As in "no, I don't worship anything", usually results in increasingly puzzled questions about what I actually do or do not 'believe'.

Condolences. Judaism is much more based in practice than belief, to the point where it's not even necessary to be a secular Jew to be a Jewish atheist; it's kind of an orthagonal question, which did nothing to help me in not bouncing off this book.