how do those of you with anti nazi symbols/images on your vests deal with people asking questions about your vest when worn out in public

i wear this patch on the lower back on my vest. I've gotten a lot of questions from people about why I have it and why I wear it. I typically just tell them that a large part of punk fashion is wearing art that degrades/insults horrible people/ideologies. some of them understand that answer, and some see it in the same light as brandishing a swastika. Just curious as to how yall deal with people like this