Yeah don't be surprised if the entrance to your favorite spa is next to an outdoor dining patio where diners stare at you and judge you while your waiting for the mamasan to buzz you in.
Do you notice every time you dine next to a nail salon or a hair dresser? Nope. You don't go to these establishments, so you completely ignore them and where they are located. They are almost invisible to you.
I think you are describing an over-sensitivity to seeing the word "Spa" or a neon "Open" sign in a window because you are in this hobby. Most diners are completely unaware there is a spa next to the restaurant. They are even less aware of the activity behind that little white bell. They just see a guy waiting to be buzzed into an upstairs apartment or business. Just relax, ring the bell, and ignore the diners, because they are ignoring you.
If you are still having trouble, carry a small paper bag with a receipt stapled to it. People will think you are a delivery guy. Wear a hat and a mask like
@Bryanpunk said. Take it a step further and get some fake, non-prescription glasses for mongering. Now the locals really won't recognize you.