I wear my scrubs home from work a lot because I'm lazy and never sterile anyway. I'm a short female frequently wearing a backpack, so General-Populationville usually thinks I'm a nursing student, which I've stopped really giving half a crap about since it's easier than explaining what I actually do.
The CSI watchers jump to autopsy (Yes, I carry a gun and chase criminals around crime scenes. In heels. I'm badass that way. I also used my kelp knife to fight great white sharks rather than using it to avoid being murdered by algae), and any further explanation of what pathologists do, ie, most of our job, is lost on people.
I would say that etiquette would dictate erring on the side of highest education/deepest loan burden, thus I insist all people noticing my scrubs address me as "You must be the program director of neurosurgery at Johns Hopkins!" else be branded as sexist pigs. On the other hand, I'd note that my hospital is one of the biggest employers in this city, as well as being a medical school, so seeing someone in scrubs should usually prompt people to not guess what you do, since they have as much chance of getting your profession correct as you do seeing someone in a suit.
Still, imagine my surprise:
Rite Aid cashier: "Oh, are you a doctor?"
Me (shocked stutter): "Yes!"
Rite Aid cashier: "Surgeon?"
Me (double shocked stutter for the double un-sexism): "No, uh.. pathologist."
Rite Aid cashier: "Ah! Heh. 'Are the margins clear?? Are the margins clear??' "
Me: "You appear to know more about pathology than most doctors."
It's true too. In med school, I had a frigging anesthesiologist hear my interest and go "Oh, but don't you mostly deal with dead people?" Dude, you're in the OR when they send specimens. You have to sit there and wait for the phone to ring so the operation can proceed. How do you not know what we do?
Then I remember I've been on autopsy for two months and feel like I'm fulfilling some sort of stereotype, like if I'd been caught painting my toenails and watching Fried Green Tomatoes while crying into a kitten's fur and asking a man what he's thinking.