Using a ruler or tape measure, measure the front of each pair of glasses temple to temple. Compare the results. A bigger head dosen't necessarily mean a bigger brain, but anatomy and bone structure alone determine wider styles for men.
Measure one lens on each pair of glasses and compare the results. Men's lens are bigger, spaced further apart, with bulkier frames. The angles are very abstract and sharp, square or rectangle. Women's lens are rounder. Cat eyes and oval shaped lens wouldn't generally be found in men's styles.
Compare the aesthetic style of the glasses. Although it's truly a matter of personal taste, men's styles are traditionally more conservative with basic color choices like black, brown, gray and metallic. Women's styles are sleek, bold, colorful and creative. Design elements on glasses like rhinestones and studs also set them apart from the men's shades.
Compare the functionality of the glasses. Sunglasses serve a purpose -- to protect and shield your eyes from harmful UV rays. Men's designs are marketed more for their purpose rather than their look. Yours might be cute, but his are scratch proof with some kind of extreme UV protectant lens. Even if you do grab his shades, you'll be safe and still look good.