I think the orientation of the viewer IS important. It's coloring how you're preceiving what you're viewing. I am not looking at other females as potential mates. Whereas with males, there's always some part of the 'rodent-brain' section that's eyeing them as potential partners. It's not a DEFINING factor--there are obviously other influences on what a person percieves as beautiful or pretty--but the reason you are looking colors your perception of what is beautiful. About the only place where everyone would generally agree is that symmetrical features are more pleasing than asymmetrical features. That's apparently hardwired into the brain.
Example--to me, Tanith's features (and Sasha Cohen's, come to that) aren't particularly appealing. I can see why people think she's "hot", but to me she simply isn't beautiful. In Sasha's case, I could (but won't unless someone asks) pinpoint exactly what bothers me about her face and why I consider her not even all that pretty, except when she lets her hair down which helps fix the problem. With TT/MM, my issue is indeed he's got rather unfinished, rough features, and in her case she's just not particularly notable.