Row over AI that 'identifies gay faces'(bbc.co.uk)
bbc.co.uk
Row over AI that 'identifies gay faces'
http://www.bbc.co.uk/news/technology-41188560
68 comments
The other criticisms seemed perfectly cogent to me. A lot of people just don't know what "peer reviewed" actually means. I think that is the most likely cause of the error, rather than malice.
It's pretty normal for charities to encourage donations, so I'm not sure what your problem is with that. You seem to be hinting at the idea that they have deliberately released an inaccurate press release to encourage donations, but there's no reason for them to bother doing that when there are many uncontroversial instances of homophobia that they could refer to.
It's pretty normal for charities to encourage donations, so I'm not sure what your problem is with that. You seem to be hinting at the idea that they have deliberately released an inaccurate press release to encourage donations, but there's no reason for them to bother doing that when there are many uncontroversial instances of homophobia that they could refer to.
Thank you for responding. You're right about the donations; I'm sure this pop-over exists across their site and is not inherently sinister, but including a call for donations specifically over this divisive, misleading press release irks me, because it seems to encourage people to wage scientific debate via lobbying rather than more research, which is the opposite of what the world needs right now (see: climate change). However, you're right that it's irrelevant to the actual debate here.
On a more specific note, claims like "The study assumed there were only two sexual orientations -- gay and straight -- and does not address bisexual individuals." (without mention that this is addressed in the paper and that better labels would likely improve results) and repeated hand-waving about hairstyles (without mention of the sections of the paper devoted to feature analysis which point to structural facial features giving the strongest signal to the model) do not present methodological flaws with the study and/or are misleading. Prof. Kosinski and Yilun Wang have issued a very polite rebuttal (here)[https://docs.google.com/document/d/1UuEcSNFMduIaf0cOWdWbOV3N...] which details these and many other weaknesses with the press release. I don't think it's unreasonable to expect someone at GLAAD or HRC to understand what peer review means, and in fact, I'm sure they do.
On a more specific note, claims like "The study assumed there were only two sexual orientations -- gay and straight -- and does not address bisexual individuals." (without mention that this is addressed in the paper and that better labels would likely improve results) and repeated hand-waving about hairstyles (without mention of the sections of the paper devoted to feature analysis which point to structural facial features giving the strongest signal to the model) do not present methodological flaws with the study and/or are misleading. Prof. Kosinski and Yilun Wang have issued a very polite rebuttal (here)[https://docs.google.com/document/d/1UuEcSNFMduIaf0cOWdWbOV3N...] which details these and many other weaknesses with the press release. I don't think it's unreasonable to expect someone at GLAAD or HRC to understand what peer review means, and in fact, I'm sure they do.
The response isn't great either. For example, the contradiction alleged in footnote 1 isn't a contradiction at all, and they don't adequately address the limitations of the study in relation to the absence of non-white individuals in the dataset, or the absence accurate information about individuals' sexuality. All they do is insist without evidence that none of this makes any difference.
In fact, the original paper is deeply confused about sexual orientation:
>Their sexual orientation was established based on the gender of the partners that they were looking for (according to their profiles).
> [...]
> Another issue pertains to the quality of the ground truth: it is possible that some of the users categorized as heterosexual were, in fact, gay or bisexual (or vice versa). However, we believe that people voluntarily seeking partners on the dating website have little incentive to misrepresent their sexual orientation.
Obviously, bisexual people who say that they are looking for a partner of one or other gender are not "misrepresenting" their sexual orientation!
In fact, the original paper is deeply confused about sexual orientation:
>Their sexual orientation was established based on the gender of the partners that they were looking for (according to their profiles).
> [...]
> Another issue pertains to the quality of the ground truth: it is possible that some of the users categorized as heterosexual were, in fact, gay or bisexual (or vice versa). However, we believe that people voluntarily seeking partners on the dating website have little incentive to misrepresent their sexual orientation.
Obviously, bisexual people who say that they are looking for a partner of one or other gender are not "misrepresenting" their sexual orientation!
Science isn't about debate, you either falsify the null hypothesis or you don't.
The last thing an issue like climate change needs is a falsifiable hypothesis which is why the IPCC never puts one forward.
If we want people to stop driving cars, take shorter showers and do other progressive things than the last thing we should do is propose something that could be falsified. It would be as devastating to climate change as the catholic church putting forth a testable hypothesis for the existence of God.
The last thing an issue like climate change needs is a falsifiable hypothesis which is why the IPCC never puts one forward.
If we want people to stop driving cars, take shorter showers and do other progressive things than the last thing we should do is propose something that could be falsified. It would be as devastating to climate change as the catholic church putting forth a testable hypothesis for the existence of God.
[deleted]
Not sure why downvoted--OP is pointing out clear, obvious lies from organizations with a vested interest in the topic.
Is the assertion here that lies from gay people should be ignored?
Is the assertion here that lies from gay people should be ignored?
Not one of the downvoters, but did want to respond to a few points.
On peer review: 1) The uploaded version of the preprint that was linked is currently version 7. Version 1 did not contain the "this is a peer reviewed article" boiler-plate at the top. It's very possible that GLAAD/HRC saw the original manuscript without this notice. 2) Given that all 7 versions have been posted in the past 9 days, one might reasonably conclude that this is still a work-in-progress and peer review is still currently on-going, (the boilerplate being added in anticipation of the review being completed. Again, the paper is very clearly still being worked on.)
On misinformation: Yes I will give you that some of the criticisms might seem over-the-top to the HN crowd, but I think they are understandable given what GLAAD's / HRC's focus is, plus the fact that they aren't addressing "trained scientists." That being said, I don't think their criticisms are completely baseless. I think anyone with technical training, particularly in data analysis / machine learning, is familiar with the phrase: "Garbage In - Garbage Out." And I think it is absolutely correct to point out that maybe if your training data is dating site photos, predominantly of white people in a narrow age range, that you may not have exactly proven your claims that you have "advance[d] our understanding of the origins of sexual orientation and the limits of human perception." That maybe you've just learned some interesting features of current trends in dating site photos.
The tragic thing is that both sides (GLAAD & HRC vs. the researchers) claim to care about the privacy and safety of individuals. And kudos to them for thinking of the social implications of this kind of research! (We all know how good phrenology turned out) Because of these implications, I absolutely think it's right for us to question the conclusions of this research (that you can detect sexual orientation from facial morphology, and that this is evidence of the prenatal hormone theory of sexual orientation), given their actual methods (we threw a bunch of dating site photos at a neural-net classifier, and compared to self-reported sexual orientation)
All that said, I do think that the paper is likely to be more nuanced than the conclusions reported / the abstract make it out to be (have not finished reading the paper yet). But unfortunately, because of the way science is reported, where the headline is going to be some bombastic "AI can tell if you are gay or straight from just a picture!", there is probably some pressure to voice their criticisms just as loudly.
On peer review: 1) The uploaded version of the preprint that was linked is currently version 7. Version 1 did not contain the "this is a peer reviewed article" boiler-plate at the top. It's very possible that GLAAD/HRC saw the original manuscript without this notice. 2) Given that all 7 versions have been posted in the past 9 days, one might reasonably conclude that this is still a work-in-progress and peer review is still currently on-going, (the boilerplate being added in anticipation of the review being completed. Again, the paper is very clearly still being worked on.)
On misinformation: Yes I will give you that some of the criticisms might seem over-the-top to the HN crowd, but I think they are understandable given what GLAAD's / HRC's focus is, plus the fact that they aren't addressing "trained scientists." That being said, I don't think their criticisms are completely baseless. I think anyone with technical training, particularly in data analysis / machine learning, is familiar with the phrase: "Garbage In - Garbage Out." And I think it is absolutely correct to point out that maybe if your training data is dating site photos, predominantly of white people in a narrow age range, that you may not have exactly proven your claims that you have "advance[d] our understanding of the origins of sexual orientation and the limits of human perception." That maybe you've just learned some interesting features of current trends in dating site photos.
The tragic thing is that both sides (GLAAD & HRC vs. the researchers) claim to care about the privacy and safety of individuals. And kudos to them for thinking of the social implications of this kind of research! (We all know how good phrenology turned out) Because of these implications, I absolutely think it's right for us to question the conclusions of this research (that you can detect sexual orientation from facial morphology, and that this is evidence of the prenatal hormone theory of sexual orientation), given their actual methods (we threw a bunch of dating site photos at a neural-net classifier, and compared to self-reported sexual orientation)
All that said, I do think that the paper is likely to be more nuanced than the conclusions reported / the abstract make it out to be (have not finished reading the paper yet). But unfortunately, because of the way science is reported, where the headline is going to be some bombastic "AI can tell if you are gay or straight from just a picture!", there is probably some pressure to voice their criticisms just as loudly.
>maybe you've just learned some interesting features of current trends in dating site photos.
Again, I hate to keep parroting what another HN said, but I hate to see misinformation repeated:
>The paper states that they used an existing net, VGG-Face, to exact facial features: https://osf.io/fk3xr/ (page 13) >VGG-Face aims at representing a given face as a vector of scores that are as unaffected as possible by facial expression, background, lighting, head orientation, image properties such as brightness or contrast, and other factors that can vary across different images of the same person. All the heavy lifting for this paper was done by logistic regression on the facial features reported by VGG-Face, they didn't train a DNN specifically for identifying sexuality. The algo wouldn't have seen clothing at all.
Basically, it's not as simple as "All the gay participants had rainbow flags in the background of their photo so let's ignore this study"
Again, I hate to keep parroting what another HN said, but I hate to see misinformation repeated:
>The paper states that they used an existing net, VGG-Face, to exact facial features: https://osf.io/fk3xr/ (page 13) >VGG-Face aims at representing a given face as a vector of scores that are as unaffected as possible by facial expression, background, lighting, head orientation, image properties such as brightness or contrast, and other factors that can vary across different images of the same person. All the heavy lifting for this paper was done by logistic regression on the facial features reported by VGG-Face, they didn't train a DNN specifically for identifying sexuality. The algo wouldn't have seen clothing at all.
Basically, it's not as simple as "All the gay participants had rainbow flags in the background of their photo so let's ignore this study"
I never said anything about rainbow flags in the background, nor that we should ignore this study. And fine to be pedantic, it wasn't throwing a bunch of photos at a NN classifier, it was throwing a bunch of photos through a NN to get a descriptor of facial features, then training a logistic regression on that. (Note: they did also train a DNN specifically for this, which they used for example to say things like: "Male facial image brightness correlates 0.19 with the probability of being gay, as estimated by the DNN based classifier. While the brightness of the facial image might be driven by many factors, previous research found that testosterone stimulates melanocyte structure and function leading to a darker skin." which some might argue is quite a jump given the data at hand.
What I am saying is that the authors may have been a bit far-reaching in their conclusions, and that "trends in dating site photos" doesn't just mean "is there a rainbow flag present?", but also things like how photos are taken w.r.t angles, and lighting, and retouched, makeup, expressions, etc. (and yes VGG-Face is supposed to be resilient to some of these things, they also used Face++ for studies having to do with morphology, such as using facial contour features. But saying they are resilient to these things and quantifying that are two very different things).
To get to even a more fundamental question - Does the population of gay and straight individuals on a dating website necessarily reflect the gay and/or straight populations as a whole?
And once the authors start talking about the femininity or masculinity of faces, and whether certain features are gender atypical or not, they are entering very murky and ill-defined territory, which the authors even acknowledged when relating previous results on this front.
To be very clear about it: I am not saying that the study should be ignored completely, or that there were no interesting findings. Quite the contrary! What I am saying is that the conclusions being reached ("Our results provide strong support for the PHT, which argues that same gender sexual orientation stems from the underexposure of male fetuses and overexposure of female fetuses to prenatal androgen responsible for the sexual differentiation of faces, preferences, and behavior") are quite a bit of a leap from what was found in the research given the input data and methods, and I think should have been more accurately reported as: "interesting differences in social media photographs of gay vs straight individuals.", with a little less speculation by the authors on what might be causing these differences.
What I am saying is that the authors may have been a bit far-reaching in their conclusions, and that "trends in dating site photos" doesn't just mean "is there a rainbow flag present?", but also things like how photos are taken w.r.t angles, and lighting, and retouched, makeup, expressions, etc. (and yes VGG-Face is supposed to be resilient to some of these things, they also used Face++ for studies having to do with morphology, such as using facial contour features. But saying they are resilient to these things and quantifying that are two very different things).
To get to even a more fundamental question - Does the population of gay and straight individuals on a dating website necessarily reflect the gay and/or straight populations as a whole?
And once the authors start talking about the femininity or masculinity of faces, and whether certain features are gender atypical or not, they are entering very murky and ill-defined territory, which the authors even acknowledged when relating previous results on this front.
To be very clear about it: I am not saying that the study should be ignored completely, or that there were no interesting findings. Quite the contrary! What I am saying is that the conclusions being reached ("Our results provide strong support for the PHT, which argues that same gender sexual orientation stems from the underexposure of male fetuses and overexposure of female fetuses to prenatal androgen responsible for the sexual differentiation of faces, preferences, and behavior") are quite a bit of a leap from what was found in the research given the input data and methods, and I think should have been more accurately reported as: "interesting differences in social media photographs of gay vs straight individuals.", with a little less speculation by the authors on what might be causing these differences.
Note: I haven't yet read that paper.
Funny thing though is R=0.19 being used as evidence. First of all, effect size is what matters. Second, this is near zero and essentially worthless. Third, they didn't really check how their methods work or know why they work, just guess.
For fun, let's see how easily you can make someone gay with evolutionary approach used in the 2015 fooling face detectors paper. The algorithms are in opencv_contrib.
I bet it is essentially worthless.
Funny thing though is R=0.19 being used as evidence. First of all, effect size is what matters. Second, this is near zero and essentially worthless. Third, they didn't really check how their methods work or know why they work, just guess.
For fun, let's see how easily you can make someone gay with evolutionary approach used in the 2015 fooling face detectors paper. The algorithms are in opencv_contrib.
I bet it is essentially worthless.
GLAAD and HRC are advocacy groups, not organizations dedicated to truth. If a lie creates advocacy that's perfectly within their purview as advocates.
As with all organizations the purpose of organizations are to perpetuate themselves.
As with all organizations the purpose of organizations are to perpetuate themselves.
I'm inclined to believe that an ML algorithm can do what the authors claim- but maybe only for the data set they're using.
The inputs are self-submitted photos to a dating website. Can we be certain that how people pose, smile, look, etc, isn't influenced by their intention- to find a mate of a specific gender? What if those same people were each asked to submit a second photo when they're told to 'act gay/straight' in order to fool the algorithm?
It could be that this ML algorithm is picking up on very different cues than intended. But I wouldn't discourage the researchers over it- try it again with a better data set, one built for this purpose, and discover just what the algorithm is picking up.
The inputs are self-submitted photos to a dating website. Can we be certain that how people pose, smile, look, etc, isn't influenced by their intention- to find a mate of a specific gender? What if those same people were each asked to submit a second photo when they're told to 'act gay/straight' in order to fool the algorithm?
It could be that this ML algorithm is picking up on very different cues than intended. But I wouldn't discourage the researchers over it- try it again with a better data set, one built for this purpose, and discover just what the algorithm is picking up.
This particular algorithm is being talked about heavily in the gayer parts of the internet, and I keep telling everyone precisely this. This is interesting in as far as it shows that, in the context of trying to attract a partner, gay and straight people present themselves in quantifiably different ways, which I suspect most people sufficiently familiar with app dating culture would have already known.
Again, I hate to keep parroting what another HN said, but I hate to see misinformation repeated:
The paper states that they used an existing net, VGG-Face, to exact facial features: https://osf.io/fk3xr/ (page 13) >VGG-Face aims at representing a given face as a vector of scores that are as unaffected as possible by facial expression, background, lighting, head orientation, image properties such as brightness or contrast, and other factors that can vary across different images of the same person. All the heavy lifting for this paper was done by logistic regression on the facial features reported by VGG-Face, they didn't train a DNN specifically for identifying sexuality. The algo wouldn't have seen clothing at all.
Basically, it's not as simple as "All the gay participants had rainbow flags in the background of their photo so let's ignore this study"
The paper states that they used an existing net, VGG-Face, to exact facial features: https://osf.io/fk3xr/ (page 13) >VGG-Face aims at representing a given face as a vector of scores that are as unaffected as possible by facial expression, background, lighting, head orientation, image properties such as brightness or contrast, and other factors that can vary across different images of the same person. All the heavy lifting for this paper was done by logistic regression on the facial features reported by VGG-Face, they didn't train a DNN specifically for identifying sexuality. The algo wouldn't have seen clothing at all.
Basically, it's not as simple as "All the gay participants had rainbow flags in the background of their photo so let's ignore this study"
> Basically, it's not as simple as "All the gay participants had rainbow flags in the background of their photo so let's ignore this study"
Well, you've made me laugh and made an excellent point.
I suppose I should read the paper more in-depth.
Well, you've made me laugh and made an excellent point.
I suppose I should read the paper more in-depth.
This is very interesting thanks for sharing, but I am not sure if it totally kills my point. I assumed face was isolated from clothing/hairstyle/background already, but believed facial expression (etc) was the main thing being looked at. If this VGG-Face tool is sufficiently good at making expression (etc) not be a factor, this is potentially a huge deal and some of the worries in the gay community may be justified. Excuse my generic HN skepticism but I will need to look into how legit VGG-Face is before I start thinking about what this means.
I don't think it does kill your point. The set of images used in the sample are obviously biased. They are by people posting pictures they hope to use to attract others of a certain gender (or genders). Even if it is specifically taking dimensional measurements surely some facial expressions bias the dimensions. This could still be the equivalent of a duck-face detector.
I don't have a working "gaydar" but I do believe that it's real (meaning there are people who can predict better than chance). In the same way people can learn to sex chicks based on intuition after some training: http://www.businessinsider.com/the-incredible-intuition-of-p...
Is it that doubtful that a ML algorithm can reach/exceed human-levels on this "gaydar" problem?
Is it that doubtful that a ML algorithm can reach/exceed human-levels on this "gaydar" problem?
What disturbs me is the potential for a government, like in Iran or Russia, to use this technology to automatically screen and label people, as a precursor to mass-scale systematic repression.
I get that this study is problematic, but there is no reason to automatically assume it's not possible. What evidence do we have that there is no correlation between ML consumable physical characteristics and our sexuality?
I get that this study is problematic, but there is no reason to automatically assume it's not possible. What evidence do we have that there is no correlation between ML consumable physical characteristics and our sexuality?
Doesn't that risk exist for many emergent AI scenarios?
"Bad actors" using AI to influence debate, or writing "fake news" then promoting it with social media ads, etc?
"Bad actors" using AI to influence debate, or writing "fake news" then promoting it with social media ads, etc?
> Doesn't that risk exist for many emergent AI scenarios?
Yes, and it is actually the bigger problem. Facial recognition and license plate scanners are already producing a "papers, please" situation.
The gay community, however, is more activist and more likely to fight back. Which is why this is getting such traction.
Yes, and it is actually the bigger problem. Facial recognition and license plate scanners are already producing a "papers, please" situation.
The gay community, however, is more activist and more likely to fight back. Which is why this is getting such traction.
[deleted]
The study addresses this:
"Additionally, given that companies and governments are increasingly usingcomputer vision algorithms to detect people’s intimate traits, our findings expose a threat to the privacy and safety of gay men and women." (page 2)
"Additionally, given that companies and governments are increasingly usingcomputer vision algorithms to detect people’s intimate traits, our findings expose a threat to the privacy and safety of gay men and women." (page 2)
xienze(3)
I posted this as a reply elsewhere, but I think anyone inclined to believe this model/study supports the prenatal hormone theory should read this from Calling Bullshit: http://callingbullshit.org/case_studies/case_study_ml_sexual...
The setup and modeling may be perfectly sound, but the conclusions are a big stretch: they didn't control for intentional differences in presentation (the data was dating site photos), nor did they appropriately test for significance of the supposedly subtle facial differences picked up by the model (gay men having more feminine faces, gay women more masculine ones).
The setup and modeling may be perfectly sound, but the conclusions are a big stretch: they didn't control for intentional differences in presentation (the data was dating site photos), nor did they appropriately test for significance of the supposedly subtle facial differences picked up by the model (gay men having more feminine faces, gay women more masculine ones).
People keep saying this, but as another HN pointed out:
The paper states that they used an existing net, VGG-Face, to exact facial features: https://osf.io/fk3xr/ (page 13) >VGG-Face aims at representing a given face as a vector of scores that are as unaffected as possible by facial expression, background, lighting, head orientation, image properties such as brightness or contrast, and other factors that can vary across different images of the same person. All the heavy lifting for this paper by was done logistic regression on the facial features reported by VGG-Face, they didn't train a DNN specifically for identifying sexuality. The algo wouldn't have seen clothing at all.
It's pretty disturbing how quick people are to dismiss papers that make them uncomfortable, latching onto the first excuse to dismiss it without even confirming if the excuse is valid or not. Clear Cognitive Dissonance in action.
The paper states that they used an existing net, VGG-Face, to exact facial features: https://osf.io/fk3xr/ (page 13) >VGG-Face aims at representing a given face as a vector of scores that are as unaffected as possible by facial expression, background, lighting, head orientation, image properties such as brightness or contrast, and other factors that can vary across different images of the same person. All the heavy lifting for this paper by was done logistic regression on the facial features reported by VGG-Face, they didn't train a DNN specifically for identifying sexuality. The algo wouldn't have seen clothing at all.
It's pretty disturbing how quick people are to dismiss papers that make them uncomfortable, latching onto the first excuse to dismiss it without even confirming if the excuse is valid or not. Clear Cognitive Dissonance in action.
That's a really important point about this not being a novel DNN, but I'm not sure how it addresses either of the main criticisms from the link I posted.
The first is that claiming this methodology outperforms humans is unfair; there's no expectation that Turkers are particularly good at identifying sexuality from photographs, they haven't been able to train in a remotely analogous way, and there's no comparison to "experts" in orientation recognition (if those exist).
The second is that the authors of this study use the differences extracted by VGG-Face to claim support for PHT, despite not applying an appropriate statistical test to the differences in the features between groups. This is the bigger scientific misstep in my mind, the willingness to make a strong, apparently controversial claim (I'm not too familiar with PHT or its history) without properly validating it.
The first is that claiming this methodology outperforms humans is unfair; there's no expectation that Turkers are particularly good at identifying sexuality from photographs, they haven't been able to train in a remotely analogous way, and there's no comparison to "experts" in orientation recognition (if those exist).
The second is that the authors of this study use the differences extracted by VGG-Face to claim support for PHT, despite not applying an appropriate statistical test to the differences in the features between groups. This is the bigger scientific misstep in my mind, the willingness to make a strong, apparently controversial claim (I'm not too familiar with PHT or its history) without properly validating it.
For your second point, what in your opinion would be the right statistical test to apply?
They're "unaffected as possible", but not necessarily completely unaffected.
I think the signal that the AI bot has identified is very likely to be related to the effects of differing hormonal exchanges and releases over long periods of time.
We know that hormonal derivatives like steroids can influence appearance, and also that ambient secretions offer signaling at a distance among humans, in circumstances not limited to the synchronization of menstrual cycles, at a minimum.
Take these factors together, and exclusive close proximity to gender concentrations based on preference likely results in mild changes in appearance that automated analysis can determine as a non-zero signal, and hazard an informed guess with. People are not wrong in claiming that this is a form of stereotyping, and that sometimes the stereotype will be incorrect.
People may not like it, but just because you can't see it, doesn't mean it's not there.
We know that hormonal derivatives like steroids can influence appearance, and also that ambient secretions offer signaling at a distance among humans, in circumstances not limited to the synchronization of menstrual cycles, at a minimum.
Take these factors together, and exclusive close proximity to gender concentrations based on preference likely results in mild changes in appearance that automated analysis can determine as a non-zero signal, and hazard an informed guess with. People are not wrong in claiming that this is a form of stereotyping, and that sometimes the stereotype will be incorrect.
People may not like it, but just because you can't see it, doesn't mean it's not there.
Or their dataset is a dating site and they are training a neural net to see features that are not there. 70-80% accuracy for a neural net is pretty bad if you have taken a neural networks course. The fact that Stanford students are publishing this is probably not because it's sound scientifically, but because ostentatiously riotous subject matter will increase the number of citations dramatically. Frankly, I don't think it makes sense to infer sexual orientation based on facial recognition, and I don't think it is sound science behind any of these findings. Yes, there may be features that imply correlation one way or the other, but the fact that these features would be "undetectable by the naked eye but still visually present" leads to all sorts of murky science that is "based" on neural networks. Also, this sort of "science" will no doubt be used wrongly to classify people who have not even investigated their own sexual leanings at all. People will be saying that their 7 year old has a propensity to be gay or straight based on the length of their nose? Come now, this is just balderdash and it's no surprise that it's contentious. Trying to provide some sort of defense for what is clearly just the Academia version of Clickbait is also a waste of time.
Saying 70-80% is bad for a neural network is like saying that $1000 is too high a price without specifying what's for sale. The paper was peer-reviewed and accepted by a leading psychology journal. The lead author is an associate professor, not a student. At my first read, nothing unscientific stands out - claims that you don't think it's sound science, clickbait, and balderdash don't do anything to change the facts or advance the debate. That's not how science works.
Inner Preference versus External Appearance. If making a connection between these two things makes sense, then we should bust out the cranial measuring devices and get rid of college entrance exams.
Making a broad claim about the usual accuracy of deep nets is extremely misleading; the bounding factor is the task, not the model.
You don't even have to take issue with the modeling or any other part of the methodology to debunk this study; it's the interpretation that is plainly ridiculous. As you said, these are pictures from a dating site; awfully likely that people choose to present themselves differently based on who they want to attract. Also no appropriate statistical test applied to differences in faces picked up by the model. See this for more: http://callingbullshit.org/case_studies/case_study_ml_sexual...
You don't even have to take issue with the modeling or any other part of the methodology to debunk this study; it's the interpretation that is plainly ridiculous. As you said, these are pictures from a dating site; awfully likely that people choose to present themselves differently based on who they want to attract. Also no appropriate statistical test applied to differences in faces picked up by the model. See this for more: http://callingbullshit.org/case_studies/case_study_ml_sexual...
It's not clear to me -- your comment seems to contain conflicting statements -- do you think no correlation could exist between facial characteristics and sexuality? If you think there might be, do you think no program could ever detect it?
To answer your first question: Correlation could exist, but the idea that they are correlated implies that my physical features will change when my sexual preference changes. Ponder that for a minute.
Sexuality is not a boolean value, it's not a yes or no, it's not a this or that, even for people who are attracted to females they are not attracted to every female, and people who are attracted to males are not attracted to every male. It's such a silly thing to try and hone in on, because it removes the primary deciding component of human sexuality from the study: personal, intimate, real human preference. It's like attempting to discover one's favorite flavor of bagel shmear based on high school portraits.
You may find some surprising results, realize neural nets are trainable phenomena, and then run around giving everyone plastic surgery because more pronounced cheek bones tend to prefer Strawberry Shmear, and if peoples' cheek bones were adjusted their flavor preference would change.
Sexuality is not a boolean value, it's not a yes or no, it's not a this or that, even for people who are attracted to females they are not attracted to every female, and people who are attracted to males are not attracted to every male. It's such a silly thing to try and hone in on, because it removes the primary deciding component of human sexuality from the study: personal, intimate, real human preference. It's like attempting to discover one's favorite flavor of bagel shmear based on high school portraits.
You may find some surprising results, realize neural nets are trainable phenomena, and then run around giving everyone plastic surgery because more pronounced cheek bones tend to prefer Strawberry Shmear, and if peoples' cheek bones were adjusted their flavor preference would change.
> To answer your first question: Correlation could exist, but the idea that they are correlated implies that my physical features will change when my sexual preference changes. Ponder that for a minute.
Why couldn't physical features change?
I believe hormone changes can affect appearances, so it doesn't seem beyond the realm of possibility.
> Sexuality is not a boolean value, it's not a yes or no, it's not a this or that
Did it occur to you that possibly the correlation with facial characteristics could be a matter of degree?
Consider the correlation between facial characteristics and racial background. I'm half European, half Asian. There's going to be a stronger correlation between people with near 100% European ancestry and "European facial characteristics" than people with 50% European ancestry, than people with 25% European ancestry, etc
Why couldn't physical features change?
I believe hormone changes can affect appearances, so it doesn't seem beyond the realm of possibility.
> Sexuality is not a boolean value, it's not a yes or no, it's not a this or that
Did it occur to you that possibly the correlation with facial characteristics could be a matter of degree?
Consider the correlation between facial characteristics and racial background. I'm half European, half Asian. There's going to be a stronger correlation between people with near 100% European ancestry and "European facial characteristics" than people with 50% European ancestry, than people with 25% European ancestry, etc
My point is that Sexuality is not classifiable like Eye Color is classifiable, and we should not treat it as such.
>the idea that they are correlated implies that my physical features will change when my sexual preference change
No, it doesn't. It means sexual orientation and hormonal/chemical composition might be related, and hormonal/chemical composition might affect facial composition.
Everything you wrote afterwards is just an emotional outburst to an uncomfortable results from a study you obviously didn't bother reading.
No, it doesn't. It means sexual orientation and hormonal/chemical composition might be related, and hormonal/chemical composition might affect facial composition.
Everything you wrote afterwards is just an emotional outburst to an uncomfortable results from a study you obviously didn't bother reading.
"Sexual Orientation and Hormonal Composition" is a whole field of research on its own. Sexual Identity and Portraits we scraped off a dating site is too tenuous a leap for me to appreciate as any sort of "data" for serious science. If they had taken 5000 people, asked them to self-identify, and took a blood sample and measured various hormones, that would be a lot more satisfying.
Yes, I am emotionally bothered by this whole enterprise, because sexual orientation is not something you can understand through portraits, and this study makes the claim that 7/10 times a computer can do it. That's not the case.
Let's say there's a 50% chance that hormonal/chemical composition and sexual orientation are related, and let's say there's a 50% chance that hormonal composition affects facial features. That means that for the both of them to work out we have a 25% chance they are correlated. That's a pretty generous estimate, given that there is no data supporting the second half of that equation where "hormonal composition affects facial composition" as far as I'm aware.
The best we are doing with this dataset is inferring hormonal composition from what someone put down as their sexual preference. Or am I missing a large piece of the puzzle?
Yes, I am emotionally bothered by this whole enterprise, because sexual orientation is not something you can understand through portraits, and this study makes the claim that 7/10 times a computer can do it. That's not the case.
Let's say there's a 50% chance that hormonal/chemical composition and sexual orientation are related, and let's say there's a 50% chance that hormonal composition affects facial features. That means that for the both of them to work out we have a 25% chance they are correlated. That's a pretty generous estimate, given that there is no data supporting the second half of that equation where "hormonal composition affects facial composition" as far as I'm aware.
The best we are doing with this dataset is inferring hormonal composition from what someone put down as their sexual preference. Or am I missing a large piece of the puzzle?
[deleted]
A good accuracy for any solution is based far more on the problem than the model trying to solve it. 70-80% for one problem space might be horrible, but be a dream for others.
Yes but if evolution provided you with a 70-80% accurate way of deciding which berries were poisonous, or which potential mates were going to kill you instead of procreate with you, the neural net would be of rather negligible benefit.
> People will be saying that their 7 year old has a propensity to be gay or straight based on the length of their nose?
What's the big deal? We've already got people thinking their toddlers are transgendered because they picked up a doll one time...
What's the big deal? We've already got people thinking their toddlers are transgendered because they picked up a doll one time...
Please stop using this site primarily for politics and ideology—especially inflammatory tangents.
https://news.ycombinator.com/newsguidelines.html
https://news.ycombinator.com/newsguidelines.html
You should probably stop allowing stories where the discussion can be taken in that direction from being posted then.
This bot's sample is a dating website. I'm going to bet my socks on the fact that all it does is detect fashion styles and trends and associate them to groups. The "male homosexual" group is most likely filled with individuals dressed in a more fashinable style who are also well groomed (eyebrows, etc.)
I am certain that you could introduce a lot of errors by adding a bunch of metrosexual but straight individuals into the mix.
I am certain that you could introduce a lot of errors by adding a bunch of metrosexual but straight individuals into the mix.
The paper states that they used an existing net, VGG-Face, to exact facial features: https://osf.io/fk3xr/ (page 13)
>VGG-Face aims at representing a given face as a vector of scores that are as unaffected as possible by facial expression, background, lighting, head orientation, image properties such as brightness or contrast, and other factors that can vary across different images of the same person.
All the heavy lifting for this paper was done by logistic regression on the facial features reported by VGG-Face, they didn't train a DNN specifically for identifying sexuality from image pixel values. The algo wouldn't have seen clothing at all.
>VGG-Face aims at representing a given face as a vector of scores that are as unaffected as possible by facial expression, background, lighting, head orientation, image properties such as brightness or contrast, and other factors that can vary across different images of the same person.
All the heavy lifting for this paper was done by logistic regression on the facial features reported by VGG-Face, they didn't train a DNN specifically for identifying sexuality from image pixel values. The algo wouldn't have seen clothing at all.
Thank you for the link. My respect for the researchers has increased.
I still think that while the results are accurate(ish), they reveal fashion trends and not genetic changes. Can in point, the composite average of homosexual male has glasses. Facial features of homosexual are often gender atypical but that's trough a lot of effort to conform with fashion trends (eyebrow reshaping, smoother skin, "kinkier" smile, etc.). This is especially true on dating sites. The whole nature vs nurture thing.
While I agree that hormone levels can impact facial features, I have yet to see a strong study proving that hormones levels can be directly linked to sexual orientation or gender identity.
I still think that while the results are accurate(ish), they reveal fashion trends and not genetic changes. Can in point, the composite average of homosexual male has glasses. Facial features of homosexual are often gender atypical but that's trough a lot of effort to conform with fashion trends (eyebrow reshaping, smoother skin, "kinkier" smile, etc.). This is especially true on dating sites. The whole nature vs nurture thing.
While I agree that hormone levels can impact facial features, I have yet to see a strong study proving that hormones levels can be directly linked to sexual orientation or gender identity.
There was an interesting comment by hn user aub3bhat a while back about the political pitfalls of facial recognition tech:
"But face recognition is a sensitive/politically-charged topic, I know several grad students (including me) who inspite of having ready-to-deploy scalable software and datasets (~10M instagram images) stay away from doing this type of a demo because reputational risks are enormous. Consider the controversy around geofeedia etc. The last thing you want as a PhD student is press interpreting your research incorrectly and blaming you for causing widespread harm. It happened to a student/professor in my department and even then the infamous study in question was in collaboration with the social network."
https://news.ycombinator.com/item?id=14580985
"But face recognition is a sensitive/politically-charged topic, I know several grad students (including me) who inspite of having ready-to-deploy scalable software and datasets (~10M instagram images) stay away from doing this type of a demo because reputational risks are enormous. Consider the controversy around geofeedia etc. The last thing you want as a PhD student is press interpreting your research incorrectly and blaming you for causing widespread harm. It happened to a student/professor in my department and even then the infamous study in question was in collaboration with the social network."
https://news.ycombinator.com/item?id=14580985
Worth considering:
I think the reason this topic is so hot is not because of the study, but the "What if other people use this as a justification for X?" fear.
For example, if it turns out that there are certain physical traits that predict sexuality, does this imply a physiological component (rather than a purely psychological one)? If there is a physiological component, I think some people worry this can be used to "pathologize" alternative sexualities.
However, what if all the endocrine disrupters we're exposed to daily [1] that act as estrogens play some role in this? What if we make the topic so unspeakable that we fail to recognize environmental contamination causes both physical differences [2] and sexual differentiation differences [3]?
This is part of the reason I think we can't reject information that may allow some socially-backward conclusion. Because we don't know all the other good things (i.e. medical) that information may be necessary for.
1. https://en.wikipedia.org/wiki/Bisphenol_A#Uses 2. https://en.wikipedia.org/wiki/Anogenital_distance#In_humans 3. https://en.wikipedia.org/wiki/Diethylstilbestrol#DES_sons
I think the reason this topic is so hot is not because of the study, but the "What if other people use this as a justification for X?" fear.
For example, if it turns out that there are certain physical traits that predict sexuality, does this imply a physiological component (rather than a purely psychological one)? If there is a physiological component, I think some people worry this can be used to "pathologize" alternative sexualities.
However, what if all the endocrine disrupters we're exposed to daily [1] that act as estrogens play some role in this? What if we make the topic so unspeakable that we fail to recognize environmental contamination causes both physical differences [2] and sexual differentiation differences [3]?
This is part of the reason I think we can't reject information that may allow some socially-backward conclusion. Because we don't know all the other good things (i.e. medical) that information may be necessary for.
1. https://en.wikipedia.org/wiki/Bisphenol_A#Uses 2. https://en.wikipedia.org/wiki/Anogenital_distance#In_humans 3. https://en.wikipedia.org/wiki/Diethylstilbestrol#DES_sons
If homosexuality is genetic, which I firmly believe it is, then it's an interesting question as to whether or not this genetic trait brings with it other physical manifestations. From a purely scientific point of view, it would be an interesting test of ML to see if some sort of relationship could be discovered. It's dangerous though to jump to conclusions that there is a physical manifestation without reams and reams of scientific study, including gene sequencing, etc.
However, it's both interesting and dangerous, as the GLAAD press release states, because of the repercussions to gays in countries where it's not nearly as accepted. It probably won't stop Russia and other very homophobic countries from conducting the same experiments using their own datasets.
In addition, very deep research including gene sequencing might inadvertently turn up the "gay" gene, which would then lead to it's own Pandora's box of issues.
This might be one of those situations where we might not want to study too hard because of the repercussions associated with it.
However, it's both interesting and dangerous, as the GLAAD press release states, because of the repercussions to gays in countries where it's not nearly as accepted. It probably won't stop Russia and other very homophobic countries from conducting the same experiments using their own datasets.
In addition, very deep research including gene sequencing might inadvertently turn up the "gay" gene, which would then lead to it's own Pandora's box of issues.
This might be one of those situations where we might not want to study too hard because of the repercussions associated with it.
Anti-gay stigma is useful to those wishing to exert control over a group via establishing in-group / out-group dynamics. One can remove a social competitor from a group by outing them as gay (even if they are not). The fact that it is capricious and the accusation unfalsifiable is an essential component. A 3rd party tool that can objective establish gayness would take away the subjectively. Society will have to move on to other mechanisms of group control just like we moved on from witchery and heresy.
We moved on from witchery and heresy by establishing objective criteria for determining witches and heretics?
(I know you didn't claim we did, just pointing out that your examples don't seem to support your argument.)
(I know you didn't claim we did, just pointing out that your examples don't seem to support your argument.)
Its possible that due to the hormonal and genetic differences in gay vs straight people, they have slightly different facial structures on average. Just like people of different genders, despite having very similar genetics, generally have different faces on average. Whether using this tech on a mass scale is for good or ill remains to be seen. The tech could be used to oppress, or it could be used for good.
Phrenological Neural Networks
Edit: Can anyone downvoting me please stand up and tell me why?