AI ethics research conference suspends Google sponsorship(venturebeat.com)
venturebeat.com
AI ethics research conference suspends Google sponsorship
https://venturebeat.com/2021/03/02/ai-ethics-research-conference-suspends-google-sponsorship/
283 comments
The problem with AI ethics as a field is that it attracts a whole lot of activists, most of whom have mediocre contributions to the field at best, zero contributions at worst. Watching highly entitled, and often narcissistic people with no tangible accomplishments attack Yan LeCun is pretty insane.
The head of AI at Nvidia, who is very aggressive in her AI Ethics activism, recently put a screenshot of a list of names, which included mine, on a public tweet. In the tweet, she described the list as containing "fan boys" of an academic critical of "wokeness" as he calls it. I didn't know anything about the academic. I was added to the list because I commented on her tweet attacking him and asked her if she had engaged in any verbal discussions with him. That's it. I asked a question, and was put on a list proclaiming everyone in it as alt-right bigots. I found out about this because a former colleague called me and told me I was on the list. I didn't follow the guy she was attacking or anything like that. Not that it would have been justified if I had.
It should be noted that I was a leading advocate at a large enterprise software company for getting CUDA integration and helping NVIDIA sell more hardware in the enterprise. NVIDIA didn't fire her, and they should have.
The head of AI at Nvidia, who is very aggressive in her AI Ethics activism, recently put a screenshot of a list of names, which included mine, on a public tweet. In the tweet, she described the list as containing "fan boys" of an academic critical of "wokeness" as he calls it. I didn't know anything about the academic. I was added to the list because I commented on her tweet attacking him and asked her if she had engaged in any verbal discussions with him. That's it. I asked a question, and was put on a list proclaiming everyone in it as alt-right bigots. I found out about this because a former colleague called me and told me I was on the list. I didn't follow the guy she was attacking or anything like that. Not that it would have been justified if I had.
It should be noted that I was a leading advocate at a large enterprise software company for getting CUDA integration and helping NVIDIA sell more hardware in the enterprise. NVIDIA didn't fire her, and they should have.
> It should be noted that I was a leading advocate at a large enterprise software company for getting CUDA integration and helping NVIDIA sell more hardware in the enterprise.
Should have paused the project and reached out to the sales team responsible for your company's corporate account. Would have been interesting to see who pulls more weight (sales and paying customers or ethics "researchers").
Should have paused the project and reached out to the sales team responsible for your company's corporate account. Would have been interesting to see who pulls more weight (sales and paying customers or ethics "researchers").
I was already contemplating leaving it for a startup, and frankly my response to her behavior was to become even less motivated about pushing that rock up the hill in the large organization I was in. I've since departed for the startup.
Claiming that Gebru has no tangible accomplishments is factually wrong. She's lead author on multiple highly impactful papers.
> I asked a question, and was put on a list proclaiming everyone in it as alt-right bigots.
Were you, or are you editorializing her actions? Because what you actually described is that she put you on a list
> of "fan boys" of an academic critical of "wokeness" as he calls it
That's not calling you alt-right. I believe the stated purpose of the list was to have a set of people that allies could engage with constructively if they wanted to. In other words, it was a list of people whom might be good to engage in verbal discussions with. Because of that, you're calling for her to be fired. Why?
> I asked a question, and was put on a list proclaiming everyone in it as alt-right bigots.
Were you, or are you editorializing her actions? Because what you actually described is that she put you on a list
> of "fan boys" of an academic critical of "wokeness" as he calls it
That's not calling you alt-right. I believe the stated purpose of the list was to have a set of people that allies could engage with constructively if they wanted to. In other words, it was a list of people whom might be good to engage in verbal discussions with. Because of that, you're calling for her to be fired. Why?
Edit: I was mistaken in my response, since the commenter named Gebru. The head of AI at Nvidia is not Gebru, it's Anima Anandkumar. Gebru had nothing to do with this.
She specifically said the things I stated, in a now deleted tweet. I know the cognitive dissonance is causing you to question my account, but that's what she did.
Look up articles where screenshots are posted.
I wasn't claiming Gebru has no accomplishments either. Many of her fellow ethicists fall into that category. She's not an ethicist, just won't shut up about it, along with race stuff, and has nasty, unprofessional habit of attacking people publicly. She's a bully, and I'm sick of people like you defending her because you've bought into her idiotic religion.
She specifically said the things I stated, in a now deleted tweet. I know the cognitive dissonance is causing you to question my account, but that's what she did.
Look up articles where screenshots are posted.
I wasn't claiming Gebru has no accomplishments either. Many of her fellow ethicists fall into that category. She's not an ethicist, just won't shut up about it, along with race stuff, and has nasty, unprofessional habit of attacking people publicly. She's a bully, and I'm sick of people like you defending her because you've bought into her idiotic religion.
> She specifically said the things I stated, in a now deleted tweet. I know the cognitive dissonance is causing you to question my account, but that's what she did.
I agree that she said the specific things you quoted. I'm not debating what she actually said. However when you say she characterized you as a member of the alt-right, that is your editorializing, and not something she said (nor did you claim it was). I'm simply clarifying that she never accused you of being a member of the alt-right, and that you're projecting that accusation.
> Edit: I was mistaken in my response, since the commenter named Gebru. The head of AI at Nvidia is not Gebru, it's Anima Anandkumar. Gebru had nothing to do with this.
Right, but Gebru had a highly discussed disagreement with Yann, so when you said
> Watching highly entitled, and often narcissistic people with no tangible accomplishments attack Yan LeCun is pretty insane.
You were saying "Gebru is a narcissistic person with no tangible accomplishments". Either that, or you're making up generic events and hypotheticals, but that would be a really strangely specific hypothetical. Anandkumar also has very significant achievements, so there's no one you could be directing that comment at that wouldn't be belittling.
I agree that she said the specific things you quoted. I'm not debating what she actually said. However when you say she characterized you as a member of the alt-right, that is your editorializing, and not something she said (nor did you claim it was). I'm simply clarifying that she never accused you of being a member of the alt-right, and that you're projecting that accusation.
> Edit: I was mistaken in my response, since the commenter named Gebru. The head of AI at Nvidia is not Gebru, it's Anima Anandkumar. Gebru had nothing to do with this.
Right, but Gebru had a highly discussed disagreement with Yann, so when you said
> Watching highly entitled, and often narcissistic people with no tangible accomplishments attack Yan LeCun is pretty insane.
You were saying "Gebru is a narcissistic person with no tangible accomplishments". Either that, or you're making up generic events and hypotheticals, but that would be a really strangely specific hypothetical. Anandkumar also has very significant achievements, so there's no one you could be directing that comment at that wouldn't be belittling.
Ok, now I get why you were pissed. Gebru was in conflict with LeCun, but the mob of other ethicists that went all in on him were who I was talking about.
And let's be clear:
Gebru's accomplishments aren't shit compared to LeCun's, and everyone knows that in the AI field. Doesn't mean she's not accomplished, because she's a hell of a lot more accomplished than I am.
And let's be clear:
Gebru's accomplishments aren't shit compared to LeCun's, and everyone knows that in the AI field. Doesn't mean she's not accomplished, because she's a hell of a lot more accomplished than I am.
> Gebru was in conflict with LeCun, but the mob of other ethicists that went all in on him were who I was talking about.
Which ones? I followed things at the time, most of the other people who disagreed with Yann were...also PhDs, many of whom have significant accomplishments in their own right. I mean there are certainly random twitter people saying things, but that wasn't unique to the sides of the ethicists. Random twitter people also attacked and insulted Gebru and co too. So saying "Watching highly entitled, and often narcissistic people with no tangible accomplishments attack Timnit Gebru is pretty insane." is equally correct, but you didn't say that. That's suspicious and perhaps revealing of your biases on the subject.
> Gebru's accomplishments aren't shit compared to LeCun's
Of course. She's also had her PhD for 3 years, as opposed to 30. Its no surprise LeCun has accomplished more in a career 10x as long.
Which ones? I followed things at the time, most of the other people who disagreed with Yann were...also PhDs, many of whom have significant accomplishments in their own right. I mean there are certainly random twitter people saying things, but that wasn't unique to the sides of the ethicists. Random twitter people also attacked and insulted Gebru and co too. So saying "Watching highly entitled, and often narcissistic people with no tangible accomplishments attack Timnit Gebru is pretty insane." is equally correct, but you didn't say that. That's suspicious and perhaps revealing of your biases on the subject.
> Gebru's accomplishments aren't shit compared to LeCun's
Of course. She's also had her PhD for 3 years, as opposed to 30. Its no surprise LeCun has accomplished more in a career 10x as long.
Fair point on her being young in the field and her career. I think the reason it bothered me so much is that people act like LeCun just cruised on easy street for 30 years.
He went through hell. He was rejected from conferences despite having algorithms that exceeded all others, just because neural nets had fallen out of fashion. Meanwhile, the folks who attacked him have cushy jobs earning massive salaries, working on technology that he popularized.
He went through hell. He was rejected from conferences despite having algorithms that exceeded all others, just because neural nets had fallen out of fashion. Meanwhile, the folks who attacked him have cushy jobs earning massive salaries, working on technology that he popularized.
> I believe the stated purpose of the list was to have a set of people that allies could engage with constructively if they wanted to. In other words, it was a list of people whom might be good to engage in verbal discussions with.
Is this some poor attempt at a joke? That's exactly what Twitter mobs do, right? Especially when the instigator calls the targets "fanboys" and "fanatics", and explicitly calls for them to be "cancelled"?
https://news.ycombinator.com/item?id=25419871
https://archive.is/9sS3o
Is this some poor attempt at a joke? That's exactly what Twitter mobs do, right? Especially when the instigator calls the targets "fanboys" and "fanatics", and explicitly calls for them to be "cancelled"?
https://news.ycombinator.com/item?id=25419871
https://archive.is/9sS3o
> NVIDIA didn't fire her, and they should have.
Talk about cancel culture.
Talk about cancel culture.
Cancel culture is when a mob of people, usually on Twitter, most of whom have no direct relation to the institutions in question, try to create a bunch of bad publicity for a target at an institution for the purposes of making that institution decide to cut ties with the target.
I think JPKab is saying that an NVIDIA employee mistreated someone who is effectively a prospective customer for NVIDIA (or who is working to create prospective customers), and that this by itself should lead NVIDIA management to fire that employee even in the absence of public attention to the event.
I think JPKab is saying that an NVIDIA employee mistreated someone who is effectively a prospective customer for NVIDIA (or who is working to create prospective customers), and that this by itself should lead NVIDIA management to fire that employee even in the absence of public attention to the event.
Not at all. The way she acted created serious risks to her employer, and anyone seeing her postings could reasonably conclude that she was conducting personal attacks on others and doing it on behalf of NVIDIA.
When you declare your employer in your bio and talk about your field of work, it becomes relevant to your continued employment.
When you declare your employer in your bio and talk about your field of work, it becomes relevant to your continued employment.
Forcing its advocates to play by their own rules is probably the best way to beat it.
I think Alinsky's fourth rule is commonly understood to have the addendum "and actively dissuade them from trying to make you live up to your own through appeals to their better nature".
In fact, for the last several weeks a whole bunch of misogynistic Pedro Domingos fanboys (full of resentment and dishonesty, like JPKab) harassed her so much on Twitter that she deleted her account - but remember, this is a Good Cancelling because she criticized a emeritus crank.
Perhaps if she makes wild accusations like that she should be cancelled?
most researchers have mediocre, marginal, or straight up "un-sexy" contributions to a field, that's just how most research happens. it doesn't make anyone "less of a researcher" though
I also would also everyone look at Gebru's work [1] - certainly seems like they're an established research who comes from a top university, has published in top conferences, and has a good understanding of the technical parts of the field
one's past experiences with something they are criticizing or defending is probably a good prior to have, but also shouldn't be the sole criterion. "leading" scientists are more often than not wrong on plenty of things
http://ai.stanford.edu/~tgebru/
I also would also everyone look at Gebru's work [1] - certainly seems like they're an established research who comes from a top university, has published in top conferences, and has a good understanding of the technical parts of the field
one's past experiences with something they are criticizing or defending is probably a good prior to have, but also shouldn't be the sole criterion. "leading" scientists are more often than not wrong on plenty of things
http://ai.stanford.edu/~tgebru/
I agree with this - "ethicists" without much experience in the field tend to be the worst people.
> it attracts a whole lot of activists, most of whom have mediocre contributions to the field at best, []zero contributions at worst.[]
That is absurdly, unjustifiably optimistic. The worst activists have larger negative contributions than the best contributors have positive contributions. And as you yourself have observed, even typical activists often have worse-than-zero contributions.
That is absurdly, unjustifiably optimistic. The worst activists have larger negative contributions than the best contributors have positive contributions. And as you yourself have observed, even typical activists often have worse-than-zero contributions.
Citation please.
Why would you be an "AI expert" to make judgements on AI?
In fact, it's more politically consistent to have equal representation. We already know what AI experts believe - make money in the end.
In fact, it's more politically consistent to have equal representation. We already know what AI experts believe - make money in the end.
The real problem with ethics is that it's not a science nor a rigorous discipline: it's a trend. It's whatever is trendy to say and think at the moment. You certainly don't have to understand the tech to practice "ethics" (if any technical knowledge was required the field would be pretty much empty).
The way it works is you basically shame people and tech. You don't need to make a strong or valid case against a behavior or a tech, like a law would, because it's not legally binding. It's not real engineering regulations like a building or fire code. At best it's a mob made of folks who convinced other ethics PhDs to grant them a PhD that will rally against you and shame you [0].
Great move from Google to stop feeding the ethics racket.
[0] https://syncedreview.com/2020/06/30/yann-lecun-quits-twitter...
The way it works is you basically shame people and tech. You don't need to make a strong or valid case against a behavior or a tech, like a law would, because it's not legally binding. It's not real engineering regulations like a building or fire code. At best it's a mob made of folks who convinced other ethics PhDs to grant them a PhD that will rally against you and shame you [0].
Great move from Google to stop feeding the ethics racket.
[0] https://syncedreview.com/2020/06/30/yann-lecun-quits-twitter...
> It's not real engineering regulations like a building or fire code
AI or Data Science ethicists tend to be qualified data scientists who have a better understanding of concepts like differential privacy than the vast majority of software developers. Don't dismiss them all because you disagree with some of them.
AI or Data Science ethicists tend to be qualified data scientists who have a better understanding of concepts like differential privacy than the vast majority of software developers. Don't dismiss them all because you disagree with some of them.
While you both take two sides of the argument, the truth, as it tends to be, lies somewhere in the middle. While I agree that there are some qualified scientists, etc. that have taken up this issue, there is still a sort of extension of cancel culture of "LinkedInfluencers" and such that just hop on the bandwagon of hating anything vaguely AI adjacent. I think the playground dismissal of his point: "Don't dismiss them all because you disagree with some of them." was rather thoughtless.
> "Don't dismiss them all because you disagree with some of them." was rather thoughtless.
Th OP refered to them as a back scratching "mob", but when I say don't dismiss them all you accuse me of being "rather thoughtless" and using a "playground dismissal" when I gave specific reasons why they are qualified for the job.
Th OP refered to them as a back scratching "mob", but when I say don't dismiss them all you accuse me of being "rather thoughtless" and using a "playground dismissal" when I gave specific reasons why they are qualified for the job.
[deleted]
> Great move from Google to stop feeding the ethics racket.
Read the article again - it's Google being uninvited as a sponsor. The title is poorly translated from German and should probably be changed to something like "Google no longer wanted as a sponsor of AI ethics conference".
Read the article again - it's Google being uninvited as a sponsor. The title is poorly translated from German and should probably be changed to something like "Google no longer wanted as a sponsor of AI ethics conference".
It's a nice way for the conference to make it seem they are the ones who decided to terminate the relationship. That's like quitting right before getting fired to save face.
You're holding Google to falsely high regard.
While I could agree to a point, it's hard to separate the flavour of ethics from the need for reflection itself.
A thorny ethics example from my own role was a decade+ ago I was hired to do a privacy impact assessment (PIA) on an ML solution for detecting benefits fraud. It had all the ingredients for a complete debacle: "artificial intelligence"(?!), a vulnerable population, credit reporting companies, arms length government agencies, highly paid consulting firms, etc. What we delivered was modifications to the system to solve their core fraud risk problem without making peoples lives (more of) a dystopian hell.
Privacy is the currently the technology ethics department in large institutions, and I really see the AI ethics people as fancy privacy people with more math. Privacy has moved on from being a technical discipline that old hacker dudes like me did to being one largely done by women with law degrees. The tension between the security architect competency and the privacy policy analyst is still there in the field, but now we all see how they add a lot of value without needing the bottom-up view of a technologist.
The flavour of ethics in the privacy field has evolved, and I was absolutely suspicious of the change, as if you weren't a cypherpunk in privacy in 2005, you were a 5th column infiltrator looking to co-opt it for corporate ends - but of course, that's not true. Anyway a lot of text to say don't throw out babies with bathwater on AI ethics, but I'm saying they're not as new as anyone thinks, and if you think privacy is important, then you will probably recognize ethics in AI is important as well.
A thorny ethics example from my own role was a decade+ ago I was hired to do a privacy impact assessment (PIA) on an ML solution for detecting benefits fraud. It had all the ingredients for a complete debacle: "artificial intelligence"(?!), a vulnerable population, credit reporting companies, arms length government agencies, highly paid consulting firms, etc. What we delivered was modifications to the system to solve their core fraud risk problem without making peoples lives (more of) a dystopian hell.
Privacy is the currently the technology ethics department in large institutions, and I really see the AI ethics people as fancy privacy people with more math. Privacy has moved on from being a technical discipline that old hacker dudes like me did to being one largely done by women with law degrees. The tension between the security architect competency and the privacy policy analyst is still there in the field, but now we all see how they add a lot of value without needing the bottom-up view of a technologist.
The flavour of ethics in the privacy field has evolved, and I was absolutely suspicious of the change, as if you weren't a cypherpunk in privacy in 2005, you were a 5th column infiltrator looking to co-opt it for corporate ends - but of course, that's not true. Anyway a lot of text to say don't throw out babies with bathwater on AI ethics, but I'm saying they're not as new as anyone thinks, and if you think privacy is important, then you will probably recognize ethics in AI is important as well.
What's we're the differences you has noticed between "old hacker" privacy and "female lawyer" privacy?
> The real problem with ethics is that it's not a science
This is simply untrue. The goal of AI ethics research is (also) to build algorithms where one of the _inputs_ is a set of ethics. It doesn't matter what those ethics are. It just so happens that the "set of ethics" currently fed in tends to have a particular flavor ("woke liberal" ethics), which you seem to disagree with. Finding the algorithms matters, and it's likely that the standard ethics put in are going to come from some dominant ideology, but we still need the algorithms if we are to understand how to make AI that aligns with humanity's interests (however they are defined).
I'd call that computer science.
This is simply untrue. The goal of AI ethics research is (also) to build algorithms where one of the _inputs_ is a set of ethics. It doesn't matter what those ethics are. It just so happens that the "set of ethics" currently fed in tends to have a particular flavor ("woke liberal" ethics), which you seem to disagree with. Finding the algorithms matters, and it's likely that the standard ethics put in are going to come from some dominant ideology, but we still need the algorithms if we are to understand how to make AI that aligns with humanity's interests (however they are defined).
I'd call that computer science.
>> I'd call that computer science.
Yes it's Computer Science but it's implementing normative-ethics.
Why not provide the AI descriptive-ethics instead and let it decide for itself?
Yes it's Computer Science but it's implementing normative-ethics.
Why not provide the AI descriptive-ethics instead and let it decide for itself?
As you pointed out, people don't agree on a set of rules, so what's the point of the algorithm and field?
Here's an analogy: laws vary from country to country, from state to state, city to city.
But we understand the principle of law-making, what laws mean and how they're applied. This is fairly uniform. What really changes from place to place is the content of those laws.
AI ethics and AI safety are attempting to give us a set of "law-making" rules but for AI. We get to decide (democratically ideally), in countries, states, cities, what "ethics" (what laws) we want, but AI Ethics as a field gives us tools to achieve that regardless of what the ethics/laws are.
For example, how do we encode the trolley problem in self-driving cars? We could decide democratically that cars should act and kill 1 instead of 5. Or the opposite. But then how do we translate that into ifs and else? No one really knows how to do that.
But we understand the principle of law-making, what laws mean and how they're applied. This is fairly uniform. What really changes from place to place is the content of those laws.
AI ethics and AI safety are attempting to give us a set of "law-making" rules but for AI. We get to decide (democratically ideally), in countries, states, cities, what "ethics" (what laws) we want, but AI Ethics as a field gives us tools to achieve that regardless of what the ethics/laws are.
For example, how do we encode the trolley problem in self-driving cars? We could decide democratically that cars should act and kill 1 instead of 5. Or the opposite. But then how do we translate that into ifs and else? No one really knows how to do that.
this just isn't true; if you skim some abstracts of papers in the field, you will see that in fact there is a lot of technical content.
a lot of the big famous names in AI ethics (e.g. Moritz Hardt, Cynthia Dwork) also have very strong contributions outside this field.
a lot of the big famous names in AI ethics (e.g. Moritz Hardt, Cynthia Dwork) also have very strong contributions outside this field.
> The real problem with ethics is that it's not a science nor a rigorous discipline: it's a trend. It's whatever is trendy to say and think at the moment.
Maybe it's trendy to say and think that? but that doesn't sound anything like ethics. Ethics is a part of philosophy, not a part of science or maths. That it is what it is, is not a "problem with ethics". (Sorry if I misunderstand you. Maybe you were just talking about "AI ethics". ...In which case, what do people who call AI ethics "ethics" call ethics, a.k.a. moral philosophy?)
Maybe it's trendy to say and think that? but that doesn't sound anything like ethics. Ethics is a part of philosophy, not a part of science or maths. That it is what it is, is not a "problem with ethics". (Sorry if I misunderstand you. Maybe you were just talking about "AI ethics". ...In which case, what do people who call AI ethics "ethics" call ethics, a.k.a. moral philosophy?)
Hell, why not get rid of ethics entirely? Especially since people have been arguing about this for thousands of years already. And really it’s been pointless if you think about it, considering we can’t just answer it like a math problem. What good has it done us, other than forming the basis of modern societies, jurisprudence, religion – basically the substrate upon which humanity is built. We’ve clearly moved past the need for it though.
Ethics can last centuries and are the basis of customary law and the idea of a "reasonable and prudent person". AI engineers don't have their hide in the game, they just work. It's not interesting.
Introducing AI as automation for human jobs has consequences beyond one company's profits. So there needs to be non-profits or committees that take a stance and represent themselves. There doesn't need to be some scientific papr on "AI hut my job" because all the science is on the company's part, making the AI in the first place.
Introducing AI as automation for human jobs has consequences beyond one company's profits. So there needs to be non-profits or committees that take a stance and represent themselves. There doesn't need to be some scientific papr on "AI hut my job" because all the science is on the company's part, making the AI in the first place.
> AI ethics researchers are like hiring hackers working in security
One of these can be objectively evaluated for skill. There may be unwanted side effects. But the skill is there or it isn’t. That does not apply to the other, which makes their hiring and firing fundamentally different.
One of these can be objectively evaluated for skill. There may be unwanted side effects. But the skill is there or it isn’t. That does not apply to the other, which makes their hiring and firing fundamentally different.
AI Ethics isn't a non-technical field. See the accepted papers at the conference in question:
https://facctconference.org/2021/acceptedpapers.html
it's a mix of "soft" humanities, applied ML, methodology, algorithms/theory, and HCI.
it's not any harder to evaluate an AI ethics researcher than it is any other kind of researcher.
https://facctconference.org/2021/acceptedpapers.html
it's a mix of "soft" humanities, applied ML, methodology, algorithms/theory, and HCI.
it's not any harder to evaluate an AI ethics researcher than it is any other kind of researcher.
Hacked or not hacked is easier to say than ethical or not. Your ethical standards maybe racism to others, everyone have their own interpretation of ethics.
Expanding on this, if you had a machine that let you see 100 years in the future, you could nearly perfectly and objectively evaluate your security team's performance by looking at the number/severity/value lost through data breaches over that period.
With AI ethicists... if you could look 100 years in the future.. there's not an objective measure of ethical outcomes you could use to evaluate the performance of your AI Ethics team. I suppose a cynic would look at the total cost spent defending and paying settlements / judgements in ethics-related lawsuits. As far as a non-cynical objective metric for AI Ethics performance, I can't think of any.
With AI ethicists... if you could look 100 years in the future.. there's not an objective measure of ethical outcomes you could use to evaluate the performance of your AI Ethics team. I suppose a cynic would look at the total cost spent defending and paying settlements / judgements in ethics-related lawsuits. As far as a non-cynical objective metric for AI Ethics performance, I can't think of any.
An "objective" metric would be customers and dollars lost due to losing trust.
You could also look at compliance costs?
You could also look at compliance costs?
Don't get me wrong. I think ethics work is hugely important, particularly in this new field. However, ethics are much more abstract than security, and therefore much harder to evaluate objectively.
> dollars lost due to losing trust.
Trust, ethics, and public moral outrage are hopefully related, but not the same thing. Also, measuring loss of trust and allocating changes in revenue to that change is very difficult.
> compliance costs?
I think many people would agree that there's often a big difference between legal requirements and ethics.
> dollars lost due to losing trust.
Trust, ethics, and public moral outrage are hopefully related, but not the same thing. Also, measuring loss of trust and allocating changes in revenue to that change is very difficult.
> compliance costs?
I think many people would agree that there's often a big difference between legal requirements and ethics.
So there is an objective measure for deciding how good a programmer is? Why don’t we just adopt that measure in interviews?
I’d say a better measure is whether the person can currently handle most of what will be asked and how quickly they can get up to speed on new things. It’s easier to figure out and more useful for the company. It also de-prioritizes those that create a buzzword salad (sometimes out of necessity) on their resume.
Ok, so what's the unit of this number, so I can rank all the members of my team?
Because an interview only lasts for a few hours.
> But the skill is there or it isn’t
If I understand what you saying correctly, you're saying AI ethics researchers have no skill?
If I understand what you saying correctly, you're saying AI ethics researchers have no skill?
Charitably, I think what they are trying to say is that the output of an AI ethics researcher is not directly related to the application of a measurably skill.
It is instead related much more closely to a person's sense of right and wrong.
Of course AI ethics researchers have skills. They have skill in researching, they have knowledge of AI, and presumably education in formal logic and ethics. They have skill in writing and communicating the results of their findings.
However the product they produce is not writing and it's not research. It's the commentary on the research, the recommendations based on the results they have found. It's very subjective.
It's very different than a skilled hacker applying their skill and knowledge to uncover a zero-day exploit in your software that needs to be fixed. That's a much more objective process.
It is instead related much more closely to a person's sense of right and wrong.
Of course AI ethics researchers have skills. They have skill in researching, they have knowledge of AI, and presumably education in formal logic and ethics. They have skill in writing and communicating the results of their findings.
However the product they produce is not writing and it's not research. It's the commentary on the research, the recommendations based on the results they have found. It's very subjective.
It's very different than a skilled hacker applying their skill and knowledge to uncover a zero-day exploit in your software that needs to be fixed. That's a much more objective process.
I agree that his statement is inflammatory, but we can steel man the statement into something truthful and palatable:
It is far easier to evaluate a security researcher's output than an ethics researcher.
I think many of us are familiar with this concept, as many of us likely preferred math over english in school as we preferred being deterministically evaluated.
It is far easier to evaluate a security researcher's output than an ethics researcher.
I think many of us are familiar with this concept, as many of us likely preferred math over english in school as we preferred being deterministically evaluated.
Is it? If the security researchers don't find vulnerabilities, does that mean they don't exist? If the AI ethics researchers embarrass you, aren't they doing their jobs?
Just as a security researcher’s job is to find vulnerabilities and promote best practices, it’s not to hack and then publicly shame their employer. Ethics researchers should be the same, no? Find ethical issues and promote best practices, embarrassing your employer seems counterproductive except perhaps as a last resort.
This depends. Google project zero routinely discovers vulns in Google products.
However they're empowered to fix them, essentially no questions asked. So you don't often get p0 posts about unfixed bugs in Google products.
However they're empowered to fix them, essentially no questions asked. So you don't often get p0 posts about unfixed bugs in Google products.
I imagine that depends on your contract?
Since you, the employer, decide what the job is, maybe or maybe not. If you define the job as, "develop ideas and frameworks where we can accomplish our goals in the most ethical way possible," then publicly embarrassing the company might not be doing your job.
A good ethicist & communicator will probably be able to embarrass you if you're doing something wrong, but so will a comedian.
I don't think OP is claiming that there ethicists have no skill, but rather that it's fundamentally a lot more difficult to evaluate.
I don't think OP is claiming that there ethicists have no skill, but rather that it's fundamentally a lot more difficult to evaluate.
In that case, let’s look at the differences. In the case of ML researchers, careers are built on publications. I imagine a hacker who discovers a big vulnerability in their employer probably would be burning bridges if they published the vulnerability before the employer could fix it. In AI ethics, or at least the most notorious recent case, it seemed like publishing was the primary function and change was secondary.
Then there’s the ease of changing things. In security, I’d imagine fixing a vulnerability might require lots of work, but rarely novel research. In ML that’s not the case. and after all that research, the finding might be a way to improve the worst cases by worsening the average cases, lowering revenue. And then there’s the clarity of what even is an issue. If someone shouldn’t be able to access a file, but they can, it’s clear. The goal then is convincing stakeholders that the problem should be prioritized. In AI ethics, you have to convince them that the problem even is a problem, convince them to allocate research, then convince them that the technique is worth lowering revenue for, and only then get to the question of allocating resources to implement the fix. Convincing stakeholders to prioritize fixes is a wholly different game.
I like your analogy, and there are surely lessons to be learned (e.g. maybe the ethics researchers’ primary job shouldn’t be publishing papers), but the differences are strong enough that the lessons might not extend too far.
Then there’s the ease of changing things. In security, I’d imagine fixing a vulnerability might require lots of work, but rarely novel research. In ML that’s not the case. and after all that research, the finding might be a way to improve the worst cases by worsening the average cases, lowering revenue. And then there’s the clarity of what even is an issue. If someone shouldn’t be able to access a file, but they can, it’s clear. The goal then is convincing stakeholders that the problem should be prioritized. In AI ethics, you have to convince them that the problem even is a problem, convince them to allocate research, then convince them that the technique is worth lowering revenue for, and only then get to the question of allocating resources to implement the fix. Convincing stakeholders to prioritize fixes is a wholly different game.
I like your analogy, and there are surely lessons to be learned (e.g. maybe the ethics researchers’ primary job shouldn’t be publishing papers), but the differences are strong enough that the lessons might not extend too far.
Yeah, the firing of Timnit Gebru strongly reminded me of the Michael Lynn saga: https://www.schneier.com/blog/archives/2005/07/cisco_harasse...
I think the analogy breaks down a little since hackers were generally outsiders to the companies (even Michael Lynn), and hiring them and starting programs around them was the thing Microsoft did to fix their relationship to the community, whereas the folks here were already insiders at Google, but maybe that was the problem, that they were explicitly hired as academics with the expectation that they would keep doing the same work, rather than internally focussed researchers working on improving the products and systems being created.
I think the analogy breaks down a little since hackers were generally outsiders to the companies (even Michael Lynn), and hiring them and starting programs around them was the thing Microsoft did to fix their relationship to the community, whereas the folks here were already insiders at Google, but maybe that was the problem, that they were explicitly hired as academics with the expectation that they would keep doing the same work, rather than internally focussed researchers working on improving the products and systems being created.
She was fired because she threatened to quit, and Google is just as incapable of stopping themselves from firing people as they are of not deleting all the Terraria guy's accounts. It's all robots and business processes, it doesn't matter what "the problem" was or if there was one.
> hiring hackers working in security
Hackers offer solution to your loophole, AI ethnics researchers only call you out and hijacking the narrative for their own benefits.
Hackers offer solution to your loophole, AI ethnics researchers only call you out and hijacking the narrative for their own benefits.
This is not generally true.
Security consultants / pros come in with the bias that your software is insecure. Then they strive to prove it.
If they do prove it, it is verifiable.
By any other researcher pretty quickly.
It is verifiable.
When a professional opinionator is brought in with a bias that this data / company is racist/ transphobic / not inclusive then drafts a report proving it.
It is much harder to validate it.
It cannot even be evaluated as a legal argument with references (or missing reference) to laws.
Ethics is inherently philosophical field. From it stems understanding and ideas that may be codified in an interpretation of them into laws. Then those laws are tested and case law and valid interpretations are formulated.
When a professional opinionator is brought in with a bias that this data / company is racist/ transphobic / not inclusive then drafts a report proving it.
It is much harder to validate it.
It cannot even be evaluated as a legal argument with references (or missing reference) to laws.
Ethics is inherently philosophical field. From it stems understanding and ideas that may be codified in an interpretation of them into laws. Then those laws are tested and case law and valid interpretations are formulated.
The difference is that the security is objectively verifiable - either they were able to compromise data/access/..., or not. Despite the different incentives for the company and the hackers, if you can't break in, you cannot just fabricate a vulnerability because your paycheck depends on it.
Whereas "ethics" is a subjective field, so you can invent arbitrary "ethics violations", and you would - precisely because your paycheck depends on it. This is not new, see also bioethics - at best it's completely useless, at worst it's basically self-sabotage and your competitors would be happy to see you engage in it.
Whereas "ethics" is a subjective field, so you can invent arbitrary "ethics violations", and you would - precisely because your paycheck depends on it. This is not new, see also bioethics - at best it's completely useless, at worst it's basically self-sabotage and your competitors would be happy to see you engage in it.
[deleted]
We need something like ombudsmans. Roles with true independence. Built in checks and balances.
I have no idea how to make that work.
Maybe something like legal codification, so that publicly traded companies and the various parties are granted some temporary liability armor when vulnerabilities (or equiv) are identified.
PS- Just had a notion. Probably dumb: Maybe the liability (re)insurer gets a seat on the board of directors.
I have no idea how to make that work.
Maybe something like legal codification, so that publicly traded companies and the various parties are granted some temporary liability armor when vulnerabilities (or equiv) are identified.
PS- Just had a notion. Probably dumb: Maybe the liability (re)insurer gets a seat on the board of directors.
Set up a regulatory project on github and allow people to post annonymous "issues" to be addressed in a public forum?
Been chewing on this idea. Thanks.
In the broadest sense, your suggestion is The Correct Answer™.
Every org should have some kind of funnel for feedback. Discoverable, transparent, accountable. Probably both public (external) and private (internal) funnels. CRMs by another name. Your notion of a GitHub repo-based issue system would leverage existing infra. Both familiar and cheap. No invention necessary.
In the cases with potential liability, it seems to me privileged access to confidential information will still be necessary. Even if that limited access is only time boxed (eg quiet periods before earnings reports).
In the broadest sense, your suggestion is The Correct Answer™.
Every org should have some kind of funnel for feedback. Discoverable, transparent, accountable. Probably both public (external) and private (internal) funnels. CRMs by another name. Your notion of a GitHub repo-based issue system would leverage existing infra. Both familiar and cheap. No invention necessary.
In the cases with potential liability, it seems to me privileged access to confidential information will still be necessary. Even if that limited access is only time boxed (eg quiet periods before earnings reports).
[deleted]
>I've written here once before that AI ethics researchers are like hiring hackers working in security
Except that afaik game studio actually hire people that worked on cheats - e.g Riot Games.
In order to be defender you also need to know how to attack.
Except that afaik game studio actually hire people that worked on cheats - e.g Riot Games.
In order to be defender you also need to know how to attack.
Google hired these researchers to give them some basic facade of having concerns about AI ethics. They were supposed to write some nonsense papers and just act as shields when questions about AI ethics were raised.
When these people bit Google on the hand, Google got rid of them. Simple story.
Google is a monopoly and needs to be broken up under anti trust rules.
When these people bit Google on the hand, Google got rid of them. Simple story.
Google is a monopoly and needs to be broken up under anti trust rules.
As sad as I think this reasoning is, I think it's true. Google wanted someone to shield them from scrutiny with some hand wavy papers, making them feel good about what they are doing (disrupting the market!11!1!), just as pharma pays studies to prove what they want. Those researched probably just didn't get the implicit agreement or requirement, that Google already did everything right and well. And obviously Google isn't racist or biased, and if they are, they are doing everything they can to combat that /s.
>I've written here once before that AI ethics researchers are like hiring hackers working in security
Exactly, PoC or GTFO!
(i'll be here all week...)
(edit: maybe)
Exactly, PoC or GTFO!
(i'll be here all week...)
(edit: maybe)
Google should have never started hiring ethical AI researchers. They should have hired AI researchers and made their stuff available to independent ethics researchers.
These independent researchers should be employed by i.e. a university and funded by government funds, maybe even additionally with an AI regulation fee (i.e. like car makers have to pay to certify their cars).
It makes no sense for them to hire their critics and then think they can stay independent. You are basically paying people to shit on your own products. Makes no sense.
These independent researchers should be employed by i.e. a university and funded by government funds, maybe even additionally with an AI regulation fee (i.e. like car makers have to pay to certify their cars).
It makes no sense for them to hire their critics and then think they can stay independent. You are basically paying people to shit on your own products. Makes no sense.
Yup, exactly. This is also the stance taken in the book Lingua Franca:
> Ethics in AI should not be so convoluted as to require a separate profession; ethics should be distributed across an organization, flowing through and between individuals so that conversations of design naturally arrive at ethical questions. More crucially, everyone should have the tools to speak up and suggest improvements, so that no one person or set of people hold the keys to building ethical technology.
https://linguafranca.polytopal.ai/handbook/agency
> Ethics in AI should not be so convoluted as to require a separate profession; ethics should be distributed across an organization, flowing through and between individuals so that conversations of design naturally arrive at ethical questions. More crucially, everyone should have the tools to speak up and suggest improvements, so that no one person or set of people hold the keys to building ethical technology.
https://linguafranca.polytopal.ai/handbook/agency
If we look at biology and medicine, there's both. The NIH requires everyone funded on their training grants to be given instruction in basic bioethics, and at the same time there is a profession of bioethics that is involved in grant review and oversight committees because a lot of cases aren't simple and covered by the basics. Bioethics enters the picture from regulation rather the way independent financial auditing does.
Also even with CS you have this. To use security as the comparable system, everyone involved in an application should be engaged and aware of security as a component and goal. Anyone should feel empowered to raise security concerns. You still have security experts who are more knowledgeable than the average developer who build (or collaborate on) security critical subcomponents and who audit/red-team the system to ensure security.
The same can be true of ethics.
The same can be true of ethics.
That's a great resource (that links to even more great resources) - thanks for posting it.
I disagree that professional ethicists aren't needed in an organisation like Google though. Just as security should be lived and breathed by every developer, you still need some experts to consult when things get gnarly, light the way when you need many years of study to understand the concepts, and spot things that laypeople would miss.
The large language models Timnit was studying are a great example of this; people like her were (and are) needed to help the engineers understand the breadth and scope of the issues. Everyone in the org having their hearts in the right place doesn't replace expertise.
I disagree that professional ethicists aren't needed in an organisation like Google though. Just as security should be lived and breathed by every developer, you still need some experts to consult when things get gnarly, light the way when you need many years of study to understand the concepts, and spot things that laypeople would miss.
The large language models Timnit was studying are a great example of this; people like her were (and are) needed to help the engineers understand the breadth and scope of the issues. Everyone in the org having their hearts in the right place doesn't replace expertise.
I think the article is taking the stance that ethics should be a concern at the point of development, and those that create these large language models and those that use them should be accountable. Right now, those groups are not accountable, so instead a separate group of people is asked to take responsibility. This is where AI Ethics as a field has ended up, and in my opinion, stalled.
I don't think these two agree? The go suggests keeping a separation between who's doing the researching and who's checking its ethics. Like, the ethics researchers should not be accountable to people who are incentivized to ignore them.
Instead, the quote suggests that the separation is unnecessary, and that elevating some people to specifically look at ethics is bad since it's everyone's job
Instead, the quote suggests that the separation is unnecessary, and that elevating some people to specifically look at ethics is bad since it's everyone's job
> You are basically paying people to shit on your own products. Makes no sense.
It's good to be critical of your own products to make them better. But you have to actually be willing to do that for it all to work.
It's good to be critical of your own products to make them better. But you have to actually be willing to do that for it all to work.
It’s a great resource to pay people to do that internally.
It’s a complete disaster to hire people who see their job description as shitting on the company publicly, with no constructive proposals for improvement.
It’s a complete disaster to hire people who see their job description as shitting on the company publicly, with no constructive proposals for improvement.
One of the criticisms in that paper was that training AI used a lot of electricity. That's technically true, but what's the proposal there? The power efficiency of hardware is already being increased.
How often is the electricity use considered a constraint in algorithm design for AI models?
Moore's law isn't the only way to gain efficiency, and even if the available compute goes up, doesn't mean the total power usage will go down
Moore's law isn't the only way to gain efficiency, and even if the available compute goes up, doesn't mean the total power usage will go down
Do less AI training, so less AI could be a valid alternative.
The proposal is not to continue the trend of simply adding titanic amounts of parameters to models and develop more efficient ones.
> what's the proposal there?
That's what Google was afraid of: the alternative is to decide that the cost of the model isn't worth the effort (externalities < benefit) and to abandon it. Which might be the correct decision, but one that loses Google a lot of potential money.
That's what Google was afraid of: the alternative is to decide that the cost of the model isn't worth the effort (externalities < benefit) and to abandon it. Which might be the correct decision, but one that loses Google a lot of potential money.
The paper didn't attempt to examine if the benefits of the model outweighed the costs. They calculated the total energy used and called it a day.
[deleted]
Correct. It left it up to the model creators to potentially evaluate those tradeoffs for future models.
I find the conclusions ludicrous. Should we also abandon particle accelerators because they use too much electrity for ill-defined benefits?
Really the claim is toxic academia style knife fighting for tenure at best.
Really the claim is toxic academia style knife fighting for tenure at best.
I bet all the big tech companies spend way more electricity on things other than training big models.
"You use too much electricity" is tantamount to "tech is a flawed business model". Arguably solvable with increased green energy sources but that's more a job for society in general than just tech companies.
"You use too much electricity" is tantamount to "tech is a flawed business model". Arguably solvable with increased green energy sources but that's more a job for society in general than just tech companies.
Part of the issue in these events is that the paper that was written by Google's own ethics researchers simply repeats estimates of AI energy consumption from outside academic papers, instead of examining them. The reality is that Google's ML infrastructure is dramatically more energy efficient than the rest of the industry. A person with access to these facts who chooses to ignore them isn't really doing their best as a researcher, they're just grinding an ax.
There are other examples against the meme of AI using "a lot of electricity", for example a recent paper from Google where they show that a learned computational fluid dynamics technique uses only 2% the energy of standard techniques.
https://arxiv.org/pdf/2102.01010.pdf
There are other examples against the meme of AI using "a lot of electricity", for example a recent paper from Google where they show that a learned computational fluid dynamics technique uses only 2% the energy of standard techniques.
https://arxiv.org/pdf/2102.01010.pdf
Not only are ML techniques finding ways to do physics with less energy, the amount of waste by non-ML computing absolutely swamps ML computing. I don't care if training a model uses the amount of CO2 that a translatlanic flight uses, because those flights are flying constantly, some of them not filled to the max. The whole criticism about energy in the paper seemed to miss major details that I'd expect top researchers to get right.
BTW, I ran a large scale computing platform at Google for several years. It made absolutely enormous, stunningly huge amounts of CPU available to scientists and we used it to make several important breakthroughs. However, after a few years, I finally was shown what the power cost implications were, and I shut down the project because the costs were not justified by the scientific benefits. Compared to what I was doing, ML is relatively skimpy on CPU.
BTW, I ran a large scale computing platform at Google for several years. It made absolutely enormous, stunningly huge amounts of CPU available to scientists and we used it to make several important breakthroughs. However, after a few years, I finally was shown what the power cost implications were, and I shut down the project because the costs were not justified by the scientific benefits. Compared to what I was doing, ML is relatively skimpy on CPU.
I'm not sure that your scientific facilities cumulatively spent as much CPU time as DeepMind spent playing StarCraft. If we want to talk about energy and ethics in machine learning I guess that's where I would start.
Yes, my calculations used far more time than DM, with the caveat that my jobs ran on CPUs while most of their training is on accelerators.
I believe we reported that Exacycle provided 700K high speed Xeon cores for over a year, and the actual number is far higher (I can't share it because you'd be able to make a reasonable estimate on how many computers Google has).
I believe we reported that Exacycle provided 700K high speed Xeon cores for over a year, and the actual number is far higher (I can't share it because you'd be able to make a reasonable estimate on how many computers Google has).
> I'm not sure that your scientific facilities cumulatively spent as much CPU time as DeepMind spent playing StarCraft.
You gotta pay if you wanna play. Humans are the result of a much longer and more expensive evolutionary process. It's only normal to have to pay a little to do the same for AI.
You gotta pay if you wanna play. Humans are the result of a much longer and more expensive evolutionary process. It's only normal to have to pay a little to do the same for AI.
The criticism would need to be constructive to be useful, or at least specific enough to act on. Also, you can't go embarrassing the parent company, especially one as image-conscious as google.
Any kind of public criticism is embarrassing from a PR perspective.
Either you accept that, or you don't.
"Ethics researches" that can't publish public criticism , with minimal oversight restricted to preventing IP leaks, are a pointless exercise in fake responsibility posturing.
Either you accept that, or you don't.
"Ethics researches" that can't publish public criticism , with minimal oversight restricted to preventing IP leaks, are a pointless exercise in fake responsibility posturing.
Agree to disagree. One of the main “enforcement arms” of ethics research is to say the unsaid things out loud. Saying them introduces legal liability onto the company in future lawsuits.
I.e. “Google knew that X would result in Y and they still released X; they should pay for the damages of Y.”
The PR side of ethics research barely matters in comparison.
I.e. “Google knew that X would result in Y and they still released X; they should pay for the damages of Y.”
The PR side of ethics research barely matters in comparison.
> Saying them introduces legal liability onto the company in future lawsuits.
My first thought was ExxonMobil when I read this comment.
> In July 1977, a senior scientist of Exxon James Black reported to the company's executives that there was a general scientific agreement at that time that the burning of fossil fuels was the most likely manner in which mankind was influencing global climate change.
> According to the Union of Concerned Scientists, "The funding of academic research activity has provided the corporation legitimacy, while it actively funds ideological and advocacy organizations to conduct a disinformation campaign."
Nothing has happened to Exxon Mobile. None of the executives are in prison as far as I know. You'd think the company would be bankrupt by now...
https://en.wikipedia.org/wiki/ExxonMobil_climate_change_cont...
My first thought was ExxonMobil when I read this comment.
> In July 1977, a senior scientist of Exxon James Black reported to the company's executives that there was a general scientific agreement at that time that the burning of fossil fuels was the most likely manner in which mankind was influencing global climate change.
> According to the Union of Concerned Scientists, "The funding of academic research activity has provided the corporation legitimacy, while it actively funds ideological and advocacy organizations to conduct a disinformation campaign."
Nothing has happened to Exxon Mobile. None of the executives are in prison as far as I know. You'd think the company would be bankrupt by now...
https://en.wikipedia.org/wiki/ExxonMobil_climate_change_cont...
Humanity has a fair history of introducing products that are later shown to have large externalized costs: tetraethyl lead, PVC, etc.
I interpret your point to be that publicly investigating the harm of future products---by the only people who can do so---is a bad idea. What is the alternative? Regulation? (Gasp!)
Perhaps we should start the disinformation campaigns before the products are developed. "If you are arrested on AI-provided evidence, you are guilty. Period."
I interpret your point to be that publicly investigating the harm of future products---by the only people who can do so---is a bad idea. What is the alternative? Regulation? (Gasp!)
Perhaps we should start the disinformation campaigns before the products are developed. "If you are arrested on AI-provided evidence, you are guilty. Period."
Which is why it’s idiotic to hire people whose entire job is to create bad publicity for the company.
Few (none?) companies would be happy to pay someone to be publicly and loudly critical of their products. Especially if it wasn't constructive criticism that they could act on.
Completely agree with independent auditing. If automotive safety had been deregulated, the body count would be orders of magnitude higher. This chart says it all:
https://en.wikipedia.org/wiki/Motor_vehicle_fatality_rate_in...
The regulation of automotive safety in the 60's/70's had a huge impact.
> It makes no sense for them to hire their critics and then think they can stay independent. You are basically paying people to shit on your own products. Makes no sense.
Shit on your own products?
So if I'm a good engineer and fix mistakes, I'm "shitting on my company's product?"
There's this belief in this post that un-examined AI is some shiny golden award, and ethics researchers are tarnishing it. It's the other way around.
https://en.wikipedia.org/wiki/Motor_vehicle_fatality_rate_in...
The regulation of automotive safety in the 60's/70's had a huge impact.
> It makes no sense for them to hire their critics and then think they can stay independent. You are basically paying people to shit on your own products. Makes no sense.
Shit on your own products?
So if I'm a good engineer and fix mistakes, I'm "shitting on my company's product?"
There's this belief in this post that un-examined AI is some shiny golden award, and ethics researchers are tarnishing it. It's the other way around.
Does that not depend on your perspective? There might not be an alignment between what Google’s executives and shareholders want out of the company, and what an ethics researcher might consider ethical behavior.
> Does that not depend on your perspective?
Well, it depends on your breadth of understanding of the issue. If you have a limited scope, it doesn't seem like a big problem. And it also assumes that we all agree on certain basic human rights. If we don't then yes, it is a matter of perspective.
Read "Weapons of Math Destruction". It discusses how existing algorithms discriminate in the following case-studies:
- courtroom sentencing (and recidivism prediction)
- mortgage and loan rate determination
- educator performance
- job applications that use 3rd party screening tools
The book looks at specific examples of where these black-box algorithms are deployed that have ruined people's lives with no accountability. The ethics concerns that are being raised were needed a decade ago, or more, but it is only getting worse with the ad-hoc deployment of this un-baked technology to almost every industry.
I have to stress again that these black-box algorithms are ALREADY IN USE and cannot be subpoenaed by courts because it is considered intellectual property.
So yes, it is urgent to crack open this technology because it is all too easily being sold without any investigation, accountability, or thought to the consequences. "Move fast and break things" doesn't work if it ruins peoples lives by landing them in jail, or pushing them into poverty.
The book demonstrates this is not a what-if strawman, but reality.
Well, it depends on your breadth of understanding of the issue. If you have a limited scope, it doesn't seem like a big problem. And it also assumes that we all agree on certain basic human rights. If we don't then yes, it is a matter of perspective.
Read "Weapons of Math Destruction". It discusses how existing algorithms discriminate in the following case-studies:
- courtroom sentencing (and recidivism prediction)
- mortgage and loan rate determination
- educator performance
- job applications that use 3rd party screening tools
The book looks at specific examples of where these black-box algorithms are deployed that have ruined people's lives with no accountability. The ethics concerns that are being raised were needed a decade ago, or more, but it is only getting worse with the ad-hoc deployment of this un-baked technology to almost every industry.
I have to stress again that these black-box algorithms are ALREADY IN USE and cannot be subpoenaed by courts because it is considered intellectual property.
So yes, it is urgent to crack open this technology because it is all too easily being sold without any investigation, accountability, or thought to the consequences. "Move fast and break things" doesn't work if it ruins peoples lives by landing them in jail, or pushing them into poverty.
The book demonstrates this is not a what-if strawman, but reality.
Thanks, I definitely understand the human rights implications and the importance. I mostly expect a large corporation to make decisions based on profits over ethics though, unless the former depends on the latter (through pressure from the public/PR, legislation and so on). From that perspective, I could see Google regarding an employee remarking negatively on the ethical aspects of an otherwise popular product as ”shitting on it”.
"Weapons of Math Destruction" was our textbook for CS 6603 at Georgia Tech. It was a fascinating/terrifying look into some of the systems that seem to silently impact the lives of millions in the U.S. Cannot recommend it enough.
> it is urgent to crack open this technology
Just as easy as putting the baby back in. It would be more constructive to work on ways to detect and reduce harm.
Just as easy as putting the baby back in. It would be more constructive to work on ways to detect and reduce harm.
Bullshit. It is literally being made right now. Legislation can help solve this pretty quick. No audit? No sell.
To follow your analogy, it is more like, "wear a damn condom and take a sex-ed class before having an unwanted baby."
To follow your analogy, it is more like, "wear a damn condom and take a sex-ed class before having an unwanted baby."
I don't understand how your conclusion follows from the graph. That data would appear to indicate the opposite: that regulation had no effect.
The graph shows that vehicle safety improved steadily throughout the entire 20th century, without any sharp discontinuities. The key line is deaths per billion VMT. That is the figure which is invariant to how much driving people do, and which thus accurately measures how safe driving as an activity is. You appear to be looking at "deaths per million people" but that figure is confounded by big changes driven by the economy, which is why the 1970s energy crisis is highlighted on the graph. All you're seeing there is economic change, not change driven by regulation.
If regulation would have actually made a difference then it'd show up in the red line, but it doesn't. Conclusion: the regulation is useless. Private companies were doing fine at improving safety and always have been. And as regulations have costs, that in turn suggests they should be scrapped (zero impact + non-zero costs = repeal).
The graph shows that vehicle safety improved steadily throughout the entire 20th century, without any sharp discontinuities. The key line is deaths per billion VMT. That is the figure which is invariant to how much driving people do, and which thus accurately measures how safe driving as an activity is. You appear to be looking at "deaths per million people" but that figure is confounded by big changes driven by the economy, which is why the 1970s energy crisis is highlighted on the graph. All you're seeing there is economic change, not change driven by regulation.
If regulation would have actually made a difference then it'd show up in the red line, but it doesn't. Conclusion: the regulation is useless. Private companies were doing fine at improving safety and always have been. And as regulations have costs, that in turn suggests they should be scrapped (zero impact + non-zero costs = repeal).
- You are basically paying people to shit on your own products. Makes no sense.
They aren't really researching ethics. This is what i think is going on there. recycling an old comment.
{ AI Ethical research can never be an accurate representation of the majority of humanity - or even the majority of its users! If it can it is not sustainable for any long period of time.
What about far more stable principles, such as murder and racism, you ask?
They are prone to being overplayed or downplayed, are state executions murder, or justice? What if the victim/ hangman happens to be black? Why should it matter? Just ignore some issues? That's misleading by omission.
It would be better to just admit "yes, we at Giant Tech know our ethics are bs, but we had to put something down or our machines won't work. Maybe we are not ready for advanced ai. Maybe there's a limit on what programmers can do. Yea. We know. turns out computers DO have limits. We'll have to find other ways to make money."
"But why should we say that!" cry all the executives when this speech is proposed to them.
"because it's true, and if we don't act now the company is screwed. And our clients will also get screwed" is the answer of the timid executive who first suggested this.
"how will the truth help us?" respond all the execs in unison.
"if we can keep up the lie for long enough, we shall all long be millionaires and retired before it implodes! We shall long be out of danger! who cares if some people lose money?"
"Yes, but don't you feel bad for all the shareholders? And how can we possibly fool people for decades to come that our ai isn't bs?" Responds the poor executive weakly.
"By creating a fake team and telling everyone they are ethics researchers" they say. "Really they are just pawns to help us earn more money. Fish get eaten by bigger fish you know?"
"can you just help me change a few lines on my press statement?" Asks the first executive.}
They aren't really researching ethics. This is what i think is going on there. recycling an old comment.
{ AI Ethical research can never be an accurate representation of the majority of humanity - or even the majority of its users! If it can it is not sustainable for any long period of time.
What about far more stable principles, such as murder and racism, you ask?
They are prone to being overplayed or downplayed, are state executions murder, or justice? What if the victim/ hangman happens to be black? Why should it matter? Just ignore some issues? That's misleading by omission.
It would be better to just admit "yes, we at Giant Tech know our ethics are bs, but we had to put something down or our machines won't work. Maybe we are not ready for advanced ai. Maybe there's a limit on what programmers can do. Yea. We know. turns out computers DO have limits. We'll have to find other ways to make money."
"But why should we say that!" cry all the executives when this speech is proposed to them.
"because it's true, and if we don't act now the company is screwed. And our clients will also get screwed" is the answer of the timid executive who first suggested this.
"how will the truth help us?" respond all the execs in unison.
"if we can keep up the lie for long enough, we shall all long be millionaires and retired before it implodes! We shall long be out of danger! who cares if some people lose money?"
"Yes, but don't you feel bad for all the shareholders? And how can we possibly fool people for decades to come that our ai isn't bs?" Responds the poor executive weakly.
"By creating a fake team and telling everyone they are ethics researchers" they say. "Really they are just pawns to help us earn more money. Fish get eaten by bigger fish you know?"
"can you just help me change a few lines on my press statement?" Asks the first executive.}
It may also be that Google didn't realize the extent to which they were inviting some of the more organizationally destructive and divisive elements of modern politically correct culture into their organization: https://jakeseliger.com/2020/12/03/dissent-insiders-and-outs.... It may be that we're going to see a trend towards "Leave politics and religion at the door."
To be honest pretty much any bigger professional organization already operates under "Leave politics and religion at the door." umbrella. It's pretty much the only way how you can have an organization filled with mixed political viewpoints function in the same direction.
For some reason however, Google and their employees kinda fail to understand this basic tenement of work culture. They're not a scrappy 10 people startup anymore which can be both a workplace and a political party fully agreeing on political issues.
For some reason however, Google and their employees kinda fail to understand this basic tenement of work culture. They're not a scrappy 10 people startup anymore which can be both a workplace and a political party fully agreeing on political issues.
> It may be that we're going to see a trend towards "Leave politics and religion at the door."
Don't get my hopes up.
Don't get my hopes up.
-It may be that we're going to see a trend towards "Leave politics and religion at the door."
what are you left with? misinformation is far more prolific than truth. it needs to be held in check, somehow. ignoring it will make it waorse
what are you left with? misinformation is far more prolific than truth. it needs to be held in check, somehow. ignoring it will make it waorse
The bigger the company, the stronger the motive to control politics and religion (for profit etc.)
I imagine that the conversation beneath that umbrella will be flattened in a day.
I imagine that the conversation beneath that umbrella will be flattened in a day.
>and racism
Excuse me, but the definition of racism in the west is anything but stable. The definition has changed dramatically in the last decade, and it could change again.
Excuse me, but the definition of racism in the west is anything but stable. The definition has changed dramatically in the last decade, and it could change again.
that merely reinforces my point. Though i accept it was inaccurate of me.
> You are basically paying people to shit on your own products.
In reality, if you want them to fulfill their true role, they are preventing you from producing wrong products (which should be good for the Company).
But Google understood that as you say (which is not what I think but this is beside the point), and that was their mistake.
In reality, if you want them to fulfill their true role, they are preventing you from producing wrong products (which should be good for the Company).
But Google understood that as you say (which is not what I think but this is beside the point), and that was their mistake.
That makes sense to pay someone to perform that function privately and internally.
It’s been a disaster for Google to have that happen in public.
It’s been a disaster for Google to have that happen in public.
I think you will find that hiring skilled researchers and telling them that they cannot say anything publicly is a hard row to hoe.
Apple seems to be doing that just fine and with huge success. They are exceedingly good at silencing every single employee to not speak about any kind of internal dissent or HR issues like this.
Perhaps, but hiring skilled researchers who do not think they need to say whatever they want is probably not as hard.
Importantly, Google offers great work conditions, access to their intellectual property and different evaluation criteria. Many prefer that and other benefits over the stresses of a proper academic career.
Importantly, Google offers great work conditions, access to their intellectual property and different evaluation criteria. Many prefer that and other benefits over the stresses of a proper academic career.
This is the most curious part: hiring those two individual researchers, who clearly didn't have an understanding of the nature of public communication, was incredibly naive and asking for disaster. There is no other outcome. It kind of implies a total lack of understanding of human nature on the part of those doing the hiring. "What were they thinking?" is the only thing that comes to mind.
It's a very, very tricky thing to do to hire your own public critics, that has to be done very carefully and with a lot of parameters.
It's a very, very tricky thing to do to hire your own public critics, that has to be done very carefully and with a lot of parameters.
> You are basically paying people to shit on your own products. Makes no sense.
They hired them to control them. And it's a diversion to keep people thinking about the wrong things.
They hired them to control them. And it's a diversion to keep people thinking about the wrong things.
Another useful (to Google) aspect of hiring an ethical AI team is also that is allows Google to shut down ethical concerns raised in other parts of its own organization by going "we agree that ethics is an important, but please focus on your own job", i.e. an internal AI team can act as a sort of "controlled opposition". The fact that the ethical AI team is now being undermined within the company after trying to do their job and making clear that they are not content to be relegated to such a role is pretty consistent with this motive.
Yes, and just to expand on what I meant with "thinking about the wrong things", they've got everyone thinking about killer robots and automated weapons systems and surveillance (by government entities). When in reality Google's AI is, and has long been, vastly deployed to understand human behavior for the express purpose of maximizing advertising revenue and modification of behavior through pervasive messaging, recommendations, addictive dark patterns, etc. They've basically built a completely automated real distortion system designed to trap people into spending their time staring at whatever the highest (fully automated) bidder thinks they should see. It's dystopian and all the local incentives are wrong now. They've got a crapton of money, computational power, and self-justification. The reality distortion field is powerful. We should be very wary.
Yeah, it was a PR move.
No, Google should never have messed with their researchers’ publications; it’s always the coverup that becomes the story. Contrast Google Research with Microsoft Research, which enjoys a stellar reputation and independence.
This is upside down.
There is no reason for Google to pay researchers and to not have some ownership over the process or outcome in some manner, in fact, it's unreasonable to contemplate otherwise.
Google was exceedingly gracious by sponsoring the work and having it make public in the first place.
The demands made of the researchers in question were quite minor, and completely appropriate, and frankly immaterial to the thrust of concerns anyhow.
To boot, accusations of 'racism' are vicious and repulsive.
If researchers want to be completely independent in the terms they might expect in academia - that's entirely understandable - perfectly fine - but they'll have to find jobs elsewhere.
Google has a Search Engine and bunch of other products, and to suggest that their major investment in research should entitle them to some kind of fair parameterize of the process is very reasonable.
To remedy the situation, and an obvious failure on Google's part, they should spend some time thinking about where that line is drawn, and what kind of expectations should be in place. In retrospect, it seems like a management failure for there to have been any misunderstanding on that front. But the point is -> there is a line. Researchers that don't want to with those parameters can make the choice not to. That's why we have both private and publicly funded research.
There is no reason for Google to pay researchers and to not have some ownership over the process or outcome in some manner, in fact, it's unreasonable to contemplate otherwise.
Google was exceedingly gracious by sponsoring the work and having it make public in the first place.
The demands made of the researchers in question were quite minor, and completely appropriate, and frankly immaterial to the thrust of concerns anyhow.
To boot, accusations of 'racism' are vicious and repulsive.
If researchers want to be completely independent in the terms they might expect in academia - that's entirely understandable - perfectly fine - but they'll have to find jobs elsewhere.
Google has a Search Engine and bunch of other products, and to suggest that their major investment in research should entitle them to some kind of fair parameterize of the process is very reasonable.
To remedy the situation, and an obvious failure on Google's part, they should spend some time thinking about where that line is drawn, and what kind of expectations should be in place. In retrospect, it seems like a management failure for there to have been any misunderstanding on that front. But the point is -> there is a line. Researchers that don't want to with those parameters can make the choice not to. That's why we have both private and publicly funded research.
Absolutely, that's the only sane stance towards it. The original poster seems to imply that it's impossible to do sustainable ethical business because of conflicts of interests, but there are many ways to deal with such conflicts of interests without firing researchers or suppressing papers.
This case was really just mismanagement by Google. They could have just as well taken those researchers seriously, allow them to investigate whether there is systemic discrimination and then try to correct it, as was the whole point of hiring them in the first place. AI ethics researchers all over the world are publishing articles about the dangers of serious discrimination by large language models. Not addressing this and firing researchers who were working on it on their behalf is not going to help Google in the long run at all.
This case was really just mismanagement by Google. They could have just as well taken those researchers seriously, allow them to investigate whether there is systemic discrimination and then try to correct it, as was the whole point of hiring them in the first place. AI ethics researchers all over the world are publishing articles about the dangers of serious discrimination by large language models. Not addressing this and firing researchers who were working on it on their behalf is not going to help Google in the long run at all.
How do you know that the publication wasn't pulled because they genuinely thought it didn't meet their quality standards? When this happens at Microsoft research, the author makes a few revisions and moves on rather than accuse Microsoft of racism.
Review of the quality of publications should never be made by your indirect superiors. If you want to have an internal quality review process, fine, but the people have to be peers or immediate superiors, and the criticism must be public and must not be related to the company's bottom line, in order to allow for revisions to increase the quality before peer review.
Gebru and Mitchell were both previously employed by MSR in the same context without issue.
Google didn't allow the team to make revisions. They stated the paper couldn't be published, period. The difference here is very clearly the management, not the researchers.
Google didn't allow the team to make revisions. They stated the paper couldn't be published, period. The difference here is very clearly the management, not the researchers.
The paper was submitted late. Because of this, and probably previous actions, there was a special go/no-go review instead of a weeks-long back and forth.
If the paper was good it could go through the process and be ready for the next conference. Nobody is seriously suggesting that those words were axed forever.
If the paper was good it could go through the process and be ready for the next conference. Nobody is seriously suggesting that those words were axed forever.
This is factually inaccurate in a variety of ways.
> The paper was submitted late.
This is not true. The paper was submitted for internal review with about the median amount of notice. The normal internal review process was and is usually very quick, it is not normally a weeks-long back and forth. Importantly, the paper actually passed the normal review process before being submitted to the conference.
> there was a special go/no-go review
It's unclear to what extent the special review was a go/no-go review. It may have been. The review however was not due to the paper being submitted late. As far as I know the special review was actually only done retroactively, after the paper had been submitted externally and approved by the normal internal process.
> If the paper was good it could go through the process and be ready for the next conference.
Even after submission for conference review, there was ample time for back and forth and review internally. Timnit asked if there was feedback she could incorporate into the paper, as even after the paper was submitted to the conference, they could revise it. Google's response was initially to not provide any specific feedback, and then to say that the feedback could not be addressed, the paper could only be unsubmitted.
> Nobody is seriously suggesting that those words were axed forever.
Actually yes, that appears to be precisely what happened.
> The paper was submitted late.
This is not true. The paper was submitted for internal review with about the median amount of notice. The normal internal review process was and is usually very quick, it is not normally a weeks-long back and forth. Importantly, the paper actually passed the normal review process before being submitted to the conference.
> there was a special go/no-go review
It's unclear to what extent the special review was a go/no-go review. It may have been. The review however was not due to the paper being submitted late. As far as I know the special review was actually only done retroactively, after the paper had been submitted externally and approved by the normal internal process.
> If the paper was good it could go through the process and be ready for the next conference.
Even after submission for conference review, there was ample time for back and forth and review internally. Timnit asked if there was feedback she could incorporate into the paper, as even after the paper was submitted to the conference, they could revise it. Google's response was initially to not provide any specific feedback, and then to say that the feedback could not be addressed, the paper could only be unsubmitted.
> Nobody is seriously suggesting that those words were axed forever.
Actually yes, that appears to be precisely what happened.
> This is not true. The paper was submitted for internal review with about the median amount of notice.
This is true. There was a requested amount of time and it was not met. To show discrimination you'd also have to show median quality and median editing work required. This paper had a lot of flaws that would be hard to correct without writing it again.
> Importantly, the paper actually passed the normal review process before being submitted to the conference.
But not the corporate review where the people paying the money decided if they thought it was good to attach their name to.
> It's unclear to what extent the special review was a go/no-go review.
Not at all. There was a strong 'No Go' message given. That's how you know.
> The review however was not due to the paper being submitted late.
The "don't bother fixing stuff, we're just kiboshing it" part was.
> after the paper had been submitted externally and approved by the normal internal process.
The normal internal process is about writing quality and fact-checking. The corporate review is entirely separate and always last in the process in order to see the latest version.
> Even after submission for conference review, there was ample time for back and forth and review internally.
No, not for the level of rework they thought it should receive. If it had passed then sure. The reason to meet the deadline is to handle the situation where it doesn't pass on the first go.
> Timnit asked if there was feedback she could incorporate into the paper
And her bosses decided that the paper wasn't worth the last-minute work. I imagine they felt she submitted it late to game the system.
> Google's response was initially to not provide any specific feedback, and then to say that the feedback could not be addressed, the paper could only be unsubmitted
Right, because it wasn't feedback to her. That ship had sailed. It was confidential feedback to her boss about their belief in the quality of the work. To share that feedback would put innocent employees in a Damore position, attacked for feedback they'd been asked to write. Given how she attacked her boss by name it's reasonable to think she'd have included the coworkers, where they'd be pounced on by an angry twitter mob.
> Actually yes, that appears to be precisely what happened.
No. The preprint was released. Google just said "not in our name". And now that she resigned she has the freedom.
This is true. There was a requested amount of time and it was not met. To show discrimination you'd also have to show median quality and median editing work required. This paper had a lot of flaws that would be hard to correct without writing it again.
> Importantly, the paper actually passed the normal review process before being submitted to the conference.
But not the corporate review where the people paying the money decided if they thought it was good to attach their name to.
> It's unclear to what extent the special review was a go/no-go review.
Not at all. There was a strong 'No Go' message given. That's how you know.
> The review however was not due to the paper being submitted late.
The "don't bother fixing stuff, we're just kiboshing it" part was.
> after the paper had been submitted externally and approved by the normal internal process.
The normal internal process is about writing quality and fact-checking. The corporate review is entirely separate and always last in the process in order to see the latest version.
> Even after submission for conference review, there was ample time for back and forth and review internally.
No, not for the level of rework they thought it should receive. If it had passed then sure. The reason to meet the deadline is to handle the situation where it doesn't pass on the first go.
> Timnit asked if there was feedback she could incorporate into the paper
And her bosses decided that the paper wasn't worth the last-minute work. I imagine they felt she submitted it late to game the system.
> Google's response was initially to not provide any specific feedback, and then to say that the feedback could not be addressed, the paper could only be unsubmitted
Right, because it wasn't feedback to her. That ship had sailed. It was confidential feedback to her boss about their belief in the quality of the work. To share that feedback would put innocent employees in a Damore position, attacked for feedback they'd been asked to write. Given how she attacked her boss by name it's reasonable to think she'd have included the coworkers, where they'd be pounced on by an angry twitter mob.
> Actually yes, that appears to be precisely what happened.
No. The preprint was released. Google just said "not in our name". And now that she resigned she has the freedom.
Exactly this. Indeed, looking at Timnit Gebru as an example of someone who worked at both, I'd be very surprised if her paper that was censored at Google would have had any issues in being published if she were still working for Microsoft. Despite the research being, by implication, critical of a growing market segment in Azure, regarding its environmental impact and so on.
> You are basically paying people to shit on your own products. Makes no sense.
Self regulation is preferable (to a private company) to industry-wide regulation, which in turn is preferable to government regulation.
If you want to avoid government oversight, either have internal oversight with the appearance of independence. Alternatively, have an "independent" overseeing body for your industry (whose behavior you may still indirectly control via nominations, seats on boards, funding, etc). Having an actual independent body providing oversight, appointed and controlled by an arm of government body is the worst outcome, yet it is inevitable when lapses arise, and voters demand "something be done" and you failed to provide a veneer of oversight, like Google is doing here.
Self regulation is preferable (to a private company) to industry-wide regulation, which in turn is preferable to government regulation.
If you want to avoid government oversight, either have internal oversight with the appearance of independence. Alternatively, have an "independent" overseeing body for your industry (whose behavior you may still indirectly control via nominations, seats on boards, funding, etc). Having an actual independent body providing oversight, appointed and controlled by an arm of government body is the worst outcome, yet it is inevitable when lapses arise, and voters demand "something be done" and you failed to provide a veneer of oversight, like Google is doing here.
> funded by government funds, maybe even additionally with an AI regulation fee
Ahh, rent-seeking at it's finest. Do we really want to use the tax payer's money to fund this type of behaviors[0]?
[0] https://syncedreview.com/2020/06/30/yann-lecun-quits-twitter...
Ahh, rent-seeking at it's finest. Do we really want to use the tax payer's money to fund this type of behaviors[0]?
[0] https://syncedreview.com/2020/06/30/yann-lecun-quits-twitter...
It's good for Yann LeCun to quit Twitter if he's going to say silly things like "AI bias can always be fixed by collecting data" which is not true and not how his own employees fix issues - it's certainly not how the other FAANGs solve problems involving bias. Problems are perfectly capable of happening even if your data is the best it can be!
> he's going to say silly things like "AI bias can always be fixed by collecting data"
Except he never said that. He pointed out one algorithm was lazily trained on a non-representative dataset and that it was, therefore, producing biased results.
Except he never said that. He pointed out one algorithm was lazily trained on a non-representative dataset and that it was, therefore, producing biased results.
The model was also new, so you can't be sure improving the data without doing anything else would fix it. You have to actually try it first.
> Google should have never started hiring ethical AI researchers. They should have hired AI researchers and made their stuff available to independent ethics researchers.
If you're going to submit your work to an external, independent review board, it's even more important that you have internal people working to address related concerns.
If you're going to submit your work to an external, independent review board, it's even more important that you have internal people working to address related concerns.
[deleted]
And what qualifies anyone to be an 'AI ethicist' anyhow?
This is a heavy question - it's problematically like elevating an individuals personal moral position, with all their little trivial baggage to something more than it is.
The challenge with these roles is very real: self awareness, objectivity, and processes for arriving at that are probably key. I suggest it would require a lot of engagement, feedback, possibly legal knowledge, reference etc..
AI credibility aside, the individuals chosen by Google previously were not remotely up for the job on a professional level, this was apparent from their public posture and even writing skills, and that's not slander, it's hard job.
As the OP indicates, some degree of objectivity as well, which may be difficult from within the company.
Finally, I would say it's entirely doubtful if AI requires it's own special branch of ethics. Everything Google does touches on ethical issues, and 'plain old search' much more so than perhaps anything else - certainly more than any of their core AI. It's a business and technology issue, not an AI issue. It just may require some insight from those familiar with the company.
All of that said, Google may lose the 'PR cover' of having a woman or PoC in the driver's seat.
This is a heavy question - it's problematically like elevating an individuals personal moral position, with all their little trivial baggage to something more than it is.
The challenge with these roles is very real: self awareness, objectivity, and processes for arriving at that are probably key. I suggest it would require a lot of engagement, feedback, possibly legal knowledge, reference etc..
AI credibility aside, the individuals chosen by Google previously were not remotely up for the job on a professional level, this was apparent from their public posture and even writing skills, and that's not slander, it's hard job.
As the OP indicates, some degree of objectivity as well, which may be difficult from within the company.
Finally, I would say it's entirely doubtful if AI requires it's own special branch of ethics. Everything Google does touches on ethical issues, and 'plain old search' much more so than perhaps anything else - certainly more than any of their core AI. It's a business and technology issue, not an AI issue. It just may require some insight from those familiar with the company.
All of that said, Google may lose the 'PR cover' of having a woman or PoC in the driver's seat.
While I don’t personally think her race or gender should matter, the new head of their AI ethics org is Marion Croak, so they’ll retain the PR cover you’re describing.
The last sentence here makes no sense - this is Google RESEARCH we’re talking about, and the issue is around the suppression of research, the sneaky modification of the papers by lawyers and then doubling down on the ensuing shitshow of recriminations, dirty-laundry-airing and anger of 2700 employees by firing people. Including the founder of the AI ethics unit.
What’s their top priority?
Optics? Or better AI? People doing research are not automatically critics, but it is their literal job in this area to ask the hard, hard questions, this is not Harry and Megan go Oprah, this is technology with far-reaching consequences.
What’s their top priority?
Optics? Or better AI? People doing research are not automatically critics, but it is their literal job in this area to ask the hard, hard questions, this is not Harry and Megan go Oprah, this is technology with far-reaching consequences.
Well, Google says they didn't surpress research, but that the researchers didn't follow process. Now we are all having an argument - we wouldn't have that if the researchers were truly independent.
not just that, they fired a toxic employee who was bullying her colleagues on social media and even tried to bully her bosses by throwing ultimatums.
This whole drama has nothing to do with neither AI nor ethics.
This whole drama has nothing to do with neither AI nor ethics.
> You are basically paying people to shit on your own products.
I mean, the only way to fix bad stuff is to find it?
I mean, the only way to fix bad stuff is to find it?
Sure, but who is going to have a fair look? Someone who is employed by the company or someone who is independent?
VW faked their emissions - EPA uncovered it. Google is harming people with bad AI models - Google is uncovering it?
This works if the company is actually interested in finding the mistakes (i.e. MS finding security bugs in Windows), but not if the findings threaten your business.
MS has an interest in fixing security bugs, because it makes their product better.
No big corp has an interest in fixing their ethics, it's usually bad for their profit margins.
VW faked their emissions - EPA uncovered it. Google is harming people with bad AI models - Google is uncovering it?
This works if the company is actually interested in finding the mistakes (i.e. MS finding security bugs in Windows), but not if the findings threaten your business.
MS has an interest in fixing security bugs, because it makes their product better.
No big corp has an interest in fixing their ethics, it's usually bad for their profit margins.
Actually it wasn't even the EPA that uncovered it. It was academia. Corporations and governments just don't have the curiosity built in to their culture that academia does.
Well, there's at least 2 perspectives here.
If the two people were fired due to race, gender, or even ethical opinion, this reaction is more than appropriate.
If the two people were fired due to breaking established rules and processes, exfiltrating files or similar — this can very well backfire on minorities. It indirectly creates another hurdle to hiring a minority member if the company needs to consider that they can't viably fire them for breaking rules.
I'm unaware of good public information supporting either perspective. And both the ex-employees and Google may be restricted from publishing sensitive information to support their cause. I'm not sure how to form an informed opinion on this, honestly.
If the two people were fired due to race, gender, or even ethical opinion, this reaction is more than appropriate.
If the two people were fired due to breaking established rules and processes, exfiltrating files or similar — this can very well backfire on minorities. It indirectly creates another hurdle to hiring a minority member if the company needs to consider that they can't viably fire them for breaking rules.
I'm unaware of good public information supporting either perspective. And both the ex-employees and Google may be restricted from publishing sensitive information to support their cause. I'm not sure how to form an informed opinion on this, honestly.
The idea of something “backfiring on minorities” as an entire group is part of the problem here. The fact that there was a backlash after someone was fired can’t be used as a justification for not hiring other people whose only relationship to the original person is “also some kind of minority”.
Of course it's part of the problem. But it's also a somewhat unavoidable side effect of supporting minorities.
In a very weird way, anti-discriminative behaviour itself needs to be discriminative, because it needs to support minorities. And rightfully so, there needs to be a positive bias for minorities who were historically disenfranchised. Otherwise the historical disadvantage will propagate itself further. But then when you look at firings like this, it becomes very hard to distinguish whether it's discrimination or rightful. These two are members of minority groups. Their firing is, rightfully, examined to fine detail. But that same magnification lens has that chilling effect for the decision of hiring minority members.
It's really a have-your-cake-and-eat-it problem. It f_cking sucks. And I don't know how to do better. I absolutely think we need affirmative action to relieve historic deficits, but at the same time this affirmative support is in itself discriminatory and can have negative effects. What to do? :(
In a very weird way, anti-discriminative behaviour itself needs to be discriminative, because it needs to support minorities. And rightfully so, there needs to be a positive bias for minorities who were historically disenfranchised. Otherwise the historical disadvantage will propagate itself further. But then when you look at firings like this, it becomes very hard to distinguish whether it's discrimination or rightful. These two are members of minority groups. Their firing is, rightfully, examined to fine detail. But that same magnification lens has that chilling effect for the decision of hiring minority members.
It's really a have-your-cake-and-eat-it problem. It f_cking sucks. And I don't know how to do better. I absolutely think we need affirmative action to relieve historic deficits, but at the same time this affirmative support is in itself discriminatory and can have negative effects. What to do? :(
>there needs to be a positive bias for minorities who were historically disenfranchised
>I absolutely think we need affirmative action to relieve historic deficits, but at the same time this affirmative support is in itself discriminatory and can have negative effects. What to do?
Quit affirmative action at anything above the high school level and poor resources into fixing the issues before high school. Personally I reject the framing that we need it and fully agree with you about the negative effects. The long term effects of people "being given" jobs or school placement I think are more negative than positive and you are seeing that. The Irish, Italian, Jews, Chinese, and Koreans were all able to get to parity or better without affirmative action.
>I absolutely think we need affirmative action to relieve historic deficits, but at the same time this affirmative support is in itself discriminatory and can have negative effects. What to do?
Quit affirmative action at anything above the high school level and poor resources into fixing the issues before high school. Personally I reject the framing that we need it and fully agree with you about the negative effects. The long term effects of people "being given" jobs or school placement I think are more negative than positive and you are seeing that. The Irish, Italian, Jews, Chinese, and Koreans were all able to get to parity or better without affirmative action.
> The Irish, Italian, Jews, Chinese, and Koreans were all able to get to parity or better without affirmative action.
This of course assumes that genetic (racial) differences in intellectual (and physical, for that matter) ability do not exist. This has never been proven to be the case [1], and never will.
We all know dog breeds have different intellectual and physical characteristics. Why are humans supposedly immune from this? Because it is a morbid reality that throws a wrench in the "all men are created equal" liberal lie.
This is not me advocating for gross eugenics or even discrimination. I just want to point out something frequently missing from these conversations.
"The worst form of inequality is to try to make unequal things equal." – Aristotle
[1]: https://en.wikipedia.org/wiki/Heritability_of_IQ
This of course assumes that genetic (racial) differences in intellectual (and physical, for that matter) ability do not exist. This has never been proven to be the case [1], and never will.
We all know dog breeds have different intellectual and physical characteristics. Why are humans supposedly immune from this? Because it is a morbid reality that throws a wrench in the "all men are created equal" liberal lie.
This is not me advocating for gross eugenics or even discrimination. I just want to point out something frequently missing from these conversations.
"The worst form of inequality is to try to make unequal things equal." – Aristotle
[1]: https://en.wikipedia.org/wiki/Heritability_of_IQ
Biologist with a background in genetics here- there is no compelling evidence for statistically valid differences in measured intelligence on a racial genotypic basis. Most attempts to sort races by intelligence (e.g. The Bell Curve) end up being poorly designed junk science.
Dr. Adam Rutherford is a great pop science presence for debunking, or at the very least, addressing the claims that intelligence tracks with race on a genetic basis if you're truly interested in this topic.
https://youtu.be/3F0YSyeQavU
Dr. Adam Rutherford is a great pop science presence for debunking, or at the very least, addressing the claims that intelligence tracks with race on a genetic basis if you're truly interested in this topic.
https://youtu.be/3F0YSyeQavU
I agree with some of Taleb's critiques of IQ as a whole, and am quite interested in this topic. I will check out Dr. Rutherford's talk, thanks!
It’s fascinating that you’ve so casually compared people to dogs.
Out of curiosity, of the groups of people that are impacted by affirmative action, which groups remind you of which breeds and why? For example, are women border collies? Or some other breed?
Out of curiosity, of the groups of people that are impacted by affirmative action, which groups remind you of which breeds and why? For example, are women border collies? Or some other breed?
Every day people have innocuous conversations such as "Ohhh, look at how cute that Border Collie is. I heard they're herding dogs and are very intelligent. I wish my Lab was half as intelligent as those Border Collies..."
When it comes to humans, it's just "purely socioeconomic factors" causing intelligence differences between races. The alternative is simply too morbid for our "nice" liberal society to accept.
I would argue denial of reality always causes more suffering than it purportedly "prevents."
When it comes to humans, it's just "purely socioeconomic factors" causing intelligence differences between races. The alternative is simply too morbid for our "nice" liberal society to accept.
I would argue denial of reality always causes more suffering than it purportedly "prevents."
I understand that it might be difficult for nice liberal society to accept that dogs and people differ identically.
Considering that it appears as though you’ve accepted this concept as being true, can you please give me an example of an actual category of people that you feel is analogous to a category of dog? I don’t see how that would be difficult.
I understand your desire to make the point about things that may be hard for squares to swallow, but what exactly is this truth that you’re privy to? So far this point seems a bit like gesticulating without actually committing enough to make a statement of substance.
Considering that it appears as though you’ve accepted this concept as being true, can you please give me an example of an actual category of people that you feel is analogous to a category of dog? I don’t see how that would be difficult.
I understand your desire to make the point about things that may be hard for squares to swallow, but what exactly is this truth that you’re privy to? So far this point seems a bit like gesticulating without actually committing enough to make a statement of substance.
BTW your own linked source, the Wikipedia article states clearly:
"The current scientific consensus is that there is no evidence for a genetic component behind IQ differences between racial groups."
"The current scientific consensus is that there is no evidence for a genetic component behind IQ differences between racial groups."
Yes, it's the biggest lie of the 21st century. Every scientist that does not agree with the "consensus" on the genetic-IQ link is almost always fired and ostracized from academia.
Even James Watson saw the same fate.
Even James Watson saw the same fate.
[deleted]
You believe there's a conspiracy against racists and promotion of racist ideas in science?
Yeah, I don't know what the answer is. It feels like any attempt to design a solution ends up having to work around racism or, as you said in another comment, having to "take the devil's (racist's) advocate position". And sometimes the racism even takes the form of someone playing devil's advocate, in an honest attempt to make sure the solution is "realistic", but actually advancing the racists' position for them. The goal ultimately has to be to get to a place where we never have to think about what the racists will do because they don't have power any more.
"I have a dream that my four little children will one day live in a nation where they will not be judged by the color of their skin but by the content of their character."
Oh, how far we have strayed.
Oh, how far we have strayed.
You know who is at the end disadvantaged by affirmative action? Minorities / women.
Even if a PoC / woman is actually talented, people will subconsciously perceive them as diversity hires.
There were plenty of instances of people telling on this forum that they don't want to be seen as diversity person, but actual engineer.
Even if a PoC / woman is actually talented, people will subconsciously perceive them as diversity hires.
There were plenty of instances of people telling on this forum that they don't want to be seen as diversity person, but actual engineer.
I think we've officially reached the point where you can stomp lumping all women together. Some companies are now prioritizing interns and junior devs from HBCUs, because they ended up with a new problem where the overwhelming majority of non-male hires for those roles were Asian.
[deleted]
Legally, no. From a practical standpoint, people do whatever they think they can get away with.
We have to resort to legal solutions because the sad truth is that you can't at all trust entire demographics to alter or even reflect on their ethical stance.
I mean, there are enough people on this very board whose response to Black diversity initiatives in particular, is that they're too low IQ on average and therefore will be disruptive to team productivity and cohesion, despite people like Naseem Taleb going after IQ measurements.
I mean, there are enough people on this very board whose response to Black diversity initiatives in particular, is that they're too low IQ on average and therefore will be disruptive to team productivity and cohesion, despite people like Naseem Taleb going after IQ measurements.
Can you find any examples of HN comments that have suggested Black people are too low IQ on average and will be disruptive? I really doubt such a thing exists.
Reasonably, no. It's not a reasonable conclusion to make. Most people are minorities.
In America? Are we cutting up the European-American demographic by gender and sexuality, cause that'd be the most immediate way for me to grasp this idea?
Gender, sexuality, neurodivergence, “race” (however you define it), “caste” (however you define it), disabilities (however you define them, tbh, though it generally makes more sense than “race” or “caste”)… Either I just have a really skewed social circle, or most people are minorities in at least one category.
I don’t think this is an example of “can’t viably be fired”. They don’t seem to be saying Google or its employees are banned from the conference; they just don’t want Google’s money. (And Gebru used to be on the executive committee for this conference, so it can’t be a surprise to Google that they might be on her side.)
I wish you were right, but IMHO you need to take the devil's (racist's) advocate position here. Even an average not-really-racist but-slightly-prejudiced ("non-woke") HR manager can look at this and go "oh, Google got f_cked so hard for firing these two minority members... I really don't want to possibly end up in that situation"... and, even if that's certainly discriminatory thinking, it's not "wrong" on the pure logical level. And even worse, it's unlikely to be noticed and very hard to fight. The decision not to hire someone won't come with an explainer why, and proving discriminatory practice there is quite hard.
... and the HR manager may even do this subconsciously! In that case noone is even aware discrimination is happening. (Which is why blanking applications is so important!)
... and the HR manager may even do this subconsciously! In that case noone is even aware discrimination is happening. (Which is why blanking applications is so important!)
I have no doubt that there are bad consequences to the attitude that anything which happens to anyone is about their race. I just don't think that this particular happening is an example.
I'm not following your second perspective.
If I read you comment correctly, your second perspective would be "Gebru and Mitchell were fired with good cause, but people are rushing to their defense only because they are members of a minority".
If that's what you meant, I would counterargue: I haven't seen anything in this saga that implies anything of the sort. Simplified, it is my understanding that people either defend the firing claiming that Gebru and Mitchell acted in bad faith or object to it claiming that their only fault was speaking the truth back to Google (who hired them to do so).
If I read you comment correctly, your second perspective would be "Gebru and Mitchell were fired with good cause, but people are rushing to their defense only because they are members of a minority".
If that's what you meant, I would counterargue: I haven't seen anything in this saga that implies anything of the sort. Simplified, it is my understanding that people either defend the firing claiming that Gebru and Mitchell acted in bad faith or object to it claiming that their only fault was speaking the truth back to Google (who hired them to do so).
[deleted]
It is, unfortunately, a pretty common thing in our society to have rules that are selectively enforced. How often did the NYPD stop-and-frisk a trader walking down Wall Street?
In that case, the pressure isn't so much "don't hire minorities" as "don't be more stringent about enforcing your established rules and processes on certain groups of people," which is a great pressure.
Gebru has spoken publicly about the allegation that she broke processes. See this thread: https://twitter.com/timnitgebru/status/1335017526112227329
Notably, the official description of the process includes the sentence, "There is no such thing as the perfect policy. Fortunately Googlers like to do the right thing. Please do that here—read the policy and do what makes sense."
In that case, the pressure isn't so much "don't hire minorities" as "don't be more stringent about enforcing your established rules and processes on certain groups of people," which is a great pressure.
Gebru has spoken publicly about the allegation that she broke processes. See this thread: https://twitter.com/timnitgebru/status/1335017526112227329
Notably, the official description of the process includes the sentence, "There is no such thing as the perfect policy. Fortunately Googlers like to do the right thing. Please do that here—read the policy and do what makes sense."
This title is confusing in English. It makes it sound like Google is the entity that made the decision not to sponsor, instead of the conference organizers. A better title would be "AI ethics conference organizers no longer want Google as sponsors".
I did submit it under a completely different title but unfortunately someone changed it to this abomination. I don't know what if anything I can do about it because I also thought it can now be construed as the exact opposite.
I think
"Persona non grata" to me was the best translation of "unerwünscht" in the original German headline. There might have been other options but "wanted" certainly is extremely awkward and ambiguous in comparison. But stuff like that keeps happening on HN, it has certainly been a "known bug" for many years, which sometimes leads me to just not visiting or commenting/submitting for a couple of weeks until I've stopped fuming over the unappealable, authoritative actions by someone who doesn't know their limits... ;-)
I think
Google declared "persona non grata" as sponsor of AI ethics conference by ACM
"Persona non grata" to me was the best translation of "unerwünscht" in the original German headline. There might have been other options but "wanted" certainly is extremely awkward and ambiguous in comparison. But stuff like that keeps happening on HN, it has certainly been a "known bug" for many years, which sometimes leads me to just not visiting or commenting/submitting for a couple of weeks until I've stopped fuming over the unappealable, authoritative actions by someone who doesn't know their limits... ;-)
I think this pretty much tells me everything I suspected about HN.
HN, in the general case, values civility over decency when the two conflict.
I regard it as a fundamental flaw; your mileage may vary.
I regard it as a fundamental flaw; your mileage may vary.
I think adding "is" - "Google is no longer wanted..." would be enough
It's confusing because passive voice is used.
No, it's confusing, because the sentence (which isn't actually in the passive voice) is word-by-word identical to another sentence, but with with a totally different meaning. So that many people will parse it as the other sentence.
(The German sentence doesn't have that ambiguity)
(The German sentence doesn't have that ambiguity)
I believe whoever changed the headline to this abomination doesn't understand any German - or it was done automatically by something like GPT-3 (wouldn't put it past HN ;-)).
Why? Because it's the exact result that Google Translator spits out if you translate the original page in Chrome ("Google no longer wanted to sponsor the AI ethics conference"). That's of course just plain incorrect on several levels. I put a lot of effort into finding a suitable and exact translation when I originally submitted it but apparently we are now at the point where well-thought-through and correct work can simply be erased and overwritten with the result of a stupid Google AI. How fitting for this topic ;-))
Why? Because it's the exact result that Google Translator spits out if you translate the original page in Chrome ("Google no longer wanted to sponsor the AI ethics conference"). That's of course just plain incorrect on several levels. I put a lot of effort into finding a suitable and exact translation when I originally submitted it but apparently we are now at the point where well-thought-through and correct work can simply be erased and overwritten with the result of a stupid Google AI. How fitting for this topic ;-))
It's not really "incorrect", it's just ambiguous to the point that an average native English speaker would interpret it incorrectly.
The problem is the word "wanted" in that sentence can either signify present tense passive voice (e.g. "Google (is) no longer wanted to sponsor..."), which is actually the desired interpretation here, or past tense active voice (which is how I think most native speakers would interpret it).
The problem is the word "wanted" in that sentence can either signify present tense passive voice (e.g. "Google (is) no longer wanted to sponsor..."), which is actually the desired interpretation here, or past tense active voice (which is how I think most native speakers would interpret it).
In particular, the passive interpretation (omitting "is") is only valid in headlinese, while the active interpretation is valid in ordinary English.
https://en.wikipedia.org/wiki/Headline#Headlinese
Many English speakers understand headlinese easily, but maybe not so easily when there's a more natural non-headlinese parse of the same phrase!
https://en.wikipedia.org/wiki/Headline#Headlinese
Many English speakers understand headlinese easily, but maybe not so easily when there's a more natural non-headlinese parse of the same phrase!
I'm not a native english speaker but I understood it as; "The conference is no longer accepting the sponsorship by Google" or "The AI ethics research conference (an entity) no longer wants Google (another entity) as their sponsor and suspends them from sponsoring."
How would it be interpreted as being the other way around?
Edit: This is the headline when I read it: "AI ethics research conference suspends Google sponsorship"
I realized that it might have been changed now.
How would it be interpreted as being the other way around?
Edit: This is the headline when I read it: "AI ethics research conference suspends Google sponsorship"
I realized that it might have been changed now.
Google translation of the original German text:
Google is not allowed to sponsor the Conference of Fairness, Accountability, and Transparency (FAccT) this year. This has been announced by the Association for Computing Machinery (ACM). The reason for this is the dismissal of two AI researchers and allegations of racism. The business relationship is considered paused, but not ended. The British company DeepMind, which belongs to the Google group, is still allowed to participate - the break does not affect relationships with other big tech companies.
Michael Ekstrand, co-chair of the conference sponsors, justifies the decision with the layoffs of Timnit Gebru and Margaret Mitchell, former leaders of the ethics and artificial intelligence (AI) team at Google. Layoffs reason for a break
Google had made headlines with the dismissal of the two employees in the past few months. The trigger was the planned publication of a paper. Together with colleagues, Gebru criticized the dangers posed by large AI language models. As a result, Google spoke out against publication of the paper as long as Gebru is mentioned as a co-author. Officially, the paper "did not meet the requirements for a publication", but there was suspicion that the company was just trying to get rid of an unpleasant critic. This approach caused unrest among Google employees, and accusations of racism were also loud.
Margaret Mitchel, founder and co-head of the ethical AI team, was fired a short time later for attempting to use automated scripts to find evidence of the discriminatory treatment of her colleague in emails. Google's mail system did not escape this and locked the AI researcher out of the system. Recently, the company announced some internal changes in how it handles its AI teams and their employees, as well as the research results. Suresh Venkatasubramanian, member of the FAccT program committee, announced on Twitter last Friday that he would like to re-examine the framework conditions for sponsorship in the coming year.
Google is not allowed to sponsor the Conference of Fairness, Accountability, and Transparency (FAccT) this year. This has been announced by the Association for Computing Machinery (ACM). The reason for this is the dismissal of two AI researchers and allegations of racism. The business relationship is considered paused, but not ended. The British company DeepMind, which belongs to the Google group, is still allowed to participate - the break does not affect relationships with other big tech companies.
Michael Ekstrand, co-chair of the conference sponsors, justifies the decision with the layoffs of Timnit Gebru and Margaret Mitchell, former leaders of the ethics and artificial intelligence (AI) team at Google. Layoffs reason for a break
Google had made headlines with the dismissal of the two employees in the past few months. The trigger was the planned publication of a paper. Together with colleagues, Gebru criticized the dangers posed by large AI language models. As a result, Google spoke out against publication of the paper as long as Gebru is mentioned as a co-author. Officially, the paper "did not meet the requirements for a publication", but there was suspicion that the company was just trying to get rid of an unpleasant critic. This approach caused unrest among Google employees, and accusations of racism were also loud.
Margaret Mitchel, founder and co-head of the ethical AI team, was fired a short time later for attempting to use automated scripts to find evidence of the discriminatory treatment of her colleague in emails. Google's mail system did not escape this and locked the AI researcher out of the system. Recently, the company announced some internal changes in how it handles its AI teams and their employees, as well as the research results. Suresh Venkatasubramanian, member of the FAccT program committee, announced on Twitter last Friday that he would like to re-examine the framework conditions for sponsorship in the coming year.
Gebru was fired for throwing tantrums and ultimatums to doxx her colleagues or she would resign, then blasting group email literally calling for sabotage.
her colleagues were 'walking on eggshells' around her for the fear of retaliation for any minor criticism https://www.reddit.com/r/MachineLearning/comments/k77sxz/d_t...
Mitchell was fired for exfiltrating thousands of files and sending to external accounts. https://venturebeat.com/2021/01/20/google-targets-ai-ethics-...
her colleagues were 'walking on eggshells' around her for the fear of retaliation for any minor criticism https://www.reddit.com/r/MachineLearning/comments/k77sxz/d_t...
Mitchell was fired for exfiltrating thousands of files and sending to external accounts. https://venturebeat.com/2021/01/20/google-targets-ai-ethics-...
This will further marginalize AI ethics and can only be seen as a good thing.
NPR did yet another piece on the topic last weekend and interviewed an AI ethics expert. When, once again, I’m being told that the reason a system has a hard time identifying the features of black faces is systemic racism, and doesn’t even mention the optic and sensor issues, I think the field needs a reboot.
NPR did yet another piece on the topic last weekend and interviewed an AI ethics expert. When, once again, I’m being told that the reason a system has a hard time identifying the features of black faces is systemic racism, and doesn’t even mention the optic and sensor issues, I think the field needs a reboot.
It is precisely systemic racism.
People who do not have dark skin make a system. It turns out that the system performs poorly on people with dark skin, and possibly causes discriminatory outcomes.
Perhaps the training data didn't have enough dark-skinned people. Or perhaps the optics and sensors didn't work well.
The point is that the outcome is biased against a certain group of people, and it's (edit: partly) because that affected group of people lacks representation in the group of people developing the system (edit: , because if they were represented they would have flagged the issue and it would have all been presented and handled differently).
No, it does not imply that the individual researchers are racist as such.
People who do not have dark skin make a system. It turns out that the system performs poorly on people with dark skin, and possibly causes discriminatory outcomes.
Perhaps the training data didn't have enough dark-skinned people. Or perhaps the optics and sensors didn't work well.
The point is that the outcome is biased against a certain group of people, and it's (edit: partly) because that affected group of people lacks representation in the group of people developing the system (edit: , because if they were represented they would have flagged the issue and it would have all been presented and handled differently).
No, it does not imply that the individual researchers are racist as such.
Is there a way to prove that it is not caused by systemic racism? Having some unfalsifiable quality is pretty useless, and if it is the sort of thing that's never meant to end then that feels more like religion.
It would depend on the details of the particular situation (was there a specific news story associated with the GP?), but you'd have to show that people with the skin color in question signed off on the product.
It's one thing if you produce a new model and present it at a conference, and one of the conclusion of your paper is that "this model does great on light-skinned faces but performs poorly on dark-skinned faces, we need to make improvements in optics and low-light sensing to bring up the accuracy on dark-skinned people." That's not racist, that's doing science.
It's another if you are a for-profit image recognition company, or if you otherwise deploy that model in production without so much as checking to see if your model generalized well to dark-skinned faces, or without attempting to add some post-hoc bias correction, etc.
We are currently in a world where the latter happens quite often. It's not just a race thing; there all kinds of inadequate "AI" systems being pushed on people that don't really work as advertised, to the detriment of some group or another (sometimes to everyone). But when it becomes a race thing, it's especially distressing to a lot of people, because Europe and the USA have long and grim histories of racism that extend up to and including the present day.
It's one thing if you produce a new model and present it at a conference, and one of the conclusion of your paper is that "this model does great on light-skinned faces but performs poorly on dark-skinned faces, we need to make improvements in optics and low-light sensing to bring up the accuracy on dark-skinned people." That's not racist, that's doing science.
It's another if you are a for-profit image recognition company, or if you otherwise deploy that model in production without so much as checking to see if your model generalized well to dark-skinned faces, or without attempting to add some post-hoc bias correction, etc.
We are currently in a world where the latter happens quite often. It's not just a race thing; there all kinds of inadequate "AI" systems being pushed on people that don't really work as advertised, to the detriment of some group or another (sometimes to everyone). But when it becomes a race thing, it's especially distressing to a lot of people, because Europe and the USA have long and grim histories of racism that extend up to and including the present day.
> ... people with the skin color in question signed off on the product.
> ... if you otherwise deploy that model in production without so much as checking to see if your model generalized well to dark-skinned faces, or without attempting to add some post-hoc bias correction, etc
Neither of these would rule out systemic racism in either case, though, since, as you said, the problem is that "the outcome is biased against a certain group of people". Regardless of whether there were BIPOC people signing off or whether you had a perfect training set/etc, the only thing that matters is the outcome.
> ... if you otherwise deploy that model in production without so much as checking to see if your model generalized well to dark-skinned faces, or without attempting to add some post-hoc bias correction, etc
Neither of these would rule out systemic racism in either case, though, since, as you said, the problem is that "the outcome is biased against a certain group of people". Regardless of whether there were BIPOC people signing off or whether you had a perfect training set/etc, the only thing that matters is the outcome.
That's right. My example stated two very different outcomes. One outcome: an academic result is published. Other outcomes: something is put into production.
so you agree that attributing an outcome to systemic racism is unfalsifiable?
I see what you're saying.
Mind you, I'm not a sociologist, so anything I say here is going to be a suboptimal paraphrasing.
Systemic racism is not a cause of something, but a description of a state of existence.
Systemic racism is itself the existence of inequitable outcomes in the context of a society that was literally founded on overt racism.
That something "is" systemic racism is falsifiable by demonstrating that one or both of those criteria is not met.
So this particular outcome will always be definitionally systemic racism until the outcome is no longer inequitable, or the USA has moved meaningfully past its white supremacist origins to the point where individual cases of accidental bias are truly individual cases and not part of a pattern.
Personally, I am okay with that. Ceteris non paribus sunt.
Mind you, I'm not a sociologist, so anything I say here is going to be a suboptimal paraphrasing.
Systemic racism is not a cause of something, but a description of a state of existence.
Systemic racism is itself the existence of inequitable outcomes in the context of a society that was literally founded on overt racism.
That something "is" systemic racism is falsifiable by demonstrating that one or both of those criteria is not met.
So this particular outcome will always be definitionally systemic racism until the outcome is no longer inequitable, or the USA has moved meaningfully past its white supremacist origins to the point where individual cases of accidental bias are truly individual cases and not part of a pattern.
Personally, I am okay with that. Ceteris non paribus sunt.
Agreed, but I've found that when discussing these things logic isn't often taken into account anymore due to matters of sensitivity and bias. It's frustrating.
I understand what you're saying, but it sounds like you're aware of how prone this term is to misinterpretation. Virtually everyone outside of a particular small subculture understands "systemic racism" to be an accusation of racism. It would be like calling a correctness bug "systemic fraud" - how are the authors of the system supposed to avoid taking that as a personal attack?
[deleted]
> It would be like calling a correctness bug "systemic fraud" - how are the authors of the system supposed to avoid taking that as a personal attack?
Perhaps they aren't.
Perhaps they aren't.
Anecdata:
At work one time I created a machine learning "algorithm" that turned out to be systemically biased in a way that was both unintended and distasteful.
I'm sure I've done it other times, but this is the one time that we actually realized it.
The issue was flagged internally and the model was improved until we felt like we were no longer producing bigoted results.
I certainly did not take offense at the suggestion that my model might be racist. On the contrary, I considered it a very important and serious problem that needed to be resolved.
It's worth mentioning that part of the reason we even caught the problem is that some of my colleagues at the time would have been affected by the adverse outcome, had they been subject to the judgement of the model. If I didn't personally work with people who fell into the affected group, I might never have noticed the problem in the first place.
At work one time I created a machine learning "algorithm" that turned out to be systemically biased in a way that was both unintended and distasteful.
I'm sure I've done it other times, but this is the one time that we actually realized it.
The issue was flagged internally and the model was improved until we felt like we were no longer producing bigoted results.
I certainly did not take offense at the suggestion that my model might be racist. On the contrary, I considered it a very important and serious problem that needed to be resolved.
It's worth mentioning that part of the reason we even caught the problem is that some of my colleagues at the time would have been affected by the adverse outcome, had they been subject to the judgement of the model. If I didn't personally work with people who fell into the affected group, I might never have noticed the problem in the first place.
Invictus0(1)
[deleted]
Alternative take:
Computers can most easily pick out facial features when there is significant color contrast, shadows cast on white skin create greater contrast than it can with darker skin, and the systems end up inherently working better on people with lighter skin due to basic math and physics.
Why hasn't some company out there designed a powerful system that works well with darker skinned people? It would be trumpeted by the tech press all over if they did, instead we just see articles about companies like Google manually tweaking the outcomes to appease the people complaining about racism.
Computers can most easily pick out facial features when there is significant color contrast, shadows cast on white skin create greater contrast than it can with darker skin, and the systems end up inherently working better on people with lighter skin due to basic math and physics.
Why hasn't some company out there designed a powerful system that works well with darker skinned people? It would be trumpeted by the tech press all over if they did, instead we just see articles about companies like Google manually tweaking the outcomes to appease the people complaining about racism.
A system that works poorly on dark-skinned people would be a nonstarter in most parts of Africa and South Asia.
But in North America it's hailed as highly successful and put into production, despite some 44 million US citizens being Black or African, with demographic concentrations as high as 70-80% in some cities.
But in North America it's hailed as highly successful and put into production, despite some 44 million US citizens being Black or African, with demographic concentrations as high as 70-80% in some cities.
From looking at articles that actually give hard numbers, misidentification for minorities is something like 5 to 10 times more common, with a 1 in 1000 error rate for black women for example:
https://www.wired.com/story/best-algorithms-struggle-recogni...
Being able to identify 999 out of 1,000 black women accurately is highly successful. Just because it gets 9,999 out of 10,000 on white women doesn't make 999 out of 1,000 a failure.
Mind you I'm opposed to the tech in general because it will inevitably be used to create dystopian hellholes. But not because I falsely correlate different error margins for different races with "structural racism."
https://www.wired.com/story/best-algorithms-struggle-recogni...
Being able to identify 999 out of 1,000 black women accurately is highly successful. Just because it gets 9,999 out of 10,000 on white women doesn't make 999 out of 1,000 a failure.
Mind you I'm opposed to the tech in general because it will inevitably be used to create dystopian hellholes. But not because I falsely correlate different error margins for different races with "structural racism."
This will further marginalize AI ethics and
can only be seen as a good thing.
In recent years, "AI ethics" generally refers to the problem of building regression models whose mechanics and outputs follow a certain set of political sensibilities. People investigating the problem you're describing typically identify their work as "AI safety".
The reason why we don't care and still use passive optics to do face recognition when people with black skin exist is literally systematic racism.
Or maybe it's just one of the simpler and least computationally expensive ways to do it? No systemic racism needed (unless you're stretching the meaning of the word beyond all recognition)
I'm not stretching the word beyond anything past it's intended meaning. If passive optical image based image recognition was as performant for most people as it is in general for black people it wouldn't be acceptable. The fact that this trade-off is accepted is systematic racism - the systematic incentives of implementing facial recognition leads to discrimination.
Do you know what the classification error is for different ethnicities?
Or are you just basing this on a small handful of misclassifications that you've seen in the news?
Or are you just basing this on a small handful of misclassifications that you've seen in the news?
We're talking specifically about passive vision based facial recognition, right?
I've personally seen such systems that simply do not have the dynamic range to be able to, in the same exposure, extract a sufficient amount of detail from black skinned faces and white skinned faces.
I'm sure there are systems that can, at a much higher cost. But largely there are many that simply fail.
I've personally seen such systems that simply do not have the dynamic range to be able to, in the same exposure, extract a sufficient amount of detail from black skinned faces and white skinned faces.
I'm sure there are systems that can, at a much higher cost. But largely there are many that simply fail.
Google's push to participate in AI ethics discussions has always been about controlling the narrative. The fact that it took them firing two people for ACM to notice is rather hilarious. Just read the papers they promote. The meta-narrative is rather clear. They want to present AI/ML as something that is dangerous in the hands of individuals or smaller companies and that requires constant and direct control from experts with tons of data and resources. So they're perfectly happy to talk about stuff like model bias, but the moment you start asking uncomfortable questions about ML in general you better watch out.
These topics should be handled by people without corporate ties. Period. The conflict of interests is so in-your-face, it's mind-boggling more people don't speak about it.
These topics should be handled by people without corporate ties. Period. The conflict of interests is so in-your-face, it's mind-boggling more people don't speak about it.
Considering that "AI ethics" basically means we shouldn't have AI until we can guarantee that AI doesn't commit thought crimes(according to the current groupthink morality prevalent among the educated class), it's not surprising.
that's disingenuous at best. AI systems are already deployed in the judicial system, surveillance systems, healthcare, and who knows how many other domains that effect people's lives - with many instances to suggest that a large % of systems treat people differently along undesirable features like race or gender due to encoding negative biases or whatever the cause may be
I've changed the URL from https://www.heise.de/news/Google-als-Sponsor-fuer-KI-Ethik-K... to which seems to be the main 3rd party article on this in English.
Submissions to HN need to be in English. We have deep respect for German and other languages, but HN is an English-language site. It's important that the community be able to read an article. When people can't read an article they react purely to the title, which leads to shallower discussion.
https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor...
I've also changed the title from "Google no longer wanted to sponsor the AI ethics conference", which unless my rudimentary German is betraying me, actually is not what the heise.de headline says. If I understand the story correctly, it's the conference that didn't want Google.
Submissions to HN need to be in English. We have deep respect for German and other languages, but HN is an English-language site. It's important that the community be able to read an article. When people can't read an article they react purely to the title, which leads to shallower discussion.
https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor...
I've also changed the title from "Google no longer wanted to sponsor the AI ethics conference", which unless my rudimentary German is betraying me, actually is not what the heise.de headline says. If I understand the story correctly, it's the conference that didn't want Google.
It's ironic how one of the biggest ban-hammers on earth got banned
The title is misleading. It's confusing because the German original title make clear that ACM does not want Google to be a sponsor anymore, whereas the English title adds the possibility that it might be Google who wanted to sponsor the conference, but then decided not to.
I wish ACM would act ethically in regards to SciHub.
>For the avoidance of doubt, an example of a site ACM authors may post all versions of their work to, with the exception of the final published "Version of Record", is ArXiv.
>Examples of sites ACM authors may not post their work to are ResearchGate, Academia.edu, Mendeley, or Sci-Hub, as these sites are all either commercial or in some instances utilize predatory practices that violate copyright
https://www.acm.org/publications/openaccess
Not exactly an onerous policy, nor one that makes content hard to obtain.
>Examples of sites ACM authors may not post their work to are ResearchGate, Academia.edu, Mendeley, or Sci-Hub, as these sites are all either commercial or in some instances utilize predatory practices that violate copyright
https://www.acm.org/publications/openaccess
Not exactly an onerous policy, nor one that makes content hard to obtain.
Ah, the sweet irony of accusing a free distributor of academic papers of "predatory practices", as opposed to the bloodsucking leeches at Elsevier & co.
They take sponsorship from for profit journals and that’s way worse for their field than anything Google does.
I find them hypocritical for cancelling this one sponsorship for “ethical” reasons but not addressing the actual problem that is the relationship of their professional society and for profit publishing.
I find them hypocritical for cancelling this one sponsorship for “ethical” reasons but not addressing the actual problem that is the relationship of their professional society and for profit publishing.
What's the value of an AI ethics conference if the largest AI powerhouse is not allowed to take part?
So I have to accept all of Heise's cookies or I have to pay?
Wow. Heise had a better reputation than that in my mind.
Wow. Heise had a better reputation than that in my mind.
It's even worse. The new cookie banner reads "USA Consent: By clicking on "Accept", you consent to your data being processed by providers in the USA in accordance with Art. 49 (1) DSGVO. The USA is considered to be a country with an insufficient level of data protection according to EU standards. There is a risk that your data may also be passed on to US authorities."
Good. Imagining a world where Lockheed no longer wants to sponsor the cluster-munitions ethics conference and that's viewed as a negative.
Now that sounds like a fun conference to attend. I'd be curious about the positive impacts of cluster munitions, altho I'd be hard pressed to find any. I would imagine an ethics conference on this topic would have one session: "do not use".
"build bigger cluster munitions to deter our enemies"?
This is an ambiguous, and therefore bad, title.
In other news, Google suspends indexing of the said AI ethics research conference, so that no one knows about it.
[deleted]
Bold move from ACM but good for them!
Big G sponsoring an AI ethics conference would be like Bonnie Parker sponsoring an ethical banking committee ...
(Apparently somebody -did- remember who Parker was :)
I suffer silently the wrath of the big G!
Shall I compare them to the Bonnie and Clyde
due their ethical struggling for AI?
Big G can do no evil, plain as that
their critics only will avail squat.-
(... or down votes, apparently :)
While Parkers in the end got their due trap
Our every browse does big G still now track
Who am I meager I to claim that G
cannot do evil, but lone evil be?
As father time allowed for a short while
that Bonnie and Clyde their banking tryst did hide
now does the G control our searches all
But not with evil! No! how dare we call!
An ethical conundrum fore us lies
Is big G fit to master our AIs?
There's only one big G, I need not tell.
The rest of them can burn wholesale in ...
Shall I compare them to the Bonnie and Clyde
due their ethical struggling for AI?
Big G can do no evil, plain as that
their critics only will avail squat.-
(... or down votes, apparently :)
While Parkers in the end got their due trap
Our every browse does big G still now track
Who am I meager I to claim that G
cannot do evil, but lone evil be?
As father time allowed for a short while
that Bonnie and Clyde their banking tryst did hide
now does the G control our searches all
But not with evil! No! how dare we call!
An ethical conundrum fore us lies
Is big G fit to master our AIs?
There's only one big G, I need not tell.
The rest of them can burn wholesale in ...
[deleted]
Quis custodiet ipsos custodes?
one for /r/nottheonion
Since google doesn't need ethicists but vice versa I think they will be fine after this crushing blow.
Google had it coming when they supported autonomous warfare AI
I've worked for companies who hired and then had to fire hackers for publishing vulnerabilities in their own and in customer products. Security is fundamentally a technology governance function, and AI ethics piece is also a technology governance function, where both hackers and AI ethics researchers are in-effect activists to create awareness for change, but aren't typically who you would put in the actual governance role.
An AI ethics conference has analogies to Defcon or Blackhat, in that shunning vendors because the attendees perceived that some prominent hackers were treated unfairly may have some precedents. Microsoft's relationship with Defcon in the 90's vs. post 2000 is an example of how this both happened and changed.
Extending that analogy, the disconnect in the relationship between researchers and vendors was a symptom of what a disadvantage the vendors were at to the asymmetric risk that activists/hackers posed because the media story of the conflict between giant corporations being vulnerable to scrappy hackers writes itself. In terms of how to handle it, google can probably use precedents from MSFT vs. defcon, and AI ethics researchers can look at how hackers both succeeded and failed to change security and tech governance.