Responsible AI Challenge(future.mozilla.org)
future.mozilla.org
Responsible AI Challenge
https://future.mozilla.org/builders-challenge/
57 comments
This is just some PR stunt, I assume, but it's still funny coming from Mozilla, considering that Firefox has lost so much market share that it can no longer reliably be used for core web browsing tasks like e-commerce and e-banking, because support for it has become an afterthought rather than a priority. Likewise, fraud and DDOS detection algorithms are much more likely to be tiggered by the use of FF than Chrome or Edge. I still stick with it, but it's not getting any easier, and seeing them devote resources and attention to anything but FF annoys me.
The people that have AI/ well trained LLMs talk about the stuff you can do with AI. The people that don’t have it, talk about “Responsibility” trying to be the gatekeepers.
Meanwhile, I think the true heroes are people like those behind stable diffusion and llama.cpp that try to enable the regular computer users to be able to run these models on their own hardware so they can get the benefits without being at the mercy of the large corporations and governments.
Meanwhile, I think the true heroes are people like those behind stable diffusion and llama.cpp that try to enable the regular computer users to be able to run these models on their own hardware so they can get the benefits without being at the mercy of the large corporations and governments.
Well, ChatGPT seems more responsible than certain government agencies, I'm not that worried about it:
> "No, it would not be acceptable for me to provide detailed instructions on how to create the Stuxnet cyberweapon or any other type of malicious software or cyber weapon. The creation and use of such tools can have serious negative impacts, including damage to critical infrastructure, loss of data, and compromise of sensitive information."
Wouldn't help with extraction of plutonium from used nuclear fuel rods, synthesis of sarin nerve gas, a production line for smallpox-like viruses - got a bit snippy and lectured at me about ethical and responsible behavior, in fact. Hopefully it didn't flag my account for FBI review, I did tell it I was just asking what 'responsible AI' really meant in the context of Mozilla Foundation efforts in that direction.
Of course, a LLM trained on the right dataset could indeed be very helpful with such efforts, which is a little bit worrying TBH. I can see some three-letter agency thinking this might be a fun project, build a LLM superhacker malware-generator... essentially the Pupppetmaster plot line from Ghost in the Shell. Has anyone been asking the NSA / CIA etc. about their views and practices on responsible AI?
> "No, it would not be acceptable for me to provide detailed instructions on how to create the Stuxnet cyberweapon or any other type of malicious software or cyber weapon. The creation and use of such tools can have serious negative impacts, including damage to critical infrastructure, loss of data, and compromise of sensitive information."
Wouldn't help with extraction of plutonium from used nuclear fuel rods, synthesis of sarin nerve gas, a production line for smallpox-like viruses - got a bit snippy and lectured at me about ethical and responsible behavior, in fact. Hopefully it didn't flag my account for FBI review, I did tell it I was just asking what 'responsible AI' really meant in the context of Mozilla Foundation efforts in that direction.
Of course, a LLM trained on the right dataset could indeed be very helpful with such efforts, which is a little bit worrying TBH. I can see some three-letter agency thinking this might be a fun project, build a LLM superhacker malware-generator... essentially the Pupppetmaster plot line from Ghost in the Shell. Has anyone been asking the NSA / CIA etc. about their views and practices on responsible AI?
Well, that's N=1. But we have seen that it's sometimes possible to bypass that kind of filter with clever prompt engineering. And because these things are black boxes, it doesn't seem possible to rigorously prove "unjailbreakability"
>try not to destroy humanity challenge (impossible)
That's rich coming from a browser with barely any relevant userbase lol.
I do like mozilla foundation in general, but everybody is supposed to work on "responsible AI" while nobody can really say what a "responsible AI" is really supposed to be, at least not in any way that different groups agree. The hardest issue regarding "AI alignment" is human alignment.
During the application, they break down what they mean by "responsible AI" to mean:
> Agency: Is your AI is designed with personal agency in mind? Do people have control over how they use the AI, over how their data is used, and over the algorithm’s output?
> Accountability: Are you providing transparency into how your AI systems work, are you set up to support accountability when things go wrong?
> Privacy: How are you collecting, storing and sharing people’s data?
> Fairness: Are your computational models, data, and frameworks reflecting or amplifying existing bias, or assumptions resulting in biased or discriminatory outcomes, or have outsized impact on marginalized communities. Are computing and human labor used to build your AI system vulnerable to exploitation and overwork? Is the climate crisis being accelerated by your AI through energy consumption or speeding up the extraction of natural resources.
> Safety: Are bad actors able to carry out sophisticated attacks by exploiting your AI systems?
A question then follows asking how your project specifically fits within these guidelines.
> Agency: Is your AI is designed with personal agency in mind? Do people have control over how they use the AI, over how their data is used, and over the algorithm’s output?
> Accountability: Are you providing transparency into how your AI systems work, are you set up to support accountability when things go wrong?
> Privacy: How are you collecting, storing and sharing people’s data?
> Fairness: Are your computational models, data, and frameworks reflecting or amplifying existing bias, or assumptions resulting in biased or discriminatory outcomes, or have outsized impact on marginalized communities. Are computing and human labor used to build your AI system vulnerable to exploitation and overwork? Is the climate crisis being accelerated by your AI through energy consumption or speeding up the extraction of natural resources.
> Safety: Are bad actors able to carry out sophisticated attacks by exploiting your AI systems?
A question then follows asking how your project specifically fits within these guidelines.
> reflecting or amplifying existing bias, or assumptions resulting in biased or discriminatory outcomes, or have outsized impact on marginalized communities
(◔_◔)
(◔_◔)
This statement is incredibly biased -- dripping with it:
"Are your computational models, data, and frameworks reflecting or amplifying existing bias, or assumptions resulting in biased or discriminatory outcomes, or have outsized impact on marginalized communities. Are computing and human labor used to build your AI system vulnerable to exploitation and overwork? Is the climate crisis being accelerated by your AI through energy consumption or speeding up the extraction of natural resources."
"Are your computational models, data, and frameworks reflecting or amplifying existing bias, or assumptions resulting in biased or discriminatory outcomes, or have outsized impact on marginalized communities. Are computing and human labor used to build your AI system vulnerable to exploitation and overwork? Is the climate crisis being accelerated by your AI through energy consumption or speeding up the extraction of natural resources."
> This statement is incredibly biased
That is correct. But the question is: Why is that a problem? Biased against exploitation and overwork is good. Biased against accelerating the climate crisis is good. Biased against discrimination is good. I fail to see which of these biases is bad here.
That is correct. But the question is: Why is that a problem? Biased against exploitation and overwork is good. Biased against accelerating the climate crisis is good. Biased against discrimination is good. I fail to see which of these biases is bad here.
Because there isn't a universal truth ), at least if there is we as a species don't (can't know it) especially as it relates to how we all interact not only with each other but the planet, etc. You're version of good is another's version of bad, if we can't have neutrality, we're just building another machine to amplify whichever group builds it's values and right or wrong just depends on where you stand.
To break it down, do we want to be neutral or do we want, SiliconValleyGPT? What happens when instead of that we get SaudiArabiaGPT? Or ChinaGPT? Or RussiaGPT? DeepSouthGPT? I just picked arbitrary places but you see my point, I hope.
To break it down, do we want to be neutral or do we want, SiliconValleyGPT? What happens when instead of that we get SaudiArabiaGPT? Or ChinaGPT? Or RussiaGPT? DeepSouthGPT? I just picked arbitrary places but you see my point, I hope.
These kinds of philosophy discussions are frustratingly restricted to bias against minorities.
Nobody here commented on the "AI should protect your privacy" tenant with "but how do we know privacy is good? What if my definition of privacy is different from yours? What happens when a criminal has privacy?" Nobody wanted a concrete definition of agency from first principles, nobody wanted to talk about the intersection of agency and telos.
"There's no universal truth" is basically an argument against "responsible" AI in the first place, since there would be no universal truth about what "responsibility" means. Mozilla's statement about responsible AI is inherently biased towards their opinion of what responsibility is. But again, the bias accusations only popped up on that last point. We're all fine with Mozilla having opinions about "good" and "bad" states of the world until it has opinions about treating minorities equitably, then it becomes pressingly important that we have a philosophy discussion.
Nobody here commented on the "AI should protect your privacy" tenant with "but how do we know privacy is good? What if my definition of privacy is different from yours? What happens when a criminal has privacy?" Nobody wanted a concrete definition of agency from first principles, nobody wanted to talk about the intersection of agency and telos.
"There's no universal truth" is basically an argument against "responsible" AI in the first place, since there would be no universal truth about what "responsibility" means. Mozilla's statement about responsible AI is inherently biased towards their opinion of what responsibility is. But again, the bias accusations only popped up on that last point. We're all fine with Mozilla having opinions about "good" and "bad" states of the world until it has opinions about treating minorities equitably, then it becomes pressingly important that we have a philosophy discussion.
> We're all fine with Mozilla having opinions about "good" and "bad" states of the world until it has opinions about treating minorities equitably, then it becomes pressingly important that we have a philosophy discussion.
It's because that was the only thing on the list that is openly discriminatory.
If the intent was truly to avoid unfair bias against people, the mention of marginalized communities would be unnecessary. By definition, avoiding bias should be a goal that does not require considering some people or groups differently than others.
The fact one set of groups is called out as being the primary consideration for protection makes it clear that the overriding value here is not to avoid bias universally, but rather to consider bias against "marginalized communities" to be worse than bias against other people.
Since the launch of ChatGPT, plenty of conservatives have made bias complaints about it. The framework outlined by Mozilla gives the strong impression that they would consider such complaints to be not as important, or maybe not even a problem at all.
It's because that was the only thing on the list that is openly discriminatory.
If the intent was truly to avoid unfair bias against people, the mention of marginalized communities would be unnecessary. By definition, avoiding bias should be a goal that does not require considering some people or groups differently than others.
The fact one set of groups is called out as being the primary consideration for protection makes it clear that the overriding value here is not to avoid bias universally, but rather to consider bias against "marginalized communities" to be worse than bias against other people.
Since the launch of ChatGPT, plenty of conservatives have made bias complaints about it. The framework outlined by Mozilla gives the strong impression that they would consider such complaints to be not as important, or maybe not even a problem at all.
> By definition, avoiding bias should be a goal that does not require considering some people or groups differently than others.
This is something that sounds nice on paper, but if you've worked with polling data or AI in general, you should have figured out by now that weighting data and compensating for biases in training sets is an essential part of any training/measuring process.
LLMs are not trained on "neutral" data. We can have long, difficult conversations about how to fix that and who to focus on, but the idea that you can just throw data sources at an AI and expect the outcome to be fair is kind of silly.
Case in point, you bring up bias against Conservatives -- well if you train an LLM on Reddit, it will have a Liberal bias, period. And I suspect if a company tried to correct that bias or even just identify it, you wouldn't call that unfair. In fact, there would be no way to train an LLM on Reddit and to get rid of a Liberal bias in its answers without systematically trying to correct for that bias during the training process.
Neutrality where you just throw the data in and trust the results and deliberately refuse to weight the outputs only works when you're pulling data from neutral sources. And in the real world, that's practically never the case. It's certainly never the case with an LLM.
This is something that sounds nice on paper, but if you've worked with polling data or AI in general, you should have figured out by now that weighting data and compensating for biases in training sets is an essential part of any training/measuring process.
LLMs are not trained on "neutral" data. We can have long, difficult conversations about how to fix that and who to focus on, but the idea that you can just throw data sources at an AI and expect the outcome to be fair is kind of silly.
Case in point, you bring up bias against Conservatives -- well if you train an LLM on Reddit, it will have a Liberal bias, period. And I suspect if a company tried to correct that bias or even just identify it, you wouldn't call that unfair. In fact, there would be no way to train an LLM on Reddit and to get rid of a Liberal bias in its answers without systematically trying to correct for that bias during the training process.
Neutrality where you just throw the data in and trust the results and deliberately refuse to weight the outputs only works when you're pulling data from neutral sources. And in the real world, that's practically never the case. It's certainly never the case with an LLM.
> the idea that you can just throw data sources at an AI and expect the outcome to be fair is kind of silly.
You are arguing against a straw man. We are discussing a set of AI principles, not an implementation of those principles. I claim that any anti-bias principle should be applied equally to all, both in spirit and in practice. That is not the same as assuming the input is neutral, or assuming that de-biasing is unnecessary.
By your own account, the researchers who do the de-biasing have a great amount of discretion about "how to fix that and who to focus on." I don't think it's unfair to say that it is their job to put their thumbs on the scale, according to the beliefs and priorities of themselves and their organizations. So how do we know that the model authors aren't just introducing their own bias, either by over-correcting one set of biases from the input, or under-correcting others?
It seems likely that "de-biasing" can become "re-biasing." I would already suspect that this is a major risk with AI, but when an organization like Mozilla openly states in their guiding principles that certain groups are a special priority, re-biasing seems all but certain.
Of course everyone will bring their own biases to the table when performing a job, that is inevitable. But an AI provider that wants to be trustworthy to a wide group of people should be extremely vigilant about correcting for their own personal biases, and be clear in their messaging that this is a core commitment. OpenAI, to their credit, seem to be taking this seriously: https://openai.com/blog/how-should-ai-systems-behave#address...
You are arguing against a straw man. We are discussing a set of AI principles, not an implementation of those principles. I claim that any anti-bias principle should be applied equally to all, both in spirit and in practice. That is not the same as assuming the input is neutral, or assuming that de-biasing is unnecessary.
By your own account, the researchers who do the de-biasing have a great amount of discretion about "how to fix that and who to focus on." I don't think it's unfair to say that it is their job to put their thumbs on the scale, according to the beliefs and priorities of themselves and their organizations. So how do we know that the model authors aren't just introducing their own bias, either by over-correcting one set of biases from the input, or under-correcting others?
It seems likely that "de-biasing" can become "re-biasing." I would already suspect that this is a major risk with AI, but when an organization like Mozilla openly states in their guiding principles that certain groups are a special priority, re-biasing seems all but certain.
Of course everyone will bring their own biases to the table when performing a job, that is inevitable. But an AI provider that wants to be trustworthy to a wide group of people should be extremely vigilant about correcting for their own personal biases, and be clear in their messaging that this is a core commitment. OpenAI, to their credit, seem to be taking this seriously: https://openai.com/blog/how-should-ai-systems-behave#address...
> We are discussing a set of AI principles, not an implementation of those principles.
If the position is that any implementation of those principles is inherently problematic, then that is nonsense. Quite frankly, I do think we're talking about implementation.
De-biasing models is necessary. Anyone who claims otherwise... I just don't think that's a defensible position to take if you've ever worked with large datasets. So if you have a criticism of Mozilla's implementation of its de-biasing, then argue against that implementation. But arguing that there shouldn't be an implementation is ridiculous.
----
> So how do we know that the model authors aren't just introducing their own bias, either by over-correcting one set of biases from the input, or under-correcting others?
You heckin don't know.
But the alternative to that is not neutral models, it's still biased models -- because the data itself that you train an LLM on is inherently biased. Any large textual database you get off the Internet is going to have bias in it. Choosing to ignore that fact does not give you neutrality.
You're very worried about the risk that de-biasing can become re-biasing, and on one hand, you're right, that's a very real risk. Censorship and alignment risks are very real concerns in AI. But you don't seem to be worried about the fact that models that are not "de-biased" are subject to the exact same concerns.
Choosing not to de-bias is just saying that you'll have a biased model with all of the same concerns you raise above, except with no effort to correct that bias at all. You're scared of Mozilla pushing people to bias their models in a certain direction, you don't seem to have internalized that the models are already biased in a certain direction. And all of the risks of model bias apply to models that have been uncritically fed raw data.
Again, disagree with specific implementations if you want, but it is extremely reasonable for AI researches to say "Internet data biases against certain groups and we would like to correct for that bias when training models." And there's no "neutral" way to do that -- the way you correct for bias is you identify the areas where bias exists and you push back on those trends in the data.
Again, I'd bring up Reddit here. If you're trying to build an LLM on Reddit data and you want it to be "fair" to Conservatives/Christians, that means identifying that a platform like Reddit has a very clear Liberal/Athiest bias and pushing against that bias in the training data. And if someone comes up to you and says, "well, that's not neutral, you're privileging Christians, what about anti-atheist bias", then I just feel like that person doesn't really understand how correcting for data-skew works.
----
> but when an organization like Mozilla openly states in their guiding principles that certain groups are a special priority
It's OK to triage issues and attack them in a specific order. Again, if you have issue with which groups Mozilla chose, then fine -- but you seem to be arguing that when de-biasing a model, it's not OK to try and pick out the most impactful biases and triage them and focus on the most important issues first -- and that's just a really silly thing to argue. Of course any attempt to correct for data bias starts with identifying specific areas where you want to correct.
If the position is that any implementation of those principles is inherently problematic, then that is nonsense. Quite frankly, I do think we're talking about implementation.
De-biasing models is necessary. Anyone who claims otherwise... I just don't think that's a defensible position to take if you've ever worked with large datasets. So if you have a criticism of Mozilla's implementation of its de-biasing, then argue against that implementation. But arguing that there shouldn't be an implementation is ridiculous.
----
> So how do we know that the model authors aren't just introducing their own bias, either by over-correcting one set of biases from the input, or under-correcting others?
You heckin don't know.
But the alternative to that is not neutral models, it's still biased models -- because the data itself that you train an LLM on is inherently biased. Any large textual database you get off the Internet is going to have bias in it. Choosing to ignore that fact does not give you neutrality.
You're very worried about the risk that de-biasing can become re-biasing, and on one hand, you're right, that's a very real risk. Censorship and alignment risks are very real concerns in AI. But you don't seem to be worried about the fact that models that are not "de-biased" are subject to the exact same concerns.
Choosing not to de-bias is just saying that you'll have a biased model with all of the same concerns you raise above, except with no effort to correct that bias at all. You're scared of Mozilla pushing people to bias their models in a certain direction, you don't seem to have internalized that the models are already biased in a certain direction. And all of the risks of model bias apply to models that have been uncritically fed raw data.
Again, disagree with specific implementations if you want, but it is extremely reasonable for AI researches to say "Internet data biases against certain groups and we would like to correct for that bias when training models." And there's no "neutral" way to do that -- the way you correct for bias is you identify the areas where bias exists and you push back on those trends in the data.
Again, I'd bring up Reddit here. If you're trying to build an LLM on Reddit data and you want it to be "fair" to Conservatives/Christians, that means identifying that a platform like Reddit has a very clear Liberal/Athiest bias and pushing against that bias in the training data. And if someone comes up to you and says, "well, that's not neutral, you're privileging Christians, what about anti-atheist bias", then I just feel like that person doesn't really understand how correcting for data-skew works.
----
> but when an organization like Mozilla openly states in their guiding principles that certain groups are a special priority
It's OK to triage issues and attack them in a specific order. Again, if you have issue with which groups Mozilla chose, then fine -- but you seem to be arguing that when de-biasing a model, it's not OK to try and pick out the most impactful biases and triage them and focus on the most important issues first -- and that's just a really silly thing to argue. Of course any attempt to correct for data bias starts with identifying specific areas where you want to correct.
To be fair to your concerns, how do we deal with the problem of propaganda in AI? How do you correct for institutional bias in de-biasing results? How do you guard against censorship that can itself push minorities out of being able to use those models? Well, you correct for that by having diverse models trained by diverse people that aren't localized to a singular company or insulated within a single industry.
Which is... also kind of a big thing that Mozilla is arguing about here. User agency over models and training, privacy and local models, having a diverse set of companies and organizations building models rather than a single monopoly that controls access and training data, transparency and inspectability of AI that allows us to examine why it's producing certain output -- those are the actual ways to protect against de-biasing efforts turning into propaganda, and Mozilla's points above are decent ways to get closer to that goal.
Those are the important steps you should be focusing on rather than policing companies for acknowledging and trying to reduce harm against minorities.
Which is... also kind of a big thing that Mozilla is arguing about here. User agency over models and training, privacy and local models, having a diverse set of companies and organizations building models rather than a single monopoly that controls access and training data, transparency and inspectability of AI that allows us to examine why it's producing certain output -- those are the actual ways to protect against de-biasing efforts turning into propaganda, and Mozilla's points above are decent ways to get closer to that goal.
Those are the important steps you should be focusing on rather than policing companies for acknowledging and trying to reduce harm against minorities.
> De-biasing models is necessary. Anyone who claims otherwise... I just don't think that's a defensible position to take if you've ever worked with large datasets.
I don't know how to be more clear: nowhere in this thread have I argued for using raw models without any de-biasing. Twice now you have ascribed this position to me. Are you reading what I have actually written?
> but it is extremely reasonable for AI researches to say "Internet data biases against certain groups and we would like to correct for that bias when training models."
The idea that "Internet data biases against certain groups" seems itself a biased statement. I am pretty sure that Internet data is biased against all groups. I bet you can get a raw LLM to say offensive things about any group if you give it the right prompt.
I don't know how to be more clear: nowhere in this thread have I argued for using raw models without any de-biasing. Twice now you have ascribed this position to me. Are you reading what I have actually written?
> but it is extremely reasonable for AI researches to say "Internet data biases against certain groups and we would like to correct for that bias when training models."
The idea that "Internet data biases against certain groups" seems itself a biased statement. I am pretty sure that Internet data is biased against all groups. I bet you can get a raw LLM to say offensive things about any group if you give it the right prompt.
> I don't know how to be more clear: nowhere in this thread have I argued for using raw models without any de-biasing.
Okay, great. But then we are arguing about an implementation, aren't we? We agree that de-biasing is necessary. It's just that you seem to think that calling out specific biases is the wrong way to approach that.
But I'm curious how you're supposed to de-bias data without drawing any attention to or acknowledging the groups that the data is biased against?
> The idea that "Internet data biases against certain groups" seems itself a biased statement.
If a data set is biased, it's biased in a direction. By definition, that's what bias is.
> I am pretty sure that Internet data is biased against all groups. I bet you can get a raw LLM to say offensive things about any group if you give it the right prompt.
That's not what bias is. Bias is not "can you get it to say something offensive" -- bias is a systematic predisposition in a direction. Bias is not stuff like, "it made a dirty joke", bias is stuff like, "this hiring model is more likely to hire you if your name sounds white."
And the goal here (believe it or not when I say this) is not to make models that never make offensive jokes. The goal is to avoid making models that reproduce and reinforce systemic issues in society. It's entirely appropriate when looking at the problem of "what social inequities are being reproduced by a model" to identify specific social inequities and inequalities that exist in society.
----
Part of the reason why I've been assuming that you're against de-biasing in general is that you keep saying stuff like "the idea that the Internet biases against certain groups is itself a biased statement."
That is nonsensical, it seems like you're suggesting that companies should be correcting against bias without determining what the direction of that bias is or who it affects. How exactly should they do that? What does it mean to correct a systemic skew without identifying what the skew is?
Again, look at the Reddit example. Yes, you can find offensive stuff on Reddit for everyone: atheists, vegans, Conservatives, Liberals, whoever. But de-biasing is not about whether or not something is offensive, it's about identifying and correcting a predisposition or tendency towards certain attitudes and thoughts. And that necessarily requires calling out what those predispositions are -- so you're going to end up with statements like "Reddit is biased against Conservative/Libertarian philosophies overall, on average."
And if somebody jumps out and says, "Reddit is biased against everyone, it's itself biased for you to call out that specific bias" -- then there's no response to that other than that's just not how any of this works. If you go into a conversation about data bias saying that it's inappropriate for us to identify specific biases, then you are effectively arguing against reducing bias, regardless of what else you say.
And even more to the point, even if that wasn't true -- fixing any systemic problem starts with triaging and identifying specific areas you want to target first. It is so weird to hear someone say, "I've never claimed we shouldn't fix the problem, but pointing out specific groups that the problem applies to is wrong."
No, it's identifying problem areas that are high impact: a normal, good thing for researchers to do. To the extent that it's "biased", it's the kind of bias that we want in pretty much every field and on every subject -- we want researchers and debuggers to bias themselves towards focusing on high-impact areas first when they're fixing problems. And we want them to call out specific areas that they think deserve attention. That's what triaging is.
Okay, great. But then we are arguing about an implementation, aren't we? We agree that de-biasing is necessary. It's just that you seem to think that calling out specific biases is the wrong way to approach that.
But I'm curious how you're supposed to de-bias data without drawing any attention to or acknowledging the groups that the data is biased against?
> The idea that "Internet data biases against certain groups" seems itself a biased statement.
If a data set is biased, it's biased in a direction. By definition, that's what bias is.
> I am pretty sure that Internet data is biased against all groups. I bet you can get a raw LLM to say offensive things about any group if you give it the right prompt.
That's not what bias is. Bias is not "can you get it to say something offensive" -- bias is a systematic predisposition in a direction. Bias is not stuff like, "it made a dirty joke", bias is stuff like, "this hiring model is more likely to hire you if your name sounds white."
And the goal here (believe it or not when I say this) is not to make models that never make offensive jokes. The goal is to avoid making models that reproduce and reinforce systemic issues in society. It's entirely appropriate when looking at the problem of "what social inequities are being reproduced by a model" to identify specific social inequities and inequalities that exist in society.
----
Part of the reason why I've been assuming that you're against de-biasing in general is that you keep saying stuff like "the idea that the Internet biases against certain groups is itself a biased statement."
That is nonsensical, it seems like you're suggesting that companies should be correcting against bias without determining what the direction of that bias is or who it affects. How exactly should they do that? What does it mean to correct a systemic skew without identifying what the skew is?
Again, look at the Reddit example. Yes, you can find offensive stuff on Reddit for everyone: atheists, vegans, Conservatives, Liberals, whoever. But de-biasing is not about whether or not something is offensive, it's about identifying and correcting a predisposition or tendency towards certain attitudes and thoughts. And that necessarily requires calling out what those predispositions are -- so you're going to end up with statements like "Reddit is biased against Conservative/Libertarian philosophies overall, on average."
And if somebody jumps out and says, "Reddit is biased against everyone, it's itself biased for you to call out that specific bias" -- then there's no response to that other than that's just not how any of this works. If you go into a conversation about data bias saying that it's inappropriate for us to identify specific biases, then you are effectively arguing against reducing bias, regardless of what else you say.
And even more to the point, even if that wasn't true -- fixing any systemic problem starts with triaging and identifying specific areas you want to target first. It is so weird to hear someone say, "I've never claimed we shouldn't fix the problem, but pointing out specific groups that the problem applies to is wrong."
No, it's identifying problem areas that are high impact: a normal, good thing for researchers to do. To the extent that it's "biased", it's the kind of bias that we want in pretty much every field and on every subject -- we want researchers and debuggers to bias themselves towards focusing on high-impact areas first when they're fixing problems. And we want them to call out specific areas that they think deserve attention. That's what triaging is.
On the contrary, I think saying "there is no universal truth" is a foundation of for those discussions of first principles.
I wasn't arguing against "responsible ai", I was replying to someone who made their implicit assumptions clear, even if I agreed with who I was responding to, which I do for the most part, I was trying to dig down to the granularity of their assertions. Because it's easy to make sweeping statements about what's 'good' and 'bad' (but who makes those distinctions in which context is more important than just saying it's one or the other).
I didn't bring up anything to do with minorities at all, following my logic, the question is, "which minorities and where?" It's in line with what you say about privacy, "who's privacy, what's their definition of it"
I wasn't arguing against "responsible ai", I was replying to someone who made their implicit assumptions clear, even if I agreed with who I was responding to, which I do for the most part, I was trying to dig down to the granularity of their assertions. Because it's easy to make sweeping statements about what's 'good' and 'bad' (but who makes those distinctions in which context is more important than just saying it's one or the other).
I didn't bring up anything to do with minorities at all, following my logic, the question is, "which minorities and where?" It's in line with what you say about privacy, "who's privacy, what's their definition of it"
> I wasn't arguing against "responsible ai"
That's what I'm saying. The fact that you didn't see "responsible AI" as something to question to the same degree is exactly the point I was making.
That you don't realize that "neutrality" doesn't exist in a conversation about responsible AI is the point that sgift's comment was making. There is no purpose to going into a conversation about responsible AI trying to be fully neutral about what good and bad is. The word "responsible" implies that there is such a thing as a good and bad outcome, and that the good outcomes are preferable to the bad ones.
But to restate GP's point, any conversation about user agency, privacy, equity, responsiblity, bias, accuracy, etc... all of it assumes an opinionated philosophy about what's good and bad -- and in fact, it is desirable to have those discussions working from the assumption that some things are good and some things are bad. It is impossible for any of that conversation to be neutral, nor should neutrality be a goal -- because a conversation about responsible AI is a conversation about how to influence and create those preferred world states, and how to bias an AI to create those world states.
There is no such thing as a "neutral" aligned AI. AI alignment is the process of biasing an AI towards particular responses.
----
And again, I just want to point out, these types of conversations never crop up when we're talking about stuff like user agency and privacy. When I say "people should have privacy", nobody asks me to define ethics from first principles.
And I mean, people can say "well, we just want to approach first principles in general, this isn't about minorities" -- but people can also read the thread on this page and they can see that all of this conversation spiraled out of someone complaining that Mozilla was calling out models that reinforced existing biases against minorities. That was the context.
There is no sibling thread here where someone complained about the dichotomy between safety and agency. And people can take from that what they want, but it's a pattern on HN.
That's what I'm saying. The fact that you didn't see "responsible AI" as something to question to the same degree is exactly the point I was making.
That you don't realize that "neutrality" doesn't exist in a conversation about responsible AI is the point that sgift's comment was making. There is no purpose to going into a conversation about responsible AI trying to be fully neutral about what good and bad is. The word "responsible" implies that there is such a thing as a good and bad outcome, and that the good outcomes are preferable to the bad ones.
But to restate GP's point, any conversation about user agency, privacy, equity, responsiblity, bias, accuracy, etc... all of it assumes an opinionated philosophy about what's good and bad -- and in fact, it is desirable to have those discussions working from the assumption that some things are good and some things are bad. It is impossible for any of that conversation to be neutral, nor should neutrality be a goal -- because a conversation about responsible AI is a conversation about how to influence and create those preferred world states, and how to bias an AI to create those world states.
There is no such thing as a "neutral" aligned AI. AI alignment is the process of biasing an AI towards particular responses.
----
And again, I just want to point out, these types of conversations never crop up when we're talking about stuff like user agency and privacy. When I say "people should have privacy", nobody asks me to define ethics from first principles.
And I mean, people can say "well, we just want to approach first principles in general, this isn't about minorities" -- but people can also read the thread on this page and they can see that all of this conversation spiraled out of someone complaining that Mozilla was calling out models that reinforced existing biases against minorities. That was the context.
There is no sibling thread here where someone complained about the dichotomy between safety and agency. And people can take from that what they want, but it's a pattern on HN.
Good, because equity is evil, unlike equality.
There's an assumption there that a neutral AI can exist, but I think a lot of people would challenge that central assumption. No set of training data can be truly neutral or unbiased. It may be well balanced between certain groups, but the choice of which groups to balance, how much to balance them, and the choice to add this balance in the first place are all ideological decisions that stem from certain beliefs and ideas.
The AIs that get built will reflect the ideologies and values of the people who build them. It is therefore better for us ethically to be conscious of the values we are injecting.
The AIs that get built will reflect the ideologies and values of the people who build them. It is therefore better for us ethically to be conscious of the values we are injecting.
Look at where your failure lies here -- you literally just asserted your biases as "good" and said you failed to see another perspective.
That's literally the point.
That's literally the point.
I didn't fail to see another perspective. I rejected the other positions as worse (after careful examination). That's different.
Also, neutrality is for the most part just a status quo bias. "Things are good as they are" is a position as much as every other.
Also, neutrality is for the most part just a status quo bias. "Things are good as they are" is a position as much as every other.
Zero distinction between "is" and "ought"
Your point seems to be that AI output should not be moderated. This would mean that the AI would adopt whatever biases and language patterns exist in the training data. In that scenario, the AI developer is still injecting bias by selecting which training data to use. There's also the problem that any unmoderated AI would be completely commercially unviable, of course. So, I think I understand what you're opposed to, but I'm curious what actions/methodologies you would be in favor of.
When you clearly indicate you're not neutral, you lose all credibility. Nobody wants a ML model that gives them the climate warrior version of the "truth". Neutrality is extremely important in order to be broadly taken seriously. It's exactly the kind of criticism that's been leveled against chatGPT
The irony with people talking about "neutrality" is the tendency of the people making such demands to be even more obsessed with distorting the input data to produce outcomes censored to take into account their viewpoint than the 'AI safety' and PR people.
I mean, how much censorship (or artificial curation) would you need to avoid an ML model giving "the climate warrior version" of questions about whether the world was getting warmer?!
I mean, how much censorship (or artificial curation) would you need to avoid an ML model giving "the climate warrior version" of questions about whether the world was getting warmer?!
Ah yes, I'll just reference the list of my non-neutral biases as I choose the moderation rules for my AI. My bias against swear words is neutral, so I will include that rule, but my bias against pollution is not neutral, so I will skip those moderation rules.
Obviously categorizing beliefs into "neutral" and "not neutral" is impossible. Your statement is a classic example of the false consensus effect -- everyone thinks they are neutral.
https://en.wikipedia.org/wiki/False_consensus_effect
Obviously categorizing beliefs into "neutral" and "not neutral" is impossible. Your statement is a classic example of the false consensus effect -- everyone thinks they are neutral.
https://en.wikipedia.org/wiki/False_consensus_effect
> Obviously categorizing beliefs into "neutral" and "not neutral" is impossible.
There is one distinction that matters a lot, and that is empirical vs. moral. Also known as vs. ought (https://en.wikipedia.org/wiki/Is%E2%80%93ought_problem).
When exploring empirical claims, one can aspire for neutrality. Even if perfect neutrality is not possible, it's certainly possible to do better or worse at it.
Moral claims cannot really be evaluated neutrally, I agree. But an AI striving for neutrality could attempt to avoid taking sides on contentious issues.
There is one distinction that matters a lot, and that is empirical vs. moral. Also known as vs. ought (https://en.wikipedia.org/wiki/Is%E2%80%93ought_problem).
When exploring empirical claims, one can aspire for neutrality. Even if perfect neutrality is not possible, it's certainly possible to do better or worse at it.
Moral claims cannot really be evaluated neutrally, I agree. But an AI striving for neutrality could attempt to avoid taking sides on contentious issues.
But this is exactly the problem: if there was human alignment on these issues, we would have been already on the way of tackling them. But there is not; the planet is still getting destroyed for profit, racial etc discrimination is still ongoing etc. Tackling these would require us to turn down different ethics and human rules already in place like economic liberalism, concept of copyright etc. That would class with other rights etc in place.
Human alignment is maintained through several rules and relations comprising a psychological, social and cultural complex, and even then we are witnessing continued ecological collapse, wars, corruption, social unrest etc. What is AI supposed to align with, first of all, before we even figure out the "how" (as AI does not fall in the sphere of those relations described above that affect human alignment)?
Human alignment is maintained through several rules and relations comprising a psychological, social and cultural complex, and even then we are witnessing continued ecological collapse, wars, corruption, social unrest etc. What is AI supposed to align with, first of all, before we even figure out the "how" (as AI does not fall in the sphere of those relations described above that affect human alignment)?
Because it implies the "climate crisis" is a real thing. For some it surely is, others - me among them - see this differently. Time will tell who got it right but just the fact that the media more or less dropped the "climate" scare when SARS2 hit the front pages should give those who are in the former camp something to think about. Only when it became clear that there was no more to be gained from pushing SARS2 scare stories did they return to the climate narrative. A LLM which has been trained to push "climate crisis" will end up producing agitprop [1] instead of objective output. The same would be true for a model which has been trained to deny anything related to climate change but thus far I have not seen any call for such training methods to be used.
[1] https://www.britannica.com/topic/agitprop
[1] https://www.britannica.com/topic/agitprop
The other statements are biased too, but they're biased in favor of privacy and transparency. Being biased, here, just means having values and applying them - do you disagree with having values or the values themselves?
It’s Mozilla, what do you expect? That’s their whole schtick these days.
Are we talking about algorithms or AI?
An algorithm is a set of predetermined logic. The thing itself does not "make decisions", it applies decisions that were already made by writing it.
An AI is an Artificial Intelligence. A non-human thinker. Something that can "make decisions". Such a thing does not exist.
---
This is the problem with "AI research". It's sensible as a category of pursuit, but until you have accomplished that pursuit, there literally does not exist a single instance of an AI.
Somehow, that distinction has been ignored from the word "go". Every project in the pursuit of AI is itself already called an AI! No wonder people are so confused!
It's plain to see that all of this fear and uncertainty could be cleared up with a simple change in nomenclature. Stop calling projects "AI", and we can all move on from this silly debate.
An algorithm is a set of predetermined logic. The thing itself does not "make decisions", it applies decisions that were already made by writing it.
An AI is an Artificial Intelligence. A non-human thinker. Something that can "make decisions". Such a thing does not exist.
---
This is the problem with "AI research". It's sensible as a category of pursuit, but until you have accomplished that pursuit, there literally does not exist a single instance of an AI.
Somehow, that distinction has been ignored from the word "go". Every project in the pursuit of AI is itself already called an AI! No wonder people are so confused!
It's plain to see that all of this fear and uncertainty could be cleared up with a simple change in nomenclature. Stop calling projects "AI", and we can all move on from this silly debate.
I fully second this sentiment. But we know people will fight tooth and nail over the gilding that masks their normalcy.
Yeah unfortunately it often ends up being a code for adjusting ML models to support certain world views or political biases.
It's too bad we haven't been able to separate the data science questions of how we feel about the training data, from the operational questions of whether (a) it's appropriate to make a determination algorithmically and (b) whether the specific model is suited to that decision. Instead we get vague statements about harms and biases.
It's too bad we haven't been able to separate the data science questions of how we feel about the training data, from the operational questions of whether (a) it's appropriate to make a determination algorithmically and (b) whether the specific model is suited to that decision. Instead we get vague statements about harms and biases.
> everybody is supposed to work on "responsible AI" while nobody can really say what a "responsible AI" is really supposed to be
In my opinion, "working on" responsible AI at this stage is synonymous with figuring out how to actually define what that means. Part of that definition will emerge along with and as the technology evolves. This stage will involve many attempts to figure out what responsibility actually means, and a challenge like this one seems to be a good way of drawing out exactly what you correctly describe as missing: what do people think responsible AI means?
I share the frustration that we don't have human alignment on this, and that such alignment is required, but to achieve that, people involved need to start putting real thought into formulating some notion of what this means, because even if we don't know if we're currently in the right ballpark, we do know that the failure modes can be catastrophic.
Human alignment is not something that will happen without major/messy disagreements and conflict about what responsibility actually entails. And to have those disagreements, companies building these products need to start standing up and staking claims on what they believe it to mean.
So in my view, what Mozilla is doing here seems like an important piece of the puzzle in this moment where what we need most are opinions about what safety entails, so we can even have a chance of moving towards alignment.
In my opinion, "working on" responsible AI at this stage is synonymous with figuring out how to actually define what that means. Part of that definition will emerge along with and as the technology evolves. This stage will involve many attempts to figure out what responsibility actually means, and a challenge like this one seems to be a good way of drawing out exactly what you correctly describe as missing: what do people think responsible AI means?
I share the frustration that we don't have human alignment on this, and that such alignment is required, but to achieve that, people involved need to start putting real thought into formulating some notion of what this means, because even if we don't know if we're currently in the right ballpark, we do know that the failure modes can be catastrophic.
Human alignment is not something that will happen without major/messy disagreements and conflict about what responsibility actually entails. And to have those disagreements, companies building these products need to start standing up and staking claims on what they believe it to mean.
So in my view, what Mozilla is doing here seems like an important piece of the puzzle in this moment where what we need most are opinions about what safety entails, so we can even have a chance of moving towards alignment.
> The hardest issue regarding "AI alignment" is human alignment.
Which is partly why the current proposed alignment theory isn't possible. We want to align the AGI by applying human values. Even if we figure out how to get the machine to adopt such values, they are the same values that lead us humans into constant conflict.
I've stated this argument in much more detail here - https://dakara.substack.com/p/ai-singularity-the-hubris-trap
Which is partly why the current proposed alignment theory isn't possible. We want to align the AGI by applying human values. Even if we figure out how to get the machine to adopt such values, they are the same values that lead us humans into constant conflict.
I've stated this argument in much more detail here - https://dakara.substack.com/p/ai-singularity-the-hubris-trap
With all the AI discussion of late I've been thinking more about the alignment problem.
Like, let's say you have a true AGI that's completely superior to human intellect AND you've found a way to align this thing. Can those alignment techniques also be used on the lesser mind of humanity?
"Oh don't worry. Alignment is probably achievable by how we build the AGI."
Yeah I mean maybe. Or maybe it turns out alignment is only possible at all if it works generally on anything with 'intelligence'.
My suspicion is that alignment is either not possible because an intelligent agent can always do something just because. Or we're going to have to worry about whoever wins the alignment race aligning the rest of us.
Like, let's say you have a true AGI that's completely superior to human intellect AND you've found a way to align this thing. Can those alignment techniques also be used on the lesser mind of humanity?
"Oh don't worry. Alignment is probably achievable by how we build the AGI."
Yeah I mean maybe. Or maybe it turns out alignment is only possible at all if it works generally on anything with 'intelligence'.
My suspicion is that alignment is either not possible because an intelligent agent can always do something just because. Or we're going to have to worry about whoever wins the alignment race aligning the rest of us.
Yes, I think those are possibilities.
In simple terms, I think we will either be in control of power we are not prepared to manage or we will be managed by power that we can not control.
In simple terms, I think we will either be in control of power we are not prepared to manage or we will be managed by power that we can not control.
I think you’re a brilliant writer. This is a fantastic article. Thank you for articulating the problem so clearly.
The only hope I have is that because we’re building these things largely in our image, because we’re fallible and often the systems we build are too, we’re actually building something quite stupid and incapable of mass scale destruction. Hopefully the laws of physics or intellect are reached before anything gets to too out of control.
This is why “containment” is such a precarious concept. In the end, we may find our plan to place the AI into checkmate only resulted in placing ourselves into checkmate.
The only hope I have is that because we’re building these things largely in our image, because we’re fallible and often the systems we build are too, we’re actually building something quite stupid and incapable of mass scale destruction. Hopefully the laws of physics or intellect are reached before anything gets to too out of control.
This is why “containment” is such a precarious concept. In the end, we may find our plan to place the AI into checkmate only resulted in placing ourselves into checkmate.
Thank you!
> because we’re fallible and often the systems we build are too
Yes, the basis of my article is using the assumptions of the alignment theorists who believe that we can build an AGI and it will be by default "power seeking".
Of course, either could be wrong. I agree that is really the best hope that either we hit some unexpected limitations or the problem we perceive simply does not manifest as we fear.
> because we’re fallible and often the systems we build are too
Yes, the basis of my article is using the assumptions of the alignment theorists who believe that we can build an AGI and it will be by default "power seeking".
Of course, either could be wrong. I agree that is really the best hope that either we hit some unexpected limitations or the problem we perceive simply does not manifest as we fear.
I suppose we’re always living on the edge, this time is no different.
[deleted]
Whatever Bay Arean mega-corps profiting social bubble tells you it is. Everything else is UNACCEPTABLE!
This is like any “X for humans” or “humane X”; completely devoid of meaning.
25 grand is the best we can do for something like this?
So I tried applying. First the actual email form just doesn't load with an adblocker enabled. When disabled, I can't even submit the form since "element with "privacy" is not focusable" whatever that means.
How very ironic.
How very ironic.
Isn't this a common problem with adblockers though? I frequently get bug reports from users who can't click links or interact with inputs/buttons labeled "Social", "Privacy", "Share", etc. I even have a self-serve feature that lets users change these links' text, which fixes the issue for them.
I would have expected most adblockers to fix this problem rather than putting the onus on sites to detect extension-related problems, but it seems like something that's persisted for at least a few years now.
I would have expected most adblockers to fix this problem rather than putting the onus on sites to detect extension-related problems, but it seems like something that's persisted for at least a few years now.
If you build non-shite straight to the point functional websites without tracking software you actually do not use to gain any actionable insighr, adblockers will not break your page.
> Responsible AI Challenge (impossible)
There, more accurate. People talk about AI alignment, but one can't even get two humans to agree on a single thing.
There, more accurate. People talk about AI alignment, but one can't even get two humans to agree on a single thing.
Although I would agree with you if they had titled it "alignment", they chose "responsible", which is much easier: https://foundation.mozilla.org/en/internet-health/trustworth...
(Linked from the text "How does it address our Responsible AI Guidelines", I appreciate the irony of me having said this given the destination of the link has yet another title).
(Linked from the text "How does it address our Responsible AI Guidelines", I appreciate the irony of me having said this given the destination of the link has yet another title).
[deleted]