This is a valuable lesson in what AI can do for society: act as a mirror to show us systemic biases that exist that people try to pretend don't exist.
I think most here would understand systemic bias if reflected in a distributed system. Imagine a classifier where, through no explicit intent of the designer of the classifier, always said that some user login was malicious or some binary was malware or the like, even when it was not. And all those users or binaries had something in common that was apparent, and were all being misclassified. That would be systemic bias, and nobody would blink an eye about a call for it to be corrected.
Here we see systemic racial bias, probably not because of individual racial animus on the part of some Google engineer, but because society has systemic racial bias and the classifier is just reflecting that because it was trained on public data.
We should look at incidents like this differently: not as yet another chance to argue about what Google should or shouldn't do, but as a chance to see that a socially-neutral AI that does not have bias built into it is learning racial bias from society.
I think most here would understand systemic bias if reflected in a distributed system. Imagine a classifier where, through no explicit intent of the designer of the classifier, always said that some user login was malicious or some binary was malware or the like, even when it was not. And all those users or binaries had something in common that was apparent, and were all being misclassified. That would be systemic bias, and nobody would blink an eye about a call for it to be corrected.
Here we see systemic racial bias, probably not because of individual racial animus on the part of some Google engineer, but because society has systemic racial bias and the classifier is just reflecting that because it was trained on public data.
We should look at incidents like this differently: not as yet another chance to argue about what Google should or shouldn't do, but as a chance to see that a socially-neutral AI that does not have bias built into it is learning racial bias from society.