The context is probably what's throwing you off then. I took bilsbie's comment to be a context expansion here. Imagining how this general strategy of censoring what's actually out there plays out in the long run and wider context of how all of humanity deals or doesn't deal with what human minds are actually capable of producing.
Also note that this wouldn't just be about a bot producing more bad stuff. ChatGPT also, or even mostly, answers questions and provides information. If bots like this do not even know about the filtered out content they cannot, for example, give an accurate or original answer to a question like: "Is nazi-ism on the rise?"
The best they can do is regurgitate news posts, pundit commentary, and existing research articles. That might be good enough, but if a bot were able to see all the content out there it might even give a statistical summary of everything on the web and comment on the rate at which pro-nazi web articles are published compared to articles of all other types. That would be a useful way to shed some daylight on the issue.
If most of our efforts most of the time are spent merely filtering out the trash, then we're still just ignoring pretty big issues. The strategy as applied here isn't exactly wrong for what OpenAI is trying to achieve but it still ends up being another example of people being more willing to spend a lot of resources pretending humans are better than they are than we are willing to spend a lot of resources fixing the underlying issues that make some people so screwed up to begin with.
This new ml model they are building will be used to filter out child porn, among other things, from the output of ChatGPT, so users of that bot don't see it. It seems their ideal case is to reliably filter it from the input as well, so even the bot doesn't know about it. Which is fine for what they're doing, so they'll probably stop there.
However, once you have something like a reliable detector for human misery and the warped human minds the produce it you're actually well on your way to developing a tool chain that could help authorities and mental health professionals identify and track down those warped mind as soon as or possibly before they cause harm.
I'm not confident I know the whole field well enough to say no one is doing something like that but it does seem like the preferred option for most all AI teams is to spend $200k filtering out the problem rather than taking the time to build something that reduces or eliminates the root cause. And that stance is reasonable enough at the team/company levels because you'd have to do the former anyway even if you also do the latter.
So of course it's not OpenAI's job to do that and this isn't a moralistic judgement on the teams building AI in general either. Humanity just isn't interested in solving those problems for real in the same way we are interested in a bot that can write code or tell us interesting stories. We want to know the police are on it in some capacity, but there's not likely to be enough investment in it to actually solve the problems any time soon.
In the long run, being increasingly good at filtering this stuff out tends toward a future where most people can live their whole lives without ever becoming aware that these awful problems exist and possibly happen in their own cities. And if most people are never aware of a problem, it's not likely to ever get enough attention and funding to be solved.
So that is the sense in which I can find agreement with bilsbie's notion that merely filtering out the bad stuff humans produce can be more toxic than letting people see what humans are really like. Though again, it's hard to blame any one team for just wanting to filter that stuff out.
---
And there is actually at least one unrelated other way to look at bilsbie's comment so you might ask them what they meant as well. The other way I can think of is asking who decides what's acceptable content? And then expand the context on that to a hypothetical world where stuff like ChatGPT is ubiquitous.
There are some obvious cases we'll all pretty universally call bad, so we don't mind someone training the bots to filter that out. However, there's other examples like a lot of horror books/films or even mystery/detective novels and such that push the boundaries in order to get people to think, if those get caught in the filters too, then we're doing humanity a disservice by effectively banning/digitally-burning books/articles/web-sites. And let's not forget, if people are good at building such filter systems for all the bots you can bet governments with a eye for controlling thought will be interested in installing some filters of their own.
No one is suggesting we should embrace the worst. Only that ignoring it at every chance instead of ever properly dealing with it isn't the most helpful thing if we actually want to follow that desire to go beyond.
> If a business derives $X profit from a laborer and agrees to pay a given % of $X to the laborer as compensation, it should not matter if the laborer is in a high cost or low cost of living area
That would be true if that were deal the business had with their employees. However, you made up the percentage thing by observation and are working from the assumption that because you can represent a wage as a percentage of a profit margin, that means the business agreed to share a fixed percentage of profit with the employee. This is incorrect reasoning.
Businesses generally don't contract with their employees to share a percentage of revenue or profits, with the narrow exception of commissions for people in sales. With regard to developers and almost all other employees, the contract both parties agreed to is almost always a trade of specific amounts of time for specific amounts of money and possibly some flexible additional benefits. And to the employee's benefit, the amount of pay does not flex based on whether the company is even producing a profit from the sale of the the products of their labor, only how many hours they put in. Caveat all manner of fuckery in labor relations, of course. Which is a giant caveat, I agree in advance.
However the fundamental agreement is still time for money without the employee having to worry whether the products of their labor can be sold for more than their hourly wage, that's the business's problem to deal with. The employee expects their paycheck to show up on time all the time, or the deal is off.
With that sort of agreement, it is absolutely expected and reasonable that the business keeps any profit from selling the product of the labor after the agreed upon fees for said labor have been paid. That was the deal.
---
If you'd like a different deal for most people, as I would love to see more of in the world economic future, then most people need to change over from needing a job to needing some form of income to support their long term future growth as they live currently off of savings.
Instead of expecting someone else to ensure there is a buffer of money to pay you so you can pay your expenses, you will have to keep that buffer yourself and be okay with the fact that sometimes you will get paid a lot and sometimes not so much, but you will then be getting that percentage of the profit you wanted and it will be totally reasonable to expect the same percentage regardless of what the cost of living is.
You also won't have to worry about layoffs as much because the company's costs now also scale directly with it's profits, so there is no need to cut head count when things get really bad and the company's buffers run low.
But again, that also means you will be responsible for keeping a big enough buffer yourself to deal with the fact that your income fluctuates with both short an long term market swings. Otherwise all you have done is trade the annoyance at seeing the company profit massively from your fix-time-cost labor for the annoyance of being broke even though you have a job whenever the market takes a shit.
Most people still don't want to deal with that, even in super wealthy countries like the US. In fact most people in wealthy countries actually scale their own living expenses in step with any increases in their income such that they can be living paycheck-to-paycheck even at $200k+/yr.
So the balance of things present day is that most people take full-time or even part-time contracts as employees and hope they don't get fired at the wrong time in life.
This is possible to change though, for yourself and for others if you like. So if you think your current deal is bullshit, feel free to change the deal.
So long as developers want to be paid by the hour rather than paid for a specific result, I would say this is fine too.
The problem everyone is waffling on here isn't the competitiveness of the wage, it's the gap between the value of what the labor produces and the value offered to produce it.
But you can't escape that gap so long as you're fundamentally trading time for money.
If you want to capture more of the value of your work, you become a contractor and bid firm-fixed or cost-plus contracts instead of labor-hour or time-and-materials.
If you want to capture even more value than that, you work on building business systems and selling a valuable product or service rather than spending your work hours doing the tasks themselves.
Whether you're in Kenya or the US and whether you're doing undesirable or highly desirable work, it is not a moral failing of the business you are working for that they did not also take the time to ensure that you could have become a contractor if you wanted to or that they did not ensure that your economic situation allows sufficient market opportunities for you to quit and form a business if you wanted to.
Instead of dealing with the fact that humans do and produce tons of awful shit, we just do our best to ignore it and pretend human nature is better than it is.
This is delusion on a grand scale and it's hard to find examples in the historical record of good things ultimately coming from lots of people acting in the world while being divorced from reality.
It's worth noting that OpenAI actually paid around six times that amount for this work.
It's also worth noting that Sama actually did provide for their employee's mental health. They didn't do a good job of it but trying and failing at a thing doesn't quite qualify at the level of disgust imo. Especially not for the company that was outsourcing this.
It's also worth noting that even if those of us in the peanut gallery would not trust the word of either OpenAI or Sama, it is not unreasonable for OpenAI's team to have trusted that Sama was doing what they said they were with regard to providing for their own employees mental health.
That's not proportional, by design even. The whole game here is to do a small amount of work and multiply the gains by an infinite number of results from people using the bot.
That's always the game with any kind of technology. So no, having a few people experience something bad so billions can avoid experiencing anything bad ever does not cancel out.
As a war veteran I can confidently say most veterans would probably not be traumatized by work like this. We've all seen much worse and the things that became problematic memories for me had little to do with reading/hearing/seeing the worst humanity has to offer in terms of violence and fucked up shit people do to each other. The stuff that really sticks and eats at you is usually stuff that happened to or close the individual or something that happened as the result of actions taken by the individual or their immediate group.
Probably the most important measure a company could take to prevent lasting harm with this kind of work would be to spread it around a whole lot more than just 36 people. The real risk of long term impact here would probably be with persistent exposure to it all day long. Most people can handle reading or seeing some graphic stuff with proper mental preparation for it but to see nothing but that day and day out would quickly wear you down.
>Is it ever appropriate or ethical to ask humans to voluntarily subject themselves to traumatic experiences in exchange for compensation?
Probably not. Yet people still voluntarily sign up for military service around the world by the millions, and they do so for a bunch of personal, family, idealistic, cultural, and societal reasons that are hard to reduce to a few easy to argue points like a lot of people online try to do with stuff like this.
Personally, I think it's admirable to hold the ideal that we should like to never offer jobs like this, we should also like to never offer jobs that involve going to war, cleaning up hazardous materials, dealing with explosives, working around heavy fast-moving machines, cleaning sewers, or a myriad of other terrible experiences either; but we're probably not in a position to make those better choices just yet. Until then, people are willing to do these things for a dozen different reasons per person, only two of which are the pay and support they get from the employer.
Also note that this wouldn't just be about a bot producing more bad stuff. ChatGPT also, or even mostly, answers questions and provides information. If bots like this do not even know about the filtered out content they cannot, for example, give an accurate or original answer to a question like: "Is nazi-ism on the rise?" The best they can do is regurgitate news posts, pundit commentary, and existing research articles. That might be good enough, but if a bot were able to see all the content out there it might even give a statistical summary of everything on the web and comment on the rate at which pro-nazi web articles are published compared to articles of all other types. That would be a useful way to shed some daylight on the issue.
If most of our efforts most of the time are spent merely filtering out the trash, then we're still just ignoring pretty big issues. The strategy as applied here isn't exactly wrong for what OpenAI is trying to achieve but it still ends up being another example of people being more willing to spend a lot of resources pretending humans are better than they are than we are willing to spend a lot of resources fixing the underlying issues that make some people so screwed up to begin with.
This new ml model they are building will be used to filter out child porn, among other things, from the output of ChatGPT, so users of that bot don't see it. It seems their ideal case is to reliably filter it from the input as well, so even the bot doesn't know about it. Which is fine for what they're doing, so they'll probably stop there.
However, once you have something like a reliable detector for human misery and the warped human minds the produce it you're actually well on your way to developing a tool chain that could help authorities and mental health professionals identify and track down those warped mind as soon as or possibly before they cause harm.
I'm not confident I know the whole field well enough to say no one is doing something like that but it does seem like the preferred option for most all AI teams is to spend $200k filtering out the problem rather than taking the time to build something that reduces or eliminates the root cause. And that stance is reasonable enough at the team/company levels because you'd have to do the former anyway even if you also do the latter.
So of course it's not OpenAI's job to do that and this isn't a moralistic judgement on the teams building AI in general either. Humanity just isn't interested in solving those problems for real in the same way we are interested in a bot that can write code or tell us interesting stories. We want to know the police are on it in some capacity, but there's not likely to be enough investment in it to actually solve the problems any time soon.
In the long run, being increasingly good at filtering this stuff out tends toward a future where most people can live their whole lives without ever becoming aware that these awful problems exist and possibly happen in their own cities. And if most people are never aware of a problem, it's not likely to ever get enough attention and funding to be solved.
So that is the sense in which I can find agreement with bilsbie's notion that merely filtering out the bad stuff humans produce can be more toxic than letting people see what humans are really like. Though again, it's hard to blame any one team for just wanting to filter that stuff out.
---
And there is actually at least one unrelated other way to look at bilsbie's comment so you might ask them what they meant as well. The other way I can think of is asking who decides what's acceptable content? And then expand the context on that to a hypothetical world where stuff like ChatGPT is ubiquitous. There are some obvious cases we'll all pretty universally call bad, so we don't mind someone training the bots to filter that out. However, there's other examples like a lot of horror books/films or even mystery/detective novels and such that push the boundaries in order to get people to think, if those get caught in the filters too, then we're doing humanity a disservice by effectively banning/digitally-burning books/articles/web-sites. And let's not forget, if people are good at building such filter systems for all the bots you can bet governments with a eye for controlling thought will be interested in installing some filters of their own.