Publishers want billions, not millions, from AI(semafor.com)
semafor.com
Publishers want billions, not millions, from AI
https://www.semafor.com/article/07/23/2023/publishers-want-billions-not-millions-from-ai
77 comments
I'm a mod/admin on a topic site, similar to HN in structure... one of the posts I removed a few weeks ago literally made me feel ill... it was a guide on setting up a website, then using an LLM to generate hundreds of "articles" for said site.
I've started seeing content sites that are obviously generated content looking at it... mostly in terms of recipe content for lower carb, or sugar free... some list ingredients, but no measurements, others list directions that don't match ingredients. TBH, these kinds of activities make the internet less useful overall.
Between the above and sock puppet accounts, I can't help but think it may be a time to return to self-hosted systems like the BBSes of the 80's and early 90's. I know some still exist, but they're mostly around those that were into the tech, or otherwise into the older turn based text games or artwork. I know there's various systems from discorse to lemmy that are a bit more modern. I can't help but think something closer to a single server version of Facebook Groups +_Chat might go a long way... not even federated, but single-interrest groups that aren't beholden to a larger org (reddit, etc).
I've started seeing content sites that are obviously generated content looking at it... mostly in terms of recipe content for lower carb, or sugar free... some list ingredients, but no measurements, others list directions that don't match ingredients. TBH, these kinds of activities make the internet less useful overall.
Between the above and sock puppet accounts, I can't help but think it may be a time to return to self-hosted systems like the BBSes of the 80's and early 90's. I know some still exist, but they're mostly around those that were into the tech, or otherwise into the older turn based text games or artwork. I know there's various systems from discorse to lemmy that are a bit more modern. I can't help but think something closer to a single server version of Facebook Groups +_Chat might go a long way... not even federated, but single-interrest groups that aren't beholden to a larger org (reddit, etc).
That's fascinating. I'm also seeing the sort of effects that this recession has been having on people especially online, where a lot of the guides you're talking about are being pushed more and more as a way to "side hustle". Having a side hustle is pushed more and more these days because a "main" hustle doesn't exist anymore due to wage stagnation.
Part of the popularity of these sort of things stems entirely from how much money is reportedly up for grabs. If you tell people they can do this and bring in 2k a month on the side, they're all in. Likewise, if we had enough funds to not have to worry, these sort of things would rarely be used as there's no real joy in the pursuit of it. My theory on why AI is accelerating garbage content much faster than any other potential benefits.
Even the highlight of recipes is notable, recipe blogs are great earners due to the stability of ads and output of content. This becomes a target for AI (when I say AI here I mean people consuming AI API) due to the high ad value target. That way even if they do eventually drop, get found out, reported to google, all of that, they would have still made enough to do it again.
Part of the popularity of these sort of things stems entirely from how much money is reportedly up for grabs. If you tell people they can do this and bring in 2k a month on the side, they're all in. Likewise, if we had enough funds to not have to worry, these sort of things would rarely be used as there's no real joy in the pursuit of it. My theory on why AI is accelerating garbage content much faster than any other potential benefits.
Even the highlight of recipes is notable, recipe blogs are great earners due to the stability of ads and output of content. This becomes a target for AI (when I say AI here I mean people consuming AI API) due to the high ad value target. That way even if they do eventually drop, get found out, reported to google, all of that, they would have still made enough to do it again.
My Twitter verse is filled with people making websites with thousands of pageviews from ai generated content. Google obviously is turning a blind eye to this possibly because they are planning on something a lot bigger than chatgpt with ai. And when it happens the internet will be littered with ai generated content that it is impossible for anyone to accuse them and them alone for stealing content.
>I can't help but think it may be a time to return to self-hosted systems like the BBSes of the 80's and early 90's
I couldn't agree more. We need to stop using other people's computers to host our stuff. (Or, at least we need to rent the machines ourselves). These days, the BitTorrent protocol is popular enough that you could likely host the equivalent of a YouTube channel for very little actual bandwidth cost.
Text, forums, etc... without media, are almost free until you scale into millions of users.
I couldn't agree more. We need to stop using other people's computers to host our stuff. (Or, at least we need to rent the machines ourselves). These days, the BitTorrent protocol is popular enough that you could likely host the equivalent of a YouTube channel for very little actual bandwidth cost.
Text, forums, etc... without media, are almost free until you scale into millions of users.
Doesn’t the BitTorrent protocol use other people’s computers to host your stuff?
It uses everyone who has already downloaded a file to help share it, so the more popular a file is, the more people host it.
You don't have to keep sharing though, once you've downloaded it. It's voluntary.
You don't have to keep sharing though, once you've downloaded it. It's voluntary.
To the above, have thought about trying to create some kind of turn-key option or close to it that can run on Cloudflare workers + D1 or similar... just to let people get started relatively inexpensively.
> If you train your AI on a bunch of someone elses work, you can make it produce something very simiilar
As a blogger of 10+ years this has been very interesting to realize. I can use ChatGPT to write really good first drafts because “in the style of Swizec Teller” works as a prompt. The results are way better than the usual generic corporate drone style of output.
BUT! And this is an important but. The insights it produces remain at that generic corporate drivel level. Even with a full list of bullet points of cool insights, an LLM takes those and averages them out into a big pile of meh. Every sharp insight gets dulled by “AI protections”, every hot take is two-sided to death, etc.
Despite superficially following my style, the AI seems unable to catch your attention. Everything it writes is a little boring, a little too long, a dash tiring to read. The spark is missing.
Also my style has evolved since 2021 and that’s missing.
As a blogger of 10+ years this has been very interesting to realize. I can use ChatGPT to write really good first drafts because “in the style of Swizec Teller” works as a prompt. The results are way better than the usual generic corporate drone style of output.
BUT! And this is an important but. The insights it produces remain at that generic corporate drivel level. Even with a full list of bullet points of cool insights, an LLM takes those and averages them out into a big pile of meh. Every sharp insight gets dulled by “AI protections”, every hot take is two-sided to death, etc.
Despite superficially following my style, the AI seems unable to catch your attention. Everything it writes is a little boring, a little too long, a dash tiring to read. The spark is missing.
Also my style has evolved since 2021 and that’s missing.
> Every sharp insight gets dulled by “AI protections”, every hot take is two-sided to death, etc.
This is a OpenAI thing.
Run a "uncensored" llama finetune, like airoboros 70b. They are as hot and spicy as you want them to be.
EDIT: Actually someone is hosting 65b here, so you can test it and see what I mean: https://lite.koboldai.net/
Note that you have to get the instruct prompting syntax right.
This is a OpenAI thing.
Run a "uncensored" llama finetune, like airoboros 70b. They are as hot and spicy as you want them to be.
EDIT: Actually someone is hosting 65b here, so you can test it and see what I mean: https://lite.koboldai.net/
Note that you have to get the instruct prompting syntax right.
If you train an LLM to mimic your own style, is that even plagiarism?
Relevant Zizek: https://www.newsweek.com/slavoj-zizek-self-plagiarized-new-y...
Relevant Zizek: https://www.newsweek.com/slavoj-zizek-self-plagiarized-new-y...
I think one of the flaws is: nobody seems to want to agree that editing is far more expensive than writing (the to point where editing is routinely skipped). I think it is well established in software engineering debugging is far more expensive then specification, writing, or testing- so one does not leave the big jobs for debugging. I personally believe the same is true for writing.
As a blogger, would you consider that approach to produce content? Don’t you find it incredibly boring and uninspiring?
> Don’t you find it incredibly boring and uninspiring?
That depends. For me the fun part is in producing the novel insight. But nobody wants to read 5 bullet points and they wouldn't get it either.
Turning those 5 bullet points into a narrative that guides readers towards understanding, that part feels like work. If AI can create the skeleton that I then edit into good writing, that feels like a win.
The writing process for me goes something like this: 1) Read a bunch of books, 2) Percolate/simmer/gestate for weeks, 3) Observe reality and seek anecdotes, 4) Simmer some more, 5) Wake up one day with 4 insightful bullet points, 6) Turn that into 600+ words (this part feels like work)
That depends. For me the fun part is in producing the novel insight. But nobody wants to read 5 bullet points and they wouldn't get it either.
Turning those 5 bullet points into a narrative that guides readers towards understanding, that part feels like work. If AI can create the skeleton that I then edit into good writing, that feels like a win.
The writing process for me goes something like this: 1) Read a bunch of books, 2) Percolate/simmer/gestate for weeks, 3) Observe reality and seek anecdotes, 4) Simmer some more, 5) Wake up one day with 4 insightful bullet points, 6) Turn that into 600+ words (this part feels like work)
Acknowledging these LLMs are not human, but isn’t this kinda what humans do? Take in lots of different examples and produce something similar but distinctly different and not paying royalties or being considered plagiarism.
I mean, yes, kinda, but the sheer scale of what AI can do vastly, vastly changes the impact on society.
To give another "scale makes all the difference" analogy. Very early on when Google Maps Street View was originally released, there was debate about whether it was OK to show individuals faces. In the US at least, the argument went something like "People are out in public, they should have no expectation of privacy. If it's legal for a person to take a picture of someone else on the street (which it is in the US), why should Street View have any privacy concerns?"
The difference is that while I may expect that other people may see and even take a picture of me if I'm outside, that's different from making my picture searchable, geolocated, instantly available to billions of people across the world and online forever. And I think it's totally rational to think these 2 different situations require different approaches.
Lots of our previously common, intuitive notions of what is OK can change greatly when large scale automation enters the mix.
To give another "scale makes all the difference" analogy. Very early on when Google Maps Street View was originally released, there was debate about whether it was OK to show individuals faces. In the US at least, the argument went something like "People are out in public, they should have no expectation of privacy. If it's legal for a person to take a picture of someone else on the street (which it is in the US), why should Street View have any privacy concerns?"
The difference is that while I may expect that other people may see and even take a picture of me if I'm outside, that's different from making my picture searchable, geolocated, instantly available to billions of people across the world and online forever. And I think it's totally rational to think these 2 different situations require different approaches.
Lots of our previously common, intuitive notions of what is OK can change greatly when large scale automation enters the mix.
Humans also typically pay up-front for the content that inspires them and assimilate it over the long course of their natural lives, and are constrained in overall output.
Mechanizing that process so that it can be wielded at scale and automated is a pretty significant "change to the contract", and worth exploring what compensation is fair under these new circumstances.
Mechanizing that process so that it can be wielded at scale and automated is a pretty significant "change to the contract", and worth exploring what compensation is fair under these new circumstances.
Perhaps it's the "being human" piece that's important for determining whether royalties can be skipped or not?
Sometimes they do, sometimes they don’t. I think most people want credit for the original ideas and effort they put in, even for something like fanfiction. And fanfic is already legally very gray! Street artists in NYC selling copies of copyrighted work aren’t there to be creative, of course. But they’re also definitely breaking copyright law.
Yeah, I mean we have this term “inspiration” already for some time. I’ve long thought that when a person hits a dead end creatively what they need is to to go out and have more experiences and be exposed to new things.
The real "deniability" of copying will come when the NLP community gets off of its collective rear end and implements actual prompt engineering (i.e. any technique I mention in here: https://gist.github.com/Hellisotherpeople/45c619ee22aac6865c...). Using "prompt blending" (i.e. https://github.com/ljleb/prompt-fusion-extension) will give genuine "deniability" on the grounds that "A beautiful story by {stephen_king|plato|aristotle|virgina_wolf}" will be very unique and moreover such kinds of interpolation are for the most part how real human writers write.
Taxi drivers already were independent contractors.
And taxi drivers themselves caused Uber to be so popular.
The amount of times I've gotten into a cab in London before Uber and the cabbie was like: "Cash only". What's that machine for then? "Credit card machine down". Yea right...
I absolutely hate what Uber did with 'contractors' but I cannot deny the fact that I like being able to travel for work without having to worry about whether the cab has a credit card machine or not.
The amount of times I've gotten into a cab in London before Uber and the cabbie was like: "Cash only". What's that machine for then? "Credit card machine down". Yea right...
I absolutely hate what Uber did with 'contractors' but I cannot deny the fact that I like being able to travel for work without having to worry about whether the cab has a credit card machine or not.
> And taxi drivers themselves caused Uber to be so popular.
That entirely depends on where you are. Where I am, ride-share services were never better than taxis in any way except for price.
That entirely depends on where you are. Where I am, ride-share services were never better than taxis in any way except for price.
I wish I could say the same about my location. Where I live they can't even tell you how much the ride will be.
I am out of sympathy for big publishers. They have been greedy and predatory for decades, at the expense of their own content.
"Publishers want billions (for themselves)" is a headline we could have published at nearly any point in post-Gutenberg history.
> “Search was designed to find the best of the internet”
Says a publisher who clearly hasn't used search in the past 10 years.
> “Search was designed to find the best of the internet”
Says a publisher who clearly hasn't used search in the past 10 years.
Yes, I agree, they could have invested in the various AI companies, instead the kept bleeding their content creators of all they have. Well the boat sailed, see you later big publishers.
> they could have invested in the various AI companies
Not really what I meant.
> instead the kept bleeding their content creators of all they have
But yes, that. And their customers too.
Not really what I meant.
> instead the kept bleeding their content creators of all they have
But yes, that. And their customers too.
So this was the outcome I guessed would need to happen, but I didn't think publishers would be the ones to push for it, but I guess they are.
The thing is if this upsets you on some gut level, this is just a sign for how valuable information to train on is. May be billions is too much, but hundreds of millions? If your immediate thought is "this must be stopped, this isn't fair" it is a signal of how reliant AI at all is on the data it trains. An untrained nn is useless, it's only useful if it has information to train on. It's not like value creation is zero sum but it certainly doesn't come from zero, and the "something" it comes from is definitely more of the data you train on than the training method you choose.
The thing is if this upsets you on some gut level, this is just a sign for how valuable information to train on is. May be billions is too much, but hundreds of millions? If your immediate thought is "this must be stopped, this isn't fair" it is a signal of how reliant AI at all is on the data it trains. An untrained nn is useless, it's only useful if it has information to train on. It's not like value creation is zero sum but it certainly doesn't come from zero, and the "something" it comes from is definitely more of the data you train on than the training method you choose.
Don’t worry, Sam will give you a pittance in the form of worldcoin for allowing his company to commandeer everything that humanity has ever or will ever produce. Nothing to see here.
> “The thing that everyone wants to talk about is whether AI is going take over the world to eliminate humans and all that stuff,” IAC CEO Joey Levin
"What I want to talk about is whether we will get royalties from the elimination of all humans"
"What I want to talk about is whether we will get royalties from the elimination of all humans"
Presumably OpenAI and others (for the most part) are checking the licenses for content they use in training?
Let’s say for example that they trained on the content of the entire archive of the New York Times… isn’t it safe to say they’d have purchased a license for that content from NYT?
Wouldn’t “commercial use” cover this?
Seems like the publishers just want a piece of the money because they want a piece of the money.
Let’s say for example that they trained on the content of the entire archive of the New York Times… isn’t it safe to say they’d have purchased a license for that content from NYT?
Wouldn’t “commercial use” cover this?
Seems like the publishers just want a piece of the money because they want a piece of the money.
OpenAI absolutely does not have licenses for 99%+ of the content used to build their models. They're following the standard tech company model of "negotiate forgiveness rather than ask for permission".
Yep, my understanding is that one of their datasets is basically the ebook dump of z-lib. Honestly, they're likely to get away with training on copyrighted work unless someone can get the LLMs to spit out whole pages of copyrighted material. I don't even think small excerpts would be breaking fair use
This is big enough, though, that we'll likely see new laws made in response, the same way the rise of the internet caused previously ambiguous things to get clear laws around it. So I think they also thought they could force the ambiguity to resolve in their direction with a combination of "facts on the ground" and a big budget for lobbying and litigation.
I remember a thread a while back where someone found out that ChatGPT refused to output the "litany against fear" from Dune.
https://news.ycombinator.com/item?id=36374429
https://news.ycombinator.com/item?id=36374429
Exactly. When you have enough money, you think laws are for other people, as with Uber or cryptocurrency.
The legal status of copyright in this context is unclear, because the laws were written without imitation machines in mind. Presumably they blew right on by "is this right?" and went for "can we get away with this and pay less in fines than we'd make in profits?" As long as there's a 51% chance of that being the case, they saw it as worth a go.
The legal status of copyright in this context is unclear, because the laws were written without imitation machines in mind. Presumably they blew right on by "is this right?" and went for "can we get away with this and pay less in fines than we'd make in profits?" As long as there's a 51% chance of that being the case, they saw it as worth a go.
> Presumably OpenAI and others (for the most part) are checking the licenses for content they use in training?
Ha.
There is a difficult to describe culture in the ML community, from the top researchers to casual tinkerers... Everything is bleeding edge and moving at blinding speed. Some devs don't even stop to think about things like licensing or easy reproducibility/packaging, because they are too busy moving onto the next thing.
This thread on how a popular repo was unlicensed and violating other licenses for months is a good example: https://github.com/AUTOMATIC1111/stable-diffusion-webui/issu...
OpenAI comes from this culture, even if they are a more commercial company now.
Ha.
There is a difficult to describe culture in the ML community, from the top researchers to casual tinkerers... Everything is bleeding edge and moving at blinding speed. Some devs don't even stop to think about things like licensing or easy reproducibility/packaging, because they are too busy moving onto the next thing.
This thread on how a popular repo was unlicensed and violating other licenses for months is a good example: https://github.com/AUTOMATIC1111/stable-diffusion-webui/issu...
OpenAI comes from this culture, even if they are a more commercial company now.
I don't think that's a safe presumption at all. Getty, e.g., claims to have found their watermark in the output of Stable Diffusion.
More broadly, it's not a settled question of what license you need. The article presents a perspective that LLMs "would turn a Food & Wine review into a simple text recommendation of a bottle of Malbec, without attribution." I doubt that anyone has a license to republish the entire archive of the New York Times without attribution.
At the other extreme, I can "train" myself on newspapers I find discarded on the subway with no license at all. I don't need to cite my sources when I state an opinion based on everything I've ever read, seen, or heard. Is that a fair analogy to apply to LLMs?
More broadly, it's not a settled question of what license you need. The article presents a perspective that LLMs "would turn a Food & Wine review into a simple text recommendation of a bottle of Malbec, without attribution." I doubt that anyone has a license to republish the entire archive of the New York Times without attribution.
At the other extreme, I can "train" myself on newspapers I find discarded on the subway with no license at all. I don't need to cite my sources when I state an opinion based on everything I've ever read, seen, or heard. Is that a fair analogy to apply to LLMs?
> At the other extreme, I can "train" myself on...
Well, how much harm would you, a single person who also needs to earn a living and probably wishes to write your own original content, be for the original author?
Compare that to a datacenter full of servers ready to output millions of articles per minute in the exact style of the original author for a per-article cost close to zero.
Well, how much harm would you, a single person who also needs to earn a living and probably wishes to write your own original content, be for the original author?
Compare that to a datacenter full of servers ready to output millions of articles per minute in the exact style of the original author for a per-article cost close to zero.
> Wouldn’t “commercial use” cover this?
The problem is that companies see that they undersold their content relative to the value it’s providing LLMs, and now want to renegotiate past and future deals with AI companies. Exactly as you said: they just want a bigger slice of pie.
The problem is that companies see that they undersold their content relative to the value it’s providing LLMs, and now want to renegotiate past and future deals with AI companies. Exactly as you said: they just want a bigger slice of pie.
How big is the pie? How much money are the AI companies actually making?
>Seems like the publishers just want a piece of the money because they want a piece of the money.
Sure, but of course, we all know the actual value came from the writers.
Sure, but of course, we all know the actual value came from the writers.
And the writers were already compensated for their work.
Oh were they? So they get paid once and the LLM trainer gets paid unlimited times?
Ah, publisher rent seeking. To get independent again. Like they were in the haydays of the cold War or the Iraq war.not.
Why not a different rent seeking model.. For pharma companies? If your product cures a patient, all other pharma companies own you a percentage from everything sold ever after.
Why not a different rent seeking model.. For pharma companies? If your product cures a patient, all other pharma companies own you a percentage from everything sold ever after.
Back when it was OpenAI, I easily could side with the open source software.
Now that one company closed it off, is trying to regulate things, and is profiting from it: Let the lawyers win with fees.
Now that one company closed it off, is trying to regulate things, and is profiting from it: Let the lawyers win with fees.
Well, "open"ai opened pandora's box. Let the games begin.
Easy, humorous fix for the problem: https://www.smbc-comics.com/comic/copyright
Big media companies go fuck yourself.
Nobody should be allowed to bend the law and extort money from society for writing wine reviews.
That nightmare scenario, for Levin,
would turn a Food & Wine review into
a simple text recommendation of a
bottle of Malbec.
Nobody should be allowed to bend the law and extort money from society for writing wine reviews.
AI is not my “brother” nor is it a human. This is a predictive model controlled by a for profit business and yea, they should be paying out if they are training on data owned by someone.
>they should be paying out if they are training on data owned by someone
Virtually everything you "know" is because of current and past humans. Have you been paying everyone for all of that?
If you want to be a great author and you read books by great writers.. then become a successful writer, are you paying all those authors for training you?
I realize there is a difference with being able to absorb bulk data and all that- however, at the core the argument is still the same.
Virtually everything you "know" is because of current and past humans. Have you been paying everyone for all of that?
If you want to be a great author and you read books by great writers.. then become a successful writer, are you paying all those authors for training you?
I realize there is a difference with being able to absorb bulk data and all that- however, at the core the argument is still the same.
But you do pay for all of that in todays society.
Teachers are paid, tuition to schools is paid, feedback from tutors is paid, textbooks are paid for, copies of books are paid for, movies and TV are paid for. The experience you learn on the job is paid for via yours or your coworker’s salary.
I get the argument that it’s not copying it’s learning, but it’s also very different than a human learning by observation.
And if learning weights is okay, is it okay to train an AI with the structure designed to reproduce Shrek, then sell the AI? I think ShrekAI would be illegal in most countries. It’s a blurry question that we need to think about.
Teachers are paid, tuition to schools is paid, feedback from tutors is paid, textbooks are paid for, copies of books are paid for, movies and TV are paid for. The experience you learn on the job is paid for via yours or your coworker’s salary.
I get the argument that it’s not copying it’s learning, but it’s also very different than a human learning by observation.
And if learning weights is okay, is it okay to train an AI with the structure designed to reproduce Shrek, then sell the AI? I think ShrekAI would be illegal in most countries. It’s a blurry question that we need to think about.
The training is not the issue. It’s the reproduction. I expect long term the models will still be trained on copyrighted material, but it will refuse to recite any of it.
It is not the same.
One way to tell is that we think protecting human creators is so important that the US Constitution says Congress has the power to "To promote the Progress of Science and useful Arts, by securing for limited Times to Authors and Inventors the exclusive Right to their respective Writings and Discoveries".
So yes, we absolutely try to pay all those authors. And we also have a variety of cultural mechanisms to reward them in other ways, from honorary doctorates to literary prizes to genius grants to homages. Not to mention a widespread hatred of plagiarism that goes well beyond what copyright covers.
None of that is respected by the current crop of text generators. They chose to release these things knowing of these lacks. And that lack of respect is being richly returned, as it should be.
One way to tell is that we think protecting human creators is so important that the US Constitution says Congress has the power to "To promote the Progress of Science and useful Arts, by securing for limited Times to Authors and Inventors the exclusive Right to their respective Writings and Discoveries".
So yes, we absolutely try to pay all those authors. And we also have a variety of cultural mechanisms to reward them in other ways, from honorary doctorates to literary prizes to genius grants to homages. Not to mention a widespread hatred of plagiarism that goes well beyond what copyright covers.
None of that is respected by the current crop of text generators. They chose to release these things knowing of these lacks. And that lack of respect is being richly returned, as it should be.
uh yes? Did you pay for textbooks in school?
Do you regularly copy protected software and claim it as your own creation?
I’ve seen this argument every time someone states that these for profit ai companies should not have complete access to a all of human knowledge and I just don’t see how we’re living in the same world.
Do you regularly copy protected software and claim it as your own creation?
I’ve seen this argument every time someone states that these for profit ai companies should not have complete access to a all of human knowledge and I just don’t see how we’re living in the same world.
You don't own data. You can copyright data, maybe.
https://www.americanbar.org/groups/science_technology/public...
https://www.americanbar.org/groups/science_technology/public...
It is not a human but it is a fellow intelligent system.
Mankind had this discussion before. When Darwin published his theory about evolution. He faced a lot of hatred because it made humans less special.
Now we go through the same dance again. This time, intelligence in silicon makes humans less special.
Mankind had this discussion before. When Darwin published his theory about evolution. He faced a lot of hatred because it made humans less special.
Now we go through the same dance again. This time, intelligence in silicon makes humans less special.
It is not intelligent, any more than découpé [1] means scissors and paste are intelligent.
I understand that people like to anthropomorphize things. That's harmless fun when people are imagining tree spirits or thunder gods or whatever. But just because fancy autocomplete produces semi-intelligible text does not mean there is an intelligence behind it. I believe we will eventually be able to create synthetic minds. But our understanding of actual minds is so poor that it's going to be quite a while before we manage.
More than 200 years ago, Mary Shelley told a story of a scientist creating life via the then-new technology of electricity. Since the we keep re-telling the story with the latest technology. It's a good story, and like all good art helps us understand what it means to be human. Which is why I think it's especially bad to use those stories to muddy the many differences between actual humans and something that produces statistically plausible generated text.
[1] https://en.wikipedia.org/wiki/Cut-up_technique
I understand that people like to anthropomorphize things. That's harmless fun when people are imagining tree spirits or thunder gods or whatever. But just because fancy autocomplete produces semi-intelligible text does not mean there is an intelligence behind it. I believe we will eventually be able to create synthetic minds. But our understanding of actual minds is so poor that it's going to be quite a while before we manage.
More than 200 years ago, Mary Shelley told a story of a scientist creating life via the then-new technology of electricity. Since the we keep re-telling the story with the latest technology. It's a good story, and like all good art helps us understand what it means to be human. Which is why I think it's especially bad to use those stories to muddy the many differences between actual humans and something that produces statistically plausible generated text.
[1] https://en.wikipedia.org/wiki/Cut-up_technique
What makes your reply different from a reply ChatGPT would have written?
If nothing - where is that intelligence you are talking about? That intelligence that only humans have?
If nothing - where is that intelligence you are talking about? That intelligence that only humans have?
If you can't tell the difference, I think that says more about you than it does about the text. And I'm definitely not going to try to bring you up to date on centuries of the humanities, or even the decades of cognitive science research. If you'd like to know, perhaps start with a book or two? "The Mind's I", "Kinds of Minds", and "Architects of Intelligence" are accessible places to start.
That is not correct. LLMs have no inventory of their input datasets. If you give it a billion reviews recommending Malbec, it does not know that many reviews recommend Malbec. It simply has a very good comprehension of what a Malbec review looks like.
In order to know that many reviews recommend Malbec, it needs to be taught this fact specifically, or infer it from context.
In order to know that many reviews recommend Malbec, it needs to be taught this fact specifically, or infer it from context.
Prompt:
Do more Wikipedia pages mention the word "blue" or do more Wikipedia pages mention the word "Accismus"?
ChatGPT:
To determine whether more Wikipedia pages mention the word "blue" or the word "Accismus," we can use a simple search on the Wikipedia website.
Searching for the term "blue" on Wikipedia yields a large number of results, as the color blue is a common topic across various fields. On the other hand, "Accismus" is a relatively obscure term, and it is less likely to be mentioned in a significant number of Wikipedia pages.
Therefore, it is highly probable that more Wikipedia pages mention the word "blue" compared to the word "Accismus."
Do more Wikipedia pages mention the word "blue" or do more Wikipedia pages mention the word "Accismus"?
ChatGPT:
To determine whether more Wikipedia pages mention the word "blue" or the word "Accismus," we can use a simple search on the Wikipedia website.
Searching for the term "blue" on Wikipedia yields a large number of results, as the color blue is a common topic across various fields. On the other hand, "Accismus" is a relatively obscure term, and it is less likely to be mentioned in a significant number of Wikipedia pages.
Therefore, it is highly probable that more Wikipedia pages mention the word "blue" compared to the word "Accismus."
That is not relevant to what I said. The model can have knowledge about its corpus or tooling to ask questions about it.
But the training process is not a library it can draw upon or answer questions about intrinsically.
But the training process is not a library it can draw upon or answer questions about intrinsically.
>Nobody should be allowed to ... extort money from society for writing wine reviews.
So how do you propose to pay for someone to write wine reviews?
So how do you propose to pay for someone to write wine reviews?
"Publishers want billions, not millions, from AI"
But why ask for billions when you can have ... millions?
But why ask for billions when you can have ... millions?
First they came for writers and I didn't say anything because I wasn't a writer.
Hilarious and desperate.
They are in no position to make demands. The current data that has been scraped for AI models is probably good enough to be able to generate synthetic data.
Even if it's not, what are publishers going to do about people scraping their content? Absolutely nothing.
Trying to force this issue feels like a good way to be disintermediated quicker.
They are in no position to make demands. The current data that has been scraped for AI models is probably good enough to be able to generate synthetic data.
Even if it's not, what are publishers going to do about people scraping their content? Absolutely nothing.
Trying to force this issue feels like a good way to be disintermediated quicker.
Two rent seekers fighting each other - will be interesting to see what happens.
Capitalism needs growth. Growth needs constant new demand.
Generating overpriced intangible goods, the only value of which is novelty is almost necessary in capitalism to keep some form of constant demand. The alternative is continuously. burning something, like we do with oil.
I wonder how we are going to keep all that money in circulation, once we do not need a trillion dollar industry of entertainment anymore.
Generating overpriced intangible goods, the only value of which is novelty is almost necessary in capitalism to keep some form of constant demand. The alternative is continuously. burning something, like we do with oil.
I wonder how we are going to keep all that money in circulation, once we do not need a trillion dollar industry of entertainment anymore.
LLMs are a genuinely exciting technology, but I am worried that a part of what they enable will turn out to be a sort of "content laundering." If you train your AI on a bunch of someone elses work, you can make it produce something very simiilar. You can extract whole aspects of an artist or writer's style and replicate their ideas much more easily now, and because it is generated inside this black box you have a sort of deniability when it comes to accusations of copying.