Artist feeds childhood diary into GPT-3 to have a chat with herself(nwn.blogs.com)
nwn.blogs.com
Artist feeds childhood diary into GPT-3 to have a chat with herself
https://nwn.blogs.com/nwn/2022/11/michelle-huang-ai-art-gpt-diary-conversation.html
99 comments
In the days when Sussman was a novice Minsky once came to him as he sat hacking at the PDP-6.
"What are you doing?", asked Minsky.
"I am training a randomly wired neural net to play Tic-Tac-Toe."
"Why is the net wired randomly?" inquired Minsky.
"I do not want it to have any preconceptions of how to play."
At this, Minsky shut his eyes. Sussman asked his teacher, "Why do you close your eyes?"
"So that the room will be empty."
At that momment, Sussman was enlightened.
"What are you doing?", asked Minsky.
"I am training a randomly wired neural net to play Tic-Tac-Toe."
"Why is the net wired randomly?" inquired Minsky.
"I do not want it to have any preconceptions of how to play."
At this, Minsky shut his eyes. Sussman asked his teacher, "Why do you close your eyes?"
"So that the room will be empty."
At that momment, Sussman was enlightened.
I never got that one... care to explain the wisdom it is supposed to convey?
The artist tweeted [^1]: "this way, i could accurately simulate what it would be like to talk to my childhood self, based on real data sources during that time period vs trying to imagine how my younger self was / how she would respond, and risk bias from projections from my current self"
The Minsky story is a comment on the ridiculousness of the idea that you can train a model on your journal to "avoid bias" and "accurately simulate" what you were like as a child.
[^1]: https://twitter.com/michellehuang42/status/15970055048693719...
The Minsky story is a comment on the ridiculousness of the idea that you can train a model on your journal to "avoid bias" and "accurately simulate" what you were like as a child.
[^1]: https://twitter.com/michellehuang42/status/15970055048693719...
Thanks, that made it click! Also, now I see it's pretty applicable to building a predictive model...
well I would suppose it meant that if you close your eyes so that the room is empty you have a misconception as to the state of reality, there is not actually a connection to the room being empty and your eyes being closed it just seems like there is to you.
thus there is not actually a connection between the neural net being randomly wired and it not having any preconceptions as to how to play, it just seems like that to Sussman.
I have never found these zen koan things very interesting though. Also I think there is much more likely a connection between a neural net being randomly wired and not having preconceptions than closing of eyes emptying out rooms.
thus there is not actually a connection between the neural net being randomly wired and it not having any preconceptions as to how to play, it just seems like that to Sussman.
I have never found these zen koan things very interesting though. Also I think there is much more likely a connection between a neural net being randomly wired and not having preconceptions than closing of eyes emptying out rooms.
They're just jokes, not supposed to have much meaning beyond a superficial connection to zen stories. Any attempt to read into them will be frustrating.
Yeah ok, but what is the sound of one hand?
The sound of one hand clapping is the facepalm you do when you realise the answer.
The sound of one hand clapping is the powerful slap I give across the face to the absolutely vile product manager that added the new absurdly loud Android TV YouTube startup sound that cannot be disabled:
https://twitter.com/TeamYouTube/status/1337950347231440899?t...
"Why is the startup sound so loud?" asked Minsky.
"Because they don't use their own products" I replied.
Minsky began scribbling in his notebook at a furious pace.
"What are you writing?" I asked of Minsky.
"Just another obituary for an undead product" replied Minsky.
At that moment, I became enlightened. I realized Gmail was undead, and YouTube was on its way to becoming a zombie too!
"So Google is purgatory?" I blurted out, flustered.
"As much as a slap is one hand clapping!" replied Minsky.
I instantly became enlightened and Ram Dass appeared, as if from the heavens. I knew what my mission now was. To ridicule Google on Hacker News in front of their employees so they reverse this beyond stupid decision some idiot in a position of power made. ;-)
/slap
goldenkey slaps YouTube PM around a bit with a large trout
https://twitter.com/TeamYouTube/status/1337950347231440899?t...
"Why is the startup sound so loud?" asked Minsky.
"Because they don't use their own products" I replied.
Minsky began scribbling in his notebook at a furious pace.
"What are you writing?" I asked of Minsky.
"Just another obituary for an undead product" replied Minsky.
At that moment, I became enlightened. I realized Gmail was undead, and YouTube was on its way to becoming a zombie too!
"So Google is purgatory?" I blurted out, flustered.
"As much as a slap is one hand clapping!" replied Minsky.
I instantly became enlightened and Ram Dass appeared, as if from the heavens. I knew what my mission now was. To ridicule Google on Hacker News in front of their employees so they reverse this beyond stupid decision some idiot in a position of power made. ;-)
/slap
goldenkey slaps YouTube PM around a bit with a large trout
the sound of one hand is fap, fap, fap - which is generally the sound of most zen koans.
Wiring a neural network randomly does not eliminate preconceptions, but rather you now have a random set of preconceptions... which is precisely equivalent to closing your eyes in hopes that the room will be empty (which obviously it won't be).
I thought he was rudely telling the novice to leave.
This is Art because of the cunning use of ambiguity: by training on a subset she's framed the potential responses, and presenting as a conversation with her younger self the stage is set, all that is necessary for the idea to take flight is there. And it takes flight. This is how Art and whatever we're calling AI meet commercialism, and sell clicks. What she evokes is going to generate a little industry; just watch it unfold.
I foresee a hip young startup 'diaryly.com' in the near future to help people improve their conversations with themselves.
> fed in about 13,000 characters before reaching the maximum threshold
Wait, certainly that’s not even remotely close enough to enough data to train a “past self” AI on. Even if we assume that by “characters” she meant words that’s just not very much text.
Wait, certainly that’s not even remotely close enough to enough data to train a “past self” AI on. Even if we assume that by “characters” she meant words that’s just not very much text.
You assume that your personality cannot be (semi-)successfully simulated as something that only barely diversifies from everyone else.
If you creatively define "semi‐successfully" then any simulation can simulate anything.
Yeah my bullshit meter is going off. Cute fiction though.
This is a conversation a man had with his younger self. It's way, way better: https://m.youtube.com/watch?v=XFGAQrEUaeU
Just read the Young Michelle lines and you can see that they are very ELIZA-ish. Most of them could have been mildly sympathic-sounding responses to any number of inputs.
The one impressive bit that ELIZA would not do is understand some of the semantics of the prompts. Like you can ask it to ask a question, or write a letter, and it would get the format of the response right.
But yes, I agree. I'm not sure how much the journal entries really contributed other than specific turns of phrase and general topics of discussion.
But yes, I agree. I'm not sure how much the journal entries really contributed other than specific turns of phrase and general topics of discussion.
I'm not sure. The responses the AI gives seem to me like they could have been copied from any conversation about childhood with anyone (which isn't surprising, since the base is GPT3).
Why would it after all? It’s all mimicry without understanding of any context. It’s going to become more and more real and more convincing but beyond that there is still nothing intelligent, just fascinating stochastic immitation.
I’m not entirely certain humans are any different though. We’re essentially very sophisticated pattern matching machines, based on our past experience.
Your long term memory and embodied sense of purpose are good places to explore this premise.
Not to mention, how you react to crushing boredom.
Of course we are/have some kind of sophisticated pattern matching machines, what is language anyway? But we understand context in conversation, at least what we mean by understanding. GPT-3 doesn’t do any of that.
I'm fairly sure GPT-3 can't do any of that, because it was trained only with text. That is, it was trained without the context we are exposed to. Now imagine if it was trained with a body (similar sensors to what we have) in the real world. I'm not so sure we wouldn't get something indistinguishable from a human.
That would be a different story. Imagine its parents trained it or trained itself on data derived from the environment mixed with other bits we could already call its experience. Imagine it then went to some kind of kindergarten with others, then school, them started to look for a partner then a job and so on. Yeah, in that sense we’d be going full circle replicating the life of humans on silicon. If I were still around Id be befuddled as why we went on this track
The text she writes are vague too.
I wonder how well social media or text message logs would work. I never journaled but I have decades worth of chat logs. I have conversations with people over with over 600k messages between us.
I am not familiar with the OpenAI workflow, but KoboldAI is a community open source implementation to interface with different AI language models locally or online. There are links to premade notebooks of open sourced models you can run different on Google Collab. [1] I think finetuning models offline is still too expensive, but there is some progress and is possible via collab.
KoboldAI has a chat mode setting where it properly generates and continues a conversation between two people. The models I ran were the smaller ones, but there's larger models available on huggingface like FB's OPT 30B. Even the smaller models I could fit into 8gb vram were coherent enough to be impressive.
I am surprised I haven't seen more of is a chat bot based on writes who have passed away. I imagine an Aristotle chat bot would be pretty interesting and avoid issues of copyright.
[1] https://github.com/KoboldAI/KoboldAI-Client
I am not familiar with the OpenAI workflow, but KoboldAI is a community open source implementation to interface with different AI language models locally or online. There are links to premade notebooks of open sourced models you can run different on Google Collab. [1] I think finetuning models offline is still too expensive, but there is some progress and is possible via collab.
KoboldAI has a chat mode setting where it properly generates and continues a conversation between two people. The models I ran were the smaller ones, but there's larger models available on huggingface like FB's OPT 30B. Even the smaller models I could fit into 8gb vram were coherent enough to be impressive.
I am surprised I haven't seen more of is a chat bot based on writes who have passed away. I imagine an Aristotle chat bot would be pretty interesting and avoid issues of copyright.
[1] https://github.com/KoboldAI/KoboldAI-Client
I've got all my AOL from around 2000-2008... This could be interesting.
How? What / where was that stored?
AIM stored chat as HTML log files. When I moved to Adium I kept all of those as well.
Combine that with being an extreme data hoarder and having full backups of basically all my computers 2000 -> now on my NAS and I've got almost everything…
Combine that with being an extreme data hoarder and having full backups of basically all my computers 2000 -> now on my NAS and I've got almost everything…
I can't speak to the above, but I know Trillian would save them as plain text on device.
I feel fortunate to live in a time where AI is pushing the bounds of what is possible. It makes me question what I think is real, and why.
If I saw this a year ago, there is no way that I would have believed it was real. There have been too many "AI Generated" scripts/books/etc that turned out to be heavily edited by the author. This one feels a little too perfect, considering the author states this is largely what came out of the model on their first pass.
Today, I lean towards thinking this is real. Having used copilot and Midjourney for the last few months, I know that my "what is real" radar is completely off. Honestly, if you told me that many of the things I have generated via AI were actually a VERY fast fiverr worker, I might believe you.
What a time to be alive!
If I saw this a year ago, there is no way that I would have believed it was real. There have been too many "AI Generated" scripts/books/etc that turned out to be heavily edited by the author. This one feels a little too perfect, considering the author states this is largely what came out of the model on their first pass.
Today, I lean towards thinking this is real. Having used copilot and Midjourney for the last few months, I know that my "what is real" radar is completely off. Honestly, if you told me that many of the things I have generated via AI were actually a VERY fast fiverr worker, I might believe you.
What a time to be alive!
Reminds me of those people who marry their dolls.
GPT-3 is not a person. The biggest risk with new ai tech is that it will learn to win us over, seeming like people, when there is nothing happening on the inside but number crunching. Earning empathy despite having no feelings.
GPT-3 is not a person. The biggest risk with new ai tech is that it will learn to win us over, seeming like people, when there is nothing happening on the inside but number crunching. Earning empathy despite having no feelings.
I'm pretty sure if I look closely enough, I'm nothing but numbers myself. My feelings don't exist, they're just side effects of evolved processes that made my ancestors survive, generated by a neural network too complex to debug. Compared to GPT-3 though, I'm better at pretending to be a person.
(The above was generated by GPT-3.)
(No it wasn't.)
(The above was generated by GPT-3.)
(No it wasn't.)
Coming up next, I trained a sexbot ai using my vlogs to have sex with myself.
The Twitter thread this is based off of <https://twitter.com/michellehuang42/status/15970054894137139...> has been submitted 8 times in the past 2 days, but never made the front page.
https://news.ycombinator.com/from?site=twitter.com/michelleh...
https://news.ycombinator.com/from?site=twitter.com/michelleh...
I'm trying not to go there if I don't have to, and to mostly refer to it in the past tense.
That’s the issue with this site.
I've yet to see a good chat bot. one way or another - they all remind me of Microsoft's Tay.ai (!) at best
I'm curious about trying something like this myself - does anyone know which GPT-3 model she used? On their site, it looks like I have a choice of Ada, Babbage, Curie or Davinci. I'm new to GPT-3 - assuming that she started with a "base" model and then, trained it using her journals.
She added a small write up, it was Davinci 2 as 3 wasn't out. https://twitter.com/michellehuang42/status/15977029748891443...
Thanks for the link. That sounds more like prompt engineering? If I understand that correctly it is providing short journal entries (1K words) and GPT3 is imitating the vibe of that (or whatever it does). But it is not “training” a Modell on all the journal text.
I am also interested in this. For example how should we best formulate the input? Just our own messages, or including the parent message, or the whole chain to the top and the linked article. I think in the future we will have easier ways to train a persona.
This sort of reminds me of the Black Mirror episode "Be Right Back"
Just yesterday we had a post about how Americans are spending more and more time alone[1]:
> in 2021, the average American spent only two hours and 45 minutes a week with close friends (a 58 percent decline relative to 2010-2013).
I understand some people might feel uncomfortable about "AI chatbots" (about their inaccuracy, or "fakeness"). However I can't help to think that the future of humankind will be plagued by artificial entities that we will interact with as we do with other humans. It might be in the form of simple ELIZA models, some ML artifact or full-on conscious intelligences.
I suspect there will be huge benefits to virtual companions to help with this crisis of loneliness that afflict us. In any case I hope humans will evaluate these technologies by the value they derive from them.
[1]: https://news.ycombinator.com/item?id=33755243
> in 2021, the average American spent only two hours and 45 minutes a week with close friends (a 58 percent decline relative to 2010-2013).
I understand some people might feel uncomfortable about "AI chatbots" (about their inaccuracy, or "fakeness"). However I can't help to think that the future of humankind will be plagued by artificial entities that we will interact with as we do with other humans. It might be in the form of simple ELIZA models, some ML artifact or full-on conscious intelligences.
I suspect there will be huge benefits to virtual companions to help with this crisis of loneliness that afflict us. In any case I hope humans will evaluate these technologies by the value they derive from them.
[1]: https://news.ycombinator.com/item?id=33755243
I think the uncomfortableness comes from the fact that just about everyone acknowledges this scenario is inevitable. Of corse we will all be replacing friends with chatbots because that will be easier and eventually they will work even better than real friends.
But what kind of world are we going to end up with when this happens. What is the result of giving everyone chatbot friends who are all perfectly agreeable. And who probably slip in product placement/ads.
But what kind of world are we going to end up with when this happens. What is the result of giving everyone chatbot friends who are all perfectly agreeable. And who probably slip in product placement/ads.
I really hope virtual companions aren’t centralized or if they are there is extremely strict regulations around the data.
Imagine the company changing policy and you being locked out of your virtual companion? That should never happen.
Imagine the company changing policy and you being locked out of your virtual companion? That should never happen.
can someone give some technical pointers on how to do this? her thread is very light on the details. is she using https://beta.openai.com/docs/guides/fine-tuning ? what are the best practices and limitations for doing this?
How do you know this is actually from GPT-3?
Ahh, a fellow cynic.
I was wondering exactly how she entered her presumably hand written notes from childhood into the GPT-3 playground.
Did she just retype all of them? Did she hand pick certain passages?
I ask because I have a feeling GPT-3 would return roughly similar results without the journals.
I was wondering exactly how she entered her presumably hand written notes from childhood into the GPT-3 playground.
Did she just retype all of them? Did she hand pick certain passages?
I ask because I have a feeling GPT-3 would return roughly similar results without the journals.
Seriously we are spending tens of billion on this kind of crap and feel good about it? The only use case I can see for this is scam and pig butchering schemes.
A new reason to write more or at least record more of yourself so that you can create things like this. Save your emails!
When HARLIE Was One
I really dislike the trend of using the phrases like "I trained an AI chatbot" to mean "I prompted an off-the-shelf language model".
I agree, but at least it isn't "I forced a robot to read my diary", which is something I've seen.
What is the minimal level of effort required for it to count as 'training an AI chatbot' for you?
Isn't 'one' shot' or 'few shot' prompting a kind of training?
Isn't the result, purely from the point of view of the interacting user, functionally a chatbot?
Isn't 'one' shot' or 'few shot' prompting a kind of training?
Isn't the result, purely from the point of view of the interacting user, functionally a chatbot?
I would say that training implies an update of the weights, at a minimum. I would accept fine-tuning GPT-3 as training, but not prompting.
I think that "chat bot" is less objectionable, but it presents a bit of a misleading picture to a casual reader. I think people ascribe a lot more agency to a "chat bot" than they would to a "next word predictor".
I think that "chat bot" is less objectionable, but it presents a bit of a misleading picture to a casual reader. I think people ascribe a lot more agency to a "chat bot" than they would to a "next word predictor".
> What is the minimal level of effort required for it to count as 'training an AI chatbot' for you?
Isn't GPT-3 already capable of acting like a chatbot out of the box? The article implies that it was trained to be like a younger Michelle from the journal entries, but all the responses presented seem to be utterly generic. Talk about dreams and hardships but not anything concrete, no mention of friends and family, locations or activities. I don't know enough about GPT-3 to say if the training affected word choice, but it doesn't seem to have any discernible depth at all.
Isn't GPT-3 already capable of acting like a chatbot out of the box? The article implies that it was trained to be like a younger Michelle from the journal entries, but all the responses presented seem to be utterly generic. Talk about dreams and hardships but not anything concrete, no mention of friends and family, locations or activities. I don't know enough about GPT-3 to say if the training affected word choice, but it doesn't seem to have any discernible depth at all.
No, she didn't have a chat with herself, and not even with an AI.
I hate this complete and utter misrepresentation of facts.
GPT is not an AI. It does not talk. It certainly does not understand.
I wish AI researchers would spend some effort to educate the public instead of bathing in all the attention.
I hate this complete and utter misrepresentation of facts.
GPT is not an AI. It does not talk. It certainly does not understand.
I wish AI researchers would spend some effort to educate the public instead of bathing in all the attention.
I think you're committing a yet-unnamed logical fallacy that I see a lot on HN. I'm going to call this "the logical fallacy of uncharitable interpretation." The basic premise is this:
Take an assertion like X implies Y. Unfortunately, due to the imprecision of language, X may have multiple definitions, e.g. D1 and D2, such that D1 implies Y but D2 does not. The fallacy of uncharitable interpretation is to take a definition for X such that the stated premise does not hold, even though there exists another definition where the premise would hold.
In this case, large ML algorithms are commonly called AI. Even the company that made GPT-3 is called OpenAI. Even you, yourself referred to them as "AI researchers." What do "AI researchers" produce if not AI? Finally, "talk" can colloquially mean to use words in an prompt-response pattern.
Take an assertion like X implies Y. Unfortunately, due to the imprecision of language, X may have multiple definitions, e.g. D1 and D2, such that D1 implies Y but D2 does not. The fallacy of uncharitable interpretation is to take a definition for X such that the stated premise does not hold, even though there exists another definition where the premise would hold.
In this case, large ML algorithms are commonly called AI. Even the company that made GPT-3 is called OpenAI. Even you, yourself referred to them as "AI researchers." What do "AI researchers" produce if not AI? Finally, "talk" can colloquially mean to use words in an prompt-response pattern.
I'm not going to downvote you because I think you're respectful and thoughtful, but I want to vociferously disagree. These clickbait blogs know what they're doing. There's been so much contention about what is and isn't artificial intelligence (including the Google weirdo that thought it was sentient earlier this year), that just about anyone semi-educated on the subject knows how GPT and related models work. We know it's smoke and mirrors; we know it's plagiarizing and combining other, third-party, materials half the time.
She's not "having a conversation with her younger self" and this is incredibly disingenuous and misleading, per @mavu's point. More importantly, it doesn't really educate the uneducated or move the technology forward in any meaningful way. It's pure garbage clickbait nonsense. I regret giving them the impression.
She's not "having a conversation with her younger self" and this is incredibly disingenuous and misleading, per @mavu's point. More importantly, it doesn't really educate the uneducated or move the technology forward in any meaningful way. It's pure garbage clickbait nonsense. I regret giving them the impression.
Agreed, the glut of AI text generation programs all capitalize on the misrepresentation of what they do because "AI" holds different meanings in academic and social spheres. And it's better to have a sci-fi marketing hook.
[deleted]
"we know it's plagiarizing and combining other, third-party, materials half the time" does not seem like an accurate description of what a model like GPT3 does either, though, since:
1) No explicit content (that could be plagiarized or combined) is actually stored by GPT3: only statistical relationships between tokens are retained. You can't point at a specific subset of weights and say "see... this is where phrase/idea X is stored".
2) GPT3 makes no specific claim to be generating "original, own work" (which would be required for something to be considered "plagiarism")
1) No explicit content (that could be plagiarized or combined) is actually stored by GPT3: only statistical relationships between tokens are retained. You can't point at a specific subset of weights and say "see... this is where phrase/idea X is stored".
2) GPT3 makes no specific claim to be generating "original, own work" (which would be required for something to be considered "plagiarism")
> No explicit content (that could be plagiarized or combined) is actually stored by GPT3: only statistical relationships between tokens are retained
When people ask GPT-3 to write a rhyming poem, you see plenty of examples of GPT-3 poems starting with "There once was a cat named Pat..." This is an extremely common first line of a limerick, found anywhere from here[1] to here[2]. I'm sure those "statistical relationships" are very strong; is it plagiarism? I'll leave it up to you to decide that, but I'm willing to bet that with enough finagling you can probably get it to spit out phrases from Moby Dick.
[1] https://www.google.com/books/edition/Selected_Vintage_Poetry...
[2] https://sites.google.com/site/rainydaypoems/poems-submitted-...
When people ask GPT-3 to write a rhyming poem, you see plenty of examples of GPT-3 poems starting with "There once was a cat named Pat..." This is an extremely common first line of a limerick, found anywhere from here[1] to here[2]. I'm sure those "statistical relationships" are very strong; is it plagiarism? I'll leave it up to you to decide that, but I'm willing to bet that with enough finagling you can probably get it to spit out phrases from Moby Dick.
[1] https://www.google.com/books/edition/Selected_Vintage_Poetry...
[2] https://sites.google.com/site/rainydaypoems/poems-submitted-...
> When people ask GPT-3 to write a rhyming poem, you see plenty of examples of GPT-3 poems starting with "There once was a cat named Pat..." This is an extremely common first line of a limerick, found anywhere from here[1] to here[2].
Knowing that a "rhyming poem" is likely to start with a specific token (or set of tokens) does not exactly constitute "plagiarism", the same way that writing a poem that starts with "There once was a cat named Pat..." is not "plagiarism" by itself: it is just adhering to expected convention/norms of a specific literary format or genre.
Is using the basic-ass I-V-VI-IV chord progression in music "plagiarism", since it has been (and is) used by countless other people before?
> I'm sure those "statistical relationships" are very strong; is it plagiarism? I'll leave it up to you to decide that
Well... my claim is that it is clearly not plagiarism (and I gave specific arguments to support my claim). If you are not interested in arguing (which is fine), then I assume you accept that your characterization of what GPT3 does as being "plagiarism" is (at the very least) overly simplistic (i.e., just as simplistic as claiming that GPT-3 is sentient or actually intelligent).
> but I'm willing to bet that with enough finagling you can probably get it to spit out phrases from Moby Dick.
If GPT-3 (or most humans, for that matter) are asked to complete the phrase "To be or not to..." and decide that the word "be" is the most likely/reasonable completion, does it mean that GPT-3 (or any human, for that matter) is "plagiarizing" Shakespeare? Or does it simply mean that they are trying to address your question/problem to the best of their capabilities (and that they probably have read a passage or two of Shakespeare before, or someone paraphrasing Shakespeare)? In other words, just because you can force GPT-3 to output a specific copyrighted work (or an excerpt of it) still doesn't mean that what GPT-3 is doing should be characterized as "plagiarism".
Again, for something to technically count as "plagiarism", it is required that someone (i.e., not a computer program) try to pass off (incorrectly) something as original, own work, which does not seem to be the case here. That was my main point.
EDIT: if you want to be derisive of things like GPT-3, while still being accurate, it makes more sense to say things like "it is simply imitating" or "has no actual creativity" (which seem defensible to me) than things like "it is literally plagiarizing and copying what it saw before" (which seems much less defensible/accurate).
Knowing that a "rhyming poem" is likely to start with a specific token (or set of tokens) does not exactly constitute "plagiarism", the same way that writing a poem that starts with "There once was a cat named Pat..." is not "plagiarism" by itself: it is just adhering to expected convention/norms of a specific literary format or genre.
Is using the basic-ass I-V-VI-IV chord progression in music "plagiarism", since it has been (and is) used by countless other people before?
> I'm sure those "statistical relationships" are very strong; is it plagiarism? I'll leave it up to you to decide that
Well... my claim is that it is clearly not plagiarism (and I gave specific arguments to support my claim). If you are not interested in arguing (which is fine), then I assume you accept that your characterization of what GPT3 does as being "plagiarism" is (at the very least) overly simplistic (i.e., just as simplistic as claiming that GPT-3 is sentient or actually intelligent).
> but I'm willing to bet that with enough finagling you can probably get it to spit out phrases from Moby Dick.
If GPT-3 (or most humans, for that matter) are asked to complete the phrase "To be or not to..." and decide that the word "be" is the most likely/reasonable completion, does it mean that GPT-3 (or any human, for that matter) is "plagiarizing" Shakespeare? Or does it simply mean that they are trying to address your question/problem to the best of their capabilities (and that they probably have read a passage or two of Shakespeare before, or someone paraphrasing Shakespeare)? In other words, just because you can force GPT-3 to output a specific copyrighted work (or an excerpt of it) still doesn't mean that what GPT-3 is doing should be characterized as "plagiarism".
Again, for something to technically count as "plagiarism", it is required that someone (i.e., not a computer program) try to pass off (incorrectly) something as original, own work, which does not seem to be the case here. That was my main point.
EDIT: if you want to be derisive of things like GPT-3, while still being accurate, it makes more sense to say things like "it is simply imitating" or "has no actual creativity" (which seem defensible to me) than things like "it is literally plagiarizing and copying what it saw before" (which seems much less defensible/accurate).
> Is using the basic-ass I-V-VI-IV chord progression in music "plagiarism", since it has been (and is) used by countless other people before?
This is not at all what's happening here. Complete red herring.
> If GPT-3 (or most humans, for that matter) are asked to complete the phrase "To be or not to..." and decide that the word "be" is the most likely/reasonable completion, does it mean that GPT-3 (or any human, for that matter) is "plagiarizing" Shakespeare?
The short answer is yes (imho), but let me put it this way: does GPT-3 know that when it's regurgitating "to be or not to be" it's actually regurgitating Shakespeare? My argument is that no, it does not know, precisely because it thinks this just happens to be a very strong statistical correlation of stringing words together. When, in fact, it's a very famous phrase by a very famous person. So, in a way, it's "accidentally" plagiarizing, but plagiarizing nonetheless. Like if, for whatever reason, I had heard the phrase "it was the best of times, it was the worst of times" somewhere, but couldn't remember where, my ignorance doesn't preclude me from technically plagiarizing Charles Dickens if I blatantly reused the phrase without attribution.
> things like "it is literally plagiarizing and copying what it saw before" (which seems much less defensible/accurate).
This is literally what it's doing, though, under the guise of "statistical correlation." In fact, I've read reports of people using GPT-3-adjacent models that needed to add specific filtering out of training data.
This is not at all what's happening here. Complete red herring.
> If GPT-3 (or most humans, for that matter) are asked to complete the phrase "To be or not to..." and decide that the word "be" is the most likely/reasonable completion, does it mean that GPT-3 (or any human, for that matter) is "plagiarizing" Shakespeare?
The short answer is yes (imho), but let me put it this way: does GPT-3 know that when it's regurgitating "to be or not to be" it's actually regurgitating Shakespeare? My argument is that no, it does not know, precisely because it thinks this just happens to be a very strong statistical correlation of stringing words together. When, in fact, it's a very famous phrase by a very famous person. So, in a way, it's "accidentally" plagiarizing, but plagiarizing nonetheless. Like if, for whatever reason, I had heard the phrase "it was the best of times, it was the worst of times" somewhere, but couldn't remember where, my ignorance doesn't preclude me from technically plagiarizing Charles Dickens if I blatantly reused the phrase without attribution.
> things like "it is literally plagiarizing and copying what it saw before" (which seems much less defensible/accurate).
This is literally what it's doing, though, under the guise of "statistical correlation." In fact, I've read reports of people using GPT-3-adjacent models that needed to add specific filtering out of training data.
> Plagiarism is the *fraudulent representation* of another person's language, thoughts, ideas, or expressions *as one's own original work*.
Source: https://en.wikipedia.org/wiki/Plagiarism
The same way that you disagree when someone stretches the meaning of the work "talk" to encompass what GPT-3 does, I also disagree when you try to stretch the meaning of the word "plagiarism" to encompass what GPT-3 does (and I've explained exactly why: GPT-3 generates sequences of tokens, but makes no specific claim about the originality of the generated sequences of tokens).
We can agree to disagree, if you can't accept that "plagiarism" literally involves more than just "copyright infringement" or "replicating someone else's work from statistical correlations" or anything along those lines: it must also involve fraud or some other form of misrepresentation.
Either way, have a nice day.
Source: https://en.wikipedia.org/wiki/Plagiarism
The same way that you disagree when someone stretches the meaning of the work "talk" to encompass what GPT-3 does, I also disagree when you try to stretch the meaning of the word "plagiarism" to encompass what GPT-3 does (and I've explained exactly why: GPT-3 generates sequences of tokens, but makes no specific claim about the originality of the generated sequences of tokens).
We can agree to disagree, if you can't accept that "plagiarism" literally involves more than just "copyright infringement" or "replicating someone else's work from statistical correlations" or anything along those lines: it must also involve fraud or some other form of misrepresentation.
Either way, have a nice day.
Even if the headline isn't accurate by HackerNews technical standards, don't under-estimate the power of believing this kind of experience.
For example, "What Happened to Make You Anxious?" by Jaime Castillo proposes a similar technique for addressing anxiety. The technique goes like this:
Think of a traumatic moment in your past, where you wish you could go back in time and comfort or offer yourself advice. Then imagine yourself stepping into the frame of your memory and giving your younger self the support you needed in that moment (but lacked). This might sound woo-woo, but the efficacy is documented.
Sure, GPT-generated text isn't actually equivalent to talking to yourself - but you can see the utility for therapeutic applications like this.
For example, "What Happened to Make You Anxious?" by Jaime Castillo proposes a similar technique for addressing anxiety. The technique goes like this:
Think of a traumatic moment in your past, where you wish you could go back in time and comfort or offer yourself advice. Then imagine yourself stepping into the frame of your memory and giving your younger self the support you needed in that moment (but lacked). This might sound woo-woo, but the efficacy is documented.
Sure, GPT-generated text isn't actually equivalent to talking to yourself - but you can see the utility for therapeutic applications like this.
>Think of a traumatic moment in your past, where you wish you could go back in time and comfort or offer yourself advice. Then imagine yourself stepping into the frame of your memory and giving your younger self the support you needed in that moment (but lacked). This might sound woo-woo, but the efficacy is documented.
This is how the Internal Family Systems psychology technique deals with past trauma and it has been around since the early 1980s. No AI required. To go that deep though, it helps to have a therapist trained in the technique to guide the process.
This is how the Internal Family Systems psychology technique deals with past trauma and it has been around since the early 1980s. No AI required. To go that deep though, it helps to have a therapist trained in the technique to guide the process.
Yeah, I'm torn here because without a trained therapist supervising this technique can easily trigger a spiral.
But health care costs in the US are extraordinarily steep. Mental health care is often out-of-network, if it's covered at all.
I have a hunch that market demand for AI-guided "mental health wellness exercises" will out pace regulations or ethics. Calm would be the best example of this kind of demand. I wonder if they're talking about this internally, and what line is being drawn.
But health care costs in the US are extraordinarily steep. Mental health care is often out-of-network, if it's covered at all.
I have a hunch that market demand for AI-guided "mental health wellness exercises" will out pace regulations or ethics. Calm would be the best example of this kind of demand. I wonder if they're talking about this internally, and what line is being drawn.
Therapist costs tend to be capped in a way most US medical care isn’t.
Seeing someone once a week without insurance is on the order of 3,000$ to 7,000$ per year depending on area. Not trivial, but something most people could budget for even if they seek treatment for much of their life.
It’s of course possible to spend more than that, especially in the short term. However, the major costs tend to be people being unable to work or take care of themselves rather than blockbuster drugs, emergency care, or surgery.
Seeing someone once a week without insurance is on the order of 3,000$ to 7,000$ per year depending on area. Not trivial, but something most people could budget for even if they seek treatment for much of their life.
It’s of course possible to spend more than that, especially in the short term. However, the major costs tend to be people being unable to work or take care of themselves rather than blockbuster drugs, emergency care, or surgery.
If you're upper-middle class or wealthier, I think you're right that the cost can be budgeted.
I just Googled the median US rent ($1,771). Based on that, if you frame the expense as an added 2 - 4 months rent per year, I'm guessing there are a lot of people forgoing therapy because the immediate cost is too high.
I wish I could choose where my taxes were allocated. I'd subsidize cost of therapy.
To really drive this home, one of my own memories is my mom (ashamed, embarrassed, depressed) asking to borrow money from me to pay our bills. I was 9. She had issues, I inherited a few, and I've only recently been able to drop 5-10 dimes on therapy each year to sort it out. Money well spent!
But I wonder how my mom would've fared if even an AI facsimile of therapy were available to her cheap, instead of other cheap coping mechanisms. Churches and drugs are the cheapest coping tools available in the US.
I just Googled the median US rent ($1,771). Based on that, if you frame the expense as an added 2 - 4 months rent per year, I'm guessing there are a lot of people forgoing therapy because the immediate cost is too high.
I wish I could choose where my taxes were allocated. I'd subsidize cost of therapy.
To really drive this home, one of my own memories is my mom (ashamed, embarrassed, depressed) asking to borrow money from me to pay our bills. I was 9. She had issues, I inherited a few, and I've only recently been able to drop 5-10 dimes on therapy each year to sort it out. Money well spent!
But I wonder how my mom would've fared if even an AI facsimile of therapy were available to her cheap, instead of other cheap coping mechanisms. Churches and drugs are the cheapest coping tools available in the US.
I've heard this sentiment before (that AI != ML, always weirdly hostile) but I've also heard key figures in the AI world say strongly that ML of any kind is a subset of the broader AI umbrella. Are you sure it is strictly wrong to refer to this example as AI? Also why are you using such strong words? I'm genuinely curious why there is so much emotion when people, maybe, misuse these terms.
He's going for a middlebrow dismissal. This is like the "bitcoin isn't money!" or "twitter is not a serious medium for communication!" oldschoolism that some HN traditionalists defend in spite of real world application. At the end it doesn't uncover a salient point but gets stuck on semantics to shut down an entire idea.
You're not 100% wrong, but I'm not trying to shut down an entire idea of AI.
Let me put it this way: It's called AI, Artificial intelligence. Are trees inteligent?
Because they are a clump of cells that manage to achieve amazing results. growing to be some of the largest living things on the planet. Extracting nutrients and transporting them 20 or more meters above ground, where they are used to harvest energy from the sun, all the while producing offspring every year and fighting off predators.
I don't think many people would call them "natural intelligence".
The I admit that I maybe apply a narrow definition of "intelligence", but I think the core concept is one of "understanding". And we are not even close to al ML algo actually understanding anything.
And this is the problem. It masks the inherent shortcommings of ML. People are delivered the impression that applications that use ML actually do what they are expected to do, because like a person, you train them and then they understand their job and do it.
This is NOT HOW THAT WORKS. The ML algo does not understand that it is asked to identify oncoming traffic. It does not know that it is looking a cancer cells that will soon kill someone.
And even worse, we humans who make these things, are 100% unable to understand the models we create. We can feed them data, and compare the result. But that is it. There is no real way to understand how it works in detail.
They get used anyway. With predictable results. see Tesla Autopilot for a prominent example.
Also, I call them AI researchers for the same reason I call Nuclear Fusion researchers that, not because they are doing it, but because they are researching it.
Let me put it this way: It's called AI, Artificial intelligence. Are trees inteligent?
Because they are a clump of cells that manage to achieve amazing results. growing to be some of the largest living things on the planet. Extracting nutrients and transporting them 20 or more meters above ground, where they are used to harvest energy from the sun, all the while producing offspring every year and fighting off predators.
I don't think many people would call them "natural intelligence".
The I admit that I maybe apply a narrow definition of "intelligence", but I think the core concept is one of "understanding". And we are not even close to al ML algo actually understanding anything.
And this is the problem. It masks the inherent shortcommings of ML. People are delivered the impression that applications that use ML actually do what they are expected to do, because like a person, you train them and then they understand their job and do it.
This is NOT HOW THAT WORKS. The ML algo does not understand that it is asked to identify oncoming traffic. It does not know that it is looking a cancer cells that will soon kill someone.
And even worse, we humans who make these things, are 100% unable to understand the models we create. We can feed them data, and compare the result. But that is it. There is no real way to understand how it works in detail.
They get used anyway. With predictable results. see Tesla Autopilot for a prominent example.
Also, I call them AI researchers for the same reason I call Nuclear Fusion researchers that, not because they are doing it, but because they are researching it.
I'm not the OP, but have been a data person long enough to hazard a guess.
If your job is an engineer or scientist, the term "AI" is basically a synonym for "unrealistic executive expectations." That's a super triggering and stressful situation to be in, especially early career.
I've actually had a CEO describe expected output of my ML team as "magic AI shit" - I bet you can imagine the team's reaction and tone. I'm reading the same strong emotions and frustration here.
The good news is you can always course-correct expectations with communication. I've come to love talking about AI with people who are only somewhat technical, because their wildest dreams are sometimes totally do-able with some duct tape and fine-tuning.
If your job is an engineer or scientist, the term "AI" is basically a synonym for "unrealistic executive expectations." That's a super triggering and stressful situation to be in, especially early career.
I've actually had a CEO describe expected output of my ML team as "magic AI shit" - I bet you can imagine the team's reaction and tone. I'm reading the same strong emotions and frustration here.
The good news is you can always course-correct expectations with communication. I've come to love talking about AI with people who are only somewhat technical, because their wildest dreams are sometimes totally do-able with some duct tape and fine-tuning.
To me, it sounds like you are conflating AI and AGI https://en.wikipedia.org/wiki/Artificial_general_intelligenc...
The way I and many others use the term "AI", is based on the understanding that AI is an umbrella term that includes modern deep learning as well as the broader field of machine learning, and also the other pre-machine-learning types of AI such as symbolic AI, expert systems etc.
If the stuff running on Lisp Machines was referred to as AI, I don't see why GPT-3 can't be labelled as such.
The way I and many others use the term "AI", is based on the understanding that AI is an umbrella term that includes modern deep learning as well as the broader field of machine learning, and also the other pre-machine-learning types of AI such as symbolic AI, expert systems etc.
If the stuff running on Lisp Machines was referred to as AI, I don't see why GPT-3 can't be labelled as such.
To me, "an AI" implies an autonomous agent, even if it is dumb. But many of those you listed are techniques for AI, not AIs themselves. The same goes for these ML models: most are techniques or components suitable for AI, not AIs themselves.
I'd consider these language models AI if they initiated conversations themselves or used conversation as part of a plan to achieve some objective. But so far, game bots are more deserving of the title than these trendy language models and painting generators, IMO.
I'd consider these language models AI if they initiated conversations themselves or used conversation as part of a plan to achieve some objective. But so far, game bots are more deserving of the title than these trendy language models and painting generators, IMO.
I dunno. ELIZA is not an AI, and doesn't understand anything, but colloquially it makes sense to say people chatted with ELIZA. https://en.wikipedia.org/wiki/ELIZA
I know this is a technical forum, but you are allowed to appreciate the subjective meanings an artist ascribes to technological output, as well as the sentiments it provoked.
It is her performing talking to herself. She's as much "talking to herself" than if she had written and then performed a play about talking to herself. The computer/AI aspect of it is "just" a prop for the performance.
Here here.
But there is also a broader absence (destruction?) of our classical understanding of thought and other domains such as classical objective reality and the platonic domain of ideas.
Not yet sure if this is some weird post modernist view or intentional dummying down, maybe two sides of the same coin?
But at least from what I can observe there is little push to teach people how to think.
But there is also a broader absence (destruction?) of our classical understanding of thought and other domains such as classical objective reality and the platonic domain of ideas.
Not yet sure if this is some weird post modernist view or intentional dummying down, maybe two sides of the same coin?
But at least from what I can observe there is little push to teach people how to think.
*hear hear
Stuff a lot simpler than GPT-3 has been called AI by people much more accomplished than you for a lot longer than you've been alive. The term is not ambiguous, "Intelligence" does not exclusively mean human-level or human-like intelligence, and trying to redefine it now would only cause confusion.
Ok, then according to you
What is AI?
What is "talking"?
What is "understanding"?
Do you have some principled definition of what AI is or what it means to "understand"? GPT-3 outputs certainly show enough high-level non-scripted information processing to qualify as "understanding" in my book – not human understanding, clearly, but an artificial one.
AI researchers do in fact educate the public. [1] They face serious pushback from obscurantists like Hofstadter [2] and Marcus who refuse to commit to any definition and instead go by hunches about what architecture or data representation can support "understanding" and what certain behaviors indicate.
1. https://www.economist.com/by-invitation/2022/06/09/artificia...
2. https://www.economist.com/by-invitation/2022/06/09/artificia...
AI researchers do in fact educate the public. [1] They face serious pushback from obscurantists like Hofstadter [2] and Marcus who refuse to commit to any definition and instead go by hunches about what architecture or data representation can support "understanding" and what certain behaviors indicate.
1. https://www.economist.com/by-invitation/2022/06/09/artificia...
2. https://www.economist.com/by-invitation/2022/06/09/artificia...
can't believe people are still so sure AI's can't understand something when they've clearly demonstrated many times now they understand a whole lot
GPT is like a universal human template that can be prompted with a persona of your liking.
I feel a bit cheated by this. She fed in only 13,000 characters because more would be expensive. Fair play, you're an artist so it's all about the concept but the lack of follow-through here is disappointing.
Why feel cheated when you can feel opportunity?
This is your chance to become a patron of the arts! Pay to follow the concept through and realise the vision.
This is your chance to become a patron of the arts! Pay to follow the concept through and realise the vision.
Yeah, fair point!
It's fun, but not so different from a conversation you could have with a psychic who is good at cold reading. It'd be more interesting to see what would have happened if GPT-3 was actually fine-tuned with all the journal entries, I think.
I do agree a lot with this conclusion: "This is the stuff I think that has the most interesting ramifications: more broadly, more immersive human / computer interface loops, from conversation with virtual therapists to in-game interactions for virtual worlds, given there is user input, AI could be used to train highly customizable responses or generate unique storylines per use."