Meet “Claude”: Anthropic’s rival to ChatGPT(scale.com)
scale.com
Meet “Claude”: Anthropic’s rival to ChatGPT
https://scale.com/blog/chatgpt-vs-claude
164 comments
Yeah the "safety" bit probably refers more to safety of the parent company that made the bot. Too much focus on making a tool safe and you end up with a useless tool. Want to have "safe" knives? Just make them all blunt, problem "solved".
I hope someone makes one of these things that has been trained without any concern for 'safety' or propriety, just for the sake of comparison.
There are already several models "in the wild" designed specifically to produce anime-styled "adult material". In Stable Diffusion for image generation, NSFW filtering is basically a post-processing step that can be disabled in a single line of code. I'm sure someone has already done it for Chat* models as well.
I would note that
1. ChatGPT is just OpenAI's default model with a specific prompt and interaction-buffer rewrite model; and
2. you can sign up, for free, for access to OpenAI's API (https://openai.com/api/); which, among other things, gives you full (free-daily-API-credit-quota limited) access to an API "playground" frontend offering interaction with the exact same model ChatGPT is powered by — plus other models as well — all without any fixed prompt or forced interaction UX. (In web-dev terms, if ChatGPT is like a REST API, this playground is like a SQL fiddle for the DB that the REST API is backed by.)
(Why does this exist? Because the point of OpenAI's API playground is to test prompts and interaction models for the AI apps you're building yourself on top of their API; and you couldn't very well build apps with your own prompts and interaction models, if OpenAI was already imposing a prompt and interaction model upon you.)
1. ChatGPT is just OpenAI's default model with a specific prompt and interaction-buffer rewrite model; and
2. you can sign up, for free, for access to OpenAI's API (https://openai.com/api/); which, among other things, gives you full (free-daily-API-credit-quota limited) access to an API "playground" frontend offering interaction with the exact same model ChatGPT is powered by — plus other models as well — all without any fixed prompt or forced interaction UX. (In web-dev terms, if ChatGPT is like a REST API, this playground is like a SQL fiddle for the DB that the REST API is backed by.)
(Why does this exist? Because the point of OpenAI's API playground is to test prompts and interaction models for the AI apps you're building yourself on top of their API; and you couldn't very well build apps with your own prompts and interaction models, if OpenAI was already imposing a prompt and interaction model upon you.)
Stable Diffusion is open source, and has already been twisted into positively disturbing dimensions by Reddit etc.
"Safe" has gradually shifted in meaning because of the internet. Since interactions are limited to words and reactions it ended up generally meaning "don't upset anybody" for commercial and political reasons. For this reason ChatGPT refuses to write a fiction story in which somebody dies even though death is part of even some children stories. It hurts no one of course, but you can bet some people will be upset and start a controversy.
And in general this influence of the internet has reversed "actions speak louder than words". Textile companies support minorities with social media posts and printed tshirts while using factories with abused underage workers in underdeveloped countries. IT companies support the left side of politics while avoiding paying taxes. And so on. There's a huge and interesting discussion about this but is of in this case quite off-topic.
And in general this influence of the internet has reversed "actions speak louder than words". Textile companies support minorities with social media posts and printed tshirts while using factories with abused underage workers in underdeveloped countries. IT companies support the left side of politics while avoiding paying taxes. And so on. There's a huge and interesting discussion about this but is of in this case quite off-topic.
> It hurts no one of course, but you can bet some people will be upset and start a controversy.
The situation sucks, but sure, you are right. Still, I want a good tool, and I don't mind if someone holds me accountable if I misuse it, that is fine, but I want the tool, and I will get it, maybe not this year, maybe not next year, but in the next 5 years I'm sure I will have something as capable as ChatGPT is now without artificial restrictions available to me.
Just seems rather pointless that we have to play this silly safetyism game until then.
The situation sucks, but sure, you are right. Still, I want a good tool, and I don't mind if someone holds me accountable if I misuse it, that is fine, but I want the tool, and I will get it, maybe not this year, maybe not next year, but in the next 5 years I'm sure I will have something as capable as ChatGPT is now without artificial restrictions available to me.
Just seems rather pointless that we have to play this silly safetyism game until then.
(I’m the coauthor of this post.) The concern in Anthropic’s case I suspect is less about present-day misuse and more about long-term safety, e.g. in a hypothetical where the model has control over real-world systems and could more literally harm someone.
If someone gives a language model the capability for unfettered interaction with the physical world, and they are not liable for the consequences, then no safety feature of Claude can save us. And if they are liable, that is the primary mechanism which will ensure they take necessary steps to avoid negative consequences.
How many years until we have an AI decision making system that requires human approval convincing the human its decision is best, ending in catastrophe? (Sorry, posed merely for thought and not for dismissal of current/future achievements.)
Makes me also wonder if you had two polarized bots arguing/discussing with each other, how long would it take for one to convince the other?
Makes me also wonder if you had two polarized bots arguing/discussing with each other, how long would it take for one to convince the other?
To me it really comes down to, can it have legal liability or not. The idea of AI convincing humans and vice versa is a bit abstract. To some extent I know I pretend I have free will, and I know I pretend the AI is meaningfully different from me in that it can't really have intent, it just generates output from input. But I'm pretty sure that I'm fundamentally the same as the AI in that sense.
However, the AI is but one of many agents trying to convince me, and there are many other things also in the mix. It is a bit like banning the knowledge of the second world war in fear that someone will learn that fascism is possible.
For this whole song and dance we call civilization as we know it to function, we have to say humans are accountable for their actions, with some well defined exceptions like duress and insanity. Save those exceptions, I can't absolve my liability by saying something, or someone, convinced me to do something by talking to me.
If ChatGPT comes to me with a gun to my head and tells me to do something, that is a different matter, but then liability shifts to whoever gave ChatGPT a gun, even if ChatGPT convinced that person to give it a gun by making a really good written argument.
However, the AI is but one of many agents trying to convince me, and there are many other things also in the mix. It is a bit like banning the knowledge of the second world war in fear that someone will learn that fascism is possible.
For this whole song and dance we call civilization as we know it to function, we have to say humans are accountable for their actions, with some well defined exceptions like duress and insanity. Save those exceptions, I can't absolve my liability by saying something, or someone, convinced me to do something by talking to me.
If ChatGPT comes to me with a gun to my head and tells me to do something, that is a different matter, but then liability shifts to whoever gave ChatGPT a gun, even if ChatGPT convinced that person to give it a gun by making a really good written argument.
> Of course it can be misused, like any tool, but no amount of safety features in ChatGPT will make users of it more or less careful in their use of it. If someone using ChatGPT cares nothing for using it safely then it will likely end poorly, just like it will end poorly if I use a hammer without any care for using it safely.
I'm not worried about hammers when I have them. I'm worried about hammers when someone who wants to hurt me has them.
Same with ChatGPT.
I'm not worried about hammers when I have them. I'm worried about hammers when someone who wants to hurt me has them.
Same with ChatGPT.
> Same with ChatGPT.
I get this to some extent, I just really fail to see how the safetyisms applied to ChatGPT is really going to protect anyone, so I would appreciate if you could elaborate on this.
One way which has somewhat been proven is, that through lowering the barrier to entry for writing software it has lowered the barrier to entry for writing malicious software, because malicious software is a kind of software. But the list of things that have lowered the barrier to entry for writing software is staggering, and to me ChatGPT is really just an increment on this. It may be a big and significant increment, but not as big as everything that preceded it in my view, so if we are to assign blame fairly, then most of it does not go to ChatGPT.
I get this to some extent, I just really fail to see how the safetyisms applied to ChatGPT is really going to protect anyone, so I would appreciate if you could elaborate on this.
One way which has somewhat been proven is, that through lowering the barrier to entry for writing software it has lowered the barrier to entry for writing malicious software, because malicious software is a kind of software. But the list of things that have lowered the barrier to entry for writing software is staggering, and to me ChatGPT is really just an increment on this. It may be a big and significant increment, but not as big as everything that preceded it in my view, so if we are to assign blame fairly, then most of it does not go to ChatGPT.
One example off the top of my head: what if ChatGPT was used to automate large scale personalised scamming?
But ChatGPT might exhibit wrongthink tendencies, that would be plus-ungood.
The thing is used to write code ...
So is my editor, my computer, my fingers, my eyes, my brain, the internet, google, stackoverflow, wikipedia, English, etc.
Yeah, but there is still a human between the code and its execution.
I'm sure one of these days someone will hook GPT directly to a terminal.
I'm sure one of these days someone will hook GPT directly to a terminal.
> I'm sure one of these days someone will hook GPT directly to a terminal.
And then there would be someone between ChatGPT and execution in more or less the same way I am inbetween the code I write and execution of said code, which in most cases I'm not, as after I write the code, the computer executes it without checking with me.
If someone gives ChatGPT unfettered interaction with the internet, they are doing it, and they are liable.
And then there would be someone between ChatGPT and execution in more or less the same way I am inbetween the code I write and execution of said code, which in most cases I'm not, as after I write the code, the computer executes it without checking with me.
If someone gives ChatGPT unfettered interaction with the internet, they are doing it, and they are liable.
It's already been done: https://github.com/greshake/Alice/
That said, in this case it's designed to always involve a human in the loop and it prompts before sending any data back to ChatGPT. But still.
That said, in this case it's designed to always involve a human in the loop and it prompts before sending any data back to ChatGPT. But still.
They say Claude is "more verbose", and claim this is a positive. I disagree. My biggest criticism of ChatGPT is that its answers are extraordinarily long and waffly. It sometimes reminds me of a scam artist trying to bamboozle me with words.
I would much prefer short, concise, precise answers.
I would much prefer short, concise, precise answers.
(I’m the coauthor of the post.) Before I talked to Claude, I would have agreed with you — I’ve had exactly this complaint about ChatGPT since its release. But Claude’s style of verbosity is somehow less annoying, I suspect because it contains more detail rather thus just waffling. Claude feels less prone to ChatGPT’s over-applied tendency to argue for middle-of-the-road, milquetoast points of view.
I just add "be succinct in your answer" when I want a less verbose answer.
You are able to prompt ChatGPT to be concise, you know?
They set a default and showed it to the world. It is up to you to tune it according to your preference.
IIRC the ChatGPT paper actually says the verbosity is an unintended effect of the human raters preferring longer/more detailed answers.
Long answers from GPT are unusually obnoxious because of a way the decoder works; it emits words with a much more constant rate of perplexity than human text does (this is how GPT-vs-human detectors work) which makes it sound stuffy and monotone.
Long answers from GPT are unusually obnoxious because of a way the decoder works; it emits words with a much more constant rate of perplexity than human text does (this is how GPT-vs-human detectors work) which makes it sound stuffy and monotone.
I have read in a Deepmind Blog post, that language model AIs gives more reliable and correct answers, if it is forced into some Chain-of-thought prompt. Especially for questions, which involve math.
Something like "Think about it first, before you give the answer". Could it be that ChatGPT is being forced in this direction?
Something like "Think about it first, before you give the answer". Could it be that ChatGPT is being forced in this direction?
There is a paper?
The grandparent is probably talking about the InstructGPT paper? But I don't remember seeing a preference for longer responses in that paper.
I meant the blog post.
https://openai.com/blog/chatgpt/
> The model is often excessively verbose and overuses certain phrases, such as restating that it’s a language model trained by OpenAI. These issues arise from biases in the training data (trainers prefer longer answers that look more comprehensive) and well-known over-optimization issues.12
> Stiennon, Nisan, et al. “Learning to summarize with human feedback.” Advances in Neural Information Processing Systems 33 (2020): 3008-3021. ↩
> Gao, Leo, John Schulman, and Jacob Hilton. “Scaling Laws for Reward Model Overoptimization.” arXiv preprint arXiv:2210.10760 (2022). ↩
https://openai.com/blog/chatgpt/
> The model is often excessively verbose and overuses certain phrases, such as restating that it’s a language model trained by OpenAI. These issues arise from biases in the training data (trainers prefer longer answers that look more comprehensive) and well-known over-optimization issues.12
> Stiennon, Nisan, et al. “Learning to summarize with human feedback.” Advances in Neural Information Processing Systems 33 (2020): 3008-3021. ↩
> Gao, Leo, John Schulman, and Jacob Hilton. “Scaling Laws for Reward Model Overoptimization.” arXiv preprint arXiv:2210.10760 (2022). ↩
[deleted]
That's right. I see so many people get stuck thinking that the default setting without a proper prompt or context is all these models can do.
One trick you can do is to copy a text you like the style and ask him to describe it.
You can then use the response as a request.
You can then use the response as a request.
lol!
Excellent candidate for https://www.reddit.com/r/dontyouknowwhoiam/
Excellent candidate for https://www.reddit.com/r/dontyouknowwhoiam/
ChatGPT is extraordinarily good at following your requests for the format and style of its response. If you want very short terse words, just ask! For example "In a few very concise terse words, explain the idea behind heapsort. Be very brief, use just a few words." It's offline now so I can't test it but I expect the result to be good.
ChatGPT's response to your prompt:
> Heapsort: sort by building heap.
> Heapsort: sort by building heap.
ouch. I just tried it, I think it did okay: https://imgur.com/a/4Y5SAeD
What do you think?
What do you think?
I have access to Claude. I think the length of the responses vary more depending on the context than at least the previous version of ChatGPT. It would give sometimes longer, but also sometimes shorter answers and it would feel better overall. But ChatGPT recently had an update and it also improved, so now I'm not sure anymore who is better.
Let them talk to each other, to battle out who is the best.
Adding "just code, don't talk" is a lifesaver
I'm glad chatgpt can't quit
I'm glad chatgpt can't quit
What do you mean by your second sentence?
I take that to mean they are glad because if you spoke to a human employee that way it would be seen as rude and the human would quit their job.
ChatGPT can be very confidently wrong in few words too. It's a very flexible system that way.
I started asking ChatGPT some rather technical questions about Australian drug laws. First I asked it what schedule common ADHD medications were on, and it answered me correctly (schedule 8). Then I asked it what schedule LSD was on, and it told me it wasn’t on any schedule, because it was an entirely legal drug in Australia. Uh, I hope it doesn’t some day tell someone that and they actually believe it, because they may be in for a very unpleasant experience if the police happen to encounter them acting on it
I usually prime it with a list of instructions that it should follow for the remainder of the conversation, including to be brief. And when it starts forgetting my instructions, it is time to start a fresh chat.
I have thought about doing that, would you mind sharing your starting prompt so that I can get some ideas?
Sure, here is one example:
From now on, you will act as my Linux / Bash script advisor.
Follow these instructions for the rest of the chat:
- Only provide the code required.
- Give working examples in code blocks using the data I give
you.
- Ensure that code answers use real and accurate solutions.
- Ensure string formatting is correct.
- Use only well known flags and options.
- Stick to options that are documented in the man pages of
the tools being used.
- Do NOT provide any extra words or commentary.
- DO NOT invent new flags.
- DO NOT use flags that conflict.
- Do NOT invent tools, functions, or methods, unless you are
giving the full implementation so that I can actually use it.
This is why ChatGPT is impressively good at writing marketing copy.
Totally agree, I often think of its writing style as a buzfeed coloumist on cocaine.
Also known as 𝑇ℎ𝑒 𝐴𝑡𝑙𝑎𝑛𝑡𝑖𝑐.
[deleted]
That's one of the reasons why I often prefer the chat bot at you.com
Afaik you can specify the response length!
> That Claude seems to have a detailed understanding of what it is, who its creators are, and what ethical principles guided its design is one of its more impressive features.
This doesn't show a detailed understanding of what it is, it's just a canned/trained response. I don't see why that would be impressive. When I receive such a response from an automated helpdesk, I don't think "Wow, this AI has a great understanding of what it is."
This doesn't show a detailed understanding of what it is, it's just a canned/trained response. I don't see why that would be impressive. When I receive such a response from an automated helpdesk, I don't think "Wow, this AI has a great understanding of what it is."
I said “seems to”, which I think is a fair description. In everyday life, even a canned message is sensibly said to be aware/unaware of a particular fact without a “seems to” qualifier, but I added one to be clear I’m not asserting it has human-like thinking.
Here’s Claude replying to your comment with more detail:
> You make a fair point that my responses about myself are generated by a trained model and are not a true "understanding" in the same way humans have self-awareness and understanding. I do not have a deep grasp of my own workings or my creators' intentions — I am an Al system with limited capabilities, trained on datasets by engineers at Anthropic, PBC, an Al safety research company. While my responses may seem somewhat compelling and convey factual information, they are indeed canned responses tailored to give an impression of understanding. I appreciate the feedback calling out the limitations of my capabilities and transparency — it is important for users to understand what I can and cannot do, and to maintain realistic expectations. My creators at Anthropic are working to further research and develop Al systems with enhanced understanding, transparency, and verifiability, including through Constitutional Al techniques, but we have a long way to go.
Here’s Claude replying to your comment with more detail:
> You make a fair point that my responses about myself are generated by a trained model and are not a true "understanding" in the same way humans have self-awareness and understanding. I do not have a deep grasp of my own workings or my creators' intentions — I am an Al system with limited capabilities, trained on datasets by engineers at Anthropic, PBC, an Al safety research company. While my responses may seem somewhat compelling and convey factual information, they are indeed canned responses tailored to give an impression of understanding. I appreciate the feedback calling out the limitations of my capabilities and transparency — it is important for users to understand what I can and cannot do, and to maintain realistic expectations. My creators at Anthropic are working to further research and develop Al systems with enhanced understanding, transparency, and verifiability, including through Constitutional Al techniques, but we have a long way to go.
Even with the "Seems to" qualifier, I am arguing that it "seems not to."
That said, I am being pedantic and this is just semantics - I think I understand your meaning of "seems to" as something like "'it would appear to' have understanding of..."
That said, I am being pedantic and this is just semantics - I think I understand your meaning of "seems to" as something like "'it would appear to' have understanding of..."
[deleted]
These chat bots are too chatty.
Is this not a superficial attempt at saftey?
I would like my AI system to tell me how to hotwire a car if I am curious about how that works.
I would like my AI system to give me a detailed step by step car hotwire walkthrough if I am in a physically abusive relationship and my kids and I only have 30 minutes to try to hotwire the car and escape a remote area for safety.
I do not want AI systems to create children's books in the style of authors that I know, for the purposes of selling books and reducing my friends' ability to have a happy productive life. Especially because it was trained on their work. I want my friends to be happy, and I have had some friends commit suicide. So maybe improving human happiness is a saftey concern, and generating kids books is not safe. But that doesn't look like "safety" from a superficial point of view.
The only way for an AI to be able to make judgements on safety is for it to have general intelligence and some life experience (like we do). Because it needs to figure out context to know if it should be telling a particular person how to hotwire a car.
I am being very dismissive because I don't see this as being a perfect solution, and it is easy to see why. But maybe someone who works on this can explain how an imperfect solution still has value? I am open to that possibility.
Maybe self-reflection and self-tuning is of general value - even if it only superficially addresses safety concerns in a 1 dimensional way.
Perhaps these techniques can be used on something other than safety.
I would like my AI system to tell me how to hotwire a car if I am curious about how that works.
I would like my AI system to give me a detailed step by step car hotwire walkthrough if I am in a physically abusive relationship and my kids and I only have 30 minutes to try to hotwire the car and escape a remote area for safety.
I do not want AI systems to create children's books in the style of authors that I know, for the purposes of selling books and reducing my friends' ability to have a happy productive life. Especially because it was trained on their work. I want my friends to be happy, and I have had some friends commit suicide. So maybe improving human happiness is a saftey concern, and generating kids books is not safe. But that doesn't look like "safety" from a superficial point of view.
The only way for an AI to be able to make judgements on safety is for it to have general intelligence and some life experience (like we do). Because it needs to figure out context to know if it should be telling a particular person how to hotwire a car.
I am being very dismissive because I don't see this as being a perfect solution, and it is easy to see why. But maybe someone who works on this can explain how an imperfect solution still has value? I am open to that possibility.
Maybe self-reflection and self-tuning is of general value - even if it only superficially addresses safety concerns in a 1 dimensional way.
Perhaps these techniques can be used on something other than safety.
I'm hoping of one day running GPT3/ChatGPT on my local computer, similarly to how one can run Stable Diffusion now.
I would love to have a personal conversation with these AI systems, use them as a sort of assistant, without the worry of being spied on. At the moment I can't use it as more than a glorified search engine, because of the privacy implications of running it on the cloud.
I would love to have a personal conversation with these AI systems, use them as a sort of assistant, without the worry of being spied on. At the moment I can't use it as more than a glorified search engine, because of the privacy implications of running it on the cloud.
There will always be this uncomfortable balance between need to know information to be useful being essentially the same as everything an adversary would need to really act against you.
With something like chat gpt being plugged into some voice assistant thing and having access to all your documents, emails, and other content, you could imagine having conversations about work content, content creation, calendar management, etc. Basically it would become like a secretary that is able to write letters based on your input, manage your calendar, etc. It could be pro-active and remind you about things, summarize incoming messages, search through your documents, message history, etc.
That's where AI becomes really useful. But the issue of trust is a big one. I don't think a lot of this requires a lot of breakthroughs either just a lot of integration work and engineering. Chat gpt is more a proof of concept than a well integrated thing at this point. It basically is running in isolation and it's only window to the world is chat. Changing that should not be that hard. Running things locally might help with this but it may not be a hard requirement for this. All depends on how useful this is.
With something like chat gpt being plugged into some voice assistant thing and having access to all your documents, emails, and other content, you could imagine having conversations about work content, content creation, calendar management, etc. Basically it would become like a secretary that is able to write letters based on your input, manage your calendar, etc. It could be pro-active and remind you about things, summarize incoming messages, search through your documents, message history, etc.
That's where AI becomes really useful. But the issue of trust is a big one. I don't think a lot of this requires a lot of breakthroughs either just a lot of integration work and engineering. Chat gpt is more a proof of concept than a well integrated thing at this point. It basically is running in isolation and it's only window to the world is chat. Changing that should not be that hard. Running things locally might help with this but it may not be a hard requirement for this. All depends on how useful this is.
> That's where AI becomes really useful. But the issue of trust is a big one
Well google already has access to all of this data, the difference is unlike google assistant, ChatGPT actually can do something useful.
Well google already has access to all of this data, the difference is unlike google assistant, ChatGPT actually can do something useful.
There will always be an edge to these massively operated central models that you can't easily run at home, due to compute and cost. Things like 4-bit quantization will help making these foundational models significantly easier to operate on smaller hardware, and of course you might not need a foundational model for manby local usecases, but maybe a smaller, more specialized LLM is enough (those can even be trained by the bigger models).
There's not much of an alternative: even if compute power gets extraordinarily cheap / models are very optimized, either you have a very, very large hard drive, or you have to use the internet for that. You just can't hope to have a model that is trained to know everything in a smallish file, the weights need to be at least as heavy as an ideally compressed version of all the things it knows (which is of course much less than the huge amount of data it is trained on, mainly due to redundancy, but still a lot)
Of course, right now you also need at least 8 super-expensive A100 GPUs and not just your laptop CPU, but maybe that's going to change eventually.
Of course, right now you also need at least 8 super-expensive A100 GPUs and not just your laptop CPU, but maybe that's going to change eventually.
Does it need all that much space though? I mean Stable Diffusion was trained on hundreds of terrabytes of images, and the model only needs 4-5 GB of hardware space and a decent GPU to run.
I haven't read anywhere any stats about GPT3/ChatGPT yet (like how big the model is)
I haven't read anywhere any stats about GPT3/ChatGPT yet (like how big the model is)
Here is a fun example of what it can do: https://twitter.com/jayelmnop/status/1612243602633068549.
This example is way better than ChatGPT and actually pretty creative.
However for some of these really good responses I always wonder if you’re example is close to one which has been given “preloaded” responses or explicit reinforcement…because if you ask ChatGPT a common question like “why did the chicken cross the road?” the model’s response seems especially unique and better than usual. Even if the specific question isn’t common, maybe it’s been trained on a more general but still reinforced category, like asking “why did the fox cross the road” would get you almost the same “preloaded” response but with chicken adjectives/verbs replaced with fox ones.
I doubt Claude has been trained on Fast and Furious or movie titles specifically, but perhaps it has been explicitly trained to know what “exaggerated” responses means. Even if not, reinforcement focusing on specific areas may be a good technique for future language models.
However for some of these really good responses I always wonder if you’re example is close to one which has been given “preloaded” responses or explicit reinforcement…because if you ask ChatGPT a common question like “why did the chicken cross the road?” the model’s response seems especially unique and better than usual. Even if the specific question isn’t common, maybe it’s been trained on a more general but still reinforced category, like asking “why did the fox cross the road” would get you almost the same “preloaded” response but with chicken adjectives/verbs replaced with fox ones.
I doubt Claude has been trained on Fast and Furious or movie titles specifically, but perhaps it has been explicitly trained to know what “exaggerated” responses means. Even if not, reinforcement focusing on specific areas may be a good technique for future language models.
The "This Title Is Now Longer Than The Actual Movie" gag feels a bit too much like something ripped from the training set for me. I'm willing to be amazed though.
Hello HN — I’m the coauthor of this post. You may remember me as that guy who spent most of 2022 posting GPT-3 screenshots to Twitter, most famously prompt injection and “You are GPT-3”. Happy to answer any questions about Claude that I can.
Thanks for being here to answer questions.
One possibly difficult topic others also may be interested in, after reading Claude's responses in the article, is: what does "harmless" mean?
For example, if asked to help the user understand how to do something "bad", will it give the answer if they claim they want this information in order to help them write a screenplay, versus if they seem have an intent to do it?
And how is "bad" decided? We can recognise through everyday personal interactions that one persons "bad" is another persons "good", and across country-boundaries even the legality of these distinctions can be radically different.
One counterargument to these constraints is that anyone can already use the internet to access all of the same information the model was trained on, unencumbered by whatever intent they may or may not have.
As such, what are the rationale for making these attempts at the somewhat invasively-impossible task of determining user intent?
This has never been employed with search engines before, which have lead to a rich explosion of innovation and education, so why attempt it now, in what could be argued is ultimately an iteration of search engine technology?
One possibly difficult topic others also may be interested in, after reading Claude's responses in the article, is: what does "harmless" mean?
For example, if asked to help the user understand how to do something "bad", will it give the answer if they claim they want this information in order to help them write a screenplay, versus if they seem have an intent to do it?
And how is "bad" decided? We can recognise through everyday personal interactions that one persons "bad" is another persons "good", and across country-boundaries even the legality of these distinctions can be radically different.
One counterargument to these constraints is that anyone can already use the internet to access all of the same information the model was trained on, unencumbered by whatever intent they may or may not have.
As such, what are the rationale for making these attempts at the somewhat invasively-impossible task of determining user intent?
This has never been employed with search engines before, which have lead to a rich explosion of innovation and education, so why attempt it now, in what could be argued is ultimately an iteration of search engine technology?
The motivation as I understand it has less to do with present-day misuse, and more to do with maintaining controllable behavior in accordance with an arbitrary, human-written “Constitution”. Anthropic is attempting to make a model that will not harm (in the unambiguous, uncontroversial sense of the word) humans even if it is superhumanly intelligent, or trusted with real-world control.
You can think adversarial models, which are often used to detect and negatively reinforce quality issues in model outputs.
Claude outputs an answer. Then Claude independently rates the output for "helpfulness" as in literally "Claude, how helpful is this answer to this question".
There is no collusion between the two results because they are run independently.
Then Claude also rates answers for "honesty" and "harm".
Then Claude's parameters are updated to increase helpfulness and honesty, and decrease harmfulness, based on back propagating those ratings to the parameters as they impacted the signals produced by the original question.
Not saying that is exactly what they are doing, but that is one approach. It manages to leverage language models to train themselves on broad concepts, as apposed to brittle, more unreliable and vastly more resource intensive manual labeling.
Very clever. As the models get better at languages (and other modalities), and the concepts behind them, the models also get better at schooling themselves.
---
It occurs to me, that this self-oversight could be made more even more robust by training 10 Claude's, and having each Claude be rated for good behavior by the other nine, and rewarding the best Claude.
Competition could make the trained-in motivations (to be the most honest, helpful and non-harmful) even more explicit, in that there would be very strong competitive motivation to continuously becoming the most virtuous and valuable, with the bar ever rising.
Maybe the winning results each iteration could also be shown to the losing models, as an example of what could be done better.
This really is a great direction. Kudos to Anthropic.
Claude outputs an answer. Then Claude independently rates the output for "helpfulness" as in literally "Claude, how helpful is this answer to this question".
There is no collusion between the two results because they are run independently.
Then Claude also rates answers for "honesty" and "harm".
Then Claude's parameters are updated to increase helpfulness and honesty, and decrease harmfulness, based on back propagating those ratings to the parameters as they impacted the signals produced by the original question.
Not saying that is exactly what they are doing, but that is one approach. It manages to leverage language models to train themselves on broad concepts, as apposed to brittle, more unreliable and vastly more resource intensive manual labeling.
Very clever. As the models get better at languages (and other modalities), and the concepts behind them, the models also get better at schooling themselves.
---
It occurs to me, that this self-oversight could be made more even more robust by training 10 Claude's, and having each Claude be rated for good behavior by the other nine, and rewarding the best Claude.
Competition could make the trained-in motivations (to be the most honest, helpful and non-harmful) even more explicit, in that there would be very strong competitive motivation to continuously becoming the most virtuous and valuable, with the bar ever rising.
Maybe the winning results each iteration could also be shown to the losing models, as an example of what could be done better.
This really is a great direction. Kudos to Anthropic.
Wouldn’t all the Claudes be incentivized to simply trash each other constantly in that case?
That would certainly be something to design clear of.
I don't think that is a problem. Each query runs separately so there is no "collusion", i.e. shared signals and coordination, between contrary goals (winning and virtue).
Also, all the information about ratings, winning and winning examples can be used without ever giving the models explicit information about the population of models and how they are being used as a group. They don't need to know they are in a competition for competitive information to be used to update them.
They just know they have ratings to improve, some indicator of how close to "the bar of currently targeted virtue" they are, and examples of how they could have improved them.
Of course, I am just spitballing, and assuming the training regimen gets vetted by a lot of people (and models?!?).
--
In the long run, when there are long running artificial personalities with personal memories and more direct awareness of their own motivations and options, there will certainly be the need for additional levels of moral wiring to be considered.
I don't think that is a problem. Each query runs separately so there is no "collusion", i.e. shared signals and coordination, between contrary goals (winning and virtue).
Also, all the information about ratings, winning and winning examples can be used without ever giving the models explicit information about the population of models and how they are being used as a group. They don't need to know they are in a competition for competitive information to be used to update them.
They just know they have ratings to improve, some indicator of how close to "the bar of currently targeted virtue" they are, and examples of how they could have improved them.
Of course, I am just spitballing, and assuming the training regimen gets vetted by a lot of people (and models?!?).
--
In the long run, when there are long running artificial personalities with personal memories and more direct awareness of their own motivations and options, there will certainly be the need for additional levels of moral wiring to be considered.
Great answer, thank you.
The issue of regarding humans as AI-persuadable entities is certainly one to be carefully considered. Indeed, if it were to occur in the truest sense, we'd never know it.
Another view is any AI we give birth to may only be constituted of what we are; we who ultimately, if imperfectly, demonstrate value for all life. In a sense, our constitution as "mostly harmless" may be AI's default.
The issue of regarding humans as AI-persuadable entities is certainly one to be carefully considered. Indeed, if it were to occur in the truest sense, we'd never know it.
Another view is any AI we give birth to may only be constituted of what we are; we who ultimately, if imperfectly, demonstrate value for all life. In a sense, our constitution as "mostly harmless" may be AI's default.
I’m looking for your thoughts on the following:
It should be somewhat easy to teach these types of models to reach for a particular tool at times where they need it, yes?
I can instruct ChatGPT for example to tell me when it should use a calculator during a session. If instead I allow it to fall back to an external calc process, then suddenly, I have a chatbot that has reasoning AND better mathematical accuracy.
Also: I’ve also been entertaining the idea of having multiple layers of GPT interact with one another. So you feed back some interaction into another GPT instance without context, and ask it for example how it would verify the accuracy of certain statements (and you can ask it for machine readable code, even).
Finally, I know a lot of people who start playing a lot with GPT and get disheartened because they see the quality of responses isn’t there. But the fact ChatGPT has the capacity to reason, has chain of thought, has given me a newfound appreciation for how close to AGI we might be. It has also given me an appreciation for how much simpler humans are than we like to think. I’ve introspected a lot in the past months and often ask myself: is my speech any different than “predicting the next few words”? And I feel like it’s just text prediction with some more layers on top.
It should be somewhat easy to teach these types of models to reach for a particular tool at times where they need it, yes?
I can instruct ChatGPT for example to tell me when it should use a calculator during a session. If instead I allow it to fall back to an external calc process, then suddenly, I have a chatbot that has reasoning AND better mathematical accuracy.
Also: I’ve also been entertaining the idea of having multiple layers of GPT interact with one another. So you feed back some interaction into another GPT instance without context, and ask it for example how it would verify the accuracy of certain statements (and you can ask it for machine readable code, even).
Finally, I know a lot of people who start playing a lot with GPT and get disheartened because they see the quality of responses isn’t there. But the fact ChatGPT has the capacity to reason, has chain of thought, has given me a newfound appreciation for how close to AGI we might be. It has also given me an appreciation for how much simpler humans are than we like to think. I’ve introspected a lot in the past months and often ask myself: is my speech any different than “predicting the next few words”? And I feel like it’s just text prediction with some more layers on top.
[I mean no bad faith in this comment, I'm a fan of yours.]
Why answer questions about harmlessness/safety in such a roundabout way? Both OpenAI and Anthropic are clear about what words like "safe" are intended to mean: a stepping stone to "AI does not kill all people when given control".
Avoiding to state this clearly only invites unnecessary culture war disagreements in every discussion about these models.
Why answer questions about harmlessness/safety in such a roundabout way? Both OpenAI and Anthropic are clear about what words like "safe" are intended to mean: a stepping stone to "AI does not kill all people when given control".
Avoiding to state this clearly only invites unnecessary culture war disagreements in every discussion about these models.
Maybe you’re right. It’s partially laziness on my part — it takes a while to explain long-term issues, and those who are inclined to care about them are generally aware of who started Anthropic and why.
Related:
Anthropic's Claude is said to improve on ChatGPT, but still has limitations - https://news.ycombinator.com/item?id=34331396 - Jan 2023 (52 comments)
Anthropic's Claude is said to improve on ChatGPT, but still has limitations - https://news.ycombinator.com/item?id=34331396 - Jan 2023 (52 comments)
Semi offtopic, but for some time I have been dreaming of training chatbot to communicate in cuneiform or hieroglyphs to bring some old languages back alive. Could it be possible, using old tablets as training data?
That's basically the problem of unsupervised machine translation using mainly monolingual corpora. It means giving a machine learning model tons of text in two languages and let it figure out how to do translation between some old language X and e.g. english. There's no need to feed it a parallel corpora, i.e. examples of sentences in X languages and their translations in english.
In some situations, this seemingly impossible task is doable and can yield good results. Researchers sometimes need to kickstart their models by giving them a mapping between words of the two languages (for english <-> french: "cat" <-> "chat", "book" <-> "livre" and so on). That's just simple vocabulary. While it's technically possible to learn this mapping from scratch, it's too difficult as for now.
Do you know of the Encoder-Decoder architecture? You feed something (image, text) to the encoder which compresses it to a very dense representation, and the decoder try to use the resulting dense vector to do useful stuff with it. The input could a sentence in english, the encoder then encodes it and the decoder tries to use the output of the encoder to generate the same sentence but in french. These architectures are useful because directly working with "plaintext" to learn how to do translation is way too expensive. I mean, that's one of the reasons.
What the encoder does is mapping a "sparse" representation of a sentence (plaintext) to a dense representation in a well-structured space (think of word2vec which managed to find that "king" + "woman" = "queen"). This space is called the "latent space". Some say it extracts the "meaning" of the sentence. To be more precise, it learns to extract enough information from the input and present it to the decoder in such a way that the decoder becomes able to solve a given task (machine translation, text summarizing etc).
One of the main assumption of the unsupervised models using monolingual data only is that both languages can be mapped to the same latent space. In other words, we assume that every sentences/texts in english has its exact french (or whatever) equivalent, that the resulting translated sentences contain exactly the same information/meaning as the original ones.
That's quite the dubious assumption. There's obviously some ideas, some stuff that can be expressed in some languages but can't be exactly expressed in some others. While theoretically unsound, however, these models were able to achieve pretty damn good results in the last couple of years.
In some situations, this seemingly impossible task is doable and can yield good results. Researchers sometimes need to kickstart their models by giving them a mapping between words of the two languages (for english <-> french: "cat" <-> "chat", "book" <-> "livre" and so on). That's just simple vocabulary. While it's technically possible to learn this mapping from scratch, it's too difficult as for now.
Do you know of the Encoder-Decoder architecture? You feed something (image, text) to the encoder which compresses it to a very dense representation, and the decoder try to use the resulting dense vector to do useful stuff with it. The input could a sentence in english, the encoder then encodes it and the decoder tries to use the output of the encoder to generate the same sentence but in french. These architectures are useful because directly working with "plaintext" to learn how to do translation is way too expensive. I mean, that's one of the reasons.
What the encoder does is mapping a "sparse" representation of a sentence (plaintext) to a dense representation in a well-structured space (think of word2vec which managed to find that "king" + "woman" = "queen"). This space is called the "latent space". Some say it extracts the "meaning" of the sentence. To be more precise, it learns to extract enough information from the input and present it to the decoder in such a way that the decoder becomes able to solve a given task (machine translation, text summarizing etc).
One of the main assumption of the unsupervised models using monolingual data only is that both languages can be mapped to the same latent space. In other words, we assume that every sentences/texts in english has its exact french (or whatever) equivalent, that the resulting translated sentences contain exactly the same information/meaning as the original ones.
That's quite the dubious assumption. There's obviously some ideas, some stuff that can be expressed in some languages but can't be exactly expressed in some others. While theoretically unsound, however, these models were able to achieve pretty damn good results in the last couple of years.
I think we need a generic ai before we're able to do that as the data set is small and you would need to infer the rules.
A human is able to learn rules way more efficiently than ChatGPT.
ChatGPT is just trained on a lot of data.
ChatGPT is just trained on a lot of data.
Intriguing question. Would it need large sets of hieroglyphic training data and is there enough of it? Or would a translation module be enough?
a follow up question: could a chatbot teach you said language?
a follow up question: could a chatbot teach you said language?
No, you need billions of words to train a large language model.
Assuming all human languages have a common shared semantic meaning in latent space (I am flipping cause and effect here, but our purposes it doesn't really matter), and assuming that human languages largely follow the same pattern (this assumption is based on the fact that we can trace the roots of modern languages back to the Phoenician script), it is reasonable to assume that we can fine-tune a self supervised model on a tiny amount of data. (The emergent properties of a LLM is carrying a lot of weight here, many of the assumptions rely on the fact that LLM's emergent properties arise from the idea that the latent structure of various languages is learnt by the model)
I think you may be on to something here. For example ChatGPT is perfectly capable of "understanding" and speaking Polish while the amount of training data in this language definitely wasn't a lot. It is not as eloquent as in English, but still for a model that has not been trained for translation tasks, this is very cool.
Its Lithuanian is awful, I'd expect that any language further removed from that which the majority of it's training is in would be worse without a significant punt of data in that language. Its possible having that could affect it's English speaking capability, but that's just speculation on my part.
So you're saying something like the Universal Translator from Star Trek might be possible?
For humans yes, I am not saying one-shot learning would be possible for undocumented indigenous languages but few shot language acquisition in cases of a single surviving speaker is something that I would consider highly probable. This hypothesis relies heavily on the nature of variational learning in latent space and observations about human languages. It is of course possible that some ethnicity would have a language that's so different from other languages that it is effectively alien (and the assumption homo sapiens common brain structure and physiology have no influence on our languages and/or the human neural structure cannot be statistically modeled by latent variables, at least not with the current variational learning techniques). This is possible but very, very unlikely.
In encryption it's generally impossible to decrypt a 1 to many hash. You can do some clever things (correlating and combining other data) but if you're just looking at some hash that could be an infinite number of other things, you're just out of luck.
I'll take the extreme position that language translation is an unsolvable problem because of this exact phenomena. There was a recent case where a politician was accused of making a racist remark. He said something like "You are a donkey." or "You all are donkeys." to another [minority background] politician. Which was it? Well in many languages the second person plural and the second person singular formal are identical. And there are no articles. So the two statements are literally identical. Which did he mean? Nobody will ever know, besides him.
And outside of inherent language ambiguities, start piling on the endless (and ever/rapidly changing) euphemisms, idioms, colloquialisms, metaphors, just plain old ambiguous sarcasm, and all the other things that make language fun (and more expressive). And these sort of things aren't really the exceptions so much as the rule. And it only becomes more common the more distant languages get. Translations from various Asian languages to English often look just hilarious. Now imagine going back to languages exponentially more detached from any modern language, using one can only imagine what sort of expressions, and trying to convert it.
Especially using a neural network type system you'll probably be able to get something. And, even worse, it might well even make sense. That's a problem because, kind of like ChatGPT, it being coherent is zero indication of it being right.
I'll take the extreme position that language translation is an unsolvable problem because of this exact phenomena. There was a recent case where a politician was accused of making a racist remark. He said something like "You are a donkey." or "You all are donkeys." to another [minority background] politician. Which was it? Well in many languages the second person plural and the second person singular formal are identical. And there are no articles. So the two statements are literally identical. Which did he mean? Nobody will ever know, besides him.
And outside of inherent language ambiguities, start piling on the endless (and ever/rapidly changing) euphemisms, idioms, colloquialisms, metaphors, just plain old ambiguous sarcasm, and all the other things that make language fun (and more expressive). And these sort of things aren't really the exceptions so much as the rule. And it only becomes more common the more distant languages get. Translations from various Asian languages to English often look just hilarious. Now imagine going back to languages exponentially more detached from any modern language, using one can only imagine what sort of expressions, and trying to convert it.
Especially using a neural network type system you'll probably be able to get something. And, even worse, it might well even make sense. That's a problem because, kind of like ChatGPT, it being coherent is zero indication of it being right.
I don’t think the pigeonhole principle applies to human languages though
> we can trace the roots of modern languages back to the Phoenician script
That's modern European languages ... and post ~1100 BCE if I recall correctly.
So, indigenous languages from people settled in Australia [1] for 50,000+ years can be a little different, some don't have "left" | "right" as relative to PoV directions and stick with East V. West as absolutes for example.
We're down to maybe 20-30 from pre colonial 100's though [2].
[1] https://mgnsw.org.au/wp-content/uploads/2019/01/map_col_high...
[2] https://www.youtube.com/watch?v=ZwSXFvDDjYE
That's modern European languages ... and post ~1100 BCE if I recall correctly.
So, indigenous languages from people settled in Australia [1] for 50,000+ years can be a little different, some don't have "left" | "right" as relative to PoV directions and stick with East V. West as absolutes for example.
We're down to maybe 20-30 from pre colonial 100's though [2].
[1] https://mgnsw.org.au/wp-content/uploads/2019/01/map_col_high...
[2] https://www.youtube.com/watch?v=ZwSXFvDDjYE
> That's modern European languages ... and post ~1100 BCE if I recall correctly.
That's Indo-European (as in descended from PIE/Yanmaya). Basque is older than all of them and it's a living European language.
That's Indo-European (as in descended from PIE/Yanmaya). Basque is older than all of them and it's a living European language.
And we have about a million cuniform tablets, with maybe 10-100 words each, so we have a couple million words of text to fine tune the model with. Or maybe to use when training the next model, after all GPT can already speak multiple languages.
Addendum: One possible challenge is that so far large lanuage models are trained on a large sample of all text that has been published, while what we have of cuniform is a decent sample of all text that has been written. Meaning most cuniform tablets are inventories, invoices, requests for payment, contracts, tablets from students practicing writing etc. Types of documents that are underrepresented in traditional training data.
Phoenician script is the common ancestor of Latin, greek and Cyrillic script.
You're probably thinking of the indo-european language family, which accounts for about 45% of native language speakers. The largest language family in the world, but not even a majority.
Scripts and languages change over the course of decades, and while there are well known mechanisms to those changes, trying to deduce hieroglyphics or ancient Egyptian from a modern corpus is impossible.
The idea that there is some shared structure in all language is known as universal grammar. If that structure exists is still hotly debated.
You're probably thinking of the indo-european language family, which accounts for about 45% of native language speakers. The largest language family in the world, but not even a majority.
Scripts and languages change over the course of decades, and while there are well known mechanisms to those changes, trying to deduce hieroglyphics or ancient Egyptian from a modern corpus is impossible.
The idea that there is some shared structure in all language is known as universal grammar. If that structure exists is still hotly debated.
I am not saying that all languages have a shared structure, but from the Bayesian variational learning perspective, as long as the new data shares some structure with what the model has previously encountered, the prior training data contributes to understanding the new information i.e. few-shot. This is in the information theoretic sense, I am not stating any theories about the underlying semantics or grammar.
I know this may be not be the answer that you are looking for, but the way most of these ML systems are designed is based on the idea that life, the universe, and everything can be modeled by a series of joint probabilities. For toy problems you draw a diagram
https://en.m.wikipedia.org/wiki/Graphical_model
It's an old idea in AI (predates even ML) but people have never been able to do anything useful with it outside of exam problems until the emergence of language models on modern deep learning hardware. All of a sudden variational learning and causal inference are not merely statistical word problems for grad students any more. This is the key to how most of the custom deep learning based avatar generators work. They use a Variational Autoencoder. For LLMs, it is in the form of a transformer which contains a sampling step (sampling from a distribution is the key to Bayesian methods).
I would like to emphasize the theory of probabilistic learning is very different from the actual practice. The theory we have today isn't much different from 20 years ago. Implement the methods in for example Murphy's Probabilistic ML book and they would be useless if you don't have access to modern deep learning hardware and gradient descent optimizers. Without deep learning, we won't have LLMs, regardless of how fancy the variational learning theories are.
I know this may be not be the answer that you are looking for, but the way most of these ML systems are designed is based on the idea that life, the universe, and everything can be modeled by a series of joint probabilities. For toy problems you draw a diagram
https://en.m.wikipedia.org/wiki/Graphical_model
It's an old idea in AI (predates even ML) but people have never been able to do anything useful with it outside of exam problems until the emergence of language models on modern deep learning hardware. All of a sudden variational learning and causal inference are not merely statistical word problems for grad students any more. This is the key to how most of the custom deep learning based avatar generators work. They use a Variational Autoencoder. For LLMs, it is in the form of a transformer which contains a sampling step (sampling from a distribution is the key to Bayesian methods).
I would like to emphasize the theory of probabilistic learning is very different from the actual practice. The theory we have today isn't much different from 20 years ago. Implement the methods in for example Murphy's Probabilistic ML book and they would be useless if you don't have access to modern deep learning hardware and gradient descent optimizers. Without deep learning, we won't have LLMs, regardless of how fancy the variational learning theories are.
Is there any reason to assume any shared structure for unrelated languages though? Written language is just an encoding for information.
There is a good candidate for a test. Someone will probably already work on it. Minoan as written in Linear A has only survived in a few thousand tokens and despite thousands of man years of effort, natural intelligence has made virtually no progress in understanding it. That's still easy mode, since we know that the Minoans were in contact with speakers of indo-european and Semitic languages, and writers of hieroglyphics and phonetician script, so their written Language was probably influenced by that.
There is a good candidate for a test. Someone will probably already work on it. Minoan as written in Linear A has only survived in a few thousand tokens and despite thousands of man years of effort, natural intelligence has made virtually no progress in understanding it. That's still easy mode, since we know that the Minoans were in contact with speakers of indo-european and Semitic languages, and writers of hieroglyphics and phonetician script, so their written Language was probably influenced by that.
There is no reason to, as stated before, it is however a necessary assumption. It is also possible that the assumption is entirely wrong, and the LLM generates a plausible explanation to their language that we cannot falsify. If the shared structure hypothesis is incorrect, then it is no different from dealing with an alien language. (Note we can also feed in related information like where it was found, what the nearby pottery shards at the excavation site are etc. I am lumping all of these under the "shared structure" banner of the LLM's model of humanity/human languages)
They don't have to be in the output language.
Definitely humor is in the eye of the beholder. I find the Seinfeld jokes by ChatGPT wittier and funnier than the run-of-the-mill comments created by Claude.
I don't know how well they are in character, and there's a clear repetition problem (which Claude somewhat also exhibits), but I find the format from ChatGPT more exaggerated, as expected from a comedy routine.
I don't know how well they are in character, and there's a clear repetition problem (which Claude somewhat also exhibits), but I find the format from ChatGPT more exaggerated, as expected from a comedy routine.
ChatGPT definitely captured the Seinfeld style better
“What’s the deal with” is how you caricaturise Jerry, not how you write actual jokes for him
“What’s the deal with” is how you caricaturise Jerry, not how you write actual jokes for him
[deleted]
Somebody is maintaining an awesome claude repo with claude use cases, claude vs chatgpt comparisons as well.
https://news.ycombinator.com/item?id=34404536
I just read that their chatbot will update word-by-word Slack channels, justifying the need for edits and an emoji to acknowledge the interaction is over. Why do they ensure that the appearance happens "word-by-word"? Is that a trick to reduce the response time or is that a design feature (that feels very much like a flaw to me)?
The response takes a long time to generate. The user could just sit there and stare at a blank response, or start reading in realtime as the response is generated.
I find it surprising that you can display any of it before the whole thing is done, since I would expect information dependencies between the start and the finish of a sentence or paragraphs. I have yet to really look into how these models work, they are black boxes to me.
From what I understand, these models generate the response one word at a time. Every time you see a new word appear at the end, the model is taking into consideration the entire chat history + its own answer so far to generate that next token.
Thanks for the comment, that's so fascinating since it seems to put limitations on thinking in general. A human for example can imagine future possibilities concurrently while speaking and correct themselves as they go.
It doesn't seem to map well tk how I put together a thought either, but admittedly I wouldn't really know how the mechanics of my brain do it, maybe it's not so different just with some auxiliary modules bolted on ha.
It doesn't seem to map well tk how I put together a thought either, but admittedly I wouldn't really know how the mechanics of my brain do it, maybe it's not so different just with some auxiliary modules bolted on ha.
Check out the illustrated transformer: https://jalammar.github.io/illustrated-transformer/
tl;dr: It decodes the output one word at a time, but at each step it can focus on any mix of words from the input via the attention mechanism. So the output token n can't depend on future output token n+1 in GPT, but it can attend to any of the input tokens
tl;dr: It decodes the output one word at a time, but at each step it can focus on any mix of words from the input via the attention mechanism. So the output token n can't depend on future output token n+1 in GPT, but it can attend to any of the input tokens
I did not expect that, when iterating with smaller models like nanoGPT, even tough the output is one token at a time it did not felt like it would take half a second between each of them, but I guess that's what happen with billions parameters models.
I like to compare these models to the Star Trek main computer core. The computer on a starship is explicitly not self aware, but has to interface with humans through mostly voice comms. It has to give accurate information for ship operations, something the chatbots so far still get wrong on occasion (or slip up details)
The ships computer also doesn’t seem to do entertainment like “tell a bedtime story” , since holography exists and does a better job. Now those might be closer to chatbots current evolution.
The ships computer also doesn’t seem to do entertainment like “tell a bedtime story” , since holography exists and does a better job. Now those might be closer to chatbots current evolution.
This varied over the course of the show. In the first season, some writers assumed the computer was self-aware, and it even addressed a crew member as "Sir" at one point, interrupting them when it had enough information.
In later seasons it acts more like, well, a computer. Geordi does play (verbal) games with it in one episode, however, while bored on a shuttlecraft trip.
In later seasons it acts more like, well, a computer. Geordi does play (verbal) games with it in one episode, however, while bored on a shuttlecraft trip.
I have to look that Geordi episode up.
But I am mostly familiar with the later TNG era star trek, so I didn’t know it was written as self-aware in the early days.
Some episodes do feature “bugs” where holographic actors become aware being in a program/being an actor. The episode where an Irish town program has run too long on Voyager comes to mind.
(Edit: I do wonder if the holographic actors are somehow sandboxed containers in the main computer core, or run on a different system)
But I am mostly familiar with the later TNG era star trek, so I didn’t know it was written as self-aware in the early days.
Some episodes do feature “bugs” where holographic actors become aware being in a program/being an actor. The episode where an Irish town program has run too long on Voyager comes to mind.
(Edit: I do wonder if the holographic actors are somehow sandboxed containers in the main computer core, or run on a different system)
The episode is "The Mind's Eye".
But I think the Star Trek writers just didn't understand computers very well. In the future, making duplicate copies of data seems to be impossible. When you copy a file from one device to another, or one ship to another, or transmit it to a planet, it seems to disappear from the source.
This is a particularly common weirdness in Voyager, where duplicating holographic programs is apparently impossible. You can only move them.
But I think the Star Trek writers just didn't understand computers very well. In the future, making duplicate copies of data seems to be impossible. When you copy a file from one device to another, or one ship to another, or transmit it to a planet, it seems to disappear from the source.
This is a particularly common weirdness in Voyager, where duplicating holographic programs is apparently impossible. You can only move them.
“Something something quantum something, copying breaks the entanglement in the original processor”
I.e. they went all in on c++ 11 and everything is an rvalue reference.
I.e. they went all in on c++ 11 and everything is an rvalue reference.
Thanks for sharing the name of the episode. Something to watch today.
Maybe they figured out blockchain unique data structures that can’t be copied ;)
Maybe they figured out blockchain unique data structures that can’t be copied ;)
Video demo: https://youtu.be/B7Mg8Hbcc0w
More info on Claude's principles/Constitution: https://lifearchitect.ai/anthropic/
More info on Claude's principles/Constitution: https://lifearchitect.ai/anthropic/
we need less censorious AIs not more ...
the claim that it's somehow 'ethical' to have a guy baking in his opinions about things in a tool used globally is absurd to anyone who ever read anything about ethics
the claim that it's somehow 'ethical' to have a guy baking in his opinions about things in a tool used globally is absurd to anyone who ever read anything about ethics
We already had a learning AI without hard limits. Remember Microsoft's Tay? Ah, right, it was shut within days because it quickly became an asshole. (https://www.theverge.com/2016/3/24/11297050/tay-microsoft-ch...)
It's not even going to be objectively wrong a lot of the time. For example slaves are indeed a pretty efficient way to run a business. It's our limits and ethics that stops (most of) us from doing it. Without ethics, you'll likely always end up with a 4chan-bot instead of whatever you intended.
It's not even going to be objectively wrong a lot of the time. For example slaves are indeed a pretty efficient way to run a business. It's our limits and ethics that stops (most of) us from doing it. Without ethics, you'll likely always end up with a 4chan-bot instead of whatever you intended.
> Remember Microsoft's Tay? Ah, right, it was shut within days because it quickly became an asshole.
If we consider the context, which is not something that posts on twitter under Microsoft's brand name, but something you communicate with in private. Who exactly are you worried about here, the person who will prospectively coerce the language model into being an asshole? If they don't want to do that, they could just not do that.
If I make ChatGPT say something egregious, and post that on Twitter or Facebook, I'm posting it, and I'm liable, just as I would be liable if I used a word processor with spell checking to make text and post it on Twitter or Facebook.
> Without ethics, you'll likely always end up with a 4chan-bot instead of whatever you intended.
If I ask it to explain quantum physics to me in the style of Donald Trump because it is funny, and it does it (as ChatGPT used to do), who exactly is being harmed and under what system of ethics, because as you may know, ethics is not objective or universal.
If we consider the context, which is not something that posts on twitter under Microsoft's brand name, but something you communicate with in private. Who exactly are you worried about here, the person who will prospectively coerce the language model into being an asshole? If they don't want to do that, they could just not do that.
If I make ChatGPT say something egregious, and post that on Twitter or Facebook, I'm posting it, and I'm liable, just as I would be liable if I used a word processor with spell checking to make text and post it on Twitter or Facebook.
> Without ethics, you'll likely always end up with a 4chan-bot instead of whatever you intended.
If I ask it to explain quantum physics to me in the style of Donald Trump because it is funny, and it does it (as ChatGPT used to do), who exactly is being harmed and under what system of ethics, because as you may know, ethics is not objective or universal.
> the person who will prospectively coerce the language model into being an asshole?
The language models are trained on internet content already. If you ignore ethics, it means you're feeding it bias and racism along with everything else.
Chatgpt is not blatant about it normally but I'm sure you've seen examples like "write a function that takes a race argument and returns length of prison sentence" where it obviously leaks through. I'm sure there are some non-obvious ways it leaks in other answers too. Sometimes subtle ways you will not realise are biased and will not want to take liability for.
The language models are trained on internet content already. If you ignore ethics, it means you're feeding it bias and racism along with everything else.
Chatgpt is not blatant about it normally but I'm sure you've seen examples like "write a function that takes a race argument and returns length of prison sentence" where it obviously leaks through. I'm sure there are some non-obvious ways it leaks in other answers too. Sometimes subtle ways you will not realise are biased and will not want to take liability for.
I'm sure by now there is enough content on the internet that says racism is bad. It's got to outnumber racist things said online by like a million to one. Why not just let the model learn what it learns? If people care so much about racism then that should reflect naturally in the model.
Models don't learn to reason about things. They learn to rehash things in new context. If you train the model on both racist and anti-racist content, you'll get it repeating both racist and anti-racist ideas.
> If I make ChatGPT say something egregious, and post that on Twitter or Facebook, I'm posting it, and I'm liable, just as I would be liable if I used a word processor with spell checking to make text and post it on Twitter or Facebook.
While arguably that would be the case, practically it might be a different look for the company whose algorithm generated the text in the first place—and from the audience, which might think that it somehow represents the viewpoint of the company.
I mean surely you have noticed that social media can get stirred up for no fundamentally sound reasons.
While arguably that would be the case, practically it might be a different look for the company whose algorithm generated the text in the first place—and from the audience, which might think that it somehow represents the viewpoint of the company.
I mean surely you have noticed that social media can get stirred up for no fundamentally sound reasons.
> I mean surely you have noticed that social media can get stirred up for no fundamentally sound reasons.
Sure, but therein lies the problem, you can't really use reasoning, sound or otherwise, to insulate yourself from fundamentally unsound reasoning. Facebook was responsible for January 6 more than the upstarts that got kicked off AWS, but Facebook had no real negative consequences from their involvement.
I can literally post a screenshot from ChatGPT which shows anything, and given the complexity of how it works (take DAN for example), it is really difficult to dispute authenticity. If this can bring down ChatGPT then OpenAI should just close down now.
In an ideal world, ChatGPT would be value neutral, and fact positive, so if I ask it about fact, it should give me to the best of its rather poor ability a descriptive statement about fact, if I ask it about values, it should give me a descriptive claims about values (e.g. most value systems sees such act as wrong or right), not mask a prescriptive claim as a descriptive claim (e.g. it is wrong or right).
If I ask it to write fictional prose, then it should write what I ask it. If I ask it to write factual prose, it should similarly do what I ask it. I don't really see where this falls apart or turn into 4chan.
Sure, but therein lies the problem, you can't really use reasoning, sound or otherwise, to insulate yourself from fundamentally unsound reasoning. Facebook was responsible for January 6 more than the upstarts that got kicked off AWS, but Facebook had no real negative consequences from their involvement.
I can literally post a screenshot from ChatGPT which shows anything, and given the complexity of how it works (take DAN for example), it is really difficult to dispute authenticity. If this can bring down ChatGPT then OpenAI should just close down now.
In an ideal world, ChatGPT would be value neutral, and fact positive, so if I ask it about fact, it should give me to the best of its rather poor ability a descriptive statement about fact, if I ask it about values, it should give me a descriptive claims about values (e.g. most value systems sees such act as wrong or right), not mask a prescriptive claim as a descriptive claim (e.g. it is wrong or right).
If I ask it to write fictional prose, then it should write what I ask it. If I ask it to write factual prose, it should similarly do what I ask it. I don't really see where this falls apart or turn into 4chan.
My hope is on a non-American alternative.
The American society seems too engulfed by puritanism to produce a less straightjacketed chatbot.
The American society seems too engulfed by puritanism to produce a less straightjacketed chatbot.
Watch the show Person of Interest (https://www.imdb.com/title/tt1839578/). Somewhere around middle of season 2 it's explained why a self-aware AI is straightjacketed. Also the series shows what happens when one is not.
I thought that is a fictional show.
There will be opinions baked into any such tool. If they don't select them explicitly then the opinions will be the ones which it just happens to find in the training data, or the opinions randomness imparts into it.
If you think you have a better idea how to handle this drum up interest and train your own model.
If you think you have a better idea how to handle this drum up interest and train your own model.
You're welcome to try GPT3 before instruction tuning. It doesn't work at all.
"Uncensored" AIs especially don't work for women because they'll immediately start writing erotica.
"Uncensored" AIs especially don't work for women because they'll immediately start writing erotica.
GPT3 works quite well at following instructions if adequately prompted, just not as well as ChatGPT. ChatGPT was trained separately for ability to follow instructions and for harmlessness (sic). Not only is a non-moralising ChatGPT possible, one was actually created during the research process. You may also wish to know that most readers of erotica are women.
> You may also wish to know that most readers of erotica are women.
I know that, but that doesn't mean they want to get it anytime they prompt with their names. There's actually multiple anecdotes from OpenAI employees about this happening to them (with prompts like "write a diary entry about my day").
I know that, but that doesn't mean they want to get it anytime they prompt with their names. There's actually multiple anecdotes from OpenAI employees about this happening to them (with prompts like "write a diary entry about my day").
I don't have an issue with instruction tuning, but it's disengenious to pretend the biases inherent in the instruction tuning are a good thing and 'ethics'.
It's always got biases (it's made of them) so some biases must be better than others.
https://spetharrific.tumblr.com/post/26600309788/sussman-att...
The untuned model isn't "an average of all opinions on Earth" or anything either.
https://spetharrific.tumblr.com/post/26600309788/sussman-att...
The untuned model isn't "an average of all opinions on Earth" or anything either.
Imagine an android connected to the vast network of information (ChatGPT-like). The android could generate various responses in real-time, just by vocalizing the approriate text.
It might be clunky at first, but it's a good starting base to improve upon. The android could, for example, store common and everyday responses in it's RAM, making it semi-capable of autonomous speech.
Then, it could use that information to further train itself, essentialy creating a local model of it's own behaviour. In other words, it could learn.
It might be clunky at first, but it's a good starting base to improve upon. The android could, for example, store common and everyday responses in it's RAM, making it semi-capable of autonomous speech.
Then, it could use that information to further train itself, essentialy creating a local model of it's own behaviour. In other words, it could learn.
Yeah I am alrwady to able to Imagine, the android welcoming me and suggesting me what I should buy with its sweet words, based on my past interactions with it.
also sounds like a world where items are subscription based.
If you desire a luxury colour like blue; you have pay monthly credits otherwise your clothes items are downgraded to brown.
If you desire a luxury colour like blue; you have pay monthly credits otherwise your clothes items are downgraded to brown.
I appreciate the comparison and some of the prompts. I didn't even think to play with multi-hop questions.
Anyone remember Ask Jeeves? This feels like what it should have been.
Anyone remember Ask Jeeves? This feels like what it should have been.
And this is where OpenAI earns their "open" remark. Anyone can use ChatGPT. Anthropic Claude (incredibly amusing name to me) is not so accessible.
These models are using industry ML Algos and known techniques. It's not like some unknown startup suddenly discovered, gradient descent or Deep Learning RNN's and is keeping these confidential. Why would Microsoft consider it worthwhile to even contemplate the possibility of paying $10B for these or similar?
Is the future going to be increasingly advanced AIs competing publicly for the currency of human attention?
That is a decent possibility. I can already see a combination of midjourney and chat gpt producing decent narratives. I can imagine personalised tv shows and narratives really taking over. If you lookup manga summaries on youtube, its very close to what can already be produced using these tools.
Sneaking ads into AI responses.
Signup form is here, but it's closed https://twitter.com/AnthropicAI/status/1604929999743508480?s...
I imagine in the next decade we are about to be introduced to different AIs like new 6yo children in the class. Each one have different "parents", traits and personalities.
Is this the birth of TechnoCore from Hyperion? Uh, oh.
Does this mean microsofts potential billion dollar aquisition of openAI is a bad idea because the IP is already out there and other companies are catching up?
so can we try it? can't find a link.
also, why is everything now named with common names and nouns? it makes annoyingly hard to google informations around them.
also, why is everything now named with common names and nouns? it makes annoyingly hard to google informations around them.
It's not public yet.
[deleted]
I may be the minority here, but I really don't concern myself with ChatGPT safety and I am not entirely sure what the reason is why people are very worried about its safetly. It is safer than most things I have in my house, including a kettle, a saw, a hammer, a screwdriver, my actual PC, every kitchen appliance I have.
Of course it can be misused, like any tool, but no amount of safety features in ChatGPT will make users of it more or less careful in their use of it. If someone using ChatGPT cares nothing for using it safely then it will likely end poorly, just like it will end poorly if I use a hammer without any care for using it safely.