ChatGPT O3 Preview Announcement(openai.com)
openai.com
ChatGPT O3 Preview Announcement
https://openai.com/12-days/
42 comments
Since OpenAI bills you for tokens expended on chain-of-thought, I assume this "deliberative alignment" has the funny side-effect of making you pay for the time the model spends ruminating on whether it should refuse to do what you asked it to do. Talk about adding insult to injury.
Maybe the most convincing argument yet that this is similar to how humans think.
Okay, but is this similar to how humans bill you?
Yes, a lawyer could refuse your case after consulting them and their time considering the case.
How often does this happen? I’ve consulted lawyers for an hour who refused to support my case , and was billed nothing
At the lowest end of the industry it’s common to get free consultations because it’s too much expense for individuals otherwise, but in corporate law or higher end services I believe it’s much less common.
I’ve been to a lawyer once, and this happened. I got billed and told to go to a different lawyer.
and sometimes they charge $1k per hour and still not have solutions after that hour
The point of technology is to make our lives easier and cheaper. There's no point to have chatgpt to charge this way other than to make the capitalist owners more money.
Otherwise the self-serve machines at Mcdonalds should cost the company the same as an employee with a fair salary. Mass assembly lines building cars and machines should cost the same as 100+ workers including costs of health benefits and breaks.
Otherwise the self-serve machines at Mcdonalds should cost the company the same as an employee with a fair salary. Mass assembly lines building cars and machines should cost the same as 100+ workers including costs of health benefits and breaks.
I’d imagine the point is for you to pay for resources used… considering your prompt still costs money.
I don’t really see this any differently from an API billing you per call and still billing you for calls that where you pass in wrong credentials and get unauthorized error responses
I don't think I've ever seen an API bill for incorrect credentials, or even anything that returns a non-200?
So … hypothetically, I Spent $5 on tokens to figure out the answer - then got an answer that my answer is blocked !
using the service is optional and they're clear about how pricing works, though
one can always refuse to use
one can always refuse to use
This video format is killing me. Would love to keep up with the announcement but I have no desire for 10-20 minute videos.
AI video summary at your service:
In this video, Sam Altman and team introduce OpenAI o3 and o3-mini, showcasing their advanced reasoning capabilities and performance benchmarks compared to earlier models. The session highlights the models' robust accuracy in coding and mathematical tasks, as well as new safety testing initiatives involving community participation. The o3-mini is noted for its cost-efficient performance, while safety strategies are enhanced through deliberative alignment.
## Key Points
### Introduction to o3 and o3-mini models
During the final day of their 12-day event, OpenAI announces two new models: o3 and o3-mini. These models are positioned as advancements in AI reasoning capabilities, following the success of the earlier o1 model.
### Performance benchmarks
The o3 model achieves significant improvements in capabilities, scoring 71.7% on software benchmarks and reaching near-expert levels on competitive math exams. In comparison to o1, it shows over 20% improvement in coding tasks.
### Safety testing initiatives
OpenAI emphasizes safety testing for the new models, opening access for researchers to facilitate public testing. The goal is to ensure models are safe for general use while their capabilities are further validated.
### Introduction of o3-mini
The o3-mini is presented as a cost-efficient alternative to o3, also equipped with strong reasoning powers, allowing adjustable thinking time to optimize performance based on user needs.
### New safety techniques: deliberative alignment
A new safety training technique, deliberative alignment, uses reasoning capabilities of the models to create better safety benchmarks, enhancing the ability to accurately reject unsafe prompts.
### Future developments and availability
The video concludes with information on how to apply for early access to test these models with expectations of full public availability in early 2025, alongside a call for safety researchers to contribute.
In this video, Sam Altman and team introduce OpenAI o3 and o3-mini, showcasing their advanced reasoning capabilities and performance benchmarks compared to earlier models. The session highlights the models' robust accuracy in coding and mathematical tasks, as well as new safety testing initiatives involving community participation. The o3-mini is noted for its cost-efficient performance, while safety strategies are enhanced through deliberative alignment.
## Key Points
### Introduction to o3 and o3-mini models
During the final day of their 12-day event, OpenAI announces two new models: o3 and o3-mini. These models are positioned as advancements in AI reasoning capabilities, following the success of the earlier o1 model.
### Performance benchmarks
The o3 model achieves significant improvements in capabilities, scoring 71.7% on software benchmarks and reaching near-expert levels on competitive math exams. In comparison to o1, it shows over 20% improvement in coding tasks.
### Safety testing initiatives
OpenAI emphasizes safety testing for the new models, opening access for researchers to facilitate public testing. The goal is to ensure models are safe for general use while their capabilities are further validated.
### Introduction of o3-mini
The o3-mini is presented as a cost-efficient alternative to o3, also equipped with strong reasoning powers, allowing adjustable thinking time to optimize performance based on user needs.
### New safety techniques: deliberative alignment
A new safety training technique, deliberative alignment, uses reasoning capabilities of the models to create better safety benchmarks, enhancing the ability to accurately reject unsafe prompts.
### Future developments and availability
The video concludes with information on how to apply for early access to test these models with expectations of full public availability in early 2025, alongside a call for safety researchers to contribute.
the mini models are the most exciting to me. they seem to hit the balance of general utility and cost.
people are fond of comparing large bleeding edge models, but when we compare the small ones to say gpt3.5, it is still astonishing.
to me the smaller models being the balance of intelligence and cost is the best indicator of what the general population can use and afford.
the larger models end up being used mostly by individuals teams and orgs who can afford to pay for it and learn how to use it.
keep in mind at this day and age there is still a lot of people who dont know these things exist. it's just outside of their reality. then there are who have the slightest idea about it, but won't commit the time and money needed to learn it and fully experience it.
this is like the new generational transfer of wealth and information. even if you're an individual or a small team, with enough levers you can use these technologies to your advantage, and with more levers (like yc) you can enter the battlefield and compete with existing companies.
people are fond of comparing large bleeding edge models, but when we compare the small ones to say gpt3.5, it is still astonishing.
to me the smaller models being the balance of intelligence and cost is the best indicator of what the general population can use and afford.
the larger models end up being used mostly by individuals teams and orgs who can afford to pay for it and learn how to use it.
keep in mind at this day and age there is still a lot of people who dont know these things exist. it's just outside of their reality. then there are who have the slightest idea about it, but won't commit the time and money needed to learn it and fully experience it.
this is like the new generational transfer of wealth and information. even if you're an individual or a small team, with enough levers you can use these technologies to your advantage, and with more levers (like yc) you can enter the battlefield and compete with existing companies.
Can this “expert in math” do binary addition? o1 falls apart after 10 bits, which is easily memorizable.
Can this "expert in math" write a function that performs binary addition?
this is the way.
its internal mechanism is still statistical prediction of text in units of tokens. that math seemed lucid enough to be able to use functions.
its internal mechanism is still statistical prediction of text in units of tokens. that math seemed lucid enough to be able to use functions.
It’s like everyone suddenly forgot you can’t do O(n^2) compute in O(n) time.
Binary addition is O(n)
I meant this in the general case, not specifically binary addition. Also, returning an token by ChatGPT is technically an O(1) operation, so the same principle applies. Returning a computation answer of O(n_required_tokens) cannot be delivered in O(1) time without some sort of caching.
The issue isn’t performing the specific addition. Rather, you’re asking o1 to take n-bits of data and combine them according to some set of rules. Isn’t that what these models are supposed to excel at, following instructions? Binary addition is interesting because the memorization space grows at 2^n, which is impossible to memorize for moderate values of n.
It's a bit ironic that it got the availability date wrong no?
It’s typical I’d say. I corrected it now.
(It originally said “early 2024” instead of “early 2025”. The YouTube transcript says “around the end of January”.)
(It originally said “early 2024” instead of “early 2025”. The YouTube transcript says “around the end of January”.)
I strongly agree. I stopped checking their page for news after day 2 due to video only announcements.
Safety: not from causing you physical harm or risking property damage, since there are no guardrails against it confidently walking you through a flawed breadboard circuit that will cause components to explode or immediately catch fire. Then say
“You’re absolutely right. We shouldn’t have run 120v through that capacitor rated for 10v. Here’s an updated design that fixes the issue”, then proceed to explain the same thing with a resistor placed at random.
No we mean safety from our LLM saying something that pisses off a politician, or even worse… hurting someone’s /feelings/.
“You’re absolutely right. We shouldn’t have run 120v through that capacitor rated for 10v. Here’s an updated design that fixes the issue”, then proceed to explain the same thing with a resistor placed at random.
No we mean safety from our LLM saying something that pisses off a politician, or even worse… hurting someone’s /feelings/.
It is more accurate to say they are thinking in terms of “brand safety” or liability, that is, being safe from being blamed for making a mean robot.
Well, if one of OpenAI's LLMs started quoting Mein Kampf in its answers to people, would you laugh, or would you not laugh? That's the question.
your fallacy is: reductio ad hitlerum
> Reductio ad Hitlerum (Latin for "reduction to Hitler"), also known as playing the Nazi card, is an attempt to invalidate someone else's argument on the basis that the same idea was promoted or practised by Adolf Hitler or the Nazi Party.
reduction ad hitlerum =/= "mentioning hitler in any way"
Ok. Usually if someone suggests that you might laugh at a Mein Kampf quote in response to your argument, they are not doing it in good faith.
Did I miss it or did they say whether o3 is going to be available under ChatGPT Plus or Pro?
I watched the live stream and don't recall any mention of it.
It's not at that stage!
The title is very wrong, "ChatGPT O3" is not a thing, OpenAI's new o3 model is not even demoed in ChatGPT, they don't call it "O3".
The subtitle at the top is "o3 preview & call for safety researchers".
The web page title is "12 Days of OpenAI".
We’d really likt to use this today.
def should_upvote(headline: str) -> bool:
if "o3" in headline.lower():
return True
return False
return "o3" in headline.lower()
This is more painful than any downvote. Have an upvote.
These threads are my favorite kind of pedantic :)