nonhaver · HackerTrans

vor 11 Monatendiscuss

ill leave it at this: if “zero-hallucination omniscience” is your bar, you’ll stay disappointed - and that’s on your expectations, not the tech. personally i’ve been coding/researching faster and with fewer retries every time a new model drops - so my opinion is based on experience. you’re free to sit out the upgrade cycle

nonhaver

vor 11 Monatendiscuss

not to offend - but it sounds like your response/worries are based more on an emotional reaction. and rightly so, this is by all means a very scary and uncertain time. and undeniably these companies have not taken into account the impact their products will cause and the safety surrounding that.

however, a lot of your claims are false - progress is being made in nearly all the areas you mentioned

> hallucinations

are reduced with GPT-5

https://cdn.openai.com/pdf/8124a3ce-ab78-4f06-96eb-49ea29ffb...

"gpt-5-thinking has a hallucination rate 65% smaller than OpenAI o3"

> limited context window

same deal. gemini 2.5-pro has a 1 million token context window and GPT-5 is 400k up from 200k with o3

https://blog.google/technology/google-deepmind/gemini-model-...

"native multimodality and a long context window. 2.5 Pro ships today with a 1 million token context window (2 million coming soon)"

> expensive to operate and train

we don't know for certain but GPT-5 provides the most intelligence for the cheapest price at $10/1 million output tokens which is unprecedented

https://platform.openai.com/docs/models/gpt-5

> guardrails

are very well implemented in certain models like google who provide multiple safety levels

https://ai.google.dev/gemini-api/docs/safety-settings

"You can use these filters to adjust what's appropriate for your use case. For example, if you're building video game dialogue, you may deem it acceptable to allow more content that's rated as Dangerous due to the nature of the game. In addition to the adjustable safety filters, the Gemini API has built-in protections against core harms, such as content that endangers child safety. These types of harm are always blocked and cannot be adjusted."

now id like to ask you for evidence that none of these aspects have been improved - since you claim my examples are vague but make statements like

> Inability to recall simple information

> inability to stay on task

> (doesn't) support its output

> (no) long term planning

ive experienced the exact opposite. not 100% of the time but compared to GPT-4 all of these areas have been massively improved. sorry i cant provide every single chat log ive ever had with these models to satisfy your vagueness-o-meter or provide benchmarks which i assume you will brush aside.

as well as the examples ive provided above - you seem to be making claims out of thin air and then claim others are not providing examples up to your standard.

nonhaver

vor 11 Monatendiscuss

i didnt bring examples because i said personal experience. heres my "evidence" - gpt 4 took multiple shots and iterations and couldnt stay coherent with a prompt longer than 20k tokens (in my experience). then when o4 came out it improved on that (in my experience). o1 took 1-2 shots with less iterations (in my experience). o3 zero shots most of the tasks i throw at it and stays coherent with very long prompts (in my experience).

heres something else to think about. try and tell everybody to go back to using gpt-4. then try and tell people to go back to using o1-full. you likely wont find any takers. its almost like the newer models are improved and generally more useful

nonhaver

vor 11 Monatendiscuss

you dont remember deepseek introducing reasoning and blowing benchmarks led by private american companies out of the water? with an api that was way cheaper? and then offered the model free in a chat based system online? and you were a big fan?

nonhaver

vor 11 Monatendiscuss

this is a very odd perspective. as someone who uses LLMs for coding/PRs - every time a new model released my personal experience was that it was a very solid improvement on the previous generation and not just meant to "confuse". the jump from raw GPT-4 2 years ago to o3 full is so unbelievable if you traveled back in time and showed me i wouldn't have thought such technology would exist for 5+ years.

to the point on hallucination - that's just the nature of LLMs (and humans to some extent). without new architectures or fact checking world models in place i don't think that problem will be solved anytime soon. but it seems gpt-5 main selling point is they somehow reduced the hallucination rate by a lot + search helps with grounding.

nonhaver

vor 11 Monatendiscuss

i think this is more an effect of releasing a model every other month with gradual improvements. if there was no o-series/other thinking models on the market - people would be shocked by this upgrade. the only way to keep up with the market is to release improvements asap

nonhaver

vor 11 Monatendiscuss

also wondering this. had to pause the livestream to make sure i wasnt crazy. definitely eyebrow raising

comments