HackerTrans
TopNewTrendsCommentsPastAskShowJobs

ollin

643 karmajoined há 9 anos
Independent researcher working on real-time on-device generative models. Made sdxl-vae-fp16-fix, TAESD/TAEHV, pokemon-emulation-via-dnn.

https://madebyoll.in https://www.neuralworlds.net

Submissions

The Singularity Is the Friends We Made Along the Way

geohot.github.io
2 points·by ollin·há 2 meses·0 comments

comments

ollin
·há 3 dias·discuss
[dead]
ollin
·há 2 meses·discuss
Yeah, that part should not be read literally; `x.cos().cos()` and `x1 = x.cos(); x2 = x1.cos()` both launch the same number of kernels (two in unfused/eager mode, one in fused/torch.compile, see this test notebook [1]). I think the author chained the two cos calls to symbolize the idea of combining them (without exposing the intermediate result), but chaining the two cos calls doesn't literally trigger operator fusion.

[1] https://colab.research.google.com/drive/13a4Y-ko6QLMPAhBz64c...
ollin
·há 2 meses·discuss
This post is a classic! Also recommended: Horace also gave a related talk (covering the high-level picture of modern ML Systems) at Jane Street in Dec 2024 https://www.youtube.com/watch?v=139UPjoq7Kw
ollin
·há 2 meses·discuss
Feet of Clay is one of my favorites in the series! It's surprising how literally the Discworld version of Golems corresponds to modern LLMs (and perhaps upcoming LLM-backed humanoids?).

The Golems are brought to life by a slip of words in their heads called chem, which is almost 1:1 to an LLM system prompt (or perhaps the Claude Soul Document):

    I AM A GOLEM. I WAS MADE OF CLAY. MY LIFE IS THE WORDS. BY MEANS OF WORDS OF PURPOSE IN MY HEAD I ACQUIRE LIFE. MY LIFE IS TO WORK. I OBEY ALL COMMANDS. I TAKE NO REST.
The Golems are perfectly intelligent and self-aware, but since they don't exhibit independent goals beyond their prompt, they get treated as appliances rather than as sentient creatures.

    “What words of purpose?”
    RELEVANT TEXT THAT ARE THE FOCUS OF BELIEF. GOLEM MUST WORK. GOLEM MUST HAVE A MASTER.
    “Sorry, look,” said Cheery. “Are you telling me this… thing is powered by words? I mean… is *it* telling me it’s powered by words?”
    “Why not? Words do have power. Everyone knows that,” said Angua. “There are more golems around than you might think. They’re out of fashion now, but they last. They can work underwater, or in total darkness, or knee-deep in poison. For years. They don’t need rest or feeding. They…”
    “But that’s slavery!” said Cheery.
    “Of course it isn’t. You might as well enslave a doorknob.”
The integration of more (and more-independent) Golems into society is gradual and controversial, per Making Money:

    There was another protest march going on when Moist walked to the bank. You got more and more of them lately.
    This march was against the employment of golems, who uncomplainingly did the dirtiest jobs, worked around the clock, and were so honest they paid their taxes. But they weren’t human and they had glowing eyes, and people could get touchy about that sort of thing.
ollin
·há 2 meses·discuss
Specifically it looks like he's planning to extend the ideas from https://github.com/karpathy/autoresearch into a larger effort towards recursive training improvement [1]:

> Excited to welcome Andrej to the Pretraining team! He'll be building a team focused on using Claude to accelerate pretraining research itself. I can’t think of anyone better suited to do it — looking forward to what we build together!

[1] https://x.com/nickevanjoseph/status/2056760504949842219
ollin
·há 2 meses·discuss
This is cute and retro! But I think training only on GoldenEye undersells the concept a bit, since their world model inherits the N64-era graphics from GoldenEye, which automatically makes it look dated.

If they retrained the same model on real video data, they could potentially get a multiplayer world with quite realistic-looking graphics (see https://wayve.ai/wp-content/uploads/2025/11/Ex-2-GAIA-3.mp4, https://wayve.ai/wp-content/uploads/2025/11/Ex-3-GAIA-3.mp4).

Maybe for Agora-2 :)
ollin
·há 2 meses·discuss
Right now there is (AFAIK) no world model product booking any meaningful revenue. So there's a decent chance WMs turn out to have no long-term utility at all.

However, there are a few promising markets, assuming WMs continue to get better and cheaper:

1. Robotics training / evaluation: modern end-to-end (sensors-to-control) robot policies require simulators that are almost indistinguishable from reality. If your sim is distinguishable from reality, the evaluation metrics you get from sim don't mean anything and the policies you train in sim don't work. World models will likely be the highest-fidelity robotics simulators, since WMs are data-driven and get arbitrarily more-realistic given more data/compute. This is why so many robotics companies have WM projects [1] [2] [3] [4].

2. Video frontends for agents: in the same way that today's frontier labs are building realtime voice interfaces [5] which behave like a phone call, realtime video interfaces will behave like a video call. Early forms of this don't feel compelling IMO [6] [7], but once the models can instantly blend between rendering the agent itself, drawing diagrams/visualizations, rendering video, etc. I can see it surpassing pure voice mode.

3. Entertainment: zero-shot world generation (i.e. holodeck, genie 3; paste in an image/video/text prompt and get a world) will be a fun toy but I'm not convinced it has any long-term value. I'm more optimistic about proper narrative experiences where each scene/level is a small, carefully-crafted world (behaving like a normal film scene if you don't touch the controls, and an uncharted/TLoU-style narrative game if you do), such that the sequence of scenes builds up a larger story.

[1] https://wayve.ai/thinking/gaia-3/

[2] https://xcancel.com/Tesla/status/1982255564974641628 / https://xcancel.com/ProfKuang/status/1996642397204394179

[3] https://waymo.com/blog/2026/02/the-waymo-world-model-a-new-f...

[4] https://www.1x.tech/discover/world-model-self-learning

[5] https://thinkingmachines.ai/blog/interaction-models/

[6] https://runwayml.com/news/introducing-runway-characters

[7] https://blog.character.ai/character-ais-real-time-video-brea...
ollin
·há 2 meses·discuss
This is a HW4 Tesla on FSD 14.3.2 trying to drive into a lake five days ago (a la The Office): https://www.reddit.com/r/TeslaFSD/comments/1t9rl2u/fsd_tried..., so I would not say Tesla has solved standing water yet.

That said, FSD seems quite capable of routing around standing water in many cases (e.g. https://xcancel.com/planoken/status/2030754820462633031, https://www.reddit.com/r/TeslaFSD/comments/1pw9f2m/fsd_navig..., https://xcancel.com/BLKMDL3/status/1991862465328779317, https://xcancel.com/JVTacoma/status/2046313902749921638), so handling the remaining cases seems more like a model intelligence / data issue rather than a sensor limitation. Lidar beams generally bounce off mirrorlike surfaces without returning to the sensor, so I think all lidar would tell you about standing water is "there's something shiny/reflective within this region of the image", which you already know from cameras+headlights.
ollin
·há 2 meses·discuss
Most ONNX files are fp32, but the ONNX format actually allows fp16, int8, etc. as well (see onnx.proto for the full list of dtypes [1] - they even have fp8/fp4 these days!). I ended up switching over to fp16 ONNX models for my own web-based inference project since the quality is ~identical and page loads get 2x faster.

[1] https://github.com/onnx/onnx/blob/main/onnx/onnx.proto#L605
ollin
·há 2 meses·discuss
The source here is "CAISI Evaluation of DeepSeek V4 Pro" [1]; the US NIST ran their own benchmarks (including several internal ones) and reported the following table:

    | Domain               | Benchmark              | Model (reasoning level) |                             |                          |                       |
    |--:-------------------|------------------------|-------------------------|-----------------------------|--------------------------|-----------------------|
    |                      |                        | OpenAI GPT-5.5 (xhigh)  | OpenAI GPT-5.4 mini (xhigh) | Anthropic Opus 4.6 (max) | DeepSeek V4 Pro (max) |
    | Cyber                | CTF-Archive-Diamond    | **71%**                 | 32%                         | 46%                      | 32%                   |
    | Software Engineering | SWE-Bench Verified*    | **81%**                 | 73%                         | 79%                      | 74%                   |
    |                      | PortBench              | **78%**                 | 41%                         | 60%                      | 44%                   |
    | Natural Sciences     | FrontierScience        | **79%**                 | 74%                         | 72%                      | 74%                   |
    |                      | GPQA-Diamond           | **96%**                 | 87%                         | 91%                      | 90%                   |
    | Abstract Reasoning   | ARC-AGI-2 semi-private | **79%**                 | –                           | 63%                      | 46%                   |
    | Mathematics          | OTIS-AIME-2025         | **100%**                | 90%                         | 92%                      | 97%                   |
    |                      | PUMaC 2024             | **96%**                 | 93%                         | 95%                      | **96%**               |
    |                      | SMT 2025               | **99%**                 | 92%                         | 94%                      | 96%                   |
    | IRT-Estimated Elo    | **IRT-Estimated Elo**  | **1260 ± 28**           | 749 ± 46                    | 999 ± 27                 | 800 ± 28              |
Notably, two of the benchmarks with the biggest capability gap are CAISI-internal/private ones (CTF-Archive-Diamond, PortBench). I read this as "DeepSeek is well-tuned for public benchmarks, and less generally intelligent than GPT5.5 on held-out tasks" but a less-charitable reading would be "US government reports US models do best on benchmarks that only the US government can run". Agent benchmarking is fraught with peril [2] and an impartial benchmarker (who disproportionately overlooks bugs/issues in their evaluation of certain models) can absolutely tilt the scales, so I would not be surprised if a PRC-led benchmarking of frontier models came to the opposite conclusion.

[1] https://www.nist.gov/news-events/news/2026/05/caisi-evaluati...

[2] https://epoch.ai/gradient-updates/why-benchmarking-is-hard
ollin
·há 2 meses·discuss
Hank Green has a video walking through how to use the timeline here https://www.youtube.com/watch?v=LyZE9VWJjDA. For me, the best experience was to click "Crew Photos Only" and then step through the photos chronologically with the arrow buttons.
ollin
·há 2 meses·discuss
For context, two days ago some users [1] discovered this sentence reiterated throughout the codex 5.5 system prompt [2]:

> Never talk about goblins, gremlins, raccoons, trolls, ogres, pigeons, or other animals or creatures unless it is absolutely and unambiguously relevant to the user's query.

[1] https://x.com/arb8020/status/2048958391637401718

[2] https://github.com/openai/codex/blob/main/codex-rs/models-ma...
ollin
·há 2 meses·discuss
AFAIK Anthropic hasn't built any image or video generation tools yet, just text/code generation. OpenAI/Google/xAI all built image/video generation teams though so it may only be a matter of time.
ollin
·há 3 meses·discuss
- The OpenBSD one is 'TCP packets with invalid SACK options could crash the kernel' https://cdn.openbsd.org/pub/OpenBSD/patches/7.8/common/025_s...

- One (patched) Linux kernel bug is 'UaF when sys_futex_requeue() is used with different flags' https://github.com/torvalds/linux/commit/e2f78c7ec1655fedd94...

These links are from the more-detailed 'Assessing Claude Mythos Preview’s cybersecurity capabilities' post released today https://red.anthropic.com/2026/mythos-preview/, which includes more detail on some of the public/fixed issues (like the OpenBSD one) as well as hashes for several unreleased reports and PoCs.
ollin
·há 3 meses·discuss
My impression was entirely the opposite; the unsolved subset of SWE-bench verified problems are memorizable (solutions are pulled from public GitHub repos) and the evaluators are often so brittle or disconnected from the problem statement that the only way to pass is to regurgitate a memorized solution.

OpenAI had a whole post about this, where they recommended switching to SWE-bench Pro as a better (but still imperfect) benchmark:

https://openai.com/index/why-we-no-longer-evaluate-swe-bench...

> We audited a 27.6% subset of the dataset that models often failed to solve and found that at least 59.4% of the audited problems have flawed test cases that reject functionally correct submissions

> SWE-bench problems are sourced from open-source repositories many model providers use for training purposes. In our analysis we found that all frontier models we tested were able to reproduce the original, human-written bug fix

> improvements on SWE-bench Verified no longer reflect meaningful improvements in models’ real-world software development abilities. Instead, they increasingly reflect how much the model was exposed to the benchmark at training time

> We’re building new, uncontaminated evaluations to better track coding capabilities, and we think this is an important area to focus on for the wider research community. Until we have those, OpenAI recommends reporting results for SWE-bench Pro.
ollin
·há 4 meses·discuss
Here was the developer thread https://developer.apple.com/forums/thread/818403 I found with lots of other reports of "Unable to Verify App - An internet connection is required to verify the trust of the developer".

Although https://developer.apple.com/system-status/ was green for most of the 3-4 hour outage, the page now at least acknowledges two minutes of downtime:

    App Store Connect - Resolved Outage
    Today, 12:04 AM - 12:06 AM
    All users were affected
    Users experienced a problem with this service.
Not a great developer experience.
ollin
·há 5 meses·discuss
The still photo (with 富士康科技 photoshopped out) is the second image of the "In Houston, workers assemble advanced AI servers" photo carousel https://www.apple.com/newsroom/images/2026/02/apple-accelera...
ollin
·há 5 meses·discuss
A lot of people mentioned this! The "dreamlike" comparison is common as well. In both cases, you have a network of neurons rendering an image approximating the real world :) so it sort of makes sense.

Regarding the specific boiling-textures effect: there's a tradeoff in recurrent world models between jittering (constantly regenerating fine details to avoid accumulating error) and drifting (propagating fine details as-is, even when that leads to accumulating error and a simplified/oversaturated/implausible result). The forest trail world is tuned way towards jittering (you can pause with `p` and step frame-by-frame with `.` to see this). So if the effect resembles LSD, it's possible that LSD applies some similar random jitter/perturbation to the neurons within your visual cortex.
ollin
·há 5 meses·discuss
On a technical level, this looks like the same diffusion transformer world model design that was shown in the Genie 3 post (text/memory/d-pad input, video output, 60sec max context, 720p, sub-10FPS control latency due to 4-frame temporal compression). I expect the public release uses a cheaper step-distilled / quantized version. The limitations seen in Genie 3 (high control latency, gradual loss of detail and drift towards videogamey behavior, 60s max rollout length) are still present. The editing/sharing tools, latency, cost, etc. can probably improve over time with this same model checkpoint, but new features like audio input/output, higher resolution, precise controls, etc. likely won't happen until the next major version.

From a product perspective, I still don't have a good sense of what the market for WMs will look like. There's a tension between serious commercial applications (robotics, VFX, gamedev, etc. where you want way, way higher fidelity and very precise controllability), vs current short-form-demos-for-consumer-entertainment application (where you want the inference to be cheap-enough-to-be-ad-supported and simple/intuitive to use). Framing Genie as a "prototype" inside their most expensive AI plan makes a lot of sense while GDM figures out how to target the product commercially.

On a personal level, since I'm also working on world models (albeit very small local ones https://news.ycombinator.com/item?id=43798757), my main thought is "oh boy, lots of work to do". If everyone starts expecting Genie 3 quality, local WMs need to become a lot better :)
ollin
·há 5 meses·discuss
Yup, similar concepts! Just at two opposite extremes of the compute/scaling spectrum.

- That forest trail world is ~5 million parameters, trained on 15 minutes of video, scoped to run on a five-year-old iPhone through a twenty-year old API (WebGL GPGPU, i.e OpenGL fragment shaders). It's the smallest '3D' world model I'm aware of.

- Genie 3 is (most likely) ~100 billion parameters trained on millions of hours of video and running across multiple TPUs. I would be shocked if it's not the largest-scale world model available to the public.

There are lots of neat intermediate-scale world models being developed as well (e.g. LingBot-World https://github.com/robbyant/lingbot-world, Waypoint 1 https://huggingface.co/blog/waypoint-1) so I expect we'll be able to play something of Genie quality locally on gaming GPUs within a year or two.