I'm making a game engine based on dynamic signed distance fields (SDFs) [video](youtube.com)
youtube.com
I'm making a game engine based on dynamic signed distance fields (SDFs) [video]
https://www.youtube.com/watch?v=il-TXbn5iMA
80 comments
Seems you're not familiar with how game projects with a custom engine typically go. Let me elaborate on this - the steps involved are:
1. Create a custom game engine.
1. Create a custom game engine.
Why must you attack me personally like this?! ⠀(:
Heh, I'm as guilty of this myself. Here's one example of mine: https://github.com/nnevatie/paybacktime
"Is rust good for game development? Rust is good for everything. There are currently... 5 games written in rust. And 50 game engines."[0]
[0] https://www.youtube.com/watch?v=TGfQu0bQTKc
[0] https://www.youtube.com/watch?v=TGfQu0bQTKc
So similar to the average game project with an off the shelf engine, just with an additional fun step.
That's how the overwhelming majority of all game projects go, custom engine or not.
I am interested in the small series and also the game engine (if not covered in the series). I will read these, thank you for sharing.
Go Mike! Been seeing his progress for a while on Bluesky, I knew exactly (and who) what this was when I saw it on HN.
I was rendering-curious when we overlapped together at Figma. Mike was super patient and giving with his time, answering all my dumb questions and aiding with my Maker Week projects. Excited to see him take on something so ambitious next.
I was rendering-curious when we overlapped together at Figma. Mike was super patient and giving with his time, answering all my dumb questions and aiding with my Maker Week projects. Excited to see him take on something so ambitious next.
Really impressive work. He covers it at the end, but being able to create tunnels into terrain, walk through them and then make the terrain disappear. Or make holes in the ground and move them around to suck items in. Or dynamically erase or add to any terrain in real time. A lot of interesting gameplay opportunities here and surprisingly performant!
I really respect his work on the engine, the math, and the engineering is excellent, but I'm not sure it makes for an interesting game above and beyond what we already have.
Adding additional smoothness to existing voxel engines I'm not sure would have much effect, unless you have something specific like moving a ball on a smooth surface (but the SDF I'm seeing here wouldn't support that either).
As for "create a hole, then close it behind you", this is about as game-changing as open/closing doors (or tunnels with doors). I'm open to suggestion, but honestly this is amazing tech, I just don't think it will create very fun games.
Kind of feel the same about the demos I see of spherical or non-euclidean geometry games. Its very interesting, and impressive, but seems like it is an engine in search of a game.
Adding additional smoothness to existing voxel engines I'm not sure would have much effect, unless you have something specific like moving a ball on a smooth surface (but the SDF I'm seeing here wouldn't support that either).
As for "create a hole, then close it behind you", this is about as game-changing as open/closing doors (or tunnels with doors). I'm open to suggestion, but honestly this is amazing tech, I just don't think it will create very fun games.
Kind of feel the same about the demos I see of spherical or non-euclidean geometry games. Its very interesting, and impressive, but seems like it is an engine in search of a game.
Yeah that's what I felt watch that video.
HyperRogue pulled it off.
Maybe one other thing to consider -- I think its usually best to have a killer game idea that seems fun, then design around it (and select the proper engine), rather than simply "I want to build an engine with some capabilities, then I'll figure out what games to make later."
Have seen this with a lot of software "frameworks" (web, game, graphics, etc.). Nothing wrong with writing an amazing tech demo just for the hell of it, but then when it comes time to do "real world" tasks, the frameworks are often in search of a fitting problem.
Have seen this with a lot of software "frameworks" (web, game, graphics, etc.). Nothing wrong with writing an amazing tech demo just for the hell of it, but then when it comes time to do "real world" tasks, the frameworks are often in search of a fitting problem.
Great video. A new game engine powered by SDFs is the sort of thing I want to find out about. Not the next game in a long running franchise that looks the same as all the others that preceded it. One for the from-scratch game dev nerds like myself!
Tangentially related, but here's another game based on SDFs and geometry manipulation, for 4D movement: https://www.youtube.com/watch?v=yeCqfqE4ylk
Stylistically there are similarities...
Stylistically there are similarities...
The physics engine mentioned towards the end, Jolt Physics [0] is used in the frankly blockbuster games Horizon: Forbidden West and Death Stranding 2 and yet opens its description with
> Why create yet another physics engine? Firstly, it has been a personal learning project.
which is really rather wonderful and inspiring to see.
[0] https://github.com/jrouwe/JoltPhysics
> Why create yet another physics engine? Firstly, it has been a personal learning project.
which is really rather wonderful and inspiring to see.
[0] https://github.com/jrouwe/JoltPhysics
Its use in those games is no mere coincidence though, the creator of that physics engine, Jorrit Rouwé, has worked at Guerilla Games since the Killzone days.
https://jrouwe.nl/games.php
https://jrouwe.nl/games.php
It has also become the default physics engine in Godot.
Also increasingly well integrated into Godot.
I wonder if ReLU fields could help reduce cache grid resolution while improving reconstruction precision? See https://arxiv.org/abs/2205.10824
the problem with SDF engines is that you have to reinvent everything, as current pipelines rely on triangles.
That means:
- Software to model using SDF (like Womp)
- Technique to animate skeletons using SDFs
- Tool to procedural texture surfaces using SDFs
At least he solved the physics part, which is also complex.
And also, his way of carving is by instantiating new elements, which works for small carves, but if you plan to have lots of tunels, then the number of instances is going to skyrocket.
That means:
- Software to model using SDF (like Womp)
- Technique to animate skeletons using SDFs
- Tool to procedural texture surfaces using SDFs
At least he solved the physics part, which is also complex.
And also, his way of carving is by instantiating new elements, which works for small carves, but if you plan to have lots of tunels, then the number of instances is going to skyrocket.
Although a decent chunk of modern tooling is there to handle the limitations of triangles. And modelling is often using higher-level abstractions that are only turned into triangles at the end of the process.
That's true if you're using a CAD-like tool, but that's typically not used for art (more for engineering / mechanical design)
Game / VFX artists heavily use mesh-based tools such as Maya or Blender.
Game / VFX artists heavily use mesh-based tools such as Maya or Blender.
Both have many tools that aren't raw triangle editing
Almost every 3D game uses textured polygons almost everywhere (except sometimes for fog or clouds), so this SDF engine is nice to see.
However, he doesn't mention animations, especially skeletal animations. Those tend to work poorly or not at all without polygons. PS4 Dreams, another SDF engine, also had strong limitations with regards to animation. I hope he can figure something out, though perhaps his game project doesn't need animation anyway.
However, he doesn't mention animations, especially skeletal animations. Those tend to work poorly or not at all without polygons. PS4 Dreams, another SDF engine, also had strong limitations with regards to animation. I hope he can figure something out, though perhaps his game project doesn't need animation anyway.
His SDF probably puts out a depth buffer, so with some effort (shadows might be hard?) you can just mix it with traditional polygons. The same way raytracing and polygons mix in AAA games.
He's using the SDFs to fill a space sort of like Unreal's Nanite virtual geometry. Nanite also doesn't support general animation. They only recently added support for foliage. So you'd use SDF / Nanite for your "infinite detail" / kit-bashing individual pebbles all the way to the horizon, and then draw polygon characters and props on top of that.
In fact I was surprised to see that Nanite flipped from triangle supremacy to using voxels in their new foliage tech. So maybe the two technologies will converge. The guy who did the initial research for Nanite (his talk also cites Dreams ofc) said that voxels weren't practical. But I guess they hit the limits of what they can do with pixel-sized triangles.
He's using the SDFs to fill a space sort of like Unreal's Nanite virtual geometry. Nanite also doesn't support general animation. They only recently added support for foliage. So you'd use SDF / Nanite for your "infinite detail" / kit-bashing individual pebbles all the way to the horizon, and then draw polygon characters and props on top of that.
In fact I was surprised to see that Nanite flipped from triangle supremacy to using voxels in their new foliage tech. So maybe the two technologies will converge. The guy who did the initial research for Nanite (his talk also cites Dreams ofc) said that voxels weren't practical. But I guess they hit the limits of what they can do with pixel-sized triangles.
I think they do now support skeletal meshes with virtual geometry: https://dev.epicgames.com/documentation/en-us/unreal-engine/...
Though it says "experimental". Unclear what that means in practice.
This also mentions "skinning": https://dev.epicgames.com/documentation/en-us/unreal-engine/... I believe thats just another term for skeletal meshes / "bones".
Though it says "experimental". Unclear what that means in practice.
This also mentions "skinning": https://dev.epicgames.com/documentation/en-us/unreal-engine/... I believe thats just another term for skeletal meshes / "bones".
I'm not super familiar with this area so I don't follow... Why is animation any more difficult? I would think you could attach the basic 3D shapes to a skeleton the same way you would with polygons.
There are lots of reasons you don’t see a lot of SDF skeletal rigging & animation in games. It’s harder because the distance evaluations get much more expensive when you attach a hierarchy of warps and transforms, and there are typically a lot of distance evaluations when doing ray-marching. This project reduces the cost by using a voxel cache, but animated stuff thwarts the caching, so you have to limit the amount of animation. Another reason it’s more difficult to rig & animate SDFs is because you only get a limited set of shapes that have analytic distance functions, or you have primitives and blending and warping that break Lipschitz conditions in your distance field, which is a fancy way of saying it’s easy to break the SDF and there are only limited and expensive ways to fix it. SDFs are much better at representing procedural content than the kind of mesh modeling involved in character animation and rendering.
One possibility, a little backwards maybe, is to produce a discrete SDF from e.g. a mesh, by inserting it in an octree. The caching becomes the SDF itself, basically. This would let rendering be done via the SDF, but other logic could use the mesh (or other spatial data structure).
Or could the engine treat animated objects as traditional meshed objects (both rendering and interactions)? The author says all physics is done with meshes, so such objects could still interact with the game world seemingly easily. I imagine this would be limited to characters and such. I think they would look terrible using interpolation on a fixed grid anyways as a rotation would move the geometry around slightly, making these objects appear "blurry" in motion.
Or could the engine treat animated objects as traditional meshed objects (both rendering and interactions)? The author says all physics is done with meshes, so such objects could still interact with the game world seemingly easily. I imagine this would be limited to characters and such. I think they would look terrible using interpolation on a fixed grid anyways as a rotation would move the geometry around slightly, making these objects appear "blurry" in motion.
Sampling an implicit function on a grid shifts you to the world of voxel processing, which has its own strengths and weaknesses. Further processing is lossy (like with raster image processing), storage requirements go up, recovering sharp edges is harder...
But isn't this what the author is doing already? That's what I got from the video. SDF is sampled on a sparse grid (only cells that cross the level set 0) and then values are sampled by interpolating on the grid rather than full reevaluation.
Dreams on PS4 had an SDF modeler, but I’m not sure if the runtime was SDF. Now that I think about it, the rendering engine had a Gaussian splat look to it years before that paper.
The Dreams team made a nice talk at SIGGRAPH 2015, if you want to check it out:
* Slides (good notes): https://advances.realtimerendering.com/s2015/AlexEvans_SIGGR...
* video: https://www.youtube.com/watch?v=u9KNtnCZDMI
* Slides (good notes): https://advances.realtimerendering.com/s2015/AlexEvans_SIGGR...
* video: https://www.youtube.com/watch?v=u9KNtnCZDMI
They discuss Dreams in the video and even explain Brick rendering.
GameGlobe from Haptico and Square Enix, the engine of which also powered Project Spark from Microsoft, also used an SDF engine. Former colleagues of mine built the tech in Copenhagen and I remember getting a super impressive demo back then. This was the first time I heard of SDFs.
I had the same idea but with voxels. The idea works fine, more work on photorealism needed : https://youtu.be/LBzuXj21_bY?t=128
Reminded me of Red Faction, a FPS where you could destroy the environment.
Kind of like Quake in a Lemmings world.
This is quite more polished to say the least.
Kind of like Quake in a Lemmings world.
This is quite more polished to say the least.
Using layers with settings about which SDFs interact with which layers for operations seems interesting. Like put trees in a layer and then have an axe that can negatively deform the trees but not the ground layer or something. Or a predator layer that can absorb things in the prey layer. Haven't really thought through.
You can't talk about terrain modification without mentioning "From Dust", which did it at a grand scale... 15 (!) years ago.
It allowed you to shape terrain with sand, water and lava. So terrain modification PLUS fluid simulation!
https://www.youtube.com/watch?v=ZYUU3dv7WC4
https://en.wikipedia.org/wiki/From_Dust
It allowed you to shape terrain with sand, water and lava. So terrain modification PLUS fluid simulation!
https://www.youtube.com/watch?v=ZYUU3dv7WC4
https://en.wikipedia.org/wiki/From_Dust
I watched this over the weekend and loved the approach. I’ve played with SDF for 3D modeling (even though the current libraries generate meshes for slicing using marching cubes, which is slow as heck and can lead to imprecision on small features), and wish I had more time for playing around with it.
The interaction with the terrain reminds me of Astroneer, although I don't know if that game uses SDFs.
Such impressive demos and great explanations in the video. Mike, if you're reading this, keep making videos!
Excellent technical presentation!
Though the style itself is a bit too "clay-like", like I wouldn't expect a cube melding with the terrain sand to be a smooth glued connection. Is that some "inherent" SDF thing or just a style of the demo?
I think it is an artifact of the optimizations he uses and while it's artistically limiting, I think a right game with the right visual language could make this work to its advantage in terms of uniqueness/distinctiveness. It's a one trick pony if not avoidable though.
Basically it's the result of a smoothing function that blends the sampled SDF value of the two nearest bodies. You can simply pick the minimum SDF value and get no blending at all.
> You can simply pick the minimum SDF value and get no blending at all.
While this true for traditional SDF rendering (e.g. raymarching), the method of "interpolating cached distances" used here means that you will always get blending between objects.
While this true for traditional SDF rendering (e.g. raymarching), the method of "interpolating cached distances" used here means that you will always get blending between objects.
Oh right, of course. I wasn’t thinking about the specific implementation in the video. Thanks.
I believe you can do regular hard edged intersections. You can see in his operator list some are listed as “smoothSubtract” and some are just “subtract”
It’s just easy to do the melding thing with SDFs so a lot of people do it
It’s just easy to do the melding thing with SDFs so a lot of people do it
From his description of the approach I suspect its also to smooth over sharp edges that the grid optimization doesn't like so much.
I use SDFs on my render engine to render text. It allows me to debug values inside shaders. (I can print value of an uniform, texture value etc.)
(It's a combination of line segments for each letter and digit)
(It's a combination of line segments for each letter and digit)
Are you aware of Valve's paper[0] for glyph rendering via SDFs? You can get amazing results from a low res glyph atlas.
[0] https://steamcdn-a.akamaihd.net/apps/valve/2007/SIGGRAPH2007...
[0] https://steamcdn-a.akamaihd.net/apps/valve/2007/SIGGRAPH2007...
From what I understand, that paper was extremely influential and the technique has been widely adopted.
UE4/5 implement fonts this way.
Is there an easy way to go from Blender model to SDF?
The way he approaches the problem, which essentially uses voxels, it shouldn't be too hard: for each voxel, compute the distance to the closest triangle and you have your SDF.
The thing is, you have a SDF and now what? What about textures and materials, animation, optimization, integration within the engine,... None of it seem impossible, but I won't call it easy.
The thing is, you have a SDF and now what? What about textures and materials, animation, optimization, integration within the engine,... None of it seem impossible, but I won't call it easy.
Really nice video. I wonder how these "sandbox" worlds with transformations can be done will be efficiently stored.
Will the project be open source?
Dang! Very nice!
how does this compare to MudBun?
Mudbun renders using raymarching - the video explains why he has avoided doing this.
really cool! CSG: The Game!
There’s game developers who develop games.
And there’s game developers who develop game engines thinking they are developing games.
And there’s game developers who develop game engines thinking they are developing games.
Nothing wrong with that - Engine developers often have "defining" titles - aka tech demos they work on to push the field
Townscaper is a nice one like this; it has very few features or gameplay, just a sandbox tech demo for a very cool take on model synthesis / "wave function collapse" on an irregular grid. The game is mostly carried by the novelty of this mechanic (and also the pleasant art.)
stupid question to anyone reading this: not a gamedev, not even by a long shot but i had to ask
- with the advent of all the AI tools, is it actually possible to vibe code a 3D FPS shooter from scratch like if you wrote a 2000 page prompt, can it actually be done?
- with the advent of all the AI tools, is it actually possible to vibe code a 3D FPS shooter from scratch like if you wrote a 2000 page prompt, can it actually be done?
A big challenge of game dev is the asynchronous nature of all the requirements, and that the game will develop its direction continuously throughout dev. That is to say you don't know what assets etc you need until you've developed the part of the game that generates that requirement. I find it hard to imagine even a 2000 page pre-planning could capture that process.
You could try planning ahead and restricting assets to an asset library, that could fix some of that problem. But having used coding agents for complex software work, and games being one of the most complex software tasks in the industry, I just don't see it happening quite that easily.
I also think the outcome would be shit, pure and simple. The development of a game is usually the stylistic input of dozens to thousands of humans over the course of years. They are not trivial pursuits. There's a lot of variance in there, but generally speaking I expect this to be one of the final frontiers for AI development. There's not heaps of training data since game code is usually proprietary, which doesn't help.
You could try planning ahead and restricting assets to an asset library, that could fix some of that problem. But having used coding agents for complex software work, and games being one of the most complex software tasks in the industry, I just don't see it happening quite that easily.
I also think the outcome would be shit, pure and simple. The development of a game is usually the stylistic input of dozens to thousands of humans over the course of years. They are not trivial pursuits. There's a lot of variance in there, but generally speaking I expect this to be one of the final frontiers for AI development. There's not heaps of training data since game code is usually proprietary, which doesn't help.
out of curiosity, i want to experiment creating a third person shooter from scratch with vibe coding (yes third person, i wrote FPS above by mistake). think of a proper military game with actual uniforms, movements like walk, crouch, jump, take cover etc. and being able to fire bullets, ballistics, grenades, explosions etc. what do you think is the process to vibe code something like this. obviously i ll need to give it models or assets for characters, map locations etc. how does this sorta thing work?
I would recommend _not_ vibe coding it if it's a game you actually want to see become real, and instead pick up Godot or Unreal or Unity.
I'm sure an LLM could output something or other that resembles a vague concept of a game but you're not going to prompt your way into something that's actually fun for a human to play.
I'm sure an LLM could output something or other that resembles a vague concept of a game but you're not going to prompt your way into something that's actually fun for a human to play.
I think the best approach right now would still be to pick a game engine, and start learning it, in this case with heavy assistance from LLMs. Unity will have the most training data in it for this approach. There is actually a lot of game development that happens outside of code, and in the editor. So having coding done for you can only help so much anyway.
I don't think you can describe all of that in an HN comment. There are lots of videos of people vibe coding games though.
Probably, yes. But it's not 1997 any more, you can "code" a vanilla FPS in Unity in 15 minutes too. Games are more about artwork and design, which agents aren't great at (yet).
You can pretty much drag and drop a working FPS in unity.
But I have half vibed an FPS in Pygame so its 100% viable (Mine is First and Person, and has motion, but its more of a flight simulator. I am sure the rest of the features would be piss easy)
But I have half vibed an FPS in Pygame so its 100% viable (Mine is First and Person, and has motion, but its more of a flight simulator. I am sure the rest of the features would be piss easy)
Depends on what you mean really. In the context of making a game people would actually want to play, no.
I mean you can download a free sample project for Unity or Unreal and have a 3D FPS Shooter even without AI. If you want to make one from scratch using AI, you’ll still need to provide some kind of art…
With that said, yeah Claude Code CAN build one, but for action games a big part of them comes down to “game feel”, something that can’t be captured in a screenshot. You really need to have taste and the ability to describe what isn’t working and why.
With that said, yeah Claude Code CAN build one, but for action games a big part of them comes down to “game feel”, something that can’t be captured in a screenshot. You really need to have taste and the ability to describe what isn’t working and why.
Not a good one.
[deleted]
Elon Musk is working on this(XAi)
I'm curious to know where he takes the gameplay. He mentions it being digging-focused, and also mentions the digging/terrain deformation aspects in other games like No Man's Sky are relatively low-fidelity. I wonder what a "high-fidelity digging game" looks like (:
Aside, if I may self-plug: I wrote a small series on SDFs, for those who might be interested[1]. I'm also using them in my game engine (though it's 2D, for me).
[1]
* https://festina-lente-productions.com/articles/sdfs-1/
* https://festina-lente-productions.com/articles/sdfs-2/
* https://festina-lente-productions.com/articles/sdfs-3/