Indeed, it entirely depends on which axis you want to focus on. A loose trade-off chart would be speed, stability and accuracy. You can only have two of these in a simulator.
Robotics folks probably want speed and accuracy. I'm from the video game industry so I generally look for speed and stability.
Note: This is a loose analogy and recent techniques are already blurring the lines between these axis.
Thanks for the additional reading. I've often thought about LLMs and their ability to represent the physical world with its laws. And always concluded it is not really possible to do so with "just" text tokens and their relations in a latent space. It looks to me there are different approaches being taken to tackle this:
* You could instruct your LLM to interact with a simulator to run experiments and infer behaviour
* You could edit the transformer model and inject spatially relevant data rather than text as is done in above paper
* You could change the architecture to be more condusive for representating a world state. I.e., LeCun's JEPA world model.
* You could further enhance some of the above by using a differentiable physics engine (eg. NVIDIA Newton) to calculate losses directly.
But at the end of the day if a model has any hope to always produce realistic physics, it HAS to learn the laws of nature in some form or other. It looks to me that the next big leap could be achieved by combining the last two approaches.
P.S.: I like discussing such topics. If anyone knows a forum or discord with like-minded people, please let me know :)
In my day job I program rigid body behaviour in real time amongst other simulations.
I think rigid body contact is hard to learn as it is inherently discontinuous.. something you discover when trying to code a solver.
As such I always use this prompt as a test:
"A video of a jenga brick tower falling over as a brick is removed. The physics of each brick must be realistic."
It gave me a video of where bricks suddenly disapper or morph into others[1]. The linked video is after 2-3 iterations of me insisting on realistic physics. If you are just glancing at this, you would believe it is realistic.
That said this is still very impressive and one more step towards .. IDK what. But I am a bit reasurred that at least my job won't be fully replaced with AI :)
200% agree. If you decide to use this power you must accept the tiny risk and huge consequences of it going wrong. The article seems like it was written by AI, and quoting the agent's "confession" as some sort of gotcha just demonstrates the author does not really understand how it works...
LLMs are not deterministic per my understanding. A program always produces the same output for the same input and instructions (ignore FP accuracy for now). How is determinism achieved here?
I think the diagrams look very similar to what Keenan Crane uses in his papers, perhaps they used that tool. I think his students have now fleshed it out for general use.
One esoteric route would be to try and specialize in an area where talent is scarce. There's a lot of gameplay programmers, few engine programmers, fewer graphics programmers, and very few physics programmers (in my experience at least).
As such you could try to specialize in this area (collision detection, ray queries, rigid body simulation, constraints, solvers, softbody sim, fluid sim etc.). Of course this isn't for everyone as it requires skills and interest in: low level concurrent programming, maths/linear algebra and physical behavior intuition.
If you do find these topics fascinating and can demonstrate some ability in them, your skills will certainly be in demand.
I've always observed a curious thing within India regarding the Devnagari (Hindi) and Latin (English) scripts. Essentially all English words are always written in Devnagari, but it's rarely the other way around. For example it is much more likely to see इंग्लिश टू हिंदी than "angrezi se hindi".
My personal theory is that this is because you can make every sound you hear in English using the Devnagari script, but not the other way around.
Thank you for asking and forwarding the answer. I appreciate your comments.
Speculation as well: Looking at the trailer for Bitcraft, it looks like they have articulated characters and cloth physics. This raises more questions for me.
I see, that's extremely interesting. As mentioned earlier, I do respect efforts to push on something new, especially if it's against conventional wisdom and would be interested in poking around with an open mind. I'll join the Discord.
Thank you!
Then again looking at the FAQ for Bitcraft, we have this:
"What engine does BitCraft use?
The BitCraft client is developed using the Unity game engine. For the server, we have developed a sophisticated distributed system called SpacetimeDB, which hosts the entirety of BitCraft’s server architecture. "
Correct me if wrong, but this suggests the physics engine is Unity's, and not something implemented in SpacetimeDB.