This is not exactly the claim. I’m kind of certain they do “fair tests” where they test the same scenarios in different simulators. The problem however is that these tests are not indicative of anything useful. Why should one care if you get 430,000x real time speed simulating a robot arm not colliding or moving any objects with low sim accuracy? This number is fairly meaningless since if you do machine learning (specifically RL) to solve some task (manipulation, locomotion etc) that speed will drop significantly. Moreover in the past most simulators typically benchmark against previous versions of themselves and on scenarios with more collisions / somewhat more realistic situations (picking up a cube).
They test on 3 environments, all with multiple collisions spanning locomotion and manipulation and get some nice speeds. These tests are far more realistic and grounded in reality than the test that made the 430K number.
This is kind of correct. For the demos they show those will simulate quite slowly due to the soft body / fluid simulation. Their current code does not parallelize soft body/fluid simulation like they do with rigid bodies. Running their fluid sim code now with default settings (which are tuned for speed, not accuracy) gives maybe max 10-100x real time speed (their rigid body sim is magnitudes faster but definitely not faster than existing GPU simulators like Isaac or MJX). Rendering at that quality will also take forever but everyone in this area of research always runs the slow high quality ray tracing pipelines for the demo videos (which run in the 10-100 FPS range depending on settings). Another note is they claim realism but this is only visual realism, there doesn’t exist soft body sim solvers that transfer to the real world accurately enough yet for eg robotics. They are good enough for visual data generation which will be pretty cool.
(full disclosure I work on ManiSkill/SAPIEN which runs on PhysX from NVIDIA, Genesis authors claim they are essentially faster than PhysX by 10-80x).
This is not exactly the claim. I’m kind of certain they do “fair tests” where they test the same scenarios in different simulators. The problem however is that these tests are not indicative of anything useful. Why should one care if you get 430,000x real time speed simulating a robot arm not colliding or moving any objects with low sim accuracy? This number is fairly meaningless since if you do machine learning (specifically RL) to solve some task (manipulation, locomotion etc) that speed will drop significantly. Moreover in the past most simulators typically benchmark against previous versions of themselves and on scenarios with more collisions / somewhat more realistic situations (picking up a cube).
The original paper that started the trend of GPU parallelized sims is this one isaac gym: https://arxiv.org/pdf/2108.10470
They test on 3 environments, all with multiple collisions spanning locomotion and manipulation and get some nice speeds. These tests are far more realistic and grounded in reality than the test that made the 430K number.