Open-sourcing the MuJoCo physics simulator for robotics(deepmind.com)
deepmind.com
Open-sourcing the MuJoCo physics simulator for robotics
https://deepmind.com/blog/announcements/mujoco
77 comments
I similarly did not get in to using MuJoCo because I want all my work to be reproducible with open source. So I spent time trying to get RL working in Godot video game engine, but ran in to some road blocks. I know MuJoCo already has all the necessary pipelines so this is exciting indeed!
Hi. Do you mind sharing more of your work with Godot? I'm working in a similar project for software testing.
It would still be a while that we achieve emergence (if that's even possible) anyway, so time is kind of on our side.
Pretty sure it had free educational license tho
My uni was not rich enough to have an email server for students.
So, I did not have a uni mail address.
That was required.
So, I did not have a uni mail address.
That was required.
Somewhat. The free educational license was only acceptable for 'personal' projects, so not suitable for class work.
Surely that kind of use would have been fine. What they're trying to do with that questionnaire must be to determine if you're making money from the venture. You can share your diary with others and still call it personal, after all. (Once you get a publisher involved, not so much.)
you could install that license on one physical machine. so say you want to use a Colab or cloud GPU or something, or your university compute cluster, it wouldn’t work.
I tend to be more than a little skeptical of organizations like DeepMind and (especially) OpenAI lately, when it comes to their pledges to democratize their advances in AI/ML/robotics. This, however, is 100% a great move by DeepMind in the support of the community as a whole. Kudos to the team, and a big thanks as well.
Pretty sure this was just a reaction to losing so much mindshare to Brax since that came out.
My understanding is that DeepMind did not previously own MuJoCo, but today they have acquired it and released it for free.
On the other hand, Brax is developed by Google and is written using Jax (which DeepMind loves and uses extensively).
Why would DeepMind have been concerned that people were switching to Brax?
On the other hand, Brax is developed by Google and is written using Jax (which DeepMind loves and uses extensively).
Why would DeepMind have been concerned that people were switching to Brax?
Good point, I was not aware that Deepmind aquired MuJoCo recently.
Brax is dope, but it's a long long way from feature parity with MuJoCo, which is world class.
Great news! It was a real issue that so much RL research relied on non-free software. It was a problem that a big pile of money could solve, so I'm glad someone with a big pile of money did :)
This is really good news. It was a really contentious issue in RL and robotics research to be so reliant on proprietary (and very expensive) software.
For more context on "very expensive," a MuJoCo license was several hundreds of dollars.
For undergrad students trying to dabble in reinforcement learning, several hundreds of dollars can be quite a lot.
I got MuJoCo for free as an undergrad and I'm pretty sure all I needed was a .edu email address.
as an undergrad in the 90s I invested thousands of dollars of my own money to work on machine learning. Worth every penny, even if I had to miss out on some cool parties.
Wonderful that you had thousands of dollars as an undergrad to be able to do so.
I'm not even sure what virtue you're trying to signal here. Just because someone used the limited funds they had available to obtain the software and educational resources they were interested in, they've exercised some sort of inappropriate privilege?
Isn't that the whole idea behind college? Spend money to learn cool stuff?
Isn't that the whole idea behind college? Spend money to learn cool stuff?
Ok, I'll spell it out. Presenting the idea that MuJoCo having a license fee rather than being free is better, not worse, or even that the difference doesn't matter because "well, I paid for stuff" is the inappropriate part of the privilege. Given a choice between "I can learn X but I need to spend significant money on it" and "I can learn X for free", with everything else equal arguing for the former is exclusionary.
with everything else equal arguing for the former is exclusionary.
Sure. But who argued for that? They merely pointed out that they considered this particular physics engine worth spending money on. The implication was that because money is typically limited for a college student, the software must really be worthwhile. The implication was not "LOL, you must be broke and can't afford good software."
Sure. But who argued for that? They merely pointed out that they considered this particular physics engine worth spending money on. The implication was that because money is typically limited for a college student, the software must really be worthwhile. The implication was not "LOL, you must be broke and can't afford good software."
The implication j was responding to was "if you really cared, a mere few hundred dollars wouldn't matter because you'd be able to find the money." If you don't read that into the message chain, fine, but I do.
Worked a job to pay for ram. It atypical.
Working in undergrad isn't a common thing?
Well, if you're working to pay your rent, bills, and food expenses, and you're only working part time (because you're a full time undergraduate student), you may not have a lot left over to pay for software licenses.
which is precisely the reason I went with linux: so I could maximize my hardware resources while eliminating license costs. By not buying whatever MS compiler in those days, I was able to max out my workstation with 32MB RAM.
Working in undergrad and having thousands left over after paying for food and a place to sleep certainly can't be assumed to be. That's where my pay went.
I was very fortunate to not have to pay tuition or board, but if I had to, I wouldn't have spent my money to pay for that (I would have taken out loans) and still spent my money on RAM.
But they can accept that the world is not geared towards amateur dabblers, and that a few hundreds of $ for a license (a license, meaning to legalize production usage, you can always copy from somewhere to "dabble") is nothing.
The CS world has certainly been geared towards the hobbyist budget for some time now. Would be good to keep it that way, even if it meant sacrificing a supposed 1% extra accuracy in the latest "top conference" (sure if their sponsors are the companies publishing huge models it won't happen even if it should).
[deleted]
Well now they don't have to, because it really is nothing.
the price was reasonable for a good piece of software, but for the way people want to do ML research now (launch tons of parallel jobs on lots of cloud machines) the price, and more importantly, the burden of managing licenses, was prohibitive.
For reference here is the price before as seen on november 2020 (previous internet archive screenshot), https://web.archive.org/web/20201111235930/https://www.robot...
TLDR: personal non commercial 500$/year for usage on up to 3 computers personal commercial 2000$/year
TLDR: personal non commercial 500$/year for usage on up to 3 computers personal commercial 2000$/year
There was a free license for individual users, with the non-free license applicable to those who were receiving financial support (i.e. academic / industry researchers, etc.). The individual license was very popular and heavily utilized.
+1 - plus, it's an immediate barrier for attracting new people to the field.
This is huge for the open source ML community because so many academic and commercial projects rely on MuJoCo which is out of reach for folks working on hobby projects or at tiny startups. This makes a lot more AI environments work for everyone. Major props to DeepMind
The demo object that turns itself over is nice.
I used to use a rattleback as a test back when I was working on accurate but slow physics engines in the 1990s. That's an ellipsoid cut by a plane that isn't axis-aligned. You spin it, and it spins, stops, rocks, and spins a bit in the other direction. Very similar to what they're doing.
The key to this is a multi-point contact model. If your contacts are single points, the frictional effects are wrong. It's not that hard to do this. I held a patent on that (#6,067,096), and there was a product called Falling Bodies, the first ragdoll system that really worked.
It was just too early in the 1990s. 100 MIPS was not enough.
I used to use a rattleback as a test back when I was working on accurate but slow physics engines in the 1990s. That's an ellipsoid cut by a plane that isn't axis-aligned. You spin it, and it spins, stops, rocks, and spins a bit in the other direction. Very similar to what they're doing.
The key to this is a multi-point contact model. If your contacts are single points, the frictional effects are wrong. It's not that hard to do this. I held a patent on that (#6,067,096), and there was a product called Falling Bodies, the first ragdoll system that really worked.
It was just too early in the 1990s. 100 MIPS was not enough.
It's hard to do it if some asshat patents it, that's for sure. At least in commercial applications.
Does anyone know how this compares to e.g. Chrono?
https://github.com/projectchrono/chrono
https://github.com/projectchrono/chrono
I wonder what this means for the future of Brax [1].
1. https://github.com/google/brax
1. https://github.com/google/brax
Google Research already maintains the Bullet Engine as well (although it mostly seems to be Erwin Coumans maintaing it)
Each have their own strengths, there is no "definitive" physics engine. Bullet is probably the most generalist engine, but it suffers a bit because of this approach (it's split between the old 2.0 C++ API vs. the new PyBullet API with vastly different functionalities)
Each have their own strengths, there is no "definitive" physics engine. Bullet is probably the most generalist engine, but it suffers a bit because of this approach (it's split between the old 2.0 C++ API vs. the new PyBullet API with vastly different functionalities)
One strong advantage of MuJoCo is its plain vanilla C bindings. Hardwiring an API like this to C++ or Python makes life much harder than necessary for users of other languages. Conversely, it's very easy to wrap a well-thought-out C API in your language of choice.
Super impressed with the data structures (and documentation) they've already released.
Super impressed with the data structures (and documentation) they've already released.
MuJoCo going open source is good for the open source community. PyBullet is build on top of a plain C API (wrapping C++ Bullet API).
Note that there is also Brax and Tiny Differentiable Simulator, developed within Google.
> MuJoCo is a second-order continuous-time simulator, implementing the full Equations of Motion
Are there any books describing how to implement this kind of physics engine?
Are there any books describing how to implement this kind of physics engine?
The Mujoco documentation explains the mathematics behind it in detail.
It seems great, but the simulation geometry options feel bit limited? What would be really great would be the ability to simulate scenes with objects imported from STEP/IGES files, so you can do more of the modeling in actual CAD tools. Instead, MuJoCo seems more oriented towards using its own primitives, not sure how that works out with more complex situations.
If you can turn it into a collection of STL or MSH files then you can simulate it in MuJoCo. There are some caveats - MuJoCo will use the convex hull of the mesh for collisions, so if you want to accurately simulate collisions with concave features then you'll need to split your mesh into convex submeshes (there are various third party tools that can automate this). Sometimes you need to tweak the mesh geometry a bit in order to get the right contact behaviour, and in some cases it's easier to use primitives for key collision geometry and just use the meshes for visualisation.
They're not in the file-reading business. There's a staggering variety of 3D object file formats, both open and proprietary, so it's logical enough to let the application worry about reading the geometric data and converting it to the representation needed for physics simulation.
[deleted]
[deleted]
[deleted]
This is an amazing outcome. The license cost was a huge barrier to entry and the progress of deep RL generally.
A little off topic but is there a good library for physics simulations that involve cloth modeling.
interesting. will they be including any biomechanical models as part of the open sourced stuff?
Is this a competitor to physX?
Nope. PhysX is all about realtime and gaming is the end use it was designed around. TFA has a section about why that’s bad. The different end uses mean there are miles of differences between them even though they’re both physics simulators.
A lot of RL problems can be adequately simulated in either Mujoco or PhysX. So even though as you pointed out they have key differences, there is a pretty big overlap in use cases. So in that sense, I believe it still does qualify as competition.
I have personally spent tons of time dealing with models exploiting physics issues. It’s a major issue. It’s true that either can adequately simulate the setups, but is it true that either can be equally exploited? I genuinely don’t know. I kinda assumed physx is more exploitable.
I agree with you that PhysX is definitely easier to exploit, and you will lose time because of that.
But on the other hand, for many problems, the gain in training time using PhysX is just so massive (2 or 3 orders of magnitude), that you can then train with a much more aggressive sim2real policy to mitigate that to some extent, or even spend time debugging manually and still ending up more productive.
But on the other hand, for many problems, the gain in training time using PhysX is just so massive (2 or 3 orders of magnitude), that you can then train with a much more aggressive sim2real policy to mitigate that to some extent, or even spend time debugging manually and still ending up more productive.
What kind of exploits are you talking about? Cheating in games for example?
I think this is referring to inaccuracies of simulation which the RL model can learn to exploit. Sort of a local minimum that doesn't exist in the real world.
Say you're teaching a robot to walk, but the joint constraints in the simulator aren't quite right. It would be possible for the policy to exert more torque on the joint than it's supposed to be able to in real life. Take that policy and put it on a real robot and it won't work.
[deleted]
Can it do fluid simulations for aquatic robots?
The docs talk about that a little [1], with the answer boiling down to "Yes, the necessary hooks are there, but no, you don't want to use this particular engine for fluid dynamics."
The docs are definitely a strong point, nicely written and presented.
1: https://mujoco.readthedocs.io/en/latest/computation.html# , about 1/4 of the way down the page
The docs are definitely a strong point, nicely written and presented.
1: https://mujoco.readthedocs.io/en/latest/computation.html# , about 1/4 of the way down the page
I wish someone would do this for MATLAB.
I assume you know about GNU Octave? I used it years ago for Andrew Ng's ML course. It seems very good.
Since it has a C API, I guess binding it with Matlab won’t be that hard?
sfdsdf
You should favorite it, if you want to leave a placeholder on it. It's a link near the top of the thread.
[deleted]
When I was learning Deep RL, I edged on what could be done through OpenAI gym, or self-made environments, I decided to begin working with MuJoCo.
But for an undergrad student from a developing country, the fees were extremely high.
So, I just didn't go forward with Deep RL much. I dabbled for a while and tried using PyBullet. Did not go that well.
Also thought about creating custom environments in Unity and then apply Deep RL in there. But it took too much effort.
Creating a complex enough environment and getting used to it were, in themselves- full projects that took long term commitment.
As a result, I did not learn much Deep RL to be able to apply it to complex, real-world-like scenarios.
I think that it is going to change.
I am excited about books, blogs, and videos that are going to be created as MuJoCo is open-sourced and available for free.
I, personally, will get back into Deep RL the next opportunity I get.