Open Source Goes to Mars(github.blog)
github.blog
Open Source Goes to Mars
https://github.blog/2021-04-19-open-source-goes-to-mars/
60 comments
Hi! PM at GitHub here. This achievement only accounts for packages used for the Ingenuity Helicopter, as it celebrates the first ever powered, controlled flight on another planet. Based on the article you shared, FFmpeg was used on the Rover.
[deleted]
That's a big one. Probably hundreds of contributors. But what percentage of FFmpeg code they would actually run?
Weird that the big projects mentioned there does not have a link.
Ironically, people hated the contribution graph stats because it gamified code contributions, and now we're introducing "achievements".
This is kind of neat as a one-off, but it seems like it'll get weird with scale. Isn't it disappointing if your code is on the rover but you don't get a badge because your code isn't on the helicopter?
This is kind of neat as a one-off, but it seems like it'll get weird with scale. Isn't it disappointing if your code is on the rover but you don't get a badge because your code isn't on the helicopter?
I've started making a decent amount of commits to a public repository, and I almost feel embarrassed having my stats there because the commits have been rather marginal. I wish there was a way to block the specific numbers or block certain repositories from filling the stats.
> I wish there was a way to block the specific numbers or block certain repositories from filling the stats.
This is somewhat related, but I'm working on impact based metrics that doesn't focus on commits, but is based on a contribution formula that I'm currently working on. You can get a sense of how the formula works by switching to the impact view in the following link:
https://public-001.gitsense.com/insights/github/repos?q=wind...
As a side note, if you want, DM me your public repos and I'll add them to my public index list, as your contributions might have a greater impact that you may think. Plus it will provide me with more data points for refining my formula.
This is somewhat related, but I'm working on impact based metrics that doesn't focus on commits, but is based on a contribution formula that I'm currently working on. You can get a sense of how the formula works by switching to the impact view in the following link:
https://public-001.gitsense.com/insights/github/repos?q=wind...
As a side note, if you want, DM me your public repos and I'll add them to my public index list, as your contributions might have a greater impact that you may think. Plus it will provide me with more data points for refining my formula.
First, yes, commits is a terrible metric, because it lacks both the nuance of the size of commits and the substance of the commits. Also different projects have different norms around commits. Some people commit each step along a process, some force-push updates to their branches so a given improvement sits in a single commit. There's no good direct comparison between work across projects.
That being said, don't feel like "rather marginal" commits is unhelpful. Support, docs, and tweaks are woefully undervalued parts of public projects, and you don't have to be building core features to be a big part of adding to the long-term success of a project. Don't sell yourself short.
That being said, don't feel like "rather marginal" commits is unhelpful. Support, docs, and tweaks are woefully undervalued parts of public projects, and you don't have to be building core features to be a big part of adding to the long-term success of a project. Don't sell yourself short.
And on the flip side, I got the badge because I have a few commits in the Linux kernel, even though the code I touched is almost certainly not compiled into any of Ingenuity's systems. Feels a little undeserved.
Sound like the National Defense Medal you get in the US Military if you served during an ongoing war, regardless of what you did. I think it's the only medal you can get before you even finish boot camp.
It's called the Pizza Stain medal for a reason. Service-members are still eligible for it 20 years later, handily beating the Vietnam's 13 years of eligibility. See also the GWOT [0].
[0]: https://en.wikipedia.org/wiki/Global_War_on_Terrorism_Servic...
[0]: https://en.wikipedia.org/wiki/Global_War_on_Terrorism_Servic...
I think it's important to remember that the people who "hated the contribution graph stats" were the people who specifically complained about hating them. Many multiples of that number of people comprise the set of people who were indifferent about them + people who liked them. (Probably a lot more in the first subgroup.)
It's selection set bias. You only heard from the complainers, making it seem like "everyone" disliked them. I think the contribution graphs are cool, and so are badges/achievements.
It's selection set bias. You only heard from the complainers, making it seem like "everyone" disliked them. I think the contribution graphs are cool, and so are badges/achievements.
> Ironically, people hated ____ and now we're ____.
All that's changed is your own vague impression of what the current most popular opinion is.
All that's changed is your own vague impression of what the current most popular opinion is.
This such an achievement for open source developers whose software has made significant impact.
I wonder what world would look like where we open source software didn't existed?
Anyhow I really appreciate the work of open source developers as myself who uses open source software on daily basis
I wonder what world would look like where we open source software didn't existed?
Anyhow I really appreciate the work of open source developers as myself who uses open source software on daily basis
The pace of innovation would have slowed. Some projects that didn’t succeed would have thrived and others that did succeed would have failed. I don’t think it would have fundamentally altered the trajectory of technology, just the pace and environment (which is still amazing but let’s keep things in perspective).
Every day would be a field day for Oracle/Microsoft lawyers and sales guys.
And AWS wouldn't exist.
I’m concerned why chardet is on that list haha
You know, to figure out the encoding of files found in the recently unearthed (unmarsed?) ancient datacenter.
It would be nice if GitHub could allow opting out of all future achievements things so folks don't have to do it manually every time there is a new achievement.
Would also be nice to know what I did that counts for each of the achievements.
Would also be nice to know what I did that counts for each of the achievements.
Hovering on the achievement shows which contribution(s) were the cause
Looks like that feature requires JavaScript.
Luckily almost every browser supports it these days
There are a lot of browsers that don't and a lot of people that disable it in browsers that do.
An anecdotal "a lot of" is not an authoritative source. >99% of browsers in active use support JS (https://gs.statcounter.com/browser-market-share) and <1% of people have JS disabled (https://deliberatedigital.com/blockmetry/javascript-disabled).
You say "a lot of", but the statistics say "a statistically insignificant percentage". I'm all for designing your static webpages with no need for JS, but the claim that a lot of people have it turned off is demonstrably false.
Anyway, hovering on an image to get a description shouldn't need javascript, that's an alt tag that is definitely supported by anything but hobby project browsers.
You say "a lot of", but the statistics say "a statistically insignificant percentage". I'm all for designing your static webpages with no need for JS, but the claim that a lot of people have it turned off is demonstrably false.
Anyway, hovering on an image to get a description shouldn't need javascript, that's an alt tag that is definitely supported by anything but hobby project browsers.
> the claim that a lot of people have it turned off is demonstrably false
Not unless you've taken a concrete definition for "a lot of people".
Even 1% of internet users is still a lot of people IMO. Like, on the order of tens of millions, right?.. That's bigger than whole countries.
Also keep in mind how outraging it feels to find yourself in that 1% that someone decided it was fine to ignore and discriminate against the mainstream. We all been there.
Not unless you've taken a concrete definition for "a lot of people".
Even 1% of internet users is still a lot of people IMO. Like, on the order of tens of millions, right?.. That's bigger than whole countries.
Also keep in mind how outraging it feels to find yourself in that 1% that someone decided it was fine to ignore and discriminate against the mainstream. We all been there.
The group of people (and their choice of browser and it's possible security related constraints) accessing github probably does not correlate 1:1 with "all of the world"?
If Google could make a few cents from every comment on HN disparaging JS they wouldn't have to show us ads anymore
(edit: I realize there are legitimate reasons to block JavaScript so I just wanted to clarify this is a joke and I don't mean to insult OP, just commenting on how often this discussion pops up in comment sections here and how predictable they all are.)
(edit: I realize there are legitimate reasons to block JavaScript so I just wanted to clarify this is a joke and I don't mean to insult OP, just commenting on how often this discussion pops up in comment sections here and how predictable they all are.)
Hey my ToDo list made it into the arctic vault. I think I will be remembered by future generations.
Judging from my todo list, some of the items will still be undone by the time future generations roll around.
So did my keyboard layout for WASD!
What does the Mars 2020 Helicopter team use Elastic Search for? Searching logs/data?
To search for extra terrestrials.
[deleted]
By my count, six of the sixty projects cited were initiated by the amazing Armin Ronacher (aka mitsuhiko), whose genius for designing and implementing clean, intuitive library interfaces has, for the past decade or so, been a boon to the world of open-source software development. Kudos to him.
[*] click, flask, itsdangerous, jinja, markupsafe, and werkzeug
[*] click, flask, itsdangerous, jinja, markupsafe, and werkzeug
Armin's work is great but I'm curious if the Mars rover really implements a web interface? Are all the boto AWS libs just dependencies of opencv with its mess of supported data sources? Why are they deploying unit testing code to Mars? Given the 28 minute average round trip time between Mars and Earth for a light speed signal, they can't realistically be running distributed workloads partly happening on Earth.
All the PKI libs are amusing, too. JPL isn't sending signals to Mars via public networks, are they?
All the PKI libs are amusing, too. JPL isn't sending signals to Mars via public networks, are they?
It looks like Python libraries are somewhat over-represented here because they credited every library in the Python dependency graph, but didn't do the same for dependencies in other languages. For example: lxml depends on libxml and libxslt, which are not credited.
The reason for the discrepancy is probably the lack of a straightforward, programmatic way to view the dependency graph of a C library. But they could have probably gotten a pretty good approximation by looking at Debian package dependencies (e.g., 'apt-rdepends python3-lxml').
This would massively expand the credits, but that's sort of the point here, no? (i.e., to celebrate the massive tapestry that is the open source ecosystem.)
(edit: I e-mailed Nat with this suggestion.)
The reason for the discrepancy is probably the lack of a straightforward, programmatic way to view the dependency graph of a C library. But they could have probably gotten a pretty good approximation by looking at Debian package dependencies (e.g., 'apt-rdepends python3-lxml').
This would massively expand the credits, but that's sort of the point here, no? (i.e., to celebrate the massive tapestry that is the open source ecosystem.)
(edit: I e-mailed Nat with this suggestion.)
I think part of that is because these libraries do not live on github, or have different identities attached to them (different email addresses like @gnome.org for libxml2 stuff).
It is essentially a hard problem, one that ohloh.net (now openhub) tried to make fun a while ago, to encourage people to claim their own contributions to tie their identities together.
I do wonder if they could have used openhub data to improve identity links though. Still doesn't solve the dependency graph lacking.
It is essentially a hard problem, one that ohloh.net (now openhub) tried to make fun a while ago, to encourage people to claim their own contributions to tie their identities together.
I do wonder if they could have used openhub data to improve identity links though. Still doesn't solve the dependency graph lacking.
> All the PKI libs are amusing, too. JPL isn't sending signals to Mars via public networks, are they?
They're using radio waves so yeah, it's public. In fact I suspect the protocols are pretty much standardized because why reinvent those wheels?
I'm sure there's some blog post at the bottom of an internet rabbit hole where someone figured out the protocol, or just knows it, or is the one who invented it.
It's not a defense thing, NASA may well have published the details themselves. In fact, that's pretty much the point of NASA.
E: quick quack rabbit hole seeds:
https://mars.nasa.gov/msl/mission/communications/
https://mars.nasa.gov/mars2020/mission/communications/
https://en.wikipedia.org/wiki/NASA_Deep_Space_Network
Of course the HAMs got there first:
http://www.prutchi.com/2020/10/15/recap-of-x-band-dsn-activi...
https://hackaday.com/2020/10/17/it-came-from-outer-space-lis...
They're using radio waves so yeah, it's public. In fact I suspect the protocols are pretty much standardized because why reinvent those wheels?
I'm sure there's some blog post at the bottom of an internet rabbit hole where someone figured out the protocol, or just knows it, or is the one who invented it.
It's not a defense thing, NASA may well have published the details themselves. In fact, that's pretty much the point of NASA.
E: quick quack rabbit hole seeds:
https://mars.nasa.gov/msl/mission/communications/
https://mars.nasa.gov/mars2020/mission/communications/
https://en.wikipedia.org/wiki/NASA_Deep_Space_Network
Of course the HAMs got there first:
http://www.prutchi.com/2020/10/15/recap-of-x-band-dsn-activi...
https://hackaday.com/2020/10/17/it-came-from-outer-space-lis...
The airwaves can be intercepted, sure, but that isn't what I mean. I've done quite a bit of work that involved ground processing of signals from government owned spacecraft and all of it was encrypted using pre-shared hardware keys, not PKI. Granted, that was military applications and maybe JPL and NASA don't have the same security requirements, but seemingly they should? Maybe the Mars rover is less of a desirable attack target for a nation state adversary than a spy satellite, but it is a pretty expensive project you would seemingly still want to protect as much as you can.
The radio side of things is quite well standardised around Mars actually.
The majority of missions now carry an Electra radio unit https://en.wikipedia.org/wiki/Electra_(radio)
The Electra radio implements the Consultative Committee for Space Data Systems (CCSDS) Proximity-1 Space Link Protocol (https://en.wikipedia.org/wiki/Proximity-1_Space_Link_Protoco...) which is available online as PDFs, divided into three sections: Data Link Layer CCSDS 211.0-B https://public.ccsds.org/Pubs/211x0b6.pdf ; Physical Layer CCSDS 211.1-B - https://public.ccsds.org/Pubs/211x1b4e1.pdf ; and Coding and Synchronization Sublayer CCSDS 211.2-B - https://public.ccsds.org/Pubs/211x2b3.pdf
The CCSDS provides full access to all their recommended standards on their website ( https://ccsds.org )
There are some recommended standards published regarding crypto and security: "Security Architecture for Space Data Systems" (CCSDS 351.0-M), "Space Data Link Security Protocol" (CCSDS 355.0-B) , "CCSDS Cryptographic Algorithms" (CCSDS 352.0-B), and others.
And to wrap up, they also have standards regarding deep space communications inside "Radio Frequency and Modulation Systems—Part 1: Earth Stations and Spacecraft" (CCSDS 401.0-B) https://public.ccsds.org/Pubs/401x0b31.pdf which has been an active recommended standard for over 30 years now, so everything currently operating at Mars (NASA, ESA, and the China National Space Administration (CNSA) are all participating in CCSDS) is likely conforming pretty closely to the radio communications standards in CCSDS 401.0-B
Overall its a pretty amazing amount of detail provided by them, a real wealth of technical information regarding spacecraft operations and engineering.
The majority of missions now carry an Electra radio unit https://en.wikipedia.org/wiki/Electra_(radio)
The Electra radio implements the Consultative Committee for Space Data Systems (CCSDS) Proximity-1 Space Link Protocol (https://en.wikipedia.org/wiki/Proximity-1_Space_Link_Protoco...) which is available online as PDFs, divided into three sections: Data Link Layer CCSDS 211.0-B https://public.ccsds.org/Pubs/211x0b6.pdf ; Physical Layer CCSDS 211.1-B - https://public.ccsds.org/Pubs/211x1b4e1.pdf ; and Coding and Synchronization Sublayer CCSDS 211.2-B - https://public.ccsds.org/Pubs/211x2b3.pdf
The CCSDS provides full access to all their recommended standards on their website ( https://ccsds.org )
There are some recommended standards published regarding crypto and security: "Security Architecture for Space Data Systems" (CCSDS 351.0-M), "Space Data Link Security Protocol" (CCSDS 355.0-B) , "CCSDS Cryptographic Algorithms" (CCSDS 352.0-B), and others.
And to wrap up, they also have standards regarding deep space communications inside "Radio Frequency and Modulation Systems—Part 1: Earth Stations and Spacecraft" (CCSDS 401.0-B) https://public.ccsds.org/Pubs/401x0b31.pdf which has been an active recommended standard for over 30 years now, so everything currently operating at Mars (NASA, ESA, and the China National Space Administration (CNSA) are all participating in CCSDS) is likely conforming pretty closely to the radio communications standards in CCSDS 401.0-B
Overall its a pretty amazing amount of detail provided by them, a real wealth of technical information regarding spacecraft operations and engineering.
Public maybe but probably encrypted or I guess anyone could just reverse engineer it and hack the rover
Most people don’t have an antenna and RF transmitter capable of reaching Mars.
Chinese or Russians, heck even the Australians do
Happy to see my favourite library Python Requests[1] on the list. Congrats team.
[1] - https://github.com/psf/requests
[1] - https://github.com/psf/requests
This is so awesome, most interesting aspect for me is fprime documentation is organized, so far.
It's informative to think about what is missed in the listed dependencies (https://docs.github.com/en/github/setting-up-and-managing-yo...), such as C and Fortran libraries vendored within NumPy and SciPy, packaging infrastructure like pip, interactive tooling (Jupyter, editors/IDEs), compilers and build systems.
Just like with humans, a figurehead takes all the credit ;)
I wonder why they chose to use Kernel 3.4 on the helicopter (https://docs.github.com/en/github/setting-up-and-managing-yo...). I wonder if they'll ever upgrade the kernel aboard -- it seems like they can do upgrades, given they have Python 3.9.2 aboard.
An aside -- I'm surprised how much Python is aboard. It seems like they're using Python + OpenCV for something. I'm particularly surprised by Boto. Perhaps this is the beginning of the newest AWS region -- mars-1.
An aside -- I'm surprised how much Python is aboard. It seems like they're using Python + OpenCV for something. I'm particularly surprised by Boto. Perhaps this is the beginning of the newest AWS region -- mars-1.
Presumptively not all this code is running on Mars, but some of it in Mission Control to deal with data coming from the helicopter. Either that, or they’re running Flask on the helicopter with some really large timeouts set.
So what has changed since Tcl was rejected for use on the Mars Pathfinder because "interpretive language code is too difficult to test" - https://wiki.tcl-lang.org/page/Mars+Rover+Project ??
As usual, this looks rather like Python making a big fuss about achieving something Tcl did long ago :-)
See also https://wiki.tcl-lang.org/page/International+Space+Station .
As usual, this looks rather like Python making a big fuss about achieving something Tcl did long ago :-)
See also https://wiki.tcl-lang.org/page/International+Space+Station .
But not to GitHub. Ironic.
Looking forward to reading on haxx.se about how someone contacted Daniel for help with their Martian helicopter that, according to the manual, runs curl.
The RTEMS project should be there - it's provided the realtime OS for the radio communication unit both on rovers and satellites - https://en.wikipedia.org/wiki/Electra_(radio)
See https://link.springer.com/article/10.1007/s11214-020-00765-9
"The core EDL data processing and compression engine is powered by FFMPEG (ffmpeg.org)."