I moved from CS into Earth science a decade ago and I was similarly appalled at the state of affairs in the beginning. First, this doesn't have anything to do with how smart people are, it's a question of time and other resources. Our job simply isn't to write pretty code, it's just a means to an end. If you know how to do something in FORTRAN, why would you invest into learning how to do it with <lastest trend>? It's a huge cost! Second, the evolution of such code occurs often over decades; as the article states, with students and post-docs hammering along until it finally does what they need to get their project to move forward. The code usually begins with "Do X" and then the rest of the alphabet is tacked on later. We all know it's bad. But in the end, it's usually only your own lab group using this stuff; maybe a handful of people around the world. The cost of polishing something that started out as hastily written code to get something done in time for {conference,paper,proposal} a decade ago is simply to high. It works. Done.
I wish the state of affairs was prettier, but I doubt it will change any time soon. And frankly, I don't think it has to. Software engineers will always find something to complain about others' code (coding style discussions come to mind). If there's value in commercializing scientific prototypes, experts should rush in and do it. There's no need for us to invest in perfect code just in case someone may want to reuse it. Having said that, NSF is starting to fund initiatives to improve the state of affairs. For my field, it's EarthCube: https://www.earthcube.org/ but this seems to be fumbling along; all of their goals would need to be adapted by scientists outside of that community (standards, etc). Without the right incentives (funding, publications), it's just not going to happen.
It's problematic when data publisher != data user/paper writer. I'm not familiar enough with DOI minting and therefore don't know what issues DOI generation on large scales for miniscule changes in the data might bring. Ultimately, if I make data openly available the worst case is that every change to the data requires a new DOI as I don't know how many people have downloaded earlier versions and not published on those yet / don't care about my added cleaning (or think it's wrong). I haven't done it in a while, but github's collaboration with Zenodo results in a zip file hosted there. Obviously, that reduces the amount of DOIs created but it's not great. As soon as my code changes, and someone uses that version in a paper, they'll use the old DOI. Potentially resulting in not reproducible results. The same is true for data. On the researcher side, you may end up with 100s of DOIs, each with zero-few citations. Also not great. A happy medium might be to leave it to the data generator to create DOIs for set versions, and drop anyone trying to resolve the DOI on a landing page that provides links to that original version and any updates since (maybe indicating later releases that have DOIs attached separately). Certainly would make me as data user / supplier happy.
Yes, but it's not globally recognizable. that's why doi's are standardized through ISO https://www.doi.org/ Internally you could implement a DOI->HASH mapping, but a quilt hash isn't going to help in the reference list of a paper (if you're lucky you can copy'n'paste it. How do you know where to go? What happens if your package organization changes internally and so forth.
Any thoughts on adding DOIs? It's a complex subject wrt versioning, in particular (new DOI per version? How to keep track?). It would help tremendously with the academic community; for the bean counting.
I wish the state of affairs was prettier, but I doubt it will change any time soon. And frankly, I don't think it has to. Software engineers will always find something to complain about others' code (coding style discussions come to mind). If there's value in commercializing scientific prototypes, experts should rush in and do it. There's no need for us to invest in perfect code just in case someone may want to reuse it. Having said that, NSF is starting to fund initiatives to improve the state of affairs. For my field, it's EarthCube: https://www.earthcube.org/ but this seems to be fumbling along; all of their goals would need to be adapted by scientists outside of that community (standards, etc). Without the right incentives (funding, publications), it's just not going to happen.