Distill Hiatus(distill.pub)
distill.pub
Distill Hiatus
https://distill.pub/2021/distill-hiatus/
12 comments
Strongly agree on the archival concerns. The reason scientific publishing is nowadays built upon 2D PDFs is that those are the digital analogue of paper.
Distill used a new medium and media, with all the good and bad that comes with it. In my view the biggest challenge is archiving to ensure readability and accessibility in 20, 200, 2000 years. We can read things written in parchment 2000 years ago, we should aspire to properly view digital media 2000 years from now. Yet 2 years from now much of digital media on the internet is already broken. The internet archive is humankind's savior in this regard, but we need to do more, better and faster (because so much digital content is being created and lost before we can save it...).
Regarding Distill specifically, short of a GitHub repo for each article archived in other mirrors, I don't see much else that is straightforward and flexible enough. Even the Distill arXiv idea mentioned would likely have to run on a combination of GitHub + mirrors...
Distill used a new medium and media, with all the good and bad that comes with it. In my view the biggest challenge is archiving to ensure readability and accessibility in 20, 200, 2000 years. We can read things written in parchment 2000 years ago, we should aspire to properly view digital media 2000 years from now. Yet 2 years from now much of digital media on the internet is already broken. The internet archive is humankind's savior in this regard, but we need to do more, better and faster (because so much digital content is being created and lost before we can save it...).
Regarding Distill specifically, short of a GitHub repo for each article archived in other mirrors, I don't see much else that is straightforward and flexible enough. Even the Distill arXiv idea mentioned would likely have to run on a combination of GitHub + mirrors...
> Another significant risk factor is having unachievable goals. We set extremely high standards for ourselves: with early articles, volunteer editors would often spend 50 or more hours improving articles that were submitted to Distill and bringing them up to the level of quality we aspired to. This invisible effort was comparable to the work of writing a short article of one’s own. It wasn’t sustainable, and this left us with a constant sense that we were falling short. A related issue is that we had trouble setting well-defined boundaries of what we felt we owed to authors who submitted to us.
As someone finishing a PhD, I think that doing LESS and doing it SLOWER is actually a very desirable thing for most of science. We are limited by how fast humans can wrap their heads around articles, and if we have fewer articles that are better written, that's a huge compound gain!
To the Distill team, if anyone is reading this: I don't think you should feel bad for being slow, or for doing "few" things at all. We humans to place big emphasis on superficial large numbers in the heat of the moment, but only good things withstand the test of time. I've only read a few Distill articles, but they were all really good and I can see myself coming back to most them 5-10 years from now. I don't think any other academic journal comes close in the ratio of (total goodness)/(total content). Good job Distill team for making a great thing, and summarizing the lessons learned so well in this goodbye article!
My only wish would be that you could find a way to continue to do auch good work that does not entirely rely on unpaid volunteering. In the end of the day volunteering only means some other institution bears the cost of supporting the volunteers.
For example: Could you get a Distill editor endowment to pay editors using donations throughout a non-profit fiscal sponsorship partner? Could you partner with a university, or even publisher, to support long term writing?
GOOD work takes TIME and is SLOW and we are bad at appreciating that. I hope the distill team keeps taking their time to put out good work, whatever it is they go do next!
Cheers to Distill!
As someone finishing a PhD, I think that doing LESS and doing it SLOWER is actually a very desirable thing for most of science. We are limited by how fast humans can wrap their heads around articles, and if we have fewer articles that are better written, that's a huge compound gain!
To the Distill team, if anyone is reading this: I don't think you should feel bad for being slow, or for doing "few" things at all. We humans to place big emphasis on superficial large numbers in the heat of the moment, but only good things withstand the test of time. I've only read a few Distill articles, but they were all really good and I can see myself coming back to most them 5-10 years from now. I don't think any other academic journal comes close in the ratio of (total goodness)/(total content). Good job Distill team for making a great thing, and summarizing the lessons learned so well in this goodbye article!
My only wish would be that you could find a way to continue to do auch good work that does not entirely rely on unpaid volunteering. In the end of the day volunteering only means some other institution bears the cost of supporting the volunteers.
For example: Could you get a Distill editor endowment to pay editors using donations throughout a non-profit fiscal sponsorship partner? Could you partner with a university, or even publisher, to support long term writing?
GOOD work takes TIME and is SLOW and we are bad at appreciating that. I hope the distill team keeps taking their time to put out good work, whatever it is they go do next!
Cheers to Distill!
Thanks for the kind remark!
> I don't think you should feel bad for being slow, or for doing "few" things at all.
Unfortunately, I think it's tricky to do this in a journal format. If you accept submissions, you'll have a constant flow of articles -- which vary greatly in quality -- who's authors very reasonably want timely help and a publication decision. And so it's very hard to go slow and do less, even if that's what would be right for you.
> Could you get a Distill editor endowment to pay editors using donations throughout a non-profit fiscal sponsorship partner? ...
I don't think funding is the primary problem. I'm personally fortunate to have a good job, and happily spend a couple thousand a year out of pocket to cover Distill's operating expenses.
I think the key problem is that Distill's structure means that we can't really control how much energy it takes from us, nor chose to focus our energy on the things about Distill that excites us.
> I don't think you should feel bad for being slow, or for doing "few" things at all.
Unfortunately, I think it's tricky to do this in a journal format. If you accept submissions, you'll have a constant flow of articles -- which vary greatly in quality -- who's authors very reasonably want timely help and a publication decision. And so it's very hard to go slow and do less, even if that's what would be right for you.
> Could you get a Distill editor endowment to pay editors using donations throughout a non-profit fiscal sponsorship partner? ...
I don't think funding is the primary problem. I'm personally fortunate to have a good job, and happily spend a couple thousand a year out of pocket to cover Distill's operating expenses.
I think the key problem is that Distill's structure means that we can't really control how much energy it takes from us, nor chose to focus our energy on the things about Distill that excites us.
Yeah, the real question shouldn't be whether something looks productive on the short term, but whether it has a good chance to help us move forward or not. Sometimes we move a lot... without going anywhere; and I don't think that's necessarily terrible, we can also learn a lot from it, but it definitely shouldn't be the only model. At some point you should be allowed to try to go on a long journey. Everyone needs to decide for themselves whether what they are making is relevant or not, and from there on we just need to trust people.
Too bad. In my opinion, Distill was the best thing that had happened to publish in the last 5 years.
I do understand that there is a lot of burden for editors. Also, from what I see - a lot of serious research now ends up on arXiv, and it does not matter if it gets published in a reviewed journal. (At least, in the last 2 years in deep learning, some breakthroughs are only as a PDF on OpenAI or as a website by Nvidia.)
Still - for static papers, it is acceptable to submit them to arXiv. For interactive ones, Distill is (was?) the only suitable venue.
On a more personal note, I am in the process of writing a paper (interactive, on tensor diagrams) for Distill. So well, it will end up as a blog post. It is OK-ish - most of my quality blog posts had orders of magnitude higher impact than my peer-reviewed papers. Still - a persistent DOI, editorial help from Chris Olah, would be game-changers.
I do understand that there is a lot of burden for editors. Also, from what I see - a lot of serious research now ends up on arXiv, and it does not matter if it gets published in a reviewed journal. (At least, in the last 2 years in deep learning, some breakthroughs are only as a PDF on OpenAI or as a website by Nvidia.)
Still - for static papers, it is acceptable to submit them to arXiv. For interactive ones, Distill is (was?) the only suitable venue.
On a more personal note, I am in the process of writing a paper (interactive, on tensor diagrams) for Distill. So well, it will end up as a blog post. It is OK-ish - most of my quality blog posts had orders of magnitude higher impact than my peer-reviewed papers. Still - a persistent DOI, editorial help from Chris Olah, would be game-changers.
I can't help with regard to editorial help from COlah but in terms of a DOI you could save your blog to a Zenodo repository and get the DOI from that. Alternatively if you're building your blog directly from GH repos you can link Zenodo to them and have it update the DOI automatically with each version change.
Thank you Chris, Nick, Sam and Janelle for putting things so eloquently. Distill is probably my favorite project I've had the fortune to contribute to, and that's largely due to you and everyone else involved. I'll continue to do my best to keep our "beautiful artifact" online indefinitely. <3
Distill has been an amazing journal, and I appreciate the analysis for suspending operations. Distill provides a detailed explanation of what didn’t work
Sad to see this. But also not very surprising, given distill has had relatively few or no articles from outside the ediotiral team for years. The combination of extremely polished, interactive articles, and usually great depth made writing for distill a huge challenge. Which makes it not very surprising it was a huge amount of work for the editorial team - I wonder how many articles they helped with that never reached completion. Combined with the lack of incentive for authors to take on the huge amount of work (relative to working on research papers). Still, I hope it indeed inspires more researchers to have personal blogs with in depth articles.
Out of curiosity I just ran some counts using `jq` on our metadata[0]:
We seem to have published 48 articles overall[1], of which 22 list at least one of our editors as an author. 10 of those list an editor as the first author. That leaves 26 articles entirely from outside the editorial team. Those are indeed predominantly from the more recent years; I'll try to perform an aggregation to quantify that.
[0] I'm sure there are more elegant ways to form these queries: https://gist.github.com/ludwigschubert/cd2bc4686e3fc6c8202c8...
[1] excluding editorial updates; if that still seems high: this includes every individual article in a "thread"; check out the [Differentiable Self-organizing Systems](https://distill.pub/2020/selforg/) thread, the [Circuits](https://distill.pub/2020/circuits/) thread, or the [Discussion of Adversarial Examples Are Not Bugs, They Are Features](https://distill.pub/2019/advex-bugs-discussion/).
[0] I'm sure there are more elegant ways to form these queries: https://gist.github.com/ludwigschubert/cd2bc4686e3fc6c8202c8...
[1] excluding editorial updates; if that still seems high: this includes every individual article in a "thread"; check out the [Differentiable Self-organizing Systems](https://distill.pub/2020/selforg/) thread, the [Circuits](https://distill.pub/2020/circuits/) thread, or the [Discussion of Adversarial Examples Are Not Bugs, They Are Features](https://distill.pub/2019/advex-bugs-discussion/).
This is so sad, it had such enormous potential. (Might be an opportunity for a lone blog to take the helm.)
A serious concern I have about self publishing (which doesn't appear to be addressed, hopefully I didn't miss it) is bit rot and long term format accessibility. arXiv (and traditional journals more generally) provides a reliable, versioned central repository. The contents appear in a predictable format and (mostly) adhere to an accepted set of styling conventions.
Today, you can retrieve scanned PDFs of many (most?) papers from the early 1900s. Other than the lack of color photographs and more simplistic figures, they're largely the same as their more modern counterparts. (But not quite - the lack of DOIs is downright painful.) I suspect that the entirety of arXiv will still be readily available in some form 100 years from now provided society doesn't suffer a total collapse; the same can't be said of a one off website as described in the article. (Perhaps versioning it on GitHub will prove to be enough?)
Related, probably the only frustration I had will Distill was that articles had a tendency to rely on assets hosted by third parties. Those are particularly prone to disappearing unexpectedly in my experience. I fear that one off websites will make this issue significantly more common.