Literate Programming: Articles(literateprogramming.com)
literateprogramming.com
Literate Programming: Articles
http://literateprogramming.com/articles.html
72 comments
> It adds overhead to refactoring. Refactoring code now means editing and rewriting a book
In my opinion, this is a good thing. The problem with ordinary refactoring is that it's too easy to just shuffle things around until they work/seem nicer/do whatever you want to. However, with literate programming, you have to think why you're refactoring, and how does it reflect in the explanation.
In my opinion, this is a good thing. The problem with ordinary refactoring is that it's too easy to just shuffle things around until they work/seem nicer/do whatever you want to. However, with literate programming, you have to think why you're refactoring, and how does it reflect in the explanation.
I use it in a commercial team to write technical documentation. I write it while, for example, exploring an API that I am integrating with. It produces a nice little document with important integration apis, limitations, open questions, integration decisions as well as interactive ways to check certain parts of the integration.
I do this with Typescript (deno) and a neovim plugin called sniprun.
I do this with Typescript (deno) and a neovim plugin called sniprun.
> It messes with tooling. If you're lucky then your editor will be smart enough to syntax highlight inside code blocks, or can be taught to do so easily. It's unlikely that more advanced IDE-style features will work.
Probably a niche example and the exception that proves the rule, but in Racket, the included literate programming (Scribble/LP2) is itself a language implemented in Racket. Racket’s IDE and tools for exposing and inspecting syntax work just as well in that environment as in any other Racket-implemented language.
https://docs.racket-lang.org/scribble/lp.html
Probably a niche example and the exception that proves the rule, but in Racket, the included literate programming (Scribble/LP2) is itself a language implemented in Racket. Racket’s IDE and tools for exposing and inspecting syntax work just as well in that environment as in any other Racket-implemented language.
https://docs.racket-lang.org/scribble/lp.html
> Obviously tools could be adapted with custom plugins but literate programming isn't popular enough for these to already exist as far as I'm aware.
I’ve heard that org-babel [https://orgmode.org/worg/org-contrib/babel/intro.html] works pretty well for this, though I’ve never used it myself.
I’ve heard that org-babel [https://orgmode.org/worg/org-contrib/babel/intro.html] works pretty well for this, though I’ve never used it myself.
Org Babel is really nice, but it doesn't really solve the problem, rather it sidesteps it my giving you the option to edit a block of code in a dedicated buffer. I don't know how well that works with LSP (I don't use LSP) but it does allow you to make full use of SLIME.
lsp-mode does support tooling in source code blocks without opening it in a dedicated buffer by creating a virtual one backing it.
See https://emacs-lsp.github.io/lsp-mode/manual-language-docs/ls...
See https://emacs-lsp.github.io/lsp-mode/manual-language-docs/ls...
their is no tooling for it because no one has tried to build it. all you need is a markdown file with links to the code bookmarks. every modern editor can resolve those links to the actual location of the code
>* The "includes" problem. Most language require you to include/import/require packages used in a file. These are usually all placed at the start. This means a lot of chapters will start with "here are all the includes we'll need" if you're using a linear format. More complex formats that rearrange the code to generate outputs can do a better job but it's still a little clunky.
Real literate programming, rather than rich text comments, can order the code blocks in any order. You can add then at the very end of the book/chapter/section if you feel like it.
I did some literary programs about a previous generation of our program. Every new hire reads them and asks me for copies.
Real literate programming, rather than rich text comments, can order the code blocks in any order. You can add then at the very end of the book/chapter/section if you feel like it.
I did some literary programs about a previous generation of our program. Every new hire reads them and asks me for copies.
> Real literate programming, rather than rich text comments, can order the code blocks in any order. You can add then at the very end of the book/chapter/section if you feel like it.
And yet Knuth still does it the other way. <http://akkartik.name/post/literate-programming>
And yet Knuth still does it the other way. <http://akkartik.name/post/literate-programming>
Does it matter though? The primary purpose of using literate programming is expository. If Knuth feels that putting the includes at the start makes sense, then that's what he does. If you don't, then fine you move it to the end or the middle. That's the benefit of WEB (and WEB-derived systems). The presentation order is not dependent on the code order, it's dependent on what makes the most expository sense.
When I've written literate programs, I almost always shuffle long lists of includes to the end. They add little or no value at the top and distract from the material I want to present. It's also a simple cut and paste to move them back to the top if I wanted to. The only time I leave them at the top is if there's something actually informative about having them at the top, or it's a short program, or I've not actually finished working on it (a lot of my literate programs start as traditional "live in source files" programs that I slurp into org files).
When I've written literate programs, I almost always shuffle long lists of includes to the end. They add little or no value at the top and distract from the material I want to present. It's also a simple cut and paste to move them back to the top if I wanted to. The only time I leave them at the top is if there's something actually informative about having them at the top, or it's a short program, or I've not actually finished working on it (a lot of my literate programs start as traditional "live in source files" programs that I slurp into org files).
> Does it matter though?
Yes.
> The primary purpose of using literate programming is expository.
Right. A big, zero-context block of includes prefaced with a comment that you should just "skip ahead" past it to the "interesting stuff" and offering no insight about the stuff that you're skipping over is the opposite of expository. The main program—what it actually does—is sufficient to serve as exposition for the includes that it ends up needing to be put it. So actually write it that way.
> That's the benefit of WEB[...] The presentation order is not dependent on the code order, it's dependent on what makes the most expository sense.
The argument is that it doesn't make expository sense to write the includes the way Knuth is. It's just yet another form, as taeric observes, of boilerplate—which makes us a slave to what the (LP-unaware) compiler expects, and which LP is supposed to liberating us from...
* * * *
It strikes me that there are probably only two ways to really deal with includes in the spirit of literate programming, which is to either (a) put them at the end after having already shown us why they're necessary (accompanied with with e.g. a comment along the lines of "Recall that our program is using printf—a part of the C standard library and defined in stdio.h—so it's necessary, therefore, that we include it in order for our program to compile") or (b) to use WEB/CWEB's appending facilities—after having written a routine that uses printf for the first time, you write an interlude not altogether different from what I just described that adds that include to the "running sum" of necessary includes, which the tangle step will take care of as part of the build.
(I say "only" because these are the only ways that occur to me that it can reasonably be done. I'm not committed to that being true—I'm open to the possibility of there being more—and it's not as if I started with that in mind, but it's hard for to imagine others. The only thing I'm really committed to is that what the article says about Knuth's two examples being wrong is accurate. I know Kartik has softened his stance since originally writing it, but I haven't. The examples given are individually each an awful way to demonstrate LP considering how antithetical that is to the whole thing. Totally indefensible.)
Yes.
> The primary purpose of using literate programming is expository.
Right. A big, zero-context block of includes prefaced with a comment that you should just "skip ahead" past it to the "interesting stuff" and offering no insight about the stuff that you're skipping over is the opposite of expository. The main program—what it actually does—is sufficient to serve as exposition for the includes that it ends up needing to be put it. So actually write it that way.
> That's the benefit of WEB[...] The presentation order is not dependent on the code order, it's dependent on what makes the most expository sense.
The argument is that it doesn't make expository sense to write the includes the way Knuth is. It's just yet another form, as taeric observes, of boilerplate—which makes us a slave to what the (LP-unaware) compiler expects, and which LP is supposed to liberating us from...
* * * *
It strikes me that there are probably only two ways to really deal with includes in the spirit of literate programming, which is to either (a) put them at the end after having already shown us why they're necessary (accompanied with with e.g. a comment along the lines of "Recall that our program is using printf—a part of the C standard library and defined in stdio.h—so it's necessary, therefore, that we include it in order for our program to compile") or (b) to use WEB/CWEB's appending facilities—after having written a routine that uses printf for the first time, you write an interlude not altogether different from what I just described that adds that include to the "running sum" of necessary includes, which the tangle step will take care of as part of the build.
(I say "only" because these are the only ways that occur to me that it can reasonably be done. I'm not committed to that being true—I'm open to the possibility of there being more—and it's not as if I started with that in mind, but it's hard for to imagine others. The only thing I'm really committed to is that what the article says about Knuth's two examples being wrong is accurate. I know Kartik has softened his stance since originally writing it, but I haven't. The examples given are individually each an awful way to demonstrate LP considering how antithetical that is to the whole thing. Totally indefensible.)
My point, which you missed: It doesn't matter what Knuth does.
thr-nrg rightly pointed out in their comment that you can place the code wherever you want. You then say, "Knuth puts it at the top" (meant as a criticism of his style).
My point (so you don't miss it again): You can place the code wherever you want when doing a literate programming style. It also doesn't matter what Knuth does with his includes because his style does not dictate your style or my style or anyone else's style. So put it wherever you want. His style does not matter.
thr-nrg rightly pointed out in their comment that you can place the code wherever you want. You then say, "Knuth puts it at the top" (meant as a criticism of his style).
My point (so you don't miss it again): You can place the code wherever you want when doing a literate programming style. It also doesn't matter what Knuth does with his includes because his style does not dictate your style or my style or anyone else's style. So put it wherever you want. His style does not matter.
I didn't miss it. I replied to your question of whether it matters: yes, it does.
Your point doesn't make any sense. ("_You_ can place the code wherever _you_ want when doing a literate programming style.") Literate programs are supposed to be read. Of course a writer can write however they want. You could write a book where everything that isn't a proper noun ends with an uppercase letter, except for proper nouns, which are all lowercase. It will be shit.
Your point doesn't make any sense. ("_You_ can place the code wherever _you_ want when doing a literate programming style.") Literate programs are supposed to be read. Of course a writer can write however they want. You could write a book where everything that isn't a proper noun ends with an uppercase letter, except for proper nouns, which are all lowercase. It will be shit.
What matters is the exposition. Does the presentation (order, style, etc.) communicate what you intend for it to communicate, and does it communicate effectively. That's what matters.
Does it matter what Knuth does? Nope. I don't care for that aspect of his style. I already said I put the includes (when a text gets past the draft stage at least) toward the end. We're in violent agreement with respect to how we (if I understood you correctly) often or generally put them into LP texts.
You're just weirdly caught up on criticizing Knuth's style for some reason. Have fun shouting into the wind. The rest of us know, it doesn't matter. The goal is to communicate effectively and not whine about things that don't matter.
Does it matter what Knuth does? Nope. I don't care for that aspect of his style. I already said I put the includes (when a text gets past the draft stage at least) toward the end. We're in violent agreement with respect to how we (if I understood you correctly) often or generally put them into LP texts.
You're just weirdly caught up on criticizing Knuth's style for some reason. Have fun shouting into the wind. The rest of us know, it doesn't matter. The goal is to communicate effectively and not whine about things that don't matter.
I don't think I'm the one missing the point here. You're adamant that I'm complaining about Knuth's style. I'm complaining that programs he's written don't show us that exposition matters (rather than him just telling us that it does).
Exposition does matter. He's not doing it. That's the problem.
Dumping a big lump of include statements or defines at the beginning of the text and then saying don't worry about it isn't an example of someone who actually believes that exposition matters. It's incontrovertibly devoid of exposition—not just a matter of it being one's "style".
> What matters is the exposition.
> Does it matter what Knuth does? Nope.
Either it matters, or it doesn't. Pick one.
Exposition does matter. He's not doing it. That's the problem.
Dumping a big lump of include statements or defines at the beginning of the text and then saying don't worry about it isn't an example of someone who actually believes that exposition matters. It's incontrovertibly devoid of exposition—not just a matter of it being one's "style".
> What matters is the exposition.
> Does it matter what Knuth does? Nope.
Either it matters, or it doesn't. Pick one.
> Exposition does matter. He's not doing it. That's the problem.
I agree. Here's an example I give of an emacs configuration in the typical style which I don't find especially useful versus one that does use exposition:
https://www.paretooptimal.dev/finding-the-best-style-of-lite...
Here's a sample:
I agree. Here's an example I give of an emacs configuration in the typical style which I don't find especially useful versus one that does use exposition:
https://www.paretooptimal.dev/finding-the-best-style-of-lite...
Here's a sample:
* My emacs use-cases
** Version control with Git
*** use magit as a porcelain
#+begin_src elisp
(use-package magit)
#+end_src
*** enable easily showing history of a file
#+begin_src elisp
(use-package git-time-machine)
#+end_srcI'm not entirely sure that trying to not have boilerplate of any kind is a worthy goal. Consider, no book is wanting to get rid of the title page. Even a dedication page has grown to be a near required page that we literally teach students to write.
To that end, the goal is to not have parts that can be the same between programs vary. That has pedagogical value. Is a type of stability that we don't really value much anymore.
Though, I also give a +1 to jtsummers' point. Knuth has a style of writing literate programs. And if you find some old videos of his reviewing student's code, he explicitly calls out points of style that he likes from different students and how they work together. He didn't necessarily want to push a "this is how you style your programs," but instead was working how you talk about programs into a way that you can also write them. While not having to relearn either, necessarily.
To that end, the goal is to not have parts that can be the same between programs vary. That has pedagogical value. Is a type of stability that we don't really value much anymore.
Though, I also give a +1 to jtsummers' point. Knuth has a style of writing literate programs. And if you find some old videos of his reviewing student's code, he explicitly calls out points of style that he likes from different students and how they work together. He didn't necessarily want to push a "this is how you style your programs," but instead was working how you talk about programs into a way that you can also write them. While not having to relearn either, necessarily.
> I'm not entirely sure that trying to not have boilerplate of any kind is a worthy goal.
I think it's a worthy goal for nearly all readers. However I also believe that literate programmers should be written with many different readers in mind.
So you could have one version that is written in a "typical code" style like:
For example:
I think it's a worthy goal for nearly all readers. However I also believe that literate programmers should be written with many different readers in mind.
So you could have one version that is written in a "typical code" style like:
* library 1
** module 1
*** source file 1
** module 2
*** source file 2
For example:
* typical program style
** library 1
*** module 1
**** source file 1
*** module 2
**** source file 2
* new to language X presentation of program
** link to various parts of library 1 alongside prose to describe concepts to language newcomer
* typical hire presentation of program
** business domain focused exposition presenting code via transclusion as needed from typical program stylePeople will inevitably complain that what you're advocating for involves too much duplication, but I've independently reached the same conclusion that you have. Besides, in a good LP system, the toolchain would be able to consume the second part and transform and emit the first part.
See also this conversation about the unreadability of Knuth's .web files: <https://news.ycombinator.com/item?id=29203798#29205840>
See also this conversation about the unreadability of Knuth's .web files: <https://news.ycombinator.com/item?id=29203798#29205840>
> Consider, no book is wanting to get rid of the title page.
Lists of imports are not title pages.
> the goal is to not have parts that can be the same between programs vary
Lists of imports vary. (And in today's world of extreme software reuse, which other pieces of software a program depends on is of greater interest than ever.)
I'd take your point on boilerplate if it were necessary and unavoidable. I've already described, however, that it isn't and how.
Lists of imports are not title pages.
> the goal is to not have parts that can be the same between programs vary
Lists of imports vary. (And in today's world of extreme software reuse, which other pieces of software a program depends on is of greater interest than ever.)
I'd take your point on boilerplate if it were necessary and unavoidable. I've already described, however, that it isn't and how.
Depends on the list? The ones you linked from Knuth are basically fixed imports that are, in fact, very common and easy to take as a whole.
I think the disconnect is still that you are writing a C program with CWEB. If I'm using a literate style to write JavaScript, I'm still writing a JavaScript program. As such, the point isn't to just throw any established norms out, but to rearrange for presentation. If some items go well together, might as well keep them together.
Most programming tools that can do this require you to "bundle" things such that you have to have well formed containers, if you will. Having the "add this to a section" constructs really help, as then you can say things like "add this to the imports" the first time you have need of a new import. But, if you don't add any non-standard imports, you can forego doing that.
The other approach is to try and make everything implied, such that new users can skip any of the boilerplate and jump straight to programming. That is fine, for what it is, but does little to build understanding of the programs as a whole.
That said, I may have missed your described solution. I'm kind of scattershot today and not keeping fully on top of this. Apologies if I'm talking past you.
I think the disconnect is still that you are writing a C program with CWEB. If I'm using a literate style to write JavaScript, I'm still writing a JavaScript program. As such, the point isn't to just throw any established norms out, but to rearrange for presentation. If some items go well together, might as well keep them together.
Most programming tools that can do this require you to "bundle" things such that you have to have well formed containers, if you will. Having the "add this to a section" constructs really help, as then you can say things like "add this to the imports" the first time you have need of a new import. But, if you don't add any non-standard imports, you can forego doing that.
The other approach is to try and make everything implied, such that new users can skip any of the boilerplate and jump straight to programming. That is fine, for what it is, but does little to build understanding of the programs as a whole.
That said, I may have missed your described solution. I'm kind of scattershot today and not keeping fully on top of this. Apologies if I'm talking past you.
I missed this conversation earlier, but see my response to that post here: https://news.ycombinator.com/item?id=29871047 (also see https://news.ycombinator.com/item?id=30762055) — in short, literate programming is writing; writing is done with an audience in mind; you don't necessarily have to explain everything; Knuth only chooses to explain the tricky stuff rather than language features or obvious (to him) #includes, especially in these throwaway programs that he wrote for himself.
Note the end of the sentence you quoted: “if you feel like it” — Knuth clearly doesn't feel like it in these programs. (BTW, it appears he generally adds a comment next to each #include mentioning why it's being included.)
Literate programming, for Knuth, is supposed to be a fun way to write programs that leads to them being easier to write and read and debug and maintain; it's not some sort of purity discipline that requires everything to be maximally expository.
Note the end of the sentence you quoted: “if you feel like it” — Knuth clearly doesn't feel like it in these programs. (BTW, it appears he generally adds a comment next to each #include mentioning why it's being included.)
Literate programming, for Knuth, is supposed to be a fun way to write programs that leads to them being easier to write and read and debug and maintain; it's not some sort of purity discipline that requires everything to be maximally expository.
> Now, writing is always (best) done with a specific reader in mind: you assume the reader has a certain background/prerequisites: some things that don't need explaining, and some things that do.
I don't buy it. The same explanation can be used to excuse the use of traditional (non-LP) programming systems—raising questions about why make a fuss about LP at all at that point if an LP text is going to treat the same types of shortcomings as a given.
As the commenters on Ward's wiki pointed out, Knuth's examples come across as still being written for the compiler—what he's mostly succeeded at is just coming up for a different syntax for it to consume in a roundabout way.
> Depending on the reader you're targeting (e.g., yourself a few years from now), and how polished you're trying to make your presentation, you may well choose to take it for granted and not bother explaining that a C program will have some obvious #includes at the top.
But of all the things that Knuth could explain, that's the one thing he does explain. It's not a lack of an explanation that the program will have some includes that is the problem. It's the impertinence of immediately dumping a list of includes on the reader, in a total failure in exposition.
We don't have to focus exclusively on the includes to see this problem. The define at the top of the Symmetric Hamiltonian cycles exhibits the same thing just as clearly.
I have another theory that I've floated, which is that Knuth realized that there's something wrong with how traditional Pascal and C compilers force you to write/read your programs bottom-up, but by the time he came up with literate programming he was already so warped and tainted from years of doing work in the bottom-up tradition to satisfy the compiler that it ends up clouding his vision, even when he knows that's the mindset he's trying to actively work against.
I don't buy it. The same explanation can be used to excuse the use of traditional (non-LP) programming systems—raising questions about why make a fuss about LP at all at that point if an LP text is going to treat the same types of shortcomings as a given.
As the commenters on Ward's wiki pointed out, Knuth's examples come across as still being written for the compiler—what he's mostly succeeded at is just coming up for a different syntax for it to consume in a roundabout way.
> Depending on the reader you're targeting (e.g., yourself a few years from now), and how polished you're trying to make your presentation, you may well choose to take it for granted and not bother explaining that a C program will have some obvious #includes at the top.
But of all the things that Knuth could explain, that's the one thing he does explain. It's not a lack of an explanation that the program will have some includes that is the problem. It's the impertinence of immediately dumping a list of includes on the reader, in a total failure in exposition.
We don't have to focus exclusively on the includes to see this problem. The define at the top of the Symmetric Hamiltonian cycles exhibits the same thing just as clearly.
I have another theory that I've floated, which is that Knuth realized that there's something wrong with how traditional Pascal and C compilers force you to write/read your programs bottom-up, but by the time he came up with literate programming he was already so warped and tainted from years of doing work in the bottom-up tradition to satisfy the compiler that it ends up clouding his vision, even when he knows that's the mindset he's trying to actively work against.
I was thinking of this again this weekend, after spending a couple of hours reading some literate programs again, in light of Knuth's recent (via https://twitter.com/ksmeel/status/1661890306370072576) note https://cs.stanford.edu/~knuth/papers/cvm-note.pdf and associated program https://shreevatsa.github.io/knuth-literate-programs/program..., a CWEB program he wrote a few days ago. It's been 9–10 days here and I'm not sure anyone will read this (I'm surprised HN still lets me reply), but anyway…
It seems that your main complaint is that in Knuth's literate programs, he does not explain enough. To which my answer is: Sure! LP doesn't mean you have to explain everything, you can choose to explain as much you want. It's not a "moralism" like "structured programming"; it's just a tool.
I think everything you're saying is along the same lines:
> The same explanation can be used to excuse the use of traditional (non-LP) programming systems—raising questions about why make a fuss about LP at all at that point if an LP text is going to treat the same types of shortcomings as a given.
Firstly, "excuse" seems to suggest an accusation that something is wrong with non-literate programming, which has not been made except as a joke — LP is just presented as a tool, with the expectation that it won't work for everyone. (Knuth is against moralism in programming style, and has often complained about it, e.g. in the context of defending GOTO and comparing pointers and what not.) So your question is basically asking: why attempt to explain at all, if one is not going to explain everything? The answer is, even without explaining everything, explaining as much as he does seems to work for him; he's been doing this for 40 years now and continues to rave about it.
> But of all the things that Knuth could explain, that's the one thing he does explain.
(I didn't understand this part.)
> immediately dumping a list of includes on the reader
As a rule of thumb, Knuth has said somewhere (maybe in the early days of LP, for TeX), that he targeted around 12 lines per LP section. So I would imagine that if he ever wrote a program that had more than about 12 includes (which is pretty hard to imagine :-)), he would split up the list of includes into multiple sections, presented separately. Below that (like the four or five includes in these examples), there's not much value in splitting it up further. I guess there's a lesson/hint here that splitting something up into sections is not "free" and below some threshold starts having more cost than benefit. (Just like the cost with having lots of short functions in modern-day non-LP programs: Ousterhout's A Philosophy of Software Design has some succinct words about that: https://web.stanford.edu/~ouster/cgi-bin/aposd2ndEdExtract.p....)
In any case, not having separate sections for #includes is not a failure of imagination: if you look at the programs on his webpage, the very first example ("used as a handout for a lecture on literate programming") has #includes separated out: https://shreevatsa.github.io/knuth-literate-programs/program... — note that in section 2, after a self-mocking joke ("First we brush up our Shakespeare by quoting from The Merchant of Venice … This makes the program literate."), he uses printf, and in the next section includes stdio.h ("Since we’re using the printf routine"), and then in section 7 he has another include, with the words "UNIX’s localtime function does most of the work for us, but we need to include another system header file before we can use it." So the fact that he presented includes like this in his early demonstration program (October 1992), and does not bother to do so in later programs (including last week: May 2023) seems worth thinking about.
> The define at the top of the Symmetric Hamiltonian cycles exhibits the same thing just as clearly.
As mentioned before, this is one of the programs that "use the conventions and library of The Stanford GraphBase". If you look at the program (https://shreevatsa.github.io/knuth-literate-programs/program...), it starts with "We use a utility field to record the vertex degrees" and then has "#define deg u.I". This is in fact part of those conventions — in the SGB book index, there's an entry for "utility fields" pointing to pages 38–39 and 284, where this is explained:
> Every Vertex record contains eight subfields. We have already mentioned name and arcs; the other six subfields are called utility fields because they can be used for many different purposes. […] The six utility fields are named u, v, w, x, y, z, and their five possible interpretations are distinguished by adding one of the respective suffixes .I, .S, .G, .V, .A; thus, for example, v->w.I stands for the integer in utility field w of the Vertex record pointed to by v.
> Utility field names are usually given meaningful aliases by means of macro definitions. For example, GB-GAMES defines nickname to mean y.S.
and so on, and the book is full of definitions like "#define source a.V" and "#define back_arcs z.A" — so when the SHAM program begins with "we use a utility field" and then "#define deg u.I", this is established convention, not some wild thing pulled out of nowhere. When Kartik says “presumably a struct whose definition — whose very type name — we haven't even seen yet. (The variable name for the struct is also hard-coded in”, all of these presumptions are wrong: `u` is not a hard-coded variable name but the name of a field in Vertex (over the course of the program we can see v->deg, x->deg, u->deg, a->tip->deg: there's an index entry for "deg"), and the Vertex struct's definition and type name are well-documented (in a published book). I find it easy to take at face value that this was really the order in which Knuth thought of things, and also the level of exposition that he finds most useful (for his intended audience, which is himself).
This points at a problem with literate programming, that I also mentioned in the other thread: because it can be so personal, everyone who uses LP has their own idea of what is most worth explaining, and even ends up building their own LP tools. (Similar things are said about every Lisp programmer ending up with their own idioms and mini-languages.)
> he was already so warped and tainted from years of doing work in the bottom-up tradition to satisfy the compiler that it ends up clouding his vision
Another way of saying this is that Knuth never believes in hiding the fact that you're writing code that a compiler will read and that a computer will execute — in fact he continues to annotate variables with "register" even though compilers ignore it, simply because he likes to be conscious about what instructions the (ideal) machine will execute — he's not a big believer in abstraction and hiding the details by passing to a higher level; he believes in (and somehow manages) constantly being aware of all levels at once.
Yet another way of saying this (my theory) is that when machine code / assembly programs became too hard to maintain, the rest of the world solved the problem by, over time, settling on abstraction, interfaces, higher-level languages, style conventions, information-hiding, and all that. Instead, Knuth has forged his own programming path/style, still basically writing machine-sympathetic programs, but "explaining to human beings what we want a computer to do". It is up to others to merge LP's human-orientation with mainstream ideas… if it will ever happen.
Anyway, it seems that the criticisms we're discussing are mostly based on first impressions. While they are valuable (and indicative of what others' first impressions will be), ultimately I think criticisms based on studying these programs more closely would be more interesting. E.g. has his LP style evolved over time? What can we learn from studying recent programs (there are over a dozen since 2020 alone, like the program above or others posted on his webpage and which I just typeset yesterday at https://shreevatsa.github.io/knuth-literate-programs/program...) — what can we learn from what he chooses to explain and not; why does he make those choices; what way does this seem to help him: can we use these insights (not the same style) for our own programming practice? Things like that.
It seems that your main complaint is that in Knuth's literate programs, he does not explain enough. To which my answer is: Sure! LP doesn't mean you have to explain everything, you can choose to explain as much you want. It's not a "moralism" like "structured programming"; it's just a tool.
I think everything you're saying is along the same lines:
> The same explanation can be used to excuse the use of traditional (non-LP) programming systems—raising questions about why make a fuss about LP at all at that point if an LP text is going to treat the same types of shortcomings as a given.
Firstly, "excuse" seems to suggest an accusation that something is wrong with non-literate programming, which has not been made except as a joke — LP is just presented as a tool, with the expectation that it won't work for everyone. (Knuth is against moralism in programming style, and has often complained about it, e.g. in the context of defending GOTO and comparing pointers and what not.) So your question is basically asking: why attempt to explain at all, if one is not going to explain everything? The answer is, even without explaining everything, explaining as much as he does seems to work for him; he's been doing this for 40 years now and continues to rave about it.
> But of all the things that Knuth could explain, that's the one thing he does explain.
(I didn't understand this part.)
> immediately dumping a list of includes on the reader
As a rule of thumb, Knuth has said somewhere (maybe in the early days of LP, for TeX), that he targeted around 12 lines per LP section. So I would imagine that if he ever wrote a program that had more than about 12 includes (which is pretty hard to imagine :-)), he would split up the list of includes into multiple sections, presented separately. Below that (like the four or five includes in these examples), there's not much value in splitting it up further. I guess there's a lesson/hint here that splitting something up into sections is not "free" and below some threshold starts having more cost than benefit. (Just like the cost with having lots of short functions in modern-day non-LP programs: Ousterhout's A Philosophy of Software Design has some succinct words about that: https://web.stanford.edu/~ouster/cgi-bin/aposd2ndEdExtract.p....)
In any case, not having separate sections for #includes is not a failure of imagination: if you look at the programs on his webpage, the very first example ("used as a handout for a lecture on literate programming") has #includes separated out: https://shreevatsa.github.io/knuth-literate-programs/program... — note that in section 2, after a self-mocking joke ("First we brush up our Shakespeare by quoting from The Merchant of Venice … This makes the program literate."), he uses printf, and in the next section includes stdio.h ("Since we’re using the printf routine"), and then in section 7 he has another include, with the words "UNIX’s localtime function does most of the work for us, but we need to include another system header file before we can use it." So the fact that he presented includes like this in his early demonstration program (October 1992), and does not bother to do so in later programs (including last week: May 2023) seems worth thinking about.
> The define at the top of the Symmetric Hamiltonian cycles exhibits the same thing just as clearly.
As mentioned before, this is one of the programs that "use the conventions and library of The Stanford GraphBase". If you look at the program (https://shreevatsa.github.io/knuth-literate-programs/program...), it starts with "We use a utility field to record the vertex degrees" and then has "#define deg u.I". This is in fact part of those conventions — in the SGB book index, there's an entry for "utility fields" pointing to pages 38–39 and 284, where this is explained:
> Every Vertex record contains eight subfields. We have already mentioned name and arcs; the other six subfields are called utility fields because they can be used for many different purposes. […] The six utility fields are named u, v, w, x, y, z, and their five possible interpretations are distinguished by adding one of the respective suffixes .I, .S, .G, .V, .A; thus, for example, v->w.I stands for the integer in utility field w of the Vertex record pointed to by v.
> Utility field names are usually given meaningful aliases by means of macro definitions. For example, GB-GAMES defines nickname to mean y.S.
and so on, and the book is full of definitions like "#define source a.V" and "#define back_arcs z.A" — so when the SHAM program begins with "we use a utility field" and then "#define deg u.I", this is established convention, not some wild thing pulled out of nowhere. When Kartik says “presumably a struct whose definition — whose very type name — we haven't even seen yet. (The variable name for the struct is also hard-coded in”, all of these presumptions are wrong: `u` is not a hard-coded variable name but the name of a field in Vertex (over the course of the program we can see v->deg, x->deg, u->deg, a->tip->deg: there's an index entry for "deg"), and the Vertex struct's definition and type name are well-documented (in a published book). I find it easy to take at face value that this was really the order in which Knuth thought of things, and also the level of exposition that he finds most useful (for his intended audience, which is himself).
This points at a problem with literate programming, that I also mentioned in the other thread: because it can be so personal, everyone who uses LP has their own idea of what is most worth explaining, and even ends up building their own LP tools. (Similar things are said about every Lisp programmer ending up with their own idioms and mini-languages.)
> he was already so warped and tainted from years of doing work in the bottom-up tradition to satisfy the compiler that it ends up clouding his vision
Another way of saying this is that Knuth never believes in hiding the fact that you're writing code that a compiler will read and that a computer will execute — in fact he continues to annotate variables with "register" even though compilers ignore it, simply because he likes to be conscious about what instructions the (ideal) machine will execute — he's not a big believer in abstraction and hiding the details by passing to a higher level; he believes in (and somehow manages) constantly being aware of all levels at once.
Yet another way of saying this (my theory) is that when machine code / assembly programs became too hard to maintain, the rest of the world solved the problem by, over time, settling on abstraction, interfaces, higher-level languages, style conventions, information-hiding, and all that. Instead, Knuth has forged his own programming path/style, still basically writing machine-sympathetic programs, but "explaining to human beings what we want a computer to do". It is up to others to merge LP's human-orientation with mainstream ideas… if it will ever happen.
Anyway, it seems that the criticisms we're discussing are mostly based on first impressions. While they are valuable (and indicative of what others' first impressions will be), ultimately I think criticisms based on studying these programs more closely would be more interesting. E.g. has his LP style evolved over time? What can we learn from studying recent programs (there are over a dozen since 2020 alone, like the program above or others posted on his webpage and which I just typeset yesterday at https://shreevatsa.github.io/knuth-literate-programs/program...) — what can we learn from what he chooses to explain and not; why does he make those choices; what way does this seem to help him: can we use these insights (not the same style) for our own programming practice? Things like that.
> It's been 9–10 days here and I'm not sure anyone will read this (I'm surprised HN still lets me reply)
HN gives you 14 days.
(I wish it had a feature to "mark" a comment in lieu of writing a reply then and there. When 12 or 13 days have passed, it'd send you an email reminding you that this is your last chance to add what you have to say to the record. In this way, it would favor people going away and coming back to leave more thoughtful comments instead of impelling them to reply instantaneously i.e. to avoid the risk of not being heard.)
I suppose the hack would be to set your delay in your profile to 60 × 24 × 12.5 = 18000 (minutes), post your reply, then reset it to its initial value. You'd progressively edit your comment as the opportunity comes up, and then it will be autoposted in whatever form its in when the delay elapses. You can only edit posts for up to an hour (or something), though, and I'm not sure whether that timer starts from the time you hit submit or when it appears in the comments (i.e. after the delay has elapsed).
HN gives you 14 days.
(I wish it had a feature to "mark" a comment in lieu of writing a reply then and there. When 12 or 13 days have passed, it'd send you an email reminding you that this is your last chance to add what you have to say to the record. In this way, it would favor people going away and coming back to leave more thoughtful comments instead of impelling them to reply instantaneously i.e. to avoid the risk of not being heard.)
I suppose the hack would be to set your delay in your profile to 60 × 24 × 12.5 = 18000 (minutes), post your reply, then reset it to its initial value. You'd progressively edit your comment as the opportunity comes up, and then it will be autoposted in whatever form its in when the delay elapses. You can only edit posts for up to an hour (or something), though, and I'm not sure whether that timer starts from the time you hit submit or when it appears in the comments (i.e. after the delay has elapsed).
I think the idea of a general preamble that would have the same explanation, every time, is fine? This is literally the boilerplate concept.
The idea is more to break the code into parts you would explain and grok easily. Not every atomic part of the code.
The idea is more to break the code into parts you would explain and grok easily. Not every atomic part of the code.
See my response to Jtsummers. <https://news.ycombinator.com/item?id=35989931>
And? Knuth also used Pascal for the original Web and TeX. Both are still Pascal programs that get transpiled to C before being compiled. Just because he does something doesn't mean you should copy it religiously.
Web and its derivatives are sufficiently advanced that they suffer from the lisp curse. What in other systems are major fundamental engineering problems - try adding an include in the middle of a c file - in Web derivates are a matter of taste. Do you have a chunk that picks up includes as you need them? Is it a big one at the front or back? It's up to you.
Web and its derivatives are sufficiently advanced that they suffer from the lisp curse. What in other systems are major fundamental engineering problems - try adding an include in the middle of a c file - in Web derivates are a matter of taste. Do you have a chunk that picks up includes as you need them? Is it a big one at the front or back? It's up to you.
> And?
And nothing. For programs that are ostensibly meant to be read, Knuth's examples are poor ones.
> in Web derivates are a matter of taste
Dumping a bunch of includes at the top and saying the equivalent of "don't worry about this boring stuff for now; you can skip it if you want" (and thereby compromising the entire LP experiment) is a matter of taste in the way that putting beef in a vegan casserole is.
And nothing. For programs that are ostensibly meant to be read, Knuth's examples are poor ones.
> in Web derivates are a matter of taste
Dumping a bunch of includes at the top and saying the equivalent of "don't worry about this boring stuff for now; you can skip it if you want" (and thereby compromising the entire LP experiment) is a matter of taste in the way that putting beef in a vegan casserole is.
Actually, I have been doing this for some stuff now. I wrote my copy of tangle (based on noweb's notangle). I'm starting to write markdown files to that a literate programs for Dockerfile images (as well as the support scripts used in the docker images and support scripts to build/test/etc.).
Since all of this work is in a git repo, it is nice to pass around documents so folks understand the thinking behind things.
Also, (and the main reason I went towards it) is that when I pull what I need from the markdown files, I have steps built around to lint, test, and do additional checks to ensure the required naming, labeling, etc.
It has worked well and reduced many `silly' errors when other folks use the images.
Is it for everybody? Nope. But it works here.
I also did LP many years ago when I worked at a place that produces custom software for embedded systems. Providing them with a friggin' novel about the program was also fun. But, we never .. once .. were told we needed to provide more documentation.
LP has good points, but it isn't a one-size-fits-all.
When asked to send some coding samples, it is fun to have a PDF file that starts 'Chapter 3: And Now, Security! Encryption and Other Pain Points!'
Since all of this work is in a git repo, it is nice to pass around documents so folks understand the thinking behind things.
Also, (and the main reason I went towards it) is that when I pull what I need from the markdown files, I have steps built around to lint, test, and do additional checks to ensure the required naming, labeling, etc.
It has worked well and reduced many `silly' errors when other folks use the images.
Is it for everybody? Nope. But it works here.
I also did LP many years ago when I worked at a place that produces custom software for embedded systems. Providing them with a friggin' novel about the program was also fun. But, we never .. once .. were told we needed to provide more documentation.
LP has good points, but it isn't a one-size-fits-all.
When asked to send some coding samples, it is fun to have a PDF file that starts 'Chapter 3: And Now, Security! Encryption and Other Pain Points!'
I think there's some distinctions that need to be made. IMHO, Literate programming can be an umbrella for a bunch of different things.
Is it "notebook" style programming, like Mathematica, matlab, Jupyter, and markdown-based stuff like R/python + Quarto? In this mode, I think the focus is more on creating a document or communicating stuff typically with data analysis. And actually, "literate" is a very apt adjective for something that's ultimately intended to be a document of some kind.
Or is it more general intended for writing arbitrary software applications and having everything in the source code? In other words, to understand the source, you just read the source and there are no other documents.
The latter mode, I think, is still very much not around as far as I know (academic proof-of-concepts for papers don't count :-) ). Will it ever really exist?
Is it "notebook" style programming, like Mathematica, matlab, Jupyter, and markdown-based stuff like R/python + Quarto? In this mode, I think the focus is more on creating a document or communicating stuff typically with data analysis. And actually, "literate" is a very apt adjective for something that's ultimately intended to be a document of some kind.
Or is it more general intended for writing arbitrary software applications and having everything in the source code? In other words, to understand the source, you just read the source and there are no other documents.
The latter mode, I think, is still very much not around as far as I know (academic proof-of-concepts for papers don't count :-) ). Will it ever really exist?
I have tried a few blog posts with literate style code. Some in a notebook style, like https://taeric.github.io/cube-permutations-1.html. in general, that style seems easier nowadays.
There is Stanford GraphBase, and a really good MP3 book that are full programs. Not to mention TeX.
There is Stanford GraphBase, and a really good MP3 book that are full programs. Not to mention TeX.
Also the PCC compiler published A Retargetable C Compiler by Fraser & Hanson, as well as the PBRT renderer as described in the Academy Award winner Physically Based Rendering[1] by Pharr, Jacob, & Humphreys. As for instances where the point is more the book than the program, there’s C Interfaces and Implementations by Hanson, and while the “literate diffs” style used by Nystrom’s Crafting Interpreters[2] comes at the idea from a somewhat different angle, I’d say the effect is largely the same. Ulix[3] is arguably an academic exercise, but at the same time a full Unix-like kernel should count for something, shouldn’t it?
The overwhelming popularity of TeX all too often makes people forget about METAFONT. The type of computer fonts advocated by the latter has unfortunately failed to gain popularity, and its descendant MetaPost has a lot of competition, but I think the language itself is much nicer (while remaining true to its macroexpander roots), and it’s another literate program of about the same size.
[1] Gratis at https://pbr-book.org/
[2] Gratis at http://craftinginterpreters.com/contents.html
[3] https://www.ulixos.org/
The overwhelming popularity of TeX all too often makes people forget about METAFONT. The type of computer fonts advocated by the latter has unfortunately failed to gain popularity, and its descendant MetaPost has a lot of competition, but I think the language itself is much nicer (while remaining true to its macroexpander roots), and it’s another literate program of about the same size.
[1] Gratis at https://pbr-book.org/
[2] Gratis at http://craftinginterpreters.com/contents.html
[3] https://www.ulixos.org/
> the PCC compiler published A Retargetable C Compiler by Fraser & Hanson
You're thinking of LCC. PCC is AT&T's compiler and the work of Johnson who also did yacc.
You're thinking of LCC. PCC is AT&T's compiler and the work of Johnson who also did yacc.
Right. That’s what I get for checking the ’net instead of literally flipping back to the introduction of the book I literally already had open. :/
It might be nice if modern languages defined an official way to "flip" the interpretation of a source file, so that by default the content is treated as markdown and the code goes in fenced blocks (rather than defaulting to code and providing a way to mark comment blocks).
Then we might hope for support in most tooling to appear quite quickly.
I think one reason why literate programming is less attractive nowadays (compared to the days of Pascal) is that programming languages have got better at letting you arrange your code in whatever way you like. So maybe something like the above is enough to get 90% of the advantages of a full literate programming system.
Then we might hope for support in most tooling to appear quite quickly.
I think one reason why literate programming is less attractive nowadays (compared to the days of Pascal) is that programming languages have got better at letting you arrange your code in whatever way you like. So maybe something like the above is enough to get 90% of the advantages of a full literate programming system.
> It might be nice if modern languages defined an official way to "flip" the interpretation of a source file, so that by default the content is treated as markdown and the code goes in fenced blocks (rather than defaulting to code and providing a way to mark comment blocks)
Jef Raskin wrote and thought about this. The sentiment sounds like it would fit best in his piece "Comments are More Important than Code" (2005), but there's another Raskin article "The Woes of IDEs" that contains the most explicit agreement:
> In my method, program segments are embedded in the midst of a word processor document--like raisins in cake--so that the emphasis is on the explanation rather than the code.
<https://queue.acm.org/detail.cfm?id=864034>
> maybe something like the above is enough to get 90% of the advantages of a full literate programming system
If you're doing JS, then you can actually get pretty far with browser-based notebooks that just use vanilla HTML (and CSS), which notably doesn't require any ex ante setup/configuration for runtime support in the vein of e.g. Jupyter notebooks. (The problem being that not everyone writes JS and even most programmers these days who think they're writing JS really aren't and have no idea how to, so this won't work for them, either.)
Jef Raskin wrote and thought about this. The sentiment sounds like it would fit best in his piece "Comments are More Important than Code" (2005), but there's another Raskin article "The Woes of IDEs" that contains the most explicit agreement:
> In my method, program segments are embedded in the midst of a word processor document--like raisins in cake--so that the emphasis is on the explanation rather than the code.
<https://queue.acm.org/detail.cfm?id=864034>
> maybe something like the above is enough to get 90% of the advantages of a full literate programming system
If you're doing JS, then you can actually get pretty far with browser-based notebooks that just use vanilla HTML (and CSS), which notably doesn't require any ex ante setup/configuration for runtime support in the vein of e.g. Jupyter notebooks. (The problem being that not everyone writes JS and even most programmers these days who think they're writing JS really aren't and have no idea how to, so this won't work for them, either.)
There are tools to do exactly that, like a simple bash scripts lit.sh: https://github.com/vijithassar/lit
Pandoc is a nice swiss army knife that can do a similar conversion with some custom lua filters. It could even do wackier stuff like convert word or odt files to code if you were really diving into prose-first coding.
I've also heard it referred to as 'semi-literate programming' because it skips the reorganization functionality and just gives you nice prose to code conversion.
Pandoc is a nice swiss army knife that can do a similar conversion with some custom lua filters. It could even do wackier stuff like convert word or odt files to code if you were really diving into prose-first coding.
I've also heard it referred to as 'semi-literate programming' because it skips the reorganization functionality and just gives you nice prose to code conversion.
As long as these are external tools, IDEs and things like github's default file renderer aren't going to understand what's going on.
Yeah that's why lit.sh and most similar tools have a mode to output a version of the file with the markdown commented out except for code blocks. This lets you pipe it into any tool and get output with proper line numbers, analysis, etc.
VS code is getting pretty smart about code notebooks too, it has native support for a few different formats of markdown + code blocks (hydrogen, jupyter, etc) and applies all the right syntax highlighting, analysis, etc. to each block automatically.
VS code is getting pretty smart about code notebooks too, it has native support for a few different formats of markdown + code blocks (hydrogen, jupyter, etc) and applies all the right syntax highlighting, analysis, etc. to each block automatically.
Every language with block comments supports this, by starting the file with a comment and wrapping code in end-comment/start-comment pairs.
It's not especially useful. For literate programming you really need the ability to move blocks of code around without the ceremony of function signatures.
It's not especially useful. For literate programming you really need the ability to move blocks of code around without the ceremony of function signatures.
Haskell actually supports literate programming [0]. Oleg Kiselyov regularly writes examples of his ideas using it, e.g. [1], [2].
[0] https://wiki.haskell.org/Literate_programming#Haskell_and_li...
[1] https://okmij.org/ftp/Haskell/class-based-dispatch.lhs
[2] https://okmij.org/ftp/Computation/lightweight-guarantees/eli...
[0] https://wiki.haskell.org/Literate_programming#Haskell_and_li...
[1] https://okmij.org/ftp/Haskell/class-based-dispatch.lhs
[2] https://okmij.org/ftp/Computation/lightweight-guarantees/eli...
Haskell does not natively support literate programming -- it only allows the comment/code default to be swapped. The format named "literate Haskell" is basically just syntactic sugar for {- comment blocks -}. Literate programming requires the ability to decouple the ordering of code relative to the commentary.
I wrote more about this at <https://news.ycombinator.com/item?id=35985835> and <https://john-millikin.com/software/anansi#history>.
I wrote more about this at <https://news.ycombinator.com/item?id=35985835> and <https://john-millikin.com/software/anansi#history>.
If I use block comments like that in most contemporary IDEs, I find that by default they show the main text in flat "this is a comment" styling, even though they're in principle capable of using different kinds of highlighting for different parts of the file.
Similarly, if I preview such a file on github I get syntax highlighting for the code but I don't get rendering for the markdown.
So I think the advantage of an 'official' way of saying you're using this arrangement would be communicating intent to more-or-less generic tools.
Similarly, if I preview such a file on github I get syntax highlighting for the code but I don't get rendering for the markdown.
So I think the advantage of an 'official' way of saying you're using this arrangement would be communicating intent to more-or-less generic tools.
That's an IDE configuration problem. Most of them support mixed-mode syntax highlighting nowadays, so you just need to configure your theme to treat block comments as Markdown (or LaTeX, HTML, whatever).
You could probably even do it with Markdown directly, using fenced code blocks as part of your begin/end lines.
I think you'll find that this approach offers very little benefit over either traditional commentary or linear notebook-style inline code blocks.
You could probably even do it with Markdown directly, using fenced code blocks as part of your begin/end lines.
I think you'll find that this approach offers very little benefit over either traditional commentary or linear notebook-style inline code blocks.
I had a black/white/gray vim theme that supports that, the only color is comments are gray. Press F6 and it reverses code and comment colors.
One more tool to accomplish this is lmt [0] which, despite minimal documentation, is quite pleasing to use.
[0] https://github.com/driusan/lmt
[0] https://github.com/driusan/lmt
I've been experimenting a lot with using Jupyter Notebooks (http://jupyter.org/) to get many features of literate programming.
The tooling can be a bit clunky -- though JupyterLab and some plugins can go a long way. Importing notebooks into other notebooks or modules is still kind of weird even with the `ipynb` library.
However, on the otherside of it all, you have a collection of documents that contain documentation, code, and tests, all side by side and storable with proof of past execution.
I'll take this arrangement over someone's unit test and CI suite every day of the week.
The tooling can be a bit clunky -- though JupyterLab and some plugins can go a long way. Importing notebooks into other notebooks or modules is still kind of weird even with the `ipynb` library.
However, on the otherside of it all, you have a collection of documents that contain documentation, code, and tests, all side by side and storable with proof of past execution.
I'll take this arrangement over someone's unit test and CI suite every day of the week.
FWIW, I note emacs polymode's[1] poly-markdown, and pandoc's ast json dumps[2].
For decades, I searched for literate tooling for personal projects, but nothing stuck. Something about the design space seems to make it easy to hit design choices that are uncomfortable fits for one's use case, or personal taste, or preferences for tooling extension and customization. It reminds me of the personal information manager space.
My most recent foray seemed more promising. poly-markdown permits editing polylingual md files, and is permissive towards markdown dialects. So I could use a custom literate markdown, normalize it for rich-but-restrictive pandoc, then get the pandoc ast and emit custom html. With Julia's MLStyle[3] for pattern matching, and a bit of front-end pretty, it was unexpectedly easy to reach something comfortable.
Perhaps for literate tooling to gain greater traction, it helps that software dev progress permits folks to more easily cook their own?
[1] https://github.com/polymode/polymode [2] https://pandoc.org/filters.html [3] https://thautwarm.github.io/MLStyle.jl/latest/
For decades, I searched for literate tooling for personal projects, but nothing stuck. Something about the design space seems to make it easy to hit design choices that are uncomfortable fits for one's use case, or personal taste, or preferences for tooling extension and customization. It reminds me of the personal information manager space.
My most recent foray seemed more promising. poly-markdown permits editing polylingual md files, and is permissive towards markdown dialects. So I could use a custom literate markdown, normalize it for rich-but-restrictive pandoc, then get the pandoc ast and emit custom html. With Julia's MLStyle[3] for pattern matching, and a bit of front-end pretty, it was unexpectedly easy to reach something comfortable.
Perhaps for literate tooling to gain greater traction, it helps that software dev progress permits folks to more easily cook their own?
[1] https://github.com/polymode/polymode [2] https://pandoc.org/filters.html [3] https://thautwarm.github.io/MLStyle.jl/latest/
Do you mind sharing your comparison of polymode versus org-mode for literate programming?
That's ... a lot of articles. Is there any article in particular worth checking out?
Knuth's original article is a good starting point: http://literateprogramming.com/knuthweb.pdf.
Knuth's Programming Pearls entries are good reading as well.
Knuth's Programming Pearls entries are good reading as well.
Yeah I have read the original Literate Programming text and played with tangle/weave a bit before realising jupyter notebooks were more suitable for what I was doing. I'm just wondering if there is something in particular the submitter wanted to highlight, or whether they just wanted to share the concept of "Literate Programming" in general
Clicking through the links I find this:
> In general, literate programs combine source and documentation in a single file. Literate programming tools then parse the file to produce either readable documentation or compilable source.
http://literateprogramming.com/lpfaq.pdf#page11
By this litmus test, the Go standard library qualifies as literate programming. Document production tools are go doc or godoc. The .go files are themselves already compilable source.
> In general, literate programs combine source and documentation in a single file. Literate programming tools then parse the file to produce either readable documentation or compilable source.
http://literateprogramming.com/lpfaq.pdf#page11
By this litmus test, the Go standard library qualifies as literate programming. Document production tools are go doc or godoc. The .go files are themselves already compilable source.
I don't think this is quite the same thing. It's nice that godoc autogenerates documentation on the functions + types each module in the stdlib provides (like here: https://pkg.go.dev/compress/[email protected]) but it doesn't look like it goes any further. In LP you'd get explanation of the implementation details interspersed with little chunks of code, it wouldn't necessarily be in the same order as the final file would be (just the order it makes sense to see it in).
It might be worth checking through the PDF the sister comment to yours linked - http://literateprogramming.com/knuthweb.pdf - you'll see what I mean.
It might be worth checking through the PDF the sister comment to yours linked - http://literateprogramming.com/knuthweb.pdf - you'll see what I mean.
A literate computer algebra system
https://en.wikipedia.org/wiki/Axiom_(computer_algebra_system...
https://en.wikipedia.org/wiki/Axiom_(computer_algebra_system...
I could not help but notice the conspicuous absence of notebooks.
That is an excellent resource. Thanks very much for posting [and thanks to everyone involved in the original compilation].
That is an excellent resource. Thanks very much for posting [and thanks to everyone involved in the original compilation].
Davidnkt89(3)
* It messes with tooling. If you're lucky then your editor will be smart enough to syntax highlight inside code blocks, or can be taught to do so easily. It's unlikely that more advanced IDE-style features will work. Obviously tools could be adapted with custom plugins but literate programming isn't popular enough for these to already exist as far as I'm aware.
* It adds overhead to refactoring. Refactoring code now means editing and rewriting a book, doing a good job requires significantly more effort. If you have designed the program thoroughly ahead of time you can avoid this, but that's not generally the way most programmers work.
* The "includes" problem. Most language require you to include/import/require packages used in a file. These are usually all placed at the start. This means a lot of chapters will start with "here are all the includes we'll need" if you're using a linear format. More complex formats that rearrange the code to generate outputs can do a better job but it's still a little clunky.
Overall my conclusion was that literate programming works best if you write code like Donald Knuth: work solo, have a good idea of the entire design and a fixed scope which you can complete to declare both book and program "done forever". Unfortunately that doesn't cover most real world programming. I don't think it works well in the most common commercial/programming team style settings.