The Tools I Use to Write Books (2018)(thorstenball.com)
thorstenball.com
The Tools I Use to Write Books (2018)
https://thorstenball.com/blog/2018/09/04/the-tools-i-use-to-write-books/
41 comments
Totally agree about not obsessing over tools. That's an easy rabbit hole to go down and distract yourself from producing the actual content of the book. I deliberately chose to use Google Docs because the formatting options are so limited. But I also found it motivating to have a WYSIWYG interface so it felt like I was looking at something resembling the finished product.
In the end gdocs turned out to be a great choice because it's so easy to collaborate. All of the articles talking about markdown seem to assume that you're the only person that's going to work on it, or that everyone else is going to figure out this diagram and your git workflow. It's so easy to email someone a link and let them comment on it, or suggest edits.
For producing the final PDF I found a plugin that embeds the doc directly from Google into InDesign so I can typeset it into something that looks pretty great. Then I just upload the PDF to Leanpub.
The tool that had the biggest impact though was Beeminder, which has a hook into Docs and measures your progress by counting the number of words in the book every day. If you don't make progress you have to pay! It's a great way to make yourself accountable. I kept a 100+ day streak for the final push.
In the end gdocs turned out to be a great choice because it's so easy to collaborate. All of the articles talking about markdown seem to assume that you're the only person that's going to work on it, or that everyone else is going to figure out this diagram and your git workflow. It's so easy to email someone a link and let them comment on it, or suggest edits.
For producing the final PDF I found a plugin that embeds the doc directly from Google into InDesign so I can typeset it into something that looks pretty great. Then I just upload the PDF to Leanpub.
The tool that had the biggest impact though was Beeminder, which has a hook into Docs and measures your progress by counting the number of words in the book every day. If you don't make progress you have to pay! It's a great way to make yourself accountable. I kept a 100+ day streak for the final push.
For my pixel art book with No Starch, they insisted on Word templates and it worked out well for a couple of reasons. The first being because their editing team was so used to using the template for feedback. It was easier for me to adapt to the publisher than the other way around, and the whole process was smoother and quicker.
The second reason was to do with a change in format as the book was nearing completion. It's aimed at kids (8+) and adults, but we decided having an open and lay flat printed edition made the most sense because it makes following the tutorials easier. This resulted in the layout needing to be tweaked for a couple of hundred pages, which the publisher handled because we used their template system.
So while it may be annoying for a writer to have to change their methods, it can save you a hell of a lot of work because things change unexpectedly (and in my case for the better).
The second reason was to do with a change in format as the book was nearing completion. It's aimed at kids (8+) and adults, but we decided having an open and lay flat printed edition made the most sense because it makes following the tutorials easier. This resulted in the layout needing to be tweaked for a couple of hundred pages, which the publisher handled because we used their template system.
So while it may be annoying for a writer to have to change their methods, it can save you a hell of a lot of work because things change unexpectedly (and in my case for the better).
A pixel art book - that sounds very interesting! What's the title?
Obsession over tools, and the more trendy obsession over productivity ("My secret to be 10x more productive!!") ultimately seem tautological in the end: most people hooked up on these techniques will never stop talking about it, and sometimes even write books on the topic.
Take the 2 most well-known authors on these topics: Tim Ferriss and Cal Newport. And look at their list of published works.
I've only very rarely seen actually productive people talk about the way they do things -- probably because they know there isn't a one technique, and have tailored their own things to themselves entirely. Balzac used debt to keep motivation alive, Hemingway wrote while standing up and used the counter of his chimney -- who cares?
I've only very rarely seen actually productive people talk about the way they do things -- probably because they know there isn't a one technique, and have tailored their own things to themselves entirely. Balzac used debt to keep motivation alive, Hemingway wrote while standing up and used the counter of his chimney -- who cares?
I just wanted to add that I really liked your 'Classic CS Problems in Python' book. I found it much better than following university course notes (e.g. CS 61 A/B etc).
Thanks so much. Really appreciate it. Hearing anecdotes like that makes writing worthwhile.
As a counterpoint, I'd wanted to write a book for years, and felt overwhelmed by the production process. asciidoc, docbook, word, it's kind of overwhelming. After I finally put together a good markdown->pandoc->latex->KDP flow, it was a huge relief, and I've printed two books since then (one by me, one by me and some co-authors). It's super-fun to receive an author proof in the mail and it motivates me to write more.
Are your tools publicly available. I've created over a dozen books using my rst-based tools. But I'm planning on using on markdown and pandoc (plus custom filters). (Just write a filter to handle LaTeX index entries).
I bought books from leanpub.com, and they could really improve their PDF generation.
In Markdown, there's no concept of pages. So you can get one row in a table get pushed to the next page and the rest of the page is blank, even there are contents after. Generating PDF with pandoc will usually take care of this through LaTeX, but I don't think leanpub is doing that.
In Markdown, there's no concept of pages. So you can get one row in a table get pushed to the next page and the rest of the page is blank, even there are contents after. Generating PDF with pandoc will usually take care of this through LaTeX, but I don't think leanpub is doing that.
You and Torsten are right. Write the freakin' book. Write the next page or two before knocking off for the night. The tools don't matter as long as you and/or an editor can polish the work and then mung it as needed for publication. But that's for later, once the book is written. Tools are distractions. They give you an excuse not to write the next page. Write the book.
That being said, Markdown + git are a really nice combination.
May your mind be focused and your sitting-upon body parts be made of iron.
That being said, Markdown + git are a really nice combination.
May your mind be focused and your sitting-upon body parts be made of iron.
Unrelated, but I read much of your Classic Computer Science Problems in Python.
And it is really a great book. Especially suitable for people in junior roles right now or people in any kind of role with a knack for solving interesting problems.
And it is really a great book. Especially suitable for people in junior roles right now or people in any kind of role with a knack for solving interesting problems.
Thanks a lot. Appreciate the kind words.
the non-affiliate link: https://www.amazon.com/Writing-Interpreter-Go-Thorsten-Ball/...
Curious, what are your habits for writing a book?
- I do the code first (generally) before the writing.
- I try to do drafts of whole chapters at once, even if it’s an all nighter.
- I reread the chapter over and over again at every review stage.
- As many say to be a good writer you have to be a good reader. I read a lot.
- I get it done. I hear about people spending years on books. I keep it succinct and to the point. Most readers appreciate that. I never miss a deadline. I write in weeks what I hear about others writing in months. To be fair, I’m privileged that my job as a professor at a teaching college allows for this (over the summer for example).
- I try to do drafts of whole chapters at once, even if it’s an all nighter.
- I reread the chapter over and over again at every review stage.
- As many say to be a good writer you have to be a good reader. I read a lot.
- I get it done. I hear about people spending years on books. I keep it succinct and to the point. Most readers appreciate that. I never miss a deadline. I write in weeks what I hear about others writing in months. To be fair, I’m privileged that my job as a professor at a teaching college allows for this (over the summer for example).
Thank you for sharing.
Maybe relevant to those interested: The Snowflake Method
https://www.advancedfictionwriting.com/articles/snowflake-me...
The gist (might work better for fiction, unsure): write a single sentence summary, then, three sentences to describe what major things happen, then fractal it out - keep expanding. This way, from the start, you have a cohesive story with a planned-out arc.
https://www.advancedfictionwriting.com/articles/snowflake-me...
The gist (might work better for fiction, unsure): write a single sentence summary, then, three sentences to describe what major things happen, then fractal it out - keep expanding. This way, from the start, you have a cohesive story with a planned-out arc.
I've used this identical technique to write hourlong talks -- it works well!
The first sentence is the _core_: you write and rewrite this a lot, because the entire talk builds up to it. Next three sentences are the major supporting ideas for your core. They also get rewritten a lot, because again they're really important, they flesh out your core idea. Then you write a lot of detail, one chunk per idea.
I'm glad to know the name for this technique, it works really well!
The first sentence is the _core_: you write and rewrite this a lot, because the entire talk builds up to it. Next three sentences are the major supporting ideas for your core. They also get rewritten a lot, because again they're really important, they flesh out your core idea. Then you write a lot of detail, one chunk per idea.
I'm glad to know the name for this technique, it works really well!
I did KDP for an author, from Word. It wasn't nothing. I valued my labour at $0 to get experience, There are a LOT of fine-grained issues to resolve, bringing a book to printable state.
Binding is it's own special hell: There are huge expectations of barcoding, which plays with your cover artwork, and the positioning of things on the cover is a function of the width of the spine, which is a function of pointsize, margins, wordcount, paper density/weight/thickness. I became very dependent on other peoples templates and calculators to "get this right"
Perhaps the oddest part of this, is the join over paper and ePublishing: they are very unequal in terms of DPI expectations, for both font and artwork. I think the golden rule here is "preserve your pixels as long as you can" but there is also "think about metadata, in images and in fonts and text" because producing print ready PDF and ePub formats, the meta is what the printery/bindery is going to work with. EPSF and other stuff comes to the fore very quickly.
Page numbering is another bugbear: some places demand you DONT number pre and post, some expect it to reset, some expect one to be roman numerals, the rest in decimals. some want it spine, some want it edge, some want it centered. Get too close to the margins, now the printer hates you...
Binding is it's own special hell: There are huge expectations of barcoding, which plays with your cover artwork, and the positioning of things on the cover is a function of the width of the spine, which is a function of pointsize, margins, wordcount, paper density/weight/thickness. I became very dependent on other peoples templates and calculators to "get this right"
Perhaps the oddest part of this, is the join over paper and ePublishing: they are very unequal in terms of DPI expectations, for both font and artwork. I think the golden rule here is "preserve your pixels as long as you can" but there is also "think about metadata, in images and in fonts and text" because producing print ready PDF and ePub formats, the meta is what the printery/bindery is going to work with. EPSF and other stuff comes to the fore very quickly.
Page numbering is another bugbear: some places demand you DONT number pre and post, some expect it to reset, some expect one to be roman numerals, the rest in decimals. some want it spine, some want it edge, some want it centered. Get too close to the margins, now the printer hates you...
The link in the article to the pp tool is no longer active.
Find it here https://github.com/CDSoft/pp
If it’s interesting to someone I have a similar pipeline I used to write and deploy my book The Security Engineer Handbook [1].
I basically wrote everything using markdown files, and a pdf/epub/mobi is automatically generated from the folder using a Github Action. The action will also modify the date of the last update on the webpage, which gets deployed via cloudflare pages (although github pages could have been used). On the other side Stripe handles the payments (No server side code for me) and zapier detects new customers and sends the artifacts by email.
It’s magical :) the next time I want to write a book I’ll focus purely on the content and everything else will be taken care of automagically.
[1]: https://securityhandbook.io/
I basically wrote everything using markdown files, and a pdf/epub/mobi is automatically generated from the folder using a Github Action. The action will also modify the date of the last update on the webpage, which gets deployed via cloudflare pages (although github pages could have been used). On the other side Stripe handles the payments (No server side code for me) and zapier detects new customers and sends the artifacts by email.
It’s magical :) the next time I want to write a book I’ll focus purely on the content and everything else will be taken care of automagically.
[1]: https://securityhandbook.io/
Always cool to see what tools other authors use to write books. I took a different route with my upcoming book[0] on career advice for programmers working towards a promotion to a senior role.
I started out writing it in markdown but ended up moving to a workflow that was mostly Google Docs. My general flow was:
1. High level chapter outline
2. Outline for each individual chapter
3. Write each chapter according to the outline
4. First draft done!
5. Read through each chapter and highlight each sentence according to color code (Green: good, Orange: good but needs to be revised, Red: bad, needs to be rewritten)
6. Work through each chapter and revise/rewrite based on colors.
7. Second draft done!
8. Developmental edits (Get feedback, move chapters or sections around, add missing sections or talking points)
9. Copy edit (This is where I'm at now)
10. Prepare for publication
11. Done!
[0]: https://www.holloway.com/b/junior-to-senior
I started out writing it in markdown but ended up moving to a workflow that was mostly Google Docs. My general flow was:
1. High level chapter outline
2. Outline for each individual chapter
3. Write each chapter according to the outline
4. First draft done!
5. Read through each chapter and highlight each sentence according to color code (Green: good, Orange: good but needs to be revised, Red: bad, needs to be rewritten)
6. Work through each chapter and revise/rewrite based on colors.
7. Second draft done!
8. Developmental edits (Get feedback, move chapters or sections around, add missing sections or talking points)
9. Copy edit (This is where I'm at now)
10. Prepare for publication
11. Done!
[0]: https://www.holloway.com/b/junior-to-senior
I am at the start of writing a book and had some major freeze at the start before, much like the author here, realised that all that matters is that I write the damn thing.
I love how LaTeX lays things out, so I'm using LaTeX. I'd like to have a web version possibly but I can think about that later.
And now, I better get back to it...
I love how LaTeX lays things out, so I'm using LaTeX. I'd like to have a web version possibly but I can think about that later.
And now, I better get back to it...
I wrote a screenplay using a Markdown-based language called "Fountain" designed and implemented by the guy who wrote Tim Burton's screenplays, who I guess is apparently also a programmer. It's a great language and makes the barrier to getting started extremely low.
Fountain is pretty cool.
I used Highland, the writing app developed by the same fellow (John August), to work on a screenplay a couple years ago. It's good, but quirky, especially for non-screenplay applications. The advantage of Fountain, of course, is that it goes anywhere plain text does.
I used Highland, the writing app developed by the same fellow (John August), to work on a screenplay a couple years ago. It's good, but quirky, especially for non-screenplay applications. The advantage of Fountain, of course, is that it goes anywhere plain text does.
Fountain: https://fountain.io/
Highland 2: https://quoteunquoteapps.com/highland-2/
About John August: https://johnaugust.com/about
Highland 2: https://quoteunquoteapps.com/highland-2/
About John August: https://johnaugust.com/about
I'm about 3/4 of the way through writing my second book right now (Zefs Guide to Deep Learning https://zefsguides.com ). I have am using a somewhat similar workflow. I'm writing in Leanpub's Markua flavor of Markdown, which they use to build ebook, PDF, and HTML formats. They will also produce a reformatted PDF for you for book printing, with appropriate margins.
This is the same thing I did for my first book, except that I want a pocket book size format and there isn't one that both Leanpub and Amazon KDP currently support, so instead I will be doing some LaTeX wrangling to produce a PDF formatted for KDP. I haven't yet decided how I will do that, as there doesn't seem to be a Markua -> Markdown convertor anywhere (pandoc only goes in the other direction).
This is the same thing I did for my first book, except that I want a pocket book size format and there isn't one that both Leanpub and Amazon KDP currently support, so instead I will be doing some LaTeX wrangling to produce a PDF formatted for KDP. I haven't yet decided how I will do that, as there doesn't seem to be a Markua -> Markdown convertor anywhere (pandoc only goes in the other direction).
I find the flowchart visualizing the pipeline very intriguing, can someone link me a tool I can use to create similar charts?
For reference: https://thorstenball.com/images/book_tool_pipeline.svg
For reference: https://thorstenball.com/images/book_tool_pipeline.svg
I've used Mermaid for that before: https://mermaid-js.github.io/mermaid/#/
Not sure what the author used.
Not sure what the author used.
I think Thorsten uses Monodraw.
Yup, correct. I used Monodraw [0] for that (and the diagrams in the book). Highly recommend Monodraw, easily one of the loveliest pieces of Mac software I used. And it just got an update!
[0]: https://monodraw.helftone.com
[0]: https://monodraw.helftone.com
Thanks, sadly Monodraw isn't cross-platform. Would've loved to use it.
[deleted]
Being there. Having so many (nice) tools available out there, I spent more time playing around with such tools than I spent doing actual writing. Nowadays, this is what I do:
- I do the writing first using Word (or Google Docs)
- Once my writing is done, I use the cool tools (if needed)
It a little more work at the end (if any), but it's totally negligible because the trunk work is already done (the writing itself).
- I do the writing first using Word (or Google Docs)
- Once my writing is done, I use the cool tools (if needed)
It a little more work at the end (if any), but it's totally negligible because the trunk work is already done (the writing itself).
[deleted]
I wrote a tool that can process a number of MarkDown files with fragments of C code and put all those fragments in the right order to produce a file that can be compiled. It is grammar based and works with manipulating Abstract Syntax Trees, so I guess, it could be adapted for different programming languages. See: https://github.com/FransFaase/IParse#markdownc
Note that it looks like Amazon pulled KindleGen off the web awhile back.
Amazon accepts EPUB directly now-- MOBI's been deprecated for a while now as it doesn't support the full range of features on current devices. That means you don't need KindleGen in order to publish (its validation and previewing roles have been taken over by Kindle Previewer).
That's awesome!
No mention of org-mode?
Apress used Word templates back in 2013 when I did my first book with them. Manning let you use asciidoc or Word templates when I worked with them. I wrote my first book with them in markdown and translated to asciidoc using pandoc. There were so many formatting issues and I found the tools for asciidoc so immature that I ultimately went to using their Word templates on my next two books with them despite preferring purely text formats.
Now I'm working on my next book and I think I'm going to try self publishing. I'm liking leanpub's use of a markdown like format. But again none of this really matters. You can fix formatting later. You have to sit down and write!
PS Thorsten's "Writing an Interpreter in Go" is great: https://amzn.to/3yLTJjE