Rewriting my blog in Rust for fun and profit(jonashietala.se)
jonashietala.se
Rewriting my blog in Rust for fun and profit
https://www.jonashietala.se/blog/2022/08/29/rewriting_my_blog_in_rust_for_fun_and_profit/
115 comments
I plan on writing a static site generator soon myself. And I briefly wondered the same thing and realized that I could very easily answer that question: ruthlessly build whatever I personally need and not what others need. I don't usually follow that line of thinking, but the static site generator space is so full of so many great tools that I don't think there's really anything for me to contribute to it.
Instead, I think the killer feature of my static site generator is going to be exactly what makes it difficult for others to use: its tight coupling with my specific use case. I'll be able to hard-code configuration knobs. Avoid dealing with edge cases. Bake strange non-generic and single-blog-post-specific logic right into the tool.
(The reason why I want to build my own is because I've tried using someone else's. And I just keep getting bitten: someone else's tool gets so big and moves so rapidly, that by the time I get around to updating my site, something about the new version of the tool has broken the way my web site works. And then I usually have to spend hours figuring out how to fix it, which is hugely deflating because I had just worked up the energy to write a new post. My naive belief is that if I have my own tool that evolves at precisely the pace I want it to, I won't run into that specific problem. I'll run into other problems of course, but I'm thinking that I'll prefer those problems to the ones I currently face.)
Instead, I think the killer feature of my static site generator is going to be exactly what makes it difficult for others to use: its tight coupling with my specific use case. I'll be able to hard-code configuration knobs. Avoid dealing with edge cases. Bake strange non-generic and single-blog-post-specific logic right into the tool.
(The reason why I want to build my own is because I've tried using someone else's. And I just keep getting bitten: someone else's tool gets so big and moves so rapidly, that by the time I get around to updating my site, something about the new version of the tool has broken the way my web site works. And then I usually have to spend hours figuring out how to fix it, which is hugely deflating because I had just worked up the energy to write a new post. My naive belief is that if I have my own tool that evolves at precisely the pace I want it to, I won't run into that specific problem. I'll run into other problems of course, but I'm thinking that I'll prefer those problems to the ones I currently face.)
I went this route. At first I tried writing a bunch of Python scripts reading a YAML manifest. However, after ~2 weeks I already got warnings and errors about things which got deprecated/removed/changed by a new Python version and the YAML library I was using. So I dumped it and rewrote the thing in XML and XSLT. It's been working for a couple years with no breakages. Essentially 0 maintenance after the initial work. This design was about generating a tree of directories with Markdown files turned into HTML, PDF files turned into pages linking to them, and some directory renames, GUIDs and dates (for Atom feeds, which I generate for each directory) specified in a manifest file.
I've since taken up photography, so now I need to automatically read metadata from jpegs, add some tags and organise it on the site in albums. It's a bit outside the reach of XSLT, so first I've tried using Python with Jinja2 and PIL. Same problem as previously. So I've rewritten it in Go with the help of encoding/json (for the manifest), html/template (for the template), image/jpeg, golang.org/x/image/draw (for generating thumbnails of different sizes for different screen sizes) packages and exiftool (reading and removing EXIF tags). Pandoc has been present throughout all the rewrites, because I like its flavour of Markdown and options to shift heading levels by a specified value, so that they fit in with the surrounding template. I'm putting some finishing touches and am going to upload it in the coming days.
I know, you stopped reading at "XSLT". :D
I've since taken up photography, so now I need to automatically read metadata from jpegs, add some tags and organise it on the site in albums. It's a bit outside the reach of XSLT, so first I've tried using Python with Jinja2 and PIL. Same problem as previously. So I've rewritten it in Go with the help of encoding/json (for the manifest), html/template (for the template), image/jpeg, golang.org/x/image/draw (for generating thumbnails of different sizes for different screen sizes) packages and exiftool (reading and removing EXIF tags). Pandoc has been present throughout all the rewrites, because I like its flavour of Markdown and options to shift heading levels by a specified value, so that they fit in with the surrounding template. I'm putting some finishing touches and am going to upload it in the coming days.
I know, you stopped reading at "XSLT". :D
I've done exactly what you describe and I highly recommend it.
My personal website is served by a couple hundred lines of Rust. There is no database or ORM -- I just defined structs like `BlogPost` or whatever and stuck a `#[derive(Serialize, Deserialize)]` on it and use the `serde` library to JSONify it, store it flat in a hard-coded folder.
I even wrote my own dead-simple Markdown-ish flavor that is tailored exactly to my writing style. I use a lot of em-dashes, for example, so it translates `--` into `—` (rather, &mdash). Or, since a lot of my posts link to other posts, I can do `[post title_of_other_post]` and it'll generate the appropriate link.
Whenever I want a new feature while writing a particular blog post, e.g. pretty inline SVG, I just go add it and then keep writing.
My personal website is served by a couple hundred lines of Rust. There is no database or ORM -- I just defined structs like `BlogPost` or whatever and stuck a `#[derive(Serialize, Deserialize)]` on it and use the `serde` library to JSONify it, store it flat in a hard-coded folder.
I even wrote my own dead-simple Markdown-ish flavor that is tailored exactly to my writing style. I use a lot of em-dashes, for example, so it translates `--` into `—` (rather, &mdash). Or, since a lot of my posts link to other posts, I can do `[post title_of_other_post]` and it'll generate the appropriate link.
Whenever I want a new feature while writing a particular blog post, e.g. pretty inline SVG, I just go add it and then keep writing.
We think alike. I started out with Hugo because it's very popular and relatively easy to use. It has a bunch of features that I had to read about but didn't have a use for -- overhead.
One of the most annoying aspects of working with hugo was the friction from idea to publishing it.
I decided to go a completely different route and built https://prose.sh
With prose all I have to do is to create a markdown file and then `scp` it to prose.sh. The platform takes care of the rest and I can still enjoy my own editor and terminal tooling. Even better, authentication happens via SSH so the friction is extremely low.
It's a nice balance between writing a blog on a platform that does everything for you and being able to use my own tooling to write content.
Another big issue with Hugo is when I want to make edits to the content. My brain is weird and I only scrutinize my content after it has been published. So I end up making a lot of edits after an article has been live for a few minutes. It's a terrible habit and Hugo required deployment for every little change.
I've thought about converting prose to be SSG -- we could generate the html when a user uploads the file -- but honestly the site is fast as it is so I don't see the value in that added complexity.
One of the most annoying aspects of working with hugo was the friction from idea to publishing it.
I decided to go a completely different route and built https://prose.sh
With prose all I have to do is to create a markdown file and then `scp` it to prose.sh. The platform takes care of the rest and I can still enjoy my own editor and terminal tooling. Even better, authentication happens via SSH so the friction is extremely low.
It's a nice balance between writing a blog on a platform that does everything for you and being able to use my own tooling to write content.
Another big issue with Hugo is when I want to make edits to the content. My brain is weird and I only scrutinize my content after it has been published. So I end up making a lot of edits after an article has been live for a few minutes. It's a terrible habit and Hugo required deployment for every little change.
I've thought about converting prose to be SSG -- we could generate the html when a user uploads the file -- but honestly the site is fast as it is so I don't see the value in that added complexity.
Nice! I have a Hugo site I’ve tweaked over the years, and my workflow is similar to yours. I create/edit an MD file, then run a shell command which pushes changes to Github, which triggers a redeploy.
How do you handle images? That’s my main annoyance / biggest hurdle when publishing. I have to take each image, resize it for web, put it in static folder, and reference that pathname in the MD file. Not a big deal, but would love an easier way. I suppose I could write a simple shell script that does all of the resizing and prints out a path back for me to use.
How do you handle images? That’s my main annoyance / biggest hurdle when publishing. I have to take each image, resize it for web, put it in static folder, and reference that pathname in the MD file. Not a big deal, but would love an easier way. I suppose I could write a simple shell script that does all of the resizing and prints out a path back for me to use.
We are working on https://imgs.sh which will try to make optimizing for web automatic as well as making it easy to reference images from your blog.
btw the example image (https://erock.imgs.sh/t/iceland) 404's :(
Can you use `hugo server` (much like `jekyll/zola serve`) and a local Git repo to polish up the content locally, explore various permutations, before pushing to a cloud-based host?
No but that's a really interesting idea. I'll have to think about how we could implement that. Thanks!
Tangentially related: the delay field in your hn profile will hide your comment for n minutes for you to make edits.
I always set mine to 2.
I always set mine to 2.
> And I just keep getting bitten: someone else's tool gets so big and moves so rapidly, that by the time I get around to updating my site, []something about the new version of the tool has broken the way my web site works. And then I usually have to spend hours figuring out how to fix it[], which is hugely deflating because I had just worked up the energy to write a new post. My naive belief is that if I have my own tool that evolves at precisely the pace I want it to, I won't run into that specific problem. I'll run into other problems of course, but I'm thinking that I'll prefer those problems to the ones I currently face.
This is true, as a general principle, not just of static site generators, but of all software, and is one of the major Reasons Why We Can't Have Nice Things, and why automatic updates are evil and poisonous, and manual updates dangerous and often-avoided. See eg https://jacquesmattheij.com/why-johnny-wont-upgrade/.
This is true, as a general principle, not just of static site generators, but of all software, and is one of the major Reasons Why We Can't Have Nice Things, and why automatic updates are evil and poisonous, and manual updates dangerous and often-avoided. See eg https://jacquesmattheij.com/why-johnny-wont-upgrade/.
> but of all software
I don't think so. There's lots of software that I've used and upgraded for decades and rarely if never run into any issues.
I don't think it's a general principle. I don't like generalizing to "all" unless I have a really good reason to do so. Instead, I stuck to my specific experience with static site generators.
I don't think so. There's lots of software that I've used and upgraded for decades and rarely if never run into any issues.
I don't think it's a general principle. I don't like generalizing to "all" unless I have a really good reason to do so. Instead, I stuck to my specific experience with static site generators.
You should check out what fasterthanlime does for their blog. Theirs uses sqlite for storing all state and the write plugins into the Rust Liquid template implementation to directly query sqlite so they could do whatever they wanted without recompiling. It was pretty neat.
I have been looking to break out parts of static-site-generators to make it easier to bring one up. So far I only have two primitive building blocks
- https://docs.rs/engarde/latest/engarde/ - https://docs.rs/file-serve/latest/file_serve/
I have been looking to break out parts of static-site-generators to make it easier to bring one up. So far I only have two primitive building blocks
- https://docs.rs/engarde/latest/engarde/ - https://docs.rs/file-serve/latest/file_serve/
My needs are far simpler than that. I don't need state. I don't need dynamic content. I don't need ads. I don't need tracking. I don't need search. I just need a way to publish content and maybe an RSS feed. It doesn't even need to serve files. Everything is static. (I'm not saying fasterthanlime has all those things. Just saying what I don't need.)
I can't even imagine I need a template system beyond something I can write in a few dozen lines with some regexes.
I did the SQLite thing more than a decade ago. I ain't going back to it unless I have a really really good reason for it.
I really just need a purely static web site where I can write Markdown posts. There's some other stuff I want in terms of producing the static content, but in terms of web site functionality itself, it should just be HTML, CSS and some images.
I also don't mind re-compiling. For my needs, "compiling" my web site is likely to take less than 1 second.
Definitely don't need plugins. :)
I can't even imagine I need a template system beyond something I can write in a few dozen lines with some regexes.
I did the SQLite thing more than a decade ago. I ain't going back to it unless I have a really really good reason for it.
I really just need a purely static web site where I can write Markdown posts. There's some other stuff I want in terms of producing the static content, but in terms of web site functionality itself, it should just be HTML, CSS and some images.
I also don't mind re-compiling. For my needs, "compiling" my web site is likely to take less than 1 second.
Definitely don't need plugins. :)
I plan on trying https://luapress.org/ one of the next weeks. Looks good at a first glance -- give it a quick check if you get a free hour, would be interested what you think of it.
I don't have a free hour unfortunately. And I know I'm not going to use that because I don't want to setup a Lua environment. Indeed, I have enough Lua experience to know that they break the language in big ways for what seems like every release. That's a liability. What happens when luapress stops getting regular maintenance and the version of Lua it uses is no longer packaged in my Linux distro? Oops. Ticking time bomb.
I meant it. I'm done with static site generators built by others. They have caused me too much pain.
EDIT: That "oops" is already reality: https://github.com/Fizzadar/Luapress#luapress-v4
Static site generators are just too easy to build and too easy to let die. And the ones that don't die grow big and bloated. That's my experience. The only remedy I see is to build my own and simplify ruthlessly through tight coupling.
"You either live long enough to see yourself become the villain, or you die a hero."
I meant it. I'm done with static site generators built by others. They have caused me too much pain.
EDIT: That "oops" is already reality: https://github.com/Fizzadar/Luapress#luapress-v4
Static site generators are just too easy to build and too easy to let die. And the ones that don't die grow big and bloated. That's my experience. The only remedy I see is to build my own and simplify ruthlessly through tight coupling.
"You either live long enough to see yourself become the villain, or you die a hero."
> Static site generators are just too easy to build and too easy to let die. And the ones that don't die grow big and bloated.
Sadly you're right. I've been monitoring in the trenches for a while and what you said is covering it perfectly.
RE: maintenance, I was aware but was wondering whether Luapress is mature and complete enough to not need maintenance. I'll give it a go.
And I truly get your point about point releases of languages breaking stuff. I am severely burned out on that front myself that's why my next game plan is to buy and setup a Linux workstation and just isolate various software pieces that I [might] need in Docker containers and periodically backup those for extra good measure.
I am done with all that constantly moving and breaking crap as much as you are. Just looking to find software for all my needs and nail its versions and installs and just allow myself to not constantly keep up with the youngsters' play toys.
Appreciate all your software contributions and a good part of them are my favorite in their area.
Sadly you're right. I've been monitoring in the trenches for a while and what you said is covering it perfectly.
RE: maintenance, I was aware but was wondering whether Luapress is mature and complete enough to not need maintenance. I'll give it a go.
And I truly get your point about point releases of languages breaking stuff. I am severely burned out on that front myself that's why my next game plan is to buy and setup a Linux workstation and just isolate various software pieces that I [might] need in Docker containers and periodically backup those for extra good measure.
I am done with all that constantly moving and breaking crap as much as you are. Just looking to find software for all my needs and nail its versions and installs and just allow myself to not constantly keep up with the youngsters' play toys.
Appreciate all your software contributions and a good part of them are my favorite in their area.
I also am quite neurotic about version control, updates etc and if you're prepared to climb up a fairly steep learning curve, I'd recommend Nix/NixOS. It's a much more robust and comprehensive solution than Docker. Docker is not as good a technology as it should be in this use case, and it does stuff like cacheing CMD instructions in the Dockerfile based on their value as a string (rather than the result of the command). This can and does lead to Docker images changing between builds even when the Dockerfile itself has not changed.
Yep, thanks for reminding me about Docker -- I just haven't figured out something much better yet. Maybe mini-VMs like Firecracker or just straight up qemu will be my final stops, who knows (as I plan to buy a workstation with hefty RAM amounts; will likely start at 256GB). I still have a lot of time to think about how to approach the whole thing.
--
For Nix, I think you know what I'll tell you: you don't recommend a war veteran with PTSD "just one last war". I get what they're trying to do but their discourse on various platforms has left much to be desired and they seem opposed to offer more ergonomic CLIs and/or UIs. I hear that's changing as well so I'll be checking them out a few times a year. As it is right now, I wouldn't even mind the steep learning curve -- I am not completely burned out and I still punch quite hard in my work -- but mysterious error messages and maintainers ending discussions with "well then it's not for you" et. al. are just not appealing and feel like I'm taking a risk that will not pay off. Nix still feels like somebody's experimentation project.
I truly hope they take off as their idea deserves but that must come with simplicity on the level of your average Joe and Jane pasting 2-3 commands in a terminal (sort of how you install Rust; it's literally two pasted commands). Before something similar happens, Nix is doomed to remain a niche curiosity.
--
For Nix, I think you know what I'll tell you: you don't recommend a war veteran with PTSD "just one last war". I get what they're trying to do but their discourse on various platforms has left much to be desired and they seem opposed to offer more ergonomic CLIs and/or UIs. I hear that's changing as well so I'll be checking them out a few times a year. As it is right now, I wouldn't even mind the steep learning curve -- I am not completely burned out and I still punch quite hard in my work -- but mysterious error messages and maintainers ending discussions with "well then it's not for you" et. al. are just not appealing and feel like I'm taking a risk that will not pay off. Nix still feels like somebody's experimentation project.
I truly hope they take off as their idea deserves but that must come with simplicity on the level of your average Joe and Jane pasting 2-3 commands in a terminal (sort of how you install Rust; it's literally two pasted commands). Before something similar happens, Nix is doomed to remain a niche curiosity.
I just wrote like a Perl script that generates my site. It's just printing stuff and converting markdown to HTML, right? Never breaks, never changes, does exactly what I need it to.
Presumably there are reasons people want to use more specialised tools, but I don't have any of those.
Presumably there are reasons people want to use more specialised tools, but I don't have any of those.
I've just started to take that approach (exactly what I need, no more, no less) with personal process automation, and it's really working for me.
I made a few attempts in the past at building more general-purpose tools that would cover my use cases, but I've never followed through on them. Too much of the work was in the bits that don't actually benifit me right now.
My new tool is simply called "Other Jeff". I write onto it what I need right now. Do I have a series of pull requests that need babysitting to get them over the line? Other Jeff learns to poll their status and notify me of important events along the way. If I want to store some state about what I'm working on right now, I need a database of some kind, right? Nope. Not any more. I hard-code it into `db.rs`, because it turns out there's almost nothing I need to persist that's too burdensome to update by hand.
So what I have in the end is a single Rust program that grows and shrinks depending on my needs at the time, accumulates reusable helpers if and when they actually make sense for _me_, and slowly learns to automate more of my daily distractions.
I made a few attempts in the past at building more general-purpose tools that would cover my use cases, but I've never followed through on them. Too much of the work was in the bits that don't actually benifit me right now.
My new tool is simply called "Other Jeff". I write onto it what I need right now. Do I have a series of pull requests that need babysitting to get them over the line? Other Jeff learns to poll their status and notify me of important events along the way. If I want to store some state about what I'm working on right now, I need a database of some kind, right? Nope. Not any more. I hard-code it into `db.rs`, because it turns out there's almost nothing I need to persist that's too burdensome to update by hand.
So what I have in the end is a single Rust program that grows and shrinks depending on my needs at the time, accumulates reusable helpers if and when they actually make sense for _me_, and slowly learns to automate more of my daily distractions.
This. All the big static generators in the market are kinda of generic enough that they demand upfront investment from you. Whether its Jekyll, Hugo or any big shot you have to fit a lot of things according to them. My personal website is built by Jekyll, but I finding a lot harder to make changes because I am not comfortable with a Ruby tooling( Once, I had to debug an obscure Ruby related bug in M1 Mac). I am comfortable with Python though( I have considerable proficiency and good documentation) which then I will use it for my own static generator.
I guess writing your own non-generic own-static generator should be relatively easy and worth the effort as you wouldn't have to read some obscure documentation (like Jekyll).
I guess writing your own non-generic own-static generator should be relatively easy and worth the effort as you wouldn't have to read some obscure documentation (like Jekyll).
Sounds cool, I hope one day I stumble upon your article that utilizes those very custom hard-coded knobs. Yet I wouldn't make a tool like that since tools should be KISS.
KISS is exactly my point. :)
I went with Hugo. It is written in Go.
If I were to make my own generator from scratch, I'd go with python. It's the language I work with the most. A lot of my blog content will be generated from python (figures etc) or will show python code. So having a site generator in the same language can help me add features by hacking on the generator code in a way that a single binary cannot.
If I were to make my own generator from scratch, I'd go with python. It's the language I work with the most. A lot of my blog content will be generated from python (figures etc) or will show python code. So having a site generator in the same language can help me add features by hacking on the generator code in a way that a single binary cannot.
I migrated from Pelican (Python) to Zola (Rust) because of the ease of replication of the environment. After a few years it was a pain to recreate my pelican environment and even the tools I'd used to try make that situation easier (pipenv) were undergoing their own bitrot.
I went with Hugo too. I used Gatsby in the past, but it was slower and it felt a bit odd using GraphQL in the way it does. Maybe that'd make more sense if I was using an API to deliver content, but for simple markdown pages, it seemed unnecessary.
On the other hand, having single binary is good because you don't have to install the runtime to tinker with the tool. Yet I do agree that tinkering options are very limited in comparison.
Finding a good static site generator wasn't really the author's point. It was more for the fun of writing something in Rust.
But I appreciate the link to Zola. I haven't looked in this space for awhile and it looks good.
But I appreciate the link to Zola. I haven't looked in this space for awhile and it looks good.
You’re welcome! Just wanted to share what did I come to in the end following the same route.
> I've found myself struggling to decide on what do I want to see in the final app and what I do not; what features might be needed by other users and what features might be an overkill.
It's tricky. You can't accept all the features otherwise you just have a mess. At the same time you need to also add/supports features that a lot of people want but _you_ personally don't need if you want the tool to be used by more than 1 person. For example with SSG I don't use themes, I just write my own templates for every site. Despite that I still added it to Zola because most people want themes. Some people will not be happy when you say no to some features they want but hey, it's open-source they can always fork it.
It's tricky. You can't accept all the features otherwise you just have a mess. At the same time you need to also add/supports features that a lot of people want but _you_ personally don't need if you want the tool to be used by more than 1 person. For example with SSG I don't use themes, I just write my own templates for every site. Despite that I still added it to Zola because most people want themes. Some people will not be happy when you say no to some features they want but hey, it's open-source they can always fork it.
Storing the internal state in sqlite and allowing Lua plugins to operate on it might be a way of extending things.
I ended up making mine with Pandoc + Make. At first i used Nextjs and Remark/Unifiedjs but then I had a realization that all that is not even needed.
Have you come across Soupault yet?
I'm considering using Pandoc with Soupault to my website markup agnostic by being dependent on Pandoc. Soupault can act as a HTML processor although I'm not sure if that's enough to not need a template langauge. Or maybe I'm mistaken about Soupault.
https://soupault.app/
I'm considering using Pandoc with Soupault to my website markup agnostic by being dependent on Pandoc. Soupault can act as a HTML processor although I'm not sure if that's enough to not need a template langauge. Or maybe I'm mistaken about Soupault.
https://soupault.app/
Looks pretty cool, thank you!
My makefile is pretty simple (3 lines) but I'll see if maybe this is more suitable. :)
My static site generate is enough python to invoke mustache on every file in a directory. Maybe 70 lines
That's it.
Works fine.
Not everything is improved by creeping featuritis.
That's it.
Works fine.
Not everything is improved by creeping featuritis.
I moved from Jekyll to Zola and it's been great until I started tinkering with templates and stuff recently. Single binary and simple themes were main reasons to choose this. Wanted a feature that wasn't present in the version I started with (about 2 years ago). So, I got the newest version and a few things broke subtly (ex: syntax highlighting). Which led me to tinkering (I don't know much about HTML/CSS/etc).
It was actually good to try a few things almost blindly and then slowly narrowing with a bit more thought process. Been a long time since I had such fun investigating unknown codebase. This even helped me move my post processing scripts to templates. And then I went a step further by reading about meta tags (things like image, description, etc) and adding them via an extra variable in the post frontmatter.
It was actually good to try a few things almost blindly and then slowly narrowing with a bit more thought process. Been a long time since I had such fun investigating unknown codebase. This even helped me move my post processing scripts to templates. And then I went a step further by reading about meta tags (things like image, description, etc) and adding them via an extra variable in the post frontmatter.
That's exactly why I never used Jekyll and don't really understand people that use it nowadays. It is AWFUL. Not only it uses greatly outdated Ruby, you also HAVE to install outdated Ruby on your machine to play with themes, serve, compile and are forced to use RVM. I know GitHub Pages can compile the page for you, but when you want to customize a theme you have to go through all that hell. It's just a terrible experience. And installing plugins is another hell.
I use Jekyll for a few sites and it works perfectly. Design is bootstrap and so nothing to do with Jekyll and it always compiles the site, takes a few seconds. Works fine on my mac and linux.
Would I have picked Jekyll today? no. Is it worth the cost of switching? no.
Would I have picked Jekyll today? no. Is it worth the cost of switching? no.
I'm literally in the process of porting my blog to Zola right now (from Octopress).
One issue I hit is that taxonomies aren't available at the section-level (nor are they section-specific, in case you want to treat sections as sub-sites), this makes it seemingly impossible to have a section taxonomy view.
I might try to patch it, but it's tricky as it's like merging the taxonomy code with the page and section rendering.
One issue I hit is that taxonomies aren't available at the section-level (nor are they section-specific, in case you want to treat sections as sub-sites), this makes it seemingly impossible to have a section taxonomy view.
I might try to patch it, but it's tricky as it's like merging the taxonomy code with the page and section rendering.
I did this as my project to learn Rust, to build with all the features needed for a good static site generator was a great way to learn!
I still haven't released it as there are some bugs (localization!) that i need to fix but if you want to have a play the staging site is here[1].
[1] https://stage.uwe.app/
I still haven't released it as there are some bugs (localization!) that i need to fix but if you want to have a play the staging site is here[1].
[1] https://stage.uwe.app/
I also rewrote my blog's static site generator to a bespoke one (in Dart in my case), largely for performance reasons. Prior, I was using Jekyll and Pygments. Jekyll and Pygments are really slow. My new site generator is literally 100x faster, and I did, like, zero real optimization work. In fact, it can rebuild my entire blog and every post from scratch faster than Jekyll's watch mode (with caching set up properly) could rebuild a single page.
I wrote Zola because at that time I was using Hugo + Pygments and while Hugo was fast, Pygments was the opposite. I'm pretty sure Zola can build a 1k page website with syntax highlighting faster than Pygments can highlight 20 pages.
Is it open sourced? Would love to have a read of some dart not written by a Googler or Xoogler. Unless...
Edit: and... just read his bio
Edit: and... just read his bio
My recommendation for people using anything like Jekyll: just bite the bullet and write a custom build script. A 200 line script that takes can handle Jinja templates or markdown or something will mean you actually understand all that is happening and aren't trying to fight the system or just figure out how to do anything.
And of course don't write a plugin system or something you would share with others! Just write the thing that you need, and change it as you need to!
Static site generators are great for people less comfortable doing this, but having to deal with all the _stuff_ is a major waste of time for what it's offering for people who don't need it.
And of course don't write a plugin system or something you would share with others! Just write the thing that you need, and change it as you need to!
Static site generators are great for people less comfortable doing this, but having to deal with all the _stuff_ is a major waste of time for what it's offering for people who don't need it.
100% agreed. The things a static site generator does are typically very simple: load some Markdown into memory, run it through some HTML templates, and maybe scale some pictures with ImageMagick if you're feeling especially fancy. Writing a script to do those things is way easier than trying to learn and wrangle a Static Site Generator(tm).
The one time static stie generators are helpful is if you really, really like the out-of-the-box look and behavior of some existing theme.
The one time static stie generators are helpful is if you really, really like the out-of-the-box look and behavior of some existing theme.
To this point: Markdown is a nice format for ingest because you can just paste in HTML. I am a bit more courageous and have some custom directives for my Markdown stuff, but it's all really 20 lines of code.
I think it's tempting to add a bunch of "time saving" features for writing, but lots of time just going with the HTML is fine. Maybe if you're writing out a post every couple of days then you can focus on time saves. Until then MD will get you most of the way, and otherwise you can always hack in some special case for your unique post
I think it's tempting to add a bunch of "time saving" features for writing, but lots of time just going with the HTML is fine. Maybe if you're writing out a post every couple of days then you can focus on time saves. Until then MD will get you most of the way, and otherwise you can always hack in some special case for your unique post
Markdown is lousy in significant part because its interactions with HTML are ridiculously complex and moderately difficult to learn, and almost no one tries to actively learn Markdown in a principled fashion, and even fewer try teaching it in a principled fashion, so it ends up unreliable and fairly unpredictable to almost everyone. Things like: if you start a “paragraph” (of which “what comes after a blank line” is a fair approximation) with a block HTML element (e.g. <p>, <div>), inline Markdown syntax won’t work until you start a new “paragraph”. This is related to what I find to be the real killer for just pasting HTML: if you use blank lines followed by lines that are either indented (by at least four spaces or a tab) or don’t start with a block HTML element, then Markdown will mangle your input fiercely. And that’s extremely common (it’s basically recommended formatting for various content).
And all that is without taking into account variations in Markdown flavours, and what may happen with unprincipled and inconsistent extension of Markdown. All up, I say if you want sanity that will last, go HTML and avoid Markdown strenuously for anything beyond simple prose.
<aside>
Markdown syntax is *not* interpreted here, and this is a text node inside an aside.
</aside>
<aside>
<p>Markdown syntax is *not* interpreted here, and this is a text node inside a p inside an aside.</p>
</aside>
<aside>
Markdown syntax *is* interpreted here, and `aside > p > em` should match.
</aside>
<aside>
<p>Markdown syntax is *not* interpreted here, and this is a text node inside a p inside an aside.</p>
</aside>
<aside>
<p>This line is a code block (pre) inside an aside.</p>
</aside>
And all that is without taking into account variations in Markdown flavours, and what may happen with unprincipled and inconsistent extension of Markdown. All up, I say if you want sanity that will last, go HTML and avoid Markdown strenuously for anything beyond simple prose.
I think here you are also somewhat proving the point because you can "go HTML", since the problem is more about markdown not kicking in. I'm mostly kidding but since we are talking about "blogs" (mostly pictures and text) it's OK for there to be a bit of jank. Remember, this is a script for you! You do not have to justify the jank to anyone!
Seriously though, I think that what you're describing is a real failing of Markdown. I wanted at one point to use Restructured Text but really you want Sphinx and Sphinx is not meant to be used as a library unfortunately.
Seriously though, I think that what you're describing is a real failing of Markdown. I wanted at one point to use Restructured Text but really you want Sphinx and Sphinx is not meant to be used as a library unfortunately.
The problem is that part of the time it won’t kick in when you want it to, and part of the time it will kick in when you don’t want it to. I’ve seen plenty of examples of both.
I have a well-established track record of disliking Markdown because of how grossly technically unsound it is. I’m steadily designing a more principled replacement for at least my own use, drawing most significantly from reStructuredText, AsciiDoc and Markdown.
I have a well-established track record of disliking Markdown because of how grossly technically unsound it is. I’m steadily designing a more principled replacement for at least my own use, drawing most significantly from reStructuredText, AsciiDoc and Markdown.
I have the opposite advice: optimize for writing, minimize janitorial stuff like tweaking your own tooling. It just distracts you from writing and offers only masturbatory value.
The best path depends on your personality, but building my own blog tooling killed my writing. I'd start writing something only to realize I needed to tweak something. Then I wouldn't even write the post or I wouldn't have the energy to fix the tooling and I'd do neither. And then I stopped writing.
I didn't start writing again until I used Wordpress which let me focus on writing, though after proving myself for a year, I switched to Jekyll + Github Pages so that everything was on Github.
The best path depends on your personality, but building my own blog tooling killed my writing. I'd start writing something only to realize I needed to tweak something. Then I wouldn't even write the post or I wouldn't have the energy to fix the tooling and I'd do neither. And then I stopped writing.
I didn't start writing again until I used Wordpress which let me focus on writing, though after proving myself for a year, I switched to Jekyll + Github Pages so that everything was on Github.
This is mindblowing synchronicity. I thought this article was one I had read this weekend for a second but checked the post date and saw it wasn't available yesterday.
I'm working on trying to write a static site generator for my own personal site/blog in C++, been tinkering for a couple months. It started as a passion project in tribute to my dog that passed. GatsbyJS wasn't working (again) which is what my current site is built with, so I just said screw it, I'll write my own. Chose C to begin with and quickly gave up. Decided to switch to C++ because its what I'm supposed to be learning for work.
I named it bluesky, after my dog that passed, Sky Blue. https://github.com/mas-4/bluesky
Templating is a lot harder than I had initially thought. I wanted the templating system to be as bare bones as possible, similar to another static site generator I rather like, sergey[0]. Most of these articles I found ([1], [2]) about writing your own SSG don't go into templating much, they just use an off the shelf library. Inja[3] is available for C++ but, like I said, I want something really bare bones, like if you were designing html now, you'd obviously include html-includes and html-templates. I finally got it working for includes and templates, now I have to add the markdown support, and then I plan on migrating my personal site to using it.
[0]: https://sergey.cool/ [1]: https://www.smashingmagazine.com/2020/09/stack-custom-made-s... [2]: https://blog.hamaluik.ca/posts/build-your-own-static-site-ge... [3]: https://github.com/pantor/inja
edit: a link
I'm working on trying to write a static site generator for my own personal site/blog in C++, been tinkering for a couple months. It started as a passion project in tribute to my dog that passed. GatsbyJS wasn't working (again) which is what my current site is built with, so I just said screw it, I'll write my own. Chose C to begin with and quickly gave up. Decided to switch to C++ because its what I'm supposed to be learning for work.
I named it bluesky, after my dog that passed, Sky Blue. https://github.com/mas-4/bluesky
Templating is a lot harder than I had initially thought. I wanted the templating system to be as bare bones as possible, similar to another static site generator I rather like, sergey[0]. Most of these articles I found ([1], [2]) about writing your own SSG don't go into templating much, they just use an off the shelf library. Inja[3] is available for C++ but, like I said, I want something really bare bones, like if you were designing html now, you'd obviously include html-includes and html-templates. I finally got it working for includes and templates, now I have to add the markdown support, and then I plan on migrating my personal site to using it.
[0]: https://sergey.cool/ [1]: https://www.smashingmagazine.com/2020/09/stack-custom-made-s... [2]: https://blog.hamaluik.ca/posts/build-your-own-static-site-ge... [3]: https://github.com/pantor/inja
edit: a link
I've found you don't actually need a templating system, you just need format-strings. Write your rendering logic with normal code that just spits out a string
Templating languages probably help with performance a bit when you're rendering at request-time, but for static stuff I don't really see much point
I've got a hand-rolled static site generator in JavaScript that just uses template strings. I only have around 30 pages, to be fair, but when I make a change it re-generates the whole site before I can even alt-tab and reload the page. I assume a C++ version could be much faster
Templating languages probably help with performance a bit when you're rendering at request-time, but for static stuff I don't really see much point
I've got a hand-rolled static site generator in JavaScript that just uses template strings. I only have around 30 pages, to be fair, but when I make a change it re-generates the whole site before I can even alt-tab and reload the page. I assume a C++ version could be much faster
One great benefit of a good template language is automatic escaping (e.g. within HTML attributes or tags), which you won't get from format strings.
Is that important in a static site generator?
In a server-rendered site you have to worry about injection attacks, but that's not really relevant when you're statically generating a site from content that's fully under your control
In a server-rendered site you have to worry about injection attacks, but that's not really relevant when you're statically generating a site from content that's fully under your control
But then you'll need to watch for `<script>` in examples of your articles, so you dont need to write it as `<script>`. And how you are going to get the second in your blog? How do you like to write `&lt;script&gt;`?
It is very convenient to have an automatic escaping of everything.
It is very convenient to have an automatic escaping of everything.
Depends on the site I guess. My actual blog content is written in Markdown, which already handles escaping stuff between backticks, and that's the only place I'm likely to run into that sort of thing
Sorry for your loss.
I have some 381 posts (including landing pages, etc). It takes maybe 45 minutes to build (there is no cache process). I have a faster version that skips audio processing, but it still takes minutes.
The problem is that it does several pandoc parses for each page:
1. Plain text - Input for audio parsing (which takes a significant amount of time, I have a version of the build that skips this).
2. Simple HTML - Input for RSS feed.
3. Full HTML - Static web page.
One change I would make to pandoc is to have the option for several outputs (as was discussed years ago [1]). I think generating multiple outputs is highly common usage.
One feature I will get around to writing (eventually) is automatic image compression. I had some success using jpegoptim and optipng on another project. This takes a while, so I want to build a command line tool that caches the output of a command if both the input file(s) and command do not change, and invalidate based on time. That way I could blindly do something like:
[1] https://groups.google.com/g/pandoc-discuss/c/lex900rSpOM
The problem is that it does several pandoc parses for each page:
1. Plain text - Input for audio parsing (which takes a significant amount of time, I have a version of the build that skips this).
2. Simple HTML - Input for RSS feed.
3. Full HTML - Static web page.
One change I would make to pandoc is to have the option for several outputs (as was discussed years ago [1]). I think generating multiple outputs is highly common usage.
One feature I will get around to writing (eventually) is automatic image compression. I had some success using jpegoptim and optipng on another project. This takes a while, so I want to build a command line tool that caches the output of a command if both the input file(s) and command do not change, and invalidate based on time. That way I could blindly do something like:
image="cat-pic.png"
cache-run -i $image -c "optipng -dir out/ -o 4 -strip all -fix $image"
[1] https://groups.google.com/g/pandoc-discuss/c/lex900rSpOM
Nice work and great write up!
I’m a huge fan of treating one’s blog as a “homelab” for exploring new (or old) software stacks for the web, so this idea resonates a lot.
One thing I’d like to try (has anyone tried this?): eschew the idea of a static site generator and go the “old school” route of dynamically generating each page on the fly, but take advantage of the modern CDN layer of the web (CloudFlare, etc.) to effectively cache and give you the speed equivalent of a static site. Yes this approach has more dependencies but I think simplifies the work done on the blog side since you can build it using whichever simple web framework appeals to you most (or none at all…this is a homelab after all).
My plan (don’t we all have this plan) is to try this out in Crystal when time allows…
I’m a huge fan of treating one’s blog as a “homelab” for exploring new (or old) software stacks for the web, so this idea resonates a lot.
One thing I’d like to try (has anyone tried this?): eschew the idea of a static site generator and go the “old school” route of dynamically generating each page on the fly, but take advantage of the modern CDN layer of the web (CloudFlare, etc.) to effectively cache and give you the speed equivalent of a static site. Yes this approach has more dependencies but I think simplifies the work done on the blog side since you can build it using whichever simple web framework appeals to you most (or none at all…this is a homelab after all).
My plan (don’t we all have this plan) is to try this out in Crystal when time allows…
I did the same thing a couple years ago [0]. It's not the most elegant and my blog is miniscule, but it was a fun project and also born out of frustration with the existing site generators. I structured it as the source sitting next to the site contents, so from a fresh computer (with Rust installed), it's as straightforward as `cargo build`.
I think the best thing about this approach (writing your own generator) is that it really lets you build your site / content exactly the way you want to. Don't need categories and tags? Don't add them to begin with. Want to add an RSS feed? No problem, just add it. Want to do some fancy image compression and processing in the pipeline? No problem; you're the boss. It sounds simple but it really frees you up to write / generate your site however you want.
0: https://github.com/hamaluik/blog.hamaluik.ca
I think the best thing about this approach (writing your own generator) is that it really lets you build your site / content exactly the way you want to. Don't need categories and tags? Don't add them to begin with. Want to add an RSS feed? No problem, just add it. Want to do some fancy image compression and processing in the pipeline? No problem; you're the boss. It sounds simple but it really frees you up to write / generate your site however you want.
0: https://github.com/hamaluik/blog.hamaluik.ca
As enjoyable as this post was, i recalled my history with static site generators as follows: hand-code html pages --> hand-code html but w/ server-side includes (yes, back in the apache days) --> use php w/own custom templates and mysql db --> move to wordpress --> love the intent of wordpress, but regret the needed maintenance, so move back to custom php and custom templates --> move manage content via Lektor (python SSG) --> learn that i don't need a UI, and am content crafting my content in markdown --> start using pelican (python SSG) --> get bored, so move to Hugo as new SSG flavor of the month --> get annoyed with templating language --> move back to pelican --> realized i spent too much time bouncing around, just did away with any SSG tool set, and hand craft html pages whenever i want to publish something...and now only craft journal entries for my own private diary (with an occasional post on the fediverse). :-)
I just linked your blog post in a separate comment because I was reading it over the weekend. I was looking for articles about writing static site generators and didn't realize most people don't roll their own templating engines. I was disappointed to see not much on templating in your post but it definitely gave me the confidence to keep going. Managed to get aspects I'd been struggling to work for months working yesterday.
> Several external dependencies.
> It’s annoying to install all these and to keep them up-to-date, it would be great to just have a single thing to worry about.
I feel the author's pain.
There's a Python2 project for ebook management that I used pretty extensively years ago, but it depended on a number of external dependencies, both as compiled shared objects and other Python libraries. It felt extremely fragile and was always a pain to get it working properly, so I eventually rewrote the core parts of the entire project for my own use (in Rust, incidentally). Now I have a single app that I can be sure will pretty much always work.
> It’s annoying to install all these and to keep them up-to-date, it would be great to just have a single thing to worry about.
I feel the author's pain.
There's a Python2 project for ebook management that I used pretty extensively years ago, but it depended on a number of external dependencies, both as compiled shared objects and other Python libraries. It felt extremely fragile and was always a pain to get it working properly, so I eventually rewrote the core parts of the entire project for my own use (in Rust, incidentally). Now I have a single app that I can be sure will pretty much always work.
I also have a personal Rust ebook manager because of frustrations with calibre :)
Though Calibre has at least made it to Python 3 these days.
Though Calibre has at least made it to Python 3 these days.
I went from Jekyll to to Hugo to avoid being bothered with futzing with ruby gems.
Eventually I really appreciated the Golang approach to templating much more than Jinja/Liquid style templates, Go has kind of a composition-over-inheritance approach to templates.
If I ever have time to get proficient enough with Rust I'd love to port that Go templating approach to Rust. Maybe not so much the data interpolation syntax, but rather the approach to composing template partials. But ultimately I'm familiar enough with Jinja style, Zola would probably be a breeze to adopt as well.
Eventually I really appreciated the Golang approach to templating much more than Jinja/Liquid style templates, Go has kind of a composition-over-inheritance approach to templates.
If I ever have time to get proficient enough with Rust I'd love to port that Go templating approach to Rust. Maybe not so much the data interpolation syntax, but rather the approach to composing template partials. But ultimately I'm familiar enough with Jinja style, Zola would probably be a breeze to adopt as well.
It’s refreshing to see a front-page Rust piece that just nails it on promoting the language.
It cites concrete examples of where the rubber really meets the road: e.g. Cargo as a one-stop-shop for platform insensitive, “just works” build and dep management.
It acknowledges that it’s ok for fun and novelty to be features, which it is!
It steers well clear of A >> B nonsense. No preachy TED talk about memory safety.
It’s respectful of the value that other languages provide in their respective niches.
This is how to sell Rust. It’s novel and fun but also pragmatic and industrial grade! And if you’re into X, that’s cool too!
It cites concrete examples of where the rubber really meets the road: e.g. Cargo as a one-stop-shop for platform insensitive, “just works” build and dep management.
It acknowledges that it’s ok for fun and novelty to be features, which it is!
It steers well clear of A >> B nonsense. No preachy TED talk about memory safety.
It’s respectful of the value that other languages provide in their respective niches.
This is how to sell Rust. It’s novel and fun but also pragmatic and industrial grade! And if you’re into X, that’s cool too!
Your comment helped me want to click through and read the post. I'm very tired of all the recent Rust fanboy posts, but level-headed and earnest comparisons against real implementations make for much better understanding of a new concept or language.
Here are some select quotes from the article:
> Please note that this is not to say that Rust this [sic] much faster than Haskell, instead see it as a comparison of two implementations. I’m sure someone could make it super fast in Haskell.
> I don’t think it means that Rust is simpler or easier than Haskell, it just means that I personally had an easier time to grok Rust than Haskell.
Here are some select quotes from the article:
> Please note that this is not to say that Rust this [sic] much faster than Haskell, instead see it as a comparison of two implementations. I’m sure someone could make it super fast in Haskell.
> I don’t think it means that Rust is simpler or easier than Haskell, it just means that I personally had an easier time to grok Rust than Haskell.
I religiously refresh HN the last few months and I've seen exactly zero "fanboy" Rust articles. All of them outline what was gained by a switch to it and what problems were eliminated. The better ones also list tradeoffs i.e. they say what got a bit harder, too.
Maybe you just have an axe to grind?
Maybe you just have an axe to grind?
The author is clearly an experienced and mature hacker who knows many languages well. It’s pretty difficult to maintain the shrill “X is better than everything” tone you’re referring to with a diverse portfolio of marketable skills.
I don’t know how Rust, which is quite a cool language, attracted this sizable bloc of aggressively intermediate jihadis, but it’s really worrying because Rust is in fact innovative and important, but epic languages have crashed and burned over less.
I don’t know how Rust, which is quite a cool language, attracted this sizable bloc of aggressively intermediate jihadis, but it’s really worrying because Rust is in fact innovative and important, but epic languages have crashed and burned over less.
You’re really going to call people jihadis for talking about the technical strengths of a programming language?
And then you’re going to act like you’re the self appointed arbiter of discourse on HN, saying that this kind of article is ok but that kind isn’t? Maybe a person who calls those they disagree with jihadis isn’t qualified to police what others say.
Let’s talk about the way you talk. Here’s you from 5 days ago accusing a good tool of being a “dumpster fire” and saying “it crashes process IDs more often than Justin Bieber crashes Maseratis”. (https://news.ycombinator.com/item?id=32581005). But it gets worse, because later you start crying about being downvoted.
At this point it looks like a speed run of how many HN rules you can break. And then you follow that up with tone policing others.
And then you’re going to act like you’re the self appointed arbiter of discourse on HN, saying that this kind of article is ok but that kind isn’t? Maybe a person who calls those they disagree with jihadis isn’t qualified to police what others say.
Let’s talk about the way you talk. Here’s you from 5 days ago accusing a good tool of being a “dumpster fire” and saying “it crashes process IDs more often than Justin Bieber crashes Maseratis”. (https://news.ycombinator.com/item?id=32581005). But it gets worse, because later you start crying about being downvoted.
At this point it looks like a speed run of how many HN rules you can break. And then you follow that up with tone policing others.
While his vocabulary might be a inflammatory, it's hard to argue that any language has an evangelistic community anywhere near the level of rust's. No other community is pushes their favorite language for every single use case. No other language has a rewrite it in rust meme. No other language attacks people for unsafe code.
You want to see criticism of Rust? Here’s the best, most accurate criticism of Rust, written by avid Rustaceans.
- Why Not Rust? (https://matklad.github.io/2020/09/20/why-not-rust.html)
- A Rust match made in hell (https://fasterthanli.me/articles/a-rust-match-made-in-hell)
Compare these with the superficial criticism that benreesman was making. He couldn’t even link to the proper issue, a fact he realised 10 hours later. Even had he linked to the right issue, he has no idea what he’s talking about. He says he had some issue with the language server and then simultaneously mentions LLVM? rust-analyser doesn’t import LLVM because it doesn’t need to generate machine code. You think when such poor feedback gets pushback it means people aren’t listening. No, it’s just poor feedback. If you want high quality criticism of Rust, read the articles I linked. Rust has many shortcomings and these folks discuss it in detail.
> no other language has a rewrite it in rust meme.
Your account is a year old. I’ve been here almost 10. I remember when everyone was excited about Go and wanted to rewrite everything in Go. And before Go it was Python. A couple of years from now they’ll be rewriting everything in Carbon.
This is normal when things are new and people are still discovering it. Most of the folks you’re encountering are excited by Rust … and that’s ok. It’s not a bad thing to be excited or share that with others. If that reduces your quality of life, I apologise. But they meant no harm.
> No other language
Firstly, no language does anything. And secondly, you’re making sweeping generalisations of a large community based on a couple of incidents. I don’t know there’s anything I can say, other than I haven’t seen it in years. No one thinks it’s a good idea to attack anyone, and moderators are usually quick to nip any anti-social behaviour in the bud.
Lastly, anyone with a lick of Rust experience knows that unsafe is useful and widely used. You can’t implement std::Vec without it, for instance.
- Why Not Rust? (https://matklad.github.io/2020/09/20/why-not-rust.html)
- A Rust match made in hell (https://fasterthanli.me/articles/a-rust-match-made-in-hell)
Compare these with the superficial criticism that benreesman was making. He couldn’t even link to the proper issue, a fact he realised 10 hours later. Even had he linked to the right issue, he has no idea what he’s talking about. He says he had some issue with the language server and then simultaneously mentions LLVM? rust-analyser doesn’t import LLVM because it doesn’t need to generate machine code. You think when such poor feedback gets pushback it means people aren’t listening. No, it’s just poor feedback. If you want high quality criticism of Rust, read the articles I linked. Rust has many shortcomings and these folks discuss it in detail.
> no other language has a rewrite it in rust meme.
Your account is a year old. I’ve been here almost 10. I remember when everyone was excited about Go and wanted to rewrite everything in Go. And before Go it was Python. A couple of years from now they’ll be rewriting everything in Carbon.
This is normal when things are new and people are still discovering it. Most of the folks you’re encountering are excited by Rust … and that’s ok. It’s not a bad thing to be excited or share that with others. If that reduces your quality of life, I apologise. But they meant no harm.
> No other language
Firstly, no language does anything. And secondly, you’re making sweeping generalisations of a large community based on a couple of incidents. I don’t know there’s anything I can say, other than I haven’t seen it in years. No one thinks it’s a good idea to attack anyone, and moderators are usually quick to nip any anti-social behaviour in the bud.
Lastly, anyone with a lick of Rust experience knows that unsafe is useful and widely used. You can’t implement std::Vec without it, for instance.
I don't think I made any criticisms of rust. I just pointed out it clearly has the most evangelical communit, a point which I think you helped support. I didn't even mention GP or his criticism.
None of those languages had as many rewrites as rust (which is a testament to how good the language actually is) nor did they have the same eventually obnoxious meme.
And yes, I am making sweeping generalizations about the community based on a couple of incidents. Incidents that have never happened in any other language communities I've participated in. Didn't official rust moderation team essentially quit relaticely recently because some business with the core team? I don't really care about the details. Rust is too smart for me, but I like using it anyway. I'm happy to stay separate from the community around it though.
None of those languages had as many rewrites as rust (which is a testament to how good the language actually is) nor did they have the same eventually obnoxious meme.
And yes, I am making sweeping generalizations about the community based on a couple of incidents. Incidents that have never happened in any other language communities I've participated in. Didn't official rust moderation team essentially quit relaticely recently because some business with the core team? I don't really care about the details. Rust is too smart for me, but I like using it anyway. I'm happy to stay separate from the community around it though.
(Ignoring the language itself for a moment) I absolutely love cargo. It’s so productive & easy to use. I have never had the pleasure of using such a painless cross platform build system. I’m neutral on Rust itself but Cargo is awesome.
Cargo takes a rather brute-force approach to the hard problems, but it generally just fucking works, which is a comically under-rated feature!
On the one hand, it’s depressing that Cargo is forced to “go it’s own way” in order to achieve some sanity because it can’t can’t trust the platforms, but working is working and I can’t remember more than one or two times I’ve seen it shit the bed. It’s pragmatic, useable, discoverable, and industrial strength.
On the one hand, it’s depressing that Cargo is forced to “go it’s own way” in order to achieve some sanity because it can’t can’t trust the platforms, but working is working and I can’t remember more than one or two times I’ve seen it shit the bed. It’s pragmatic, useable, discoverable, and industrial strength.
> On the one hand, it’s depressing that Cargo is forced to “go it’s own way” in order to achieve some sanity because it can’t can’t trust the platforms
I've used Rust and Cargo a bit but I don't have a lot of insight into it, what does Cargo actually need to implement itself instead of trusting the platform?
I've used Rust and Cargo a bit but I don't have a lot of insight into it, what does Cargo actually need to implement itself instead of trusting the platform?
It may be referring to the Rust ecosystem's propensity to pull code from crates.io (and for the Rust toolchain to update itself via rustup) rather than relying on distro-level package managers.
Distros do still package Rust, and some are even relatively up to date, but finding third-party libraries that work with [insert arbitrary old version of Rust] can be a challenge, because libraries often start using new APIs quite soon after they hit stable (and obviously old versions of Rust don't know what to do with that; this is the difference between backward compatibility and forward compatibility).
Distros do still package Rust, and some are even relatively up to date, but finding third-party libraries that work with [insert arbitrary old version of Rust] can be a challenge, because libraries often start using new APIs quite soon after they hit stable (and obviously old versions of Rust don't know what to do with that; this is the difference between backward compatibility and forward compatibility).
CPAN, Maven, NuGET,...
It isn't as if it wasn't done before.
It isn't as if it wasn't done before.
[deleted]
I enjoyed this post. The premise is a good one, as in for fun. A lot of these "I wrote X in Y" go on some spree of questionable assertions but "because I wanted to" is intellectually honest and understandable.
I agree with a lot of the points about Rust too. Every language is a selection of tradeoffs and for me Rust's chosen tradeoffs I think are both important (eg memory safety) and reasonable.
Take something like Cargo. The author says "It just works". This is my experience as well. But more importantly, dependency management and packaging were built into the language and ecosystem from the start. It still boggles my mind that any language that came after Java didn't do this (yes, I'm looking at you, Go).
My heart did sink a little bit at "Using Regex of course!" though.
I agree with a lot of the points about Rust too. Every language is a selection of tradeoffs and for me Rust's chosen tradeoffs I think are both important (eg memory safety) and reasonable.
Take something like Cargo. The author says "It just works". This is my experience as well. But more importantly, dependency management and packaging were built into the language and ecosystem from the start. It still boggles my mind that any language that came after Java didn't do this (yes, I'm looking at you, Go).
My heart did sink a little bit at "Using Regex of course!" though.
I honestly highly recommend the approach. Anytime I get excited about a new technology, I rewrite my blog in it. Gives me a simple live project with real traffic and all the right moving parts.
Last time I did it was with Elixir + Phoenix + PostgreSQL, without any standard or reasonable approach. It was a lot of fun and still using it today.
https://www.brightball.com/articles/insanity-with-elixir-pho...
Last time I did it was with Elixir + Phoenix + PostgreSQL, without any standard or reasonable approach. It was a lot of fun and still using it today.
https://www.brightball.com/articles/insanity-with-elixir-pho...
Ah yes, one of my favorite projects in a newly learned language: Make a blog, based on plain text files, maybe markdown, maybe something else.
But since I have done that, I feel that many existing site generators are bloated, probably, because they cover scenarios, which I would not want on my blog anyway.
But since I have done that, I feel that many existing site generators are bloated, probably, because they cover scenarios, which I would not want on my blog anyway.
Don't forget to throw your static site generator on the pile: https://jamstack.org/generators/
347 options and counting!
347 options and counting!
Fun project! I really resonate with this part
> But I haven’t used Haskell in years, and the overhead for me to add slightly more complex things to the site is quite large
I’m starting to come around to the idea that it might be better to avoid shiny fun things in favor of having most of my active projects use the same stack, or at least the same few languages.
I’ve got a blog in Jekyll (with some Ruby-based customizations), a bunch of stuff in JavaScript, some TypeScript, an app in Elixir & Phoenix, a (desktop) app in Rust, and some Rails to maintain. For me personally, Elixir is the odd one out: I learned enough to build that app, and I really enjoyed Elixir, but that knowledge was fleeting because I haven’t needed to touch the code very much. I have to relearn how Phoenix works in order to make changes. As much as it pains me to say this, I kind of wish it was in TypeScript or something, just to reduce the mental overhead. (but then, that’s a whole other can of worms because of how piecemeal things are in the Node ecosystem, so I’d end up having to learn a web framework and an ORM and some other libraries, which feels like the sort of knowledge that might be even more fleeting!)
> But I haven’t used Haskell in years, and the overhead for me to add slightly more complex things to the site is quite large
I’m starting to come around to the idea that it might be better to avoid shiny fun things in favor of having most of my active projects use the same stack, or at least the same few languages.
I’ve got a blog in Jekyll (with some Ruby-based customizations), a bunch of stuff in JavaScript, some TypeScript, an app in Elixir & Phoenix, a (desktop) app in Rust, and some Rails to maintain. For me personally, Elixir is the odd one out: I learned enough to build that app, and I really enjoyed Elixir, but that knowledge was fleeting because I haven’t needed to touch the code very much. I have to relearn how Phoenix works in order to make changes. As much as it pains me to say this, I kind of wish it was in TypeScript or something, just to reduce the mental overhead. (but then, that’s a whole other can of worms because of how piecemeal things are in the Node ecosystem, so I’d end up having to learn a web framework and an ORM and some other libraries, which feels like the sort of knowledge that might be even more fleeting!)
There’s a lot of wisdom in that observation.
Knowing a lot of languages well is a super-power in some ways, but heavily polyglot environments create real pragmatic challenges. There’s a reason so many marquee shops carefully curate a (short) menu of production languages.
It’s been my experience that if (and it’s a big if) you have either the need or inclination to operate a big orgy of polyglot language interop, the hellacious learning curves on NixOS and Bazel are “The Way”. It’s a big investment and only worth it if you’re pretty committed/obligated, but there is a path for those whose needs justify the cost.
Knowing a lot of languages well is a super-power in some ways, but heavily polyglot environments create real pragmatic challenges. There’s a reason so many marquee shops carefully curate a (short) menu of production languages.
It’s been my experience that if (and it’s a big if) you have either the need or inclination to operate a big orgy of polyglot language interop, the hellacious learning curves on NixOS and Bazel are “The Way”. It’s a big investment and only worth it if you’re pretty committed/obligated, but there is a path for those whose needs justify the cost.
> As much as it pains me to say this, I kind of wish it was in TypeScript or something, just to reduce the mental overhead.
If you once knew how Elixir works (syntactically), then you will be able to quickly bounce back. If comparing TypeScript with Phoenix, then you would have to relearn your in TS written functionality, in contrast to relearning Phoenix.
I don't see how it being TS would help much tbh. I think what matters much more is, whether your past self left good comments and wrote readable code.
If you once knew how Elixir works (syntactically), then you will be able to quickly bounce back. If comparing TypeScript with Phoenix, then you would have to relearn your in TS written functionality, in contrast to relearning Phoenix.
I don't see how it being TS would help much tbh. I think what matters much more is, whether your past self left good comments and wrote readable code.
If you're interested in this, a while ago, for my personal photo site with image galleries[1], I wrote a fast static site generator in Rust[0], which also supports plugins now.
[0] https://www.getblades.org
[1] https://grego.site/photo
[0] https://www.getblades.org
[1] https://grego.site/photo
Great post- great project. Always fun to see someone building something (from scratch!) that filled a need they personally had.
This is a cool idea, I've wanted to find a Rust project for a while and regenerating my own site (which uses jekyll) might not be a bad idea too. Every time I come back to mine it never works because it always complains about the wrong ruby library versions being installed.
I switched from Jekyll to Zola because of my difficulty running Jekyll locally too. I chose Zola because it's a single binary and its template language (Tera) is similar to Jekyll's so my blog was fairly easy to port.
For my own blogging needs, I chose an even more pragmatic alternative to markdown - email!
Well, email is a actually a single file CMS which contains html, plaintext, attachments/images, and metadata like when the email was sent and the subject line,
With all this, it turns out you can create pretty nice looking webpages without having to futz around with md->html conversions.
The huge advantage is that since I know gmail, whenever I have an idea for a blog post, I can go from idea to published in minutes. The biggest time suck tends to be finding stock photos for the post.
As far as tech stack goes, it’s all serverless. I use SES to handle inbound emails and the excellent nodemailer package to parse .eml files.
Well, email is a actually a single file CMS which contains html, plaintext, attachments/images, and metadata like when the email was sent and the subject line,
With all this, it turns out you can create pretty nice looking webpages without having to futz around with md->html conversions.
The huge advantage is that since I know gmail, whenever I have an idea for a blog post, I can go from idea to published in minutes. The biggest time suck tends to be finding stock photos for the post.
As far as tech stack goes, it’s all serverless. I use SES to handle inbound emails and the excellent nodemailer package to parse .eml files.
> On my crappy laptop a full site rebuild took 75 seconds. (Not compile, just to generate the site.) I only have 240 posts so I don’t think it should be that slow.
Wow, what on earth was Hakyll doing all that time?
Wow, what on earth was Hakyll doing all that time?
I really liked the reactive pattern, so one of the best static site generators I found was docpad. It used (backbone?) collections, so you could subscribe to changes on things like "new files under this path" and then rerender the part of the page that needed to change. I haven't seen this done in others. Then docpad was rewritten in ES6, and I lost all love for it. Things were so expressively concise in Coffeescript.
It's kind of ironic and uncool that DocPad's website is made using GitBook.
The author is a pretty interesting guy, but I just didn't want to follow him into ES6 land. Coffeescript is still the ranking king on concision, and so many features didn't make it in. I'm going to be sad when I can't support the transpile in my projects anymore :(
[deleted]
Why even bother with a static generator when you can run rust code at essentially wire speed and dynamically generate all the required formatting on the fly?
Because Rust binaries are so secure, you can expose them over the internet safely. With a little bit of glue, you can automatically recompile the CGI binary every time the rs source is touched, for a dev experience similar to interpreted web scripts.
Because Rust binaries are so secure, you can expose them over the internet safely. With a little bit of glue, you can automatically recompile the CGI binary every time the rs source is touched, for a dev experience similar to interpreted web scripts.
> Because Rust binaries are so secure, you can expose them over the internet safely.
That doesn’t stand to reason. Safely in that you can confidently avoid things like remote code execution, sure. But that doesn’t necessarily protect you from denial of service attacks; for example, I expect most services built on Hyper to be vulnerable to slowloris attacks <https://en.wikipedia.org/wiki/Slowloris_(computer_security)>, because I believe it still defaults to basically no sane kind of timeout, and doesn’t provide the tools to handle it properly anyway <https://github.com/hyperium/hyper/issues/1628>. If that’s part of your threat model, you should avoid exposing it directly and use a reverse proxy.
Good somewhat related reading, on a technologically simple DoS attack on Rust-based code that was largely fixed by speeding things up, fixing caching and proper use of reverse proxying (via Cloudflare because it was genuinely heavy load): https://fasterthanli.me/articles/i-won-free-load-testing
That doesn’t stand to reason. Safely in that you can confidently avoid things like remote code execution, sure. But that doesn’t necessarily protect you from denial of service attacks; for example, I expect most services built on Hyper to be vulnerable to slowloris attacks <https://en.wikipedia.org/wiki/Slowloris_(computer_security)>, because I believe it still defaults to basically no sane kind of timeout, and doesn’t provide the tools to handle it properly anyway <https://github.com/hyperium/hyper/issues/1628>. If that’s part of your threat model, you should avoid exposing it directly and use a reverse proxy.
Good somewhat related reading, on a technologically simple DoS attack on Rust-based code that was largely fixed by speeding things up, fixing caching and proper use of reverse proxying (via Cloudflare because it was genuinely heavy load): https://fasterthanli.me/articles/i-won-free-load-testing
Sorry for the lack of clarity, by "exposing the binary over the internet" I certainly wasn't implying re-implementing a HTTP server, HTTPS and certificate handling etc. You would of course have an nginx frontend that would cache the request and subtleties; the Rust FastCGI would see a well formed request, and rush to execute it just like any other dynamic language, PHP, Python.
You can of course still do stupid things inside the rust application that explode your number of concurrent requests, like waiting for slow external resources, databases etc. For a simple static formatter, it should not be the case, it would just repeatedly hit the disk cache and some minimal CPU.
You can of course still do stupid things inside the rust application that explode your number of concurrent requests, like waiting for slow external resources, databases etc. For a simple static formatter, it should not be the case, it would just repeatedly hit the disk cache and some minimal CPU.
If you don’t mean exposing it directly, I’m not sure what’s special about Rust in what you’re saying. Compared to deliberately memory-unsafe languages, perhaps, but you’re normally comparing it to the likes of Python, PHP, Ruby and Go, all of which stand on level ground in that regard.
> Python, PHP, Ruby and Go
... but with an execution speed that consumes negligible CPU time from the moment the template/source is loaded into memory until it is formatted and put on the wire. Compared to that, a PHP template is dog slow. I don't have any direct experience with Go, small GC allocations for string handling might introduce non-negligible overhead.
... but with an execution speed that consumes negligible CPU time from the moment the template/source is loaded into memory until it is formatted and put on the wire. Compared to that, a PHP template is dog slow. I don't have any direct experience with Go, small GC allocations for string handling might introduce non-negligible overhead.
Yeah, but that stuff has nothing to do with the point I was responding to: “Because Rust binaries are so secure, you can expose them over the internet safely.”
[deleted]
I prefer to take your approach, but one advantage of a static generator is that it opens up hosting location options such as github pages, dropping in an S3 type public storage, many traditional web hosts, netlify, etc.
I'm seeing a few too many comments about static site generators rebuilding everything on a change.
Have the tenets of Make (and the like) from over 40 years ago been forgotten?
I guess maintaining a dependency graph just isn't a high priority feature for site generators.
Have the tenets of Make (and the like) from over 40 years ago been forgotten?
I guess maintaining a dependency graph just isn't a high priority feature for site generators.
Computers are fast, sites get updated infrequently, and compiling a static site is a lot less computational work than dealing with the utter *********** that is C++ templates.
Unless you're 'web-scale', it's complexity that doesn't solve any particular business need.
Unless you're 'web-scale', it's complexity that doesn't solve any particular business need.
For something run relatively infrequently, I can see why it wouldn't be a huge priority.
On that topic, I wish there was a generic build system like SCons written in something faster. Maybe there is (does Bazel fit my bill?) and I just don't know of it yet. What I love best about SCons is that you can just write custom builders right there. No need to drop in a shell script - you have the much better Python standard library at your fingertips. This also means it's way easier to make cross-platform.
On that topic, I wish there was a generic build system like SCons written in something faster. Maybe there is (does Bazel fit my bill?) and I just don't know of it yet. What I love best about SCons is that you can just write custom builders right there. No need to drop in a shell script - you have the much better Python standard library at your fingertips. This also means it's way easier to make cross-platform.
Not to be a negative Nancy, but the phrase “for fun and profit” is quite overused that any originality is lost.
[deleted]
[1] https://www.getzola.org/