static-server: an HTTP server in Go for static content(eli.thegreenplace.net)

87 pointsby chmaynardvor 3 Jahren63 comments

eli.thegreenplace.net

static-server: an HTTP server in Go for static content

https://eli.thegreenplace.net/2023/static-server-an-http-server-in-go-for-static-content/

66 comments

Awesome. There is a very surreal joy in building such standalone tools even just for learning. I have been working on a kitchen sync style "chat" app to experiment with new ideas (like integration with X/y/z). I just need to muster the courage to publish it!

bbkanevor 3 Jahren

100% agree. I've been "reinventing the wheel" (my own terminal color library, CLI framework, GNU Stow alternative, etc) and loving it! I get a lot of satisfaction from using these simple tools [0].

I say publish your work. Chances are, no one will care and you have a nice backup of your code. And if someone does find it useful, then great!

https://github.com/bbkane/

ssddanbrownvor 3 Jahren

I wrote something like this [1] for the team at my old workplace. We mainly worked on static html files but some functionality required loading via http. Many of the devs were quite new to development in general, so I built a simple static server in Go, as an exe which they could set as the default for `.html` files, and therefore they could just open html files on the webserver via double-click as normal. The program would watch the directory (if not already open and watching) then open the right path in the default browser. I also built-in livereload, management via tray icon and a basic web UI.

[1] https://github.com/ssddanbrown/webby

simonwvor 3 Jahren

This is cool - I didn't know you could have Go one-liners that look like this:

    go run github.com/eliben/static-server@latest

I had to upgrade to Go 1.21 for this to work - I was previously on Go 1.20. "brew upgrade go" worked for me.

Looks like almost the entire implementation is here, it's mostly CLI option parsing logic: https://github.com/eliben/static-server/blob/main/internal/s...

elibenvor 3 Jahren

Indeed, the actual heavy lifting is done by the Go standard library; this line (https://github.com/eliben/static-server/blob/main/internal/s...) is where the real magic happens (ignoring the logging middleware):

    fileHandler := serveLogger(serveLog, http.FileServer(http.Dir(rootDir)))

benhoytvor 3 Jahren

Yeah, with the "go 1.21.0" line in the go.mod file, it only works on Go 1.21. Here's what I get on Go 1.20:

  $ go run .
  go: errors parsing go.mod:
  .../static-server/go.mod:3: invalid go version '1.21.0': must match format 1.23

Eli, you might consider changing that to "go 1.20" so it works on Go 1.20 (and older, actually -- it's the 1.2.3 format that is getting in the way on Go pre-1.21).

elibenvor 3 Jahren

Done now; thanks for the note!

PhilippGillevor 3 Jahren

There are many projects doing the exact same thing. Here's one approach from myself, but haven't updated since 2019:

https://github.com/philippgille/serve

One difference is that it can create a TLS cert for you, instead of having to supply one via CLI arg.

I don't recommend using it in its current form though due to its (the compiled binaries') Go version being outdated.

quinnftwvor 3 Jahren

Looking at the code this appears [1] to be a very thin wrapper around go's built in file server from the http package.

[1]: https://github.com/eliben/static-server/blob/main/internal/s...

canadaduanevor 3 Jahren

Out of curiosity, are there alternatives to this that are already well established or distributed? For example, something you'd get in debian linux?

xmodemvor 3 Jahren

https://caddyserver.com/ is implemented in Go, production-ready, and easy to setup with a one-liner (though personally I would use official binaries or compile from source rather than use the builds from a distro package manager)

TylerEvor 3 Jahren

apache? nginx? Serving static content off the file system is braindead simple.

ameliaquiningvor 3 Jahren

What's the zero-config way to temporarily spin up an instance of Apache or Nginx that serves my current directory? The way Debian does this out of the box seems significantly higher-ceremony than that.

bryancoxwellvor 3 Jahren

python -m http.server

ameliaquiningvor 3 Jahren

Doesn't support HTTPS out of the box, unfortunately.

mholtvor 3 Jahren

These are fun to make. I've done it several times. :) That is why I wrote Caddy:

    $ caddy file-server

It does templates, TLS, and other production things really easily from the command line too, including automatically getting certificates:

    $ caddy file-server --domain example.com

Done!

I think projects like static-server are wonderful learning examples of how to get Useful Things done in Go.

mnstngrvor 3 Jahren

Thank you for Caddy! Going from a complex nginx setup to the delight that's the Caddyfile was a breeze.

And a special call-out to your dedication to providing amazing documentation. Caddy is what all projects should aspire to be.

mbakkevor 3 Jahren

Caddy is amazing. I accidentally wiped my fairly complex nginx configuration (yay apt-get purge) and have been so spoiled by Guix that I did not care about etckeeper or ansible on my one Debian server.

Anyway I decided to try Caddy to quickly get up and running and had a good setup in minutes including certificates for multiple domains. Even added wildcard certificates which I never figured out with Lets Encrypt, just because I could.

Thanks for this amazing software :-)

mholtvor 3 Jahren

That's awesome! Thank you for the experience/feedback.

mholtvor 3 Jahren

Thank you! We try very hard (collectively, as a community).

elibenvor 3 Jahren

Thanks for the comment, Matt! I love Caddy. For some reason I thought it needs a configuration file.

The simplest cmdline I found to run it as a local server on a non-priveledged port with listings is:

    $ caddy file-server --listen localhost:8099 --browse

Is there a simpler way I'm missing?

mholtvor 3 Jahren

On Mac, the defaults aren't privileged. (Not sure why. That's just how it is.)

The --browse flag is optional and unrelated to ports/privileges. Otherwise, that sounds about right. Caddy defaults to port 80 unless you give it a domain name, then it defaults to port 443 and redirects HTTP on port 80 to HTTPS.

e12evor 3 Jahren

Wait, file-server without arguments does dynamic content (templating)?!

I assume that's just poorly phrased?

mholtvor 3 Jahren

To clarify, you do need to use --templates to enable template evaluation on the static files.

But for the `caddy respond` command, minimal templates are available by default; see the docs here: https://caddyserver.com/docs/command-line#examples

skrebbelvor 3 Jahren

Only for directory listings I think. And of course you can turn that off as well.

[deleted]vor 3 Jahren

timenovavor 3 Jahren

Hey, I was just wondering if there's any timeline on when the caddy-l4 module be compatible with Caddyfile?

I know I can write it in JSON, but IMO the greatest appeal of Caddy is its simple and easy-to-read Caddyfile!

mholtvor 3 Jahren

I, too, would like to see this, but it'll be a lot of work. It's just not a top priority right now for me, unfortunately.

This is something I could prioritize if a business wanted to sponsor this; but right now the most pressing things are updating the docs, revamping our test suite, and I have a few action items for existing sponsors I need to prioritize too.

Anyway, no timeline -- but definitely something wanted!

PS. There is kind of Caddyfile support here: https://github.com/RussellLuo/caddy-ext/tree/master/layer4

chaxorvor 3 Jahren

I'm (very) unfamiliar with webdev and hosting. Is this 'oneliner' a secure way to host files? I have heard that using a reverse proxy, some 2FA on a cloud gateway that forwards to another location, with a VLAN for the file server, and containerization, etc are all the standard practice now for ensuring security. Is there anything that works toward making this goal more easily amenable (since it's not possible with one project), or should I just keep using python -m http.serve and not care?

mholtvor 3 Jahren

It depends. What is your definition of "security"? There's lots of dimensions of security in the web serving space. If you list out your requirements and specify your threat model, then answers become a lot clearer.

But yes, in general: Caddy's one-liner is a safe way to serve static files in the sense that remote services can't upload files or escape memory bounds to run arbitrary code. It encrypts your connections for you, so they're basically safe from surveillance and modification.

The elements you're describing are all external factors that have nothing to do with serving static files specifically. For example, a reverse proxy just multiplexes requests coming in on a port to various backends, maybe making modifications along the way. Authentication will restrict access to only allowed users, if that's something you require. Containers are more a way of administering a system or mitigating very specific risks that have niche relevance with static Go programs; and VLANs are just ways of isolating network traffic, but again it's somewhat orthogonal to file serving.

Overall, `caddy file-server` is better than Python's simple HTTP server in every way, though: static binary, faster performing, production-ready (handles Range requests properly, and a few other details), automatic HTTPS, folder indices, etc...

abhishekjhavor 3 Jahren

Is there more to the philosophy of why caddy exists? I am trying to jump into a few open source projects to hone my programming skills and I have a harder time comprehending things if I don't know what was the overall intention of the authors.

mholtvor 3 Jahren

Caddy 1 was created because I needed a quick and easy web server for a lot of my projects.

Caddy 2 was created when I got serious about it, and we had governments and enterprises starting to rely on the project.

Now we exist because we advocate for (and deliver!) HTTPS on every site, memory safety, dynamic configuration, extensibility, and many more features valuable in a modern web server.

Hope that helps :)

abhishekjhavor 3 Jahren

Thank you for the response. It looks interesting.

>I needed a quick and easy web server

I am trying to reason with what I know, but wouldn't something like "python -m http.server 9000" suffice? Or nginx? These are battle tested and python is there on most platforms.

Or is there more depth to this specific web server that others don't have?

mholtvor 3 Jahren

No, because python's simple HTTP server isn't production ready (doesn't support Range requests last I checked, and quite a few other limitations). And nginx is not "quick and easy" by any means, doesn't grant me the security of memory safety (it was vulnerable to Heartbleed, for example -- I wrote Caddy shortly after Heartbleed), and doesn't give me automatic HTTPS.

And Caddy is very well battle-tested too, btw. ;)

housemusicfanvor 3 Jahren

From afar, Caddy is one of those weird HN cargo cults.

It seems like in every thread about web servers, the developers are in here peddling their wares, and the disciples pour in with endless praise.

It's another product with insane footgun defaults (the admin API on localhost) in a sea of mature alternatives. That's all I need to know.

I don't mean to insult anyone's hard work but in the words of Josh Baskin, "I don't get it."

mholtvor 3 Jahren

What's wrong with the admin API on localhost, exactly?

simonwvor 3 Jahren

One of the big selling points of Caddy for me is that you can reconfigure it "live" without restarting the server.

This is great for if you want to be able to do zero-downtime deploys of new applications behind a proxy server or similar.

rcmevor 3 Jahren

I think nginx does that too; if you send it a hup signal, it will reload the config.

housemusicfanvor 3 Jahren

Indeed, many programs interpret a kill -HUP signal as a "re-read config" command.

mholtvor 3 Jahren

This is problematic as when you log out your shell, HUP is sent to the web server process unintentionally...

(One reason we don't use signals in Caddy 2. So uncivilized.)

mholtvor 3 Jahren

Absolutely! Thanks for the feedback, Simon!

unethical_banvor 3 Jahren

Caddy's appeal when I last looked at it some years ago was that the configuration file format was very easy to read and use; and that configuration itself was minimal. It came/comes with Letsencrypt integration which was novel some years ago, so setting up a secure site was even easier for people who weren't used to it.

Its syntax for being configured as a reverse proxy (secure front-end to less secure back-end servers) is similarly pretty easy.

I haven't looked at it in a long time because I recall them screwing with their terms of service so it wasn't fully FOSS by the most liberal definition.

kstrauservor 3 Jahren

It’s had an Apache-2.0 license for at least the last 4 years: https://github.com/caddyserver/caddy/blob/master/LICENSE

mholtvor 3 Jahren

Caddy has ALWAYS had the Apache 2.0 license. For a time we did additionally offer officially licensed binaries for companies, but Caddy has never deviated from Apache 2.

Caddy's ease of use is one feature, but there are many more - like the ability to massively scale your TLS to thousands of sites reliably. And to use the on-line configuration API to make changes to your server. There's a ton to discover with Caddy, it's not just a tool for beginners. ;)

[deleted]vor 3 Jahren

gabereiservor 3 Jahren

While I love Go, have we gotten this lazy that we need a package for this? Go does this in 3 lines minimum, like you describe in your blogpost. However, in your package you expose the ability to kill your server [0] without any security. That’s a huge vulnerability. I know you’ll say “It’s just a static server, meant for serving static stuff” but it will be indexed by pkg.go.dev, people will use this outside your intent. It is the way. At the very least, use a secret token.

[0] https://github.com/eliben/static-server/blob/3ce83524ed54298...

jjicevor 3 Jahren

> I know you’ll say “It’s just a static server, meant for serving static stuff” but it will be indexed by pkg.go.dev, people will use this outside your intent.

While true, I don't think the author should refrain from making code available based on the potential negatives from others using code they didn't even bother to read the documentation for.

gabereiservor 3 Jahren

You’re right, however, due to the nature of the go ecosystem, someone will use it - host their react app with it - and expose an endpoint that could shutdown their server. I think that warrants being called out for.

xcdzvynvor 3 Jahren

rm -rf /

Need I be called out, now?

elibenvor 3 Jahren

Thanks for your comment. I surely hope no one will even consider using this server for anything public-facing :) It's solely for testing on localhost.

The shutdown endpoint is used for robust testing; I suppose I can hide it a bit more, like using an environment variable or something.

gabereiservor 3 Jahren

Just check a header for a secret key you generate when you startup. Easy peasy. This keeps you able to call it for testing (granted you read from stdout or passed the key to tests as a variable). Then some scripto ransomware User from Omgodisztan doesn’t shutdown your server from the tent he’s camped in with Starlink.

elibenvor 3 Jahren

This is done now, thanks for the suggestion

d-z-mvor 3 Jahren

It's fine the way it is IMO. However, it might be worth caveating in the README that it's for local testing only, the same way you do in your blog post.

Mainly because of the shutdown endpoint, but also that the -cors flag returns "Access-Control-Allow-Origin: *" exposing you to arbitrary cross origin requests.

midwitvor 3 Jahren

A simple middleware hook for http basic auth :)

badrequestvor 3 Jahren

People are allowed to write code/tools that are useful to only themselves. They're also allowed to post about these tools. Nobody is going to make you use them.

nicoburnsvor 3 Jahren

These kind of servers are useful for quickly serving a folder of files locally. Security isn't a primary concern for these kind of use cases.

tgvvor 3 Jahren

$ python -m http.server

chrismarlow9vor 3 Jahren

https://gist.github.com/willurd/5720255

inChargeOfITvor 3 Jahren

Assuming you have python installed on the system, yeah.

sdf4jvor 3 Jahren

python 2 and not 3

maleldilvor 3 Jahren

`python -m http.server` is the one for Python3. Python 2 is `python -m SimpleHTTPServer`

joaomacpvor 3 Jahren

or the node one:

$ npm install http-server

$ http-server .

parminyavor 3 Jahren

or the go one:

$ go run github.com/eliben/static-server@latest

PinguTSvor 3 Jahren

That is the argument of the first MVP by a startup and then it is their onlien product.

I have this seen also in automotive. "This is no problem, because this is not connected to the Internet." Then a few years later you have a DefCon presentation "GM hack, you can control the whole car via the Internet".

gabereiservor 3 Jahren

Where are you hosting your client-side code? Let me see if I can shut it down…

[deleted]vor 3 Jahren