Sherver: Bash lightweight web server(github.com)

163 pointsby turrinivor 5 Jahren66 comments

https://github.com/remileduc/sherver

70 comments

> I didn't want to install and configure Apache or nGinx. In fact, I didn't want any configuration.

Personally, in that case, I'd use:

python3 -m http.server

adamddev1vor 5 Jahren

`npm install -g live-server`

Also works great (but a bit bulkier sure).

rasculvor 5 Jahren

There are many similar options. I think that awhile ago there may have been a list of one liner web servers posted to HN. Either way, I've been using python's for years and it's just muscle memory now.

em500vor 5 Jahren

Big list of http static servers:

https://gist.github.com/willurd/5720255

nickjjvor 5 Jahren

That's my goto solution for a simple server if I'm testing something locally but it has issues dealing with concurrent requests in its simple form.

For example I ran a simple hello world test service on Kubernetes once using Python's http.server.

Just having 3 Kubernetes health checks happen on regular intervals would routinely cause the server to respond with errors. I think what ends up happening is if you have 2 requests coming in within a short enough period of time one of them will fail. You have to use a more complicated threaded http server if you want a zero dependency solution.

zoomablemindvor 5 Jahren

Just an idea, along the lines of a single binary webserver: one may use Fossil SCM [1] to easily spawn a webserver ('fossil server', or even 'fossil ui'), no need for root (port 8080 by default).

The pages could be served as part of a project (all version-controlled), or as documentation pages, or as wiki. Styling is somewhat limited, but could still be done creatively.

Additionally, a static website or even a whole application (PHP or the likes) could be rolled out via /ext, the 'extension' path.

[1]: https://www.fossil-scm.org

medvvor 5 Jahren

It is a socat server, not bash. Basically a bash script calling socat.

Definitely not “pure”.

loegvor 5 Jahren

There’s always https://github.com/cemeyer/httpd.sh .

dangvor 5 Jahren

Ok, we've impured the title above.

[deleted]vor 5 Jahren

prussianvor 5 Jahren

Someone beat me to it. When I think of pure bash, I don't really think of invoking anything that isn't a builtin or a grammatical feature of bash. If socat is fair-play, almost anything you can spawn on a shell should count then.

edit: on top of that, I already found a perl script utility in the repo: https://github.com/remileduc/sherver/blob/master/scripts/uti...

bluetomcatvor 5 Jahren

Jokingly, "sudo service apache2 start" may also be considered pure Bash :-)

fragbait65vor 5 Jahren

Exactly!

fragbait65vor 5 Jahren

I think I'll just write something similar and wrap Apache with a shell scrip, claim that it's a web server written in bash, post it here and see what reaction I get, just for shits and giggles.

prussianvor 5 Jahren

Here: https://github.com/remileduc/sherver/issues/1 that way you can have HTTP/1.1 support even with arbitrary output lengths. I was planning on sending this to the linked repo so the owner doesn't need to use HTTP/1.0.

EDIT: changed link to the github issue I created with inlined utility.

junonvor 5 Jahren

~~Right. "Pure" bash would mean they somehow got sockets working in the scripting language itself, which is not possible unless you're on a system with `/dev/tcp` (which isn't many these days).~~

I am wrong! See response.

rzzztvor 5 Jahren

/dev/tcp is a Bash construct (mentioned as "pseudo-device file" in the manual): https://tldp.org/LDP/abs/html/devref1.html

junonvor 5 Jahren

Oh huh, TIL it's only a Bash thing. I stand corrected - I thought it was a Linux thing that was removed for security reasons.

Thanks for correcting me :)

tyingqvor 5 Jahren

Bash can't do listen sockets, so I don't think a pure bash web server is possible. You can make an "pure" gawk web server, though it works terribly https://gist.github.com/willurd/5720255#gistcomment-3143007

rasculvor 5 Jahren

This seems like an odd usage of cat and command substitution to me:

    HEAD_TEMPLATE=$(cat <<EOF
        <title>Sherver example</title>
        <meta name="description" content="Sherver example">
    EOF
    )

https://github.com/remileduc/sherver/blob/master/scripts/ind...

rasculvor 5 Jahren

I believe the pure bash way of assigning a multi line string to a variable, without dealing with quoting, might be as follows:

    IFS='' read -r -d '' HEAD_TEMPLATE <<'EOF'
        <title>Sherver example</title>
        <meta name="description" content="Sherver example">
    EOF

https://stackoverflow.com/questions/1167746/how-to-assign-a-...

throwaway984393vor 5 Jahren

I mean, if it's got no single quotes:

  HEAD_TEMPLATE='
  <title>Sherver example</title>
  <meta name="description" content="Sherver example">
  '

rasculvor 5 Jahren

Yes this is indeed simpler, however I noticed in some of the code single quotes were also used [0] so I wanted to use an example that would work for that.

[0] https://github.com/remileduc/sherver/blob/master/scripts/ind...

junonvor 5 Jahren

It achieves multiline block text in bash, which is otherwise a PITA to reason about. It can keep certain scripts cleaner, especially since `cat` is often inlined, without process spawning.

steerablesafevor 5 Jahren

> especially since `cat` is often inlined, without process spawning.

That would be surprising. How would that work? `cat` is definitely not a bash built-in.

junonvor 5 Jahren

Cat is not a specified built-in but many implementations do build it in to speed up scripts that use it quite heavily.

Other examples include `which`, `time`, `echo`, etc. They're usually promoted to built-ins when it's clear that's the intended command to run, but there are still analogs on the PATH if need-be - I'm not sure exactly how bash would detect a non-standard executable there to know when it's unsafe to promote to a built-in, though.

For example with echo:

    $ bash -c 'echo hi'
    hi
    
    $ strace bash -c 'echo hi' | grep 'exec'
    execve("/usr/bin/bash", ["bash", "-c", "echo hi"], 0x7fffe264aad0 /* 24 vars */) = 0
    
    $ strace bash -c '$(which echo) hi' | grep 'exec'
    execve("/usr/bin/bash", ["bash", "-c", "$(which echo) hi"], 0x7fffe36c6cd0 /* 24 vars */) = 0
    execve("/usr/bin/echo", ["/usr/bin/echo", "hi"], 0x7fffe2e16ff0 /* 25 vars */) = 0

In fact for me, `cat` always gets inlined, even if I call it with its full path or with `which cat`.

There are cases where the built-in and the actual executable differ - the most prominent case being `time`. The built-in does not have the command line arguments to print out things like pre-empted scheduling (forced context swiches).

    $ time --help
    --help: command not found
    
    $ /usr/bin/time --help
       Usage: /usr/bin/time [-apvV] [-f format] [-o file] [--append] [--verbose]
          [--portability] [--format=format] [--output=file] [--version]
          [--quiet] [--help] command [arg...]

steerablesafevor 5 Jahren

"echo" and "time" are explicitly bash built-ins, "which" and "cat" are not, and you will always get a process spawned there.

Your second strace is misleading, you didn't trace child processes.

    $ strace bash -c '$(which echo) hi' 2>&1 | grep 'exec'
    execve("/bin/bash", ["bash", "-c", "$(which echo) hi"], 0x7ffddfcdaee0 /* 59 vars */) = 0
    execve("/bin/echo", ["/bin/echo", "hi"], 0x55eb565cb0b0 /* 59 vars */) = 0
    
    $ strace -f bash -c '$(which echo) hi' 2>&1 | grep 'exec'
    execve("/bin/bash", ["bash", "-c", "$(which echo) hi"], 0x7ffd09b3e228 /* 59 vars */) = 0
    [pid 81493] execve("/usr/bin/which", ["which", "echo"], 0x5560d172e0b0 /* 59 vars */) = 0
    execve("/bin/echo", ["/bin/echo", "hi"], 0x5560d172e0b0 /* 59 vars */) = 0

rasculvor 5 Jahren

You had:

    $ strace bash -c 'echo hi' | grep 'exec'
    execve("/usr/bin/bash", ["bash", "-c", "echo hi"], 0x7fffe264aad0 /\* 24 vars */) = 0

Something seems off. strace(1) prints the trace to stderr not stdout, so you should need |& not | to catch it with grep. At least, that's how it works here.

Also, cat does not appear to be inlined for me:

    rascul@smarts:~> strace bash -c 'cat /etc/os-release' |& grep 'exec'
    execve("/usr/bin/bash", ["bash", "-c", "cat /etc/os-release"], 0x7ffe7b5334e0 /* 88 vars */) = 0
    execve("/usr/bin/cat", ["cat", "/etc/os-release"], 0x562c447076f0 /* 88 vars */) = 0*

junonvor 5 Jahren

Yeah good catch, I forgot it since I didn't copy 1:1 the command line into vim when I formatted it for HN. I use fish + powerline so the command line doesn't copy correctly. In fish, it's `&|`, which would look weird anyway. I just forgot to add the `&` after the `|`.

The output was copied 1:1 though.

The inlining is an implementation detail and I've seen it happen differently across different platforms.

rasculvor 5 Jahren

> The inlining is an implementation detail and I've seen it happen differently across different platforms.

Do you have more information on this? I would be interested to read more about it. As far as I was aware (and my knowledge is certainly not exhaustive), the only thing kinda related is as noted in the bash man page:

> The command substitution $(cat file) can be replaced by the equivalent but faster $(< file).

junonvor 5 Jahren

I don't, sorry. I just know it's something some implementations opt-in to (especially drop-in replacements to bash). It's behavior I've observed quite a bit and have leveraged it in some cases to improve the performance of some critical bash scripts over the years. I'd also be interested in some prose about it.

prussianvor 5 Jahren

Are you sure?

    strace -ff -e execve /bin/bash -c 'x="$(cat <<< "test")"'
    ... snip
    [pid 311214] execve("/usr/bin/cat", ["cat"], 0x56280da76fc0 /* 3 vars */) = 0

junonvor 5 Jahren

Yep, I'm sure. It's not a guarantee (nor is it specified, to my knowledge) that the inlining happens. You also used a subshell there, which might make a difference.

BeefWellingtonvor 5 Jahren

The original poster is using a subshell. Your example uses echo, which is both a built-in AND an external program.

They are not comparable. I've never seen a shell inline cat, probably because it would make no sense given there are POSIX builtins to do exactly what cat does.

junonvor 5 Jahren

It of course makes sense since `cat` is widely used for the purpose of the original code, just not in a subshell. It's used often to output blocks of text to stdout/err for multiline output, namely --help text.

strace doesn't lie here, and I've seen it substitute cat in trivial cases quite often.

BeefWellingtonvor 5 Jahren

Indeed strace doesn't lie but you have yet to provide an example of a system where what you're saying about cat is true. You've shown a number of examples which do not actually back up the claim you are making, and have not responded to other posters pointing out the flaws in the examples you did provide.

I'm genuinely curious to see an implementation but I suspect you've misunderstood something in the past and your statement was incorrect.

frutigervor 5 Jahren

> They're usually promoted to built-ins

There’s no runtime “promotion”. You can read about it all in bash(1).

MisterTeavor 5 Jahren

Plan 9's rc-httpd is quite useful, supports cgi and serves many plan 9 web sites, many of which run werc http://werc.cat-v.org/

It can also run on Unix via plan 9 port (p9p) or derivatives.

https://github.com/9front/9front-test/blob/master/rc/bin/rc-...

adamddev1vor 5 Jahren

Very cool. Was hoping to see Sean Connery in the README.

pkruminsvor 5 Jahren

Also `php -S` is very handy.

[deleted]vor 5 Jahren

vkazanovvor 5 Jahren

One those things that is s scarily beautiful

montroservor 5 Jahren

Fantastic. For a more sinatra-inspired take, see also https://github.com/guigo2k/ganesh

throwaway984393vor 5 Jahren

They reinvented CGI scripts but didn't actually implement CGI. Could have implicitly supported a lot of apps by default. There's some pretty decent CGI libraries for Bash, too.

Personally I use busybox's httpd (which does support CGI) for a small portable web server.

laumarsvor 5 Jahren

> I didn't want to install and configure Apache or nGinx. In fact, I didn't want any configuration.

If you ignore all of the manual installation instructions and any dev time you took to create this, then sure.

I’m not going to argue that the author should have used something else instead though. It’s a personal project used purely on an internet system of his house. For those kinds it projects “because I could” is as good a reason as any. I just think more authors should be honest and say “I just did it because it seemed fun at the time”

tjoffvor 5 Jahren

Manual instructions and dev time doesn't necessarily pollute the state of the machine though and are harder to replicate, which is how I would interpret it.

laumarsvor 5 Jahren

Not having your application packaged up for quick removal strikes me as more polluting. And I don’t really see how an independent solution with almost no documentation and zero configuration management support (eg Ansible) is easier to replicate than a battle tested industry standard with widespread support.

Don’t get me wrong, I have nothing against building things for fun. But the technical arguments made for why this is preferable over an established solution doesn’t stack up.

tjoffvor 5 Jahren

I do.

Just as I would use this (would look into it more first though), the netcat trick mentioned here or python3 -m http.server over ever considering installing Ansible and/or nginx/apache.

Totally different use case though.

laumarsvor 5 Jahren

If I was rolling my own and concerned about polluting, I’d favour a language like Rust or Go because I then don’t need to install the Python runtime, have random undocumented dependencies for the bash etc.

Or I’d use Docker.

But as said before, there’s no wrong answer here since it’s just a hobby project running on a HTPC.

tjoffvor 5 Jahren

With that I agree, I would only use python if it was already installed, and the assumption with bash was that everything needed was already installed.

montroservor 5 Jahren

For the occasional service where you are essentially just wrapping an existing binary, a bash server behind uwsgi can be totally sensible.

remotevor 5 Jahren

Kudos for scripting a great hack in BASH.

Since the title reads "pure bash" however I was expecting to see an implementation using builtin bash features such as /dev/tcp.

Have you considered solving this problem using BASH builtin support for TCP?

rasculvor 5 Jahren

I recall that bash can't create a server socket, because bash doesn't bind().

IceWreckvor 5 Jahren

If this is pure bash then so is

``` caddy file-server --browse --listen :80 ```

Brian_K_Whitevor 5 Jahren

Ah, no. Bold claim unrealized. Immediately I see many unnecessary forks and externals to do things that bash actually can do itself.

_wlduvor 5 Jahren

Here's my small webserver:

    #!/bin/bash

    while : ; do cat conference.txt | nc -l 80; done

Here's the story behind it:

https://www.go350.com/posts/finding-a-hacked-server/

tessierashpoolvor 5 Jahren

for a more complete solution, check out Bash On Balls, a Rails clone written in bash:

https://github.com/jneen/balls

laputan_machinevor 5 Jahren

I could read stories like this forever, they are educational and engrossing, thanks for sharing

[deleted]vor 5 Jahren

Tepixvor 5 Jahren

I liked the story and the impromptu web server!

Two comments:

- using "ping -a" creates an audible ping so you can find the server if you're by yourself without having to look at the screen.

- No need for cat:

    while :; do nc -l 80 < conference.txt; done

_wlduvor 5 Jahren

Interesting. I don't hear anything when I use the -a flag.

rzzztvor 5 Jahren

Windows Terminal shows a "visual bell" in the tab bar whenever BEL is emitted (even though I have it set to "audible" in advanced settings of the profile, hmm).

Edit: Uhm, right. I have also silenced all system sounds in Windows.

thanatos519vor 5 Jahren

Let's check with "reveal codes":

    $ ping -a 127.0.0.1 | less

    PING 127.0.0.1 (127.0.0.1) 56(84) bytes of data.

    64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.029 ms^G

    64 bytes from 127.0.0.1: icmp_seq=2 ttl=64 time=0.023 ms^G

Looks like you need a terminal that has a BEL sound.

vesinisavor 5 Jahren

It just echoes a Bell character to your terminal. Make sure your terminal emulator supports it, and is configured properly: https://en.wikipedia.org/wiki/Bell_character

dormentovor 5 Jahren

Whoa, nifty.

Why does firefox shows the "chose an application to open this link" when you access the http://localhost link, but shows the content correctly when you access the 127.0.0.1/0.0.0.0 ip-based version?

hericiumvor 5 Jahren

Browser expects to see some HTTP headers before the content. Adding HTTP version with response code, content type declaration and empty line dividing headers and content should do the trick:

    HTTP/1.1 200
    Content-Type: text/plan;

    1st line of content

vincnetasvor 5 Jahren

TIL. thanx for -a.

[deleted]vor 5 Jahren

moondevvor 5 Jahren

go run github.com/patrickhener/goshs@latest

notRobotvor 5 Jahren

See also:

darkttpd: When you need a web server in a hurry.

https://github.com/emikulic/darkhttpd