OpenResty – A fast web app server by extending Nginx(openresty.org)
openresty.org
OpenResty – A fast web app server by extending Nginx
http://openresty.org/
37 comments
"the underlying HttpLuaModule for nginx is awesome on its own if you're using nginx already"
This is the really important bit about OpenResty that sometimes gets overlooked, or buried under the "full-fledged web application server" motto. The "ngx_lua" module by itself basically nullifies the need for a bunch of other modules.
As anyone running nginx for something non-trivial can attest, adding modules can quickly become a support nightmare (varying levels of quality, issues you can't figure out if they're core or module related, keeping everything updated, confusing configurations, etc.).
Right now "ngx_lua" is one of my "must have" modules, together with "headers_more" and "set_misc" (which could also be replaced by "ngx_lua" rather easily, but with a slight impact in configuration readability).
I'd hope the nginx plus folks would rather make this official instead of going out with a competing module using JavaScript. Guess Lua is not hipster enough.
This is the really important bit about OpenResty that sometimes gets overlooked, or buried under the "full-fledged web application server" motto. The "ngx_lua" module by itself basically nullifies the need for a bunch of other modules.
As anyone running nginx for something non-trivial can attest, adding modules can quickly become a support nightmare (varying levels of quality, issues you can't figure out if they're core or module related, keeping everything updated, confusing configurations, etc.).
Right now "ngx_lua" is one of my "must have" modules, together with "headers_more" and "set_misc" (which could also be replaced by "ngx_lua" rather easily, but with a slight impact in configuration readability).
I'd hope the nginx plus folks would rather make this official instead of going out with a competing module using JavaScript. Guess Lua is not hipster enough.
I have been using OpenResty for years now. The ability to run Lua scripts from within is great.
Story time: Some time ago we developed a CMS for a large art/production company. One of the requirements was that everything must be access controlled including all assets (eg. images, videos). The site went live and all was good until one day when, for no apparent reason, our monitoring alreted extreme load and then the site went down. Not so good for a company that sells its tickets online... I got on the phone with them and turns out that they started using the CMS to store images used for newsletters, which they sent out to some 200K people that morning. Now normally this wouldn't be such a big problem, but since all images were ACLd, this meant that instead of serving static files all requests went to the backend and pretty much killed it. The solution we came up with was to move the initial access control check for assets directly to OpenResty using Lua and Drizzle. So whenever a request came in for an image Openresty checked if it's public and only if it was not did the request hit the backend, otherwise it was served directly. Once we pushed this live the load disappeared and never came back.
Also I wrote an init script for it (as we needed one for a DRBD setup). Maybe someone will find it useful, here it is: https://gist.github.com/pwm/d3260804b4ade0d81f29
Story time: Some time ago we developed a CMS for a large art/production company. One of the requirements was that everything must be access controlled including all assets (eg. images, videos). The site went live and all was good until one day when, for no apparent reason, our monitoring alreted extreme load and then the site went down. Not so good for a company that sells its tickets online... I got on the phone with them and turns out that they started using the CMS to store images used for newsletters, which they sent out to some 200K people that morning. Now normally this wouldn't be such a big problem, but since all images were ACLd, this meant that instead of serving static files all requests went to the backend and pretty much killed it. The solution we came up with was to move the initial access control check for assets directly to OpenResty using Lua and Drizzle. So whenever a request came in for an image Openresty checked if it's public and only if it was not did the request hit the backend, otherwise it was served directly. Once we pushed this live the load disappeared and never came back.
Also I wrote an init script for it (as we needed one for a DRBD setup). Maybe someone will find it useful, here it is: https://gist.github.com/pwm/d3260804b4ade0d81f29
How did you do the ACLs for media files? is there any code online for this? Thanks!
The approach we use for large files in our Python app is to set an X-Accel-Redirect header in a dynamic response to a location protected by the 'internal' directive.
That way your dynamic script is only responsible for access control and generating headers.
Apache supports similar functionality by setting a Location header on a 200 response from a CGI or mod_wsgi script.
That way your dynamic script is only responsible for access control and generating headers.
Apache supports similar functionality by setting a Location header on a 200 response from a CGI or mod_wsgi script.
I've had great results using OpenResty in combination with the Lapis[0] framework. Screaming fast and pretty easy to work with.
[0] http://leafo.net/lapis/
[0] http://leafo.net/lapis/
Yeah, it's pretty awesome. If anyone wants to play with it - I've created an simple example to showcase it along with some benchmarks. It has some hardcoded stuff for Postgresql example, which you may want to change.
https://github.com/slava-vishnyakov/lapis-example
It is impressively simple to code:
https://github.com/slava-vishnyakov/lapis-example/blob/maste...
https://github.com/slava-vishnyakov/lapis-example
It is impressively simple to code:
https://github.com/slava-vishnyakov/lapis-example/blob/maste...
I'm really liking the results on my end as well. I currently use this same setup for the web/API side of things for a Freeswitch-based project with a large Lua codebase and the reusability factor is a great plus.
Yep Lapis is a work of art, just like the rest of leafo's work [1].
Lapis and OpenResty (?) are actually used to power the online indie game store run and built by leafo [2].
[1]: http://leafo.net
[2]: https://itch.io
Lapis and OpenResty (?) are actually used to power the online indie game store run and built by leafo [2].
[1]: http://leafo.net
[2]: https://itch.io
I've really enjoyed poking at OpenResty and I found Lapis to be the most mature framework working with it.
However, I find the use of Moonscript for its examples sites and the obvious preference for it in the documentation to be quite maddening. Using moonc to compile things down to Lua doesn't really make tracing through code any better when trying to gain a full understanding of the stack.
I've got very little desire to go out of my way to learn Moonscript since I will not get company approval to work in it and I cannot train my coworkers on it just so they can learn to use Lapis. Already it was a stretch adopting Lua where I am and to be quite honest, I really haven't liked what I saw of Moonscript anyway. I don't want or need classes and its associated machinery, I particularly do not want semantic whitespace.
I really hope to see more focus placed on making Lua proper a first-class citizen for the framework, including provided examples. Otherwise, Lapis does look like the right tool for the job and I liked many of the other decisions made over it.
However, I find the use of Moonscript for its examples sites and the obvious preference for it in the documentation to be quite maddening. Using moonc to compile things down to Lua doesn't really make tracing through code any better when trying to gain a full understanding of the stack.
I've got very little desire to go out of my way to learn Moonscript since I will not get company approval to work in it and I cannot train my coworkers on it just so they can learn to use Lapis. Already it was a stretch adopting Lua where I am and to be quite honest, I really haven't liked what I saw of Moonscript anyway. I don't want or need classes and its associated machinery, I particularly do not want semantic whitespace.
I really hope to see more focus placed on making Lua proper a first-class citizen for the framework, including provided examples. Otherwise, Lapis does look like the right tool for the job and I liked many of the other decisions made over it.
I use Moonscript and like it, but do not use the classes (partly for "leaky abstraction" and debugging reasons, partly performance concerns). I'm a fan of significant whitespace and most of the rest of its syntax sugar. The only part I dislike is the odd choice to use backslashes instead of colons for method calls. (I know leafo explained his reasoning many times in GitHub issues, but I still think there are some better alternatives.)
I also like Coffeescript though. If someone doesn't like Coffeescript, they probably will not like Moonscript either.
I also like Coffeescript though. If someone doesn't like Coffeescript, they probably will not like Moonscript either.
Rewriting the hodge podge of C, Perl and other custom bits of Nginx almost entirely in Lua and hiring its maintainer Yichun in the process where important steps for scaling up the CloudFlare's CDN and make sure each node is running as hot as possible.
More on this from dotScale 2015: https://youtu.be/LA-gNoxSLCE?t=7m40s
More on this from dotScale 2015: https://youtu.be/LA-gNoxSLCE?t=7m40s
Combining a fast, optimized web server with Lua for fast and robust web applications is a technique proven in the field by Barracuda Application Server [1]. It's good to see people trying it for server apps. Thing I like about such a combo is that simpler web stacks are easier to secure and have fewer ways to fail in general.
[1] https://realtimelogic.com/products/barracuda-application-ser...
[1] https://realtimelogic.com/products/barracuda-application-ser...
Most of my nginx use has been to serve static assets and reverse proxy to web apps, so I'm not familiar with a lot of other use-cases here. What are some common problems I can solve with openresty vs what I can already do with nginx?
Adding authentication/login, cacheing, processing or modifying pages on the fly (eg poersonalisation) are all things that people start with in general.
One big downside was that OpenResty doesn't support SPDY, due to an apparent issue when running SPDY and Lua. SPDY was an easy double digit percentage perf gain for us, so it's omission was odd.
I've had normal nginx with all the plugins (using the nginx-extras package, I think, from the ppa). No problem with Lua and SPDY at least in my use case (sending rendered HTML snapshots for anything with an HTML mime type).
I've had normal nginx with all the plugins (using the nginx-extras package, I think, from the ppa). No problem with Lua and SPDY at least in my use case (sending rendered HTML snapshots for anything with an HTML mime type).
isnt spdy deprecated per se ? [1]
[1] http://blog.chromium.org/2015/02/hello-http2-goodbye-spdy-ht...
[1] http://blog.chromium.org/2015/02/hello-http2-goodbye-spdy-ht...
But HTTP 2 isn't on nginx yet, and SPDY still has plenty of browser support so...
I know about OpenResty, but I am getting a blank page? Can't even open Chrome developer tools on it. Very strange. Works fine in Safari and Firefox.
a note: the image serving frontend and storage backend for the original twitter photos integration was written largely in openresty. I love it.
How do I debug a web application built in Lua on OpenResty? Is there some sort of debugger where I can execute code step-by-step or do I have to resort to writing to log files to understand what's going on?
I think all the HN traffic might have crushed the website - I'm getting a blank page and when I refresh it seems to be stuck. Not very encouraging, assuming they're using it to serve their main page.
My default web server! Wrote last week a Discourse SSO implementation just using content_by_lua from OpenResty.
This is the kind of "outside the box" kind of stuff I love. Thanks for sharing.
The navigation on this site is possibly the worst ever conceived. You can't link to anything!
It's also a unique experience for people using NoScript: a whole wall of links and a warning with no content at all. Most sites aren't so broken with NS running.
While I agree that it is slightly jarring to use, there are "Permalink" buttons beneath each section.
Wiki as a homepage is not unusual, TiddlyWiki is an interesting choice...
I wonder if people insisting that they use node.js for scalability and high concurrency (two false but popular reasons) will now reconsider their decision because this blows node.js out of the water.
https://www.techempower.com/benchmarks/#section=data-r10&hw=...
https://www.techempower.com/benchmarks/#section=data-r10&hw=...
> because this blows node.js out of the water.
Using your same link I find this solution blowing node away in JSON serialization but the other benchmarks? Not so much. In fact complete opposite for some of them.
As far as I have found there is no "best at everything" language and framework so your binary question doesn't make much sense. I'm also not convinced of this solution in a large web application because debugging and unit testing would be nightmare .
Using your same link I find this solution blowing node away in JSON serialization but the other benchmarks? Not so much. In fact complete opposite for some of them.
As far as I have found there is no "best at everything" language and framework so your binary question doesn't make much sense. I'm also not convinced of this solution in a large web application because debugging and unit testing would be nightmare .
OpenResty is considerably faster than Node in the JSON serialization, single query, and plaintext benchmarks. It is slower than Node in the multiple queries benchmark, and missing from the other benchmarks.
So, it's a bit more inconclusive than you describe. I'd expect OpenResty to be much faster than Node for a lot of common web interactions. Node has better support for interacting with external services asynchronously, though.
So, it's a bit more inconclusive than you describe. I'd expect OpenResty to be much faster than Node for a lot of common web interactions. Node has better support for interacting with external services asynchronously, though.
This won't have any impact on the node.js community. OpenResty has been around since 2011 and it's had some pretty screaming fast benchmarks for quite awhile, yet it still sees little traction. Lapis is great, Lua is super simple to work with and a total pleasure as well, but it just doesn't seem to be gaining much traction.
There continually seems to be this belief that Lua is language only for game scripting; outside of that it is largely ignored.
There continually seems to be this belief that Lua is language only for game scripting; outside of that it is largely ignored.
For web applications, you're definitely going to get better performance with OpenResty. However, Node has one advantage: you can write any sort of socket server with it. With nginx/OpenResty, you're stuck with HTTP(S) servers.
The creator of OpenResty has expressed interest in eventually extending it to work for non-HTTP usecases though.
The creator of OpenResty has expressed interest in eventually extending it to work for non-HTTP usecases though.
Please don't get me wrong I'll love to use openresty, I love nginx, etc. But there are other reasons to use node: ecosystem and npm.
Try searching a driver for a not so popular db on npm and you might find two different implementations in addition to the official driver.
I'd love to see something like this on openresty.
Try searching a driver for a not so popular db on npm and you might find two different implementations in addition to the official driver.
I'd love to see something like this on openresty.
> ecosystem and npm.
Yeah. There are "Lua rocks" (https://luarocks.org/) for Lua, but they have a few hundreds of packages only. Compared to npm there's no comparison at all.
On the other hand you wouldn't want to use RDBMS with OpenResty anyway, I think. The whole point of using Nginx scripting (especially with LuaJIT, which is awesome) is to be as fast as possible. True, DB connections are async in Resty, but they are still going to take some time. For simple data that don't need to be persistent it's better to use ngx.shared - an in process dictionary shared between all scripts. For more complicated or persistent data, you have Redis and other fast solutions.
Additionally, Nginx offers a full non-blocking socket API to Lua scripts. You can write a "driver" for anything that wants to speak to you over a network.
Yeah. There are "Lua rocks" (https://luarocks.org/) for Lua, but they have a few hundreds of packages only. Compared to npm there's no comparison at all.
On the other hand you wouldn't want to use RDBMS with OpenResty anyway, I think. The whole point of using Nginx scripting (especially with LuaJIT, which is awesome) is to be as fast as possible. True, DB connections are async in Resty, but they are still going to take some time. For simple data that don't need to be persistent it's better to use ngx.shared - an in process dictionary shared between all scripts. For more complicated or persistent data, you have Redis and other fast solutions.
Additionally, Nginx offers a full non-blocking socket API to Lua scripts. You can write a "driver" for anything that wants to speak to you over a network.
* Classifying incoming requests based on various signals to detect DDOS patterns and moving those requests into different request limit zones. It's used in production at a bigger hosting provider.
* Rewriting incoming requests to a certain path of an existing website to a different shared blog hosting provider and changing back HTML/CSS/JS so everything worked. It didn't end up going into production, but it was pretty easy to build in under 100 lines of Lua code.
So when you're bored and want to learn something useful, have a look at http://wiki.nginx.org/HttpLuaModule. It might help you someday.