Lapis – A Lua, Moonscript Framework built on OpenResty(leafo.net)

164 pointsby statenjason12 tahun yang lalu46 comments

Lapis – A Lua, Moonscript Framework built on OpenResty

http://leafo.net/lapis/?

47 comments

leafo12 tahun yang lalu

Hello, I create lapis.

I'm actually on the verge of releasing a new huge update. New features include:

* first class lua support

* built in support for a lua templating language: https://github.com/leafo/etlua

* a new postgres driver written in pure lua, should play nicer with luajit and and be available in more stages of nginx request cycle: https://github.com/leafo/pgmoon

* lots of other smaller changes

All of this code is in master I just need to write all the documentation for it, sigh.

The biggest site I've created with lapis is itch.io: http://itch.io it's currently 43,209 lines of code (excluding libraries).

I run it on a really crappy 10 dollar a month vps, memory and cpu usage is tiny. I've handled well over a million requests per day in the past with no issues. The app communicates with a lot of third party payment services over http in the server, because of the non-blocking nature of openresty throughput is not affected when these services are slow (eg. paypal).

Another interesting lapis project is MoonRocks: http://rocks.moonscript.org/ it's a public lua module hosting site. It's opensource so you can check out what a developed lapis application looks like: https://github.com/leafo/moonrocks-site

Thanks for checking out lapis!

legutierr12 tahun yang lalu

We've started using openresty quite a bit in our shop. It's great for making certain things very fast. We looked at lapis, too, but it seems to be more than we need right now. Really, I'm afraid of doing too much complex development using a framework that doesn't provide you with an interactive debugger (like pdb).

Is there something out there that I'm not aware of? The following comment by the creator of openresty seems to indicate that there is not:

https://groups.google.com/forum/#!topic/openresty-en/VSIM8jY...

leafo12 tahun yang lalu

It's not something I've experimented with yet. Lua comes with a very low level debugger: http://www.lua.org/manual/5.1/manual.html#pdf-debug.debug but I haven't tried using it inside of an nginx worker. Because the workers are forked you don't have immediate access to their standard in.

For lapis I did build a server side console for running code in the same environment as the worker though, it might be useful: http://leafo.net/lapis/reference.html#lapis-console

This project looks interesting, maybe it can be used: http://keplerproject.org/remdebug/

paulclinger12 tahun yang lalu

I've been working on extending ZeroBrane Studio (the IDE mentioned in the google groups discussion you referenced) and its debugger and has posted instructions on how to debug OpenResty Lua scripts with it: http://notebook.kulchenko.com/zerobrane/debugging-openresty-....

It's an early work, but you can set breakpoints, step through the code, run ngx commands remotely, inspect stack/variables, and do all the things you'd expect from an interactive debugger. I included a screenshot in the blog post.

I'm interested in getting it to work with moonscript/lapis as well, but ran into issues with mapping source code information between moonscript and lua from the debugger.

pspeter312 tahun yang lalu

With all of these projects, I'm unsure of the relationship between Lua and MoonScript. Is it similar to the relationship between JavaScript and CoffeeScript? Also, does it matter which one is used ultimately?

erikcw12 tahun yang lalu

Yes, that is the relationship exactly. I've used both, but found MoonScript easier to get up and running with since we already use CoffeeScript extensively.

rlander12 tahun yang lalu

Are you sure you're just one person? Seriously, great stuff!

statenjason12 tahun yang lalu

Thanks for the great documentation. It makes lapis stand out from just some weekend project.

egeozcan12 tahun yang lalu

Awesome work!

By the way, does pgmoon support arrays in postgres?

leafo12 tahun yang lalu

I was just experimenting this today, select queries will be able to deserialize array results into lua tables automatically. Same for json values as well.

fasteo12 tahun yang lalu

As a sidenote, openresty (nginx+lua) is probably one of the most underrated technologies we have available nowadays. In my experience, every webapp can offload a ton of work(1) to openresty. The lua code is usually much simpler that the equivalent code in the app.

(1) think caching, auth/acl, security filters, some of the stats, some of the logs, etc

vanessarp12 tahun yang lalu

+1 At apitools.com we use openresty as well. We chose lua (over javascript) because it's super fast, and because it has good sandboxing/ isolation properties which make it safe to handle arbitrary middleware.

erikcw12 tahun yang lalu

We've been using it in production for about 6 months as the endpoint for an analytics system. We went from 5 medium EC2 app servers down to 1 micro instance in testing for our load at the time (we have an HA setup in production). Blazingly fast. Less than 100 LOC for the service.

The author of Lapis is also very responsive to PRs and bug reports.

mojoe12 tahun yang lalu

What framework were you using before that required 5 medium ec2 instances? I'd really appreciate knowing what you were comparing Lapis to.

erikcw12 tahun yang lalu

We use Django behind gunicorn for most of the stack. Not that Django is slow (we still happily use it for 99% of our public facing system) -- it's just that OpenResty is so fast[1].

Besides the fact that OpenResty is built on C/Lua, Nginx's evented architecture makes it scale concurrently really well. We use Gunicorn in a multiprocessing configuration for the rest of the stack -- so each worker consumes much more RAM than the equivalent evented configuration.

We were toying with the idea of implementing the service with Tornado (also evented, but in Python) -- but OpenResty/Lapis has been so great for our use case that it's tough to justify the test at this point.

[1] http://www.techempower.com/benchmarks/

xfalcox12 tahun yang lalu

I just need to say that OpenResty is great, and the mailing list is awesome.

After a topic here in HN saying about doing auth in lua+nginx, we successfully migrated our auth from inside the applications (all java) and now it's a 100 LOC of Lua, easy and decoupled from the application.

justincormack12 tahun yang lalu

Doing auth in OpenResty is a good gateway into using it for more stuff, it is really easy to do whatever auth you need without your application having to know.

srean12 tahun yang lalu

Are there any Lua libraries targeting openresty that makes it easy to use integrate with oauth2 services of Google, facebook, 4square etc ? Since everyone does oauth differently it would be nice to have a go to library for this.

xfalcox12 tahun yang lalu

Yeah, a coworker got excited and did the identicons in Lua too, with openresty generating the png on the first visit of the user. We're a intranet app, in a company with 120k workers, some with intranet only access.

Keats12 tahun yang lalu

Do you have the link for that topic ?

xt12 tahun yang lalu

If someone is eager to try Lapis I have a Dockerfile available here:

https://github.com/torhve/lapis-docker

it includes a readme for getting started with Openresty in general.

_mikz12 tahun yang lalu

For plain openresty you can try https://github.com/3scale/docker-openresty It does not have fancy readme, but has batteries included. Luarocks works, supervisor is set up, cron is there...

avargas12 tahun yang lalu

Lua within Ngnix is amazingly fast. I developed a custom solution for a client last year that basically did bot detection (similar to CloudFlare). It's amazing what you can do within nginx without touching an external fastcgi server or proxy.

thinxer12 tahun yang lalu

I don't understand. How is Lua's performance compared to NodeJS, or compiled languages such as Java or Go ? Does it make sense to stuck everything into a (used to be ) reverse proxy server ?

otikik12 tahun yang lalu

> How is Lua's performance compared to NodeJS, or compiled languages such as Java or Go

This might help: http://www.techempower.com/benchmarks/

> Does it make sense to stuck everything into a (used to be ) reverse proxy server ?

Just like putting stuff into Apache or whatever other webserver you like.

networked12 tahun yang lalu

Is Unicode ever a problem when you do real-life web development in Lua? Lapis looks appealing otherwise but the lack of a built-in Unicode string type in Lua makes me somewhat wary.

plorkyeran12 tahun yang lalu

Not particularly. Lua doesn't do anything to make storing utf-8 in strings difficult. You have to bring your own Unicode library for doing anything nontrivial, but that's true of everything in Lua. Using ICU from Lua isn't substantially different from using it from any other lanugage.

shanth12 tahun yang lalu

And they said, "Lua doesn't come with Batteries included"

justincormack12 tahun yang lalu

There are unicode libraries for Lua should you need to do unicode manipulation. In real life though I rarely have needed much. What functions do you need?

networked12 tahun yang lalu

What I had in mind was Unicode string normalization, comparison and sorting as well as finding the length of strings correctly when they contain combining characters, ligatures and such. What is the library of choice for this?

For anyone curious about the potential issues there is a good article about the flaws in JavaScript's handling of Unicode at http://mathiasbynens.be/notes/javascript-unicode#accounting-.... HN discussion: https://news.ycombinator.com/item?id=7221362.

meric12 tahun yang lalu

Yes, you'll need a unicode library for that. Lua's length operator count bytes not characters.

izietto12 tahun yang lalu

This is great! I'm curious about CPU/RAM usage too

fasteo12 tahun yang lalu

An order of magnitude less than any framework you have in your mind

jbeja12 tahun yang lalu

Is so minimal that is not interesting.

dubcanada12 tahun yang lalu

I've been experimenting with this with a fairly large website. The lack of MySQL support is the only thing keeping me away.

taf212 tahun yang lalu

You migh look at this nginx module wiki.nginx.org/HttpDrizzleModule. It should be possible to access mysql this way with lua as well

reinhardt12 tahun yang lalu

Performance aside, does anyone have experience with how Lapis/OpenResty compare to, say, Django or Rails in terms of stability, included batteries, 3rd party libraries and overall developer productivity and production readiness? I guess this overlaps a lot with the respective question for lua vs python/ruby ecosystems.

shuzchen12 tahun yang lalu

As a person that uses Django to make a living (at my full-time job, and for my freelance work) and who has tinkered with a lapis/openresty site on my spare time (http://graphcake.com), I'd say productivity is on the side of Django. There are too many included batteries (form validation, django ORM more powerful than lapis ORM, staticfiles) and high quality 3rd party libraries (task queue (celery, python-rq), image processing, caching) that I'd hate to reimplement.

That said, performance is totally on the side of lapis/openresty. The performance of my side project totally depends on how fast postgres responds to the needed queries.

xt12 tahun yang lalu

The community for Lua in general, and specifically for Lapis is much smaller.

pspeter312 tahun yang lalu

I was just looking at this yesterday. It seems awesome. My one question is how you do forking dependency graphs since everything is non blocking but looks blocking. Eg. I have takes A, B and C. C depends on A and B so I want to do A and B concurrently but then wait on them for C.

justincormack12 tahun yang lalu

You can use https://github.com/openresty/lua-nginx-module#ngxlocationcap... to do multiple parallel requests

pspeter312 tahun yang lalu

Interesting. Thank you!

fab13n12 tahun yang lalu

I don't know how Lapis does it, but it's not difficult to write a coroutine-cloning function in C for Lua:

http://lua-users.org/lists/lua-l/2006-01/threads.html#00650, cf. thread "coroutine.clone and stateful web applications".

(Coroutines are essentially one-shot continuations; if you can clone them, you can bookmark execution states at will. Look at Lua's upvalue semantics to anticipate what will happen to variables shared by multiple coroutine clones).

martijn_himself12 tahun yang lalu

Absolutely amazing work. I have been meaning to give this a go for ages.

[deleted]12 tahun yang lalu

frik12 tahun yang lalu

Interesting, and I found it in the techempower benchmark too:

http://www.techempower.com/benchmarks/#section=data-r9&hw=i7...

pspeter312 tahun yang lalu

I don't get how it does so badly in the queries and writes section when compared to node.js while the serving plaintext and json is so high.