hugomg · HackerTrans

anno scorsodiscuss

We figured that it wasn't worth dealing with the hassle of unaligned addresses because the more portable alternatives worked just as well.

hugomg

2 anni fadiscuss

tl;dr: For Pallene, Lua is a scripting language. For Terra, Lua is mainly for metaprogramming.

The key difference is that Terra cannot directly manipulate Lua data. Although Terra's syntax is similar to Lua, its type systems and data structures are actually much closer to C. If we want to pass Lua table into Terra, we must first convert it into Terra arrays/structs. This cost adds up if the code uses many Lua data structures; in the worst case it can be worse than just using Lua for everything.

Passing data between Lua and Pallene should incur no cost. If the programmer rewrites a piece of code from Lua to Pallene, it should hopefully make things faster, and never slow it down.

Terra is more focused on metaprogramming. Generating code and executing things at compile-time. Think of C++ templates, but better.

hugomg

2 anni fadiscuss

The difference is the Lua-C API. The default Lua-C API is designed for humans: it is stable and safe to use, but every operation must pay the cost of function calls and passing data through the Lua stack. Pallene bypasses the API and reaches into the Lua tables directly. This is much faster, but would be impractical without the Pallene compiler. The internal struct layouts are unstable, and unsafe if you're not very careful.

hugomg

2 anni fadiscuss

> I can just as good implement it in C using the Lua C API

Pallene beats C when the code uses many Lua data structures. Acessing Lua data from C via the Lua-C API has significant overhead that can erase the gains from rewriting into C. Also, rewriting from Lua to Pallene is much less work than rewriting it in C.

Although Pallene is only a subset of Lua, the idea is that you use it together with Lua. It's not meant to replace Lua entirely.

hugomg

2 anni fadiscuss

(I'm one of the Pallene authors)

For compiled code, Pallene's performance can be comparable to LuaJIT[1]. Pallene might be better in code with unpredictable branches, which don't fit in a single trace. LuaJIT has the edge in purely-interpreted code, and can inline indirect or cross-module function calls. LuaJIT is also more featureful, and has a better FFI story at the moment.

[1] https://www.inf.puc-rio.br/~hgualandi/papers/Gualandi-2020-S...

LuaJIT is really good software, it is hard to beat it as its own game. Pallene tries to get its edge elsewhere: it tracks Lua 5.4, whereas LuaJIT diverged around 5.1 and will stay that way forever. Pallene's implementation is arguably more portable, because it compiles down to C and doesn't need hand-crafted assembly.

hugomg

5 anni fadiscuss

Pallene and Teal are in a bit of a different space. Teal is focused on compile-time type checking. The intention is to encourage you to add type annotations to your entire Lua codebase. It's another take at the problem that Typed Lua was trying to solve. A Typed Lua 2.0, if you want to call it that way.

Pallene, on the other hand, is more focused on performance. The tradeoff is that it doesn't cover the entirety of the Lua language, just the features that we can generate fast code for. The intention is that you can write the performance-critical parts in Pallene and glue everything together using Lua (or Teal).

That said, I certainly wouldn't be surprised if we saw more cross pollination of ideas between these two languages in the future :)

hugomg

6 anni fadiscuss

It wouldn't be the JS we know and love if it had been burdened with a type system designed by a committee sometime in the 90s. That said, one thing we can say for sure is that the dynamic typing doesn't make your job any easier :)

hugomg

6 anni fadiscuss

I always love reading more about JavaScriptCore internals although I have to confess that much of the time one of the main lessons I get from it is that life would be much easier if we had types and didn't need to speculate so much in the first place.

hugomg

6 anni fadiscuss

If you are curious, this is what the resulting format string looks like after expanding all the macros:

https://gist.github.com/hugomg/73e91ebca7ced3c5c6f955e9e830b...

hugomg

7 anni fadiscuss

We chose the Pallene name because it was another one of Saturn's moon names that sounded nice, and is pronounced the same way in English and Portuguese. The Mike Pall thing was just a coincidence although we certainly can't deny his influence, as you have said :)

hugomg

7 anni fadiscuss

It is no surprise that the benchmarks on the LuaJIT website are going to show it ahead of PUC-Lua, is it? In truth the performance comparison is a bit more complicated than that. :)

For a pure Lua comparison LuaJIT is typically going to beat Lua by a large margin. For code that can be JIT compiled you might see ~10x performance improvements and for code that doesn't you might still see around a ~2x improvement because the LuaJIT interpreter is written in hand-crafted assembly language (PUC-Lua sticks to portable and standards-compliant C90).

But things get a bit more complicated if you add foreign C code into the mix. Under LuaJIT, the JIT compiled code that uses the LuaJIT FFI interface is the fastest, but interpreted code using the FFI is slower than interpreted code using the traditional Lua-C interface, and can even be slower than PUC-Lua using the traditional Lua-C interface. This means that when you are using LuaJIT you need to be very careful to make sure that your code is the sort of code that can be JIT compiled. If you hit a "not yet implemented" feature you can have a big performance degradation.

------

Nevertheless, Lua 5.4 has brought significant performance improvements compared to Lua 5.3, specially in integer operations (including for loops). Another big change was that the `#` is now faster. For example, on my machine the following loop runs in 1.96 seconds in Lua 5.3, in 1.26 seconds under LuaJIT, and in 0.96 seconds in Lua 5.4

    local a = {}
    for i = 1, 1e7 do
        a[#a +1] = i
    end
    print(#a)

The new generational garbage collector is also a very big improvement. I've measured a 1.5x speedup on some GC-dominated workloads, where the final result was that Lua 5.4 would even edge out LuaJIT, which is still using the old incremental collector.

hugomg

7 anni fadiscuss

In case others aren't familiar with the termiology: #SAT is the counting version of the SAT problem, which tries to find how many different solutions exist to a set of constraints)

hugomg

7 anni fadiscuss

Some SAT solvers allow you to add additional constraints while they are running. So you can find a solution and then add additional constraints to try to get it to find a better one.

hugomg

7 anni fadiscuss

Generally speaking, if you ever find yourself coding a brute-force backtracking search for some combinatorical problem, it might be a good idea to consider using a SAT solver instead. This way you get to take advantage of the non-chronological backtracking, and all the other tricks the SAT solvers implement.

I wrote the hexiom solver that got linked above and the motivation in that case was that I saw that someone tried to solve the puzzle with a hand-written backtracking search, but that their algorithm wasn't able to find a solution to one of the levels.

My version of the SAT solver managed to quickly find a solution for every case, including the highly symmetric one where the usual search algorithm got stuck. The nicest thing is that the implementation was very declarative, in that my job was to produce a set of constraints for the SAT solver, instead of to produce an imperative algorithm. The symmetry breaking was one example of something hat was very easy to do in a SAT-solver setting but which would be trickier to do by hand.

hugomg

7 anni fadiscuss

Seeing someone cite a little project I did for fun many years ago is a very nice surprise. :)

hugomg

8 anni fadiscuss

Yes.

hugomg

8 anni fadiscuss

Currently there is no special support for metaprogramming, but maybe we could borrow some ideas from Terra in a future version...

hugomg

8 anni fadiscuss

Wow, I hadn't thought of that. Its a complete coincidence, actually!

hugomg

8 anni fadiscuss

From the point of view of Lua, Pallene works the same as C

If you want to produce a standalone executable, at the end of the day this executable will contain a copy of the Lua VM.

If you want to produce a library that can be "require"-ed from Lua, then you will produce a ".so" shared object file with just the Pallene functions in it.

hugomg

8 anni fadiscuss

If you were writing naive code, it probably wouldn't be that much. Maybe a 2to 3 times difference. But the thing with Terra is that you can be even faster than naively-written C if you know what you are doing.

comments