adsharma

hace 9 díasdiscuss

LadybugDB was forked from KuzuDB 8 months ago. Find out what's new and how it differs from KuzuDB in performance, features and market positioning.

adsharma

hace 12 díasdiscuss

A more appropriate mirror test for LLMs is to get them to state facts about their training data. Percentage of arts vs science for example.

Given the framing that they're similar to nukes and a national security issue, it's likely that the models are post trained to not answer such questions accurately.

Also the article could be trying to normalize thinking that these are more than matrix multiplication gadgets good at compression.

adsharma

hace 22 díasdiscuss

It's interesting that the blog post places SQLite and Ladybird on the spectrum, but omits it's chief open source rival: DuckDB.

Agree that Level 3 is what inspires confidence. But we need to invent new business models to sustain in the era of vibe-coded databases.

adsharma

el mes pasadodiscuss

DuckDB and LadybugDB use the same terminology to describe internals.

adsharma

el mes pasadodiscuss

The future is to write Rust in a python interpreter compatible way leveraging large parts of the python ecosystem.

This will require minor tweaks to the language and perhaps a forked interpreter (a forked parser already exists).

Then you solve the two language problem with a combination of probabilistic and deterministic translation.

adsharma

el mes pasadodiscuss

7 years of python -> rust here.

https://github.com/py2many/static-python-skill/blob/main/ass...

adsharma

hace 2 mesesdiscuss

Installing thing like this should be an agent skill, not a distribution.

adsharma

hace 2 mesesdiscuss

+1 - I was making exactly this argument in other threads. But I have a slightly different take on how software should be written.

Translating zig -> rust is more complex than writing a JPEG parser in static python and then lowering it into zig and rust differently using idiomatic construct for each language.

Towards that end, I've created a parser for a dialect of python which is suitable for this purpose. It should maintain compatibility with the vast majority of python code out there, while picking up some rust/zig features that make translation easier. JPEG parser included in the assets of the skill for a flavor.

https://github.com/py2many/static-python-skill

https://github.com/py2many/spy-ast

adsharma

hace 2 mesesdiscuss

A SMT solver may work better.

adsharma

hace 3 mesesdiscuss

Experimenting with the pglite way, this time using vlang here:

https://github.com/adsharma/vpg

  v run cmd/vpg_test.v

adsharma

hace 3 mesesdiscuss

The first one. It's forked from pgserver.

Yes, what you get is a multi-server postgres under the covers. But for many users, the convenience of "uv pip install...", auto clean up via context manager is the higher order bit that takes them to SQLite.

Of course the bundled extensions are the key differentiator.

With enough distribution and market opportunities we (yes, there is a for profit company behind it) can invest in making it truly embedded.

adsharma

hace 3 mesesdiscuss

Pgserver is not maintained. Had to fork it as pgembed, compile recent versions and bundle BM25 and vector extensions.

adsharma

hace 3 mesesdiscuss

Pgembed is pglite for native code.

adsharma

hace 3 mesesdiscuss

This is why transpilers exist.

py2many can compile static python to mojo, apart from rust and golang.

Is it comprehensive? No. But it's deterministic. In the age of LLMs, with sufficient GPU you can either:

  * Get the LLM to enhance the transpiler to cover more of the language/stdlib
  * Accept the non-determinism for the hard cases

The way mojo solves it is by stuffing two languages into one. There are two ways to write a function for example.

I don't think the cost imposed by a transpiler is worse. In fact, it gets better over time. As the transpiler improves, you stop thinking about the generated code.

adsharma

hace 3 mesesdiscuss

There is a question of what benefit would it bring even if its open sourced?

Static python can transpile to mojo. I haven't seen an argument on what concepts can only be expressed in mojo and not static python?

Borrow checker? For sure. But I'm not convinced most people need it.

Mojo therefore is a great intermediate programming language to transpile to. Same level of abstraction as golang and rust.

adsharma

hace 3 mesesdiscuss

Amazing people still keep discovering it. And google search fails to surface working implementations.

"Python to rust transpiler" -> pyrs (py2many is a successor) "Python to go transpiler" -> pytago

Grumpy was written around a time when people thought golang would replace python. Google stopped supporting it a decade ago.

Even the 2022 project by a high school student got more SEO

https://github.com/py2many/py2many/issues/518

adsharma

hace 3 mesesdiscuss

Cython uses C-API. This one doesn't.

adsharma

hace 3 mesesdiscuss

Static python as described in this skill.

https://github.com/py2many/static-python-skill

adsharma

hace 3 mesesdiscuss

Do you have LongMemEval numbers for pgvector vs pgvector+ hybrid search?

adsharma

hace 4 mesesdiscuss

Thank you for the shout out! I looked into your benchmark setup a bit. Two things going on:

- Ladybug by default allocates 80% of the physical memory to the buffer pool. You can limit it. This wasn't the main reason.

- Much of the RSS is in ladybug native memory connected to the python connection object. I noticed that you keep the connection open between benchmark runs. For whatever reason, python is not able to garbage collect the memory.

We ran into similar lifetime issues with golang and nodejs bindings as well. Many race conditions where the garbage collector releases memory while another thread still has a reference to native memory. We now require that the connection be closed for the memory to be released.

  https://github.com/LadybugDB/ladybug/issues/320
  https://github.com/LadybugDB/go-ladybug/issues/7
  https://github.com/LadybugDB/ladybug-nodejs/pull/1

Submissions

LadybugDB Flying Solo

Show HN: Vpg – Pglite for Native Code

comments