OAF lends materials and teachers to farmers to make the more productive, by the end of the program they got productive enough to pay back the loan and OAF can lend the same money to someone else.
It's super capillary, with many boots on grounds and quality problems.
Embedded into this there's a good feedback channel: farmers who don't think are getting a good service stop paying back the loan. This allows OAF to go and audit what's failing there.
I do consume the parquets with DuckDB but had to read in firebird sql stuff.
I didn't think of checking, but I now learnt there's an extension for DuckDB but it's C++ and also embeds the same DLL [0] https://github.com/flozer/duckdb-firebird
Recently I was messing around with parquet files in Python and ended up needing to ship the results on Windows, without a Windows machine to test on.
Shipping Python to end users is half mad already, and doing it on Windows is exactly the kind of thing I don't want to spend my life maintaining.
So I figured I'd rewrite it in Go. But that meant embedding a DLL, and how would I test it?
I could spin up a VM, sure. But GitHub Actions already has a Windows environment, and there was my loop: let the agent push to the repo, run tests in GHA, rinse and repeat.
In under an hour it had a full rewrite of my Python, passing every test and producing row-for-row copies of my Parquet output. And it does work on the user machine!
Spotting a loop like that is as satisfying as noticing you can walk your chess opponent into a smothered mate. Truly empowering.
`pylint . --disable=R0801` will work, `pylint profiling/ --enable=duplicate-code` doesn't seem to exit in a reasonable time. So that's likely hitting some pathological case, possibly accidentallyquadratic.tumblr.com material.
This. I've recently used both duckdb and sqlite to power a dashboard for a small restaurant of a family member. It converts all their sales to a very tiny parquet files, daily.
The file fits in memory and can do all sort of computation in the browser itself. The backend is extremely simple, it just loads the JS and serves the parquet files.
It was also trivial to let the owner do their own queries, just give the schema to an LLM and let it use the charting library, no data hallucinations. If they need it in the dashboard they can either use that one or ask me to review that query.
To be honest, given how simple some things became, it's been really fun to work on.
It's hard to predict what parts of shared understanding we have today is going to be scarce tomorrow. And one can't serialize _all_ of that current shared understanding in documents and integration tests.
Lightly edited transcript of Richard Hipp, creator of SQLite, from 28'53":
> Suppose you had a pull request for SQLite. "Hey, I've got this new feature for SQLite. Here's the pull request." When you want me to pull that into the tree, you say, "Oh, it's free."
> No, it's not free. What you're doing is asking me - you've got this cool feature, and you want me to maintain it for you, to document it for you, to test it for you, to maintain it for you for the next twenty-five years. That's not free.
> Linus Torvalds is famous for saying there's free as in beer and free as in speech. But there's another kind of freedom: free as in puppies. "Oh look, I've got a free puppy for you." You see where this is going?
> A pull request is a free puppy. And then you've just got a kennel full of puppies at the end of the day. And you can't just throw them out - you're morally obligated to take care of them for their natural life.
Related to this, the concept of "free as in puppies" from D. Richard Hipp, creator of SQLite:
> Suppose you had a pull request for SQLite. "Hey, I've got this new feature for SQLite. Here's the pull request." When you want me to pull that into the tree, you say, "Oh, it's free."
> No, it's not free. What you're doing is asking me - you've got this cool feature, and you want me to maintain it for you, to document it for you, to test it for you, to maintain it for you for the next twenty-five years. That's not free.
> Linus Torvalds is famous for saying there's free as in beer and free as in speech. But there's another kind of freedom: free as in puppies. "Oh look, I've got a free puppy for you." You see where this is going?
> A pull request is a free puppy. And then you've just got a kennel full of puppies at the end of the day. And you can't just throw them out - you're morally obligated to take care of them for their natural life.