Beyond the SQLite single-writer limitation with concurrent writes(turso.tech)
turso.tech
Beyond the SQLite single-writer limitation with concurrent writes
https://turso.tech/blog/beyond-the-single-writer-limitation-with-tursos-concurrent-writes
74 comments
> You can shard your SQLite tables into multiple database files and query across all of them from a single connection.
You mean using ATTACH statement, right? If you use WAL mode, then you cannot get transaction safety / ACID with ATTACH [0]
> If the main database is ":memory:" or if the journal_mode is WAL, then transactions continue to be atomic within each individual database file. But if the host computer crashes in the middle of a COMMIT where two or more database files are updated, some of those files might get the changes where others might not.
Moreover, ATTACH do not support more than 125 databases, so that limits the shards to 125. [1]
ATTACH does not solve the concurrency problems. That's why SQLite also has a BEGIN CONCURRENT experimental branch
[0] - https://www.sqlite.org/lang_attach.html
[1] - https://www.sqlite.org/limits.html
You mean using ATTACH statement, right? If you use WAL mode, then you cannot get transaction safety / ACID with ATTACH [0]
> If the main database is ":memory:" or if the journal_mode is WAL, then transactions continue to be atomic within each individual database file. But if the host computer crashes in the middle of a COMMIT where two or more database files are updated, some of those files might get the changes where others might not.
Moreover, ATTACH do not support more than 125 databases, so that limits the shards to 125. [1]
ATTACH does not solve the concurrency problems. That's why SQLite also has a BEGIN CONCURRENT experimental branch
[0] - https://www.sqlite.org/lang_attach.html
[1] - https://www.sqlite.org/limits.html
I would argue that BEGIN CONCURRENT doesn't solve the concurrency problems either, which is why SQLite also has an HCTree experimental branch :P
Joins and Transactions are a pretty big part of SQL. I'm no expert, but if my quick search results are right, both are lost in the separate file per table scenario.
They work just fine!
Joins: https://simonwillison.net/2021/Feb/21/cross-database-queries...
Transactions: https://www.sqlite.org/atomiccommit.html#_multi_file_commit
Joins: https://simonwillison.net/2021/Feb/21/cross-database-queries...
Transactions: https://www.sqlite.org/atomiccommit.html#_multi_file_commit
But then you need the write lock for all the databases (assuming your transaction involves writes).
Unfortunately if you use WAL mode the transactions are only isolated per database file.
https://www.sqlite.org/lang_attach.html
https://www.sqlite.org/lang_attach.html
Don't forget that the SQLite team is working on their own multi-writer mode that blows BEGIN CONCURRENT' out of the water: https://news.ycombinator.com/item?id=34434025
Though this stuff moves slowly (that announcement was almost 3 years ago!), so I'm glad to see Turso giving us options today.
Though this stuff moves slowly (that announcement was almost 3 years ago!), so I'm glad to see Turso giving us options today.
Looks like that branch is still under actively development - seven new commits this month: https://sqlite.org/hctree/timeline
[deleted](1)
Kind of cool to see work on this. I do hope that the final db file result is still binary compatible with SQLite 3 in whatever direction Turso moves towards though... Rust or not.
I've been advocating with several projects over recent years to get SQLite3 as an archive/export/interchange format for data. Need to archive 2019 data from the database, dump it into a SQLite db with roughly the same schema... Need to pass multiple CSVs worth of data dumps, use a SQLite file instead.
As a secondary, I wonder if it's possible to actively use a SQLite interface against a database file on S3, assuming a single server/instance is the actual active connection.
I've been advocating with several projects over recent years to get SQLite3 as an archive/export/interchange format for data. Need to archive 2019 data from the database, dump it into a SQLite db with roughly the same schema... Need to pass multiple CSVs worth of data dumps, use a SQLite file instead.
As a secondary, I wonder if it's possible to actively use a SQLite interface against a database file on S3, assuming a single server/instance is the actual active connection.
SQLite directly against S3 is workable if you mean querying a read-only database.
For example, from Go, you could use my driver, and point it to a database file stored in S3 using this: https://pkg.go.dev/github.com/ncruces/go-sqlite3/vfs/readerv...
For read-write it's a terrible idea. Object storage assumes objects are immutable. There may be some support for appends, but modifying the middle of an object in place involves copying the entire thing.
What is on the verge of becoming viable is to use Litestream to do asynchronous replication to S3, and have read replicas that stream the data directly from S3. But what's stored in S3 isn't a database file, but a format created for the purpose called LTX.
For example, from Go, you could use my driver, and point it to a database file stored in S3 using this: https://pkg.go.dev/github.com/ncruces/go-sqlite3/vfs/readerv...
For read-write it's a terrible idea. Object storage assumes objects are immutable. There may be some support for appends, but modifying the middle of an object in place involves copying the entire thing.
What is on the verge of becoming viable is to use Litestream to do asynchronous replication to S3, and have read replicas that stream the data directly from S3. But what's stored in S3 isn't a database file, but a format created for the purpose called LTX.
ncruces helped me with some code I made for VFS. It uses [zstd seekable](https://github.com/jtarchie/sqlitezstd) for reading a file. I thought it would be really well-suited for S3.
- Support for HTTP range queries - "Fast" read times - No disk required
I was wrong.
It turns out that for specific SQL queries, it might be fine, but not fast. For queries that do aggregations, like `COUNT`, sqlite loads the whole database anyway.
- Support for HTTP range queries - "Fast" read times - No disk required
I was wrong.
It turns out that for specific SQL queries, it might be fine, but not fast. For queries that do aggregations, like `COUNT`, sqlite loads the whole database anyway.
SQLite against S3 can work with some clever tricks. The neatest version of that I've seen is still this WebAssembly one: https://phiresky.github.io/blog/2021/hosting-sqlite-database...
I also got sqlite-s3vfs working from Python a few months ago: https://simonwillison.net/2025/Feb/7/sqlite-s3vfs/
Both of these are very much read-only mechanisms though.
I also got sqlite-s3vfs working from Python a few months ago: https://simonwillison.net/2025/Feb/7/sqlite-s3vfs/
Both of these are very much read-only mechanisms though.
simon, you will be pleased to know that the python package for turso is in good shape! (and if you do find any issues please scream at us)
> As a secondary, I wonder if it's possible to actively use a SQLite interface against a database file on S3, assuming a single server/instance is the actual active connection.
You could achieve this today using one of the many adapters that turn S3 into a file system, without needing to wait for any SQLite buy in.
You could achieve this today using one of the many adapters that turn S3 into a file system, without needing to wait for any SQLite buy in.
Very excellent to see the embedded DB in the browser link at the bottom. https://shell.turso.tech/
I've been thinking Turso in the browser would be some eventual addition, is something the Rust rewrite would eventually unlock. And that would require considerable extra backing to make go. But here it is now!
I've been thinking Turso in the browser would be some eventual addition, is something the Rust rewrite would eventually unlock. And that would require considerable extra backing to make go. But here it is now!
It's very easy to do given the async model. It is one of the new features that is working the best so far.
Even in the “low-performing” zone of the single threaded SQLite and Turso, we are still talking about 50k rows per second and 1000 microseconds, aka 1 ms. It is insanely fast. 1ms is just the round-trip for my Postgres on RDS. It is amazing that SQLite is so awesome. I understand it is not for every use-case, but still awesome
How does the mvcc / row versioning changes that accumulate get shared among multiple processes? Is it another file like the WAL file? Or is it just local memory as part of the client? I guess with everything having to commit or entirely roll back at the end, it doesn't need to be shared.
How does this compare with hctree then? It sounds similar, both using an MVCC model.
Really cool though, love the technical details and would be interested to see further. I feel like there is certainly a space for sqlite+fancier features. Though it would break backwards compatibility, I've always wondered what kind of performance gains could be had by using a different varint implementation. And enforcing or always using "strict" for smarter column layouts.
Also would be really cool to have indexed views.
And columnstore seems like a logical direction as well.
How does this compare with hctree then? It sounds similar, both using an MVCC model.
Really cool though, love the technical details and would be interested to see further. I feel like there is certainly a space for sqlite+fancier features. Though it would break backwards compatibility, I've always wondered what kind of performance gains could be had by using a different varint implementation. And enforcing or always using "strict" for smarter column layouts.
Also would be really cool to have indexed views.
And columnstore seems like a logical direction as well.
Can someone explain what "ecological niche" this new Turso DB occupies in between SQLite and Postgres?
TursoDB aims to be fully compatible with sqlite, so files you create with tursodb can be read by sqlite and vice-versa. Like sqlite, turso is in-process, so it runs alongside your application, which is quite different from postgres' client-server architecture.
The only think turso has in common with postgres is mvcc, which is a rather standard concurrency control model across modern databases. Idk if I answered your question :)
The only think turso has in common with postgres is mvcc, which is a rather standard concurrency control model across modern databases. Idk if I answered your question :)
Can't use it locally (yet?) but it's definitely an interesting move in the space. For my personal projects lately I've been defaulting to sqlite in dev, and having a database wrapper layer to use something else in prod.
> Can't use it locally
I'm imagining some insane replication behind the scenes, where every write is happening concurrently on a different SQLite DB, and then merged together sequentially into some master DB.
I'm imagining some insane replication behind the scenes, where every write is happening concurrently on a different SQLite DB, and then merged together sequentially into some master DB.
why not? Turso is a fully local database, you can just download the shell and use it as you would use sqlite.
More specifically, I didn't see any commits about multiple simultaneous writers having landed in main and it's not document as a feature in the release notes https://github.com/tursodatabase/turso/commits
neither the prerelease
https://github.com/tursodatabase/turso/releases/tag/v0.3.0-p...
nor their latest release.
https://github.com/tursodatabase/turso/releases/tag/v0.2.2
The core/mvcc directory does have commits though, so maybe you can.
https://github.com/tursodatabase/turso/commits/main/core/mvc...
neither the prerelease
https://github.com/tursodatabase/turso/releases/tag/v0.3.0-p...
nor their latest release.
https://github.com/tursodatabase/turso/releases/tag/v0.2.2
The core/mvcc directory does have commits though, so maybe you can.
https://github.com/tursodatabase/turso/commits/main/core/mvc...
You just have to pass the experimental flag. I thought this was explained in the blog ?
I searched https://turso.tech/blog/beyond-the-single-writer-limitation-... with ctrl-f for "download" and "--exp" and got no results for either. There was a link to a live demo webpage at the very end. Am I looking at the right blog? That's the one linked at the top of the page.
The flag is --experimental-mvcc, it is in the help for the shell.
You are right that the blog failed to mention. I will get this fixed soon!! Thanks for noticing it.
There is a lot of work that still needs to be done to make this production-ready, both from a performance and reliability point of view, as we did our best to convey on the blog. We truly appreciate you trying it out! Report any issues, please.
Ah there it is! Thanks, I'll play around with it tonight.
While I understand that you could do this, I genuinely don't understand why.
SQLite and Postgres/MySQL/etc. occupy different niches. If you need massive concurrent writes, surely that's what Postgres/MySQL/etc. is for? Their engines are built around that from the ground up.
SQLite is built around a file that stores data for a single application, as opposed to being client-server with many clients. I've used it a ton for that, but the idea that your application would have so many threads needing to write concurrently that it would be slowed down by a file lock... just doesn't make sense to me.
What applications need this, and why wouldn't they use Postgres/MySQL/etc. if they need such a high level of concurrent performance? This feels like trying to adapt a small sports car to tow a semi-trailer. It doesn't seem like it's what it's meant for.
SQLite and Postgres/MySQL/etc. occupy different niches. If you need massive concurrent writes, surely that's what Postgres/MySQL/etc. is for? Their engines are built around that from the ground up.
SQLite is built around a file that stores data for a single application, as opposed to being client-server with many clients. I've used it a ton for that, but the idea that your application would have so many threads needing to write concurrently that it would be slowed down by a file lock... just doesn't make sense to me.
What applications need this, and why wouldn't they use Postgres/MySQL/etc. if they need such a high level of concurrent performance? This feels like trying to adapt a small sports car to tow a semi-trailer. It doesn't seem like it's what it's meant for.
sqlite is embedded. I understand that there might be scenarios in which multi-threaded sqlite is beneficial when an application has many concurrent writers. But taking a look at the company's website makes me wonder what the project's motivation is. The company offers a database service, which is a completely different scenario from embedded dbs. If the intention is to offer a cloud service, evolving from "sqlite" seems odd. The only benefit I can think of is that the new db service helps existing sqlite users migrate. My issue is that if I choose sqlite to store data locally in my browser or my cell phone, why do I suddenly want to store it in the cloud?
> My issue is that if I choose sqlite to store data locally in my browser or my cell phone, why do I suddenly want to store it in the cloud?
I don't think those solutions are necessarily that bad. There was one the other week that offered a means of guaranteed sync to a sqlite file in the cloud. Its nice to have the infra to allow users to hop around devices and have backups. What's weird to me, is trying to magic it into a performant multi-client db which the underlying technology was never designed to be.
I don't think those solutions are necessarily that bad. There was one the other week that offered a means of guaranteed sync to a sqlite file in the cloud. Its nice to have the infra to allow users to hop around devices and have backups. What's weird to me, is trying to magic it into a performant multi-client db which the underlying technology was never designed to be.
I'll give you an example of what I'm doing with SQLite.
I have an app that organizes my movie collection. I use IMDB public datasets as data source. Because versioning is hard, data changes over time (remember who directed Matrix?) it's just easier to rebuild the database every time.
A naive import of datasets is ~131M inserts. It takes 10 minutes (of which 100M takes exactly 6 minutes). Another 10 minutes to further clean up the data, set up indexes, optimize and vacuum.
It's fine for the use case, but still it would be nicer to be able to achieve that in half the time.
The app has no dependencies, database is a single file I can store on S3 and with a little bit of magic use it as a real database - meaning I can run a complex queries over http on that whole database and exchange 50kb.
The client is a standalone HTML file with some JS. No need of server for client, no need for hosting for server except of that S3 bucket.
Sure, it's definitely a niche application, but, at least for me, it makes perfect sense. Start your app with zero deps and add only what you really need. There's too much projects that start with google-esque architecture, with plenty of microservices, only to reach MVP with technical debt the size of US treasury.
I have an app that organizes my movie collection. I use IMDB public datasets as data source. Because versioning is hard, data changes over time (remember who directed Matrix?) it's just easier to rebuild the database every time.
A naive import of datasets is ~131M inserts. It takes 10 minutes (of which 100M takes exactly 6 minutes). Another 10 minutes to further clean up the data, set up indexes, optimize and vacuum.
It's fine for the use case, but still it would be nicer to be able to achieve that in half the time.
The app has no dependencies, database is a single file I can store on S3 and with a little bit of magic use it as a real database - meaning I can run a complex queries over http on that whole database and exchange 50kb.
The client is a standalone HTML file with some JS. No need of server for client, no need for hosting for server except of that S3 bucket.
Sure, it's definitely a niche application, but, at least for me, it makes perfect sense. Start your app with zero deps and add only what you really need. There's too much projects that start with google-esque architecture, with plenty of microservices, only to reach MVP with technical debt the size of US treasury.
I don't understand what you are trying to say here, this doesn't seem to have anything to do with single writer limitations.
That's not concurrent writes. You would see zero speedup.
Your use case makes perfect sense for SQLite. It's a great example of what it's for. It won't benefit. That's my point.
Your use case makes perfect sense for SQLite. It's a great example of what it's for. It won't benefit. That's my point.
Why not take something that’s great and make it support more use cases? People are doing it the other way around too: https://pglite.dev/
IMO this project kneecapped itself by being web-focused.
Perhaps they will support native builds some day?
Agreed, Sqlite docs specifically state that its not designed for this, or rather that the solution isn't appropriate, that database as a service is appropriate when you want multiple clients writing.
The entire "problem" is a side-effect of using the wrong tool for the job.
The entire "problem" is a side-effect of using the wrong tool for the job.
Linux was designed to run in home PCs, and we keep running it in supercomputers. It works just fine. Tools evolve.
Tools don't always need to evolve though. I don't want my hammer to evolve into a screwdriver too, or vice-versa.
Having separate tools for separate things makes sense when the things are different enough.
Having separate tools for separate things makes sense when the things are different enough.
Yeah but this tool is evolving. You don't get a say on that, this is open source and devs do whatever they want with their time
But I do get to ask why. And that's what I'm asking.
"Tools evolve" isn't a use case. So what's the use case where you need concurrent writes for a database that isn't client-server?
"Tools evolve" isn't a use case. So what's the use case where you need concurrent writes for a database that isn't client-server?
its a file on disk, it relies on the file handling of the OS and an OS file system doesn't give you the same guarantees as a fully fledged db. Even if you try to fix this problem you're just going to end up with trade offs which to properly mitigate you'll end up approaching running some sort of service anyway. At some point you have to ask yourself if jumping through all these hoops has saved you more effort than just running a proper service.
You can justify it as a migration strategy between a file and a service but you're just hammering in screws if you try to force Sqlite to be a multi-client database.
> If there are many client programs sending SQL to the same database over a network, then use a client/server database engine instead of SQLite[1].
[1] https://www.sqlite.org/whentouse.html
You can justify it as a migration strategy between a file and a service but you're just hammering in screws if you try to force Sqlite to be a multi-client database.
> If there are many client programs sending SQL to the same database over a network, then use a client/server database engine instead of SQLite[1].
[1] https://www.sqlite.org/whentouse.html
> SQLite is built around a file that stores data for a single application
A single application can need multiple concurrent writes
A single application can need multiple concurrent writes
So that's what asking. When? Why? I've simply never encountered that, and can't imagine what a use case would be.
SQLite already supports lots of threads writing, they just all take turns. What application needs those writes to be concurrent?
SQLite already supports lots of threads writing, they just all take turns. What application needs those writes to be concurrent?
I have such application. It scans hosts for gathering some information in several parallel threads. Then each thread stores part of this information in Sqlite database.
I’m not in a rush to use a reimplementation of SQLite — particularly from startup bros that had a very public, one-sided, and unpleasant fight with SQLite over their contribution model.
D. Richard Hipp is a genuinely fantastic human being, and SQLite is a project developed at literally the level of planetary infrastructure given how broadly and everywhere it appears.
Forking his project while using the name to garner recognition and attention is poor form, and it is difficult to have faith in the results.
D. Richard Hipp is a genuinely fantastic human being, and SQLite is a project developed at literally the level of planetary infrastructure given how broadly and everywhere it appears.
Forking his project while using the name to garner recognition and attention is poor form, and it is difficult to have faith in the results.
> Forking his project while using the name
The don't use the name. They use Turso. Even the HN title is wrong - the article title doesn't mention SQLite.
They refer to SQLite, but how could you not if that's what you forked from, and that's what has the functionality you're changing. That would be a very weird article if we didn't have that context.
The don't use the name. They use Turso. Even the HN title is wrong - the article title doesn't mention SQLite.
They refer to SQLite, but how could you not if that's what you forked from, and that's what has the functionality you're changing. That would be a very weird article if we didn't have that context.
They state, plainly on their home page:
“The next evolution of SQLite”
That is a material misrepresentation and absolutely trading on the SQLite name
“The next evolution of SQLite”
That is a material misrepresentation and absolutely trading on the SQLite name
“The next evolution of SQLite”
How? The whole point of trademark is to avoid confusing users that an alternate product is the same as the original product.
By explicitly saying "Next evolution of SQLite", or "A fork of SQLite", or a "Better SQLite" .. all of this is saying our product is distinct and different from SQLite.
If the fork were called "nue-sqlite" or "sqlitest" or "fastsqlite", there's an argument to be made.
How? The whole point of trademark is to avoid confusing users that an alternate product is the same as the original product.
By explicitly saying "Next evolution of SQLite", or "A fork of SQLite", or a "Better SQLite" .. all of this is saying our product is distinct and different from SQLite.
If the fork were called "nue-sqlite" or "sqlitest" or "fastsqlite", there's an argument to be made.
The issue isn’t that you’re mentioning SQLite or acknowledging that it’s a fork.
The problem is that the phrase “the next evolution of SQLite” conveys continuity and endorsement.
A reasonable reader could conclude that Turso is an official successor or the next release of SQLite — that it represents the official lineage of the project.
Phrasing like “SQLite-compatible,” or “a fork of SQLite” would be clear and factual.
Calling it “the next evolution of SQLite” isn’t factual; it’s marketing positioning, and it implies ownership of SQLite’s identity and lineage.
This reflects a broader pattern in how the fork has been presented publicly.
The messaging often treats the original project as something to be fixed rather than a foundation to be respected.
Referring to Turso as a product while leveraging the SQLite name reinforces that framing — co-opting a public-domain engineering gift into a commercial asset.
The problem is that the phrase “the next evolution of SQLite” conveys continuity and endorsement.
A reasonable reader could conclude that Turso is an official successor or the next release of SQLite — that it represents the official lineage of the project.
Phrasing like “SQLite-compatible,” or “a fork of SQLite” would be clear and factual.
Calling it “the next evolution of SQLite” isn’t factual; it’s marketing positioning, and it implies ownership of SQLite’s identity and lineage.
This reflects a broader pattern in how the fork has been presented publicly.
The messaging often treats the original project as something to be fixed rather than a foundation to be respected.
Referring to Turso as a product while leveraging the SQLite name reinforces that framing — co-opting a public-domain engineering gift into a commercial asset.
Seems like typical marketing fluff to me.
One ugly thing I remember was the insinuation that SQLite was somehow immoral. Glauber Costa attacked SQLite[1] with lines like:
> We take our code of conduct seriously, and unlike SQLite, we do not substitute it with an unclear alternative. We strive to foster a community that values diversity, equity, and inclusion. We encourage others to speak up if they feel uncomfortable.
Sometime after the HN backlash[2], that paragraph was removed. Eventually, Turso released https://github.com/tursodatabase/turso without a code of conduct and hasn't had one since.
For me, the saddest part of the affair is that Richard Hipp and Glauber are both committed Christians. There should be no reason for envy, enmity, or memory games. But when business and money was on the table, it seems only one of them followed Jesus.
[1] https://github.com/tursodatabase/libsql/commit/3ac3ad263c0f0... https://web.archive.org/web/20221004144141/https://glauberco...
[2] https://news.ycombinator.com/item?id=33081159
> We take our code of conduct seriously, and unlike SQLite, we do not substitute it with an unclear alternative. We strive to foster a community that values diversity, equity, and inclusion. We encourage others to speak up if they feel uncomfortable.
Sometime after the HN backlash[2], that paragraph was removed. Eventually, Turso released https://github.com/tursodatabase/turso without a code of conduct and hasn't had one since.
For me, the saddest part of the affair is that Richard Hipp and Glauber are both committed Christians. There should be no reason for envy, enmity, or memory games. But when business and money was on the table, it seems only one of them followed Jesus.
[1] https://github.com/tursodatabase/libsql/commit/3ac3ad263c0f0... https://web.archive.org/web/20221004144141/https://glauberco...
[2] https://news.ycombinator.com/item?id=33081159
What you are pointing to did happen. What you are forgetting to mention is that this was one phrase in a post that was otherwise singing their praises.
The charitable interpretation is that when people pointed this out, I have read it, and come to agree that the wording was too strong and not representative of what we wanted to convey. Therefore, it was taken down.
But why be charitable when one can just throw accusations around from the armchair towards people one barely knows ?
The one thing you are right about, is that I am failing to live up to the Lord. As much as I try, I keep falling short. It happens not only here but in all aspects of my life. If it wasn't for his grace I would be in Hell for sure.
The charitable interpretation is that when people pointed this out, I have read it, and come to agree that the wording was too strong and not representative of what we wanted to convey. Therefore, it was taken down.
But why be charitable when one can just throw accusations around from the armchair towards people one barely knows ?
The one thing you are right about, is that I am failing to live up to the Lord. As much as I try, I keep falling short. It happens not only here but in all aspects of my life. If it wasn't for his grace I would be in Hell for sure.
Glauber, your memory is off. That line of attack was not in your blogpost; it's in your now-deleted manifesto, the README.md of libsql[1]. It was a strong statement of intent, praising the software and campaigning against the maintainers. "SQLite needs to open contributions" was a spat you started in front of HN's eyes, and are now denying[2] for reasons I can only guess at. That's the reality that myself and others observed.
Of course, you can take that — all of this — as an accusation. You can choose a narrative that's kinder towards you. The question I'd ask myself is if Jesus really wants to hear our storytelling, or if he wants to see us repent.
That said: on certain key aspects of life, I think you're at least closer to the Lord than I am. So Godspeed to you on your journey.
[1] https://github.com/tursodatabase/libsql/blob/3ac3ad263c0f092...
[2] https://news.ycombinator.com/item?id=45586331
Of course, you can take that — all of this — as an accusation. You can choose a narrative that's kinder towards you. The question I'd ask myself is if Jesus really wants to hear our storytelling, or if he wants to see us repent.
That said: on certain key aspects of life, I think you're at least closer to the Lord than I am. So Godspeed to you on your journey.
[1] https://github.com/tursodatabase/libsql/blob/3ac3ad263c0f092...
[2] https://news.ycombinator.com/item?id=45586331
[deleted]
I get where you're coming from, but isnt the whole idea of open source "if you dont like the approach, you're free to fork the code and do it the way you think is right?"
As long as the fork doesnt violate trademark (turso vs sqlite) it is working-as-intended?
I, for one, encourage this kind of behavior. We should have more forks. More forks = more competition = better results for everyone.
---
To make an analogy. Would you say the same thing if this were a for-profit company?
"I cant believe someone else is competing the same space as $x. $x is hugely successful, and so many people use it. I dont know why there's an alternative"
As long as the fork doesnt violate trademark (turso vs sqlite) it is working-as-intended?
I, for one, encourage this kind of behavior. We should have more forks. More forks = more competition = better results for everyone.
---
To make an analogy. Would you say the same thing if this were a for-profit company?
"I cant believe someone else is competing the same space as $x. $x is hugely successful, and so many people use it. I dont know why there's an alternative"
Both forking sqlite, and disliking the forking of sqlite, are allowed.
Plus, it's just technically bad. Most cases where you'd want to scale up sqlite are better served by a client/server database.
Plus, it's just technically bad. Most cases where you'd want to scale up sqlite are better served by a client/server database.
>Most cases
But not all of them, right? I don't need scaling across different hosts, but I need scaling between different threads on the same host. And I don't want for that PostgreSQL, MySQL, Oracle an other stuff like that.
But not all of them, right? I don't need scaling across different hosts, but I need scaling between different threads on the same host. And I don't want for that PostgreSQL, MySQL, Oracle an other stuff like that.
Author of Turso here. Couple of points
* You are right not to rush. You should keep using SQLite until Turso matures. Some use cases are more tolerant to new tech than others. It will take time for us to reach the level of trust SQLite has for broad use cases, but we are hoping to add value for some use cases right away. Never rush, tech matters!
* I have never met Hipp, but only heard great things about him.
* We never had a fight with SQLite over their contribution model (or about anything for that matter, I never even met Hipp or anybody else from SQLite). We just disagree with it - in the sense that we believe in different things. We don't think what they do is fundamentally wrong. Different projects take different paths.
* We are not using the SQLite name. We compare ourselves to SQLite because we are file and API compatible, and we do aspire to raise the very high bar they have set. It is hard to do this without drawing the comparison, but we are a different project and state it very clearly. I am not a lawyer (and neither you seem to be), but we believe we are doing is okay. If we ever have any valid reason to believe we crossed a line here, we will of course change course.
* We are not "startup bros". We spent 20+ years of our lives building databases and operating systems.
* You are right not to rush. You should keep using SQLite until Turso matures. Some use cases are more tolerant to new tech than others. It will take time for us to reach the level of trust SQLite has for broad use cases, but we are hoping to add value for some use cases right away. Never rush, tech matters!
* I have never met Hipp, but only heard great things about him.
* We never had a fight with SQLite over their contribution model (or about anything for that matter, I never even met Hipp or anybody else from SQLite). We just disagree with it - in the sense that we believe in different things. We don't think what they do is fundamentally wrong. Different projects take different paths.
* We are not using the SQLite name. We compare ourselves to SQLite because we are file and API compatible, and we do aspire to raise the very high bar they have set. It is hard to do this without drawing the comparison, but we are a different project and state it very clearly. I am not a lawyer (and neither you seem to be), but we believe we are doing is okay. If we ever have any valid reason to believe we crossed a line here, we will of course change course.
* We are not "startup bros". We spent 20+ years of our lives building databases and operating systems.
I wasn’t describing a legal dispute, and I haven’t made any claim about infringement. “Material misrepresentation” refers to the substance of the public messaging, not to a legal violation.
The issue I raised is that the phrasing and the way the fork has been presented create the impression of continuity and endorsement that doesn’t exist. That’s a reputational and ethical concern, not a legal debate. Calling it “the next evolution of SQLite” is, in practice, absolutely trading on the SQLite name.
There was a very public, one-sided disagreement about SQLite’s contribution model at the time, and you’ve been open about your criticisms of SQLite in the years since. That’s the context for my comment; it isn’t something I’ve imagined.
The issue I raised is that the phrasing and the way the fork has been presented create the impression of continuity and endorsement that doesn’t exist. That’s a reputational and ethical concern, not a legal debate. Calling it “the next evolution of SQLite” is, in practice, absolutely trading on the SQLite name.
There was a very public, one-sided disagreement about SQLite’s contribution model at the time, and you’ve been open about your criticisms of SQLite in the years since. That’s the context for my comment; it isn’t something I’ve imagined.
I understood your criticism. I disagree with it, and I think we are not misrepresenting anything, as we make it very clear that we are not SQLite, we are just a reimplementation that goes beyond it (evolution).
The disagreement about their contribution model of course happened, but the meaning you ascribe to it, perhaps is something you imagined. It boils down to what you understand "criticism" to be.
If I see someone doing something wrong, I will criticize them. That certainly never happened. What happened is that we pointed out pros and cons of an open and closed development model. We believe a piece of technology that plays the role of SQLite would benefit from having an open model. And exactly because they are absolutely not doing nothing wrong with not being open, we created our own thing. Hard to see how that is a "criticism".
I said that a billion times, and here's a billion and one: there's absolutely nothing wrong with a closed model. SQLite is doing nothing wrong. They contributed tremendously to the databases we used every day.
I do think an Open model yields so many benefits that should someone rewrite SQLite with an open model, even starting 20 years later, they would end up ahead.
There is now a very easy way to prove or disprove this particular hypothesis.
Stay tuned!
The disagreement about their contribution model of course happened, but the meaning you ascribe to it, perhaps is something you imagined. It boils down to what you understand "criticism" to be.
If I see someone doing something wrong, I will criticize them. That certainly never happened. What happened is that we pointed out pros and cons of an open and closed development model. We believe a piece of technology that plays the role of SQLite would benefit from having an open model. And exactly because they are absolutely not doing nothing wrong with not being open, we created our own thing. Hard to see how that is a "criticism".
I said that a billion times, and here's a billion and one: there's absolutely nothing wrong with a closed model. SQLite is doing nothing wrong. They contributed tremendously to the databases we used every day.
I do think an Open model yields so many benefits that should someone rewrite SQLite with an open model, even starting 20 years later, they would end up ahead.
There is now a very easy way to prove or disprove this particular hypothesis.
Stay tuned!
You’ve reframed this as a discussion about open versus closed development. That wasn’t the point I raised.
My concern is about presentation, ethics, respect, and about the co-option of a gift to the commons — particularly how your public messaging gives readers the impression of lineage and endorsement that doesn’t exist.
Regardless of your intent, that’s the effect of calling Turso “the next evolution of SQLite.” You’re welcome to disagree; it would be very strange if you didn’t.
My concern is about presentation, ethics, respect, and about the co-option of a gift to the commons — particularly how your public messaging gives readers the impression of lineage and endorsement that doesn’t exist.
Regardless of your intent, that’s the effect of calling Turso “the next evolution of SQLite.” You’re welcome to disagree; it would be very strange if you didn’t.
Yes, this is (one of) the point(s) you raised. You said there was a public an "unpleasant fight" (never happened) because of "criticism of SQLite" on my part. I calmly explained that such thing never took place. I have a preference towards Open models but never criticized SQLite (as in stated that they are wrong). Where does the unpleasant fight comes from?
Your claim that we are doing something ethically wrong seems to be informed by your pre-existing opinion of me, that itself derives from the "unpleasant fight" (that never happened).
As for the tagline we use, most people don't get the impression that there is any violation of ethics or respect. This is evidenced by other people's reaction here. You do, and you are within your right. I can't, unfortunately, please everybody.
Some people are more relevant than others, though: in this case, if the authors of SQLite expressed their opinion to me that this crosses a line in their view, I'd change it, without blinking an eye.
I have a tremendous respect for them, and we want our messaging to convey nothing but that!
Your claim that we are doing something ethically wrong seems to be informed by your pre-existing opinion of me, that itself derives from the "unpleasant fight" (that never happened).
As for the tagline we use, most people don't get the impression that there is any violation of ethics or respect. This is evidenced by other people's reaction here. You do, and you are within your right. I can't, unfortunately, please everybody.
Some people are more relevant than others, though: in this case, if the authors of SQLite expressed their opinion to me that this crosses a line in their view, I'd change it, without blinking an eye.
I have a tremendous respect for them, and we want our messaging to convey nothing but that!
Your focus is on intent; mine is on the ethical consequences of behavior.
The earlier disagreement with SQLite was one-sided because the other side saw no reason to engage. I expect that dynamic will continue.
The earlier disagreement with SQLite was one-sided because the other side saw no reason to engage. I expect that dynamic will continue.
Awesome to see you chiming in here.
Just to set some context, I remember talking to Glauber and Pekka last year, right around when the Turso folks started toying with the idea of reimplementing SQLite in Rust.
It’s a moonshot, but if anyone can pull it off, it’s these people. If you don’t know their background, checkout ScyllaDB. It never ceases to amaze me how belligerent some HN folks can be - they won’t even take five minutes to do a Google search before calling someone a nobody or “SV tech bro.” Not that I’m saying you should do that to anyone.
Just to set some context, I remember talking to Glauber and Pekka last year, right around when the Turso folks started toying with the idea of reimplementing SQLite in Rust.
It’s a moonshot, but if anyone can pull it off, it’s these people. If you don’t know their background, checkout ScyllaDB. It never ceases to amaze me how belligerent some HN folks can be - they won’t even take five minutes to do a Google search before calling someone a nobody or “SV tech bro.” Not that I’m saying you should do that to anyone.
Thanks! Appreciate it, and you will see we came a long way with the implementation. It's now right around the corner! We'd love to see you try and report back your impressions.
I'd still give them the benefit of the doubt as most (all?) of the contributors are Finns who you can almost guarantee to have a "no bullshit" type of mentality to essentially everything. And what I'd guess, the team most probably has quite an academic background because of the local culture.
Pekka is a Finn, and there are other Finns as well, but not all of us are Finns.
Our Finns are also not the standard Finns. I know Pekka closely for 15 years, and he smiled 3 times.
Our Finns are also not the standard Finns. I know Pekka closely for 15 years, and he smiled 3 times.
I agree that "the single-writer limitation isn't just a theoretical concern", but it's also solvable without forking SQLite. ulimit's the limit! If your goal is resource maximization of a given computer, though, Postgres is likely a better fit.