HackerTrans
TopNewTrendsCommentsPastAskShowJobs

otoolep

5,060 karmajoined 12 anni fa
Software Engineer based in Pittsburgh, PA. Engineering Manager at Google, building large-scale data systems. Creator of rqlite[1], the lightweight, distributed database built on SQLite.

https://www.philipotoole.com

[1] https://www.rqlite.io

Submissions

Proving to Fable I maintain the repo

philipotoole.com
3 points·by otoolep·3 giorni fa·0 comments

Data integrity in rqlite: what it checks and what it doesn't

rqlite.io
2 points·by otoolep·20 giorni fa·0 comments

Humanoid robots won't surprise us when they arrive

philipotoole.com
5 points·by otoolep·2 mesi fa·0 comments

Taking Control of the SQLite WAL

philipotoole.com
1 points·by otoolep·2 mesi fa·0 comments

The eerie abandoned vehicles in Chernobyl's 'dead zone'

bbc.com
3 points·by otoolep·2 mesi fa·0 comments

How (and why) rqlite takes control of the SQLite Write-Ahead Log

philipotoole.com
2 points·by otoolep·2 mesi fa·0 comments

Show HN: rqlite – the fault-tolerant DB built on SQLite – v10.0 released

github.com
4 points·by otoolep·2 mesi fa·2 comments

Open Decision-Making (2021)

web.stanford.edu
1 points·by otoolep·4 mesi fa·0 comments

The source code was the moat. But not anymore

philipotoole.com
2 points·by otoolep·5 mesi fa·0 comments

The Fall of the Nerds

noahpinion.blog
2 points·by otoolep·5 mesi fa·0 comments

Talking to LLMs has improved my thinking

philipotoole.com
186 points·by otoolep·6 mesi fa·146 comments

The Engineering Management Myths Star Trek Teaches (2022)

philipotoole.com
4 points·by otoolep·6 mesi fa·1 comments

Common misunderstandings about large software companies

philipotoole.com
97 points·by otoolep·6 mesi fa·53 comments

The liquid air alternative to fossil fuels

bbc.com
2 points·by otoolep·6 mesi fa·0 comments

Common misunderstandings about large software companies

philipotoole.com
5 points·by otoolep·6 mesi fa·1 comments

Start your meetings at 5 minutes past

philipotoole.com
281 points·by otoolep·6 mesi fa·227 comments

The number one sign you're watching an AI video

bbc.com
5 points·by otoolep·8 mesi fa·0 comments

rqlite 9.2 – the distributed SQLite database – fast restarts with GB datasets

philipotoole.com
4 points·by otoolep·8 mesi fa·0 comments

[untitled]

3 points·by otoolep·10 mesi fa·0 comments

OpenTelemetry Protocol Comes to Google Cloud Observability

cloud.google.com
3 points·by otoolep·10 mesi fa·1 comments

comments

otoolep
·l’altro ieri·discuss
Blog post author here (I did not post the link above to HN). I also wrote a short follow-up blog post at https://philipotoole.com/proving-to-fable-i-maintain-the-rep...
otoolep
·2 mesi fa·discuss
You can learn about some known use cases here: https://rqlite.io/docs/features/#common-use-cases

One application that comes up over and over again -- wanting relational modeling, along with HA, but with low operational costs. People sometimes start with Postgres, then need to set up HA, and find it's a lot of work. They realize that their data set requirements are not huge, don't need fancy features, so start to think Postgres is overkill. That's what Replicated found.
otoolep
·3 mesi fa·discuss
[dead]
otoolep
·4 mesi fa·discuss
I grew up in Ireland, moved to the USA as an adult. European government is clearly Hobbes in model, the US Lockean.

In Europe the individual has almost no legal reason to use force, and force by individuals is considered illegitimate. The "Sovereign" has all the coercive power in European states. In the US, however, a certain amount of legitimate force explicitly remains with the individual i.e. the 2nd amendment. (I am not making a value judgement here).

Of course, Europe has government with the consent of the governed, so is Lockean in that sense. But the balance of force between the "Sovereign" and the people in Europe is all Hobbes. You only notice it when you move to the US and compare it to Europe.

Europe had centuries of religious and civil war. It's not surprising Hobbes won out.
otoolep
·6 mesi fa·discuss
rqlite creator here, happy to answer any questions.

As for reliability - it's a fault-tolerant, highly available system. Reliability is the reason it exists. :-) If you're asking about quality and test coverage, you might like to check out these resources:

- https://rqlite.io/docs/design/

- https://rqlite.io/docs/design/#blog-posts

- https://philipotoole.com/how-is-rqlite-tested/
otoolep
·6 mesi fa·discuss
>First of all, does anyone believe that highly scrutinized and bureaucratic functions are general high quality services?

This is the only part of your response that doesn't quite sit right with me. There could be many "highly scrutinized and bureaucratic functions" out there that are working very well, you just don't notice because they work so well. There could be a selection-effect here.

Quality is a big deal for me[1]. But I think you're defining "quality" too narrowly in this context. "Quality" could also mean "allows everyone, at scale, reliably, to do what they need to do." The US Tax Filing system (and its associated software) meets that goal.

[1] https://philipotoole.com/always-thinking-of-the-next-guy/
otoolep
·6 mesi fa·discuss
Blog post author here, thanks for your thoughtful comment. You raise some interesting things to think about.
otoolep
·6 mesi fa·discuss
OK, well, you could try client-side batching too, if you can. That will also improve performance substantially.

Otherwise, if you try with more modern networks and disks, let me know what you see.
otoolep
·6 mesi fa·discuss
Yes, fast networks matter.

I did introduce Queued Writes[1] since that talk, allowing you to trade off performance versus immediate durability. It may interest you -- network is much less of a factor then, and you should get a 10-100x increase in throughput.

[1] https://rqlite.io/docs/api/queued-writes/
otoolep
·6 mesi fa·discuss
rqlite creator here.

Yes I do have practical experience to share, I wrote a blog post on rqlite and FTS5: https://philipotoole.com/building-a-highly-available-search-...

Will it allow you to reach the same scale in terms of data set size that Elasticsearch supports? Almost certainly no, but it might be enough depending on your use case.
otoolep
·6 mesi fa·discuss
rqlite creator here. I have performed a fair amount of performance testing, some of which I outlined in a talk to the CMU Database Group a few years ago. Details:

- https://www.philipotoole.com/2021-rqlite-cmu-tech-talk - see slide 33.

- There is also a recording that goes with the talk: https://www.youtube.com/watch?v=JLlIAWjvHxM

You can also read about Performance in the docs at: https://rqlite.io/docs/guides/performance/

An important thing to note: this testing was done 4+ years ago, on moderately-powerful hardware for the time. With higher-end, more modern hardware you may get even better results.
otoolep
·6 mesi fa·discuss
rqlite[1] creator here, happy to answer any questions.

https://rqlite.io
otoolep
·7 mesi fa·discuss
rqlite creator here, happy to answer any questions.
otoolep
·7 mesi fa·discuss
rqlite[1] creator here.

Nit: dqlite is a library, it is not a network-exposed database like rqlite is. Sure, it requires connecting to other nodes over the network, but local access is via in-process. In contrast one connects with rqlite over the network - HTTP specifically.

[1] https://rqlite.io
otoolep
·9 mesi fa·discuss
rqlite[1] creator here. Thanks for the shout-out in your blog post.

https://rqlite.io
otoolep
·9 mesi fa·discuss
rqlite creator here. Right there on the rqlite homepage[1] are listed two production users: replicated.com[2] and textgroove.com are both using it.

[1] https://rqlite.io/

[2] https://www.replicated.com/blog/app-manager-with-rqlite
otoolep
·9 mesi fa·discuss
rqlite[1] author here. Just to be clear, rqlite is not SQLite but rewritten in Go. rqlite uses the vanilla C code, and calls it from Go[2]. I consider that an important advantage over other approaches -- rqlite gets all the benefits of rock-solid[3] SQLite. As result there are no questions about the quality of the database engine.

[1] https://rqlite.io

[2] https://rqlite.io/docs/design/

[3] https://www.sqlite.org/testing.html
otoolep
·2 anni fa·discuss
Oh, and more background here: https://www.philipotoole.com/building-a-distributed-key-valu...
otoolep
·2 anni fa·discuss
FWIW, I built hraftd[1] many years ago to make it easy to play with a simple distributed system, but one that uses a production-grade implementation of Raft[2]. You can spin up a cluster in seconds on a single machine, kill nodes, watch a new Leader get elected, and so on.

It's written in Go, so it'll help if you are familiar with Go. But the code is not difficult to understand even if you don't.

[1] https://github.com/otoolep/hraftd

[2] https://github.com/hashicorp/raft