HackerTrans
TopNewTrendsCommentsPastAskShowJobs

otoolep

5,060 karmajoined hace 12 años
Software Engineer based in Pittsburgh, PA. Engineering Manager at Google, building large-scale data systems. Creator of rqlite[1], the lightweight, distributed database built on SQLite.

https://www.philipotoole.com

[1] https://www.rqlite.io

Submissions

Proving to Fable I maintain the repo

philipotoole.com
3 points·by otoolep·hace 3 días·0 comments

Data integrity in rqlite: what it checks and what it doesn't

rqlite.io
2 points·by otoolep·hace 21 días·0 comments

Humanoid robots won't surprise us when they arrive

philipotoole.com
5 points·by otoolep·hace 2 meses·0 comments

Taking Control of the SQLite WAL

philipotoole.com
1 points·by otoolep·hace 2 meses·0 comments

The eerie abandoned vehicles in Chernobyl's 'dead zone'

bbc.com
3 points·by otoolep·hace 2 meses·0 comments

How (and why) rqlite takes control of the SQLite Write-Ahead Log

philipotoole.com
2 points·by otoolep·hace 2 meses·0 comments

Show HN: rqlite – the fault-tolerant DB built on SQLite – v10.0 released

github.com
4 points·by otoolep·hace 2 meses·2 comments

Open Decision-Making (2021)

web.stanford.edu
1 points·by otoolep·hace 4 meses·0 comments

The source code was the moat. But not anymore

philipotoole.com
2 points·by otoolep·hace 5 meses·0 comments

The Fall of the Nerds

noahpinion.blog
2 points·by otoolep·hace 5 meses·0 comments

Talking to LLMs has improved my thinking

philipotoole.com
186 points·by otoolep·hace 6 meses·146 comments

The Engineering Management Myths Star Trek Teaches (2022)

philipotoole.com
4 points·by otoolep·hace 6 meses·1 comments

Common misunderstandings about large software companies

philipotoole.com
97 points·by otoolep·hace 6 meses·53 comments

The liquid air alternative to fossil fuels

bbc.com
2 points·by otoolep·hace 6 meses·0 comments

Common misunderstandings about large software companies

philipotoole.com
5 points·by otoolep·hace 6 meses·1 comments

Start your meetings at 5 minutes past

philipotoole.com
281 points·by otoolep·hace 6 meses·227 comments

The number one sign you're watching an AI video

bbc.com
5 points·by otoolep·hace 8 meses·0 comments

rqlite 9.2 – the distributed SQLite database – fast restarts with GB datasets

philipotoole.com
4 points·by otoolep·hace 8 meses·0 comments

[untitled]

3 points·by otoolep·hace 10 meses·0 comments

OpenTelemetry Protocol Comes to Google Cloud Observability

cloud.google.com
3 points·by otoolep·hace 10 meses·1 comments

comments

otoolep
·anteayer·discuss
Blog post author here (I did not post the link above to HN). I also wrote a short follow-up blog post at https://philipotoole.com/proving-to-fable-i-maintain-the-rep...
otoolep
·hace 2 meses·discuss
You can learn about some known use cases here: https://rqlite.io/docs/features/#common-use-cases

One application that comes up over and over again -- wanting relational modeling, along with HA, but with low operational costs. People sometimes start with Postgres, then need to set up HA, and find it's a lot of work. They realize that their data set requirements are not huge, don't need fancy features, so start to think Postgres is overkill. That's what Replicated found.
otoolep
·hace 3 meses·discuss
[dead]
otoolep
·hace 4 meses·discuss
I grew up in Ireland, moved to the USA as an adult. European government is clearly Hobbes in model, the US Lockean.

In Europe the individual has almost no legal reason to use force, and force by individuals is considered illegitimate. The "Sovereign" has all the coercive power in European states. In the US, however, a certain amount of legitimate force explicitly remains with the individual i.e. the 2nd amendment. (I am not making a value judgement here).

Of course, Europe has government with the consent of the governed, so is Lockean in that sense. But the balance of force between the "Sovereign" and the people in Europe is all Hobbes. You only notice it when you move to the US and compare it to Europe.

Europe had centuries of religious and civil war. It's not surprising Hobbes won out.
otoolep
·hace 6 meses·discuss
rqlite creator here, happy to answer any questions.

As for reliability - it's a fault-tolerant, highly available system. Reliability is the reason it exists. :-) If you're asking about quality and test coverage, you might like to check out these resources:

- https://rqlite.io/docs/design/

- https://rqlite.io/docs/design/#blog-posts

- https://philipotoole.com/how-is-rqlite-tested/
otoolep
·hace 6 meses·discuss
>First of all, does anyone believe that highly scrutinized and bureaucratic functions are general high quality services?

This is the only part of your response that doesn't quite sit right with me. There could be many "highly scrutinized and bureaucratic functions" out there that are working very well, you just don't notice because they work so well. There could be a selection-effect here.

Quality is a big deal for me[1]. But I think you're defining "quality" too narrowly in this context. "Quality" could also mean "allows everyone, at scale, reliably, to do what they need to do." The US Tax Filing system (and its associated software) meets that goal.

[1] https://philipotoole.com/always-thinking-of-the-next-guy/
otoolep
·hace 6 meses·discuss
Blog post author here, thanks for your thoughtful comment. You raise some interesting things to think about.
otoolep
·hace 6 meses·discuss
OK, well, you could try client-side batching too, if you can. That will also improve performance substantially.

Otherwise, if you try with more modern networks and disks, let me know what you see.
otoolep
·hace 6 meses·discuss
Yes, fast networks matter.

I did introduce Queued Writes[1] since that talk, allowing you to trade off performance versus immediate durability. It may interest you -- network is much less of a factor then, and you should get a 10-100x increase in throughput.

[1] https://rqlite.io/docs/api/queued-writes/
otoolep
·hace 6 meses·discuss
rqlite creator here.

Yes I do have practical experience to share, I wrote a blog post on rqlite and FTS5: https://philipotoole.com/building-a-highly-available-search-...

Will it allow you to reach the same scale in terms of data set size that Elasticsearch supports? Almost certainly no, but it might be enough depending on your use case.
otoolep
·hace 6 meses·discuss
rqlite creator here. I have performed a fair amount of performance testing, some of which I outlined in a talk to the CMU Database Group a few years ago. Details:

- https://www.philipotoole.com/2021-rqlite-cmu-tech-talk - see slide 33.

- There is also a recording that goes with the talk: https://www.youtube.com/watch?v=JLlIAWjvHxM

You can also read about Performance in the docs at: https://rqlite.io/docs/guides/performance/

An important thing to note: this testing was done 4+ years ago, on moderately-powerful hardware for the time. With higher-end, more modern hardware you may get even better results.
otoolep
·hace 6 meses·discuss
rqlite[1] creator here, happy to answer any questions.

https://rqlite.io
otoolep
·hace 7 meses·discuss
rqlite creator here, happy to answer any questions.
otoolep
·hace 7 meses·discuss
rqlite[1] creator here.

Nit: dqlite is a library, it is not a network-exposed database like rqlite is. Sure, it requires connecting to other nodes over the network, but local access is via in-process. In contrast one connects with rqlite over the network - HTTP specifically.

[1] https://rqlite.io
otoolep
·hace 9 meses·discuss
rqlite[1] creator here. Thanks for the shout-out in your blog post.

https://rqlite.io
otoolep
·hace 9 meses·discuss
rqlite creator here. Right there on the rqlite homepage[1] are listed two production users: replicated.com[2] and textgroove.com are both using it.

[1] https://rqlite.io/

[2] https://www.replicated.com/blog/app-manager-with-rqlite
otoolep
·hace 9 meses·discuss
rqlite[1] author here. Just to be clear, rqlite is not SQLite but rewritten in Go. rqlite uses the vanilla C code, and calls it from Go[2]. I consider that an important advantage over other approaches -- rqlite gets all the benefits of rock-solid[3] SQLite. As result there are no questions about the quality of the database engine.

[1] https://rqlite.io

[2] https://rqlite.io/docs/design/

[3] https://www.sqlite.org/testing.html
otoolep
·hace 2 años·discuss
Oh, and more background here: https://www.philipotoole.com/building-a-distributed-key-valu...
otoolep
·hace 2 años·discuss
FWIW, I built hraftd[1] many years ago to make it easy to play with a simple distributed system, but one that uses a production-grade implementation of Raft[2]. You can spin up a cluster in seconds on a single machine, kill nodes, watch a new Leader get elected, and so on.

It's written in Go, so it'll help if you are familiar with Go. But the code is not difficult to understand even if you don't.

[1] https://github.com/otoolep/hraftd

[2] https://github.com/hashicorp/raft