Yes. & in the end my experience has been that you can write low latency code with gc, but only by expending more effort than it'd take to use manual memory management
Fable's spatial reasoning is much better. Over the weekend I had opus looking into a blank textbox issue[1] which it was spinning on for a few minutes, switching to fable immediately fixed
But yeah opus often the better workhorse given price gap
I've experienced the opposite: wishing I were dead so life wouldn't go on with me falling into just another glum existence. That at the emotionally present moment I snapshot it, preventing inevitable dullness that time moves on
Unfortunately here I am, life moves on, now I'm just wasting away on HN
Maybe we were saved by being acquired before hiring sales. Sai knew the problem & understood customers. He'd sometimes oversell a bit, but managed it: kept pulse of capacity for new development, would ask about how hard requested features were, would feel out customer intent & guide customer adapt to what was already there
When we had our pepepizza moment, there was an understanding that it wasn't going to work, took learnings of what would be involved there, but kept focus on improving what we already had
For kafka connector we had a design partner, I got to work with them directly. They wanted 30 microsecond message processing, so didn't want json. Original ask was flatbuffers. I decided to put message formatting into a scripting layer using gopher-lua. Spent a weekend getting flatbuffers working with lua (it was buggy, opened half a dozen PRs to flatbuffers repo which got ignored). It was clearly awful having to manage flatbuffer schema files & update scripts every time schema changed. But I had alternative already made: msgpack. Throughput needed work but addressed that by creating pool of lua interpreters
Overall I overworked myself (put my hands out of commission & spent months relearning how to type on split ergo colemak-dh), but I enjoyed the work. Team was very open with each other & when performance is your selling point there's an understanding that engineering quality needs to be maintained. Sure there were parts of the system I hated, & sometimes I'd try chip away at those
Hopefully that helps, hard to say the difference, but I really feel in my work that when customer has problem I'm part of conversation. Most recently there was talk of customer wanting cold data offloaded from postgres which is what inspired https://github.com/ClickHouse/pg_clickhouse/pull/298 where we get Postgres to do most the work
Raised problems trying to mix C++ into postgres extension, decided fix was to write clickhouse-c library to replace clickhouse-cpp, there was some doubt on team about value, but demonstrated value (https://github.com/ClickHouse/pg_clickhouse/pull/254) & I appreciate my colleagues not being afraid to change their mind
There's a level of trust where instead of being assigning tasks on a board I instead work on what I think is important based on information available. Nobody was asking for wal-rus, but I know my fleet
ClickHouse Cloud similarly took route of taking its time hiring sales. Better to have a small sales team that can work directly with engineering on quality leads than overwhelming everyone so that sales becomes the enemy. Guess the difference is agency. When engineering is involved in making commitments they're invested in delivering & there's push back so sales doesn't start hallucinating features
If you're running pg yourself I recommend pgbackrest. It doesn't run as a daemon, & it forks multiple processes for concurrency. But it's simple to run as archive_command & is light on resources outside concurrency
tbf it took 4 years since PG15 support was added for me to fix remote BASE_BACKUP support & wal-g base backups being inconsistent on PG15+ (parameter typo had pg_backup_stop return before wal archived far enough for consistency)
Correct. We tune overcommit so postgres reliably returns out of memory. It becomes complicated to accurately tune overcommit for every AWS instance type. We configure GOMEMLIMIT/cgroups but those are about RSS. Outliers come together: instances running queries out of memory on our service tend to also be pushing other resource limits, causing wal-g & prometheus exporters to start having more erratic memory usage at the worst time
This helps on both ends of the cost spectrum. Large 64 core instances are where our heuristics fall off the most as variance increases, & tiny instances with 8GB of memory can use every 100MB of RSS we can get
no. but wal-g & wal-rus both have parallelism over wal-e. however are you more asking about handling build up of wal / vacuum prevention caused by long running transactions? those are up to postgres, archive command only keeps pushing wal so that when postgres is ready to get rid of wal it can. seems like your scenario wouldn't care much what the archiver is since wal should be shipped long before postgres is ready to get rid of wal