HackerTrans
TopNewTrendsCommentsPastAskShowJobs

qdw

no profile record

comments

qdw
·10 месяцев назад·discuss
Punch-configured textiles go all the way back to the 1700s! The 1804 Jacquard Loom, which used cards, was a big part of the Industrial Revolution and influenced Babbage's Difference Engine and Analytical Engine, both of which use punchcards.

More background: https://en.m.wikipedia.org/wiki/Jacquard_machine
qdw
·11 месяцев назад·discuss
One of their line items complains about being unable to bind 65k PostgreSQL placeholders (the linked post calls them "parameters") in a single query. This is a cursed idea to begin with, so I can't fully blame PostgreSQL.

From the linked GitHub issue comments, it looks like they adopted the sensible approach of refactoring their ORM so that it splits the big query into several smaller queries. Anecdotally, I've found 3,000 to 5,000 rows per write query to be a good ratio.

Someone else suggested first loading the data into a temp table and then joining against that, which would have further improved performance, especially if they wrote it as a COPY … FROM. But the idea was scrapped (also sensibly) for requiring too many app code changes.

Overall, this was quite an illuminating tome of cursed knowledge, all good warnings to have. Nicely done!