JasonPunyon

पिछला वर्षdiscuss

Thanks for taking it for a spin! I'm working on why this is slow now.

2 वर्ष पहलेdiscuss

You may remember a carbon copy of this event from a year ago. https://meta.stackexchange.com/questions/389922/june-2023-da...

Discussion from then https://news.ycombinator.com/item?id=36257523

JasonPunyon

2 वर्ष पहलेdiscuss

If anyone wants their data back in a way they can use it, it's right here https://seqlite.puny.engineering

And I'd be remiss if I didn't point out that their trade dress is MIT licensed. https://stackoverflow.design

Have fun.

JasonPunyon

2 वर्ष पहलेdiscuss

Thanks so much, that’s really kind.

JasonPunyon

2 वर्ष पहलेdiscuss

Good catch! Fix going up now.

JasonPunyon

2 वर्ष पहलेdiscuss

Nice post!

Yep, my process is similar. It goes...

  - decompress (users|posts)  
  - split into batches of 10,000  
  - xsltproc the batch into sql statements  
  - pipe the batches of statements into sqlite in parallel using flocks for coordination

On my M1 Max it takes about 40 minutes for the whole network. Then I compress each database with brotli which takes about 5 hours.

JasonPunyon

2 वर्ष पहलेdiscuss

This site is on a Cloudflare R2 bucket because (and only because) they have free egress. While not datacenter sized some of these files are large. Just opening up your credit card to 10 cents a gigabyte will be a bad time anywhere else.

JasonPunyon

3 वर्ष पहलेdiscuss

I mean fine, write an article that overweights recent events to weave a tale of the decline of Stack Exchange...but don't leave out the moderator strike, the data dump fiasco, or the CEO that simply doesn't get it. Jon Ericson's writing on the issue is much more informative. https://jlericson.com/

JasonPunyon

3 वर्ष पहलेdiscuss

https://stackoverflow.blog/2023/05/10/a-message-from-prashan...

JasonPunyon

Submissions

Why is the total number of questions on Stack Overflow shrinking lately?

Stack Overflow has stopped publishing data dumps to the Internet Archive

Farey Numbers and Linked Lists

SEqlite – Minimal Stack Exchange Data Dump in SQLite Format

SEqlite: Minimal Stack Exchange Data Dump in SQLite Format

June 2023 Data Dump is missing

comments