HackerTrans
TopNewTrendsCommentsPastAskShowJobs

TheTechRobo

no profile record

comments

TheTechRobo
·2 years ago·discuss
Browsertrix (and Webrecorder tools in general) also violate the standard by modifying response data. It's supposed to be the raw bytes as they are sent over the network (minus TLS).

The entire WARC ecosystem is kind of a mess.
TheTechRobo
·2 years ago·discuss
Unfortunately the Raspberry Pi version isn't supported anymore because their version of Wget isn't well-tested on ARM. I'm surprised it's still there in the repo.

You can get Docker to emulate x86, though.
TheTechRobo
·2 years ago·discuss
That doesn't really confirm that, IMHO. The line before claims the opposite, in fact: "The Internet Archive has been around since 1996, and while that does not guarantee anything, it shows continuity of support and strong commitment to digital preservation with as much access as possible."
TheTechRobo
·2 years ago·discuss
Everything transmitted over HTTP. That includes cookies, passwords, etc. You need to be careful when writing WARCs.
TheTechRobo
·2 years ago·discuss
Yeah, the format really needs an update. For starters, WARC only officially supports HTTP/1.1. Webrecorder has started faking HTTP/1.1 data in WARC files in order to save other versions, but I don't think faking data is great for an archival format, especially if it isn't standardized.
TheTechRobo
·3 years ago·discuss
ArchiveTeam's general idea is to archive it first, and make it usable later. Wayback Machine is generally how stuff is "made usable", but in cases like this, a searchable index might need to made. (That's been done before, though - it's not like it's never happened.)