HackerTrans
TopNewTrendsCommentsPastAskShowJobs

digitaldragon

no profile record

comments

digitaldragon
·11 miesięcy temu·discuss
The data is saved as a WARC file, which contains the entire HTTP request and response (compressed, of course). So it's much bigger than just a short -> long URL mapping.
digitaldragon
·2 lata temu·discuss
Unfortunately, Browsertrix relies on the Chrome Devtools Protocol, which strips transfer encoding (and possibly transforms the data in other ways). This results in Browsertrix writing noncompliant WARC files, because the spec requires that the original transfer encoding be preserved.