Show HN: Updated GiantJSON Viewer – Opening 100GB JSONs on Android (Rust+SIMD)(giantjson.com)
giantjson.com
Show HN: Updated GiantJSON Viewer – Opening 100GB JSONs on Android (Rust+SIMD)
https://giantjson.com/
5 comments
It has a 7-day trial if you apply, but I also have some 30-day trial codes for Google Play. Send me an email with "HN promo" to support [at] giantjson [dot] com and I'll send you a code back (within max 24 hours).
I'll do it but first upvote me , I don't have karma
Whats the use case for opening a 100gb file on a phone?
The 100GB was just me having fun and pushing its breaking point.
Nice, I'm creator of Logdy.dev [https://logdy.dev/logdy-pro] and some time ago I was developing a pro version that has similarities to your project. Would love to chat if you plan to commercialize your solution.
Around 2 months ago I presented my first version of "Giant JSON Viewer" here: https://news.ycombinator.com/item?id=46609592
Now, after a lot of ups and downs, around a hundred bugs squeezed, and deeper testing, I'm proudly presenting a newer version after a complete rework of the Rust core, pumped with some related, privacy-first tools.
First of all, I added a few more common formats: - JSON, NDJSON/JSONL - CBOR (first time convert to JSON) - MsgPack (first time convert to JSON) - HAR (dedicated features for this) - Markdown (straight, open only for view)
Then, I went further with my stress testing... Managed to open a ~100GB JSON file on an S23 Ultra. Previous version crashed. With the reworked indexer, backend, and aux files, it worked! It took 40 mins to index, but worked.
After realizing my compiler optimization mistake, the SIMD finally got its real native power and the first indexing went down to 4-5 mins for 100GB. On a phone! Handling of the file, scrolling, viewing, and jumping to elements are instant after the first indexing.
Search and filter are still (relatively) blazing fast with memchr
Then, to make the app more useful as a daily tool, I started building additional features:
- A rich REST API CLIENT (not fully featured yet, but supports GraphQL, OAuth2, and AWS SigV4).
- HAR analyzer. Since HAR is just JSON, why not utilize my existing backend to take advantage of it? (The first opening takes a bit longer, because apart from the initial JSON indexing, it requires request metadata processing for the filter/search and statistical features).
- Simple MOCK API: Nothing fancy, just host any files on the local network (Wifi, USB-tethering, USB-ethernet) statically.
- Privacy-first handy features: Why would you use online tools and risk sensitive data if you can do it locally? You might have your tools already (js, python, whatever), but if not, I can host a webUI for you from the app on the local network for all these. JWT, small JSON tools like prettify, minify, stringify, unescape, unix timestamp, hash generator... all on your own private webUI, hosted by the app. (Same features available in the app too).
Yes, this is my very first published app ever, and it's only available on Android, sorry. There are still some minor bugs, and 1 major (edge case) issue, but those will be handled too.
If you could take a look and let me know what your first impression is, I'd love to hear HONEST feedback! (I just realized recently how difficult it is to get useful info from users, even bug reports).
This is a freemium app: the JSON viewer section is completely free with no size limits, the export/transform features, api client and tools are paid.