Oh this is really cool! I didn't know Go has added this!
I went on a similar adventure but in Zig. Since I had to prepare a benchmarking suite, I put out one in case anyone needs it. If you think it might be helpful, give it a go:
https://github.com/peymanmortazavi/csv-race
In my findings, using 64 bytes (512-bits) even when possible actually degraded the performance. I also had to fine-tune the numbers for different CPUs. For instance on Apple, I could go much higher but on my CPU, if I went to 64 bytes (512-bits), It would degrade the performance.
Another thing I explored was to iterate on the fields as opposed to records. This allows you to just avoid any copying or dynamic memory allocation, which should give you a pretty decent boost. You can add utility wrappers to match Go's record based iteration when it is necessary.
What started as a small utility for a personal project turned into a month-long deep dive into performance engineering, SIMD, and the surprisingly sharp edges of CSV parsing. My goal was straightforward: build a simple, zero-allocation CSV iterator and writer in Zig that could handle real-world inputs without sacrificing performance.
Along the way, I explored a number of parsing strategies, including approaches inspired by a well-known paper on SIMD-accelerated JSON parsing. While that technique is elegant and highly effective for JSON, I found it didn’t translate cleanly to CSV—at least not without giving up more performance than I was willing to accept. CSV’s delimiter-heavy structure and quoting rules demanded a different approach.
After iterating through several designs and benchmarking them against each other, I eventually converged on a technique that consistently outperformed my earlier implementations. When I compared the final version against some of the fastest CSV libraries I could find, the results were better than I expected.
This post walks through the design decisions behind csv-zero, the tradeoffs I made, and the techniques that ended up mattering the most. It’s also a bit of a love letter to Zig: working in the language made it much easier to reason about memory, data layout, and performance, and pushed me to tackle problems I would have otherwise avoided.
If you’re curious about SIMD-based parsing, zero-allocation APIs, or just want to see how far you can push CSV, read on.
I went on a similar adventure but in Zig. Since I had to prepare a benchmarking suite, I put out one in case anyone needs it. If you think it might be helpful, give it a go: https://github.com/peymanmortazavi/csv-race
In my findings, using 64 bytes (512-bits) even when possible actually degraded the performance. I also had to fine-tune the numbers for different CPUs. For instance on Apple, I could go much higher but on my CPU, if I went to 64 bytes (512-bits), It would degrade the performance.
Another thing I explored was to iterate on the fields as opposed to records. This allows you to just avoid any copying or dynamic memory allocation, which should give you a pretty decent boost. You can add utility wrappers to match Go's record based iteration when it is necessary.
Just some thoughts! but congrats on this!!