I’ve just released the very first version of GitHub - molecule-man/go-brrr: Highly optimized pure go brotli implementation · GitHub .
What it does
It provides brotli compression and decompression capability
Why you made it
I wanted to use shared dictionary (a dictionary agreed on by both compressor and decompressor ahead of time) and non of pure go libraries had support of it.
Why it’s useful, especially compared to similar existing projects
It’s fast. Very fast. I have spent like months on optimization.
In fact it’s faster than c bindings provided by google/brotli/go/cbrotli on quality levels 2 - 9 which I initially thought would never be possible.
To be fair, the reason cbrotli ends up slower isn’t the c code itself: it doesn’t expose an API to reset the writer and reuse state (e.g. the ring buffer), so it reallocates the encoder on every compression. That’s the situation today, and it means in practice using go-brrr from Go will outperform calling into cbrotli (for levels 2-9 at least).
Go-brrr even provides higher throughput than klauspost/compress/zstd on levels 5-7 while giving a better compression ratio:
Please see other graphs and benchmarks in the readme.
The other reason go-brrr is useful is of course the support of Shared dictionaries which currently is not provided by other pure go brotli libraries.