Hello,
New to this community,
I’m glad to announce the release of go-cdc-chunkers v1.0.0, an ISC licensed open source package that provides fast and memory-efficient Content-Defined Chunking.
It allows splitting streams of data into chunks of variable size, usually but not exclusively for deduplication purposes, making it a nice tool to avoid redundant processing or storing of data.
You can read more about it here:
The github repository is at: GitHub - PlakarKorp/go-cdc-chunkers: A Golang package that implements CDC chunkers with a generic interface