I want to implement a fast IP lookup DB similar to Linux Kernel ipset.
The DB should store IP blacklist for a DNS/HTTP RBL service.
DNS is an established binary protocol and RBL is used in many places around the globe.
A simple PTR check can answer a fake domain name that will reflect if the IP is blacklisted or not and if so for what list name.
The main issue is that the IP blacklist should be held in memory.
When holding many IP addresses in memory it can consume a lot of memory and can be slow if not implemented correctly.
I am not sure I understand GoLang memory utilization and management enough to decide if a simple net.IP objects structure is better then a node list of int’s or big.int’s are better.
I have found the next library and it seems a pretty good start to something:
Which I struggle to understand.
How are they looping over the data file?
Would it be smarter to hold the data in some kind of array of structs?
Maybe a node list?