I figured it out (almost)! It shouldn’t have been a surprise, but it turns out the Chipmunk2D physics engine GoLang port I was using was actually really expensive. I took it out and it’s now a lot more efficient. It only adds about 1% per player but I think it can be better.
The algorithm I use for sending data to players is exponential and I’m pretty sure I can make it linear. I’ll fix that and then post my results here just for fun haha