Repo: https://github.com/roy2220/clbtransport
Kubernetes is the standard infra for cloud-native workloads today, and most backend developers eventually hit the same wall: calling another service through its Kubernetes Service DNS name over plain HTTP leads to uneven load across backend pods.
The usual fixes people reach for are:
- The Kubernetes Endpoints API
- A gateway / load balancer
- A service mesh
But does it really have to be this heavy?
Rethinking load balancing
Think about the ideal case: if every single request picked a target node at random from the whole pool right before sending, the cluster load would end up balanced (assuming all nodes are roughly equal in capacity).
Short-lived HTTP connections combined with disabled DNS caching get you exactly that — but at a cost nobody wants to pay: every request now pays for a DNS lookup, a TCP handshake, and a TCP teardown.
So long-lived (keep-alive) connections are non-negotiable. The real problem becomes controlling how often a request gets to re-pick its target node:
- Too rare, and traffic piles up on whichever node was picked first.
- Too frequent, and the cost of constantly re-establishing TCP connections eats the balancing benefit.
The approach
clbtransport controls that frequency along two axes:
- Bound connection lifetime. Long-lived connections aren’t allowed to live forever — after some time (with jitter, to avoid synchronized reconnects) they’re retired, giving requests a fresh chance to land on a different node.
- Spread connections per host. Since connection lifetime can’t be pushed arbitrarily low without hurting latency/SLA (TCP setup blocks the request), a small number of independent connections are kept “hot” per host in parallel, multiplying the opportunities for requests to fan out across nodes.
The result behaves like a standard http.RoundTripper — a drop-in replacement for http.Transport — that quietly rebalances traffic in the background instead of relying on cluster-side machinery.