High Availability Load Balancers with Maglev

BackgroundWe run many backend services that power our customer dashboard, APIs, and features available at our edge. We own and operate physical infrastructure for our backend services. We need an effective way to route arbitrary TCP and UDP traffic between services and also from outside these data centers. Previously, all traffic for these backend services would pass through several layers of stat