Meta engineering team introduced ZGateway , a proxy tier that now sits between client applications and ZippyDB , the Meta’s most widely used key value store. ZippyDB backs product metadata, counters, and configuration at billions of operations per second. ZGateway started as a fix for connection sprawl across more than a million client hosts and grew into the home for batching, admission control, caching, and failover. Why ZippyDB Needed a Proxy Under direct access, every ZippyDB client connected to every database host it needed. A single client could touch tens of thousands of shards across hundreds of thousands of hosts, so both a typical client and a typical database host carried tens of thousands of TLS connections. Each idle connection consumed memory, CPU, and a file descriptor on both ends, and inbound counts grew with every client cohort. Reconnection storms caused crashes from file descriptor exhaustion and OOMs; in one incident a routing bug made every client open a connection per shard and the fleet fell into a reboot loop. Client-side fixes were impractical because hundreds of teams own the client fleet. What is ZGateway ZGateway is a stateless proxy tier between ZippyDB clients and the ZServer database fleet. Per Meta, it handles more than 1 billion operations per second and carries about 40% of ZippyDB traffic, projected to pass 60%, at about 6% computational overhead for an average use case. It runs as regional tiers discovered through ServiceRouter , Meta’s service mesh, in two flavors: a pure proxy and a read-through cache. The engine is Meta’s thick C++ Zi
Source: MarkTechPost