Connection pool helps with unbounded number requests. But watch out...
Hussein Nasser@hnasr
53 views
Nov 03, 2025
~1 min read
Advertisement
1
Connection pool helps with unbounded number requests. But watch out for bad defaults.
When a frontend connects to a reverse proxy (load balancer, api gateway, cdn etc.) and sends a request, the RP turns around and establishes a connection to the origin backend and forwards the request to the backend on that connection.
The challenge is how do you handle a large fleet of requests from all the frontends? That easily can overwhelm the backend or results on connection resets error if a backend accept queue gets filled up.
One solution that is common is to implement a connection pool, which is basically a queue where we have a max number of connections and requests get queued to the pool with a timeout and will be processed by a free connection.
But even with a connection pool if the defaults are set to a very large number you can still run into these issues without realizing. Finding the good number is balance based on workloads and protocols used.
Check out my fundamentals of backend engineering course at backend.win
When a frontend connects to a reverse proxy (load balancer, api gateway, cdn etc.) and sends a request, the RP turns around and establishes a connection to the origin backend and forwards the request to the backend on that connection.
The challenge is how do you handle a large fleet of requests from all the frontends? That easily can overwhelm the backend or results on connection resets error if a backend accept queue gets filled up.
One solution that is common is to implement a connection pool, which is basically a queue where we have a max number of connections and requests get queued to the pool with a timeout and will be processed by a free connection.
But even with a connection pool if the defaults are set to a very large number you can still run into these issues without realizing. Finding the good number is balance based on workloads and protocols used.
Check out my fundamentals of backend engineering course at backend.win
