Rate Limiters
System Design Basics
Let’s say a client sends a lot of traffic to a server. If the servers are set up with a load balancer, when the traffic increases, it will auto-scale to distribute traffic across multiple servers. As a result of auto-scaling, the resources are being used for unwanted traffic, which can increase your bill and render the server unavailable later due to heavy loads.
What is a Rate Limiter?
Google says Rate Limiter is In computer networks, It used to control the rate of requests sent or received by a network interface controller.
The maximum number of client requests that can be sent in a given time frame can be regulated using a rate limiter. that the rate limiter restricts requests to 5 per second only. Within that second, the client may send up to five requests, which the server will process. The rate limiter will, however, interrupt the sixth request and deliver an HTTP 429 response status code, signifying that the client has sent too many requests.
What is the best place to place the rate limiter?
The number of requests coming from the same user, IP address, etc. must be counted, hence a counter is required.
If the counter goes over the limit, the request is turned down.
Rate limiting is frequently implemented using in-memory caching, like Redis, for example.
INCR and EXPIRE are the two commands it provides.
- INCR: It increases the stored counter by 1.
- EXPIRE: It sets a timeout for the counter. If the timeout expires, the counter is automatically deleted.
Rate restriction should not be used for each server or service. The resources are once again squandered on unnecessary traffic if a service is only permitted to provide 5 requests per second since the other services would not know whether this service has already fulfilled those 5 requests.
You may use a proxy to act as an API gateway without having to set rate limits for every service. The API gateway is aware of the amount of queries that are permitted for each individual service once a client sends requests to that service. As a result, the API gateway may monitor the count.
Advantage of using Rate Limits
- Keeping servers from being overloaded.
- Increasing an API’s performance and availability (unlimited requests cause the server’s performance to suffer and the API to run slowly; limiting the amount of requests will stop this). guarantees that each user is given fair and acceptable use without interfering with other users, as well).
- To thwart DoS/DDoS attacks (however a rate limiter is not the ideal choice because the traffic might force it to surpass and the regular user requests won’t reach the server because the rate limiter is flooded).
- To avoid unauthorized billing (When working on a cloud environment or using third-party APIs, requests can be limited to reduce costs).
