API Rate Limiting

From enterprise software to student projects, APIs are essential for enabling communication across systems, but how do developers protect their infrastructure and cloud budgets from floods of requests?

One of the most effective solutions is API rate limiting.

What Is API Rate Limiting?

Application programming interface (API) rate limiting is setting a cap on the number of requests a user or client can make to an API within a given time frame. This limit prevents users from overconsuming resources, allowing for smooth system performance and network traffic flow.

If users' API requests exceed the set threshold, the server sends an error message or blocks additional ones. For example, if you are building a chat service, you might want to limit users to a maximum of 20 messages per minute. Any further messages get delayed or rejected until a minute has passed.

How Does API Rate Limiting Work?

Implementing API rate limiting begins at the codebase or via an API gateway, with techniques tailored to system requirements. Limits can be:

• Request-based (such as calls per minute) 
• Resource-based (by data volume)

The first step involves setting quotas. These include:

• Number of requests allowed
• Time window (per second, minute, hour, day, or custom)
• Enforcement action when exceeded (such as delay or rejection)

Quotas often vary by plan tiers, with the most common being free, pro, and enterprise. Higher tiers enjoy larger allowances, typically under a service level agreement (SLA).

Next, the API tracks client requests in a fast, centralized store, like Redis. On each API call, the system retrieves the client's counter and compares the count against the quota. If under the limit, it increments the counter and allows the request. Otherwise, it rejects it.

To enforce limits effectively, the API must correctly identify each client using one of the following methods:

• IP address: This involves using clients' IP addresses as the identifier for counting calls. It prevents overwhelming traffic from a single IP source, protecting API security and performance. You can track IP addresses using open-source libraries and frameworks such as Flask's remote_addr attribute for Python or request-ip for Node.js.
• API keys: This method uses API keys as unique identifiers. Setting limits per API key ensures fair usage and prevents abuse. It works best if you plan to expose your API endpoints to the public.
• JSON Web Tokens (JWTs): JWTs provide powerful session-based tracking, allowing servers to identify users without database lookups. They work best for authenticated APIs, enabling fine-grained rate limits.

API Rate Limiting Example

The following Flask app demonstrates how to implement basic API rate limiting using the flask-limiter extension and Redis.

from flask import Flask, request, jsonify

from flask_limiter import Limiter

from flask_limiter.util import get_remote_address

import redis

app = Flask(__name__)

try:

    redis_client = redis.Redis.from_url("redis://localhost:6379")

    redis_client.ping()

except redis.ConnectionError:

    raise Exception("Redis connection failed")

limiter = Limiter(

    app=app,

    key_func=get_remote_address,

    storage_uri="redis://localhost:6379"

)

@app.route("/login", methods=["POST"])

@limiter.limit("5 per minute")

def login():

    return jsonify({"message": "Success"})

@app.errorhandler(429)

def ratelimit_handler(e):

    return jsonify(error=f"Too many attempts. Try again in {int(e.retry_after)} seconds."), 429

if __name__ == "__main__":

    app.run(port=5000)

This example sets a limit of 5 login attempts per minute per IP address. If the user exceeds the limit, the server returns an error 429 with a retry time.

Here is an example of a server response if the API call falls within the limit:

HTTP/1.1 200 OK

Content-Type: application/json

RateLimit-Limit: 5

RateLimit-Remaining: 4

RateLimit-Reset: 1715276060

{

  "message": "Success"

}

However, if the user exceeds the limit, the server will block the request and return an error message:

HTTP/1.1 429 TOO MANY REQUESTS

Content-Type: application/json

Retry-After: 60

RateLimit-Limit: 5

RateLimit-Remaining: 0

RateLimit-Reset: 1715276060

{

  "error": "Too many attempts. Retry in 60 seconds."

}

Benefits of API Rate Limiting

API rate limiting provides multiple benefits for developers and platform owners, including:

Protects Against Threat Actors

Rate limiting provides a first line of defense for your software against automated bot attacks designed to overwhelm your API and take your service down. These high-volume attacks make the app inaccessible to legitimate users.

By placing strict per-client limits, you prevent individual users from consuming excessive API resources or degrading performance for others.

Improves Resource Management

Even legitimate surges from factors like viral usage or unexpected user growth can exhaust your system's memory, CPU, and database connections, leading to latency spikes. Rate limiting smooths sudden demand surges, keeping API responses consistent and maintaining system responsiveness.

With consistent enforcement, APIs remain stable during high-traffic events, which is critical in applications with heavy usage, such as enterprise SaaS.

Saves Money

Most cloud services bill per resource usage (such as per CPU-second, gigabyte of bandwidth, or database transaction). High or unexpected request volumes can rapidly drive spending higher than expected.

Effective API rate limiting prevents runaway expenses by capping the maximum allowed usage, which makes monthly spending predictable.

Allows for Tier-Based Pricing

Rate limiting enables you to control access to resources based on subscription tiers.

For example, free users might have limited API calls, such as 10 per day. Paid users could have a higher limit of 200 requests per day, and enterprise customers could have custom SLAs. Users who reach their quota receive information about the next tier, incentivizing self-service plan upgrades.

Rate Limiting Mechanisms

Developers have multiple strategies for implementing API rate limiting, each with its benefits and limitations. These include:

Fixed Window Counter

In this approach, the API enforces a hard limit that resets after the end of the time window. For example, an API could limit requests to 50 in a 24-hour period, resetting at 3 am every day. When the user hits the limit before the end of the 24 hours, they must wait until 3 am for it to reset.  

Sliding Window Counter

The sliding window algorithm improves on the fixed window approach by smoothing out traffic across time. .  Instead of resetting limits at hard intervals, it breaks each window into overlapping sub-windows and sums the requests made within that sub-window.

For example, if you want a limit of 120 calls per minute, you can divide that one-minute window into six sub-windows of 10 seconds each. Each request is counted in its corresponding sub-window, and the total is evaluated across the full minute.

If the total is under 120, the request is allowed. However, if it hits the limit, the API call is rejected or delayed until some of the older sub-window counts expire.

This method prevents sudden bursts of traffic just before a window reset and allows more flexible, real-time enforcement. 

Sliding Log Counter

This method logs timestamps for user requests and uses the logs to enforce limits with higher precision. It uses a sliding window of time for tracking, such as one minute, with the count incrementing after each call made within the sliding window. If the count exceeds the set limit, additional ones are rejected. 

Token Bucket Algorithm

In the token bucket algorithm, the user is assigned tokens to make requests. These tokens are placed in a fixed bucket and consumed by each call.

This method allows users to maximize usage within the available timeline. Also, it's effective for managing traffic on sites that experience spikes, as it lets users take advantage of available capacity for temporary bursts. This approach is well-suited for platforms with variable user activity, such as social media and marketplace platforms.

Leaky Bucket Algorithm

This approach enforces a strict, constant output rate by queuing requests in a fixed bucket size and processing them regularly in the first-in-first-out (FIFO) order. The bucket fills up with requests, with processed ones being removed steadily. If the number of incoming calls exceeds the processing rate, the bucket overflows, rejecting additional ones.

IP Throttling

IP throttling controls how many requests a single IP can make within a set period. Every incoming request is mapped to its source IP, with the counter incrementing for that address. If the count exceeds the set threshold, the server blocks subsequent calls until the window resets.

Because IP throttling treats addresses equally, it can be effective against botnets and DOS attacks targeting login or signup endpoints.

Best Practices for Implementing API Rate Limiting

Implementing API rate limiting involves balancing performance, protection, and user experience. 

Below are the best practices you should follow. 

Determine the Right Quota

Start by understanding your infrastructure and user needs, then align your rate limit with available capacity and business objectives. Also, figure out what sustainable rate your database can cache and your network can handle without performance degradation.

Next, factor in your product's SLA and usage patterns. For example, OpenAI's API might safely serve thousands of calls per second from paid clients, whereas a student's hobby project may only need a few dozen per minute to protect against accidental overload.

Finally, map limits to subscription tiers or user roles to allow paid users higher quotas.

Pick the Right Limiting Method

Choose a rate-limiting method that matches traffic demands based on the following: 

• Leaky bucket algorithm for consistent traffic
• Token bucket for infrequent usage bursts
• Fixed window counter for apps with typically low traffic
• Sliding log counter for total precision, even under erratic workloads

You can also combine these methods in a single application. For example, IP throttling with a fixed window on public endpoints, a token bucket to handle bursts, and a sliding log counter for sensitive workloads.

Test Rate Limiting

After configuring rate limits, make sure it works as intended before pushing it to production. Simulate tests using tools like JMeter, Beeceptor, or k6 to generate traffic patterns and observe how the APIs behave to understand how rate limiting works under high traffic.

Inform Users of Rate Limit Errors

During development, implement ways to ensure users understand the number of requests remaining or why you denied them. For example, if you are building an AI tool like an image generator, you should indicate the number of attempts on the dashboard.

Build Mechanisms To Identify Malicious Traffic

Implement mechanisms to determine whether traffic is genuine or malicious, such as login attempts from unusual locations or devices. Some of the best methods include:

• Anomaly detection
• Signature-based detection
• System monitoring
• Network traffic analysis

These methods provide high detection accuracy, helping you differentiate between real and malicious traffic to evolve your limits appropriately.

Frequently Asked Questions