Set API Rate Limits Above P95, OWASP, NGINX and Azure for Developers

API rate limiting caps how many requests a client can make in a given window, returning a 429 Too Many Requests response with a Retry-After or X-RateLimit header once that cap is hit. Production APIs should enforce limits at multiple scopes at once, per IP address, per authenticated identity, and per endpoint, rather than relying on a single global counter.

---

TL;DR:

• Enforce rate limits across multiple scopes like IP address, user identity, and endpoint to prevent abuse and ensure fairness.
• Use algorithms like token bucket or sliding window for better accuracy and burst handling, avoiding the boundary-spike issues of fixed windows.
• Regularly test and monitor limits under realistic traffic conditions, including intentional overloads, to verify proper 429 responses and client backoff behavior.
• Combine at-edge gateway limits with application-level rules to protect against automation, abuse, and costly resource consumption.
• Always tune and review rate limits based on actual traffic data and usage patterns, adjusting thresholds for different user tiers and scenarios.

---

Table of Contents

• What rate limiting is and how it differs from throttling
• Why rate limiting matters for security and reliability
• Algorithms: token bucket, leaky bucket, and window counters
• Implementing limits at the edge, gateway, and app layer
• Testing and monitoring your rate limits in production
• Security best practices and tuning rules from OWASP
• How to choose and tune rate limits in practice
• Common pitfalls and where teams get rate limiting wrong
• Get your rate limits tested before someone else finds the gap
• Sources
• FAQ

What rate limiting is and how it differs from throttling

Rate limiting sets a hard cap: once a client crosses it, the API rejects further requests until the window resets. Throttling takes a softer approach, delaying or queuing requests to smooth out traffic rather than blocking it outright. Combining both often produces the most resilient behavior, according to a comparison of the two strategies, since throttling absorbs small spikes while rate limiting stops sustained abuse.

Each approach suits different situations:

• Login endpoints: hard rate limits stop credential-stuffing attempts after a handful of failed tries.
• Tiered APIs: throttling lets paying customers burst above their baseline briefly instead of failing instantly.
• Public, unauthenticated endpoints: strict per-IP limits protect shared infrastructure from anonymous scraping.

Why rate limiting matters for security and reliability

Rate limiting is a frontline defense against denial-of-service attempts and credential-stuffing campaigns that hammer login or search endpoints with automated requests. Without caps, a handful of misbehaving clients or a single runaway script can degrade performance for everyone else on shared infrastructure.

• Security: blocks brute-force and scraping patterns before they exhaust backend resources.
• Reliability: keeps response times predictable by preventing any one consumer from monopolizing capacity.
• Cost control: caps exposure to third-party billing when your API calls metered downstream services or triggers expensive background jobs.

Rate limiting is a foundational control that has to be tuned and monitored rather than set once and forgotten, per OWASP's guidance on bot management: limits set too aggressively generate false positives that lock out real users, while limits set too loosely let abuse through.

Algorithms: token bucket, leaky bucket, and window counters

Four algorithms cover most production needs, each with a different tolerance for bursts and a different memory cost.

1. Token bucket: tokens refill at a fixed rate and each request consumes one, allowing controlled bursts while capping the long-run average.
2. Leaky bucket: requests queue and drain at a steady rate, smoothing bursts into an even output rather than allowing spikes through.
3. Fixed window: counts requests in a set interval (like one minute) and resets to zero, which is cheap to implement but allows a burst right at the window boundary to double the effective rate.
4. Sliding window or sliding log: tracks requests across a rolling interval instead of a hard reset point, avoiding the boundary spike problem entirely.

Sliding-window and token-bucket approaches avoid the boundary-spike issue that fixed-window counters create, according to NGINX's rate-limiting documentation, because there is no single reset instant for clients to exploit. Sliding log gives the highest accuracy but costs the most memory since it stores every request timestamp, while fixed window costs the least but is the least accurate.

For most APIs, token bucket or sliding window is the right default. Avoid naive fixed-window counters unless the traffic is low-stakes and simplicity outweighs precision. In distributed systems, a single centrally updated counter creates latency and contention: NGINX's documentation recommends token bucket with consistent hashing or per-shard counters plus periodic reconciliation instead.

Statistic: RFC 6585 specifies that a server enforcing rate limits commonly responds with status code 429 and may include a Retry-After header telling the client how long to wait, which standardizes how well-behaved clients should react across any algorithm you choose.

Implementing limits at the edge, gateway, and app layer

Enforcing limits at the edge or API gateway stops abusive traffic before it reaches application servers, while application-level limits let you apply business logic tied to user tiers or specific endpoints. Most production setups use both: a gateway-level ceiling for baseline protection and app-level rules for finer-grained quotas.

NGINX implements rate limiting with leaky-bucket semantics through limit_req_zone and limit_req, and its own module documentation notes that the burst and nodelay parameters need careful tuning to avoid unintended rejections or excessive queuing delays, per the ngx_http_limit_req_module reference. Shared memory zones must be sized correctly or the limiter silently drops tracking data under load.

Azure API Management applies limits through the rate-limit and rate-limit-by-key policies and can expose remaining-calls and total-limit headers so clients know when to back off, according to Microsoft's implementation guidance.

• Apply per-IP limits as a baseline against anonymous abuse.
• Apply per-key or per-user limits so one compromised account cannot exhaust shared capacity.
• Apply per-endpoint limits so expensive routes get tighter caps than cheap ones.

Pro Tip: Test your gateway config with a burst of concurrent requests before shipping it. A misconfigured nodelay flag can silently reject legitimate traffic instead of queuing it.

Testing and monitoring your rate limits in production

Load testing with tools that simulate concurrent bursts, not just steady traffic, is the only reliable way to confirm a limiter behaves as designed under real conditions. Run tests that intentionally exceed your caps and confirm the API returns 429 with a usable Retry-After value, then check that your own client code actually respects it.

• Instrument 429 rates alongside overall latency to catch limits that are too tight.
• Track retry behavior to confirm clients back off instead of hammering the endpoint again immediately.
• Watch bucket fill levels in real time to see how close traffic runs to the ceiling before it trips.
• Alert on abnormal 429 spikes and investigate whether the cause is an attack, a bug in a client, or a limit set too low.

Security best practices and tuning rules from OWASP

Rate limiting only works as a security control when it is scoped and tuned correctly. OWASP's cheat sheet is explicit that a login limiter should never key on the combination of IP and username together, because that design lets an attacker rotate through unlimited usernames from a single IP without ever tripping the limit, per the Bot Management and Anti-Automation Cheat Sheet. Use separate per-username and per-IP buckets and require both to stay under threshold.

• Keep 429 responses generic: never reveal which bucket triggered or how many attempts remain.
• Apply limits at multiple scopes (IP, session, endpoint) instead of one combined check.
• Treat IP-based limits as a baseline only, since shared or rotating IPs make them unreliable alone.
• Enforce per-identity or per-session limits on high-value endpoints regardless of IP behavior.

Pro Tip: Attackers use diagnostic details in error responses to fine-tune evasion, so a generic "too many requests" message is safer than one that names the limit or count remaining.

How to choose and tune rate limits in practice

Start from real traffic data rather than a guessed number: pull median and P95 request rates per client over a representative period, then set the limit above P95 with enough margin to absorb normal spikes without inviting abuse.

1. Baseline first: measure actual usage patterns before picking a number, since arbitrary limits either block real users or let abuse through.
2. Tier by authentication status: give anonymous traffic tighter caps than authenticated users, and give paying tiers higher ceilings than free ones.
3. Roll out gradually: use an error-budget approach, logging what would have been blocked before actually enforcing the cap, to catch false positives early.
4. Plan a rollback: keep a fast path to loosen or disable a new limit if it starts rejecting legitimate traffic at scale.

Note that if clients connect through shared infrastructure, understanding the difference between residential, mobile, and ISP proxy traffic helps explain why IP-based limits alone sometimes misfire on legitimate users behind carrier-grade NAT.

Common pitfalls and where teams get rate limiting wrong

The most common mistake is a single combined bucket for IP and username on login endpoints, which quietly defeats the whole point of the control. The second is leaking bucket diagnostics in error messages, handing attackers a tuning signal for free. The third is limiting public endpoints carefully while leaving internal or partner-facing APIs wide open, assuming trust where none is verified.

Managed gateways like Azure API Management handle the common cases well; custom limiters make sense once you need cross-service coordination a gateway policy cannot express. Either way, the only way to know your limits hold under real attack conditions is to test them like an attacker would.

— caleb

Get your rate limits tested before someone else finds the gap

Building rate limits is only half the job. Confirming they hold up against real attack patterns, distributed clients, and edge cases like combined-bucket flaws takes hands-on testing that automated scanners tend to miss. Earthshaker Security's API and endpoint testing combines automation with manual verification, so findings on your rate limiting, authentication, and access controls come with a clear, actionable fix rather than a raw scan output.

This kind of manual verification matters most for distributed rate limiting setups, multi-scope login protection, or anything that needs to hold up under compliance review. Engagements for security testing sometimes include a re-test after fixes are deployed, and clients typically communicate directly with technical staff, not sales teams. Check current engagement pricing to find the right starting point.

Sources

• Bot Management and Anti-Automation - OWASP Cheat Sheet Series
• Rate limiting with NGINX
• RFC 6585: Additional HTTP Status Codes
• Implement rate limiting in Azure API Management - Dev Proxy
• Advanced guide to rate limiting & API traffic management - LogRocket Blog

FAQ

How do I fix a rate limit reached error?

Check the response for a Retry-After header, which tells you how long to wait before your next request, a behavior RFC 6585 documents for 429 responses. Back off for that duration, then retry with exponential backoff if the limit persists, and reduce your request frequency or batch calls to stay under the cap going forward.

What is a good rate limit for an API?

There is no universal number: the right limit comes from your own baseline traffic, typically set above your P95 request rate with margin for normal bursts. Tiered APIs often give authenticated or paying users higher ceilings than anonymous traffic to balance protection with usability.

How do I implement API rate limiting?

Choose an algorithm, usually token bucket or sliding window for accuracy and burst tolerance, then enforce it at the gateway or edge layer using tools like NGINX's limit_req module or a managed service like Azure API Management. Apply limits at multiple scopes, per IP, per authenticated identity, and per endpoint, and return 429 with a Retry-After header when clients exceed them.

How do I rate limit an API to a fixed number of requests per minute?

Configure a fixed or sliding window counter set to that request count per one-minute interval, using a directive like NGINX's limit_req_zone or a gateway policy like rate-limit-by-key. A sliding window is generally preferable to a fixed window since it avoids letting clients burst past the cap right at the boundary between two windows, as NGINX's documentation explains.

Recommended

• For developers and CTOs