Built fail‑closed rate limiting and anti‑abuse
Built fail‑closed abuse controls — 22 Redis‑backed rate limiters,
Cloudflare Turnstile, request idempotency and an origin lock — so
the platform sheds bots and floods instead of trusting its callers.
2025

Situation. A public directory of companies and professionals is a
target the day it goes live. Scrapers want the data, spam accounts
want the reach, and an endpoint that costs the platform real money
to serve — search, export, anything that touches an external API —
is worth abusing simply because it is free to call. None of that is
malice aimed at this platform in particular; it is background
weather on the open internet.

Task. The expensive and abusable paths needed limits that hold under
pressure, including the pressure of the limiter's own dependency
being unavailable.

Action. There are 22 rate limiters, each constructed for the path it
protects rather than one global cap, because a login attempt, a
search and a bulk export are abusive at wildly different rates.
State lives in Redis so a limit is shared across instances instead
of being per‑process and trivially escaped. The important decision
is what happens when Redis is not there: the limiters fail closed.
Traffic is refused rather than waved through, which is the less
convenient answer and the only defensible one. Around them sit
Cloudflare Turnstile on the paths worth challenging, 351 lines of
idempotency middleware so a retried write does not become two, an
origin lock that refuses requests not arriving through the front
door, and a bespoke challenge on the directory itself.

Result. Abuse gets expensive for the abuser and cheap for the
platform, and an outage in the limiter's own store degrades into
refusal rather than into an open door. Failing closed does mean a
Redis problem becomes a user‑visible problem — accepted
deliberately, because the alternative is a Redis problem becoming a
billing one.

Categories:
1Backend Engineering	/categories/backend/	engineer.company	70
1APIs & Integration	/categories/api/	engineer.company	70
1Reliability & Backups	/categories/reliability/	engineer.company	70
1Networking & VPN	/categories/networking/	engineer.company	70
1Security	/categories/security/	engineer.company	70
1Performance Tuning	/categories/performance/	engineer.company	70

Services:
1Backend & API Development	/services/backend-development/	engineer.company	70
1Site Reliability & Monitoring	/services/site-reliability/	engineer.company	70
1Security & Access Management	/services/security-access/	engineer.company	70
1AI Agent Security	/services/ai-security/	engineer.company	70

hThis page on the web	URL:https://engineer.company/portfolio/built-fail-closed-rate-limiting-and-anti-abuse/	engineer.company	70
