Designing a URL shortener sounds trivial until you're tasked with...

@geekyAyushi
Ayushi Choudhary@geekyAyushi
10 views May 13, 2025 ~2 min read
Advertisement
1
Designing a URL shortener sounds trivial until you're tasked with generating 100K unique URLs per second, with low latency, strict correctness, and multi-tenant isolation.

Over the weekend, I explored a brilliant breakdown of how Rebrandly engineered for this scale 🧵
Media image
2
📄 Based on:
"Distributed TinyURL Architecture" by @animesh3436
(rb.gy/svwros)

Here’s what I learned 👇
3
The challenge is that you need to:
- generate 100K short URLs per second
- guarantee uniqueness
- ensure sub-100ms latency
- be fault-tolerant, horizontally scalable, and tenant-aware

Not so "Tiny" anymore?
4
Sol 1: Async Write Architecture

- short URLs are generated and returned instantly
- writes are pushed to DynamoDB later via SQS + Lambda

Fast! Yes, but risk of collisions lies with no uniqueness guarantee at generation time.

A great example of trading correctness for speed.
5
Sol 2: Sync uniqueness check

- check if short URL exists
- if not, insert and return to user
- guarantees correctness

But, doesn’t scale. It adds ~10ms latency per write, leads to bottlenecks under concurrent load. Thus, not feasible for 100K/sec.
6
Scaling isn't just about speed. It’s about where you enforce constraints.

To achieve correctness at scale, shift guarantees to the infra level.

That’s what the third architecture gets right.
7
✅ Distributed Workers + DynamoDB Transactions

- coordinator splits work across ECS workers
- each worker handles 10K+ URLs using batched DynamoDB TransactWriteItems()
- Redis is used to cache batches + recover on failure
- transactions ensure atomic uniqueness per batch
Media image
8
This architecture combines:
- Cloud-native building blocks (Lambda, SQS, ECS, DynamoDB)
- Batched writes with transactional guarantees
- Fault-tolerant caching using Redis
- Concurrency-safe design choices

Really helped me understand how "simple" features evolve at scale.
9
Got me thinking, how would something like this be built without AWS primitives?
Would love to hear thoughts on:
- PostgreSQL + advisory locks
- Redis Streams + Lua
- Kafka with idempotent writes & deduplication

Open to perspectives, still learning my way through these ideas.
Actions
What You Can Do
  • Export as PDF or Markdown
  • Batch Export to Notion
  • Bookmark & Highlight
  • Screenshot Tweet
Create Free Account

Includes 7-day Premium trial

Advertisement