Perplexity’s database is 5x faster... Two people built it #dynamodb #aws #perplexity
Read the Portuguese translation →
Author: Better Stack · Watch on YouTube
Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.
Original language: English
Original video: Perplexity’s database is 5x faster… Two people built it #dynamodb #aws #perplexity · Channel: Better Stack · 01:35 · Watch on YouTube
Topics: AI agents, Data, databases and embeddings, Software engineering with AI, Model inference and optimization
Summary
Section titled “Summary”Perplexity replaced DynamoDB with Cobble DB, a custom key-value hot store built by two engineers. DynamoDB’s per-byte billing and limited control over partitions, caching, and replicas became challenges as Perplexity’s index grew. Cobble DB uses RocksDB nodes and parallel reads across replicas, with the option to retry a slow read on another replica. In production, batch read latency fell substantially, and Perplexity’s cost model estimates Cobble DB is at least 20% cheaper. The roughly 40,000-line Rust system was built in two months with help from AI agents, while engineers remained responsible for its architecture and review.
Key points
Section titled “Key points”- A Perplexity API request fetches around 100 to 120 page keys, divided into batches of 10 to 20.
- Each record is around 50 kilobytes, and DynamoDB charges for every byte read or written.
- Perplexity had limited control over partition placement, caching memory, and which replica handled a request.
- A slow replica could delay an entire batch read.
- Cobble DB is Perplexity’s custom key-value hot store, with each node running on RocksDB.
- Cobble DB groups keys by partition and reads across replicas in parallel, retrying a slow read on another replica.
- Median batch read latency dropped from 31.4 milliseconds to 5.6 milliseconds; P99 dropped from 123 milliseconds to 24 milliseconds.
- Perplexity’s cost model says Cobble DB is at least 20% cheaper than DynamoDB.
- Cobble DB is around 40,000 lines of Rust and was built in two months by two engineers with AI agents.
- AI agents helped with fixes, tests, monitoring agents, documentation, and tracking CI and review gates; engineers designed the architecture and reviewed changes.
Chapters
Section titled “Chapters”| Start | Chapter | Summary |
|---|---|---|
| 00:00 | Why Perplexity moved away from DynamoDB | The video explains Perplexity’s high-volume page-key reads and how DynamoDB’s per-byte billing and managed setup became limitations. |
| 00:32 | How Cobble DB handles reads | Perplexity’s replacement uses RocksDB nodes, partitioned keys, parallel replica reads, and retries when a replica is slow. |
| 01:03 | Performance, cost, and the role of AI agents | Cobble DB improved batch read latency and is estimated to cost less, while two engineers used AI agents to help build and maintain the system. |
Quotes
Section titled “Quotes”Perplexity just fired Dynamo DB and its replacement was built by two engineers. — 00:00
So Perplexity built its own key-value hot store called Cobble DB. — 00:32
In production, median batch read latency dropped from 31.4 milliseconds to 5.6. — 01:03
And it was built in two months by two engineers plus AI agents. — 01:03
The engineers still designed the architecture, reviewed the changes — 01:35
Concepts and entities
Section titled “Concepts and entities”Perplexity, DynamoDB, Cobble DB, RocksDB, Rust, AI agents
Full transcript (original language)
Section titled “Full transcript (original language)”00:00 Perplexity just fired Dynamo DB and its replacement was built by two engineers. That’s not even the crazy thing here, so let’s break it down. Perplexity search back in fetches a lot of pages at once. A single API request needs around 100 to 120 page keys split into batches of 10 to 20 with each record around 50 kilobytes. And on Dynamo DB, every byte you read or write is billed. So as Perplexity’s index grew, so did the bill. But the cost wasn’t the only problem because Dynamo DB is managed.
00:32 Perplexity couldn’t control which machine held a partition, how much memory was used for caching, or even which replica answered a request. And when you’re reading a whole batch at once, one slow replica can hold everything up. So Perplexity built its own key-value hot store called Cobble DB. Each node runs on RocksDB. Keys are grouped by a partition, reads happen across replicas in parallel, and if one replica is too slow, Cobble DB can send that same read to another one. And the difference here was massive. In
01:03 production, median batch read latency dropped from 31.4 milliseconds to 5.6. P99 dropped from 123 milliseconds to just 24, so roughly five times faster. While Perplexity’s own cost model says it’s also at least 20% cheaper than Dynamo DB at every commitment lately. But there’s the part everyone was arguing about. Cobble DB is around 40,000 lines of, you guessed it, Rust. And it was built in two months by two engineers plus AI agents. Those agents
01:35 wrote fixes, tests, monitoring agents, and docs, and even tracked CI and review gates. But they weren’t running production. The engineers still designed the architecture, reviewed the changes, and explicitly pushed on this button. If you want more engineering breakdowns like this, be sure to subscribe to the Better Stack channel.