In the ever-evolving world of tech, the story of Datadog's production migration is a fascinating glimpse into the potential and challenges of AI-assisted engineering. This narrative, shared by Datadog engineer Arnold Wakim, offers a unique perspective on how AI tools like Claude and Cursor can revolutionize critical system upgrades.
The original Stream Router, an API managing metrics, faced a significant challenge as its routing table expanded. The key-value (KV) database model, while initially efficient, began to slow down dramatically due to transaction size limits. This is where Datadog's innovative approach comes into play.
Redefining Relationships
Datadog engineers tackled the issue head-on by redesigning the schema to reflect relationships between domain entities. By introducing foreign keys, they offloaded complex logic from the application code, a move that not only streamlined the system but also set the stage for a more efficient migration process.
AI as a Refactoring Tool
The real game-changer, however, was the use of AI. Datadog employed Claude and Cursor to accelerate a systematic, test-driven refactoring. Instead of autonomous code generation, the AI models were provided with the old implementation, the new schema, and failing tests. This iterative process, guided by the test suite, allowed the models to generate code that was not only correct but also continuously improved with each iteration.
Three Pillars of Success
Wakim identifies three key elements that made this migration a success: strong code modularity, a comprehensive test suite, and a parallel infrastructure. The modularity of the new Stream Router meant that the API could be implemented on PostgreSQL without disrupting other parts of the system. The test suite, a critical component, served as a rigorous pass/fail criterion for every AI-generated change. And the parallel infrastructure, with two independent Stream Router instances running side by side, provided a safety net, allowing for continuous comparison and immediate alerts in case of divergence.
Migration Phases
The migration itself was a carefully orchestrated three-phase process. First, Claude was used to describe the intent of each key function, providing a high-level understanding of the code. Next, focused prompts were crafted to fix failing tests, guiding the AI to generate correct code. Finally, a "blue/green" deployment approach was used, with the new and old Stream Routers running in parallel and being continuously compared.
Where AI Fell Short
While the migration was largely successful, Wakim also highlights areas where AI assistance could be improved. Higher-level prompts were less effective, and Claude often generated correct but suboptimal queries, requiring human intervention for niche optimizations. Additionally, the iterative process of feeding the AI with test output, code context, and schema information resulted in high token consumption.
Impressive Results
The results of this migration are impressive. Operation times dropped from a staggering 45 minutes to a mere second, and latencies decreased significantly. Data storage became up to 40 times smaller, and the switch to PostgreSQL and DuckDB simplified relationship handling and improved query efficiency. CPU and memory usage decreased, and database costs were slashed by a remarkable 90%.
Conclusion
In my opinion, Datadog's story is a testament to the power of AI in engineering, but it also underscores the importance of human oversight and well-designed systems. The strength of their test suite, as Wakim notes, was the key success factor, determining the level of trust they could place in AI-generated code. This balance between AI assistance and human expertise is a fascinating aspect of this narrative and one that I believe will shape the future of tech innovation.
