Megabytes in milliseconds: How FireTV uses parallel queries and vertical partitioning to serve millions of customers in Amazon DynamoDB

Amazon FireTV re-architected its DynamoDB-backed Continue Watching data model to overcome the 400 KB item size limit, reduce throttling, and cut write costs. The team moved from a single compressed item per profile to vertical partitioning (one item per content entry), then addressed the resulting 1 MB query page limit by prepending sort keys with MurmurHash3 hashes to enable evenly distributed, parallel segmented reads. The redesign eliminated item size limits, dropped write costs from 50 WCUs to 1 WCU per update (97% reduction), and kept reads averaging 8ms and under 50ms at p99 regardless of profile size. The post also covers tradeoffs (loss of fine-grained access patterns), how to choose segment counts, and their migration/validation strategy using shadow reads and an EMR-based comparison job across billions of records.

Source: https://aws.amazon.com/blogs/database/megabytes-in-milliseconds-how-firetv-uses-parallel-queries-and-vertical-partitioning-to-serve-millions-of-customers-in-amazon-dynamodb. 8 Sync News only summarizes and links out; content copyright belongs to the authors and original sources.