Bridge Queries in Redpanda SQL

Redpanda SQL introduces bridge queries, a feature that lets you query live Redpanda topic data and historical Iceberg table data together through a single virtual SQL table. This eliminates the classic streaming-to-lakehouse tradeoff between data freshness and Parquet file quality. Previously, frequent flushes to Iceberg were needed for low-latency analytics, resulting in thousands of tiny Parquet files, poor compression, high S3 costs, and constant compaction overhead. With bridge queries, the topic itself serves the freshness gap at query time, so Iceberg flushes can happen on a longer cadence (hours instead of seconds/minutes), producing larger, analytics-optimized Parquet files. Configuration involves raising the iceberg lag target and optionally the flush size threshold. The result is lower query latency, reduced S3 costs, and no need for a compaction service.

Source: https://www.redpanda.com/blog/bridge-queries-in-redpanda-sql. 8 Sync News only summarizes and links out; content copyright belongs to the authors and original sources.