#apache-flink · 8sync News

Apache Flink 2.3.0 Release Announcement

Apache Flink 2.3.0 is now available, implementing 15 FLIPs with major improvements across SQL, connectors, and runtime. New SQL operators FROM_CHANGELOG and TO_CHANGELOG bridge append-only and dynamic changelog tables. Materialized tables gain DDL parity with regular tables and fine-grained refresh control via a new START_MODE clause. The SinkUpsertMaterializer is reworked with an explicit ON CONFLICT clause and watermark-based compaction to reduce state size. A new native S3 filesystem plugin built on AWS SDK v2 replaces Hadoop/Presto-based connectors with non-blocking I/O and zero Hadoop dependencies. Runtime improvements include adaptive partition selection for backpressure handling, watermark alignment redesign for faster backlog processing, checkpointing during recovery from unaligned checkpoints, and application-level lifecycle management with a new Web UI Applications tab.

Confluent: Data in motion

Batch customer data platforms can't capture user intent as it forms — by the time a nightly sync completes, the intent moment is gone. A streaming-native architecture built on Apache Kafka and Apache Flink handles the full spectrum of personalization latency windows, from sub-100ms real-time bidding to multi-day email campaigns, using the same four-job pipeline: connect, stream, process, and govern. An AI-native layer (Confluent Intelligence) sits on top, enabling streaming agents with MCP tool-calling, a real-time context engine for LLMs, and built-in ML functions (ML_PREDICT, AI_COMPLETE) for embedding, ranking, and generative copy — all running as Flink jobs with exactly-once semantics and full lineage. The guide covers three production patterns (retail product recommendations, media feed personalization, cross-channel cart abandonment orchestration), a five-capability vendor evaluation framework, and a three-phase rollout roadmap from streaming backbone to autonomous agentic personalization.

How to Eliminate Training-Serving Skew in MLOps (2026)

Training-serving skew — the divergence between features used during model training and those seen at inference time — silently degrades ML accuracy and doubles infrastructure costs. The solution is a unified kappa architecture: compute features once in Apache Flink, dual-write to an offline store (Apache Iceberg or Delta Lake) for training and an online key-value cache for serving. DoorDash measured a 35.7% feature-value mismatch in their dual-pipeline setup; Netflix replaced a $93M/year dual-pipeline backfill with a $2M/year kappa replay. The reference architecture covers Kafka ingestion via Confluent's Kora engine, serverless Flink with event-time watermarks and exactly-once semantics, Tableflow for automated Iceberg/Delta materialization, and Stream Governance for schema enforcement and lineage. A tooling comparison covers Databricks, SageMaker+Kinesis, Tecton, Feast, and Confluent, with a decision framework based on latency requirements, existing stack investment, and pipeline fragmentation. The post is authored by a Confluent employee and promotes the Confluent Data Streaming Platform throughout.