The open AI-native stack on Kubernetes: From open models to distributed inference and agents

Google Cloud outlines an open-source, Kubernetes-native stack for running AI workloads at scale, covering RL post-training, distributed inference, and agent execution. Key components include OpenRL for RL fine-tuning, llm-d 1.0 for distributed inference, KubeRay for distributed Ray workloads, and Agent Substrate plus gVisor for sandboxing untrusted agent code. The Endpoint Picker in llm-d doubled KV cache hit rates from 35% to 70% and cut median time-to-first-token by 70-85% on multi-turn agent workloads. Testing Gemma 4 with OpenRL's RL loops on Harvey.ai's Legal Agent Benchmark lifted accuracy from 50% (SFT) to 70%. The projects are presented ahead of KubeCon + CloudNativeCon North America 2026.

Nguồn: https://cloud.google.com/blog/topics/developers-practitioners/the-open-ai-native-stack-on-kubernetes-from-open-models-to-distributed-inference-and-agents. 8 Sync News chỉ tóm tắt và dẫn link; bản quyền nội dung thuộc tác giả và nguồn gốc.

Đề xuất cho bạn

Cast AI1 Hot5 phút1 giờ trước

Karpenter Enterprise for Azure AKS & NAP

Using Karpenter on AKS or NAP? Power it up to provision to app usage, not requests, with Cast AI’s Karpenter Enterprise Suit for Azure.

#kubernetes#finopsNguồn

CNCF1 Hot9 phút4 giờ trước

One flag, many guardrails: Making a platform safe enough to delegate

This post draws on a talk co-presented with Julien Semaan of Kubex at KubeCon + CloudNativeCon Europe 2026; GitOps Gone Wild: Hardening Delivery Pipelines for…

#ai-agent#kubernetes#platform-engineeringNguồn

Flavio Copes1 Hot12 phút6 giờ trước

API pricing vs subscription: what my month of Codex would cost on the API

I priced 30 days of my real Codex usage at API rates. A $100 plan covered $745 of API-equivalent work. Here's the math, the script to run on your own logs, and when the API still wins.

#ai-agent#llm#chatgptNguồn

Daily Dose of Data Science | Avi Chawla | Substack52 Hot8 phút3 ngày trướcTop

7 MoE routing strategies in production, clearly explained

+ a practical learning checklist to learn AI engineering.

#ai-inference#deepseek#mixture-of-expertsNguồn

Latent Space1 Hot4 phút22 giờ trước

What Frontier Labs Buy: A Data Deep Dive with Handshake AI — Garrett Lord & Team

Handshake went from launch to $1B in one year pushing the envelope of specialized data for frontier labs. Our first deep dive into a data lab with specific examples across Finance, Bio, and Alignment

#machine-learning#ai-agent#reinforcement-learningNguồn

Flavio Copes137 Hot6 phút9 ngày trước

How to run an LLM locally

Run an LLM locally with Ollama. See how much memory you need, where LM Studio and llama.cpp fit, and how to call the model from code.

#self-hosting#ai-inference#ollamaNguồn

Daily Dose of Data Science | Avi Chawla | Substack152 Hot7 phút10 ngày trước

RAG vs. Vectorless RAG, clearly explained!

...explained visually (with tradeoffs in production).

#llm#rag#vector-searchNguồn

Machine Learning Pills1 Hot9 phút23 giờ trước

Issue #147 - Reinforcement Learning Explained with a Practical Python Example

How agents learn from rewards, why exploration matters, and how to build your first Q-learning agent.

#python#reinforcement-learning#numpyNguồn