How Shopify built a continual learning loop with PyTorch and vLLM – PyTorch

Shopify describes a continual learning loop built with PyTorch and vLLM that compresses production failures into model weights daily. The pipeline defines a quality rubric, calibrates an LLM judge with DSPy optimizers (GEPA, ACE), improves a frontier-model baseline through an autoresearch harness-optimization loop, then mines hard negatives from production traffic, heals them with a panel of reasoning models plus human annotation from Toloka, and trains a smaller model via supervised fine-tuning and GRPO. Prompt gist compression shrinks the GraphQL agent's system prompt from ~6,000 to ~1,500 tokens. The result: the fine-tuned model beats frontier-model quality, cuts serving costs by 96% (from an estimated $27M to $1M/year), reduces end-to-end latency by ~38%, and uses ~14% fewer GPUs at 350 requests/minute.

Source: https://pytorch.org/blog/how-shopify-built-a-continual-learning-loop-with-pytorch-and-vllm. 8 Sync News only summarizes and links out; content copyright belongs to the authors and original sources.