AI On Your Gaming PC

Strata, a project by developer Niko1221, lets a 125-billion-parameter LLM run on gaming-grade hardware rather than requiring a datacenter GPU farm. It uses Qwen3.8 model variants, including a mixture-of-experts version with 24,576 small experts where only ten are active per token. VRAM acts as a cache for frequently used experts, the bulk of the model stays in system RAM, and a roughly 29 GB lookup table lives on the SSD. Speculative decoding via multi-token prediction boosts throughput, with an RTX 5070 (12 GB VRAM) reportedly hitting 50-90 tokens per second depending on quantization. It exposes OpenAI- and Anthropic-compatible local APIs, though answer quality trails larger hosted models, and realistic requirements are around 32 GB RAM, 12 GB VRAM, and 80 GB storage.

Nguồn: https://hackaday.com/2026/10/04/ai-on-your-gaming-pc. 8 Sync News chỉ tóm tắt và dẫn link; bản quyền nội dung thuộc tác giả và nguồn gốc.