Can a Local LLM Run My AI Assistant?

A developer replayed 27 real production tasks from a 90-tool personal AI agent (Jarvis) against two local models, benchmarking them against a frozen Claude baseline. On a single RTX 3090 with a 16K context window, qwen3-coder:30B scored 22.8/100 vs Claude's 89.4, with malformed tool-call syntax appearing in 25.9% of answers. After upgrading to three RTX 3090s (72 GB VRAM) and running a 122B MoE model at 256K context, the score jumped to 80.0/100 — 89.4% of Claude's performance — with zero malformed tool calls. The local model costs $0.000969 per task vs Claude's $0.763, roughly 787× cheaper. The author switched Jarvis to the local model and cancelled Claude Max, while noting the key caveat: model size and context window changed simultaneously, making it impossible to isolate which factor drove the improvement. The hypothesis is that context window size mattered more than parameter count, since the most characteristic failures in round 1 match what you'd expect from truncated tool schemas.

Nguồn: https://towardsdatascience.com/can-a-local-llm-run-my-ai-assistant. 8 Sync News chỉ tóm tắt và dẫn link; bản quyền nội dung thuộc tác giả và nguồn gốc.