How Much Does a Local LLM Actually Cost to Run? I Measured Every Watt on Apple Silicon

A hands-on measurement of the electricity cost to run five local LLMs on an Apple M3 Ultra Mac Studio with 96 GB unified memory, using a custom OpenAI-compatible proxy (TokenWatt) calibrated against a wall-socket power meter at $0.31/kWh. The key finding: parameter count is a poor predictor of cost per token. A 120B mixture-of-experts model costs ~$0.109/million output tokens, while a 27B dense model costs ~$0.554 — nearly five times more — because MoE models activate only a fraction of their weights per token, dramatically reducing memory bandwidth usage and improving throughput. Real-traffic data over 30 days confirms the ordering, with the dense 27B costing roughly 10x more per token than comparable MoEs. The post also covers Apple Silicon-specific considerations: idle power overhead from keeping large models resident, the 96 GB memory ceiling, and a frank comparison with cloud API pricing.

Nguồn: https://towardsdatascience.com/how-much-does-a-local-llm-actually-cost-to-run-i-measured-every-watt-on-apple-silicon. 8sync News chỉ tóm tắt và dẫn link; bản quyền nội dung thuộc tác giả và nguồn gốc.