#gemma · 8sync News

A Three-Phase Factual Recall Circuit in Gemma-2B and Gemma-12B-IT

An empirical mechanistic interpretability study of factual recall in Gemma-2B and Gemma-12B-IT using activation patching. The research identifies a consistent three-phase circuit: Phase 1 (Storage) encodes facts as directions in the residual stream at the entity token position in early-to-mid layers, with the residual stream contributing 40× more causally than attention outputs. Phase 2 (Routing) moves the signal to the final token position via distributed attention heads with no single dominant head. Phase 3 (Readout) retrieves the answer in late layers without additional computation. The pattern replicates proportionally at the 12B scale, with routing becoming even more distributed. The study also highlights tokenizer-induced dataset drift as a methodological concern for cross-model comparisons, and proposes path patching and SAE analysis as natural next steps.

I ran a local LLM on my underpowered Chromebook, and it actually works

A writer ran a local LLM on a severely underpowered Chromebook (4GB RAM, Intel UHD 600, 32GB SSD) and found it actually works. After an initial crash with LLM Hub loading Ministral 3B, they succeeded using LM Playground with Gemma 4 E2B. Performance is slow but functional for offline brainstorming, light research, and structured queries. The app includes built-in web search and JavaScript tools without needing MCP setup. The experiment confirms that modern small models can run on very constrained hardware, with Termux/llama.cpp as a next step for more flexibility.

Open models, global networks: How AT&T and GSMA are accelerating innovation with Gemma

AT&T and GSMA have collaborated to build OTel, a family of open telecom-specific AI models fine-tuned on Gemma (Google's open-source model family). Trained on a specialized telco dataset curated by GSMA, operators, equipment vendors, and academia, the initiative produced 30 models across various sizes. Gemma-4-E4B-it achieved 91.74% accuracy — the highest among all tested architectures. The models use RAG-based training to reduce hallucinations, critical for regulated telecom environments. OTel has surpassed 18 million downloads and ranks among the top models on Open Telco Benchmarks, demonstrating that smaller domain-specific models can outperform larger general-purpose frontier models in specialized tasks like network configuration and self-healing systems.

I tested 3 local LLMs for UI design work, and only one of them behaved like a real designer

A hands-on comparison of three local LLMs (Qwen 3.5 9B, Mistral 7B, and Gemma 4 12B) for UI design work using Open Design and LM Studio on 8GB VRAM. Qwen produced a functional but visually underwhelming result after context-length tweaks. Mistral failed to render properly in Open Design and produced a compressed, incomplete design even via direct HTML generation. Gemma 4 12B, despite failing entirely in Open Design, produced the most polished output when run directly in LM Studio and rendered via VS Code's Live Server — with proper nav, event cards, and a well-structured café menu. The author concludes that strong code generation ability is the key differentiator for local UI design models, and recommends Gemma for local vibe design workflows despite its hardware demands.