Testing AI with an Office Hours Question

An experienced SQL Server DBA tests seven different AI models (local coding models, DwarfStar/DeepSeek, ChatGPT variants, Claude variants, and Gemini) against a real, nuanced Office Hours question about running DBCC CHECKDB on a synchronous AG secondary for a 25TB database with tempdb full errors. Each answer is graded for technical accuracy from F to B+, revealing that even paid cloud models frequently give confidently wrong or incomplete advice (missing RPO/RTO discussion, misunderstanding filegroup offline behavior, wrong tempdb explanations), while Claude Fable 5 High produced the most accurate response. The conclusion: no single model reliably wins, so continuous testing per-question is necessary rather than committing to one AI answer service, and tools like Nano-GPT and OpenRouter help compare models quickly.

Source: https://www.brentozar.com/archive/2026/08/testing-ai-with-an-office-hours-question. 8 Sync News only summarizes and links out; content copyright belongs to the authors and original sources.