Why Your Best Predictive Model Gives the Wrong Treatment Effect

Predictive model selection criteria like BIC, AIC, and Lasso optimize for outcome fit, not causal validity — meaning they can systematically drop true confounders that are weakly associated with the outcome but strongly associated with the treatment. This leads to biased treatment effect estimates even when the model fits the data well. The post demonstrates this failure mode in Python, then introduces Bayesian Adjustment for Confounding (BAC), which fits both an outcome model and an exposure model and uses the latter to rescue confounders that predictive criteria would discard. The post also covers the frequentist equivalents — post-double-selection and double/debiased machine learning — and critically explains the sharp edge of the 'adjust for anything that predicts exposure' rule: instrumental variables are statistically indistinguishable from confounders in the exposure model, and adjusting for them amplifies bias. The choice of how aggressively to follow the exposure model is a causal assumption, not something data can resolve.

Source: https://towardsdatascience.com/why-your-best-predictive-model-gives-the-wrong-treatment-effect. 8sync News only summarizes and links out; content copyright belongs to the authors and original sources.