#StackBounty: #modeling #p-value #aic Does automatic model selection via AIC bias the p-values of the selected model?

Bounty: 50

Let’s say I run a procedure where I fit every possible model given some set of covariates and I select the model with the minimum AIC. I know that if my selection criteria was based on minimizing p-values, the p-values of the selected model would be misleading. But what if my selection criteria was AIC alone? To what extent would this bias the p-values?

I had assumed the effect on p-values would be negligible, but came across this paper, which proves the following:

P values are intimately linked to confidence intervals and to
differences in Akaike’s information criterion (ΔAIC), two metrics that
have been advocated as replacements for the P value.

If this is true, does it imply that p-values are misleading after automatic selection based on AIC? To what extent will they be biased, and what determines this?

Get this bounty!!!

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.