Valid F-screening in linear regression

Daniela WITTEN, Dorothy Gilford Endowed Chair, Professor of Statistics & Biostatistics, University of Washington

Suppose that a data analyst wishes to report the results of a least squares linear regression only if the overall null hypothesis—namely, that all non-intercept coefficients equal zero—is rejected. This practice, which we refer to as F-screening (since the overall null hypothesis is typically tested using an F -statistic), is in fact common practice across a number of applied fields. Unfortunately, it poses a problem: standard guarantees for the inferential outputs of linear regression, such as Type 1 error control of hypothesis tests and nominal coverage of confidence intervals, hold unconditionally, but fail to hold conditional on rejection of the overall null hypothesis. 
In this talk, I will present an inferential toolbox for the coefficients in a least squares model that are valid conditional on rejection of the overall null hypothesis. I will present selective p-values that lead to tests that control the selective Type 1 error, i.e., the Type 1 error conditional on having rejected the overall null hypothesis. Furthermore, they can be computed without access to the raw data, using only the standard outputs of a least squares linear regression, and therefore are suitable for use in a retrospective analysis of a published study. I will also present confidence intervals that attain nominal selective coverage, and point estimates that account for having rejected the overall null hypothesis. 
 will illustrate this selective procedure via re-analysis of a published result in the biomedical literature, for which the raw data is not available. 
This is joint work with Olivia McGough (U. Washington) and Daniel Kessler (UNC Chapel Hill).