Valid predictions of group-level random effects

Abstract

Gaussian linear models with random group-level effects are the standard for modeling randomized experiments carried out over groups, such as locations, farms, hospitals, or schools. Group-level effects can be summarized by prediction intervals for group-level means or responses, but the quality of such summaries depends on whether the intervals are valid in the sense they attain their nominal coverage probability. Many methods for constructing prediction intervals are available -- such as Student's t, bootstrap, and Bayesian methods -- but none of these are guaranteed to be valid, and indeed are not valid over a range of simulation examples. We propose a new method for constructing valid predictions of group-level effects based on an inferential model (IM). The proposed prediction intervals have guaranteed finite-sample validity and outperform existing methods in simulation examples. In an on-farm agricultural study the new IM-based prediction intervals suggest a higher level of uncertainty in farm-specific effects compared to the standard Student's t-based intervals, which are known to undercover.