This dissertation addresses statistical issues that arise in a multiple testing framework when each of m hypotheses is tested via permutation methods. A standard error rate to control in multiple testing situations (especially when m ~ 10⁴) is the false discovery rate which describes the expected ratio of type I errors to the total number of rejections. An adaptive approach to controlling the false discovery rate is to estimate the number of type I errors using a data-based estimate of m₀, the number of true null hypotheses. Estimation of m₀ has received much interest in recent years. Existing methods assume each of the m p-values has a continuous uniform (0,1) null distribution. This dissertation discusses numerous ways in which p-values may not have continuous uniform (0,1) null distributions and proposes how to estimate m₀ and the false discovery rate in these scenarios.

The first scenario involves a sequential permutation testing procedure that can substantially reduce computational expense when the test statistic is computationally intensive. The method is demonstrated via an application involving the genetic mapping of expression quantitative trait loci (eQTL). Other scenarios are motivated by problems that arise in genomics and proteomics.