MMCTest
MMCTest -- a safe algorithm for implementing multiple Monte Carlo tests. Consider testing multiple hypotheses using tests that can only be evaluated by simulation, such as permutation tests or bootstrap tests. This article introduces MMCTest, a sequential algorithm that gives, with arbitrarily high probability, the same classification as a specific multiple testing procedure applied to ideal $p$-values. The method can be used with a class of multiple testing procedures that include the Benjamini and Hochberg false discovery rate procedure [{it Y. Benjamini} and {it Y. Hochberg}, J. R. Stat. Soc., Ser. B 57, No. 1, 289--300 (1995; Zbl 0809.62014)] and the Bonferroni correction [{it C. E. Bonferroni} [Teoria statistica delle classi e calcolo delle probabilita. Firenze: Libr. Internaz. Seeber (1936; Zbl 0016.41103)] controlling the familywise error rate. One of the key features of the algorithm is that it stops sampling for all the hypotheses that can already be decided as being rejected or non-rejected. MMCTest can be interrupted at any stage and then returns three sets of hypotheses: the rejected, the non-rejected and the undecided hypotheses. A simulation study motivated by actual biological data shows that MMCTest is usable in practice and that, despite the additional guarantee, it can be computationally more efficient than other methods.
Keywords for this software
References in zbMATH (referenced in 8 articles , 1 standard article )
Showing results 1 to 8 of 8.
Sorted by year (- Hahn, Georg: Online multivariate changepoint detection with type I error control and constant time/memory updates per series (2022)
- Ding, Dong; Gandy, Axel; Hahn, Georg: A simple method for implementing Monte Carlo tests (2020)
- Hahn, Georg: Optimal allocation of Monte Carlo simulations to multiple hypothesis tests (2020)
- Hahn, Georg: On the expected runtime of multiple testing algorithms with bounded error (2020)
- Hahn, Georg: Closure properties of classes of multiple testing procedures (2018)
- Gandy, Axel; Hahn, Georg: QuickMMCTest: quick multiple Monte Carlo testing (2017)
- Gandy, Axel; Hahn, Georg: A framework for Monte Carlo based multiple testing (2016)
- Gandy, Axel; Hahn, Georg: MMCTest -- a safe algorithm for implementing multiple Monte Carlo tests (2014)