An algorithm to compute the power of Monte Carlo tests with guaranteed precision. This article presents an algorithm that generates a conservative confidence interval of a specified length and coverage probability for the power of a Monte Carlo test (such as a bootstrap or permutation test). It is the first method that achieves this aim for almost any Monte Carlo test. Previous research has focused on obtaining as accurate a result as possible for a fixed computational effort, without providing a guaranteed precision in the above sense. The algorithm we propose does not have a fixed effort and runs until a confidence interval with a user-specified length and coverage probability can be constructed. We show that the expected effort required by the algorithm is finite in most cases of practical interest, including situations where the distribution of the $p$-value is absolutely continuous or discrete with finite support. The algorithm is implemented in the R-package simctest, available on CRAN.
Keywords for this software
References in zbMATH (referenced in 10 articles )
Showing results 1 to 10 of 10.
- Ding, Dong; Gandy, Axel; Hahn, Georg: A simple method for implementing Monte Carlo tests (2020)
- Hahn, Georg: On the expected runtime of multiple testing algorithms with bounded error (2020)
- Hahn, Georg: Optimal allocation of Monte Carlo simulations to multiple hypothesis tests (2020)
- Silva, Ivair; Assunção, Renato: Truncated sequential Monte Carlo test with exact power (2018)
- Gandy, Axel; Hahn, Georg: MMCTest -- a safe algorithm for implementing multiple Monte Carlo tests (2014)
- Gandy, Axel; Rubin-Delanchy, Patrick: An algorithm to compute the power of Monte Carlo tests with guaranteed precision (2013)
- Oueslati, Abdullah; Lopez, Olivier: A proportional hazards regression model with change-points in the baseline function (2013)
- Silva, Ivair R.; Assunção, Renato M.: Optimal generalized truncated sequential Monte Carlo test (2013)
- Gandy, Axel: Sequential implementation of Monte Carlo tests with uniformly bounded resampling risk (2009)
- Gandy, Axel; Jensen, Uwe: Model checks for Cox-type regression models based on optimally weighted martingale residuals (2009)