Assessing Robustness of the Permutation Test by Alex Han ’22, Statistics Senior Thesis Defense

Assessing Robustness of the Permutation Test by Alex Han ’22, Statistics Senior Thesis Defense, Wednesday, May 18, 1:10 – 1:50 pm, North Science Building 017, Wachenheim.

Abstract:  The hypothesis testing procedure specifies a decision rule for accepting or rejecting the null hypothesis based on observed data.  That is, given some dataset, a hypothesis test determines the plausibility of the null.  Inverting a hypothesis test, then, determines the set of all plausible forms of the null based on a given dataset.  Unlike its parametric counterparts, the permutation test assumes exchangeability of data under the null, rather than the parametric form of the underlying distribution. For the two-sample difference in means, exchangeability asserts that the two underlying distributions are the same, and inverting a permutation test produces a permutation confidence interval for all plausible values of the true difference in means. This thesis investigates the power of permutation confidence intervals for this problem.  Through simulation studies, we quantify how well the permutation confidence interval controls the Type I error rate (that is, whether the permutation test is exact) in the non-asymptotic case and when exchangeability does not hold.