Stochastic Curtailment: A New Approach to Improve Efficiency of Variable-Length Computerized Adaptive Tests

Abstract

Stochastic curtailment (SC) is a statistical procedure that was originally developed to enhance the efficiency of clinical trials. It has been applied to psychological testing, but to sequential mastery testing only (Finkelman, 2008, 2010). This study adapted the method to detect low-precision examinees in computerized adaptive tests (CATs) (i.e., examinees whose final standard error of measurement (FSEM) at the end of a full-length CAT could not reach the pre-specified SEM termination level). Using central limit approximations, the study developed a method to estimate the distribution of test information at maximum test length and the corresponding FSEM. The study also developed a hypothesis testing procedure to implement SC. Using monte-carlo simulations, the study found that (1) the FSEM estimation procedure performed well in the middle range of theta values but less so at extreme theta values; (2) the SC procedure had good predictive accuracy, with excellent performance on positive predictive values and good performance on true positive rates and false positive rates; (3) the potential reduction in test length was substantial. Overall, the study showed that SC is a promising procedure to identify low-precision examinees and enhance efficiency in measurement CATs. A brief guide to implementing SC is provided.