Evaluating Effectiveness of Standard Error of Score Estimation as a CAT Termination Criterion

Abstract

In computerized adaptive testing (CAT), the standard error of score estimation (SEE) value is sometimes used as a test termination criterion since test forms with the same level of SEE are believed to exhibit equivalent measurement precision even when they vary in the number of items. This study aimed to evaluate the appropriateness and effectiveness of using SEE as an approximation for actual standard error of measurement (SEM) under various conditions. First, this study illustrated a potential issue with the inconsistent relationship between the SEE and actual SEM at different lengths in equivalent tests. A series of simulation studies were then conducted to investigate the relationship among SEE, SEM, and test lengths in a variable-length CAT administration. For this purpose, various combinations of score estimation methods and item selection methods were considered both with and without item exposure control. The study found that using the SEE as a sole criterion to terminate a CAT administration might result in a less than ideal level of control in measurement precision when the test was either extremely short or administered in a variable-length CAT. Also, the choice of the score estimation methods and item selection methods with or without item exposure control affected the relationship between SEE and actual SEM.