Doctoral Dissertation Research: Multidimensional Nominal Response Models in Adaptive Testing

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2). This doctoral dissertation research project will investigate the use of multidimensional tests in computerized adaptive testing. Multidimensional tests are used across many disciplines to make critical decisions on issues such as psychological health, educational ability, and vocational suitability.

Some of these tests are administered as Computerized Adaptive Tests (CATs), a mode of testing that adapts the test in real time based on the examinee's item responses. Generally, CATs are designed to work with item responses that are scored as 0 or 1, based on whether the examinee has chosen the correct response option or an incorrect response option.

However, this dichotomous scoring scheme disregards information contained in incorrect response options, which can be modeled as nominal response data. Research on single-subject CATs has demonstrated that when its components are designed to work with nominal response data, the tests were shorter and more accurate than those designed for dichotomous responses.

This project will examine the value of multidimensional adaptive tests that utilize item response options as nominal response data. Psychometric software will be developed and made publicly available. As a Doctoral Dissertation Research Improvement award, support is provided to enable a promising student to establish a strong, independent research career.

This research project will advance the use of multidimensional tests in computerized adaptive testing by incorporating item response options into the modeling, item selection, and estimation processes that constitute the adaptive test. The project will compare the accuracy, efficiency, and security of this approach to existing procedures. Adaptive test components and results will be simulated for Multidimensional Adaptive Tests (MATs) and Cognitive Diagnostic-Computerized Adaptive Tests (CD-CATs).

Both MATs and CD-CATs can be used for either summative or formative assessment purposes. However, they differ in their theoretical assumptions and thus must be investigated separately. To compare the incorporation of item response options in multidimensional adaptive testing to current dichotomous scoring procedures, item banks and item selection methods will be manipulated.

Common item selection methods for dichotomous response data will be generalized for use with nominal response data to ensure comparability.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.