Development and Validation of a Versatile and Equitable Assessment of Knowledge within the Standard First Course in Computer Science

This project aims to serve the national interest by developing an empirically validated tool to assess a student’s knowledge of computing concepts. Such well-designed knowledge assessments, vetted by rigorous design and psychometric processes, are still relatively uncommon in the computer science education community. Moreover, students learn computing concepts in a variety of increasingly different educational contexts: at home, on the job, in a course offered online, or in a traditional classroom.

Thus, there is a pressing need for an empirically validated assessment of computing knowledge that works in a variety of educational contexts and with different programming languages. The goal of this project is to design and pilot such a multi-contextual and multi-language assessment by enhancing the existing Second Computer Science 1 (SCS1) course assessment.

The new assessment, referred to as SCS1++, will be designed by addressing known issues with the existing SCS1 assessment while also increasing the number of questions on the assessment. Specifically, the project will (1) construct a coherent argument for validity claims of SCS1++ as a whole; and (2) create subscales aligned with the concepts on the assessment to improve the assessment’s formative values.

The project will directly inform the research on equitable CS assessment by updating the current SCS1 with improved questions and rigorous validation using advanced psychometric tools. A design-based research approach will be used, based on theory, and the system will be evaluated in real educational settings. The NSF IUSE: EDU Program supports research and development projects to improve the effectiveness of STEM education for all students.

Through its Engaged Student Learning track, the program supports the creation, exploration, and implementation of promising practices and tools.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.