Designing Inclusive Computational Thinking Metrics to Broaden Participation in Computer Science

Broadening participation in computer science requires helping students understand how they can use computer science in ways that they see as valuable. This includes students’ learning by designing their own games. The experience of game design has proven effective at helping them learn about algorithms, computational thinking, and programming in meaningful ways.

Tools such as Scratch programming enable students to engage in such game design learning. However, interpreting and assessing students’ understanding of computer science in these environments is challenging. Automated approaches to assessing the products students create may have biases or may not account for diverse approaches to game design.

Automated assessments have the potential to provide feedback on their work to students in real time and help them learn about computer science effectively. This project seeks to understand how to create methods for automatically analyzing students’ computational thinking that are equitable, inclusive, and diverse.

The work addresses three critical needs in computer science education: (1) improving computational thinking metrics to support automated assessment of computational thinking; (2) providing students with real-time feedback that helps them monitor their progress in computational thinking and programming to build confidence in their skills; and (3) creating context-sensitive assessment and feedback tools that promote inclusivity and encourage students to learn about computer science, computational thinking, and programming. The key research question in this project is: How do we address inclusivity and diversity in existing metrics-based automated computational thinking assessments to help broaden participation in computer science?

The project will use assessment data gathered from eighth-grade students using Scratch who created games connected to science concepts. The analysis of students’ products will document how students’ computational thinking development, design practices, and programming routines are assessed currently. Subsequent work will create metrics that are more inclusive and equitable.

The project will use machine learning techniques to analyze data (e.g., to identify patterns in the students’ products). The project will also use measures of self-efficacy in computer science to understand students’ product design. In later phases of the project, a participatory design process will be used to redesign the metrics to enhance inclusivity in the assessment of computational thinking via game design in Scratch.

This project is funded through the CS for All: Research and RPPs program

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.