Collaborative Research: Research: Development and Validation of Learning through Making Instrument (LMI)

Makerspaces are informal learning environments that facilitate multimodal learning through iterative practice, collaboration, design, building, prototype, and management. They have proliferated America’s college campuses, with initial creation costs in the thousands to millions of dollars. While emerging data demonstrate the tremendous impact makerspaces may create in the engineering learning experience especially for learning engineering-relevant skills, data also demonstrate that makerspaces fail to meet aspirations of being egalitarian, open, inclusive, and welcoming spaces for all.

Currently, it is difficult to determine the learning impact of a makerspace and to compare different makerspaces structures or makerspaces at different institutions with different populations of students. Drawing upon qualitative research that identified the modes and contents of learning in university makerspaces, this project will create and validate an instrument that measures student learning in makerspaces: the Learning through Making survey Instrument.

The Learning through Making Instrument will provide a tool for researchers and educators to assess the educational value of makerspaces, compare across different environments, experiment with different approaches, document the learning impacts, and ultimately facilitate understanding how makerspaces can enhance engineering education. This instrument will assist educators in understanding learning gaps that may be associated with obstacles for women, students of color, international students, and first-generation college students providing educators with data critical for designing more inclusive makerspaces and associated curriculum that broaden participation and cultivate more inclusive learning communities.

The Learning through Making Instrument will be developed and validated for collegiate students providing a springboard for quantitative research on learning in makerspaces and will facilitate future development of similar instruments applicable for K-12 students.

The research objective is to develop a survey for measuring learning in university makerspaces and similar environments. The proposal team’s prior work, The Learning through Making Typology will be a partial basis for development of the Learning through Making Instrument. Data for instrument development and validations with diverse engineering students will occur at very different universities, including San Jose State, Texas State, Texas A&M, Purdue University, Georgia Tech, and James Madison University.

The selection includes two Hispanic-Serving Institutions in two different regions of the country; a comprehensive, mid-Atlantic, regional, undergraduate-focused university offering only a general engineering degree; research-focused universities; and a top school for graduating Black and women engineers that also has dual-degree programs providing access to students from HBCUs. These universities also provide a contrasting set of makerspaces.

An initial pool of items will be generated, experts will evaluate the pool of items, and evidence of reliability along with structural validity will be sought by exploratory factor analysis and then confirmatory factor analysis. Standards for Educational and Psychological Testing indicate that fairness of assessments is just as crucial as validity and reliability, and fairness will be evaluated as a key part of this research.

We seek to answer the following Research Questions: To what extent are measurement construct specifications and actual assessment questions aligned? To what extent are there measurement invariances between gender and minority groups? Are there significant differences in the mean scores between groups that indicate certain groups are learning differently in makerspaces?

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.