Developing resources to support community-responsive family program evaluation in informal STEM learning institutions

The New York Hall of Science (NYSCI) will undertake a two-year Integrating Research and Practice project that will build the capacity of informal STEM educators to conduct evaluations of family STEM learning programs that are aligned with and responsive to the priorities and interests of program participants. The goal of this project is to test and develop resources that can support informal STEM educators and program developers in integrating meaningful, culturally responsive evaluation strategies into family STEM learning programs, particularly those that educators are responsible for evaluating themselves (a common scenario for ongoing programming in informal institutions).

Through a collaborative research process, the research team will develop and test evaluation strategies that empower STEM educators to conduct reflexive and iterative evaluations that leverage and integrate caregiver perspectives and feedback. A resulting toolkit, developed within the context of NYSCI’s existing Families Learning Together program, will empower informal STEM educators to gather, interpret, and apply evidence and will contribute to the expertise of the informal STEM education workforce, the quality of the experiences they provide to learners, and the richness of the evaluation data available to project funders.

The proposed project will extend and strengthen long-term partnerships between NYSCI, local community members, program evaluators, and other informal learning environments that work with largely Latine, new immigrant Spanish-speaking populations.

The project will move through four, overlapping phases of work. Phase 1 of the project will be an exploration of educator and participant alignment of metrics for success. Using a participatory research approach, researchers will collaborate with a group of caregivers that have participated in the NYSCI family program, Families Learning Together, identify the goals and values of family engagement programming, possible sources of evidence of its impact, and include caregivers in both the design and evaluation processes at NYSCI.

Phase 2 will include systematic and collaborative data analysis and sense-making. Together with STEM practitioners and caregivers, researchers will work to identify potential tensions and opportunities to align practitioner expectations with caregiver values, agency, and engagement. In Phase 3, the research and practitioner team will develop resources that build the capacity of informal STEM educators to conduct evaluations of the programs they are responsible for evaluating themselves, through collaborative work with caregivers.

Throughout the course of the project, the research and practitioner team will work with a group of four expert advisors to ensure that the strategies and resources created are relevant and feasible for educators at a range of institutions. Phase 4 will focus on communication and dissemination of project findings and resources to multiple stakeholders.

This project will directly benefit caregivers from 45 Queens families who participate in the Families Learning Together program at NYSCI. The resources developed will help to improve evaluation practices and capacity in other informal STEM program settings, including science centers and children’s museums, and other community-based organizations that conduct evaluations to improve their programs.

This Integrating Research and Practice project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments. This includes providing multiple pathways for broadening access to and engagement in STEM learning experiences.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.