Developing Conversational Videos to Support Children's STEM Learning and Engagement

Early childhood is a critical time for developing foundational knowledge, skills, and interest in science, technology, engineering, and mathematics (STEM). For that reason, the Public Broadcasting Service (PBS) places a great priority on developing early childhood STEM content, especially through its television shows that are watched by over 60% of young children in the United States.

Research suggests that adding in-the-moment interaction to television watching promotes learning and engagement. Toward this end, researchers from the University of California, Irvine and PBS KIDS have prototyped interactive versions of science shows that children view on internet-connected devices while they communicate with the main character powered by an AI conversational agent.

Pilot studies show that when children watch these new interactive videos with the main character pausing periodically to ask probing questions about the learning goals of the episode and following up with appropriate responses, they are more engaged and learn more about science, with heightened benefits for children who speak languages other than English at home. Based on these early results, in this Innovations in Development project the research team will develop, test and produce publicly available conversational episodes for two PBS KIDS television shows, one focused on science and the other on computational thinking.

The project will iteratively study and develop six conversational videos with novel forms of support for children, including extended back-and-forth conversation that builds upon a child's responses, visual scaffolding that facilitates verbal communication, and bilingual language processing so that children can answer in English or Spanish. The conversational videos will be evaluated in both lab-based and home settings.

The lab-based study will involve 600 children ages 3-7 in a predominantly low-income Latino community in Southern California, in which researchers compare children’s learning and engagement when watching the conversational videos with three other formats: (1) watching the non-interactive broadcast version of the video; (2) watching the video with pseudo-interaction, in which the main character asks questions and gives a generic response after a fixed amount of time but can’t understand what the child says; or (3) watching the broadcast version of the video with a human co-viewer who pauses the video and asks questions. The home-based study will involve 80 families assigned to watch either the non-interactive or interactive videos as many times as they want over a month at home.

In both the lab-based and home studies, pre- and post-tests will be used to examine the impact of video watching on science and language learning, and log data will be used to assess children’s verbalization and engagement while watching. Following the home study, the six videos will be further refined and made available for free to the public through the PBS KIDS apps and website, which are visited by more than 13 million users a month.

Beyond providing engaging science learning opportunities to children throughout the country, this study will yield important insights into the design, usability, feasibility, and effectiveness of incorporating conversational agents into children’s STEM-oriented video content, with implications for extending this innovation to other educational media such as e-books, games, apps, and toys.

This Innovations in Development project is funded by the Advancing Informal STEM Learning (AISL) program, which seeks to (a) advance new approaches to and evidence-based understanding of the design and development of STEM learning in informal environments; (b) provide multiple pathways for broadening access to and engagement in STEM learning experiences; (c) advance innovative research on and assessment of STEM learning in informal environments; and (d) engage the public of all ages in learning STEM in informal environments.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.