EAGER: Language Learning through Machine Theory of Mind

As natural language systems become ubiquitous (e.g. phone trees, chatbots, and smart homes) they must learn to adapt to users by modeling them each as individuals with different abilities, knowledge, and tastes. Theory of mind is the human ability to reason about the hidden mental states of others, but is a complex phenomenon that does not emerge in children until late in their development compared to other more basic communicative skills.

The questions of interest to this EAGER project are: (1) what makes this skill hard for children to learn, (2) what can computers learn from how children are taught, and (3) in what ways can machine learning models provide insight into human development. This project sits at this intersection of machine learning, developmental psychology, and pedagogy.

This project includes formal models of information sharing and teaching grounded in shared referential games. Agents and children are tasked with asking an instructor to efficiently distinguish similar objects -- a task which requires understanding common ground and identifying distinguishing features. While the learner will often make ambiguous statements, the teacher will provide corrections and instruction to guide the learning process.

This formulation allows for variation along several dimensions of relevance to successful communication: working memory, visual and lexical complexity, and specificity of instruction. Experiments with children will provide benchmarks against which computational agents can be compared, and experiments with agents will allow us to decompose the contribution of each of these factors to the difficulty of developing a theory of mind.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.