FAME: OPEN-ENDED MANIPULATION TASK LEARNING WITH FAME (FUTURE-ORIENTED COGNITIVE1 ACTION MODELLING ENGINE)

The realization of computational models for accomplishing everyday manipulation tasks for any object and any purpose would be adisruptive breakthrough in the creation of versatile, general-purpose robot agents; and it is a grand challenge for AI and robotics.Humans are able to accomplish tasks such as “cut up the fruit” for many types of fruit by generating a large variety of context-specificmanipulation behaviors.

They can typically accomplish the tasks on the first attempt despite uncertain physical conditions and novelobjects.

Acting so effectively requires comprehensive reasoning about the possible consequences of intended behavior beforephysically interacting with the real world.In the FAME project, I will investigate the research hypothesis that a knowledge representation and reasoning (KR&R) frameworkbased on explictly-represented and machine-interpretable inner-world models can enable robots to contextualize underdeterminedmanipulation task requests on the first attempt.

To this end, I will design, implement, and evaluate FAME (Future-oriented cognitiveAction Modelling Engine), a hybrid symbolic/subsymbolic KR&R framework that will contextualize actions by reasoning symbolicallyin an abstract and generalized manner but also by reasoning with “one’s eyes and hands” through mental simulation and imagisticreasoning.

Realizing FAME requires three breakthrough research results:(1) modelling and parameterization of manipulation motion patterns and understanding the resulting effects underuncertain conditions;(2) the ability to mentally simulate imagined and observed manipulation tasks to link them to the robot’s knowledge and experience;and(3) the on-demand acquisition of task-specific causal models for novel manipulation tasks through mental physics-based simulations.To assess the power and feasibility of FAME, I will use open manipulation task learning as a benchmark challenge.