EAGER: Advancing Neuro-symbolic AI with Deep Knowledge-infused Learning

The first wave of AI termed symbolic AI, focused on explicit knowledge. The current second wave of AI is termed statistical AI. The deep learning techniques have been able to exploit large amounts of data and massive computational power to improve upon human levels of performance in narrowly defined tasks.

Separately, knowledge graphs emerged as a powerful tool to capture and exploit an extensive amount and variety of explicit knowledge to make algorithms better understand the content, and enable the next generation of data processing, such as in semantic search. Now, we herald towards the third wave of AI built on what is termed as the neuro-symbolic approach that combines the strengths of statistical and symbolic AI.

Combining the respective powers and benefits of using knowledge graphs and deep learning is particularly attractive. This has led to the development of an approach we have called knowledge-infused (deep) learning. This project will advance the currently limited forms of combining the knowledge graphs and deep learning, called shallow and semi-diffusion, with a more advanced form called deep-infusion, that will support stronger interleaving of more variety of knowledge at different levels of abstraction with layers in a deep learning architecture.

This project will investigate the deep knowledge-infusion strategy in two substantial ways. The first is to infuse knowledge of different types from knowledge graphs in the deep learning pipeline. For example, in natural language processing, we will investigate the incorporation of linguistic, common sense, broad-based and domain-specific knowledge.

The second is to infuse stratified knowledge representing different levels of abstractions, such as low levels of abstractions contained in raw data measurements that focus on the physical features of an object, and higher levels of abstractions that capture more conceptual aspects of the object, such as the object's functionality in an application. Each deep network layer may take a different type of knowledge representing the intended level of abstraction at that layer.

For example in a transformer, we can reparameterize different transformer blocks (layers) such that the transformer block will take a different type of knowledge representing the intended level of abstraction at that layer. Furthermore, the deep infusion pipeline can generate explanations for the outcomes of the deep-learning pipeline from the knowledge graph at the appropriate layer leading to a clear picture of the contextual connection between parts of the input.

Both layered abstraction and explanation modules would be highly significant contributions towards improving the state of machine intelligence.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.