APTO: Constructing causal knowledge graphs for assessing and predicting technology outcomes

Assessing and predicting technology outcomes (APTO) is crucial for evaluating the impact of R&D investments on innovation, economic growth, and national competitiveness. Tackling this complex task requires appropriate datasets and effective data creation tools. The latest natural language processing (NLP) technologies have reached human-level performance in certain crucial information extraction tasks, as evidenced by results from community-organized challenges.

This project will leverage an award-winning pipeline for constructing knowledge graphs (KGs) by expanding it substantially to include a diverse set of technology related entities and their relations. KGs comprise entities like diseases, genes, drugs, etc., and their relations, including associations, bindings, positive correlations, etc. The enhanced KG will be ready for developing models for predicting technology outcomes in healthcare and drug discovery.

In addition to the dataset, the project team will also develop an end-to-end toolkit for fellow APTO teams to perform information extraction tasks and construct KGs in their own domains. By utilizing the latest advancements in NLP and predictive modeling, the project will provide a comprehensive assessment of the capabilities and applications of biomedical technologies.

This will not only inform R&D investments but will also contribute to informed decision-making in healthcare and technology policy, as well as address the disparities between healthcare spending and outcomes. Furthermore, the project's approach of extracting vast amounts of information from text to build predictive models can be applied to other sectors, advancing research and knowledge across various fields.

Ultimately, this project has the potential to drive strategic investments in technology and innovation, improving health outcomes and fostering economic prosperity on a national and global scale.

This project will leverage a pipeline recently developed that won the NIH-organized LitCoin NLP challenge, a competition that evaluated methods for constructing biomedical knowledge graphs (KGs) by extracting entities and their relations from biomedical texts. Using this pipeline, the project team created a large-scale KG by extracting information from all PubMed abstracts.

The KG, named iKraph, contains substantially more information than that in public databases. To adapt iKraph for causal inference, the project team annotated direction information for the relations in the LitCoin dataset and developed models to predict the direction of relations, which enabled the construction of a causal KG capable of inferring causality between indirectly connected entities.

In this project, iKraph will be enhanced by adding a diverse set of technology related entities and their relations such as equipment, technology, technology features, feature values, problems, methods, data types, datasets, and geographical entities etc. The project team will extract relevant information from unstructured text including PubMed abstracts, PubMed Central full-text articles, patents, marketing reports, and Wikipedia articles.

Relevant data from public databases will also be integrated into iKraph. The toolkit for constructing KGs, designed for end-to-end annotation and model building, will utilize an AI-assisted methodology. This approach incorporates AI models at every stage of the annotation process to enhance quality and significantly improve efficiency.

Finally, the project team will conduct a case study on advanced manufacturing technologies (AMT) for the production of generic off-patent drugs using the constructed KG.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.