CRII: III: Explainable Multi-Source Data Integration with Uncertainty

Sensors are all around us collecting data. Each sensor may provide complementary or reinforced information that supports tasks such as target detection, classification, or scene understanding. In remote sensing applications, for example, hyperspectral imaging sensors can provide spectral information about materials by using a wide range of wavelengths, while LiDAR (light detection and ranging) measures an object's elevation above the ground.

If a road and a building rooftop are built with the same material (e.g., asphalt), hyperspectral information alone may not be sufficient to tell them apart, while integrating height information from LiDAR data makes it easier to distinguish the two. This project will develop innovative mathematical framework and associated algorithms for integrating such sensor data from multiple sources.

The main novelty of this project will be in its capacity to learn the relationships and non-linear interactions among multiple sources during data integration, while addressing data and label uncertainties commonly observed in real-world sensor data. This work will advance the fundamental knowledge in explainable data integration and is applicable to a broad range of data-intensive applications with significant social impact including remote sensing, autonomous driving, robotic vision and perception, and precision agriculture.

This work will also support cross-disciplinary research and educational activities, including mentoring students from diverse backgrounds, outreach activities through hands-on design projects and the development of undergraduate and graduate-level courses on data integration and machine learning at the University of Michigan.

This project aims to bridge existing gaps in knowledge by developing a unified, weakly supervised learning-based framework for heterogeneous multi-sensor data integration, accounting for both data and label uncertainties. This project addresses the following three research components. The first objective of this research is to develop a scalable and efficient computational model leveraging the binary fuzzy measures under the multiple instance learning framework, which will reduce the search space of the non-linear aggregation operator such as the Choquet integral (CI) and be significantly more robust at handling data integration for larger numbers of sources.

The second objective is to leverage bi-capacities and bi-polar CI to account for conflicts and negation of evidence in data sources. Experiments will be conducted to investigate how each data source and their combinations contribute to different integration outcomes. This study will allow for greater explainability and adaptability when integrating complex and heterogeneous datasets.

The third objective is to extend the current multiple instance CI framework for sequential inputs, such as multi-view and multi-temporal data, which will uncover the untapped potential of using CI for spatiotemporal data integration. This research will make data integration manageable, interpretable, and applicable with large volumes of multi-modal data to facilitate decision making and knowledge discovery.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.