CHS: Small: DeepCrowd: A Crowd-assisted Deep Learning-based Disaster Scene Assessment System with Active Human-AI Interactions

Recent advances in artificial intelligence (AI) have transformed many important domains of modern life such as transportation, finance, education, healthcare, and entertainment. This project addresses application of AI to disaster scene assessment (DSA). For DSA, artificial intelligence can be used to automatically identify damage severity of impacted areas from imagery reports in the aftermath of a disaster such as earthquake, hurricane, or landslides.

A key limitation of AI based techniques is the black-box nature of many contemporary models and the consequent lack of interpretability of the results and failures. This project investigates the problem of troubleshooting, tuning, and eventually improving the black-box AI algorithms by integrating human intelligence with machine intelligence through active crowd-AI interactions.

The work complements the prevailing AI solutions that primarily focus on AI model design and training sample collection. The results from this project will open up unprecedented opportunities of fully exploring the wisdom from the crowd in various crowd-assisted AI application domains. This project will also provide opportunities for students in STEM and from underrepresented groups to study the interaction between AI and humans.

This project develops a DeepCrowd framework that can be used to guide the design, development, and implementation of future crowd-AI applications where the human intelligence obtained from the crowd is tightly integrated with AI deep learning models to significantly improve the system performance over the AI-only or human-only solutions. The project addresses the black-box challenges of AI and the crowdsourcing platform in DeepCrowd using an interdisciplinary approach inspired by techniques from AI, machine learning, estimation theory, and cyber-human interactions.

In particular, the research includes i) developing a crowd task generation scheme to effectively query the crowdsourcing platform for feedback; ii) creating a novel adaptive mechanism to incentivize the crowd for timely and accurate response; iii) designing an interactive attention neural network scheme that enables direct interaction between crowd and AI models; and iv) developing a crowd and AI integration engine that effectively incorporates feedback from crowd to alleviate failure scenarios of AI. The resulting DeepCrowd framework is transformative in that it will produce a set of new crowd-AI interaction models and techniques to build novel crowd-assisted AI applications with boosted system performance.

This project develops a DeepCrowd framework that can be used to guide the design, development, and implementation of future crowd-AI applications where the human intelligence obtained from the crowd is tightly integrated with AI deep learning models to significantly improve the system performance over the AI-only or human-only solutions. The project addresses the black-box challenges of AI and the crowdsourcing platform in DeepCrowd using an interdisciplinary approach inspired by techniques from AI, machine learning, estimation theory, and cyber-human interactions.

In particular, the research includes i) developing a crowd task generation scheme to effectively query the crowdsourcing platform for feedback; ii) creating a novel adaptive mechanism to incentivize the crowd for timely and accurate response; iii) designing an interactive attention neural network scheme that enables direct interaction between crowd and AI models; and iv) developing a crowd and AI integration engine that effectively incorporates feedback from crowd to alleviate failure scenarios of AI. The resulting DeepCrowd framework is transformative in that it will produce a set of new crowd-AI interaction models and techniques to build novel crowd-assisted AI applications with boosted system performance.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.