HCC: Medium: Investigating the Feasibility of Object Recognition Systems Providing Individuals with IDD with Workplace Task Support

Having a job is important in many ways, including for people with intellectual and developmental disabilities (IDD), like Down Syndrome. Jobs are associated with improved well-being, life satisfaction, confidence, and inclusion in the community. People with IDD are often the most unemployed and underemployed group in the United States, partly because they may not have the training or skills needed for many jobs.

However, there are different support mechanisms to help those with IDD to find and keep work, including Vocational Rehabilitation, the Department of Disability Services, and post-secondary education. But these supports can be expensive and time-consuming, so not everyone can use them. This project will help people with IDD be more independent by creating a mobile phone app that provides personalized job training. There is strong potential for impact to different communities through this research.

The primary goal of this project is to co-design, co-develop, and test an object recognition app. This app provides on-the-job task prompting to support people with IDD in their work. The project has five primary objectives.

First, the project will identify specific work tasks where object recognition may help people with IDD. Second, researchers will identify how people with IDD feel about object recognition systems. Third, the project team will co-design an object recognition mobile app for use by people with IDD.

Fourth, the team will train an object recognition model for in-task workplace support. Fifth, the team will test the effects of the object recognition system with people with IDD. The tests will check workplace self-sufficiency, self-efficacy, and task performance.

To achieve these goals, this project combines different research approaches. Ethnographic research will capture the detailed work experiences of people with IDD. Next, the team uses participatory design methods.

This ensures inclusivity in the technology development of object recognition systems for people with IDD. A single case multiple-probe across participant design will assess the technology’s effectiveness. This project has the potential to advance knowledge in accessible technology development.

This will make technology more accessible for people with intellectual disabilities. It will result in more inclusive and fair artificial intelligent systems. This project lays the groundwork for future explorations in the design of intelligent workplace assistive technologies.

The results of this research will extend the abilities of people with cognitive differences and will be publicized as broadly as possible.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.