Collaborative Research: DASS: Policy Design for Holding AI-Supported Systems Accountable

As society is becoming more and more dependent on software systems, questions arise about how such software can be developed responsibly. In the design of software systems, flexibility and innovation must be balanced with safeguarding the public. Regulators who design policy to steward software’s effect on society often have a hard time capturing clear guidance to hold software systems accountable and keeping up with new technologies, such as the increased use of artificial intelligence (AI).

Software engineers have often been left with vague guidance and no measurable goals, such as the EU’s “right to explanation.” While clarifying guidance or court decisions may eventually provide more actionable details, policy implementation is a slow, reactive process that can delay adoption of improvements. To reconcile software and policy design and construct a proactive policy framework for accountable software systems, guidance for qualities such as explainability, safety, and fairness requires more attention.

This project will investigate cross-domain principles for policy design, setting corresponding obligations for software engineers, particularly in the context of AI-supported systems. Results will help stakeholders to deliberately design policy with evidence-based guidance for a specific domain. The project will facilitate interactions between policy makers and software engineers in understanding capabilities and societal expectations toward forming policy goals, investigate policy dimensions and the concrete tradeoffs involved, and connect the policy to specific quality-assurance obligations that could be used in a regulatory evaluation covering both AI and non-AI parts of the system.

Specifically, the project will first investigate the important dimensions for policy design (e.g., strictness of regulation, level of evidence, policy specificity) using stakeholder interviews, historical analysis, and technical analysis. It then will evaluate, with several experiments, how design decisions affect outcomes, interact with each other, and result in tradeoffs -- which will be encoded and validated as policy design patterns.

The research will combine multiple research methods, including interviews, literature and document analyses, controlled experiments, prototyping, and writer’s workshops. It will deeply integrate social-science research on policy design and regulation with software-engineering research on system design and quality assurance. The research will be disseminated broadly across multiple communities and through engagement with regulators. Educational material will be developed for multiple courses and shared publicly.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.