RegTek: Methods for Regulatory Impact Assessments of Cutting-Edge Technology

Purpose and goal: <This text has been machine translated>

According to the OECD, 60% of Swedish impact assessments are insufficient today. The RegTek project aims to streamline rulemaking through a pilot study evaluating the accuracy and usability of a language model for faster impact assessments. An evaluated tool can increase the trust and use of language models by regulatory bodies, while ensuring objectivity and compliance with the EU standards for AI in accordance with AI Act.

Expected results and effects: <This text has been machine translated>

Results from the RegTek project complement and reinforce existing initiatives for impact investigations of cutting-edge technology at authorities through ESV assignments for method development and support. With this project, RISE can start building a capability for AI testing according to EU AI Act where metrology is mentioned for quality assurance and compliance with the law.

Approach and implementation: <This text has been machine translated>

The goal of the RegTek project is to conduct a pilot study to evaluate the accuracy and usability of a RAG-based language model for AI regulatory impact assessments, which may lead to wider acceptance and trust in language models and more comprehensive and fair regulation. The work includes exploring how metrology, specifically, the Rasch measurement method can be used to evaluate/measure the abilities and performance of AI language models such as accuracy and precision.