CAREER: Integrating Trust and Accountability into Compliance Enforcement for a Secure Internet of Things

Regulators have only recently begun to grapple with the reality of billions of vulnerable Internet of Things (IoT) products and have responded with targeted security and privacy regulations. The usefulness of such policy initiatives relies on their enforcement in practice. The enforcement strategy outlined in such regulations is similar to that used for software security compliance, wherein regulators delegate enforcement to Commercially Licensed Evaluation Facilities (CLEFs), which evaluate vendor products.

While such delegation is useful in scaling the enforcement to millions of products, it comes at a price: the affected party, i.e., the regulators and consumers who are the primary beneficiaries of security compliance, play a limited role in it, enabling an incentive structure skewed against effective enforcement. To elaborate, product vendors have little incentive to select an ideal CLEF that would thoroughly evaluate their product, instead of one that offers the fastest route to certification.

Even if a vendor searched for an ideal CLEF in good faith, they have few means to gauge the CLEF’s effectiveness aside from brochures, limited demonstrations, and the CLEF’s reputation. Moreover, CLEFs are not licensed on the basis of their performance at detecting vulnerabilities, but instead on procedural competence (e.g., adequate facilities, personnel).

Hence, traditional model, if applied as is to the IoT sector, would foster unvalidated CLEFs who have little incentive to improve, and vendors who simply view certifications as liability shields. This project seeks to avert such a future by empowering the affected party with practical tools to objectively measure the performance of CLEFs, and influence accountability in security compliance enforcement.

The systematic, data-driven, evaluation techniques developed in this research will enable regulators and standards bodies to reform the compliance infrastructure by directly evaluating the claimed performance of CLEFs as a part of the license-granting process or periodic audits. Moreover, this research will also help CLEFs and vendors improve through self-evaluation, help vendors seek effective CLEFs, and help CLEFs compete on the basis of performance.

By improving the compliance enforcement infrastructure for IoT, this project will generate tangible benefits for consumers in the form of secure IoT products, and has the potential to increase consumer confidence in and adoption of IoT technology. The research will be incorporated into graduate and undergraduate security classes at William & Mary through experiential learning activities, and disseminated to key stakeholders such as policymakers and developers, as well as the broader research community.

This project synergistically blends the approach of mutation testing with static and dynamic analysis, machine learning, and qualitative studies, to lay the foundation for empirically and systematically evaluating CLEFs, along three core research thrusts and a fourth thrust that investigates extensibility. The first thrust examines if the scope of work assumed by CLEFs is sufficient, by investigating a key underlying question: what should CLEFs look for?

To this end, the research acquires and analyzes IoT products at market-scale, in order to develop a generalizable understanding of what vulnerabilities are relevant to detect, i.e., pose risk in the IoT context, resulting in a comprehensive, risk-based IoT vulnerability taxonomy. The second thrust rigorously evaluates a CLEF’s ability to detect non-trivial variants of vulnerabilities from the taxonomy, i.e., mutants.

It develops a threat-aware mutation framework that generates mutants guided by a threat model for compliance enforcement that encapsulates the conditions CLEFs must account for, thus ensuring a non-arbitrary evaluation of CLEFs. The third thrust re-imagines security analysis for compliance enforcement with the approach of mutation-driven vulnerability prediction, which combines the strengths of machine learning and security-focused mutation for effective detection.

The fourth thrust explores the extensibility of the research to IoT product-types, application domains (e.g., smart cities), and usage paradigms. This research project leverages well-founded techniques from security, software engineering, and machine learning to make novel contributions at the intersection of security and software engineering. Finally, the initial focus on mobile-IoT apps as a target product-type will advance security research at the key intersection of mobile and IoT security.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.