Faces of Inequality: An Empirical Study Using Passport Photos and AI to Assess Appearance-Based Discrimination in Sweden

Research problem and specific questionsLookism, appearance-based discrimination, is pervasive.

Studies suggest looks-based earnings inequality in the US rivals the racial wage gap (19), and returns to beauty surpass those to cognitive ability (20).

This project will be the first to study lookism in register data, enabling a large-scale analysis of a gravely understudied basis for discrimination.

The project sets out to measure and identify subjective appearance dimensions using passport photos and AI, document appearance-based inequalities across labor and marriage markets and education sectors, and investigate the extent to which lookism is influenced by or distinct from other forms of discrimination like sexism and racism.Data and methodTo tackle these questions, the project will utilize AI to analyze Swedish passport applications, paired with extensive individual-level data.

This innovative approach, supported by recent advancements in computer vision, will enable a detailed examination of appearance-based disparities.

The methodology is grounded in theories from economics, sociology, and social psychology, incorporating an intersectional perspective to reflect the complex, multifaceted nature of appearance valuation.Societal relevance and utilisationLookism is an often-overlooked form of discrimination that leads to significant injustices, comparable in severity to sexism and racism.

By analyzing how appearance impacts life outcomes, this project will provide insights for policymakers, educators, and social activists to develop interventions and policies promoting a more inclusive society.

The findings will also support NGOs and organizations advocating for diverse representation in media and advertising.Plan for project realizationThis three-year project unfolds in four primary stages: data collection, processing, analysis, and dissemination of results. A key challenge is the acquisition and analysis of passport photographs.

The PI´s expertise in deploying machine learning to handle large, sensitive registry datasets will be instrumental.

Additionally, the team will be supported by an advisory group composed of experts in discrimination, computer vision, and AI ethics from both academic circles and the private sector.

Leveraging this expertise, the project has been designed to minimize ethical risks and uphold personal integrity, employing strict data security standards, random sampling, pseudonymization, and by letting AI, not humans, rate photos.