Scented AI? Integrating olfaction into Large Language Models

The relationship between language and sensory knowledge is a crucial question in the cognitive sciences.

With advancements in Large Language Models (LLMs), this research explores how language can form sensory knowledge without direct experiences.

It aims to empirically test and enhance LLMs´ ability to represent sensory experiences, focusing on olfaction, often termed the ´muted sense´ due to its limited language portrayal.By examining the capabilities of LLMs to develop consistent representations of olfactory concepts that align with human interpretations, this project addresses a critical gap in the development of multimodal AI systems.

Objectives include creating a comprehensive dataset reflecting human olfactory perception and developing the first olfaction-enriched open-source language model using this dataset and advanced fine-tuning techniques.

Finally, the first benchmark to systematically evaluate LLMs´ proficiency in olfactory processing will be developed to keep track of and facilitate further research.The significance of this research lies in its potential to advance our understanding of the representation of sensory experiences in language.

The model will serve both theoretical and practical benefits, such as improving diagnostic methods in healthcare and advancing product development in the perfume and flavor industries.