From Spectra To Colour: A Big-Data Computational Approach

Perceiving colour is of vital importance for our behaviour: it helps us to identify objects, see them quicker and remember them better. However, understanding the relationship between the physical properties of light and its colour is a complex computational problem. The colour of a surface cannot be predicted solely based on light spectra.

In fact, the light reflected off an object depends on both the surface spectral reflectance properties and the spectral composition of the illumination. Crucially, when the illumination changes, objects don't change colour as much as we would expect based on how much the reflected lights change. This phenomenon, usually referred to as Colour Constancy (CC), is a fundamental property of colour vision, as it allows us to recognise objects and their properties based on colour, despite the variable illumination conditions we experience in real life.

Although in our brain there is a representation of the colour of surfaces that remains constant under different illuminations, the neural computations to realise it are unknown. A good CC model should predict how colour changes across illuminations and how we perceived these changes. A significant limit for evaluating computational models of CC is the lack of naturalistic spectral images with changing illuminations. Surprisingly, perception is not often taken into account.

A fundamental computational challenge for colour vision is given by the fact that high dimensional spectral information (one dimension per wavelength, e.g. 400 dimensions) is reduced to three values: the responses of our three photoreceptors. Such values are used to build different colour representations at different stages of processing in the visual system, some of which are formally described by colour spaces.

Statistical analysis of photoreceptor responses to light reflected from natural objects showed that a simple linear transformation, meant to efficiently transmit information from the eye to the brain, could explain how colour opponency is realised in the retinae.

Non-linear approaches (e.g. Deep Learning) are able to explain perceptual mechanisms as a result of efficient encoding of the sensory input. By compressing information, it is possible to discover latent dimensions which represent the physical causes of the sensory input, i.e. the properties of the world that we perceive.

We have previously shown that by compressing tactile sensory input a deep neural network can learn sensitivity functions similar to the ones of tactile receptors on our skin. A similar approach could explain how the colour transformation implemented at different stages in the human visual system are linked with each other and have evolved as a result of being exposed to the statistical properties of our environment.

Although this idea follows a promising trend of the last years in computational neuroscience, Deep Learning requires a large dataset of spectral images.

Developments in Computer Graphics (CG) allow for physically accurate simulation of light reflections in complex scenes. Naturalism and complexity of visual scenes seem crucial to understand the mechanisms of CC, because contextual information presented by complex natural scenes (e.g., inter-reflections, specular highlights) plays an important role.

We will use CG to render a large dataset of naturalistic scenes with known spectral illuminants and surface reflectances. We could thus evaluate existing computational models for CC, based on the spectral ground-truth and on measures of colour constancy that we will collect online and in the laboratory with human participants. Building on these results, we will use our expertise on evolutionary computing to device a novel, complex, but interpretable model for CC, potentially outperforming existing ones.

Finally, we will use Deep Learning techniques to reduce spectral data to a lower dimensional representation that explains existing neurophysiological and perceptual data.