The remains of a blue day: Synesthesia and language as MULTIModal EXpertise – MULTIMEX

Synesthesia is a fascinating phenomenon, because it is a normal condition that concerns the intimacy of subjective experience, shared probably by about 5% of the population. It offers a unique opportunity to study the neural bases of subjective experience, drawing on individual differences just like in neuropsychology, but with healthy people. Synesthetes experience systematic, additional associations. For example, they may arbitrarily associate a specific color to each day of the week (‘Born On A Blue Day’ as says the title of the popular book written by synesthete – and Asperger – Daniel Tammet), or to each letter or number. This project focuses on those so-called ‘grapheme-color’ synesthetes. Most proposed explanations of synesthesia so far suggested a neurological origin: synesthetes would exhibit extra neuronal connections between the neural centers responsible for grapheme identification and those related to color perception, leading to spurious activity of ‘color areas’ by graphemes. In a recent study (currently submitted for publication), two partners of this proposal evaluated this theory with functional and structural MRI. They observed no activation of ‘color areas’ by graphemes and no structural difference between synesthetes and control subjects in ‘color regions’. However, synesthetes had more white matter in the retrosplenial cortex. These results inspired the present experiments.
The result precisely obtained by Hupé and Dojat was the absence of localized correlates of synesthetic colors in the visual cortex. It was obtained by using standard fMRI analysis, requiring averaging across subjects the voxel BOLD response contrasted against two conditions. Such methodology only permits to identify neural processes that are localized enough in the brain. Neural correlates of synesthetic colors may however be distributed within the visual color system. Distributed processing of synesthetic associations would in fact make more sense, since it may seem odd that a unique, specific region of the visual cortex had specialized to code random associations between graphemes and colors – these associations being created (and then stabilized) by children possibly at a late developmental stage. We will evaluate this ‘distributed hypothesis’ by applying recently developed Multi-Voxel based Pattern classification Approaches (MVPA) to fMRI data.
Another possibility is that the neural correlates of synesthetic associations are localized (or distributed) outside of the visual cortex. In our study, we observed that synesthetes had more white matter in the retrosplenial cortex, a region at the crossroad of memory, emotional and attentional networks, and involved in language, leading us to envision that the retrosplenial cortex could play a critical role in expertise acquisition by rallying mnesic, emotional and attentional resources in especially difficult perceptual tasks – such as reading. Color associations made by synesthetes (created when they were children and learnt to read) would be the remains of the strong multimodal connections and multiples resources involved in language-related expertise acquisition – reflected as more numerous ‘associative’ connections at the retrosplenial cortex node in the adult. This project considers therefore synesthesia within the context of expertise acquisition and its relationship to written language and speech – especially at the level of its elementary units (graphemes and phonemes) that trigger synesthetic associations. Our main hypothesis is that the synesthetic remains of children expertise acquisition strategies would reflect in a more multimodal coding of language processing in adult synesthetes. We are therefore joining forces with language researchers in audition (speech), vision (reading) and sensory-motor integration (writing and speech production) to measure the multimodal coding of language in controls and synesthetes in four modalities with MVPA fMRI methodology.