What neuroimaging tells us about sensory substitution

Embed Size (px)

Text of What neuroimaging tells us about sensory substitution

  • Neuroscience and Biobehavioral Revie





    , 4

    7; a


    itutive modality or is it determined by the nature of the information

    n? This paper reviews the recent neuroimaging studies which have

    investigated the neural bases of sensory substitution. The detailed analysis of available results led us to propose a general scheme of the

    neural mechanisms underlying sensory substitution. Two different main processes may be responsible for the visual area recruitment

    observed in the different studies: cross-modality and mental (visual) imagery. Based on our results analysis, we propose that cross-

    Introduced by Bach-y-Rita in 1969, the sensory sub- ogies (Kaczmarek et al., 1991; Meijer, 1992, Capelle et al.,

    have been reproduced in early (Arno et al., 2001a; Sampaio


    Corresponding author. Centre de Medecine Nucleaire Hopital Neuro-

    et al., 2001) and late blind subjects (Cronly-Dillon et al.,1999). Not only these performances were found to beaccessible to blind subjects (Cronly-Dillon et al., 1999;

    0149-7634/$ - see front matter r 2007 Published by Elsevier Ltd.


    Cardiologique, 59 Bd Pinel 69677 BRON Cedex, France.

    Tel.: +33 472 684 961; fax: +33 472 357 345.

    E-mail address: christian.scheiber@chu-lyon.fr (C. Scheiber).stitution concept refers to the use of one sensory modalityto supply information normally gathered from anothersense (Bach-y-Rita et al., 1969). This information isacquired through an articial organ, and then transformedinto a meaningful signal for the substitutive system (Fig. 1).In the case of blindness, visual information can betransmitted through the auditory or tactile channels.Since 1969, several sensory substitution devices have

    been developed, using more and more advanced technol-

    1998). In parallel, several behavioural studies have been ledin order to evaluate the performances allowed by theseprostheses. Tactile- or auditory-for-visual substitutiondevices have been shown to allow blindfolded sightedsubjects to match vibrotactile to visual patterns (Epsteinet al., 1989), to discriminate pattern orientations (Sampaioet al., 2001) and to recognise visual patterns (Arno et al.,1999, 2001a) and graphic representations of objects(Cronly-Dillon et al., 1999). Training is necessary toachieve these performances. Some of these experimentsmodel implies that, with training, sensory substitution mainly induces visual-like perception in sighted subjects and mainly auditory or

    tactile perception in blind subjects. This framework leads us to make some predictions that could easily be tested.

    r 2007 Published by Elsevier Ltd.

    Keywords: Sensory substitution; Blindness; Cross-modality; Mental imagery; Visual perception; Neuroimaging


    References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1069modality is the predominant process in early blind subjects whereas mental imagery is predominant in blindfolded sighted subjects. Thisprostheses. Is the perception determined by the nature of the subst

    transmitted by the device? Is it a totally new, amodal, perceptioA major question in the eld of sensory substitution concerns the nature of the perception generated by sensory substitutionRe

    What neuroimaging tells u

    Colline Poiriera, Anne G. DeaLaboratoire de Genie de la Rehabilitation Neurale, Universite catholique

    bInstitut de physique biologique, UMR 7004

    Received 5 March 200ws 31 (2007) 10641070


    about sensory substitution

    oldera, Christian Scheiberb,

    Louvain, Avenue Hippocrate, 54 UCL-54.46, B-1200 Brussels, Belgium

    rue Kirschleger, 67085 Strasbourg, France

    ccepted 19 May 2007


  • ARTICLE IN PRESSobehFig. 1. (a) Schematic representation of the auditory-for-visual sensory

    substitution device developed by Capelle et al. (1998) and named PSVA,

    adapted to the fMRI environment. (b) The PSVA and its power supply.

    (c and d) Subject using the device in the MRI environment. Normally, the

    PSVA is connected to a tiny head-xed camera. As the subjects cannot

    move their head in the scanner, this camera was replaced by a non-

    magnetic joystick connected to a PC. Using this joystick, the subjects

    could move the patterns they were supposed to recognise. These

    movements made corresponding sounds to change according to the PSVA

    C. Poirier et al. / Neuroscience and BiSampaio et al., 2001), but they also were found moreaccurate as compared to those of blindfolded sightedsubjects (Arno et al., 2001a).More recently, Renier et al. have shown that an

    auditory-for-visual substitution device can mediate visualillusions (Renier et al., 2005a; 2006) and allow depthperception in blindfolded sighted subjects (Renier et al.,2005b). Using a pattern recognition task, it has also beenfound that, as in vision, blindfolded sighted subjects usingan auditory-for-visual substitution device better recognisedvertical bars than horizontal bars, these last ones beingbetter recognised than oblique bars (Poirier et al., 2006a).Subjects were also found to better recognise the size andthe spatial arrangement of the elements constituting thepatterns than the nature of these elements (vertical,horizontal and oblique bars). It is worth noting that theseresults match very well with visual perception rules (e.g.Morrison and Schyns, 2001; Miller and Navon, 2002).All these results raise the question of the nature of the

    perception induced by sensory substitution prostheses. Isthe perception determined by the nature of the substitutivemodality (i.e. tactile or auditory) or is it determined by thenature of the information transmitted by the device (i.e.visual)? Is it a totally new, amodal, perception? Neuroima-ging studies have recently brought partial responses to thisquestion.

    code. These sounds were transmitted via transducers (in the copper box, in

    image (d)) and dedicated plastic conducts that were inserted into the

    subjects ears. Headphones were added for isolation purpose. The plastic

    tube visible in image (d) contained a microphone at its non-visible

    extremity and allowed the experimenter to hear the verbal description

    made by the subject of each pattern.Using Positron Emission Tomography (PET), Arno andcolleagues (2001b) have shown that pattern recognitiontrough an auditory-for-visual device induced the recruit-ment of extra-striate occipital areas (BA 18 and 19) in earlyblind subjects but not in blindfolded sighted controls.Using the same PET technique but another device and adifferent task, Ptito and colleagues (2005) have foundsimilar results: a pattern orientation discrimination task,performed through a tactile-for-visual device stimulatingelectrically the tongue of the subjects, was found to recruitthe extra-striate occipital areas BA 18 and 19 only in blindsubjects but not in sighted controls. Another PET studyhas investigated the neural substrates of a depth perceptiontask through an auditory-for-visual device (Renier et al.,2004, 2005b). Based on three monocular depth cues (therelative target size, the proximity of the target to thehorizon and the linear perspective), this task was found toinvolve the extra-striate area BA 19 in blindfolded sightedsubjects whereas only a slight trend to visual activation wasobserved in early blind subjects. Finally, a FunctionalMagnetic Resonance Imaging (fMRI) study has shownthat pattern recognition through an auditory-for-visualdevice can induce the recruitment of striate (BA 17) andextra-striate (BA 18 and 19) areas in blindfolded sightedsubjects (Poirier et al., 2007) (Fig. 2).The major nding of these studies lies in the recruitment

    of brain areas (BA 17, 18 and 19) usually considered asvisual areas, in addition to auditory or somatosensorycortex activation, in blindfolded sighted (Renier et al.,2005a, b; Poirier et al., 2007) and early blind subjects (Arnoet al., 2001b; Ptito et al., 2005). Two major differentinterpretations of these results can be made.First, visual area activation can reect the use of mental

    (visual) imagery strategies. Visual imagery is known toinduce the recruitment of the striate and extra-striate areasin blindfolded sighted subjects (Kosslyn et al., 1995,Kosslyn and Thompson, 2003). To a lesser extent, earlyblind subjects seem also be able to perform mental imagerytasks (Marmor and Zaback, 1976; Kerr, 1983). The natureof imagery performed by blind subjects, visual or not,remains a source of debate. Nevertheless, this process wasalso found to induce the recruitment of the striate andextra-striate areas in this subject population (De Volder etal., 2001; Vanlierde et al., 2003; Lambert et al., 2004).Second, cross-modality could account for the observed

    results. Cross-modality consists in the recruitment of brainareas normally devoted to processing information fromone sensory modality by the processing of informationcoming from another modality. This phenomenon is wellknown in blind subjects, in whom auditory and tactilestimuli induce visual area recruitment (for a review, seeTheoret et al., 2004). However, recent studies have shownthat this phenomenon also occurs in sighted subjects in amulti-sensory but also in a uni-sensory context. Variousauditory and tactile tasks were found to induce the

    avioral Reviews 31 (2007) 10641070 1065recruitment of the visual areas (e.g. Amedi et al., 2001;Blake et al., 2004). Nevertheless, this process seems to be

  • ARTICLE IN PRESSbehC. Poirier et al. / Neuroscience and Bio1066less important in sighted than in blind subjects (Poirieret al., 2006b).Both hypoth