Fmri of bilingual brain atl reveals language independent representations

Brain-Based Translation: fMRI Decoding of Spoken Wordsin Bilinguals Reveals Language-

Independent Semantic Representations in Anterior Temporal Lobe

Correia et al (2014). The Journal of Neuroscience

Statement of the problem

What are the neural mechanisms underlying the representation of language-independent semantic

concepts in the brain? “Horse” in English

“Paard” in Dutch

General findingsSeveral regions in the ATL (Anterior

Temporal Lobe) seem to be responsible for organizing language-independent semantic concepts. Specifically, at the fine-grained

level of within-semantic-category discriminations (e.g “animal” category).

“Horse” “Paard”

Claims EvidencePatients with ATL brain lesions demonstrate selective deficits of semantic knowledge, also known as semantic dementia.

A crucial function of the ATL is to act as a “semantic hub.” That is, language-independent representations of words and concepts are constructed in some regions of the ATL.

Previous work(Damasio et al., 1996; Pattersonet al., 2007)

The knowledge gap in the literature

How this paper attempts to address

it

Using bilingual participants, fMRI, a unique experimental paradigm, and a whole lot of complicated data analyses that I hope we can figure out together.

Lesion correlation evidence is insightful, but by no means abundant or consistent across patients (e.g. what counts as a lesion to ATL, what counts as semantic dementia).

Ideally, we’d want converging evidence from several kinds of data (behavioral, lesion, neurolinguistic...etc.). Some work has been done in this regard, but there exist some challenges to find reliable ATL activation in neuroimaging. These include “susceptibility artifacts” or “limited field of view.”

Recent studies have devised creative new techniques to overcome these challenges. This paper is one of them.

Important terms and concepts

● Within-language discrimination● Across-language generalization● Block experiment design

● Univariate Analysis● Multi-Voxel Pattern Analysis● Cortex-based alignment

● Repetition Time (RT)?● Acquisition Time (AT)?● Baseline?

● Classification● Training algorithms● Searchlight method● Support vector machine (SVM)

Experimental designIn separate Dutch and English blocks, bilingual participants were asked to listen to individual animal nouns (e.g. horse) and to detect occasional non-animal target nouns (e.g. bike). The goal of the task was to help maintain a constant attention level throughout the experiment to promote speech comprehension at every word presentation.

Importantly, each word was spoken by 3 different female speakers, which allowed for speaker invariant word discrimination.

Methodology

Participant criteria Ten bilinguals university students. All L1 Dutch, proficient L2 English (3 males, 7 females).Why bilinguals? Allows us to tease apart language-specific & language-independent representations in a way that monolingual data can’t.Stimuli Consisted of Dutch and English spoken words representing 4 different animals (bull, duck, horse, shark) and 3 inanimate object words (bike, dress, suit)Task Listen & detect non-animal words.Data collection fMRI Data analysis It’s complicated

Data analysis1. Functional and anatomical image acquisition2. fMRI data preprocessing3. MRI data analysis univariate statistics4. fMRI data analysis MVPA5. Classification (Feature extraction, Feature selection, Cross-

validation) 6. Discriminative maps



Whole-brain analysis was achieved with high-resolution (voxel size, 1x1x1 mm3) anatomical images.



They obtained an anatomically-aligned, group-averaged 3D surface representation so that subject data could be compared. They did this by preprocessing and analyzing fMRI data using Brain Voyager and custom-made MATLAB routines.

● Functional data were corrected for (3D motion, slice scan time differences, removing noise)

● Anatomical data were corrected for (intensity inhomogeneity, transformed into Talairach space).

● Functional & anatomical data then aligned to create 4D volume time courses (cortex-based alignment).

Data analysis1. Functional and anatomical image acquisition2. fMRI data preprocessing3. fMRI data analysis univariate statistics4. fMRI data analysis MVPA5. Classification (Feature extraction, Feature selection, Cross-


Functional contrast maps were constructed in order to identify the cortical regions involved in the processing of spoken words.

This was done by:(1) comparing activation to all animal words vs. baseline across subjects (2) combining all possible binary contrasts within nouns of the same language (3) grouping all equivalent nouns into single concepts and contrasting all possible binary combinations of concepts.

Functional contrast maps were submitted to a whole-brain correction criterion based on the estimate of the spatial smoothness of the map.

Univariate analysis is the simplest form of data analysis. It means that your analysis is considering only one variable. It doesn’t deal with causes or relationships (like MVPA does). Its major purpose is to describe, summarize and find patterns in the data with respect to that one variable.

fMRI data Univariate analysis

Here we’re looking at the fMRI responses elicited by all animal words across subjects, as computed by a functional contrastmap (t statistics) comparing all animal words versus baseline.



What can Multi-voxel pattern analysis (MVPA) do that Univariate analysis can’t?

Detect differences between conditions with higher sensitivity than conventional univariate analysis by focusing on the analysis and comparison of distributed patterns of activity. Data from individual voxels within a region are jointly analyzed.

Furthermore, MVPA is often presented in the context of "brain reading" applications reporting that specific mental states or representational content can be decoded from fMRI activity patterns after performing a "training" or "learning phase. In this context, MVPA tools are often referred to as classifiers or, more generally, learning machines. The latter names stress that many MVPA tools originate from a field called machine learning, a branch of artificial intelligence.

BrainVoyager introduces a comprehensive set of MVPA tools for locally distributed as well as more extended (sparse) patterns of activation. The tools include a multivariate searchlight mapping approach, which is used both for analyzing patterns in ROIs and for discriminating patterns that are potentially spread out across the whole brain.

BrainVoyager QX v2.8 | 2014 Rainer Goebel.

fMRI data analysis MVPA

Here we’re looking at the statistical maps of searchlight selections forwhich the word discrimination and the language generalizationanalyses yielded accuracies significantly above chance level(50%).

Data analysis1. Functional and anatomical image acquisition2. fMRI data preprocessing3. fMRI data analysis univariate statistics4. fMRI data analysis MVPA5. Classification (Feature extraction, Feature selection,

Cross-validation) 6. Discriminative maps

?



Accuracy maps of within-language word discrimination and across-language word generalization were constructed. Accuracy maps were averaged within each subject across binary comparisons and cross-validation folds. Thereafter, individual averaged accuracy maps were projected onto the group-averaged cortical surface and anatomically aligned using cortex-based alignment.

In the within-language discriminationanalysis, one map was produced per language and subsequently combined into a single map by means of conjunction analysis. The resulting discrimination map thus depicts regions with consistent sensitivity in English and Dutch. For the across-language word generalization, one map was produced from all possible binary language generalizations.

Within-language discrimination results

Conclusions

● That abstract horse representation exists in the brain and can be located. Brain-based decoding of individual spoken words at the fine-grained level of within-semantic category is possible within and across the first and second languages of bilingual adults.

● That representation exists somewhere in the ATL. Specifically, localized clusters in the left anterior temporal lobe, the left angular gyrus and the posterior bank of the left postcentral gyrus, the right posterior superior temporal sulcus/superior temporal gyrus, the right medial anterior temporal lobe, the right anterior insula, and bilateral occipital cortex.

● MVPA is a good approach for this kind of research question. Results indicate the benefits of MVPA based on the generalization of the pattern information across specific stimulus dimensions. This approach enabled examining the representation of spoken words independently of the speaker and the representation of semantic– conceptual information independently of the input language.

Questions

● So what? Why should we care where representations are or where processes happen in the brain?

● Could we translate this study (an fMRI study examining the location of abstract semantic representations in the brain) with other neurolinguistic tools (MEG, EEG...etc.). If so, how? Benefits or advantages of doing that?

● Do you feel their experimental design tested what they thought they were testing?

● If you were to change something about their experimental design or data analysis, what would it be?

Education

Fmri of bilingual brain atl reveals language independent representations