Hidden Markov event sequence models: Toward unsupervised functional MRI brain mapping1

Hidden Markov Event Sequence Models: TowardUnsupervised Functional MRI Brain Mapping1

Sylvain Faisan, PhD, Laurent Thoraval, PhD, Jean-Paul Armspach, PhD, Jack R. Foucher, PhD,Marie-Noëlle Metz-Lutz, PhD, Fabrice Heitz, PhD

Rationale and Objectives. Most methods used in functional MRI (fMRI) brain mapping require restrictive assumptionsabout the shape and timing of the fMRI signal in activated voxels. Consequently, fMRI data may be partially and mis-leadingly characterized, leading to suboptimal or invalid inference. To limit these assumptions and to capture the broadrange of possible activation patterns, a novel statistical fMRI brain mapping method is proposed. It relies on hidden semi-Markov event sequence models (HSMESMs), a special class of hidden Markov models (HMMs) dedicated to the model-ing and analysis of event-based random processes.

Materials and Methods. Activation detection is formulated in terms of time coupling between (1) the observed sequenceof hemodynamic response onset (HRO) events detected in the voxel’s fMRI signal and (2) the “hidden” sequence of task-induced neural activation onset (NAO) events underlying the HROs. Both event sequences are modeled within a singleHSMESM. The resulting brain activation model is trained to automatically detect neural activity embedded in the inputfMRI data set under analysis. The data sets considered in this article are threefold: synthetic epoch–related, real epoch–related (auditory lexical processing task), and real event–related (oddball detection task) fMRI data sets.

Results. Synthetic data: Activation detection results demonstrate the superiority of the HSMESM mapping method with respectto a standard implementation of the statistical parametric mapping (SPM) approach. They are also very close, sometimes equiva-lent, to those obtained with an “ideal” implementation of SPM in which the activation patterns synthesized are reused for analy-sis. The HSMESM method appears clearly insensitive to timing variations of the hemodynamic response and exhibits low sensi-tivity to fluctuations of its shape (unsustained activation during task). Real epoch–related data: HSMESM activation detectionresults compete with those obtained with SPM, without requiring any prior definition of the expected activation patterns thanksto the unsupervised character of the HSMESM mapping approach. Along with activation maps, the method offers a wide rangeof additional fMRI analysis functionalities, including activation lag mapping, activation mode visualization, and hemodynamicresponse function analysis. Real event–related data: Activation detection results confirm and validate the overall strategy thatconsists in focusing the analysis on the transients, time-localized events that are the HROs.

Conclusion. All the experiments performed on synthetic and real fMRI data demonstrate the relevance of HSMESMs infMRI brain mapping. In particular, the statistical character of these models, along with their learning and generalizingabilities are of particular interest when dealing with strong variabilities of the active fMRI signal across time, space, ex-periments, and subjects.

Key Words. Functional MRI; functional brain mapping; hidden Markov models; activation detection.

© AUR, 2005

Acad Radiol 2005; 12:25–36

1 From the Université Louis Pasteur, Strasbourg, France (S.F., L.T., F.H., J.-P.A., M.-N.M.L.), the Laboratoire des Sciences de l’Image, de l’Informatique et dela Télédétection, ENSPS LSIIT - Boulevard Sebastien Brant BP 10413 - 67 412 Illkirch Cedex, France (S.F., L.T., F.H.), the Institut de Physique Biologique,IPB - 4, rue Kirschleger - 67085 Strasbourg Cedex, France (S.F., J.-P.A., M.-N.M.L.), Institut National de la Santé et de la Recherche Médicale, HôpitauxUniversitaires, Place de l’Hôpital - BP 426 - 67091 Strasbourg Cedex, France (J.R.F.). Received June 8, 2004; revision received September 13, 2004; revi-sion accepted September 27 2004. Address correspondence to S.F., L.T. , [email protected], [email protected]

©
AUR, 2005doi:10.1016/j.acra.2004.09.012
25

FAISAN ET AL Academic Radiology, Vol 12, No 1, January 2005

Functional magnetic resonance imaging (fMRI) extendsconventional anatomic imaging to include localization andmapping of active brain areas engaged during perceptions,actions, and cognitive tasks. Most existing methods usedin fMRI brain mapping require strong prior assumptionsabout the shape and timing of the fMRI signal in acti-vated voxels (1–6). These assumptions are generally ex-pressed through the use of a limited set of reference func-tions to model the noise-free activated fMRI signal, as inthe popular statistical parametric mapping (SPM) ap-proach (7). Each reference function is implemented as thelinear convolution of a predefined impulse response func-tion, the putative hemodynamic response function (HRF),with a deterministic timing function encoding the sup-posed sequence of task-induced neural activations. But inpractice, shape of the HRF and activation timing mayvary across stimulus (habituation, learning, and fatiguephenomena), and across brain areas, stimulation parame-ters, experimental sessions, and subjects (8–10). Conse-quently, fMRI data may be partially and misleadinglycharacterized leading to suboptimal or invalid inference.

To capture the broad range of possible activations pat-terns, Thoraval et al. (11,12) recently proposed to resolvefMRI brain mapping based solely on the relevant nonsta-tionary items of the fMRI signal that are the hemody-namic response onsets (HROs). In their approach, activa-tion detection is performed at the voxel level by scoringthe relative alignment of the sequence of HROs detectedin the fMRI signal onto a deterministic model of the hid-den sequence of task-induced neural activation onsets(NAOs). Despite the promising results obtained, themethod suffers from the introduction of heuristic costfunctions in the time alignment scoring process. In thisarticle, we propose a novel statistical fMRI brain mappingtechnique that reformulates the time alignment principlebetween observed HRO events and hidden NAO eventswithin the statistical modeling framework of hidden semi-Markov event sequence models (HSMESMs). HSMESMsare particular hidden semi-Markov models (HSMMs) ded-icated to the modeling and analysis of event-based ran-dom processes (13). HSMESMs belong to the wide rangeof hidden Markov modeling (HMM) techniques (14). HS-MESMs were first described by the authors in the contextof cardiac arrhythmia recognition (15). The statisticalcharacter of these models proved to be of great interest infMRI brain mapping when dealing with shape and timingvariability of the active fMRI signal. Moreover, the learn-ing and generalizing abilities of HSMESMs allowed auto-
matic detection of in-phase and delayed activations em-
26

bedded in any given set of fMRI signals, making the pro-posed statistical brain mapping approach completelyunsupervised.

MATERIALS AND METHODS

fMRI Data Sets

Imaging procedure.—Functional images were acquiredon a 2-T whole-body S200 Bruker MRI system (Wissem-bourg/Cedex, France) with a head volume coil, usingecho-planar imaging (EPI) with an axial slice orientation(32 slices). Each data set consisted in a sequence of T �145 three-dimensional (3D) scans of size 64 � 64 � 32(voxel size � 4 � 4 � 4 mm), with echo time (TE) �43 milliseconds and repetition time (TR) � 5 seconds.The number of intracranial voxels per sequence was about17,000. Anatomic imaging used fast spin echo (TE �90 milliseconds; voxel size � 2 � 2 � 2 mm). BeforeHSMESM brain mapping, all fMRI scans of a sequencewere registered to the first scan using an automated regis-tration algorithm (16). A Gaussian prefiltering in space(full width at half maximum [FWHM] � 8 mm) and time(�2 � 8 s) was then performed to the raw fMRI data toincrease the fMRI signal-to-noise ratio (SNR).

Synthetic epoch-related data.—Five synthetic data setswere derived from a single real noise fMRI data set, us-ing fake activation patterns embedded at known locations.The fMRI noise was derived from a “null condition” ex-periment of no activation during which a healthy subjectwas asked to rest, to be motionless as possible, and to notperform any cognitive task. Four cubic areas 6 � 6 � 6voxels were manually selected within the brain volume astrue active areas. Magnitudes of added activation patternsat active locations were specified as some positive frac-tion, m, of series standard deviation at each location,computed after baseline drift removal.

The first synthetic data set, DS1, was designed to illus-trate variability in the shape of the hemodynamic re-sponse to a stimulation block (unsustained activation dur-ing task). To this end, four noise-free activation patternswere computed by convolving the HRF model proposedby the SPM software (difference of two gamma func-tions) with four deterministic timing functions. These fourlatter functions were all derived from the same “ex-pected” boxcar-like timing function (150 scan length, 7blocks ON interleaved with 8 blocks OFF, 10 scanseach). In order to ensure a minimum of 95% of activation
detection performance for the “ideal” SPMb implementa-

Academic Radiology, Vol 12, No 1, January 2005 UNSUPERVISED FUNCTIONAL BRAIN MAPPING

tion of SPM (see Results), the fraction m was set to 1.6in all active areas and to 0 outside these regions. Oneactivation pattern was finally assigned to each active area.

The second synthetic data set, DS2, was designed toillustrate timing variability between stimulation paradigmand neural/hemodynamic response (activation delay). Fouractivation patterns were obtained by convolving the aboveHRF model with four delayed versions (delay of 2, 4, 6,or 8 scans) of the aforementioned “expected” boxcar-liketiming function. The fraction m was also set to 1.6 in allactive areas. One delayed activation pattern was then as-signed to each active area.

The last three synthetic data sets, DS3, DS4, and DS5,were designed to illustrate signal to noise variability. Foreach data set, four activation patterns were obtained byconvolving the HRF model of SPM with four exact repli-cations of the aforementioned “expected” boxcar-like tim-ing function. The fraction m was set to 1.6, 0.8, and 0.5in all active areas of the data set DS3, DS4, and DS5,respectively, so that DS5 is the most noisy of all threedata sets. One delayed activation pattern was then as-signed to each active area.

Real epoch–related data.—Another 42 real fMRI datasets were acquired from 17 healthy subjects ages 18 to 39years who were asked to perform auditory lexical pro-cessing using EXPE 6 (17). The auditory stimuli usedwere frequently used nouns of the French lexicon checkedfor concreteness and imagery (18). These nouns were lis-tened to through earphones with high noise attenuation.The subjects were asked to not perform any active task,except to pay as much attention as possible to real words,so as to be able following the testing to recognise themout of a list of 80 written nouns. In the active state, theylistened to a series of concrete French nouns presented ata rate of 1 per second. In the control state, the subjectslistened passively to the same nouns played backwards atthe same rate (19).

HSMESMs: TheoryTo understand what HSMESMs are, one should start

from HMMs. HMMs are a statistical modeling techniquewell suited for the analysis of random processes that arestructured in time (speech is an illustrative example ofsuch processes). Let P be such a process, and � an HMMof P. Then, � is defined as a doubly stochastic process{ot,qt}, where:

{ot} is the observable process of �. The so-called ob-servation sequence O � o1o2 . . . oT is usually derived by
preprocessing of P. It is representative of the short term
statistical characteristics of P over the discrete observationinterval [1,T].

{qt} is the hidden process of �. It is modeled by aMarkov chain statistically representative of the possiblestate-based evolutions of P.

The process {qt} is said to be “hidden” because it canbe observed only through the observable process {ot}, theone statistically depends on {qt}. That is, the distributionof ot is defined as a probabilistic function of the hiddenstate qt � Si occupied by P at t. The finite set {Si} repre-sents the state space of �, or equivalently, of P. For athorough presentation of HMMs, the reader is referred tothe well-known Rabiner’s tutorial (14).

HSMMs are an extension of the standard HMMs toexplicit modeling of state occupancy duration. To accountfor this modeling aspect, the hidden process {qt} of anHSMM is modeled by a semi-Markov chain. Again, thereader is referred to the Rabiner’s tutorial (14) and to thepioneering works of Ferguson (20), Russel et al. (21), andLevinson (22) for an understanding of durational model-ing in HSMMs.

Finally, HSMESMs are a particular instance ofHSMMs dedicated to the modeling and analysis of event-based random processes. The remainder of this sectionpresents the salient features of HSMESMs within the gen-eral context of hidden Markov modeling. Further detailson HSMESMs are provided elsewhere (13,15).

Preprocessing, event sequence, observation sequence.—In an HSMESM approach, a preprocessing step detectsand characterizes events of interest in the input process Punder analysis. Let E � e1e2 . . . eL be the sequence ofdetected events and �E � {t1 � 1, t2, . . . , tL � T} theassociated set of event detection times. Let O �

o1o2 . . . oT be, as usual, the observation sequence, and �O

� {1,2, . . . ,T} the set of observation times. Then, by def-inition, O is built on E by inserting a null event, �, in Eeach time t��E� � �O � �E. Writing ot � � means a miss-ing observation occurred at time t. As an example, if T �

20 and �E � {1,3,6,12,15,20}, then E � e1e2e3e4e5e6 andO � e1�e2��e3��e4��e5��e6.

Elements of an HSMESM.—Because of the detection-based preprocessing, the HSMESM observation sequenceO derived from P is usually composed of true positiveevents (tpe) mixed with false-positive events (fpe) andmissed observations (null). Let C-2 be the number of tpeclasses that characterize P. Then, the state space of (themodel of) P results from the union of two state sub-
spaces, namely:
27


Stpe � {Si; i�1, . . . ,C}, where Si is a Markov stateproducing tpe at times tl in �E according to its associatedoutput observation pdf bj�otl�. S1 and SC are the start andfinal states of the hidden process.

Sfpe/null � {Sij; i,j�1, . . . ,C}, where Sij is a semi-Markov state interleaved between Si and Sj. Sij is occupiedd – 1 times when the model transits from Si to Sj, with aninter-tpe state duration d explicitly modeled by the dura-tional pdf pij(d) in accordance with the semi-Markoviannature of Sij. Sij produces fpe or null at times t in �E�� ac-cording to its composite output observation pdf bij

��ot� � bij

�el�·eij1·1�E�t� � eij2·1�E��t� �1X�t� � 1if t � X, 0 otherwise)built on the latent Bernoulli process of parameter eij1 � 1 �eij2.

To completely specify an HSMESM � of P, one needs todefine the C parameter, the transition probability matrix A �{aij} between states, the observation pdfs B �{bj(.)} andB= � {bij(.)}, the durational pdfs D � {pij(.)} and the emis-sion matrix E � {eij1}. A possible topology for � is depictedin Fig 1b with C � 5. Circles represent the tpe states Si,whereas the bicolor boxes in between correspond to the fpe/null states Sij. From a generative point a view, � behaves asfollows. In S1 at t1, the model emits e1. At some tk, 1 � k �L, in Si, it selects the next state Sj to visit according to thetransition probability distribution {aij}. Sj is actually visitedat tl � tk � d, l � k, with d randomly drawn according tothe inter-tpe distribution pij(d). Before reaching Sj, the modelremains d – 1 times in Sij to emit fpe or null accordingto bij

��.�. This procedure is repeated until reaching SC at tL �T to emit eL. As an example, observing the subsequence ofFig 1a implies occupying the hidden process level at the

Figure 1. Hidden semi-Markov event sequence model example.(a) Output observation sequence with some of the corresponding
visited states. (b) Topologic representation (see text).
28

state Si to emit e10, then for Sij to emit the four null mixedwith the two fpe (e11 and e12), and finally Sj to emit e13.

Algorithmic aspects.—Similarly to the HMM or HSMMcase, the three following basic problems have to be solvedfor the HSMESMs to be used in practical applications suchas fMRI brain mapping:

● Evaluation: Given an observation sequence O, and anHSMESM �, compute P[O /�], the likelihood of theobservation sequence given the model.

● Decoding: Given an observation sequence O, and anHSMESM �, infer the hidden state sequence q* �(q1q2 . . . qT)* that best explains O.

● Learning: Adjust the HSMESM parameter set(A,B,B=,D,E) of � so as to maximize the likelihoodL(�) � v�1

V P[OV��] of a training set {Ov} of V ob-servation sequences assumed mutually independent.

The evaluation, decoding, and learning problems canbe solved based on adapted versions of the forward-backward, Baum-Welch, and Viterbi algorithms, re-spectively (14,23). These versions are detailed else-where (13). Note finally that the likelihood P[O��]asymptotically decreases to 0 and may underflow as theobservation sequence O gets longer, by simply multi-plying probabilities. However, in practice, the likeli-hoods are generally computed and compared for obser-vation sequences of same size T (in our fMRI applica-tion, T is equal to the number of brain volumes thatcompose the input fMRI data set under analysis). Be-sides, normalization methods exist that can solve theproblem of probability underflow (14).

HSMESMs: Application in fMRI Brain MappingIn the fMRI brain mapping approach proposed, the

random process P under analysis is the local neuralactivation process at voxel v. This process is “hidden”but can be analyzed through the corresponding fMRIsignal f v(t). The mapping method developed hereafterrelies on this simple premise: in the presence of neuralactivation at location v, the sequence of HROs ob-served in the signal f v(t) should align, to some extent,onto the hidden sequence of task-induced NAOs(12,24). We propose to model both sequences and theirtime coupling in a single HSMESM �. In this model,the observation sequence Ov is representative of thesequence of HRO candidates observed at location v. Itis obtained by preprocessing f v(t). The hidden part of �

statistically accounts for the timing of the NAO events


underlying the HRO events. It is built from the timinginformation of the input stimulation protocol.

fMRI data preprocessing.—A wavelet-based prepro-cessing approach is used to detect HROs of interest in the(active) fMRI signal. HRO detection relies on the particu-lar behavior of these transients in the phase plane domainof the continuous wavelet transform of f v(t). Each time tl

an HRO event is detected, a feature vector el is derived.The voxel’s observation sequence Ov is then built on theevent sequence E � {el} by following the procedure de-

Figure 2. Brain activation modeling. (a) Stimulation paradigmwith C � 5 blocks, among which blocks 1 and 5 are fictiveblocks. Vertical arrows indicate the OFF-ON transition of eachblock with the corresponding occurrence time t’c. (b) Schematicrepresentation of the hidden neural event timing where the hori-zontal arrows underline timing variability of the neural activationonsets (NAOs). (c) Corresponding brain activation semi-Markovevent sequence models with the tpe states (circles) and aggre-gate states in between (boxes). The vertical dashed lines from(a) to (c) reflect the one-to-one registration assumed between aparadigm OFF-ON transition, the induced NAO, and the observedhemodynamic response onset event.

Figure 3. Unsupervised learning-mapping
their representative brain activation semi-Marko
scribed in the previous section. Technical details on thiswavelet-based approach may be found in (11,25).

Brain activation modeling.—Let us consider a binaryactivation-baseline paradigm composed of C stimulationblocks sharing the same basic OFF-ON pattern. Amongthem, two are fictive blocks introduced at the beginningand the end of the paradigm for duration modeling pur-poses (Fig 2a). Though the local sequence of task-inducedNAOs is hidden, its timing (Fig.2b) can be statisticallymodeled from the timing information of the stimulationparadigm. The modeling procedure is as follows:

● C tpe states Si are used, one per OFF-ON paradigmtransition, with a state index i reflecting the transitionorder of appearance in the paradigm.

● A semi-Markov state Sij is inserted between each validtpe state pair (Si, Sj), that is, for which a state transitionis allowed (aij � 0).

● A left-right topology is selected for the chain with theadditional constraints aii � 0, for all i, and aij � 0 ifj � i � , with C/2, to avoid an active fMRI sig-nal “jumps” too many successive stimulation blocks atonce.

An example of brain activation model is depicted inFig 2(c) with � 2 and C � 5. For simplicity of themodel and consistency of its parameter estimates, all ob-servation and durational pdfs are specified as one-dimen-sional Gaussians. Finally, to reduce the amount of param-eters to be estimated, the concept of parameter tying isextensively used (14).

Unsupervised learning and mapping of active brainfMRI signals.—Learning and mapping of brain activesignals represents the major analysis functionality of

edure of brain active signals along with
proc v event sequence model (see text).
29


HSMESMs in fMRI brain mapping. This functionalityrelies on the resolution of the HSMESM learning andevaluation problems discussed previously. Learning andmapping of brain active signals are performed jointly inthe sake of a brain activation HSMESM, �opt, well-suitedfor the input fMRI data set (DS) under analysis. The pro-cedure consists in building automatically and iteratively aseries of refined brain activation models, {�i}, along witha series of active signal sets, {Si}, where Si � DS for alli. Composed of all the signals declared active by �i, Si isused as learning set for �i�1. Both series converge respec-tively to �opt and Sopt in that learning from Sopt yields �opt,which, in turn, declares active Sopt. The overall procedureis depicted in Fig 3. The initial model �1 is derived fromsimple statistics performed on the input data set DS. Thelikelihood map LMi is made of the voxel likelihoods l �

P(O|�i) (see the Evaluation problem) for which a P valuep � �l

�P�x�H0�dx is calculated. P(x|H0) denotes the pdf

of the likelihood x under the null hypothesis H0 of cere-bral inactivity. This distribution is inferred experimentallyfrom a large set of random observation sequences, repre-sentative of chance, obtained by a random mixing proce-dure applied to the intracranial event sequences. The Pvalue map associated with LMi is thresholded at a givenP value to yield the active signal set Si. If Si coincideswith Si-1, the learning/mapping procedure is stopped withSi declared as the final brain activation map Sopt. Other-wise, Si is reused to estimate �i�1 (see the Learning prob-lem) based on the Baum-Welch algorithm. It must benoted that the procedure converged for all fMRI se-quences tested, and independently of the initial model �1.

More insight into neural activation: Viterbi decoding.—Along with activation mapping, Viterbi decoding offers awide range of additional fMRI data analysis functional-ities. Indeed, Viterbi decoding of the fMRI signals de-clared active by mapping gives access, through the localhidden state sequence qv* that best explains Ov, to the besttpe state sequence qtpe

v∗ embedded in. Therefore, it alsogives access to the occurrence time sequence of the truepositive HROs decoded at location v and to the sequenceof stimulation block numbers associated with it. Both se-quences may be exploited in various ways to informabout how brain activated regions have responded to theexternal stimulation:

● Activation lag mapping provides maps of the activa-tion lag estimate at each active voxel v, the one can be
computed from the average difference between the
30

HRO times and the stimulation onset times associatedwith.

● Activation mode visualizing provides histogram-likesubplots of the neural response rate of a given region toeach input stimulation block.

● Hemodynamic response analysis can be performedbased on subplots of the mean waveform of the hemo-dynamic response computed for a given activated re-gion and a given stimulation block.

RESULTS

The HSMESM-based brain mapping method wasapplied to the synthetic and real epoch–related datadescribed previously. Because of its widespread use inthe fMRI community, the SPM method was employed(SPM2 release) as the main comparator of theHSMESM method.

Synthetic Data SetsSince the “ground truth” is known for all synthetic

data sets, activation detection performance of theHSMESM and SPM methods are compared from activa-tion maps and receiver-operating characteristic (ROC)curves. For a better evaluation, two implementations ofthe SPM method are considered. The first implementation,SPMa, corresponds to a “real” application case where the“ground truth” is unknown. It makes use of a single re-gressor, namely, the HRF model proposed by the SPM2software convolved with the “expected” boxcar-like tim-ing function mentioned previously, and depicted in dottedline Fig 4d. Conversely, the second implementation ofSPM, SPMb, corresponds to a somewhat “ideal” applica-tion case where “ground truth” is known and exploited.The four noise-free activation patterns that were used tobuild either DS1 or DS2, are reused in SPMb as four ad-ditional regressors to the SPMa implementation. Thisdouble implementation of SPM is unnecessary for thetriplet of data sets (DS3, DS4, DS5), since no differenceexists between “expected” and observed activation pat-terns (see lower part of Fig 4d). In the particular case of(DS3, DS4, DS5), the SPMa and SPMb implementationsare strictly equivalent. They both correspond to the same“ideal” application case of SPM. Therefore, in order to beconsistent with the notations used for DS1 and DS2, thisimplementation will be denoted SPMb rather than SPMa.Let us recall that the HSMESM method has no knowl-edge about the “ground truth,” the method being com-
pletely unsupervised.


The activation detection results obtained for DS1,DS2, and the triplet of data sets (DS3, DS4, DS5), arerespectively shown in the top, middle, and bottom rowof Fig 4. SPMa and HSMESM activation maps are de-picted in Fig 4a and b, respectively, for the DS1 (top),DS2 (middle), and DS3 (bottom) data sets. Because ofthe equivalence of SPMa and SPMb for the DS3 dataset, the SPMa activation map at the lower part of Fig4a should rather be considered as an (“ideal”) SPMbactivation map. All maps are depicted for a false posi-tive rate (FPR) of 0.005. True positives are representedin black, while false positives and false negatives are

Figure 4. Results on synthetic epoch-related data, for the data serow). (a) Activation maps obtained by the SPMa (top, middle) andSPMb map was obtained for the DS3 data set. (b) Activation mapspositives are represented in black, while false positives and false ntained by the HSMESM (‘*’), SPMa (‘□’), and SPMb (‘o’) methods fSPMb methods for the DS3 (‘�’,‘o’), DS4(‘�’,‘�’), and DS5 (‘*’,‘□’)tively. (d) Synthetic noise-free activation patterns used for evaluatiotion pattern (dotted line). For convenience, all activation patterns afurther details.

represented in white. The upper subplot and middle

subplot of Fig 4c represent the ROC curves obtainedby the HSMESM (‘*’), SPMa (‘□’) SPMb (‘o’) meth-ods for DS1 and DS2, respectively. The lower subplotof Fig 4c represents the ROC curves obtained by theHSMESM and SPMb methods for the DS3 (‘�’,‘o’),DS4 (‘�’,‘�’), and DS5 (‘*’,‘□’) data sets, that is, foran SNR m equal to 1.6, 0.8, and 0.5, respectively. Thenoise-free activation patterns used to build the data setsare illustrated in Fig 4d, before convolution with theHRF model proposed by the SPM2 software. The fouractive areas in any activation map are arranged accord-ing to the lexographic order while a top-bottom order

S1 (top row) and DS2 (middle row), and (DS3, DS4, DS5) (bottomPMb (bottom) implementations of SPM for an FPR of 0.005. Theined by the HSMESM method for the same FPR of 0.005. True

ves are represented in white. (c) Top, middle: ROC curves ob-1 and DS2. Bottom: ROC curves obtained by the HSMESM andsets, that is, for an SNR m equal to 1.6, 0.8, and 0.5, respec-in line), and obtained by transformation of the “expected” activa-resented before convolution with the HRF model. See text for

ts Dthe S

obtaegatior DSdatan (th

re rep

is used for the activation patterns.

31

4 (thi


Activation detection results obtained for DS1 demon-strate the relative insensitivity of the HSMESM method tovariations of the hemodynamic response to a stimulationblock. With respect to SPMb, a maximum fall of only 7%of the activation detection performance is observed forthe HSMESM method. From the HSMESM map of Fig4b, one should note an important increase of false nega-tives for the fourth active area, that is, where the hemody-namic response to the stimulation block is the most un-sustained and the temporal support of the response is onehalf shorter than expected.

Activation detection results obtained for DS2 demon-strate clearly the complete insensitivity of the HSMESMmethod to timing variations of the hemodynamic responsewith respect to the input stimulation blocked paradigm.HSMESM detection results are excellent whatever thefalse positive rate and the activation delay are. They areequivalent to that obtained by SPMb, but without need-ing, as in SPMb, any prior knowledge about the delays ofthe synthetic activation patterns used.

Activation detection results obtained for the triplet ofdata sets (DS3, DS4, DS5) demonstrate that the HSMESMmethod proves to be very competitive when compared toan “ideal” application case of SPM. SPMb and HSMESMactivation detection results obtained for DS3 (m � 1.6)

Figure 5. Results on real epoch-related data for an auditory lexic(HSMESM) map (top left), HSMESM activation lag map (top right),phase activation map (bottom right) computed for a same subject.ing” activation (bottom) modes computed for the same active brainhemodynamic response waveforms (thick line) computed from thebrain regions of two subjects in response to the stimulation block

are equivalent and optimal. In the case of DS4, where the

32

noise is stronger than the signal (m � 0.8), a maximumdecrease of only 5% in detection performance is observedfor the HSMESM method with respect to the (here opti-mal) SPMb method, and for a low false positive rate of0.001. Finally, when noise becomes very important, as inthe DS5 data set (m � 0.5), activation detection perfor-mance of both methods suddenly decreases while theSPMb method clearly outperforms the HSMESM method.This latter result is not surprising, having in mind the realapplication case of SPM considered here. Also, it pointsout the importance of making use of all the information(the samples) of the fMRI signal when dealing with verynoisy data, as does the SPMb method, rather than relyingon sparse yet signifigant transients that are the HROs, asdoes the HSMESM method. However, even if less effi-cient in such a noisy context than the “ideal” SPMbmethod, the HSMESM strategy proves to be really fruit-ful to automatically capture the wide range of possibleactivation patterns, as demonstrated by the results ob-tained for DS1, DS2, DS3, and DS4.

Real Data SetsAn illustrative example of HSMESM mapping results

obtained for the auditory lexical processing task is shownin Fig 5a. The group of four activation maps is composed

cessing task. (a) Hidden semi-Markov event sequence modeltical parametric mapping map (bottom left), and HSMESM in-ctivation mode visualizing: “sustained” activation (top) and “fad-on of two subjects. (c) Hemodynamic response analysis: meandynamic response patterns (�) observed for the same active

n line) of the auditory lexical processing paradigm.

al prostatis(b) Aregi

hemo

of an HSMESM map (top left), an HSMESM activation


lag map (top right), an SPM map (bottom left), and anHSMESM in-phase activation map (bottom right). TheSPM map was obtained using a standard implementationof the SPM2 software, similar to the SPMa implementa-tion described previously, using Gaussian random fieldsand a corrected P value of 0.05. The HSMESM map wasobtained for a noncorrected P value of 0.01 because ofthe absence of spatial oversampling as in SPM. TheHSMESM in-phase activation map is derived by filteringthe activated voxels in the original HSMESM activationmap whose activation lag estimate falls between –2 and�2 scans. Activated brain regions are represented inwhite, except for the HSMESM activation lag map inwhich a gray level scale graduated in number of scans isused to represent the local activation lag estimate.

Figure 5a shows the activation maps on a horizontalsection 20 mm from the anterior commissure–posteriorcommissure (AC-PC) line. The SPM map shows signifi-cant activations in the inferior frontal syrus (IFG) (BA46/45), the mid-frontal gyrus (FG) (BA10), and the poste-rior mid-temporal gyrus (TG) (BA39/19). The HSMESMin-phase activation map consistently shows activations inthe same cortical areas. In addition, the HSMESM activa-tion map shows delayed activations in the right posteriorsuperior and mid-TG, left anterior superior frontal gyrus(SFG), and bilaterally in the anterior and posterior cingu-lar gyrus and the precuneus. More generally, theHSMESM brain mapping results were cross-validatedwith those of the SPMs in the 42 subjects studied. Noadditional in-phase activated brain regions were detectedby the HSMESM method.

As illustrated in Fig 5a, an activation lag map may beuseful for analyzing delayed activations with respect tothe activation task. It may also be of interest for the studyof the shifting in local brain activity when moving fromone task to the other as it is the case in the ON-OFF par-adigm. As an example, from the delayed activations ob-served within the right posterior superior and mid-TG andleft anterior SFG of Fig 5a (top left), one may speculatethat they reflect the auditory processing of backwardwords (19). Indeed, in the control task of the stimulationparadigm, the subjects face human voice articulatingspeech that does not fit either the canonical phonologic orlexical representations. Such context may induce a shift-ing of verbal auditory processing to a more suprasegmen-tal one relying on the processing of prosodic cues, whichmay implicate the right auditory associative cortex.

Figure 5b shows two activation modes that can be
computed from Viterbi decoding information associated
with the activation (lag) maps. The subplots have beenobtained from the same active brain region of two sub-jects. Before comparison, each subplot was normalized bythe number of activated voxels in the region. Note that inboth subplots, the first and last neural response rates donot have any physiologic sense because they correspondto the systematic occupancy of the start and final tpestates S1 and SC. The top subplot of Fig 5b indicates aregular level of activation from the beginning to the endof the task of interest. This sustained activity might indi-cate that the involvement of this brain area is essential forthe cognitive processing implicated in the task. The bot-tom subplot of Fig 5b would indicate that the neuronalactivation is fading before the end of the task as if thisarea were no more essential to pursuit the task. Moregenerally, the different activation modes shown by differ-ent brain areas might be the expression of the recruitmentdynamics of different components of the neural networkinvolved in the task.

Finally, Fig 5c highlights another aspect of the analysispossibilities of the HSMESM method. It shows two sub-plots of the mean waveform of the hemodynamic re-sponse obtained for two active brain regions of two sub-jects, and for block 4 of the stimulation paradigm. Beforeplotting, the hemodynamic response patterns related to theactive area and stimulation block 4 were aligned on thetime instant 0, based on their HRO detection time. Also,the OFF-ON pattern of stimulation block 4 was superim-posed on each subplot according to the mean activationlag of the hemodynamic response patterns with respect tothe block. A first remark concerns the relative insensitiv-ity of the HSMESM method to shape variations of thehemodynamic response. In addition, one may notice thesharp, transient, less invariant character of the HRO pat-tern across the two subjects—which was also observedfor the other subjects—thereby justifying to some extentthe use of the HRO pattern in our unsupervised mappingapproach.

DISCUSSION

General IssuesExperiments conducted in epoch-related fMRI illustrate

the relevance of HSMESMs in unsupervised fMRI brainmapping. In particular, the learning and generalizing abili-ties of HSMESMs are of great interest to automaticallycapture neural activation variability across time, brain,
experiments, and subjects. Beyond these models, the de-
33


veloped method demonstrates the possibility to detect andanalyze brain functional activation based solely on thefMRI signal transients that are the HROs. The benefit ofthis strategy is threefold: no prior shape assumption aboutthe hemodynamic response is needed; robustness in acti-vation detection is increased if one reasonably considersthe HRO pattern as the most significant and invariant fea-ture of the hemodynamic response; and the use of signalprocessing techniques effective in detecting and character-izing transients in low SNR ambiances, as encountered infMRI, is made possible.

So far, only a two-condition blocked paradigm and asingle HSMESM-based mapping approach have beenconsidered. When dealing with more sophisticatedblocked paradigms involving more stimulus conditions,a multiple HSMESM-based approach could be used.These HSMESMs could be representative of specificcondition types or of any combination. They could bebuilt from OFF-ON transition subsequences of interestderived from the timing of the original stimulation par-adigm.

Extension to Event-Related fMRI DataThe application of the HSMESM method to event-

related paradigms has also been considered. The modelingprinciple and methodology employed were similar to thatused for blocked paradigms, instead considering this timeall input stimuli as OFF-ON transitions. Preliminary ex-periments conducted on real event–related fMRI datademonstrated some weaknesses of the HSMESM methodin low SNR ambiances. The impaired results obtainedshould be attributed to the inherent unsustained nature ofthe hemodynamic response observed in event-relatedfMRI, thereby making the HRO transients more difficultto detect. To solve this problem, an extension of the HS-MESM mapping approach that takes into account contex-tual spatial information has been developed. It is still for-mulated in terms of temporal alignment between eventsequences. But the event sequences considered are thehidden sequence of stimulus-induced NAOs at voxel v,and the sequences of HROs observed at v and in the spa-tial neighborhood of v. This multiple event sequencealignment problem, which comes under multisensor datafusion, can be solved within the probabilistic frameworkof HMMESMs. HMMESMs can be considered as an ex-tension of the scalar HSMESMs presented so far to thevectorial case of multiple asynchronous event sequences.Although a thorough presentation of HMMESMs falls
beyond the scope of this article, it is worth presenting
34

some illustrative mapping results of these models onevent-related fMRI data. For further technical details onHMMESMs, please refer to a previous article (26).

Stimulation protocol.—The event-related fMRI proto-col included a single patient with no prior history of neu-rologic injury. He gave informed consent before partici-pating in this study approved by the local ethical commit-tee. The subject was asked to perform an oddballdetection task. Visual stimuli were presented to him, con-sisting in series of frequent distractors (letter “O”) andtwo types of oddballs: novels (black and white namablepictures, never repeated), and rare targets (letter “X”).The subject was instructed to signal targets by pressing aresponse key with the right forefinger, and to ignore otherstimuli. Interstimuli intervals were homogeneously distrib-uted from 0.4 to 3.6 seconds (median of 2 seconds) andintervals between two targets ranged between 5.2 and 96seconds. There were 40 targets and 30 novels. Detailedinformation about this experimental design may be foundelsewhere (27,28).

Imaging procedure.—Because electroencephalogramwas acquired with the fMRI, the volumes were separatedby a 1.6-second period, during which gradients were notcommuted. This allowed recording gradient artifact-freeelectroencephalogram data simultaneously with fMRI.Single-shot gradient echo, echo planar imaging was usedfor the functional study, relying on the BOLD effect(Bruker 2T/270 volumes preceded by 5 dummy scans forsteady state of T1 partial saturation effect/TR � 4 sec-onds/imaging time � 2.6 seconds/flip angle of 90°/TE �43 milliseconds/ 4 mm in-plane resolution, field of view� 256 mm, gridding 642, 4-mm slice thickness/24 slicescovering the whole brain except for the cerebellum).

Results.—Results obtained by the scalar HSMESMmethod were not satisfactory with a diffuse pattern beingdetected, either for targets and novels. Inversely, theywere very much satisfactory when using the vectorial ex-tended version of HSMESMs, with all attended regionsbeing detected in three separate sessions of the subject forboth targets and novels (Fig 6). For the target condition:the inferior frontal gyrus bilaterally and the right tem-poroparietal junction (posterior supramarginal gyrus), theright anterobasal frontal region (BA 10), the anterior cin-gulate, the supplementary motor areas, the intraparietalsulcus, and the left central sulcus and thalamus (pulvinarand dorsomedian nuclei). Interestingly, a small activationin the left anterior hippocampus could be detected by thevectorial HMMESM-based approach as published by oth-
ers, but which required the lowering of the threshold


when using the classical analysis using SPM99 on ourdata. The novel condition induces activation in the leftinferior frontal sulcus and the inferior temporal-occipitalregions bilaterally extending to the left posterior parietalareas. False positives were rare and none exceeded thesize of 1 voxel.

More generally, the preliminary results obtained inevent-related fMRI conformed to those attained in epoch-related fMRI. They confirm the relevance of hiddenMarkov modeling techniques in functional MRI brainmapping. Also, they validate the initial strategy employedthat consists of detecting neural activation based solely onthe transient, time-localized events of the fMRI signalthat are the HROs.

Computational IssuesIn conclusion, it should be added that all the experi-

ments described throughout were performed on a 1.5 GHzPentium IV platform. The overall HSMESM-based brainmapping procedure was implemented in C/C��. Thepreprocessing of an input fMRI data set requires about30 s. In average, a total time of 2 min was needed towork out an activation map from an input fMRI data set,spatial registration excluded. This short processing time isto be attributed to the low complexity of an HSMESMimplementation when compared with an HMM or HSMM

Figure 6. Results obtained on real event-related data for anoddball detection task and a noncorrected P value of 0.01 (seetext).

one.

REFERENCES

1. Bandettini PA, Jesmanowicz A, Wong EC, et al. Processing strategiesfor time-course data sets in functional MRI of the human brain. MagnReson Med 1993; 30:161–173.

2. Friston KJ, Jezzard P, Turner R. Analysis of functional MRI time-series.Human Brain Mapping 1994; 1:153–171.

3. Dale AM, Buckner RL. Selective averaging of rapidly presented individ-ual trials using fMRI. Human Brain Mapping 1997; 5:329–340.

4. Rosen BR, Buckner RL, Dale AM. Event-related functional MRI: past,present, and future. Proc Natl Acad Sci U S A 1998; 95:773–780.

5. Lange N, Zeger SL. Non-linear Fourier time series analysis for humanbrain mapping by functional magnetic resonance imaging. Appl Stat JRoy Stat Soc Series C 1997; 46:1–30.

6. Rajapakse JC, Kruggel F, Maisog JM, et al. Modeling hemodynamicresponse for analysis of functional MRI time-series. Human Brain Map-ping 1998; 6:283–300.

7. Frackowiak RSJ, Friston KJ, Frith C, et al. Human brain function. SanDiego: Acedemic Press, 1997. Available from http://www.fil.ion.ucl.ac.uk/spm/doc/books/hbf1/

8. Huettel SA, Singerman JD, McCarthy G. The effects of aging upon thehemodynamic response measured by functional MRI. NeuroImage2001; 13:161–175.

9. D’Esposito M, Zarahn E, Aguirre GK, et al. The effect of normal agingon the coupling of neural activity to the bold hemodynamic response.NeuroImage 1999; 10:6–14.

10. Richter W, Richter M. The shape of the fMRI BOLD response in chil-dren and adults changes systematically with age. NeuroImage 2003;20:1122–1131.

11. Thoraval L, Armspach JP, Namer IJ. Analysis of brain functional MRtime series based on the continuous wavelet transform and stimula-tion-response coupling distance. In: Niessen W, Viergever M, eds.Proc. Medical Image Computing and Computer Assisted Intervention(MICCAI). Utrecht, The Netherlands: 2001, Berlin, Heidelberg: Springer-Verlag, LNCS 2208; 881–888.

12. Faisan S, Thoraval L, Armspach JP, et al. Unsupervised learning and map-ping of brain fMRI signals based on hidden semi-Markov event sequencemodels. In: Ellis RE, Peters TM, eds. Proc. Medical Image Computing andComputer Assisted Intervention (MICCAI). Montreal, Canada: 2003, Berlin,Heidelberg: Springer-Verlag, LNCS 2879; 2:75–82.

13. Thoraval L. Hidden semi-Markov event sequence models. Technicalreport, LSIIT, ENSPS, Université Louis Pasteur Strasbourg I, 2002.Available from http://picabia.u-strasbg.fr/lsiit/perso/thoraval.htm. Ac-cessed November 5, 2004.

14. Rabiner LR. A tutorial on hidden Markov models and selected applica-tions in speech recognition. Proc IEEE 1989; 77:257–286.

15. Thoraval L, Carrault G, Bellanger JJ. Heart signal recognition by hiddenMarkov models: the ECG case. Meth Inform Med 1994; 33:10–14.

16. Nikou C, Heitz F, Armspach JP, et al. Registration of MR/MR and MR/SPECT brain images by fast stochastic optimization of robust voxelsimilarity measures. NeuroImage 1998; 8:30–43.

17. Pallier C, Dupoux E, Jeannin X. EXPE: an expandable programminglanguage for on-line psychological experiments. Behav Res Meth In-struments Computers 1997; 29:322–327.

18. Content A, Mousty P, Radeau M. Une base de données lexicales infor-matisée pour le francais écrit et parlé. Ann Psychol 1997; 90:551–566.

19. Metz-Lutz MN, Namer IJ, Gounot D, et al. Language functional neuro-imaging changes following focal left thalamic infarction. Neuroreport2000; 11:2907–2912.

20. Ferguson JD. Variable duration models for speech. In: Ferguson JD,ed. Proc. Symposium on the Application of Hidden Markov Models toText and Speech, Princeton, NJ, 1980; 143–179.

21. Russell M, Moore R. Explicit modeling of state occupancy in hiddenMarkov models for automatic speech recognition. Proc. IEEE Interna-tional Conference on Acoustics, Speech and Signal Processing(ICASSP). Tampa, FL, 1985; 5–8.

22. Levinson SE. Continuously variable duration hidden Markov models for
speech analysis. Computer Speech Lang 1986; 1:29–45.
35

http://picabia.u-strasbg.fr/lsiit/perso/thoraval.htm


23. Forney GD. The Viterbi algorithm. Proc IEEE 1973; 268–278.24. Faisan S, Thoraval L, Armspach JP, et al. Hidden semi-Markov event

sequence models: application to brain functional MRI sequence analy-sis. In: Proc. IEEE Int. Conf. Image Processing (ICIP). Rochester, NY,2002. Piscatoway, NJ: IEEE.

25. Senhadji L, Thoraval L, Carrault G. On the use of continuous wavelettransform magnitude and phase for ECG signal analysis. In: Aldroubi A,
Unser MA, eds. Wavelets in medicine and biology. Boca Raton, Fla:CRC Press: 1996; 439–463.
36

26. Thoraval L. Hidden Markov multiple event sequence models. Technicalreport, LSIIT, ENSPS, Université Louis Pasteur Strasbourg I, 2004.Available from http://picabia.u-strasbg.fr/lsiit/perso/thoraval.htm. Ac-cessed November 5, 2004.

27. Foucher JR, Otzenberger H, Gounot D. The BOLD response and thegamma oscillations respond differently than evoked potentials: An in-terleaved EEG-fMRI study. BMC Neurosci 2003; 4:22.

28. Foucher JR, Otzenberger H, Gounot D. Where arousal meets attention: A si-multaneous fMRI and EEG recording study. Neuroimage 2004; 22:688–697.

The Association of University Radiologists wishes to thank the followingcompanies for their generous support of the AUR Junior MembershipProgram:

4th YearRadiologyResidents

SiemensMedical Systems, Inc.

3rd YearRadiologyResidents

BerlexLaboratories

http://picabia.u-strasbg.fr/lsiit/perso/thoraval.htm

Documents

Hidden Markov event sequence models: Toward unsupervised functional MRI brain mapping1