Adaptive Emotional Information Retrieval From EEG Signals in TFD

2604 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 60, NO. 5, MAY 2012

Adaptive Emotional Information Retrieval From EEGSignals in the Time-Frequency Domain

Panagiotis C. Petrantonakis, Student Member, IEEE, and Leontios J. Hadjileontiadis, Senior Member, IEEE

Abstract—This paper aims at developing adaptive methods forelectroencephalogram (EEG) signal segmentation in the time-fre-quency domain, in order to effectively retrieve the emotion-relatedinformation within the EEG recordings. Using the multidimen-sional directed information analysis supported by the frontal brainasymmetry in the case of emotional reaction, a criterion, namelyasymmetry index , is used to realize the proposed segmenta-tion processes that take into account both the time and frequency(in the empirical mode decomposition domain) emotionally relatedEEG components. The efficiency of the -based “emotional” fil-ters was justified through an extensive classification process, usinghigher-order crossings and cross-correlation as feature-vector ex-traction techniques and a support vector machine classifier for sixdifferent classification scenarios in the valence/arousal space. Thisresulted inmean classification rates from 64.17%up to 82.91% in auser-independent base, revealing the potential of establishing sucha filtering for reliable EEG-based emotion recognition systems.

Index Terms—Electroencephalogram (EEG), emotion recog-nition (ER), empirical mode decomposition, frontal brain asym-metry, multidimensional directed information.

I. INTRODUCTION

E LECTROENCEPHALOGRAM (EEG)-based emotionrecognition (EEG-ER) systems are gaining considerable

attention, since they provide a convenient and nonintrusiveway of capturing signals related to emotional expression in thebrain, with efficient time resolution. Moreover, many studies[1]–[7] have revealed the potential of such systems to differen-tiate discrete emotions and affective states paving the way ofnew approaches to the so-called Affective Computing area [8].The latter deals with the design of systems and devices that candetect, recognize and process human emotion.A typical approach of an EEG-ER system realization consists

of three major steps: a) the emotion elicitation step, i.e., specif-ically designed experiments where emotions are artificiallyevoked to subjects by pictures [4], videos [2], and/or sounds[3] with predefined emotional reference; b) the captured datapreprocessing step, where the recorded signals are subjected tofrequency band selection, noise cancelling, and artifact removalprocedures; and c) the classification step, where the featureextraction techniques and robust classification methods areutilized to classify the recorded signals in different groups that

Manuscript received October 19, 2011; revised January 17, 2012; acceptedFebruary 02, 2012. Date of publication February 13, 2012; date of current ver-sion April 13, 2012. The associate editor coordinating the review of this manu-script and approving it for publication was Dr. Z. Jane Wang.The authors are with the Department of Electrical and Computer Engi-

neering, Aristotle University of Thessaloniki, GR-54124 Greece (e-mail:[email protected]; [email protected]).Digital Object Identifier 10.1109/TSP.2012.2187647

refer to different affective states evoked during the elicitationstep.Besides the threefold aforementioned realization, the emotion

elicitation step is the most crucial step for an efficient EEG-ERsystem. For instance, if the media used as stimulation do noteffectively evoke the appropriate subjects’ emotional reactionand arousal (different subjects may be emotionally affected bydifferent stimulation due to personality and/or personal expe-riences), the respective EEG recordings would not incorporatethe corresponding emotional information, resulting in an incom-petent emotion classification process. Based on that, and takinginto account the difficulty of evoking emotional states during anartificially designed affective situation in a lab, it is of major im-portance to discard the useless nonemotion related informationfrom the captured EEG dataset.The first attempt towards such direction was made in [9],

where a novel index, namely asymmetry index (AsI), utilizingmultidimensional directed information (MDI) [10], was intro-duced in the trial-based domain. There, it was proved that EEGsignals, which refer to distinct emotion elicitation trials, withrelatively big (above a specified threshold) AsI values, weremore effectively discriminated during the classification stepcompared to the opposite case.Besides the trial-based dependence of the emotion elicitation

efficiency in a relative experiment, the exposure of a subjectto an emotional stimulation with specific duration, e.g., a pic-ture projection, would evoke a relative, time-varying emotionalreaction. As a result, the extraction of features for classifica-tion from the signals that refer to the whole duration of the pic-ture projection would lead to problematic extraction of the emo-tion-related information during the feature extraction processand, thus, to a poor classification performance. Furthermore, thefrequency-based characteristics of the emotion related EEG ac-tivity bring out another mean of emotion related informationretrieval that would lead to even better isolation of the valuableinformation from EEGs.In this paper, in order to meet the time-varying appearance of

the emotional reaction of each subject and the frequency-baseddependency of emotional expression in EEG recordings, EEGsignals are subjected to AsI-based algorithms to be “emotion-ally” purified, i.e., to exclude as much as possible the nonemo-tion related information, in time-frequency domain. Regardingthe time-based perspective, AsI index was applied in a slidingtime-window oriented approach. In this way, each EEG trialwas segmented in targeted durations in the time domain, leadingto a more effective representation of emotion-related informa-tion in the features extracted from the segmented signal, in con-trast with the initial one. Moreover, considering the frequency-

1053-587X/$31.00 © 2012 IEEE

PETRANTONAKIS AND HADJILEONTIADIS: ADAPTIVE EMOTIONAL INFORMATION RETRIEVAL 2605

based characteristics of the emotional expression in the brain(see Section II-A), this work also aims at expanding the abovetime-based approach of EEG database segmentation to a fre-quency- and time-frequency-based one. In order to accomplishthis, empirical mode decomposition (EMD) analysis [11] is uti-lized, as a means of the time-frequency representation of theEEG signal. In the EMD domain, the local characteristic timescale of the EEG signal is used for decomposing it into a numberof intrinsic mode functions (IMFs), representing a generallysimple oscillatory mode. Having this, the index is em-ployed either globally (in the whole IMF) or locally (in a slidingwindow), leading to the aforementioned frequency- and time-frequency-based segmentation of the signal, respectively. Eachtime, a reconstruction process takes place to form the final trial-referring EEG signal, from which the feature vector for classi-fication is to be extracted. All above described approaches aimat retrieving the emotional information from EEG signals in anadaptive way, which is demonstrated from both the fundamentalalgorithmic tools used (e.g., EMD) and the fact that each emo-tion elicitation trial is faced separately and new, filtered signalsare constructed.For the evaluation of the previously described methodologies

for time-frequency-based segmentation, a thorough classifica-tion procedure was followed, incorporating two different featureextraction methods, namely higher order crossings (HOC) [4]and cross-correlation (CC) analysis [12] and six different clas-sification scenarios, in accordance with the classification setupimplemented in [9], for comparison reasons. The EEG databaseused (same as in [9]) comprised of EEG signals from 16 subjectscaptured during a specifically designed experiment to evoke cer-tain affective states. The results provided by the aforementionedclassification setup reveals the significance of such emotionalinformation retrieval that leads to a more reliable and pragmaticEEG-ER system.The rest of the paper is structured as follows. Section II

provides with some background material in regard with theemotional expression in the brain and fundamental methodolog-ical tools used in the proposed scheme. Section III explicitlydescribes the proposed approaches, whereas Section IV presentsthe feature extraction and classification approaches adoptedin this paper. Section V describes the experiments conductedfor the EEG database selection and some implementationissues, along with the description of the classification setup.Sections VI and VII present the results and provide somediscussion on the overall evaluation of the proposed method-ologies, respectively. Finally, Section VIII concludes the paper.

II. BACKGROUND

A. Emotions and Frontal Brain Asymmetry

In psychology, emotions are usually analyzed in a 2D space,i.e., the valence/arousal space (VAS), instead of being character-ized as distinct emotional states, such as happiness, fear, sadness[13]. In VAS, valence stands for one’s judgment about a situa-tion as positive or negative (including the whole range betweenthese two extreme cases) and arousal spans from calmness toexcitement, expressing the degree of one’s excitation.

The most prominent expression of emotions in brain signalswas described by Davidson et al. [14], who developed a modelthat relates the asymmetry between the left and right frontal andprefrontal lobes (expressed in alpha frequency band, i.e., 8–12Hz) of the brain with emotions, with the latter be analyzed inthe VAS. According to that model, emotions are either orga-nized around approach-withdrawal tendencies or differentiallylateralized in the frontal brain region. The left frontal area is in-volved in the experience of positive emotions, whereas the rightfrontal region is involved in the experience of negative emo-tions. Other studies [15]–[17] have also confirmed the afore-mentioned asymmetry concept and have examined frequencybands other than alpha, including theta (4–7 Hz), beta (13–30Hz), and gamma (30–100 Hz), where asymmetrical effects werealso found. Finally, Bos [3] examined the efficacy of alpha, betaand the combination of these bands to discriminate emotionswithin the VAS and concluded that both bands include importantinformation for the aforementioned discrimination. Despite thefact that there is still an ongoing discussion on the Davidson’smodel, a vast amount of neuroscience bibliography has con-tributed to the validity of that model; thus, it is adopted in thisstudy (see Section V-A).

B. Multidimensional Directed Information (MDI)

For the aforementioned asymmetry, a robust mathematicaltool is needed to express the information shared between the twobrain sides and finally define a measure for its quantification.This shared information is frequently defined as correlationsamong multiple EEG recording channels (multiple time series)simultaneously observed from a subject. If a relation of tem-poral ordering is noted, as the correlation relation among thesetime series (EEG channels), some are interpreted as causesand others as results, suggesting a cause-effect relation amongthe time series (causality analysis). When causality in such asense is noted in multiple time series, the relation is definedas directed information [18]. There are methods developed toperform causality analysis, such as directed-coherence analysis[19], directed-information analysis [18], MDI analysis [10],Kaminski’s method (DTF) [20], partial directed coherence [21],and Granger causality [22]. MDI analysis was employed as ameans to identify the causality between any two series (EEGchannels) considering all acquired series. The main advantageof MDI, compared to the other aforementioned methods, is thatthe amount of information propagation is presented as an abso-lute value in bits and not as a relative value, i.e., correlation; abrief description of the MDI [10] follows.Consider the simple case of three stationary time series ,, and of length , divided into epochs of length ;

each epoch of length is written as a sequenceof two sections of and lengths before and after the , ,and sampled values of time series , , and at time ,respectively, i.e.

(1)

(2)

(3)


Fig. 1. An illustration of what the , , and sets represent [see (1)]and how they are created from the EEG time series .

where ; ;; ; ;

. An example of the formulation of (1),for the case of an EEG signal, is shown in Fig. 1.According to the MDI analysis [10], information that is first

generated in at time and propagated with a time delay ofto taking into consideration information that is propagated toboth of them from , can be calculated from [10]

(4)

where is the covariance matrix of the stochasticvariables . Using (4), the total amount of information,namely , that is first generated in and propagated totaking into account the existence of , across the time delayrange is

(5)

It must be stressed out that if time series and contain acommon component from , i.e., there is information flow fromto both and but not between and , in conventional

directed information analysis, i.e., is excluded from (4), aninformation flowwould wrongly be identified, as if there exists aflow between and . To circumvent this ambiguity, the MDImethod is employed.In the subsequent paragraph, (5) will be used to consolidate

the measure by estimating the mutual information sharedbetween the left and right brain hemisphere, exploiting, thatway, the frontal brain asymmetry concept.

C. The Asymmetry Index (ASI)

As described in Section II-A, the experience of negative emo-tions is related with an increased right frontal and prefrontalhemisphere activity, whereas positive emotions evoke an en-hanced left-hemisphere activity. The measure is based onthat asymmetry concept and its definition is briefly describedlater.Assume that EEG channel 1 recorded from the left hemi-

sphere, EEG channel 2 from the other hemisphere, and EEGchannel 3 recorded from both hemispheres as a dipole channelrepresent the signals , , and of MDI analysis, respec-tively. In order to evaluate the asymmetry-related informationbetween signals and , taking into account the informationpropagated to both of them by the signal , the MDI approachwas applied by estimating the total amount of informationshared between the left and right hemisphere (signals and, respectively) with (5). In accordance with the frontal brain

asymmetry concept, would become maximum when the sub-ject is calm (information symmetry), whereas it would becomeminimum when the subject is emotionally aroused (informationasymmetry), corresponding to the and values, respec-tively. Consequently, the latter could be formed as

(6)

(7)

where , , , and are estimated by (5).Using (6) and (7), is defined as the distance of the

point, corresponding to a specific emotion elicitationcause (e.g., a picture), from the line , i.e.

(8)

In this paper, the concept is further expanded to be ap-plicable in the time-frequency domain, with the frequency com-ponent expressed through the EMD approach [11], in order toaccomplish an adaptive EEG time-, frequency-, and time/fre-quency-based segmentation.

D. EMD

The EMD method considers the signals at their local os-cillation scale, subtracts the faster oscillation, and iterates theresidual. The produced IMFs satisfy two important conditions,i.e.: i) the number of extrema and the number of zero crossingsdiffer by at most one and ii) the local mean is zero. In particular,the EMD procedure for a given signal is realized throughthe sifting process, summarized as follows [11]:1. Identification of the successive extrema of .2. Extraction of the upper and lower envelopes by interpola-tion.

3. Computation of the average of upper and lower envelopes.

4. Calculation of the first component:.

5. Treatment of as a new set of data, and repetitionof steps 1–4 up to times until becomes a trueIMF. Then set . Overall, should


contain the finest scale or the shortest period componentof the signal.

6. Obtainment of the residue .7. Treatment of as a new set of data and repetition ofsteps 1–6 up to times until the residue becomes aconstant, a monotonic function, or a function with only onecycle from which no more IMFs can be extracted.

8. Finally, where is the thIMF and the final residue.

The above procedure results in a decomposition of the datainto -IMFs and a residue .

III. THE PROPOSED APPROACH

The proposed emotion information retrieval scheme is real-ized through three segmentation types. In particular, the firstone, namely wAsI incorporates the application of the AsI indexestimation within a sliding across time window of the signalwith certain length. The second one, namely EMD-AsI, involvesthe application of AsI in the EMD domain, as a means for se-lecting the most appropriate IMFs that carry the emotional in-formation. The last one, i.e., the EMD-wAsI approach, focusesat each IMF in a local manner, extracting its segments that con-tribute the most to the expression of the emotional information.It is actually the implementation of the wAsI algorithm in theIMFs of a signal. The aforementioned segmentation types areelaborately described later.

A. Time-Based Segmentation (wAsI)

Fig. 2 provides a schematic representation of the windowedapproach. The case study depicted refers to a

single emotion elicitation trial. The -sample signals fromthree EEG channels, i.e., , , and , that are simultaneouslyrecorded during the emotion elicitation trial, are used for theimplementation of the algorithm. First, a lengthwindow slides in a parallel way through the three EEG signalseither for the signals that refer to the phase where the subjectis relaxed or when the subject is emotionally aroused. Thewindow moves across the signals with 1-sample step, resultingin a total number of windows . Foreach one of the triplet , , and , i.e., the parts of thesignals , , and that are within the window borders,where , the MDI analysis is applied for the relaxand the emotionally triggered (e.g., picture projection) phase.Subsequently, the respective and values are extractedand, according to the estimation formula of AsI, the new wAsIversion is defined, i.e.

(9)

For each window, a , value is ex-tracted and is assigned into the middle of the window. In orderto extract the segment of the signal that is more likely to ex-press emotional EEG activity, the EEG segments that corre-spond to those values that exhibited a concentrated mul-titude of peaks higher than 0.5 ( values are normalized inthe range ) were selected, by multiplying the initial signalwith the unit-amplitude pulse (see Fig. 2-gray pulse), namely

. The selected segments constitute a new signal,

Fig. 2. The schematic representation of the wAsI approach.

(in Fig. 2 the segmented signal corresponds to the initialsignal of the projection phase), that is supposed to correspondto an “emotionally” filtered signal. It must be stressed out thatall signals, , , and either from the relax or the projectionphase of a single emotion elicitation trial are segmented with theuse of the (in Fig. 2 only the segmentation ofthe signal of the projection phase is depicted as an example).Afterwards, the feature extraction and the classification stagestake place and the segmented signals are further investigatedregarding their ability to better discriminate emotion in relationwith the initial -sample signals.

B. Frequency-Based Segmentation (EMD-AsI)

A schematic diagram of the proposed EMD-AsI segmenta-tion approach is depicted in Fig. 3. According to Fig. 3, the threeEEG signals (channels) , , and , are initially subjected tothe EMD process and the IMFs are extracted for each one ofthem. Afterwards, each triplet of IMFs, i.e., , and ,and , passes through the MDI analysis, andthe , values are estimated. Subsequently, athresholding procedure takes place and removes the IMFs thatcorrespond to normalized (range ) values lower than


Fig. 3. The schematic representation of the EMD-AsI approach.

Fig. 4. The block-diagram of the realization of the EMD-wAsI approach.

a threshold of 0.5. Finally, by summing up the IMFs that remainafter the thresholding process, three new , , and signalsare reconstructed and used as inputs to the subsequent featureextraction methods (see Section IV).

C. Time- and Frequency-Based Segmentation (EMD-wAsI)

The schematic representation of the EMD-wAsI segmenta-tion approach is depicted in Fig. 4. As in EMD-AsI approach,

the IMFs of the three EEG signals , , and are obtainedand each one of the triplets , , and

, are subjected to the wAsI algorithm which is schemat-ically presented in Fig. 2. After the wAsI algorithm is appliedand all segmented IMFs are obtained, the EMD-wAsI approachis concluded through a reconstruction phase that takes place per

channel, e.g., , and the reconstructed , ,and signals either for the projection or the relax phase are thensubjected to the feature extraction and classification processes,described in the subsequent section.

IV. FEATURE EXTRACTION AND CLASSIFICATION METHODS

A. Feature Vector Construction

Similarly to [9], the feature vector set was constructed basedon twomethods, i.e., HOC [4] and CC [12]. The HOC techniquewas proved beneficial for feature vector extraction for the caseof emotion recognition, as shown in [4]. A brief description ofthe calculation of HOC sequences is presented in the Appendix.The HOC used to construct the feature vector , wasformed as follows:

(10)

where denotes the maximum order of the estimated HOC,is the HOC order up to which the corresponding numbers ofzero crossings were used to form the , and

is the number of zero crossings in respective order (seeAppendix). Due to the three-channel EEG recording setup (seeSection V-A) the final , where stands for “combined,”was structured as

(11)

where , , denote the channel number that participates to thecombination.The CC method was chosen here as a baseline approach.

The CC method introduced in [12] estimates the CC coefficientbetween potentials of the EEG electrodes and for

the frequency band , i.e.

(12)

where is the Fourier transform of EEG at the elec-trode site and frequency bin, and the summation is over thefrequency bins in the frequency band. In this paper, the CC co-efficient was estimated for both alpha and beta bands resultingin a six-feature vector, denoted as , three features for eachband as a three-channel EEG recording set is used.

B. Classification Techniques

As there is no single best classification algorithm, that is one-size-fits-all, the choice of the most efficient classifier is stronglydependent on the examined problem and the relevant dataset tobe classified [23]. After having tested some classifiers such as


quadratic discriminant analysis (QDA) [24], Mahalanobis dis-tance (MD) [25], -nearest neighbor ( -NN) [26], and supportvector machine (SVM) [27], the latter was chosen, as it outper-formed the others with higher recognition accuracy.In the SVM classifier, a polynomial function was used, as

a kernel function, to project the data into higher dimensionalfeature space, i.e.

(13)

where is the “support vector” and is the query fea-ture vector. The SVM kernel (13) was chosen after a thoroughexperimentation with different kernels and various parameters,resulting in the most robust results. Among several available ap-proaches to realize a multiclass SVM classification process theone-versus-all method was adopted here.

V. EXPERIMENTS

A. Datasets

In this paper, the projection of pictures with predefined emo-tional content was used as a means of emotional stimulation.The pictures used were drawn from the International AffectivePicture System (IAPS) [28] database, a widely used dataset,which contains pictures that come with their individual normvalues of valence and arousal in the range from 1 to 9 for eachmetric. The selected pictures (totally 40 pictures) were chosento refer to arousal and valence higher (H) from 6 and lower(L) from 4 in the above described range with standard devia-tion lower than 2.2. Thus, 10 pictures were chosen for each oneof the cases low arousal low valence (LALV), low arousal highvalence (LAHV), high arousal high valence (HAHV), and higharousal low valence (HALV). It should be stressed out that va-lence is usually associated with the terms “negative” or “posi-tive.” Due to the range of the IAPS norms, i.e., 1 to 9, for valenceand arousal, and for the sake of simplicity, the terms “low” and“high” are alternatively used in this paper instead of “negative”or “positive,” respectively.For the construction of the EEG data, a specifically designed

experiment was conducted through the abovementioned elicita-tion process. In this experiment, 16 (9 males and 7 females in theage group of 19–32 yr), healthy, right-handed volunteers partic-ipated. The experimental protocol included a series of discretesteps. In particular, sequentially for each one of the 40 pictures,the following procedure took place:i) projection of a 5 s black screen;ii) projection of a 5 s period in which countdown frames

were demonstrated;iii) a 1 s projection of a cross shape in the middle of the screen

to attract the sight of the subject; andiv) projection of the corresponding picture for 5 s.The 5 s countdown phase was employed to accomplish a re-

laxation phase and emotion-reset [3] before the projection of thenew picture, due to its naught emotional content. During the ex-periment, the selected pictures were projected in sequence: 10for LALV, 10 for LAHV, 10 for HAHV, and 10 for HALV. Thus,more intense emotions were chosen to be projected at the end of

Fig. 5. The Fp1, Fp2, F3, and F4 EEG sites according to the 10/20 system.

the whole process. This was adopted as a means to avoid ordereffects as, in a random picture projection, a possible case of in-tense-mild sequential emotional content would evoke an emo-tionally “cover up” effect, that is, an intensive emotion woulddominate upon a milder one.In our previous work [9], the self-assessment of the IAPS pic-

tures by the subjects participated in the experiment was also pre-sented. There, it was shown that the categorization of the EEGsignals using the self-assessment ratings did not provide betterresults than the categorization based on the IAPS norms. In astep further, it was observed that, although, the self-assessmentof the valence dimension exhibited high consistency with theIAPS norms , the one that refers to the arousal dimen-sion appeared to match with the IAPS norms approximately in50% of the cases. The latter reveals an almost random behaviorin the evaluation of the emotional arousal by the subjects. Inorder to avoid any propagation of such randomness to the sub-sequent analysis and taking into account the findings in [9], theIAPS norms was preferred for the signals categorization in thiswork.The EEG signals from each subject were recorded during the

whole experiment. The EEG signals were acquired from Fp1,Fp2, F3, and F4 positions, according to the 10–20 system [29],related to the emotion expression in the brain, based on theasymmetry concept. The Fp1 and Fp2 positions were recordedas monopole channels, whereas the F3 and F4 positions as adipole, resulting in a 3-channel EEG set (see Fig. 5). Thus, theshared information between Fp1 (time series in theMDI anal-ysis) and Fp2 (time series in theMDI analysis) channels couldbe measured, as effectively as possible, by taking into accountthe respective information propagated by the dipole F3/F4 (timeseries in the MDI analysis) to both of them. The ground andreference electrodes were placed at the right (A2) and left (A1)earlobes, respectively (see Fig. 5).EEG recordings were conducted using the g.MOBIlab (g.tec

medical and electrical engineering, Guger Technologies, www.


gtec.at) with sampling frequency of 256 Hz. After the acqui-sition part, the EEG signals were subjected to a bandpass 10thorder Butterworth filtering, to retain only the frequencies withinthe alpha (8–12 Hz) and beta (13–30 Hz) bands, as these are thefrequency bands that convey the emotion related informationexpressed by the frontal brain asymmetry concept [3], [15]. Al-though there are sophisticated approaches to remove superim-posed artifacts from various sources (e.g., Independent Com-ponent Analysis), it has been reported [30]–[32] that, in manycases, these artifacts are effectively eliminated by appropriatebandpass filtering. Specifically, the influence of eye movement/blinking is most dominant below 4 Hz, heart-functioning causesartifacts around 1.2 Hz, whereas muscle artifacts affect the EEGspectrum above 30 Hz. Nonphysiological artifacts caused bypower lines are clearly above 30 Hz (50–60 Hz). Consequently,by extracting only the alpha and beta frequency bands from theacquired EEG recordings, the most part of the noise influenceis circumvented. This is further supported by the fact that theexperimental procedure took place using a clear experimentalprotocol, where the subjects minimized their movements andeye-blinking during the image projection. Afterward, the EEGsignals were cut into 5 s segments corresponding to the durationof each picture projection. Finally, the EEG signals referring tothe countdown phase were also cut and filtered as they werealso used for the analysis described in the previous section. Itmust be stressed out that the and values [see (7) and (6)]were calculated for the signals that correspond to the projectionof the IAPS picture and the countdown phase that took placeimmediately before the projection of the corresponding picture,respectively.

B. Implementation Issues

After experimentation, the window length was setand which

resulted in a efficient number of epochs for MDI im-plementation. Moreover, the interval between two succeeding

peaks (see Fig. 2) that belong to the same ,should in maximum be samples. Furthermore, the borders ofeach wAsI-Mask were set to samples left from the farmost left wAsI peak of the concentrated multitude of peaks thatwas observed and samples right from the far most right one,due to the assignment of the wAsI value at the center of thewindow . Regarding the HOC-based feature vector wasset to . The SVM kernel function parameter in (13)was set as after a thorough experimentation. Finally, thenumber of IMFs was found to be .

C. Classification Setup

The same classification setup used in [9] was also employedhere to facilitate the direct comparison between the corre-sponding approaches. In particular, six different three-classesclassification scenarios were employed: i) S1: class1: LA,class2:HA and class3: the respective Relax signals (countdownphase); ii) S2: class1: LV, class2:HV and class3: the respec-tive Relax signals; iii) S3: class1: LALV, class2:HALV andclass3: the respective Relax signals; iv) S4: class1: LAHV,class2:HAHV and class3: the respective Relax signals; v) S5:

class1: LALV, class2:LAHV and class3: the respective Relaxsignals; vi) S6: class1: HALV, class2:HAHV and class3: therespective Relax signals. These classification scenarios werechosen in order to: a) emphasize on the discrimination of theadverse expressions of the valence and arousal coordinatesin the VAS and the relax state and b) to create different clas-sification setups for testing the consistency of the examinedapproaches to perform efficiently in every one of them.It should be pointed out that some classifications scenarios,

such as S1, demonstrate the discrimination of states across thearousal dimension of the VAS despite the fact that AsI is basedon the Davidson’s asymmetry theory, which refers only to thevalence dimension. First, it is clear that the classification proce-dure exclusively depends on the feature vector extracted by theHOC or CC algorithms and not on the AsI value estimated foreach one of the different emotion elicitation trials. AsI is usedonly for the indication of an effective emotion elicitation trialvia the detection of a frontal brain asymmetry, which demon-strates a probable deviation from the neutral affective state. Thisdeviation may be towards a positive or negative affective stateand taking this into account, it is not questionable that a respec-tive arousal component of that affective state would be broughtout, as a direct consequence from the engagement of defensiveor appetitive motivational systems [28], [33]. Thus, by assuringthat a deviation from neutral state occurs, a related low, mod-erate or high emotional excitation (arousal) will be present. Thisremark is also the reason for using pictures from the IAPS data-base with extreme low or high values of valence or arousal innine-grade scale, i.e., to foster a probable deviation from theneutral state. As soon as this deviation is tracked by the AsI, thecorresponding EEG signals are subjected to the classificationprocedure, where sates with different valence and/or arousal canbe recognized by analyzing the alpha and beta bands of EEGsignals, acquired from the frontal and prefrontal lobe [1], [3].According to the AsI concept [9], emotion elicitation trials,

i.e., corresponding pairs, with big AsI values have beenshown to elicit more effectively the respective emotional state,whereas others with smaller AsI values exhibit the opposite re-sult. According to this observation, trials that correspond to AsIlarger than 0.5 gather the Big AsI group [9]. Particularly, theBig AsI group consisted of 31, 29, 21, and 15 signals (gatheredfrom approximately all subjects) for the LALV, LAHV, HAHV,andHALV cases, respectively. In accordance with the [9], in thispaper, the classification process was conducted both for the sig-nals that belong to the Big AsI group and all signals that belongto emotion dataset and found to provide with a new signal aftera specified segmentation (see Section III).For each one of the classification scenarios, the 70% of the

signals were used as a training set, whereas the remaining 30%as a testing set. The leave- -out cross-validation technique wasadopted for a better evaluation of the classification performanceresulting in a 20-iteration classification process. Thus, for eachone of the 20 iterations 70% of the dataset was randomly ex-tracted and used for training and the remaining 30% for testing.The mean classification rate from the 20 iterations was finallyextracted.All above processes were conducted for both the HOC- and

CC-based feature vectors.


Fig. 6. values of the AsI, wAsI, EMD-AsI, EMD-wAsI, and EMD-wAsI BigIMFs approaches for HOC- and CC-based feature vector extraction methods and(a) for the Big AsI group; (b) all signals or signals with wAsI-Masks.

VI. RESULTS

A. User-Independent Case

The results of the aforementioned classification setup in auser-independent framework, that is, signals were classified re-gardless of the subject from whom they were recorded, are pre-sented in Fig. 6(a) and (b) for the Big group and the wholedataset, respectively. In Fig. 6(a), the solid black line corre-sponds to the initial Big AsI group, i.e., the initial EEG signalswithout any preprocessing except for the filtering procedure de-scribed in Section V-A. The dashed black line represents thevalues of the signals of the Big AsI group after the wAsI al-gorithm was applied and, thus, a time-based segmentation ofthem was accomplished. The black dotted and dash-dotted linescorrespond to the Big AsI group signals after the EMD-AsIand EMD-wAsI algorithms were applied to them, respectively.Finally, the gray solid line corresponds to the of the sig-nals of the Big AsI group after a combination of the algorithmsEMD-AsI and EMD-wAsI was applied. Particularly, the EMD-wAsI algorithm was applied to IMFs that was assigned from theEMD-wAsI algorithm to have AsI values above the specifiedthreshold (denoted as ). All the abovedescribed lines are marked with a circle (o) for HOC-based fea-ture vector and with a triangle for the CC-based one. Froma simple visual inspection of the Fig. 6(a) it is obvious that theEMD-wAsI and the approaches sur-pass all the others for almost all classification scenarios (seealso Table I). On the other hand, the CC feature vector pro-vides with significantly poorer classification results, but still ex-hibits improvement when the initial signal is subjected in time-or time/frequency-based segmentation (see Table I). To statis-tically justify this difference and accommodate for the limitednumber of cases involved, the Wilcoxon signed ranks non-para-metric test was used. This analysis revealed a strong statistically

TABLE IMEAN CLASSIFICATION RATES FOR BIG ASI GROUP

significant difference between the classification performance ofHOC and CC feature vectors, resulting in .For the whole EEG dataset, the corresponding classification

results are depicted in Fig. 6(b). It must be stressed out that forwAsI, EMD-AsI, and EMD-wAsI, not all of the intial signalsof the dataset were used, i.e., 160 signals per affective state(LALV, LAHV, HAHV, HALV). This is due to the nonexistenceof at least one for a signal in the wAsI approachor for all IMFs of a signal in the EMD-wAsI approach andfinally the nonexistence of IMF of AsI value higher than thepre-specified threshold for a certain signal in the EMD-AsIcase. Thus, for the {LALV, LAHV, HAHV, HALV} affectivecases, {85, 95, 93, 93}, {148, 151, 152, 153}, and {131,


TABLE IIMEAN CLASSIFICATION RATES FOR THE WHOLE EEG DATASET

135, 134, 134} signals were used for the wAsI, EMD-AsI,and EMD-wAsI, respectively. In Fig. 6(b), the annotation isexactly the same as in Fig. 6(a). Again, the EMD-wAsI and

approaches with the HOC-based featurevector surpass all the others (see Table II). All the remainingapproaches exhibit very lower classification results for all clas-sification scenarios and for both feature extraction techniques.Tables I and II provide a complete presentation of values forthe Big AsI group and whole EEG dataset, respectively. FromTables I and II, the ability of the proposed approaches at eachcategory level, i.e., classification scenarios S1 (solely arousaldomain) and S2 (solely valence domain), can be deduced. Inparticular, although the AsI is mainly related to the valencedomain, the corresponding -based mean classificationrates (S2 column) are quite similar to those from the arousaldomain (S1 column). This justifies the role of the AsI as anefficient index that fosters the corresponding to captureboth the emotional valence and arousal related activity.

B. User-Dependent Case

So far, the presented results refer to a user-independent per-spective, i.e., classification process took place considering thesignals of each subject separately. In an attempt to evaluate theefficiency of the proposed approaches in a user-dependent con-cept, the classification setup previously discussed was also im-plemented for each one of the 16 subjects using the 10-EEGsignal set for each one of the affective states per subject (as de-scribed in Section V-A, for each affective state 10 pictures fromthe IAPS database were used, resulting in 10 respective EEGsignals). In order to relate the classification performance of eachsubject with her/his mean AsI value (in accordance with[9]), all subjects were ranked according to their (in a de-scending order) as is depicted in Fig. 7. Moreover, in the samefigure, the dashed line represents the mean AsI values of thetime-based segmented signals of each user (i.e., ), whichare also presented according to the subjects’ rank derived fromthe and the time-frequency based segmented signals usingthe EMD-wAsI approach (i.e., ) also presented ac-cording to the subjects’ rank derived from the . This way, itcan be shown that the same inclination trend holds for theand values in relation to ones.

Fig. 7. values in descending order along with the andones, in a ranking provided from the descending order of .

Fig. 8. Mean classification rates (for all subjects standard deviation, not shown

here, was around 6.5%), , for HOC (circles) and CC (triangles) methods acrossall subjects, when using the initial EEG signals (solid line) and the

ones (dashed line) and the EMD-wAsI-Masked (gray solid line).

In Fig. 8 the black solid, black dashed, and gray solid lines

present themean classification rate, , derived from the six clas-

sification scenarios, i.e., is the mean of the across the sixclassification scenarios , , for the whole signals, thetime-based segmentation of the signals (wAsI) , and the time-frequency based one (EMD-wAsI), respectively. The circle (o)and triangle marks correspond to HOC and CC methods,respectively. The important conclusion from this figure is thatthe HOC and CC lines incline with the same way as the AsIline, for both initial and segmented signal with wAsI, showingthat a subject with small mean AsI tends to exhibit poorer clas-sification performance in contrast to another with higher mean

values. On the contrary, the segmented signals with theEMd-wAsI approach exhibit an equalized classification perfor-mance for all subjects concentrated around 80% (with thevalues spanning approximately from 70% to 100% for the sixclassification scenarios S1–S6) and 60% (spanning from 50%to 75% for S1-S6) for the HOC and CC methods, respectively.


VII. DISCUSSION

From the results presented in the previous section, the time-frequency segmentation, i.e., EMD-wAsI approach, led, in mostcases, to a more efficient isolation of the emotional informationin EEG signals and resulted in better classification performanceduring an EEG-ER task (Fig. 6). It should also be stressed outthat this classification rate improvement was detected for allclassification scenarios (S1–S6), either for all signals that ap-peared to have or for signals that belong to theBig AsI group, and for both feature extraction methods. Thisconfirms the effectiveness of the EMD-wAsI method to isolatethe emotion-related information, which, consequently, led to amore reliable EEG-ER system.The way of descending seen between the and

values (see Fig. 7) becomes more “horizontal” in the caseof the values. Moreover, the values arelarger, something that should be expected as thevalues were extracted from reconstructed EEG segments afterthe emotional-based filtering of the initial EEG signals withthe EMD-wAsI method, which led to the isolation of thosesegments, where the asymmetry concept was more dominant.

All the above observations explain the resulted values, de-rived from the EMD-wAsI approach for the subject dependentclassification (see Fig. 8, gray solid line). There, a balanced clas-sification performance for all subjects is exhibited in contrastwith the wAsI case and the case where the initial signal (withoutany segmentation) is used, where the classification performanceis highly dependent on the value of each subject. This factlead to the conclusion that the EMD-wAsI approach has effec-tively isolated great amount of the emotion-related informationdistributed through the time and the frequency characteristics ofthe EEG recordings that refer to a specific emotion elicitation

trial. The improvement in the values of the subjects with rel-atively low values is more dominant, whereas the subjectswith higher values exhibit a slight decrease in some cases,which may happen due to the filtering procedure that might dis-card also valuable information. Nevertheless, the overall per-formance for both the HOC and CC feature vector approachesshows that the EMD-wAsI method provides with an efficientway to isolate the desired information.As it can be seen in Fig. 8 (where the subject-dependent

classification results are depicted), the classification rates forsubject 11 for the wAsI approach are missing, as not enough

EEG signals were extracted in order to havereasonable number of signals to perform effectively the classi-fication step. For instance, subject 11 had only two trials with

in the HALV case and the prerequisite for areliable classification performance was set to at least three trialsfor each affective state. On the other hand, for the EMD-wAsI

approach, the same subject is found to have valid values,as were regularly found in the IMFs of hisEEG signals. Moreover, as noted in the previous section, thesignals that appear to have were {85, 95, 93,93}, and {131, 135, 134, 134} for the {LALV, LAHV, HAHV,HALV} cases for the wAsI and the EMD-wAsI approaches,respectively. Thus, it is obvious that the EMD-wAsI methodovercomes in some way the difficulty of the approach

Fig. 9. Number of IMFs selected from the EMD-AsI approach.

Fig. 10. The normalized mean between all the extracted fromthe wAsI approach for each one second of the 5-s period of the picture projectionphase in the form of a normalized activation level.

to extract a for a large number of signals. Thisprovides with a larger dataset to perform the classification stepand, moreover, to better evaluate the subject-dependent per-formance, where few signals are available and become evenfewer if the signals without are not taken intoconsideration.In Fig. 9, the total number of IMFs that was chosen from

the EMD-AsI method for the four affective cases is shown. Itis obvious that the first two IMFs (out of totally five IMFs) arein major chosen from the algorithm. A small number of the thirdIMF is chosen for each case. The aforementioned observationsshow that with the EMD-AsI algorithm, a significant amountof the frequency component of each signal is excluded and thisleads to a poor classification performance (see Fig. 6).In an attempt to generally identify the localization of the emo-

tion related information for the four affective states, i.e., HALV,HAHV, LALV, and LAHV, the mean between all the

extracted from the wAsI approach for each one secondof the 5-s period of the picture projection phase was estimatedand the results in the form of a normalized activation level aredepicted in Fig. 10. From this figure, two major conclusions canbe drawn. First, it is obvious that the 1st and the 5th secondsdo not exhibit emotional information, as far as the asymmetryconcept is concerned, revealing a probable transitional period


Fig. 11. The normalized mean between all the extracted fromthe EMD-wAsI approach for each one of the first four IMFs and for each onesecond of the 5 s period of the picture projection phase in the form of a normal-ized activation level.

from calmness to emotional excitement during the 1st second,and a possible loss of emotional interest during the last second.Second, it should be stressed out that for the HA affective statesthe duration of emotional activation is relatively longer in com-parison with the LA affective states, as for HALV and HAHVcases, the 2nd, 3rd, and 4th seconds appear to be activated al-most equally, whereas LALV case exhibits a significant activa-tion only during the 2nd second and LAHV case for the 3rd and,perhaps, the 4th one. This observation is probably related withthe intensiveness of the emotional content reflected by the corre-sponding pictures in the HA case. Thus, a more intense emotionmaintains for a longer period of time than a milder one.In order to identify the localization of the emotion related in-

formation for the four affective states in each one of the fre-quency components as presented by the IMFs of the signals,the mean between all the extracted from theEMD-wAsI approach for each IMF and for each one second ofthe 5 s period of the picture projection phase was estimated, andthe results in the form of a normalized activation level are de-picted in Fig. 11. It must be stressed out that none of the 5thIMFs exhibited a , and, as a result, it is absentfrom Fig. 11. In this figure, the time-localized frequency compo-nent that is excluded from the EMD-AsI approach is revealed. Aslight but profound activity is observed in the 4th IMF in all af-fective cases, proving that countable amount of emotion relatedinformation is ‘stored’ in the 4th IMF, which is totally excludedfrom the EMD-AsI approach. This fact, along with the enhancedparticipation of the 3rd IMF in the EMD-wAsI approach, in con-trast with the EMD-AsI one, vindicates the superiority of theEMD-wAsI over the EMD-AsI.The values of the EMD-wAsI and

approaches show a relative equivalence between them. Forexample, the maximum classification rate is obtained forthe EMD-wAsI approach, whereas,approach, due to its realization only in some selected IMFs,

exhibits a relative faster implementation in contrast with themethod. Thus, the selection of one of the

two approaches is totally on the personal assessment of theresearcher.The value of finally selected for the HOC based feature ex-

traction method when applied to the EMD-wAsI case was set to, as it provided with the maximum classification perfor-

mance. This could be compared with the corresponding valuesof the best cases of the EMD-AsI, wAsI, and AsI approaches,where , , and , respectively. The exclu-sion of IMFs in the EMD-AsI case and the exclusion of IMFsand/or time segments of the signals in the EMD-wAsI case re-sulted in a significant decrease of the optimum HOC order, incontrast with the case where only a time segment wasextracted or the whole signal with big AsI value was used (AsIapproach [9]). This led to even faster implementation of theEEG-ER system, as the estimation of the HOC sequence of highorders results in a more time-consuming process. For instance,for a 5-s EEG signal, the time to estimate the HOC sequence for

and are approximately 0.8 and 0.2 s, respectively(see also [5]).

VIII. CONCLUSION

In this paper, novel AsI-based algorithms are introduced inorder to effectively retrieve the emotion-related informationfrom EEG recordings. The frontal EEG asymmetry, combinedwith the multidimensional directed information approach,is exploited in order to define an efficient way for time-fre-quency-based segmentation (wAsI, EMD-AsI, and EMD-wAsIapproaches). In this way, the signal segments with less ’emo-tion-related’ information are discarded, keeping only thevaluable ones, contributing to a more reliable EEG-ER process.Based on the presented results, the EMD-wAsI method resultedin the EEG segments that seem to demonstrate better classifi-cation attitudes compared to the other approaches, evaluatedusing two feature vector extraction techniques and six classifi-cation scenarios via an SVM classifier on an experimental EEGdataset derived from 16 subjects. The promising classificationrates derived from the proposed ’emotional filtering’ of theEEG signals set a new perspective in the emotional informationretrieval approach, enhancing the potential of EEG to reflectemotional arousals. Despite the advantageous potential of thepresented EMD-wAsI approach, further justification of itsreliability is needed through its implementation to datasets oflarge-scale experiments.

APPENDIX

Consider a finite zero-mean series oscil-lating about level zero. Let be the backward difference oper-ator defined by

(A1)

The difference operator is a high-pass filter. If we define thefollowing sequence of high-pass filters:

(A2)


with being the identity filter, we can estimate thecorresponding HOC, namely simple HOC [34], by

(A3)

where is the number of zero crossings in respective order,denotes the estimation of the number of zero-cross-

ings and

(A4)

In practice, we only have finite time series and lose an ob-servation with each difference. Hence, to avoid this effect, wemust index the data by moving to the right, i.e., for the eval-uation of simple HOC, the index should be given tothe th or a later observation. For the estimation of the numberof zero-crossings in (13), a binary time series is initiallyconstructed given by

ifif

(A5)and the desired simple HOC are then estimated by countingsymbol changes in , i.e.

(A6)

In finite data records it holds [34].

REFERENCES

[1] G. O. A. Heraz and C. Frasson, “Predicting the three major dimensionsof the learner’s emotions from brainwaves,” Int. J. Comput. Sci., vol.2, no. 3, pp. 187–193, 2008.

[2] K. Takahashi, “Remarks on emotion recognition from bio-potential sig-nals,” in Proc. 2nd Int. Conf. Autonomous Robots Agents, 2004, pp.186–191.

[3] D. O. Bos, EEG-Based Emotion Recognition: The Influence of Visualand Auditory Stimuli Jan. 14, 2012 [Online]. Available: http://hmi.ewi.utwente.nl/verslagen/capita-selecta/CS-Oude_bos-danny.pdf

[4] P. C. Petrantonakis and L. J. Hadjileontiadis, “Emotion recognitionfrom EEG using higher order crossings,” IEEE Trans. Inf. Technol.Biomed. , vol. 14, no. 2, pp. 186–197, 2010.

[5] P. C. Petrantonakis and L. J. Hadjileontiadis, “Emotion recognitionfrom brain signals using hybrid adaptive filtering and higher-ordercrossings analysis,” IEEE Trans. Affect. Comput., vol. 1, no. 2, pp.81–97, 2010.

[6] K. Schaaff and T. Schultz, “Towards emotion recognition from elec-troencephalographic signals,” in Proc. 3rd Int. Conf. Affect. Comput.Intell. Interaction (ACII), Amsterdam, Netherlands, 2009, pp. 1–6.

[7] Y. Liu, O. Sourina, and M. K. Nguyen, “Real-time EEG-based humanemotion recognition and visualization,” in Proc. 2010 Int. Conf. onCyberworlds, Singapore, 2010, pp. 262–269.

[8] R. W. Picard, Affective Computing. Boston, MA: MIT Press, 1997.[9] P. C. Petrantonakis and L. J. Hadjileontiadis, “A novel emotion elic-

itation index using frontal brain asymmetry for enhanced EEG-basedemotion recognition,” IEEE Trans. Inf. Technol. Biomed..

[10] O. Sakata, T. Shiina, and Y. Saito, “Multidimensional directed infor-mation and its application,” Elect. Commun. in Japan, vol. 4, pp. 3–85,2002.

[11] N. Huang, Z. Shen, S. Long, M. Wu, H. H. Shih, N. C. Zheng, N. C.Yen, C. Tung, and H. Liu, “The empirical mode decomposition andHilbert spectrum for nonlinear and nonstationary time series analysis,”Proc. Roy. Soc. London A, vol. 454, pp. 903–995, 1998.

[12] T. Musha, Y. Terasaki, H. A. Haque, and G. A. Ivamitsky, “Feature ex-traction from EEGs associated with emotions,” Artific. Life and Robot.,vol. 1, no. 1, pp. 15–19, 1997.

[13] J. Russell, “Core affect and the psychological construction of emotion,”Psycolog. Rev., vol. 110, no. 1, pp. 145–172, 2003.

[14] R. J. Davidson, G. E. Schwartz, C. Saron, J. Bennett, and D. J.Goleman, “Frontal versus parietal EEG asymmetry during positiveand negative affect,” Psychophysiol., vol. 16, pp. 202–203, 1979.

[15] R. J. Davidson, “What does the prefrontal cortex Do affect: Perspec-tives on frontal EEG asymmetry research,” Biolog. Psychol., vol. 67,pp. 219–233, 2004.

[16] D. Hagemann, E. Naumann, A. Lurken, G. Becker, S. Maier, and D.Bartussek, “EEG asymmetry, dispositional mood and personality,”Pers. Individ. Differences, vol. 27, pp. 541–568, 1999.

[17] J. A. Coan and J. J. B. Allen, “Frontal EEG asymmetry as a moderatorand mediator of emotion,” Biolog. Psychol., vol. 67, pp. 7–49, 2004.

[18] T. Kamitake, H. Harashima, and H. Miyakawa, “Time series analysisbased on directed information,” Trans. IEICE, pp. 103–110, 1984.

[19] G. Wang and M. Takigawa, “Directed coherence as a measure of in-terhemispheric correlation of EEG,” Int. J. Psychophysiol., vol. 13, pp.119–128, 1992.

[20] M. Kaminski and K. J. Blinowska, “A new method of the descriptionof the information flow in the brain structures,” Biolog. Cybern., vol.65, pp. 203–210, 1991.

[21] L. A. Baccala and K. Sameshima, “Partial directed coherence: A newconcept in neural structure determination,” Biolog. Cybern., vol. 84,pp. 463–474, 2001.

[22] W. Hesse, E. Moller, M. Arnord, and B. Schack, “The use of time-variant EEG Granger causality for inspecting directed interdependen-cies of neural assemblies,” J. Neurosci. Methods, vol. 124, pp. 27–44,2003.

[23] R. D. King, C. Feng, and A. Shutherland, “StatLog: Comparison ofclassification algorithms on large real-world problems,” Appl. Artif. In-tell., vol. 9, pp. 259–287, 1995.

[24] W. J. Krzanowski, Principles ofMultivariate Analysis. Oxford, U.K.:Oxford Univ. Press, 1988.

[25] P. C. Mahalanobis, “On the generalized distance in statistics,” Proc.Nat. Inst. Sci. India, vol. 2, pp. 49–55, 1936.

[26] T. Mitchell, Machine Learning. New York: McGraw-Hill, 1997.[27] N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector

Machines and Other Kernel-Based Learning Methods. Cambridge,U.K.: Cambridge Univ. Press, 2000.

[28] P. J. Lang, M.M. Bradley, and B. N. Cuthbert, “International AffectivePicture System (IAPS): Affective Ratings of Pictures and InstructionManual,” Univ. Florida, Gainesville, 2008, Tech. Rep. A-8.

[29] H. Jasper, “The ten-twenty electrode system of the international feder-ation,” Electroencephalogr. Clin. Neurophysiol., vol. 39, pp. 371–375,1958.

[30] K. Coburn and M. Moreno, “Facts and artifacts in brain electrical ac-tivity mapping,” Brain Topogr., vol. 1, pp. 37–45, 1988.

[31] D. O. Olguin, “Adaptive Digital Filtering Algorithms for the Elimina-tion of Power Line Interference in Electroencephalographic Signals,”Master, Instituto Tecnologico y de Estudios Superiores de Monterrey,Monterrey, Mexico, 2005.

[32] M. Fatourechi, A. Bashashati, R. K. Ward, and G. E. Birch, “EMG andEOG artifacts in brain computer interface systems: A survey,” Clin.Neurophysiol., vol. 118, pp. 480–494, 2007.

[33] M.M. Bradley and P. J. Lang, “Affective reactions to acoustic stimuli,”Psychophysiol., vol. 37, pp. 204–215, 2000.

[34] B. Kedem, Time Series Analysis by Higher Order Crossings. Piscat-away, NJ: IEEE, 1994.

Panagiotis C. Petrantonakis (S’08) was born inIerapetra, Crete, Greece, in 1984. He received theDiploma degree in electrical and computer engi-neering in 2007 from the Aristotle University ofThessaloniki (AUTH), Thessaloniki, Greece.Currently, he is a Ph.D. researcher at AUTH, af-

filiated with the Signal Processing and BiomedicalTechnology Unit of the Telecommunications Labo-ratory. His current research interests lie in the areaof advanced signal processing techniques, nonlineartransforms, and affective computing.

Dr. Petrantonakis is a member of the Technical Chamber of Greece.


Leontios J. Hadjileontiadis (S’87–M’98–SM’11)was born in Kastoria, Greece, in 1966. He receivedthe Diploma degree in electrical engineering in 1989and the Ph.D. degree in electrical and computerengineering in 1997, both from the Aristotle Univer-sity of Thessaloniki, Thessaloniki, Greece. He alsoreceived the Diploma in Musicology from AristotleUniversity of Thessaloniki, in 2011, and the Ph.D.degree in music composition from the University ofYork, U.K., in 2004Since December 1999, he has been with the De-

partment of Electrical and Computer Engineering, Aristotle University of Thes-saloniki, as a faculty member, where he is currently an Associate Professor,working on lung sounds, heart sounds, bowel sounds, ECG data compression,seismic data analysis and crack detection in the Signal Processing and Biomed-ical Technology Unit of the Telecommunications Laboratory. His research in-terests are in higher-order statistics, alpha-stable distributions, higher-order zerocrossings, wavelets, polyspectra, fractals, neuro-fuzzy modeling for medical,

mobile, and digital signal processing applications. He is currently a Professorin composition at the State Conservatory of Thessaloniki, Greece.Dr. Hadjileontiadis is a member of the Technical Chamber of Greece, the

Higher-Order Statistics Society, of the International Lung Sounds Association,and of the American College of Chest Physicians. He was the recipient of thesecond award at the Best Paper Competition of the 9th Panhellenic MedicalConference on Thorax Diseases’97, Thessaloniki. He was also an open finalistat the Student Paper Competition (Whitaker Foundation) of the IEEE EMBS’97,Chicago, IL, a finalist at the Student Paper Competition (in memory of DickPoortvliet) of the MEDICON’98, Lemesos, Cyprus, and the recipient of theYoung Scientist Award of the 24th International Lung Sounds Conference’99,Marburg, Germany. In 2004, 2005, and 2007, he organized and served as amentor to three five-student teams that have ranked as third, second, and sev-enth worldwide, respectively, at the Imagine Cup Competition (Microsoft), SaoPaulo, Brazil (2004)/Yokohama, Japan (2005)/ Seoul, Korea (2007), New York(2011), with projects involving technology-based solutions for people with dis-abilities and pain management.

Documents

Adaptive Emotional Information Retrieval From EEG Signals in TFD