Clustering of Time Course Gene-Expression Data via Mixture Regression Models

Clustering of Time Course Gene-Expression Data

via Mixture Regression ModelsGeoff McLachlan

(joint with Angus Ng and Sam Wang)

Department of Mathematics & Institute for Molecular BioscienceUniversity of Queensland

ARC Centre of Excellence in Bioinformatics

http://www.maths.uq.edu.au/~gjm

Institute for Molecular Bioscience, University of Queensland

Time-Course Data

Time-course microarray experiments are being increasingly used to characterize dynamic biological processes.(Microarray technology provides the ability to measure the expressionlevels of thousands of genes at once.)

In these experiments, gene-expression levels are measured at different time points, possibly in different biological conditions (e.g. treatment-control). The focus here is on the analysis of gene-expression profilesconsisting of short time series of log expression ratios foreach of the genes represented on the microarrays.

CLUSTERING OF GENE PROFILES

can provide new insight into the biological proces of interest (coexpressed genes can contribute to our understanding of the regulatory network of gene expression). can also assist in assigning functions to genes that have not yet been functionally annotated.

a secondary concern is the need for imputation of missing data

The biological rationale underlying the clustering ofmicroarray data is the fact that many coexpressed genesare coregulated. It becomes a way of identifying sets ofgenes that are putatively coregulated, thereby generatingtestable hypotheses; see Boutros and Okey (2005).

It assists with: the functional annotation of uncharacterised genes

the identification of transcription factor binding sites

the elucidation of complete biological pathways

Outline of Talk

1. Mixture model-based approach to analysis of gene-expressions

2. Normal Mixtures

3. Modifications for high-dimensional and/or structured data

4. Mixtures of linear mixed models

5. Clustering of gene profiles

• Provide an arbitrarily accurate estimate of the underlying density with g sufficiently large

• Provide a probabilistic clustering of the data into g clusters - outright clustering by assigning a data point to the component to which it has the greatest posterior probability of belonging.

Finite Mixture Models

Definition

We let Y1,…. Yn denote a random sample of size n where Yj is a p-dimensional random vector with probability density function f (yj)

where the f i(yj) are densities and the i are nonnegative quantities that sum to one.

)y()y( j

1iij iff

By Bayes Theorem,

for i=1,…, g; j=1,…,n.

);( )()( kji

kij Ψy

);(/);( )()()()( kj

ki ff Ψyθy

The quantity i(yj;(k)) is the posterior probability that the jth member of the sample with observed value yj belongs to the ith component of the mixture.

A soft (probabilistic) clustering is given in terms of theestimated posterior probabilities of component membership

A hard (outright) clustering is given by assigning each yj

to the component to which it has the highest posteriorprobability of belonging; that is, given by thewhere

,ˆijz

ij iz max arg if ,1ˆ

otherwise. ,0

Multivariate Mixture Models Day (Biometrika, 1969) Wolfe (NORMIX, 1965, 1967, 1970)

It was the publication of the seminal paper of Dempster, Laird, and Rubin (1977) on the EM algorithm that greatly stimulated interest in the use of finite mixture distributions to model heterogeneous data.

Multivariate Mixture Models Day (Biometrika, 1969) Wolfe (NORMIX, 1965, 1967, 1970)

It was the publication of the seminal paper of Dempster, Laird, and Rubin (1977) on the EM algorithm that greatly stimulated interest in the use of finite mixture distributions to model heterogeneous data.

Ganesalingam and McLachlan (Biometrika,1978)

• Everitt and Hand (2001)

• Titterington, Smith, and Makov (1985)

• Everitt and Hand (2001)

• Titterington, Smith, and Makov (1985)

• McLachlan and Basford (1988)

• Lindsay (1996)

• McLachlan and Peel (2000)

• Bohning (2000)

• Fruhwirth-Schnatter (2006)

Normal Mixtures

where is the vector containing the unknown parameters.

),,;( );(1

iij Ψf yy

Suppose that the density of the random vector Yj has a g-component normal mixture form

One attractive feature of adopting mixture models with elliptically symmetric components, such as the normal or t densities, is that the implied clustering is invariant under affine transformations of the data, i.e., under operations relating to changes in location, scale, and rotation of the data.

Thus the clustering process does not depend on irrelevant factors such as the units of measurement or the orientation of the clusters in space.

Sample 1 Sample 2 Sample M

Gene 1Gene 2

Gene N

Expression ProfileE

xpression S

ignature

Microarray Data represented as N x M Matrix

N rows (genes) ~ 104

M columns (samples) ~ 102

Clustering of Microarray Data

Clustering of tissues on basis of genes:

latter is a nonstandard problem in

cluster analysis (n =M << p=N)

Clustering of genes on basis of tissues:

genes (observations) not independent and

structure on the tissues (variables) (n=N >> p=M)

The component-covariance matrix Σi is highly parameterized with p(p+1)/2 parameters.

Σi = σ2Ip (equal spherical)

Σi = σi2Ip (unequal spherical)

Σi = D (equal diagonal)

Σi = Di (unequal diagonal)

Σi = Σ (equal)

Banfield and Raftery (1993) introduced a parameterization of the component-covariance matrix Σi based on a variant of the standard spectral decomposition of Σi

(i=1, …,g).

However, if p is large relative to the sample size n, it may not be possible to use this decomposition to infer an appropriate model for the component-covariance matrices.

Even if it is possible, the results may not be reliable due to potential problems with near-singular estimates of the component-covariance matrices when p is large relative to n.

Hence, in fitting normal mixture models to high-dimensional data, we should first consider

• some form of dimension reduction and/or

• some form of regularization

Mixture Software: EMMIX

McLachlan, Peel, Adams, and Basfordhttp://www.maths.uq.edu.au/~gjm/emmix/emmix.html

EMMIX for UNIX

PROVIDES A MODEL-BASED APPROACH TO CLUSTERING

McLachlan, Bean, and Peel, 2002, A Mixture Model-Based Approach to the Clustering of Microarray

Expression Data, Bioinformatics 18, 413-422

http://www.bioinformatics.oupjournals.org/cgi/screenpdf/18/3/413.pdf

Sample 1 Sample 2 Sample M

Gene 1Gene 2

Gene N

Expression ProfileE

xpression S

ignature

Microarray Data represented as N x M Matrix

N rows (genes) ~ 104

M columns (samples) ~ 102

In applying the normal mixture model to cluster multivariate(continuous) data, it is assumed as in most typical cluster

analyses using any other method that

(a) there are no replications on any particular entity specifically identified as such;

(b) all the observations on the entities are independent of one another

For example,

,),( 21TT

iTii X

).2,1,( 1 ihhphihi

),2,1( 2

).2,1,( 2 ihIhphihi

• Longitudinal (with or without replication, for example time-course)

• Cross-sectional data

Clustering of gene expression profiles

Ng, McLachlan, Wang, Ben-Tovim Jones, and Ng (2006, Bioinformatics)

Supplementary information :

http://www.maths.uq.edu.au/~gjm/bioinf0602_supp.pdf

EMMIX-WIREEM-based MIXture analysis With Random Effects

),,1( njijiijij εVcUbXβy

In the ith component of the mixture, the profile vector yj for the jth gene follows the model

1p 1m 1bq 1cq 1p

),(~ iij N H0b

),(~ ci cqi N I0c

),(~ iA0Nij

Tiqiiii e

),,(),diag( 221 φWφA

},|1{);,( cyZprcy jijji

h hhhjjh

iiijji

1);,1|(

);,1|(

N(iiwith iii VcX T

biii UUAB

• Celeux et al. (2005). Mixtures of linear mixed models for clustering gene expression profiles from repeated microarray measurements. Statistical Modelling 5 , 243-267.

• Qin and Self (2006). The clustering of regression models method with applications in gene expression data. Biometrics 62, 526-533.

• Booth et al. (2008). Clustering using objective functions and stochastic search. J R Statist Soc B 70, 119-139.

Yeast cell cycle data of Cho et al. (1998)

n=237 genes at p=17 time points

categorized into 4 MIPS (Munich Information Centre for Protein Sequences) functional groups.

The yeast system is useful because of our ability to control and monitor the progression of cells through the cell cycle (temperature-based synchronization with temperature-sensitive genes whose product is essential for cell-cycle progression).

High-density oligonucleotide arrays were used to quanitate mRNA transcript levels in synchronized

yeast cells at regular intervals (10 min) during the cell cycle

(genes with cell-cycle dependent periodicity).

Samples of yeast cultures were taken at 17 time points after their cell cycle phase had been synchronized.

The data were reduced to a short time series of log expression ratios for each of the yeast genes represented on the microarrays (expression ratios were calculated by dividing each intensity measurement by the average for that gene.

n = 237 genes

p = 17 time points

ijiijij εVcUbXβy

)/2sin()/2cos(

ijij b171Ub

)diag()diag()cov( 217 iiij 1Wφε

Example . Clustering of yeast cell cycle time-course data

In the ith cluster,

ijkikij

ikikjk

)2sin()2cos( 21

jkkkjk eTtTty )/2sin()/2cos( 210

T is the period – estimated to be 73 min.

),0(~ 2Ne jk

0, 10, 20,…, 160

Estimated T following Booth et al. (2004)

The Number Of Components

2 3 4 5 6 7

10883 10848 10837 10865 10890 10918

Table 1: Values of BIC for Various Levels of the Number of Components g

Cluster-specific random effects term

),(~ 17IOc cii N

Tc )0.14 0.04, 0.28, ,23.0(ˆ θ

Table 2: Summary of Clustering Results for g = 4 Clusters

Model Rand Index Adjusted Rand Index

Error Rate

1 0.7808 0.5455 0.2910

2 0.7152 0.4442 0.3160

3 0.7133 0.3792 0.4093

Wong 0.7087 0.3697 NA

• The use of the cluster-specific random effects terms ci leads to a clustering that corresponds more closely to the underlying functional groups than without their use.

Figure 1: Clusters of gene-profiles obtained by mixture of linear mixed models with cluster-

specific random effects

0 50 100 150

Figure 2: Clusters of gene-profiles obtained by mixture of linear mixed models without cluster-

specific random effects

0 50 100 150

Figure 3: Clusters of gene-profiles obtained by mixtures of linear mixed models with and without cluster-specific

random effects

0 50 100 150

Figure 4: Plots of gene profiles grouped according to their functional grouping

0 50 100 150

Figure 5: Plots of clustered gene profiles versus functional grouping

0 50 100 150

Figure 6: Clusters of gene-profiles obtained by k-means

0 50 100 150

Figure 7: Plots of Clusters of gene-profiles: Model-based clustering versus k-means

0 50 100 150

Another Yeast Cell Cycle Dataset

Spellman (1998 used α-factor (pheromone) synchronization where the yeast cells were sampled at 7 minute intervals for 119 minutes; the period of the cell cycle was estimated using least squares to be T=53 min.

n = 612 genes

p = 18 time points

ijiijij εVcUbXβy

)/.2sin()/.2cos(

ijij b181Ub

)diag()diag()cov( 218 iiij 1Wφε

Example . Clustering of time-course data

Clustering Results for Spellman Yeast Cell Cycle Data

Mixtures of linear mixed models Useful in modelling biological processes that exhibit periodicity atdifferent temporal scales (not restricted to cell cycle data; e.gchanges in core body temperature, heart rate, blood pressure).

In summary, they provide a flexible tool to cluster high-dimensional data (which may be correlated and structured) for a wide range of experimental designs, e.g. - longitudinal data (with or without replication)

- cross sectional data (multiple samples at one time point).

Provide an integrated framework for the analysis of microarray data by incorporating experimental designinformation and (biological or clinical) covariates.

Clustering of Time Course Gene-Expression Data via Mixture Regression Models

Documents

MLPR: Clustering, Mixture Models, and the EM Algorithm - …€¦ · MLPR: Clustering, Mixture Models, and the EM Algorithm Machine Learning and Pattern Recognition Charles Sutton

Copula Mixture Model for Dependency-seeking Clustering - arXiv

Multilevel regression mixture models 1 - eprints.lancs.ac.ukeprints.lancs.ac.uk/72847/1/Multilevel_Regression_Mixtures... · Multilevel regression mixture models 3 A common research

Optimal interval clustering: Application to Bregman clustering and statistical mixture learning

Regression, Classification and Clustering

MIXTURE MODEL CLUSTERING FOR VERY LARGE …studentsrepo.um.edu.my/6108/1/Mixture_Model_Clustering_for_Very... · MIXTURE MODEL CLUSTERING FOR VERY LARGE DATA ... 2010 . MIXTURE MODEL

3F3: Signal and Pattern Processingsvr-mjfg/local/3F3/lect1.pdf · Mixture Modelling page. Welcome to David Dowe's clustering, mixture modelling Welcome to David Dowe's clustering,

Correlated variables in regression: clustering and …Correlated variables in regression: clustering and sparse estimation Peter Bu hlmann 1, Philipp Rutimann , Sara van de Geer ,

Sparse regression with exact clustering

Bayesian inﬁnite mixture model based clustering of gene

Subspace Clustering via Variance Regularized Ridge Regression · 2017. 5. 31. · Subspace Clustering via Variance Regularized Ridge Regression Chong Peng, Zhao Kang, Qiang Cheng

Network Regression with Predictive Clustering -

Multivariate Robust clustering via mixture models Robust clustering via mixture models INRIA-LJK équipe MISTIS (Grenoble) Florence Forbes , Darren Wraith, SenanDoyle, Eric Frichot

Lecture 3-4 Clustering (1hr) Gaussian Mixture and EM (1hr)

Gaussian Scale Mixture Models For Robust Linear Multivariate Regression With Missing … · GAUSSIAN SCALE MIXTURE MODELS FOR ROBUST LINEAR MULTIVARIATE REGRESSION WITH MISSING DATA

Bayesian Quantile Mixture Regressionthanos/BNP2019_talk.pdfPrologos Quantile Mixture Regression Data Application Epilogos Bayesian Quantile Mixture Regression Athanasios Kottas Department

Mixture Regression Modelle. Gründe für die Anwendung von Mixture Regression Modellen am Beispiel der Conjointanalyse

Model-based clustering and classiﬁcation with non-normal ... · Model-based clustering and classiﬁcation with non-normal mixture distributions 3 the components of the mixture

Clustering and Regression using WEKA

Probabilistic analog of clustering: mixture models We model uncertainty in the identification of objects to mixture components Each mixture component can