View
5
Download
0
Category
Preview:
Citation preview
UsingEnsemblesofHiddenMarkovModelsforGrandChallengesinBioinformatics
TandyWarnowFounderProfessorofEngineering
TheUniversityofIllinoisatUrbana-Champaignhttp://tandy.cs.illinois.edu
From the Tree of the Life Website, University of Arizona
Phylogenomics
1kp:ThousandTranscriptomeProject
l PlantTreeofLifebasedontranscriptomesof~1200speciesl Morethan13,000genefamilies(mostnotsinglecopy)GeneTreeIncongruence
G. Ka-Shu Wong U Alberta
N. Wickett Northwestern
J. Leebens-Mack U Georgia
N. Matasci iPlant
T. Warnow, S. Mirarab, N. Nguyen UIUC UCSD UCSD
Challenge: Alignments and trees on > 100,000 sequences
Plus many many other people…
1000-taxonmodels,orderedbydifficulty(Liuetal.,Science324(5934):1561-1564,2009)
Re-aligning on a tree A
B D
C
Merge sub-alignments
Estimate ML tree on merged
alignment
Decompose dataset
A B
C D
Align subsets
A B
C D
ABCD
SATéandPASTAAlgorithms
Estimate ML tree on new alignment
Tree
Obtain initial alignment and estimated ML tree
Use tree to compute new alignment
Alignment
Repeatuntilterminationcondition,and
returnthealignment/treepairwiththebestMLscore
1000taxonmodels,orderedbydifficulty,Liuetal.,Science324(5934):1561-1564,2009
24hourSATé-Ianalysis,ondesktopmachines
(Similarimprovementsforbiologicaldatasets)
1000-taxonmodelsrankedbydifficulty
SATé-2betterthanSATé-1
SATé-1(Liuetal.,Science2009):cananalyzeupto8KsequencesSATé-2(Liuetal.,SystematicBiology2012):cananalyzeupto~50Ksequences
RNASim
0.00
0.05
0.10
0.15
0.20
10000 50000 100000 200000
Tree
Erro
r (FN
Rat
e) Clustal−OmegaMuscleMafftStarting TreeSATe2PASTAReference Alignment
PASTA:Mirarab,Nguyen,andWarnow,JComp.Biol.2015– SimulatedRNASimdatasetsfrom10Kto200Ktaxa– Limitedto24hoursusing12CPUs– Notallmethodscouldrun(missingbarscouldnotfinish)
PASTA:evenbetterthanSATé-2
Decompositionto100-sequencesubsets,oneiterationofPASTA+BAli-Phy
ComparingdefaultPASTAtoPASTA+BAli-Phyonsimulateddatasetswith1000sequences
PASTA+BAli−Phy Better
PAS
TA B
etter
0.0
0.1
0.2
0.3
0.4
0.0 0.1 0.2 0.3 0.4PASTA
PAS
TA+B
Ali−
Phy
Total Column Score
dataIndelible M2
RNAsim
Rose L1
Rose M1
Rose S1
PASTA+BAli−Phy Better
PAS
TA B
etter
0.6
0.7
0.8
0.9
1.0
0.6 0.7 0.8 0.9 1.0PASTA
PAS
TA+B
Ali−
Phy
Recall (SP−Score)
PASTA+BAli−Phy Better
PAS
TA B
etter
0.00
0.05
0.10
0.15
0.00 0.05 0.10 0.15PASTA+BAli−Phy
PAS
TA
Tree Error: Delta RF (RAxML)
1kp:ThousandTranscriptomeProject
l PlantTreeofLifebasedontranscriptomesof~1200speciesl Morethan13,000genefamilies(mostnotsinglecopy)GeneTreeIncongruence
G. Ka-Shu Wong U Alberta
N. Wickett Northwestern
J. Leebens-Mack U Georgia
N. Matasci iPlant
T. Warnow, S. Mirarab, N. Nguyen UIUC UCSD UCSD
Challenge: Alignments and trees on > 100,000 sequences
Plus many many other people…
Length
Counts
0
2000
4000
6000
8000
10000
12000 Mean:317Median:266
0 500 1000 1500 2000
1KPdataset:morethan100,000p450amino-acidsequences,manyfragmentary
Length
Counts
0
2000
4000
6000
8000
10000
12000 Mean:317Median:266
0 500 1000 1500 2000
1KPdataset:morethan100,000p450amino-acidsequences,manyfragmentary
Allstandardmultiplesequencealignmentmethodswetestedperformedpoorlyondatasetswithfragments.
ProfileHiddenMarkovModels• Probabilisticmodeltorepresentafamilyofsequences,
representedbyamultiplesequencealignment
• IntroducedforsequenceanalysisinKroghetal.1994.PopularizedinEddy1996andtextbookDurbinetal.1998
• FundamentalpartofHMMERandotherproteindatabases
• Usedfor:homologydetection,proteinfamilyassignment,multiplesequencealignment,phylogeneticplacement,proteinstructureprediction,alignmentsegmentation,etc.
ProfileHMMsl GenerativemodelforrepresentingaMSA
l Consistsof:
l Setofstates(Match,insertion,anddeletion)
l Transitionprobabilities
l Emissionprobabilities
ProfileHiddenMarkovModelforDNAsequencealignment
HMMsforMSAl Givenseedalignment(e.g.,inPFAM)andacollectionof
sequencesfortheproteinfamily:
l RepresentseedalignmentusingHMM
l AligneachadditionalsequencetotheHMM
l UsetransitivitytoobtainMSA
l Canwedosomethinglikethiswithoutaseedalignment?
HMMsforMSAl Givenseedalignment(e.g.,inPFAM)andacollectionof
sequencesfortheproteinfamily:
l RepresentseedalignmentusingHMM
l AligneachadditionalsequencetotheHMM
l UsetransitivitytoobtainMSA
l Canwedosomethinglikethiswithoutaseedalignment?
UPPUPP=“Ultra-largemultiplesequencealignmentusingPhylogeny-awareProfiles”Nguyen,Mirarab,andWarnow.GenomeBiology,2014.Purpose:highlyaccuratelarge-scalemultiplesequencealignments,eveninthepresenceoffragmentarysequences.
UPPUPP=“Ultra-largemultiplesequencealignmentusingPhylogeny-awareProfiles”Nguyen,Mirarab,andWarnow.GenomeBiology,2014.Purpose:highlyaccuratelarge-scalemultiplesequencealignments,eveninthepresenceoffragmentarysequences.
UsesanensembleofHMMs
Simpleidea(notUPP)
• Selectrandomsubsetofsequences,andbuild“backbonealignment”
• ConstructaHiddenMarkovModel(HMM)onthebackbonealignment
• AddallremainingsequencestothebackbonealignmentusingtheHMM
RNASim
0.00
0.05
0.10
0.15
0.20
10000 50000 100000 200000
Tree
Erro
r (FN
Rat
e) Clustal−OmegaMuscleMafftStarting TreeSATe2PASTAReference Alignment
PASTA:Mirarab,Nguyen,andWarnow,JComp.Biol.2015– SimulatedRNASimdatasetsfrom10Kto200Ktaxa– Limitedto24hoursusing12CPUs– Notallmethodscouldrun(missingbarscouldnotfinish)
PASTA:evenbetterthanSATé-2StartingtreeisbasedonUPP(simple):oneprofileHMMforBackbonealignment
• Selectrandomsubsetofsequences,andbuild“backbonealignment”
• ConstructaHiddenMarkovModel(HMM)onthebackbonealignment
• AddallremainingsequencestothebackbonealignmentusingtheHMM
Thisapproachworkswellifthedatasetissmallandhaslowevolutionaryrates,butisnotveryaccurateotherwise.
One Hidden Markov Model for the entire alignment?
OneHiddenMarkovModelforthebackbonealignment?
HMM1
Or2HMMs?
HMM1
HMM2
HMM1
HMM3 HMM4
HMM2
Or4HMMs?
m
HMM2
HMM3
HMM1
HMM4
HMM5 HMM6
HMM7
Orall7HMMs?
UPPAlgorithmicApproach
1. Selectrandomsubsetoffull-lengthsequences,andbuild“backbonealignment”
2. Constructan“EnsembleofHiddenMarkovModels”onthebackbonealignment
3. AddallremainingsequencestothebackbonealignmentusingtheEnsembleofHMMs
Evaluation• Simulateddatasets(somehavefragmentarysequences):– 10Kto1,000,000sequencesinRNASim–complexRNAsequenceevolutionsimulation
– 1000-sequencenucleotidedatasetsfromSATépapers– 5000-sequenceAAdatasets(fromFastTreepaper)– 10,000-sequenceIndeliblenucleotidesimulation
• Biologicaldatasets:– Proteins:largestBaliBASEandHomFam– RNA:3CRWdatasetsupto28,000sequences
RNASimMillionSequences:treeerror
Using 12 processors: • UPP(Fast,NoDecomp)
took 2.2 days,
• UPP(Fast) took 11.9 days, and
• PASTA took 10.3 days
0.0
0.2
0.4
0.6
0 12.5 25 50% Fragmentary
Mea
n al
ignm
ent e
rror
PASTA UPP(Default)
(a) Average alignment error
0.0
0.2
0.4
0 12.5 25 50% Fragmentary
Del
ta F
N tr
ee e
rror
PASTA UPP(Default)
(b) Average tree error
Figure S32: Alignment and tree error of PASTA and UPP on the fragmentary 1000M2datasets.
80
Performanceonfragmentarydatasetsofthe1000M2modelcondition
UPPisveryrobusttofragmentarysequences
Underhighratesofevolution,PASTAisbadlyimpactedbyfragmentarysequences(thesameistrueforothermethods).UPPcontinuestohavegoodaccuracyevenondatasetswithmanyfragmentsunderallratesofevolution.
●
●
●
●
0
5
10
15
50000 100000 150000 200000Number of sequences
Wal
l clo
ck a
lign
time
(hr)
● UPP(Fast)
UPPRunningTime
Wall-clocktimeused(inhours)given12processors
OtherApplicationsoftheEnsembleofHMMs
SEPP(phylogeneticplacement,Mirarab,Nguyen,andWarnowPSB2014)
TIPP(metagenomictaxonidentification,Nguyen,Mirarab,Liu,Pop,andWarnow,Bioinformatics2014)
HIPPI(proteinclassificationandremotehomologydetection),RECOMB-CG2016andBMCGenomics2016(Nguyen,Nute,Mirarab,andWarnow)
Objective:Distributionofthespecies(orgenera,orfamilies,etc.)withinthesample.
Forexample:Thedistributionofthesampleatthespecies-levelis:
50% speciesA
20% speciesB
15% speciesC
14% speciesD
1% speciesE
AbundanceProfiling
TIPPpipeline
Input:setofreadsfromashotgunsequencingexperimentofametagenomicsample
1. AssignreadstomarkergenesusingBLAST2. Forreadsassignedtomarkergenes,perform
taxonomicanalysis3. CombineanalysesfromStep2
Highindeldatasetscontainingknowngenomes
Note:NBC,MetaPhlAn,andMetaPhylercannotclassifyanysequencesfromatleastoneofthehighindellongsequencedatasets,andmOTUterminateswithanerrormessageonallthehighindeldatasets.
“Novel”genomedatasets
Note:mOTUterminateswithanerrormessageonthelongfragmentdatasetsandhighindeldatasets.
ProteinFamilyAssignment
• Input:newAAsequence(mightbefragmentary)anddatabaseofproteinfamilies(e.g.,PFAM)
• Output:assignment(ifjustified)ofthesequencetoanexistingfamilyinthedatabase
HIPPI
• HIerarchicalProfileHMMsforProteinfamilyIdentification
• Nguyen,Nute,Mirarab,andWarnow,RECOMB-CG2016andBMC-Genomics2016
• UsesanensembleofHMMstoclassifyproteinsequences
• TestedonHMMER
Pre
cis
ion
.97
.98
.98
.99
.75 .80 .85 .90 .95 1.00
Seq Length: Full
.96
.97
.98
.99
.50 .60 .70 .80 .90
Recall
Seq Length: 50%
.92
.95
.98
.20 .40 .60 .80
Seq Length: 25%
Method HIPPI HMMER BLAST HHsearch: 1 Iteration HHsearch: 2 Iterations
FourProblems
• PhylogeneticPlacement(SEPP,PSB2012)
• Multiplesequencealignment(UPP,RECOMB2014andGenomeBiology2014)
• Metagenomictaxonidentification(TIPP,Bioinformatics2014)
• Genefamilyassignmentandhomologydetection(HIPPI,RECOMB-CG2016andBMCGenomics2016)
Aunifyingtechniqueisthe“EnsembleofHiddenMarkovModels”(introducedbyMirarabetal.,2012)
Summary• UsinganensembleofHMMstendstoimproveaccuracy,foracostofrunning
time.Applicationssofartotaxonomicplacement(SEPP),multiplesequencealignment(UPP),proteinfamilyclassification(HIPPI).Improvementsaremostlynoticeableforlargediversedatasets.
• Phylogenetically-basedconstructionoftheensemblehelpsaccuracy(note:thedecompositionsweproducearenotclade-based),butthedesignanduseoftheseensemblesisstillinitsinfancy.(Manyrelativelysimilarapproacheshavebeenusedbyothers,includingSci-PhyandFlowerPowerbySjolander.)
• Thebasicideacanbeusedwithanykindofprobabilisticmodel,doesn’thavetoberestrictedtoprofileHMMs.
Basicquestion:whydoesithelp?
• UsinganensembleofHMMstendstoimproveaccuracy,foracostofrunningtime.Applicationssofartotaxonomicplacement(SEPP),multiplesequencealignment(UPP),proteinfamilyclassification(HIPPI).Improvementsaremostlynoticeableforlargediversedatasets.
• Phylogenetically-basedconstructionoftheensemblehelpsaccuracy(note:thedecompositionsweproducearenotclade-based),butthedesignanduseoftheseensemblesisstillinitsinfancy.(Manyrelativelysimilarapproacheshavebeenusedbyothers,includingSci-PhyandFlowerPowerbySjolander.)
• Thebasicideacanbeusedwithanykindofprobabilisticmodel,doesn’thavetoberestrictedtoprofileHMMs.
Scientific challenges: • Ultra-large multiple-sequence alignment • Alignment-free phylogeny estimation • Supertree estimation • Estimating species trees from many gene trees • Genome rearrangement phylogeny • Reticulate evolution • Visualization of large trees and alignments • Data mining techniques to explore multiple optima • Theoretical guarantees under Markov models of evolution
Techniques: machine learning, applied probability theory, graph theory, combinatorial optimization, supercomputing, and heuristics
The Tree of Life: Multiple Challenges
Acknowledgments
PASTAandUPP:NamNguyen(nowpostdocatUIUC)andSiavashMirarab(nowfacultyatUCSD),undergrad:KeerthanaKumar(atUT-Austin)PASTA+BAli-Phy:MikeNute(PhDstudentatUIUC)CurrentNSFgrants:ABI-1458652(multiplesequencealignment)GraingerFoundation(atUIUC),andUIUCTACC,UTCS,BlueWaters,andUIUCcampusclusterPASTA,UPP,SEPP,andTIPPareavailableongithubathttps://github.com/smirarab/;seealsoPASTA+BAli-Phyathttp://github.com/MGNute/pasta
Nguyen et al. Genome Biology (2015) 16:124 Page 6 of 15
Table 2 Average alignment SP-error, tree error, and TC score across most full-length datasets
Method ROSE RNASim Indelible ROSE CRW 10 AA HomFam HomFam
NT 10K 10K AA (17) (2)
Average alignment SP-error
UPP 7.8 (1) 9.5 (1) 1.7 (2) 2.9 (1) 12.5 (1) 24.2 (1) 23.3 (1) 20.8 (2)
PASTA 7.8 (1) 15.0 (2) 0.4 (1) 3.1 (1) 12.8 (1) 24.0 (1) 22.5 (1) 17.3 (1)
MAFFT 20.6 (2) 25.5 (3) 41.4 (3) 4.9 (2) 28.3 (2) 23.5 (1) 25.3 (2) 20.7 (2)
Muscle 20.6 (2) 64.7 (5) 62.4 (4) 5.5 (3) 30.7 (3) 30.2 (2) 48.1 (4) X
Clustal 49.2 (3) 35.3 (4) X 6.5 (4) 43.3 (4) 24.3 (1) 27.7 (3) 29.4 (3)
Average !FN error
UPP 1.3 (1) 0.8 (1) 0.3 (1) 1.8 (1) 7.8 (2) 3.4 (2) NA NA
PASTA 1.3 (1) 0.4 (1) <0.1 (1) 1.3 (1) 5.1 (1) 3.3 (1) NA NA
MAFFT 5.8 (2) 3.5 (2) 24.8 (3) 4.5 (3) 10.1 (3) 2.3 (1) NA NA
Muscle 8.4 (3) 7.3 (3) 32.5 (4) 3.1 (2) 5.5 (1) 12.6 (3) NA NA
Clustal 24.3 (4) 10.4 (4) X 4.2 (3) 34.1 (4) 3.5 (2) NA NA
Average TC score
UPP 37.8 (1) 0.5 (2) 11.0 (3) 2.6 (2) 1.4 (1) 11.4 (1) 47.3 (1) 40.3 (3)
PASTA 37.8 (1) 2.3 (1) 48.0 (1) 5.4 (1) 2.3 (1) 12.1 (1) 46.1 (2) 50.0 (1)
MAFFT 31.4 (2) 0.4 (2) 7.8 (4) 0.6 (3) 0.7 (2) 12.1 (1) 45.5 (2) 46.9 (2)
Muscle 9.8 (3) <0.0 (2) 18.3 (2) 2.7 (2) 0.7 (2) 10.5 (2) 27.7 (4) X
Clustal 5.7 (4) 0.2 (2) X 3.1 (2) 0.1 (2) 11.8 (1) 38.6 (3) 31.0 (4)
We report the average alignment SP-error (the average of SPFN and SPFP errors) (top), average !FN error (middle), and average TC score (bottom), for the collection offull-length datasets. All scores represent percentages and so are out of 100. Results marked with an X indicate that the method failed to terminate within the time limit(24 hours on a 12-core machine). Muscle failed to align two of the HomFam datasets; we report separate average results on the 17 HomFam datasets for all methods and thetwo HomFam datasets for all but Muscle. We did not test tree error on the HomFam datasets (therefore, the !FN error is indicated by “NA”). The tier ranking for each methodis shown parenthetically
memory error message were marked as failures. Forexperiments on the million-sequence RNASim dataset,we ran the methods on a dedicated machine with 256GBof main memory and 12 cores until an alignment wasgenerated or the method failed. We also performed a lim-ited number of experiments on TACC with UPP’s internal
checkpointing mechanism, to explore performance whentime is not limited. All methods other than Muscle hadparallel implementations and were able to take advantageof the 12 available cores.On full-length datasets (Table 2) where nearly all meth-
ods were able to complete, PASTA was nearly always in
Table 3 Average alignment SP-error and tree error across fragmentary datasets
Method ROSE NT RNASim 10K Indelible 10K CRW
(16S.3 and 16S.T)
Average alignment SP-error
UPP 8.3 (1) 11.8 (1) 2.7 (1) 16.1 (1)
PASTA 25.2 (2) 47.7 (4) 8.8 (2) 23.3 (2)
MAFFT 32.5 (3) 25.5 (2) 51.3 (3) 24.5 (3)
Muscle 35.3 (4) 82.2 (5) 77.6 (4) 70.6 (5)
Clustal 62.0 (5) 35.0 (3) X 46.7 (4)
Average !FN error
UPP 1.9 (1) 3.1 (1) 2.5 (1) 7.4 (2)
PASTA 25.2 (3) 21.9 (3) 9.0 (2) 8.2 (2)
MAFFT 18.0 (2) 6.2 (2) 35.6 (3) 2.5 (1)
Muscle 27.5 (4) 43.6 (5) 45.2 (4) 30.1 (3)
Clustal 47.8 (5) 26.3 (4) X 37.4 (4)
We report the average alignment error (top) and average !FN error (bottom) on the collection of fragmentary datasets. Clustal-Omega failed to align any of the Indelible10000M2 fragmentary datasets and thus we mark the results with an X. The tier ranking for each method is shown in parentheses
.75
.80
.85
.90
.95
1.0
.25 .50 .75 1.0.75
.80
.85
.90
.95
1.0
.25 .50 .75 1.0.75
.80
.85
.90
.95
1.0
.25 .50 .75 1.0
.75
.80
.85
.90
.95
1.0
.25 .50 .75 1.0.75
.80
.85
.90
.95
1.0
.25 .50 .75 1.0.75
.80
.85
.90
.95
1.0
.25 .50 .75 1.0
.80
.90
1.0
.00 .25 .50 .75 1.0
.80
.90
1.0
.00 .25 .50 .75 1.0
.80
.90
1.0
.00 .25 .50 .75 1.0
.80
.90
1.0
.00 .25 .50 .75 1.0
.80
.90
1.0
.00 .25 .50 .75 1.0
.80
.90
1.0
.00 .25 .50 .75 1.0
.70
.80
.90
1.0
.00 .20 .40 .60 .80
.70
.80
.90
1.0
.00 .20 .40 .60 .80
.70
.80
.90
1.0
.00 .20 .40 .60 .80
.70
.80
.90
1.0
.00 .20 .40 .60 .80
.70
.80
.90
1.0
.00 .20 .40 .60 .80
.70
.80
.90
1.0
.00 .20 .40 .60 .80
Pre
cisi
on
Avg. Pairwise Sequence Identiy
> 30% 20−30% < 20%
Size
: 0−
10
0S
ize: >
10
0S
ize: 0
−1
00
Size
: > 1
00
Size
: 0−
10
0S
ize: >
10
0
Sequence
Length
: Full
Sequence
Length
: 50%
Sequence
Length
: 25%
Recall
Method HIPPI HMMER BLAST HHsearch: 1 Iteration HHsearch: 2 Iterations
1. Pre-processing seed alignments
.
.
.
.
Family A
Family B
Family N
.
.
i) Compute an ML tree on each seed alignment
ii) Build ensemble of HMMs for each seed alignment, using its ML tree
HMM A1
HMM B1
HMM N1
HMM A2
HMM A3
HMM N2
HMM N5
HMM N4
HMM N3
iii) Collect HMMs into database
HMM A1 HMM A2HMM A3
HMM B1
......HMM N1HMM N2
HMM N5
HMM N4HMM N3
2. Classification of query sequences
HMM A1 HMM A2HMM A3
HMM B1
......HMM N1HMM N2
HMM N5
HMM N4HMM N3
HMM A1 = 8.9 HMM A2 = 7.3HMM A3 = 9.4HMM B1 = 5.6......HMM N1 = 4.4HMM N2 = 5.6
HMM N5 = 6.6HMM N4 = 4.7HMM N3 = 4.3
1) Family A = 9.4
8) Family B = 5.6 ......
2) Family N = 6.6 ......
Query sequence
HMM database
i) Score query sequence against all HMMs in database, keeping only scores above inclusion threshold
ii) Rank families by best scoring HMM within family
iii) Assign query sequence to top ranking family
Family A
Query
HMM database
Recommended