• Home

  • Documents

  • Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv....
88
Kipoi: : model zoo for genomics Žiga Avsec PhD candidate, Technical University of Munich www.gagneurlab.in.tum.de @gagneurlab, @KipoiZoo, @Avsecz

Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

  • Upload
    others

  • View
    2

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

Kipoi: : model zoo for genomics

Žiga AvsecPhD candidate, Technical University of Munich

www.gagneurlab.in.tum.de

@gagneurlab, @KipoiZoo, @Avsecz

Page 2: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

GenomicsACGTGTCAGTAGTTAAGCTAGTAGCTGATCGGTAACGTAGTGCACGTGTCAGTAGTTAAGCTAGTAGCTGATC

3 billion letters (x2) = 1 genome

37 trillion cells

Page 3: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

3

Proteins = main building blocks

Genome

Protein1

Protein1

Protein2

Protein3

Protein3

Protein3

~100k - 1M

Page 4: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

4

Proteins = main building blocks

Genome

Protein1

Protein1

Protein2

Protein3

Protein2

~100k - 1M

Page 5: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

5

Proteins = main building blocks

Genome

Protein1

Protein1

Protein2

Protein3

Protein2

Protein2

Protein1

Pro

tein

2Protein3

~100k - 1M

Protein complex

Function1

Function2

Page 6: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

How to make proteins from the genome?

Page 7: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

7

atcgtatatatcatgatatggatacgcatagatcatgactcaggatacgDNA (2)

aucaugauauggauacgcauagaucaugacuca

Transcription

precursor RNA

Protein (~100,000)

Gene expression: How information in DNA is read out

aucaugauacauagaucaugacuca

Splicing

mature RNA

Translation

Gene (~20,000, 1% of DNA)

Intron Exon

Page 8: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

8

atcgtatatatcatgatatggatacgcatagatcatgactcaggatacg

Transcriptionfactor

TranscriptionFactor binding site

Gene expression: How information in DNA is read out

Page 9: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

9

atcgtatatatcatgatatggatacgcatagatcatgactcaggatacg

RNA polymerase

Gene expression: How information in DNA is read out

Page 10: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

10

atcgtatatatcatgatatggatacgcatagatcatgactcaggatacg

Gene expression: How information in DNA is read out

Page 11: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

11

atcgtatatatcatgatatggatacgcatagatcatgactcaggatacg

Gene expression: How information in DNA is read out

Page 12: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

12

The regulatory elements control:● The position of transcription initiation (what)● The frequency of transcription (how much)

atcgtatatatcatgatatggatacgcatagatcatgactcaggatacg

Gene expression: How information in DNA is read out

Page 13: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

13

atcttatatatcatgatatggatacgcatagatcatgactcaggatacg

Genetic variants can disrupt regulatory elements

ReferencePatient

atcgtatatatcatgatatggatacgcatagatcatgactcaggatacg

Page 14: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

14

aucaugauauggauacgcauagaucaugacuca

aucaugauacauagaucaugacuca

Transcription

Thousands of regulatory elementsacross all steps of gene expression

Splicing

Translation RNA degradation

Ø

Protein degradation

Ø

cttatcacagtgtatatcatgatatggatacgcatagatcatgactcaggatacg

Page 15: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

15

Experimental data

Measuring the regulatory steps via sequencing

Page 16: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

16

Experimental data Predictive models

Learning the regulatory steps

Page 17: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

17

Experimental data Predictive models

Learning the regulatory steps

Page 18: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

18

GATA TAL

cttatcacagtgtatatcatgatatggatacgcatagatcatgactcaggatacg

Detecting regulatory elementswith convolutional neural networks

Page 19: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

19Eraslan*, Avsec* et al Nature Review Genetics 2019 (In press)

Page 20: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

20Eraslan*, Avsec* et al Nature Review Genetics 2019 (In press)

Page 21: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

21Eraslan*, Avsec* et al Nature Review Genetics 2019 (In press)

Page 22: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

22Eraslan*, Avsec* et al Nature Review Genetics 2019 (In press)

Page 23: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

23Eraslan*, Avsec* et al Nature Review Genetics 2019 (In press)

Page 24: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

24Eraslan*, Avsec* et al Nature Review Genetics 2019 (In press)

Page 25: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

25

That’s why we need GPUs in regulatory genomics.

Eraslan*, Avsec* et al Nature Review Genetics 2019 (In press)

Page 26: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

26

Experimental data Predictive models

Learning the regulatory steps

Page 27: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

List of published predictive models

27

- Transcriptional regulation- TF Binding

- PWM Scanning (Jaspar, Cis-BP, MEME)- DeepBind- Improved DeepBind- FactorNet- GERV- DanQ- CKN-seq

- Chromatin- DeepSEA- DeepChrome- Basenji

- DNA methylation- CpGenie- DeepCpG

- DNA Accessibility- Basset

- TSS: - FIDDLE

- Gene-Expression- Basenji- Expecto

- Post-transcriptional regulation- RBP binding

- iDeep- rbp_eclip (Avsec et al)

- miRNA binding- TargetScan- deepMiRGene

- Splicing- MaxEntScan 5’, 3’- Labranchor- HAL- MMSplice- SpliceAI

- mRNA half-life- Cheng et al 2017

- Polyadenylation- APARENT

- Translation- Optimus_5Prime- Cuperus et al 2017

See also: https://github.com/greenelab/deep-review

Page 28: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

List of published predictive models

28

- Transcriptional regulation- TF Binding

- PWM Scanning (Jaspar, Cis-BP, MEME)- DeepBind- Improved DeepBind- FactorNet- GERV- DanQ- CKN-seq

- Chromatin- DeepSEA- DeepChrome- Basenji

- DNA methylation- CpGenie- DeepCpG

- DNA Accessibility- Basset

- TSS: - FIDDLE

- Gene-Expression- Basenji- Expecto

- Post-transcriptional regulation- RBP binding

- iDeep- rbp_eclip (Avsec et al)

- miRNA binding- TargetScan- deepMiRGene

- Splicing- MaxEntScan 5’, 3’- Labranchor- HAL- MMSplice- SpliceAI

- mRNA half-life- Cheng et al 2017

- Polyadenylation- APARENT

- Translation- Optimus_5Prime- Cuperus et al 2017

Can we easily apply these models to new data?

Can we easily re-use these models?

See also: https://github.com/greenelab/deep-review

Page 29: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

29

Lack of standardization leads to poor sharing and re-use of trained predictive models in genomics

Trained predictive models

Code repository Paper supplements Author-maintained web page

Page 30: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

30

Lack of standardization leads to poor sharing and re-use of trained predictive models in genomics

Trained predictive models

Code repository Paper supplements Author-maintained web page

Data

....

Bioinformatics software

Page 31: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

31

The “Findable Accessible Interoperable Reusable” principlenot only for data but also for trained predictive models

Page 32: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

32

Challenges

- Making predictions end-to-end- predict <model> -i input.data -o output.data

Page 33: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

33

Challenges

- Making predictions end-to-end- predict <model> -i input.data -o output.data

- Data heterogeneity (think one paper = one dataset)

Page 34: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

34

Challenges

- Making predictions end-to-end- predict <model> -i input.data -o output.data

- Data heterogeneity (think one paper = one dataset)

- Model heterogeneity (from deep learning frameworks to custom code)- Dependency issues

Page 35: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

35with A. Kundaje, Stanford, and O. Stegle, DKFZ

Kipoi.org [Kípi]

Avsec et al, Nature Biotechnology (In press)

Page 36: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

36

Trained model (model.yaml)

Page 37: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

37

Model

TGATCGAGGGTAGCTAGC

CGTGAGTTT

Output

Model

Input

Parameters

Can be implemented using:

data-loader model

“Parameterized function”

Page 38: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

38

Model data-loader model

Page 39: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

Data-loader data-loader model

chr1 1000 2000chr2 5000 7000

>chr1NNNNNNNNNNNN...

intervals.bed

genome.faresize

extract transform

array([[[1, 0, 0, 0], [0, 1, 0, 0], [0, 0, 1, 0], [1, 0, 0, 0], [ … ]],

[[0, 1, 0, 0], [0, 0, 1, 0], [1, 0, 0, 0], [0, 0, 0, 1], [ … ]]])

github.com/kipoi/kipoiseq

Page 40: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

40

Dependencies data-loader model

Page 41: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

41

Test predictions

...

TGATCGAGGGTAGCTAGC

CGTGAGTTT

TGATCGAGGGTAGCTAGC

CGTGAGTTTTGATCGAGGGTAGCTAGC

CGTGAGTTT

x

Page 42: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

42

Schema

TGATCGAGGA

...

Supports multiple inputs/outputs

Page 43: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

43

General information

Page 44: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

44

Model repository

Page 45: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

45

Page 46: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

46

Page 47: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

47

# model groups: 20

# of models: 2073

Page 48: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

48

- Install new conda environment- Test the pipeline: data -> dataloader -> model- Test the predictions match

Testing models

Page 49: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

49

- Install new conda environment- Test the pipeline: data -> dataloader -> model- Test the predictions match

Testing models

When?- Pull-request

- Nightly (all model groups)

Page 50: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

50

Using models

Page 51: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

For the impatient:30 seconds introduction to Kipoi

Page 52: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

52

Page 53: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

53

Case study 1: Benchmarking models

Page 54: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

54

Benchmarking alternative models

http://kipoi.org
Page 55: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

55

Benchmarking alternative models

http://kipoi.org
Page 56: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

56

Benchmarking alternative models

# Create and activate a new conda environment with# all model dependencies installedkipoi env create <Model>source activate kipoi-<Model># Run model predictionkipoi predict <Model> \ --dataloader_args='{

“intervals_file”: “intervals.bed”, “fasta_file”: “hg38.fa”}' \

-o ‘<Model>.preds.h5'

http://kipoi.org
Page 57: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

57

Benchmarking alternative models

# Create and activate a new conda environment with# all model dependencies installedkipoi env create <Model>source activate kipoi-<Model># Run model predictionkipoi predict <Model> \ --dataloader_args='{

“intervals_file”: “intervals.bed”, “fasta_file”: “hg38.fa”}' \

-o ‘<Model>.preds.h5'

<- Works very nicely with workflow-management tools like Snakemake

http://kipoi.org
Page 58: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

58

# Create and activate a new conda environment with# all model dependencies installedkipoi env create <Model>source activate kipoi-<Model># Run model predictionkipoi predict <Model> \ --dataloader_args='{

“intervals_file”: “intervals.bed”, “fasta_file”: “hg38.fa”}' \

-o ‘<Model>.preds.h5'

Why not a single command?

predict <model> -i input.data -o output.data

Page 59: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

59

# Run model predictionkipoi predict <Model> \ --dataloader_args='{

“intervals_file”: “intervals.bed”, “fasta_file”: “hg38.fa”}' \

-o ‘<Model>.preds.h5' --singularity

Why not a single command?

predict <model> -i input.data -o output.data

Page 60: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

60

# Run model predictionkipoi predict <Model> \ --dataloader_args='{

“intervals_file”: “intervals.bed”, “fasta_file”: “hg38.fa”}' \

-o ‘<Model>.preds.h5' --singularity

Why not a single command?

predict <model> -i input.data -o output.data

input.data

Container

Model output.data

Page 61: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

61

# Run model predictionkipoi predict <Model> \ --dataloader_args='{

“intervals_file”: “intervals.bed”, “fasta_file”: “hg38.fa”}' \

-o ‘<Model>.preds.h5' --singularity

Why not a single command?

predict <model> -i input.data -o output.data

input.data

Container

Model output.data

In-progress:- --docker- --docker-gpu <- Container will be

available on NGC

Page 62: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

62

Transfer learning

Page 63: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

Transfer learning: Adapting existing models to new tasks

Conv. layers

Dense

Dense

Dense

Model with transferred parameters

Pre-trained model

DNA accessibility in 421 cell-types

DNA accessibility in new cell type

Conv. layers

Dense

Dense

Dense Area under thePrecision-recallcurve

Training epoch

See also Kelley et al. Gen. res. 2016

Randomly initialized(>1day)

Transferred (<4h)

Takes a few days to train(Divergent421 model in Kipoi)

http://kipoi.org
Page 64: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

Transfer learning: Adapting existing models to new tasks

Training epoch

Area under thePrecision-recallcurve

See also Kelley et al. Gen. res. 2016

http://kipoi.org
Page 65: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

65

Interpreting models

Page 66: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

66Eraslan*, Avsec* et al NRG 2019 (In press)

Page 67: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

67Eraslan*, Avsec* et al NRG 2019 (In press)

Page 68: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

68Eraslan*, Avsec* et al NRG 2019 (In press)

Page 69: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

69

# Pythonimport kipoifrom kipoi_interpret.importance_scores.gradient import GradientXInput

model = kipoi.get_model("model”)

imp_score = GradientXInput(model)

scores = imp_score.score(seqs)

# CLIkipoi interpret create_mutation_map \ <Model> \ --dataloader_args='{

“intervals_file”: “intervals.bed”, “fasta_file”: “hg38.fa”}' \

-o ‘mmap.h5'

kipoi-interpret

Page 70: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

70

Feature importance score plugin● Variant rs35703285, near the beta globin gene HBB, is pathogenic (ClinVar) and

linked to Beta thalassemia

http://kipoi.org
Page 71: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

71

Feature importance score plugin● Variant rs35703285, near the beta globin gene HBB, is pathogenic (ClinVar) and

linked to Beta thalassemia

Methods- ISM- grad- input*grad- DeepLift

http://kipoi.org
Page 72: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

72

Scoring genetic variants

Page 73: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

73

atcttatatatcatgatatggatacgcatagatcatgactcaggatacg

Scoring genetic variants

ReferencePatient

atcgtatatatcatgatatggatacgcatagatcatgactcaggatacg

#CHROM POS ID REF ALT …chr22 41320486 . G T …

- Each one of us carries ca 1,000,000 variants

Page 74: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

74

Variant effect prediction plugin

● In-silico mutagenesis

http://kipoi.org
Page 75: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

75

Variant effect prediction plugin

# Annotate VCF file with variant scoreskipoi veff score_variants <Model> \ --dataloader_args='{

“fasta_file”: “hg38.fa”}' \ --vcf_path 'input.vcf' \ -o ‘annotated.vcf'

● Supported by 12/20 model groups, runnable on VCF files

● In-silico mutagenesis

http://kipoi.org
Page 76: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

76

Kipoi variant scoring as a DNAnexus applet

Page 77: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

77

atcgtatatatcatgatatggatacgcatagatcatgactcaggatacg

aucaugauauggauacgcauagaucaugacuca

Splicing: an essential step of protein production

aucaugauacauagaucaugacuca

Splicing

Page 78: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

78

atcgtatatatcatgatatggatactcatagatcatgactcaggatacg

aucaugauauggauactcauagaucaugacuca

Splicing: an essential step of protein production

aucaugauacauataucaugacuca

Splicing

Page 79: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

79

Scotti & Swanson, 2016 NRG

Splicing is a complex, multi-step process

Page 80: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

80

Different models model different regions

Donor AcceptorBranchpoint

MaxEntScan/3prime MaxEntScan/5primeHAL

labranchor

Page 81: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

81

Different models model different regions

Donor AcceptorBranchpoint

MaxEntScan/3prime MaxEntScan/5primeHAL

labranchor

Binary classification:- Pathogenic (ClinVar)- Benign (ClinVar)

Page 82: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

Ensemble model predicting pathogenic variants near splice sites

See also MMSplice from Cheng et al., Genome Biology, CAGI Splicing challenge 2018 winner

Kipoi models:KipoiSplice/4KipoiSplice/4consMMSplice

Page 83: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

Summary

83

Page 84: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

84

Experimental data Predictive models

Learning the regulatory steps

Page 85: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

85with A. Kundaje, Stanford, and O. Stegle, DKFZ

Kipoi.org [Kípi]

Avsec et al, Nature Biotechnology (In press)

Page 86: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

86

Kundaje lab Stanford● Anshul Kundaje● Avanti Shrikumar● Nancy Xu● Abhimanyu Banerjee● Chuan Sheng Foo

Gagneur lab TU Munich● Julien Gagneur● Jun Cheng

Stegle lab Cambridge EMBL-EBI● Oliver Stegle● Roman Kreuzhuber● Thorsten Beider● Lara Urban

Acknowledgements.org@KipoiZoo

Roman Kreuzhuber

Nvidia

● Jonny Israeli● Fernanda Foertter● Gary Dunn● Adam Simpson

Thorsten Beider

DNA Nexus

● Jason Chin● Maria Simbirsky● Andrew Carroll

Page 87: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

Kipoi: : model zoo for genomics

Žiga AvsecPhD candidate, Technical University of Munich

www.gagneurlab.in.tum.de

@gagneurlab, @KipoiZoo, @Avsecz

Page 88: Kipoi: : model zoo for genomics · Transfer learning: Adapting existing models to new tasks Conv. layers Dense Dense Dense Model with transferred parameters Pre-trained model DNA

88


Computing with matrix groups, or "how dense is dense"

Computing with matrix groups, or "how dense is dense"

Science
Chapter 5 “Electrons in Atoms” Chemistry. Ernest Rutherford’s Model Discovered dense positive piece at the center of the atom- “nucleus” Discovered dense

Chapter 5 “Electrons in Atoms” Chemistry. Ernest Rutherford’s Model Discovered dense positive piece at the center of the atom- “nucleus” Discovered dense

Documents
Hot Dense Matter in The Quark-Meson-Coupling …Hot Dense Matter in The Quark-Meson-Coupling Model (QMC): Equation of State and Composition of Proto-Neutron Stars. J. R. Stone1;2,

Hot Dense Matter in The Quark-Meson-Coupling …Hot Dense Matter in The Quark-Meson-Coupling Model (QMC): Equation of State and Composition of Proto-Neutron Stars. J. R. Stone1;2,

Documents
Dense Poly

Dense Poly

Documents
dense 3D.pdf

dense 3D.pdf

Documents
BRICK Dense/Medium Dense Coursing Bricks and Commons€¦ · Dense/Medium Dense Coursing Bricks and Commons Product Information BRICK Dense/Medium Dense Coursing Bricks and Commons

BRICK Dense/Medium Dense Coursing Bricks and Commons€¦ · Dense/Medium Dense Coursing Bricks and Commons Product Information BRICK Dense/Medium Dense Coursing Bricks and Commons

Documents
A Dense Gas Model of Combined Transports and Distributions ... · a compartment model for uid ow through the interstitium. The model is inspired by a similar Starling model in an

A Dense Gas Model of Combined Transports and Distributions ... · a compartment model for uid ow through the interstitium. The model is inspired by a similar Starling model in an

Documents
Processing Update – Kipoi Operation, DRC

Processing Update – Kipoi Operation, DRC

Documents
Gurobi Optimizer Example Tour · • dense - Solves a model stored using dense matrices. We don’t recommend using dense We don’t recommend using dense matrices, but this example

Gurobi Optimizer Example Tour · • dense - Solves a model stored using dense matrices. We don’t recommend using dense We don’t recommend using dense matrices, but this example

Documents
A Stochastic Performance Model For Dense Vehicular Ad-Hoc ... · A Stochastic Performance Model For Dense Vehicular Ad-Hoc Networks Thomas Blazek 1, Taulant Berisha , Edon Gashi 2,

A Stochastic Performance Model For Dense Vehicular Ad-Hoc ... · A Stochastic Performance Model For Dense Vehicular Ad-Hoc Networks Thomas Blazek 1, Taulant Berisha , Edon Gashi 2,

Documents
Effectively Dense Boolean Algebras and Their …nies/papers_till_09...2. EFFECTIVELY DENSE BOOLEAN ALGEBRAS As in [19], a c.e. Boolean algebras is given by a model (N,

Effectively Dense Boolean Algebras and Their …nies/papers_till_09...2. EFFECTIVELY DENSE BOOLEAN ALGEBRAS As in [19], a c.e. Boolean algebras is given by a model (N,

Documents
Discrete dipole model of radiative transfer in dense

Discrete dipole model of radiative transfer in dense

Documents
Photon storage in -type optically dense atomic media. II. Free-space … · 2015. 1. 20. · Photon storage in -type optically dense atomic media. II. Free-space model Alexey V. Gorshkov,1

Photon storage in -type optically dense atomic media. II. Free-space … · 2015. 1. 20. · Photon storage in -type optically dense atomic media. II. Free-space model Alexey V. Gorshkov,1

Documents
Semi-analytical 6D model of space charge force for dense …/67531/metadc736453/... · Semi-analytical 6D model of space charge force for dense electron bunches with a large energy

Semi-analytical 6D model of space charge force for dense …/67531/metadc736453/... · Semi-analytical 6D model of space charge force for dense electron bunches with a large energy

Documents
The Quark-Meson Coupling model as a description of dense matter

The Quark-Meson Coupling model as a description of dense matter

Documents
Material Datasheet DENSE AND MEDIUM DENSE SOLID …

Material Datasheet DENSE AND MEDIUM DENSE SOLID …

Documents
(Crowdsourced) Image-based 3D Modeling - microsoft.com · 3D Modeling Pipeline 13 Images Scene Graph Sparse Model Dense Model Sparse Reconstruction Initialization Triangulation Bundle

(Crowdsourced) Image-based 3D Modeling - microsoft.com · 3D Modeling Pipeline 13 Images Scene Graph Sparse Model Dense Model Sparse Reconstruction Initialization Triangulation Bundle

Documents
Extended Hard-Sphere Model for the Viscosity of Dense … · Extended Hard-Sphere Model for the Viscosity of Dense Fluids . ... model of Dymond and Assael and uses the roughness factors

Extended Hard-Sphere Model for the Viscosity of Dense … · Extended Hard-Sphere Model for the Viscosity of Dense Fluids . ... model of Dymond and Assael and uses the roughness factors

Documents
Dense Linear Algebra - courses.csail.mit.educourses.csail.mit.edu/18.337/2006/book/Lecture_04-Dense_Linear_Al… · Dense Linear Algebra 4.1 Dense Matrices We next look at dense linear

Dense Linear Algebra - courses.csail.mit.educourses.csail.mit.edu/18.337/2006/book/Lecture_04-Dense_Linear_Al… · Dense Linear Algebra 4.1 Dense Matrices We next look at dense linear

Documents
FOR THE DEGADIS 2.1 DENSE GAS DISPERSION MODEL

FOR THE DEGADIS 2.1 DENSE GAS DISPERSION MODEL

Documents
A Structural Model of Dense Network Formationmeleangelo.com/bayesnet_march2016.pdfA Structural Model of Dense Network Formation ... exercise I focus on the region of parameters with

A Structural Model of Dense Network Formationmeleangelo.com/bayesnet_march2016.pdfA Structural Model of Dense Network Formation ... exercise I focus on the region of parameters with

Documents
Stochastic Channel Model for Dense Urban Line-of … · Stochastic Channel Model for Dense Urban Line-of-Sight Street Microcell Mir Ghoraishi 214, ... be inside the building zone

Stochastic Channel Model for Dense Urban Line-of … · Stochastic Channel Model for Dense Urban Line-of-Sight Street Microcell Mir Ghoraishi 214, ... be inside the building zone

Documents
Dense gas dispersion model development and testing for the

Dense gas dispersion model development and testing for the

Documents
Dense Extreme Inception Network: Towards a Robust CNN ... › content_WACV_2020 › papers › Pom… · Dense Extreme Inception Network: Towards a Robust CNN Model for Edge Detection

Dense Extreme Inception Network: Towards a Robust CNN ... › content_WACV_2020 › papers › Pom… · Dense Extreme Inception Network: Towards a Robust CNN Model for Edge Detection

Documents
Multimi Dense

Multimi Dense

Documents
Quantum Mechanical Model of the Atom...Quantum Mechanical Model or Wave model •Small, dense, positively charged nucleus surrounded by electron clouds of probability. Does not define

Quantum Mechanical Model of the Atom...Quantum Mechanical Model or Wave model •Small, dense, positively charged nucleus surrounded by electron clouds of probability. Does not define

Documents
Dense Array EEG & Epilepsy - IntechOpen...Dense Array EEG & Epilepsy 155 Fig. 1. This model is wearing the 256 channe l dense array EEG net. Note the electrode coverage extending over

Dense Array EEG & Epilepsy - IntechOpen...Dense Array EEG & Epilepsy 155 Fig. 1. This model is wearing the 256 channe l dense array EEG net. Note the electrode coverage extending over

Documents
2008 International ANSYS Conference · 2008 International ANSYS Conference ... –Macroscopic Particle Model (MPM) ... dense gas-solid flows where a stochastic model for particle-particle

2008 International ANSYS Conference · 2008 International ANSYS Conference ... –Macroscopic Particle Model (MPM) ... dense gas-solid flows where a stochastic model for particle-particle

Documents
Implementation and Refinement of a Comprehensive Model … Library/Events/2015/crosscutting... · Implementation and Refinement of a Comprehensive Model for Dense Granular Flows

Implementation and Refinement of a Comprehensive Model … Library/Events/2015/crosscutting... · Implementation and Refinement of a Comprehensive Model for Dense Granular Flows

Documents
A kinetic model for collisional effects in dense adsorbed ......A kinetic model for collisional effects in dense adsorbed gas layers Paolo Barbante∗, Aldo Frezzotti∗, Livio Gibelli∗

A kinetic model for collisional effects in dense adsorbed ......A kinetic model for collisional effects in dense adsorbed gas layers Paolo Barbante∗, Aldo Frezzotti∗, Livio Gibelli∗

Documents