Upload
human-variome-project
View
91
Download
1
Embed Size (px)
Citation preview
© 2015 Invitae Corporation. All Rights Reserved. | 1
Improving the accuracy and reproducibility of clinical genetic tests
STEPHEN E. L INCOLN
HVP Meet ingMay 2016
2 © 2015 Invitae Corporation. All Rights Reserved.
1. Analytic Performance– Analytic Sensitivity: Can you find the DNA variants a
patient has?– Analytic Specificity: Of the variants you find, are they
really there?
2. Clinical Performance– Clinical Sensitivity: Can you find the complete
spectrum of clinically relevant variants for a patient’s condition?
– Classification: Can you interpret the medical impact of these variants appropriately?
Questions about any genetic test…
© 2015 Invitae Corporation. All Rights Reserved. | 3
1. Analytical Performance
4 © 2015 Invitae Corporation. All Rights Reserved.
Lions and Tigers and Bears, Oh My!
5 © 2015 Invitae Corporation. All Rights Reserved.
Clinical utility of panel testing for HBOC
Desmond et al., JAMA Oncol. 2015Swisher, JAMA Oncol. 2015
6 © 2015 Invitae Corporation. All Rights Reserved.
Companion technical paper
A systematic comparison of traditional and multigene panel testing for hereditary breast and ovarian cancer in more than 1000 patientsStephen E. Lincoln1, Yuya Kobayashi1, Michael J. Anderson1, Shan Yang1,Andrea J. Desmond2, Meredith A. Mills3, Geoffrey B. Nilsen1, Kevin B. Jacobs1, Federico A. Monzon1, Allison W. Kurian3, James M. Ford3, Leif W. Ellisen2,4
1. Invitae, San Francisco, CA2. Massachusetts General Hospital Cancer Center,
Boston, MA3. Stanford University School of Medicine, Palo Alto,
CA4. Harvard Medical School, Boston, MA Lincoln et al., J Mol Diag 2015
7 © 2015 Invitae Corporation. All Rights Reserved.
A Significant Fraction of Pathogenic Variants in Representative Clinical Cases are Technically Challenging
Pathogenic variants among 1062 clinical cases, by type:
Lincoln et al., J Mol Diag 2015
Small Indel
i.e. CNVs or “Large Rearrangements” (Myriad term)
“Hard stuff” (Steve’s term)
8 © 2015 Invitae Corporation. All Rights Reserved.
Analysis of 30,000 sequential cases with clinical testing across over 1,000 known disease genes (in total)…
Percentage Pathogenic Variant Type2.9% Single-exon del/dups (CNVs)
1.8% Large indel or complex sequence change
5.8% In high-GC, low-complexity, or poorly mapabie regions of the genome
Of the last 5,000 patients with pathogenic findings…
Rebecca Truty, Invitae
9 © 2015 Invitae Corporation. All Rights Reserved.
BRCA2: c.9342_9343insAlu– Novel Alu insertion (at the time we first saw it)– Not the Portuguese founder mutation (that’s c.156_157insAlu)
BRCA2: c.9203_9328del126 – Very large indel (or a very small CNV)
MSH2: c.942+3A>T– Splice-affecting mutation next to 25 bp homopolymer-A
MSH2: inv exons 1-7– Breakpoint detection in intronic regions
CDKN2A: c.9_32dup24– Third copy of a wilt-type tandem duplication in 5’ CpG island
And many other examples...
The spectrum of pathogenic variants in patients includes cases that the typical NGS workflow does not handle
10 © 2015 Invitae Corporation. All Rights Reserved.
Analytic validity in N=1105 individuals
NGS vs. Traditional MethodsIn 1105 Individuals
Sensitivity 100.0%
Specificity 100.0%
750 Comparable Variants (Pathogenic or Otherwise)
Sequence Changes 721
Del/dups (CNVs) 29
Lincoln et al., J Mol Diag 2015
Single Nucleotide 549
Small Indel 156
Large Indel* 13
Complex** 6
To achieve this, specialized NGS methods, biochemical and bioinformatics, are required.
The other challenging classes of variation tend to be not well represented in other validation studies.
* Large Indel is deletion≥10bp, insertion≥5bp
** Complex includes homo-polymer associated variants, indels in low-complexity sequence, short
range haplotypes, etc.
11 © 2015 Invitae Corporation. All Rights Reserved.
NGS vs. Traditional Methods in 250 Patients
Sensitivity 100.0%
Specificity 99.99%
3025 Variants Appropriate to Measure Sensitivity
Sequence Changes 3021
Del/dups (CNVs) 4 ?
Chong et al., PLOS One 2014
Single Nucleotide 3010
Small Indel 11 ?
Large Indel * 0
Complex ** 0* Large Indel is deletion≥10bp, insertion≥5bp** Complex includes homo-polymer associated
variants, indels in low-complexity sequence, short range haplotypes, etc.
Note: These numbers assume that Table 2 in the paper includes all of the indels and del/dups, and that the remaining variants in the are benign SNPs. This is not entirely clear to us in the paper text.
A different lab’s study: Analytic validity in 250 individuals
12 © 2015 Invitae Corporation. All Rights Reserved.
What is one to do?
Build Awareness
Develop/Implement Methods
Improve Validation Resources
Strengthen Validation Standards
© 2015 Invitae Corporation. All Rights Reserved. | 13
2. Clinical Interpretation
18 © 2015 Invitae Corporation. All Rights Reserved.
ClinVar top submitters
www.ncbi.nlm.nih.gov/clinvar > Statistics > Submitter List (as of May 3, 2016)
20 © 2015 Invitae Corporation. All Rights Reserved.
ClinVar entry for BRCA1:c.4410A>T
21 © 2015 Invitae Corporation. All Rights Reserved.
ClinVar entry for BRCA1:c.4410A>T
22 © 2015 Invitae Corporation. All Rights Reserved.
NM_007294.3(BRCA1):c.4410A>T (p.Glu1470Asp)
23 © 2015 Invitae Corporation. All Rights Reserved.
NM_007294.3(BRCA1):c.4410A>T (p.Glu1470Asp)
24 © 2015 Invitae Corporation. All Rights Reserved.
NM_007294.3(BRCA1):c.4410A>T (p.Glu1470Asp)
25 © 2015 Invitae Corporation. All Rights Reserved.
This sequence change replaces glutamic acid with aspartic acid at codon 1470 of the BRCA1 protein (p.Glu1470Asp). The glutamic acid residue is moderately conserved and there is a small physicochemical difference between glutamic acid and aspartic acid. This variant has been published in the literature and is present in population databases (rs80357075, 0.05%). This variant has been reported in individuals affected with breast cancer. Due to limited data, whether this variant segregates with disease remains uncertain (PMID: 20104584, 22682623). This variant has also been reported in a healthy individual not affected with cancer (PMID: 24728327). ClinVar contains entries for this variant (RCV000131557, RCV000077572, RCV000074594, RCV000120257). Algorithms developed to predict the effect of missense changes on protein structure and function (SIFT, PolyPhen-2, Align-GVGD) all suggest that this variant is likely to be tolerated, but these predictions have not been confirmed by published functional studies. In summary, this is a rare missense change that is not predicted to affect protein function or cause disease. However the evidence is insufficient at this time to prove that conclusively. Therefore, it has been classified as a Variant of Uncertain Significance.
27 © 2015 Invitae Corporation. All Rights Reserved.Vail et al., J Community Genet 2015
Finding: 3-14% disagreement rate between databases
(BIC, HGMD, LOVD, UMD, and ClinVar)
28 © 2015 Invitae Corporation. All Rights Reserved.Vail et al., J Community Genet 2015
The disagreement in public databases “precludes their wider use in clinical practice”
Finding: 3-14% disagreement rate between databases
(BIC, HGMD, LOVD, UMD, and ClinVar)
29 © 2015 Invitae Corporation. All Rights Reserved.Vail et al., J Community Genet 2015
The disagreement in public databases “precludes their wider use in clinical practice”
“Public databases [are] fraught with errors”“Interpretation accuracy impossible with public databases”
Analyst Day Presentation, September 2015investor.myriad.com > Events & Presentations > Financial Events
Accessed 2/1/2016
30 © 2015 Invitae Corporation. All Rights Reserved.
This issue can be misrepresented to clinicians…
• Experienced and responsible lab directors never simply copy variant classifications from any public database
• Instead, they critically evaluate underlying evidence and report classifications following established guidelinesQuestion: What is the clinical impact of discordance in public databases?
32 © 2015 Invitae Corporation. All Rights Reserved.
BRCA1/2 variant classification concordance
Positive vs. not positive result for BRCA1/2
Agree 99.8%
Disagree 0.2%
% of patients with one or more VUS in BRCA1/2
Our test 4.1%
Traditional test 3.2%
975 patients with both Myriad Genetics and independent test results. Variants uncovered by the independent test were blindly classified:
• Using only publicly available resources (literature and databases)
• Following a system based on the ACMG/AMP 2015 guidelines
All amended reports provided to clinical sites were incorporated, so the Myriad variant classifications were up to date
Lincoln et al., J Mol Diag 2015
33 © 2015 Invitae Corporation. All Rights Reserved.
Positive vs. not positive results
Pathogenic (P)
Likely Pathogenic(LP)
VUSLikely Benign (LB)
Benign (B)
P Concordant Discordant LP
VUS Discordant Concordant LB
B
Count differences that could substantially change patient
management decisions
34 © 2015 Invitae Corporation. All Rights Reserved.
Inclusion Criteria were ClinVar submitters that:– Are an established and licensed diagnostic laboratory
• or are submitting data from such a lab
– Submitted at least 200 classified variants in BRCA1/2– Most classifications were from the last 5 years
• Using “last evaluation date”, not “submission date” in ClinVar
Thus we excluded data from:– Research projects, software vendors– Smaller labs, consortia (e.g. ENIGMA)– Old sources (e.g. BIC)
• Mostly pre-2007 Myriad Genetics data
Examining the larger data set in ClinVar
Shan Yang, Invitae
35 © 2015 Invitae Corporation. All Rights Reserved.
ClinVar based data set
Shan Yang, Invitae
Name Classified Variants
ComparableVariants
Submitter Name in ClinVar
Ambry 2793 1502 AMBRY GENETICS
Myriad(via SCRP)
2067 1184 SHARING CLINICAL REPORTS PROJECT (SCRP)
Invitae 1479 1082 INVITAE
GeneDx 1214 937 GENEDX
Counsyl 272 256 COUNSYL
CHEO 257 216 MOLECULAR GENETICS DIAGNOSTIC LABORATORY, CHILDREN'S HOSPITAL OF EASTERN ONTARIO
Emory 203 183 EMORY GENETICS LABORATORY
Total 4725 1800 Non-redundant list from the above
Data from ≥2 labs
36 © 2015 Invitae Corporation. All Rights Reserved.
BRCA Exchange
Benedict Paten, Melissa Cline, Molly Zhang, David Haussler, et al., UCSC
www.brcaexchange.org
• Replicated our results • Open-source code to help mine ClinVar
• Integrating many other sources of BRCA1/2 data
37 © 2015 Invitae Corporation. All Rights Reserved.
Ambry Invitae GeneDx Counsyl CHEO Emory
Myriadvia SCRP
98.7%939/951
97.9% - 99.3%
99.2%619/624
98.3% - 99.7%
99.5%569/572
98.6% - 99.9%
99.4%171/172
97.3% - 100%
99.5%139/142
94.5% - 99.4%
97.2%103/106
92.6% - 99.2%
Ambry99.2%860/867
98.4% - 99.6%
99.6%780/783
99.0% - 99.9%
99.6%223/224
97.9% - 100%
98.3%176/179
95.6% - 99.5%
98.8%161/163
96.1% - 99.7%
Invitae99.8%593/594
99.2% - 100%
99.1%214/216
97.1% - 99.8%
98.2%161/164
95.2% - 99.5%
99.3%144/145
96.8% - 100%
GeneDx99.5%221/222
97.9% - 100%
97.9%138/141
94.4% - 99.4%
99.3%149/150
96.9% - 100%
CounsylConcordanceConcordant/All
Conf. Int.
100%82/82
97.0% - 100%
100%105/105
97.6% - 100%
CHEO 98.3%
57/5892.2% - 99.9%
38 © 2015 Invitae Corporation. All Rights Reserved.
Ambry Invitae GeneDx Counsyl CHEO Emory
Myriadvia SCRP
98.7%939/951
97.9% - 99.3%
99.2%619/624
98.3% - 99.7%
99.5%569/572
98.6% - 99.9%
99.4%171/172
97.3% - 100%
99.5%139/142
94.5% - 99.4%
97.2%103/106
92.6% - 99.2%
Ambry99.2%860/867
98.4% - 99.6%
99.6%780/783
99.0% - 99.9%
99.6%223/224
97.9% - 100%
98.3%176/179
95.6% - 99.5%
98.8%161/163
96.1% - 99.7%
Invitae99.8%593/594
99.2% - 100%
99.1%214/216
97.1% - 99.8%
98.2%161/164
95.2% - 99.5%
99.3%144/145
96.8% - 100%
GeneDx99.5%221/222
97.9% - 100%
97.9%138/141
94.4% - 99.4%
99.3%149/150
96.9% - 100%
Counsyl100%82/82
97.0% - 100%
100%105/105
97.6% - 100%
CHEO 98.3%
57/5892.2% - 99.9%
On a per-variant basis:Pairwise concordance: 97.2% -
100.0%Overall concordance: 98.5%
Only 27/1800 variants have a significant classification discordance
between any two reporting laboratories
39 © 2015 Invitae Corporation. All Rights Reserved.
How many individuals does this represent?
≈22,000 patients give ≈1800 non-redundant
variants
Melissa Cline, UCSC
40 © 2015 Invitae Corporation. All Rights Reserved.
Did SCRP data bias results from other labs?
Other lab classifications Concordance
Pre-dating SCRP release 99.0%
Post-dating SCRP release 98.9%
Apparently not
Compare ClinVar submission date of Myriad data via SCRP to evaluation date of same variant from other
labs
41 © 2015 Invitae Corporation. All Rights Reserved.
All of the discordant variants are rare
All of the 27 variants with classification discordances:– Have population allele frequencies ≤ 0.05% in all of:
• ExAC• ESP • 1000 Genomes• Invitae patient database
Only 16.6% of patients carry any such rare variant(s)
Most (98.4%) of these rare variants are completely concordant when seen by more than one laboratory
42 © 2015 Invitae Corporation. All Rights Reserved.
Per-variant and per-patient concordance are different
16.4%
0.2%
83.4%
Patients with 1 or more rare variants, all concordant (16.4%)
Patients with 1 or more rare variants having a discordance (0.2%)
Patients with no rare variants (83.4%)
Steve Lincoln, Invitae and Benedict Paten UCSC
Estimated per-patient concordance 2 orthogonal ways:– From population allele frequencies– From patient database (n≈20,000)
99.8%Concordant
43 © 2015 Invitae Corporation. All Rights Reserved.
Do not over-generalize these results
Discordance in other genes can be higher– e.g. some cardiovascular genes
Discordance from other sources can be higher
44 © 2015 Invitae Corporation. All Rights Reserved.Van Driest et al., JAMA 2016
“There was low concordance in designating SCN5A and KCNH2 variants as pathogenic.In an unselected population, the putatively
pathogenic genetic variants were not associated with an abnormal phenotype.”
45 © 2015 Invitae Corporation. All Rights Reserved.
Discussion
While substantial disagreements in BRCA1/2 are infrequent, they are important
We must resolve differences collaboratively, not competitively, to deliver the best patient care, as is done in other areas of medicine
Labs that maintain data as a proprietary asset can make arbitrary and unverifiable claims regarding interpretation accuracy
Data submission ClinVar facilitates peer review and interlaboratory quality control on a global scale, as exemplified by this study
46 © 2015 Invitae Corporation. All Rights Reserved.
Desmond et al., 2015 (open access) Swisher, 2015 (commentary)
Lincoln et al., 2015 (open access)
Jim Ford Allison Kurian Meredith Mills
Leif Ellisen Andrea Desmond Kristen Shannon
Nadine Tung
Steve Lincoln Shan Yang Yuya Kobayashi Scott Topper Bob Nussbaum
Melissa Cline Molly Zhang David Haussler Benedict Paten
For copy of slides:steve.lincoln
or www.invitae.com