Upload
peter-kenny
View
1.525
Download
1
Tags:
Embed Size (px)
DESCRIPTION
This harangue combines critiques of correlation inflation and ligand efficiency metrics. Who will be summoned to the headmasters study?
Citation preview
Data-analytic sins in property-based
molecular design
Peter Kenny
[email protected] | http://fbdd-lit.blogspot.com
TEP = [đˇđđ˘đ đż,đĄ ]đđđđ
đžđ
Target engagement potential (TEP) A basis for molecular design?
Property-based design as search for âsweet spotâ
Correlation
⢠Strong correlation implies good predictivity
â I have observed a correlation so you must use my rule
⢠Multivariate data analysis (e.g. PCA) usually involves transformation to orthogonal basis
⢠Applying cutoffs (e.g. MW restriction) to data can distort correlations
⢠Noise and range limits in data
Quantifying strengths of relationships between continuous variables
⢠Correlation measures
â Pearson product-moment correlation coefficient (R)
â Spearman's rank correlation coefficient ()
â Kendall rank correlation coefficient (Ď)
⢠Quality of fit measures
â Coefficient of determination (R2) is the fraction of the variance in Y that is explained by model
â Root mean square error (RMSE)
Preparation of synthetic data setsKenny & Montanari (2013) JCAMD 27:1-13 DOI
Add Gaussian noise (SD=10) to Y
Correlation inflation by hiding variationSee Hopkins, Mason & Overington (2006) Curr Opin Struct Biol 16:127-136 DOI
Leeson & Springthorpe (2007) NRDD 6:881-890 DOI
Data is naturally binned (X is an integer) and mean value of Y is calculated for each value of X. In some studies, averaged data is only presented graphically and it is left to the reader to judge the strength of the correlation.
R = 0.34 R = 0.30 R = 0.31
R = 0.67 R = 0.93 R = 0.996
r
N 1202
R 0.247 ( 95% CI: 0.193 | 0.299)
0.215 ( P < 0.0001)
0.148 ( P < 0.0001)
N 8
R 0.972 ( 95% CI: 0.846 | 0.995)
0.970 ( P < 0.0001)
0.909 ( P = 0.0018)
Correlation Inflation in FlatlandSee Lovering, Bikker & Humblet (2009) JMC 52:6752-6756 DOI
Masking variation with standard errorSee Gleeson (2008) JMC 51:817-834 DOI
Partition by value of X into 4 bins with equal numbers of data points and display 95% confidence interval for mean (green) and mean Âą SD (blue) for each bin.
R = 0.12 R = 0.29 R = 0.28
N Bins Degrees of Freedom F P
40 4 3 0.2596 0.8540
400 4 3 12.855 < 0.0001
4000 4 3 115.35 < 0.0001
4000 2 1 270.91 < 0.0001
4000 8 7 50.075 < 0.0001
âIn each plot provided, the width of the errors bars and the difference in the mean values of the different categories are indicative of the strength of the relationship between the parameters.â Gleeson (2008) JMC 51:817-834 DOI
The error of standard error
ANOVA for binned data sets
Know your data
⢠Assays are typically run in replicate making it possible to estimate assay variance
⢠Every assay has a finite dynamic range and it may not always be obvious what this is for a particular assay
⢠Dynamic range may have been sacrificed for thoughput but this, by itself, does not make the assay bad
⢠We need to be able analyse in-range and out-of-range data within single unified frameworkâ See Lind (2010) QSAR analysis involving assay results which are only known to
be greater than, or less than some cut-off limit. Mol Inf 29:845-852 DOI
Depicting variation with percentile plots
This graphical representation of data makes it easy to visualize variation and can be used with mixed in-range and out-of-range data. See Colclough et al (2008) BMCL 16:6611-6616 DOI
Binning continuous data restricts your options for analysis and places burden of proof on you to show that your conclusions are independent of the binning scheme. Think before you bin!
Averaging the binned data was
your idea so donât try blaming me this
time!
Correlation inflation: some stuff to think about
⢠Model continuous data as continuous dataâ RMSE is most relevant to prediction but you still need R2
â Fitted parameters may provide insight (e.g. solubility is more sensitive than potency to lipophilicity)
⢠When selecting training data think in terms of Design of Experiments (e.g. evenly spaced values of X)
⢠Try to achieve normally distributed Y (e.g. use pIC50 rather than IC50)⢠Never make statements about the strength of a relationship when
youâve hidden or masked variation in the data (unless you want a starring role in Correlation Inflation 2)
⢠To be meaningful, a measure of the spread of a distribution must be independent of sample size
⢠Reviewers/editors, mercilessly purge manuscripts of statements like, âA negative correlation was observed between X and Yâ or âA and B are correlated/linkedâ
Ligand efficiency metrics (LEMs) considered harmful
⢠We use LEMs to normalize activity with respect to risk factors such as molecular size and lipophilicity
⢠What do we mean by normalization?
⢠We make assumptions about underlying relationship between activity and risk factor(s) when we define an LEM
⢠LEM as measure of extent to which activity beats a trend?
Kenny, LeitĂŁo & Montanari (2014) JCAMD 28:699-710 DOI
Scale activity/affinity by risk factor
LE = ÎG/HA
Offset activity/affinity by risk factor
LipE = pIC50 ClogP
Ligand efficiency metrics
No reason that dependence of activity on risk factor should be restricted to one of these two linear models
Use trend actually observed in data for normalization
rather than some arbitrarily assumed trend
Thereâs a reason why we say standard free energy
of bindingâŚ
DG = DH TDS = RTln(Kd/C0)
⢠Adoption of 1 M as standard concentration is
arbitrary
⢠A view of a chemical system that changes with
the choice of standard concentration is
thermodynamically invalid
NHA Kd/M C/M (1/NHA) log10(Kd/C)
10 10-3 1 0.30
20 10-6 1 0.30
30 10-9 1 0.30
10 10-3 0.1 0.20
20 10-6 0.1 0.25
30 10-9 0.1 0.27
10 10-3 10 0.40
20 10-6 10 0.35
30 10-9 10 0.33
Effect on LE of changing standard concentration
Scaling transformation of parallel lines by dividing Y by X
(This is how ligand efficiency is calculated)
Size dependency of LE is consequence of non-zero intercept
Affinity plotted against molecular weight for minimal binding
elements against various targets in inhibitor deconstruction
study showing variation in intercept term
Hajduk PJ (2006) J Med Chem 49:6972â6976 DOI
Is it valid to combine results from different assays in LE analysis?
Offsetting transformation of lines with different slope and
common intercept by subtracting X from Y
(This is how lipophilic efficiency is calculated)
Thankfully (hopefully?) nobody has âdiscoveredâ
lipophilicity-dependent lipophilic efficiency yet
Linear fit of ÎG for published data set
Mortenson & Murray (2011) JCAMD 25:663-667 DOI
Ligand efficiency, group efficiency and residuals plotted for published data set
Some more stuff to think about
⢠Normalize activity using trend actually observed in data (this means you have to model the data)
⢠Residuals are invariant with respect to choice in standard concentration
⢠Residuals can be used with other functional forms (e.g. non-linear and multi-linear)