Realistic Synthetic Generation Allows Secure Development

© 2014 MapR Technologies 1© 2014 MapR Technologies

Realistic Synthetic Data Allows Secure Development

Ted Dunning

June 11, 2015

© 2014 MapR Technologies 2

Who am I?

Ted Dunning, Chief Applications Architect MapR Technologies

Email [email protected] [email protected]

Twitter @Ted_Dunning

mailto:[email protected]



Short Books by Ted Dunning & Ellen Friedman

• Published by O’Reilly in 2014 and 2015• For sale from Amazon or O’Reilly• Free e-books currently available courtesy of MapR

http://bit.ly/ebook-real-world-hadoop

http://bit.ly/mapr-tsdb-ebook

http://bit.ly/ebook-anomaly

http://bit.ly/recommendation-ebook



• Open source is great, if developed in the open. Privacy is great, but things have to be private. So what happens when you find an open source bug with private data? How do you even file the bug report? Likewise, how can you develop fraud detection algorithms in academic settings when the training data can't be transported outside a secure perimeter. One answer is really good fake data. Good enough to fool the bug. Good enough to emulate the fraud. I will describe log-synth and several physics based approaches that can do this and tell some real stories about fake data.


The basic idea


Anomaly Detection and Fraud Analytics

• Financial customer wants to identify zero-day attacks

• And advanced persistent threats

• By sophisticated adversaries who don’t use known vectors

• Must keep logs and other data secret– But must also collaborate on detection algorithms


Secure Development is Hard


Secure Development is Hard

Outside collaborators are outside the security perimeter

They can’t see the data and they can’t tune new algorithms to fit reality


How To Make Realistic Data


Parametric Simulation

Parametric matching of failure signatures allows emulation of complex data properties

Matching on KPI’s and failure modes guarantees practical fidelity


Do’s and Don’ts

• Do match the KPI’s and failure modes– Speed– Score distribution– False positive rates versus score

• Don’t try to match the actual data distribution precisely– Good enough is good enough and we want to imitate failures,

not create new life forms– Probably impossible to do precisely– Even if possible, it is vastly harder to match distributions


Methods for Generating Numbers

• Well-known distributions– Uniform, normal, gamma, Poisson– Truncations

• Cumulations– Random walk v1

• Mixture distributions• Hyper-parameters

– Random walk v2


Normal

data = data.frame(x=rnorm(10000), y=rnorm(10000))


Mixture of Normals


Random Walk

y = cumsum(rnorm(10000))


Pick Mean from Multinomial


Random Walk with Variable Standard Deviation

y = cumsum(rt(10000, df=0.9))


Methods for Generating Symbols

• Symbols are really just integers with a dictionary• Well-known distributions

– Multinomial– Dirichlet processes– Rich-get-richer, Pittman-Yor

• Mixture distributions• Hyper-parameters• Lookup tables!!!

– Simple tables– Data table joins for correlated components


Skewed Integers

207 3 203 0 198 7 196 4 195 12 193 10 189 2 187 1 185 13 179 6 178 9 177 5 177 25 174 21 173 8 173 14 170 18

[ {"name":"x", "class":"int", "skew":1}]


Methods for Generating Behaviors

• Use structured data!– Generate user meta-data– Generate list of transactions

• Only flatten if necessary• See Apache Drill for post-processing


Methods for Generating Databases

• Use integers (see previous) as foreign keys• Normalized form implies (approximate) independence of tables



Go get log-synth

https://github.com/tdunning/log-synth


A worked example...


Simulation Setup




Questions?


Thank You

@mapr maprtech

[email protected]@apache.org

Ted Dunning, Chief Application Architect

MapRTechnologies

maprtech

mapr-technologies


Technology

Realistic Synthetic Generation Allows Secure Development