Deep Learning for Anomaly Detection in Manufacturing · • Western Electric or Nelson rules • Automated Alerting Individual – Moving Range Control Charts • Suppose we measured

David Katz, Data Scientist, TIBCO Software

Mike Alperin, Manufacturing Industry Consultant, TIBCO Software

July, 2018

Deep Learning for Anomaly Detection in Manufacturing

© Copyright 2000-2018 TIBCO Software Inc.

2

•Anomaly Detection in Manufacturing•Deep Learning Autoencoder• Neural Networks• Autoencoders• Software Tools

•Virtual Demo

Agenda


3

Anomaly Detection in Manufacturing


4

• Detecting new problems• Supervised vs. Unsupervised Learning

• Some Types of Anomalies• Facilities Equipment - sensor & environmental data • Process Equipment – sensor FDC data• Process Results – Process history and measurements• Physical Defects – defect images & characteristics• Device and Product – PCM and Sort data

• A General Method1. Detect anomalies2. Cluster them3. Classify with fingerprints or signatures 4. Determine causes of anomaly classes 5. Develop Action Plans to address causes 6. Predict cluster for new material and intervene to mitigate potential problems

Anomaly Detection in Manufacturing


5

Univariate Statistical Process Control


Detect changes from baseline –one variable at a time

Shewhart Process Control Charts• Statistically derived Control Limits• Western Electric or Nelson rules

• Automated Alerting

Individual – Moving Range Control Charts

• Suppose we measured 2 parameters y1 and y2 (e.g., person’s height & 1/weight)• Univariate charts would not detect some obvious outliers• This happens in many real applications

The Power of Multivariate Control Charts

Bad Tester


7

Univariate & Multivariate Methods


8

Real-time equipment anomaly prediction & clustering


High Tech Manufacturing Accelerator

https://community.tibco.com/modules/high-tech-manufacturing-accelerator

M

6

https://community.tibco.com/modules/high-tech-manufacturing-accelerator

9

Deep Learning Autoencoder


10

• Frank Rosenblatt, Cornell, inventor of the Perceptron.• Brain mechanisms and models.

• Why the explosion?• New algorithms and techniques

• Convolutional NN, Recursive NN, Generative Adversarial NNs

• New Hardware capabilities• GPU

• Multicore

• Clusters

• More Data

• New Tools from the Open Source world.

From Neural Networks to Deep Learning


cs231n.github.io/neural-networks-1

11

• Create an identity transformation with constraints• Analogy to Principal Components – but much more

flexible/accurate.• Anomalies – the output is the reconstructed input, but it

does not fully match the original input => Reconstruction Error• Reconstruction Error:

• By component

• By sample.

Autoencoders


12

• Fraud

• Credit• Natural Language

• Speech

• Video

• Manufacturing


AbstractSmart manufacturing refers to using advanced data analytics to complement physical science for improving system performance and decision making. With the widespread deployment of sensors and Internet of Things, there is an …

Autoencoder Applications

13

•Sparse Autoencoders•Denoising Autoencoders•Generative Adversarial Networks•Variational Autoencoders

Autoencoders – Types and Variants


14

• H2O DeepLearning• Simple Structure of networks – just specify number of fully-

connected layers (and optionally dropout)• Settings for Sparse data can outperform GPU.• H2O Deep Water Project –

• uses GPU but no longer being developed;• H2O recommends Keras for new projects.

• Keras • Front end for Tensorflow, CNTK, Theano, MXNet• Specify complex network topologies• Use different types of layers – CNN, RNN,…• Can leverage GPU

Deep Learning Software


15

Virtual Demo


Industrial Plant: Raw Time series Data






19

Tag Training, Validation & Test Data Sets


Variable Selection


Model Configuration & Evaluation

Validation Error has clear minimumNote Distribution of Reconstruction Error


Problems converging


Problems Converging




Anomalies & Component Signatures







Incident not detected on Univariate Chart


Reconstruction Error

Identify Incidents Programmatically


Elbow Plot of Cluster Config


Cluster Similar Incidents, View Signatures


31

Visit the TIBCO Industry 4.0page

To Learn & Do More


Visit the TIBCO Community Manufacturing Solutions page

Download AI & Machine Learning Manufacturing Solutions from the

TIBCO Exchange

https://www.tibco.com/solutions/industry-40

https://community.tibco.com/wiki/tibco-spotfire-and-tibco-streambase-manufacturing-solutions

https://community.tibco.com/exchange/tags/manufacturing-industry-13931

32

• Thanks to Dr. Thomas Hill, Venkata Jagannath, Glenn Hoskins and Nico Rode for their contributions to this work

• Thanks to Michael O’Connell, Steven Hillion and HeleenSnelting for their support and encouragement.

Acknowledgments


Contacts:Mike Alperin

Manufacturing Industry [email protected]

David KatzData Scientist

[email protected]

Thank you!

Visit us in Booth 1021See this and other Manufacturing Demos

Learn more about the Technology


mailto:[email protected]

mailto:[email protected]

Documents

Deep Learning for Anomaly Detection in Manufacturing · • Western Electric or Nelson rules • Automated Alerting Individual – Moving Range Control Charts • Suppose we measured