Embed Size (px)
Citation preview
Industrial AI
Deep Learning for Predictive Maintenance, Predictive Quality &
Visual Inspection
We develop prize-winningIndustrial AI
Simon Stiebellehner
Head of AI @ craftworksLecturer @ WU Wien & FH Wien
craftworks.AI
Daniel Ressi
Data Scientist @ craftworks
Who is ?
16 Team membersBiomedical Engineering
User
Experience
Software
Engineering
Physics
Machine Learning
craftworks.AI
We develop prize-winningIndustrial AI
craftworks.at
Selection of our Clients
Predictive Quality
Predictive Analytics Predictive Analytics
Predictive Maintenance
Predictive Quality
Predictive Quality
Computer Vision Big Data InfrastructureSoftware Solution
Predictive Maintenance
A Special Set of Challenges
The Industry
Research & Design
Process Optimization & Automatization
Quality Control
Maintenance
Consumption
Logistics
Customer Engagement
Resource Management
Security
Special Challenges require a Special Set of Methods
Industrial AI
Industrial Challenges
Artificial Intelligence
“Application of machine and deep learning methods to solve industrial challenges”
Characteristics
Industrial AI
● Data○ High-frequency○ Big Data○ Lack of Standardization○ Imbalanced Classes
● Algorithms○ No off-the-shelf solutions○ Reliability Requirements○ Interpretability
Industrial AI
Deep Learning for Predictive Maintenance, Predictive Quality & Visual Inspection
Predictive Maintenance
Visual Inspection
Predictive Quality
Predictive MaintenanceWhat is it?
“Predict and forecast the condition of machines to plan maintenance ahead of time”
Predictive Maintenance
Why?
Predictive Maintenance
● Machine Faults ○ Can stop whole production○ High repair costs○ Safety risk
● Predictive Maintenance1. Machine health forecast2. Planned maintenance schedules3. Cost and time efficient work
Predictive Maintenance
Proactive
Data
Predictive Maintenance
Ground Truth Issue● Start of faults● Unnoticed faults● Few instances of
breakdowns
Maintenance
Records
time time
sen
sor
valu
e
sen
sor
valu
e
Predictive Maintenance
● Time series● Machine centric○ Temperature○ Pressure○ Vibration○ Acoustic, ....
Sensors
Predicting Faults in the District Heating System of Vienna
District Heating Vienna
Predictive Maintenance
Vienna has one of the largest district heating systems in EU
Several thousand converter stations
Checks and maintenance on a daily basis
Every warning has to be investigated
Interrupts schedules
Predictive Maintenance
Goal
Predictive Maintenance
● Predict faults in advanceBonus: determine exact type of fault
Why?Enable more targeted maintenance schedulesReduce interruption of work caused bylow-impact faults & issues
Predictive Maintenance
Predictive Maintenance
2-Stage-Solution
Predictive Maintenance
Recurrent Neural Network (RNN)
Predict fault typeGradient BoostedDecision Trees
Fault probability > threshold
Predicts probability of fault in the next X days (binary classification)
No Fault Fault Type
1
2
Predictive MaintenanceRecurrent Neural Network (RNN)
Stacked LSTM
Input(sensor data of last Y hours)
0
1
No Fault
Fault
Output(fault probability in the next X days)
Predictive Maintenance
91% of Faults can be detected 7 days in advance
Predictive Maintenance
Predictive Maintenance
Stat
us
Predicted Faults
Actual Faults
Time
Status: 0 = normal, 1 = Fault
Can the exact fault types be determined?
Predictive Maintenance
Predictive Maintenance ● several thousands unique fault types
○ Each fault type consists of 3 keywords○ High priority types form clusters
● Fault types were clustered into coherent groups● Selected 2 most high priority groups for classification
task
98% AccuracyFault Type Group
High Prio. Group 1
High Prio. Group 2
Low Priority Group
Labels
Industrial AI
Deep Learning for Predictive Maintenance, Predictive Quality & Visual Inspection
Predictive Maintenance
Visual Inspection
Predictive Quality
Visual Inspection
VisualInspection
What is it?
“Method of quality control using human vision”
Visual Inspection
VisualInspection
Automation is Wide-Spread
Visual Inspection
VisualInspection
Automated Visual Inspection is a Multi-Step Process
Visual Inspection:Detecting and Explaining Defects in Industrial
Parts
Industrial PartGlued PatchesVisual
Inspection
Visual InspectionDetecting and Explaining Defects in Industrial Parts
Defect A
Defect B
Defect C
VisualInspection
Visual InspectionDetecting and Explaining Defects in Industrial Parts
● Self-Learning System
● Perfect Classification in OK / Defect● Best Possible Classification of Type of Defect
GoalsVisual
Inspection
Visual InspectionDetecting and Explaining Defects in Industrial Parts
● Inspection Capabilities
● Explanation of Model Output
● Feedback to ensure Continuous Learning
Depth
Brightness
VisualInspection
Visual InspectionImaging Device captures two Types of Images
12,000
51918
22
327 152
Unlabelled Labelled
OK
Defect C
Defect B
Defect A
Vast majority of unlabelled data is
OK
VisualInspection
Visual InspectionHighly Imbalanced Class Frequencies
12,000
519
Unlabelled Labelled
ChallengeVery Few Labelled Images
VisualInspection
Visual InspectionData contains very few Labelled Images
Minimize Difference between Input and Output
VisualInspection
Visual InspectionDeep Convolutional Autoencoder
Reconstruction ErrorVisualInspection
Visual InspectionVisualizing Anomalies using Reconstruction Error
VisualInspection
Visual InspectionVisualizing Anomalies using Reconstruction Error
OK
Defect A
Defect B
Defect C
Latent SpaceRepresentation
ReconstructionError
Classifier
519
Labelled
VisualInspection
Visual InspectionVisualizing Anomalies using Reconstruction Error
Industrial Augmented IntelligenceAugmented Intelligence in Action
Industrial AI
Deep Learning for Predictive Maintenance, Predictive Quality & Visual Inspection
Predictive Maintenance
Visual Inspection
Predictive Quality
Predictive Quality
What is it?
“Estimating the output quality of a manufacturing process from start to finish”
Predictive Quality
What? Why?
Predictive Quality
● Automatic, Continuous, Proactive○ Predict (future) quality at every process stage○ Identify potential influences on quality
● Saves time, costs and resources○ Prevent quality issues
■ Reduce scrap → minimize resource consumption■ Know when a bad quality output is produced
BEFORE it is produced
Predictive Quality
Data Sources
● Vibration● Temperatu
re● Pressure● ...
Sensors
● RGB● Laser
scans● Infrared
images
Images ● Quality checks (ground truth)
● Shift schedules
● Maintenance records
● Fault logs
Meta-data
Predictive Quality
Predictive Quality
Predictive Quality
Predictive QualityProduction Line
Process A
Process B
Material
Material
FinishedProduct
Product Defects:● Defect 1● Defect 2● Defect 3
...
Process Parameters
Material
Meta-data F
Data source
ImagingE
Predictive Quality
Predictive QualityTypical Problem
● Production line: ○ Sequence of processes / stages○ Each process is sequence of steps itself ○ High volume of process data
● Random Quality checks○ Small number of labelled data
● Goal:○ Predict quality accurately
Miss-match
Predicting Quality in Industrial Production Lines
Predictive Quality
Predictive QualitySolution
1. Train LSTM Autoencoder for each process○ Feature generator (Latent Space Representations)○ Aggregate processes on (super-)process-level
2. Train Convolutional Autoencoder for image data3. Concatenate features4. Train supervised Gradient Boosting Model on
labelled data5. Use SHAP to explain influence of different stages on
product quality
Predictive Quality
Predictive QualitySolution
Process A
Process B
ProcessC
Process D
ImagingE
Meta-data F
Gradient Boosting Model
Con
cat
Labels
Prediction
Deep LSTM Autoencoder
Deep Convolutional Autoencoder
Latent Space Representation
Predictive Quality
Predictive QualityResult
1. Good performance (depending on data quality)
2. Overcomes problem of small labelled data size
3. Overcomes black-box problem○ Explainable on process level○ Analysis of autoencoder reconstruction errors for fine
grained insights within process
Predictive Quality
Predictive QualitySHAP Feature Importances
Predictive Quality
Predictive QualitySHAP Single Prediction
Fault 1 Fault 2 Fault 3
Prediction 0.14 0.71 0.14
SHAP Force Plot for Fault 2
SHAP Force Plot for Fault 3
Industrial AI
Deep Learning for Predictive Maintenance, Predictive Quality & Visual Inspection
Predictive Maintenance
Visual Inspection
Predictive Quality
Learnings
Start simple & don’t over-engineer details
Implement complete workflow asap (REST API, ...)
Ensembling & combination of methods wins
Keep business metrics in mind
Industrial AI requires ML all-arounders
We develop prize-winningIndustrial AI
Simon Stiebellehner
Head of AI @ craftworksLecturer @ WU Wien & FH Wien
craftworks.AI
Daniel Ressi
Data Scientist @ craftworks
Join us!
craftworks.at/working-at-craftworks/
Research with us!
Computer Vision Engineer / Data Scientist
(Senior) Web Developer
Marketeer/Growth Hacker
Agile Project Manager
No fit for you? Send us your application anyway - we are always looking for talent.
Are you a student?
We offer exciting research internships and thesis topics!
Currently, topics are available in the areas ...
Reinforcement Learning
UnsupervisedDeep Learning
