Embed Size (px)
Citation preview
Reluctant to analyze manufacturing data ◦ Requests to analyze plant data (circa 2000)
◦ Influence of J. S. Hunter … PARC Analysis … skepticism
Why am I seeing useful relationships? ◦ The bad news & the good news
◦ Bad news: Processes are “out of control”
◦ Good news: Lots of strong signals in the data
◦ Bad news: Data assembly is painful … so no one is looking at it
◦ Good news: LOTS of low hanging fruit!!! Big opportunity!!!
Probability of successful discovery higher than I anticipated
High return on investment -> increased demand
Realization: Mining mfg data has perils … mitigated by JMP®
Case studies will highlight perils related to: ◦ Data: Time scale issues … manufacturing specific
Data integration with non-matching timestamps
Integration of static & time-based data
◦ Known relationships UNKNOWN TO YOU! … value of visualization!
◦ Prediction vs Control … the need for guidance in tree building!
PhaseTwo Analytics
Data Mining Would be Great … if it weren’t for the Data
70-80% of time is spent wrestling with data issues ◦ Data Access … multiple, disparate databases
◦ Completeness … having all the right data … observability!
◦ Data cleaning
◦ Missing data
◦ Data integration & alignment … joining, concatenating, stacking
◦ Time scale differences
Aligning datasets with non-matching time stamps
Integrating Static data + time-based data (batch processes)
PhaseTwo Analytics
A rolling process produces coils “about” 1 every 20 minutes
The rolling process timestamp reflects the start of coil processing
The rolling process dataset contains process variables, process performance metrics & product quality
Lubricant, sprayed on rolls and the coil, is influential … but quantification of effects is needed
Problem: Lubricant chemistry data are measured less frequently
Lube timestamp will not match the coil process timestamp but it varies smoothly
Alignment is required to model results as a function of both process and chemistry
Here’s a way of aligning data with non-matching timestamps in JMP® using spline predictions
PhaseTwo Analytics
Objective: Align variable LubeChem1 ( measured 1/day ) with Rolling Process data measured 3/hr
LubeChem1 raw data flexible spline fit
Next: Save the prediction formula
Note: use the same name for the time stamp as in the rolling data set
Create a new variable in the rolling data set ( LubeChem1_Pred )
Use the spline prediction formula as the formula for LubeChem1_Pred
Here are the aligned (predicted) LubeChem1 data:
PhaseTwo Analytics
An Ingot Casting Example ◦ Static Data (1/cast) … static conditions … single point meas
Holding furnace temp
Coolant Additive %
Grain refiner set point
Drop rate @ cast start
Steady state drop rate
Ingot head ml set point
Metal treatment set point
Ambient dew point,
% humidity
Temperature
◦ Time-based Data … per cast trajectories … about 1,000 data points per cast Filter Temperature
Cast trough temp
Coolant temperature
Coolant flow
Dist Trough Metal level
Mold level (5)
Mold Controller(5)
Casting rate
How do we combine the data to model the impact of all of these data on some response???
PhaseTwo Analytics
Tim
e-based v
ari
able
s s
uch a
s:
Sta
tic v
ari
able
s s
uch a
s:
An Ingot Casting Example
◦ Static Data (1/cast) Holding furnace temp
Coolant Additive %
Grain refiner set point
Start drop rate
Steady state drop rate
Ingot head ml set point
Metal treatment set point
Ambient dewpoint,
% humidity
Temperature
◦ Time-based Data (cast trajectories)
FilterTemperature
Cast temp
Coolant temperature
Coolant flow
Dist Trough Metal level
Mold level (5)
Mold Controller(5)
Integrating Static Data (1/batch) with Time-based batch trajectories
PhaseTwo Analytics
Filte
r te
mp
C
ast
Tem
p
Co
ola
nt te
mp
M
old
Level
1000 data points but not 1000 pieces of information!!!
An Ingot Casting Example
◦ Static Data (1/drop)
Holding furnace temp X1
Coolant Additive % X2
Grain refiner set point X3
Start drop rate X4
Steady state drop rate X5
Ingot head ml set point X6
Metal treatment set point X7
Ambient dewpoint, X8
% humidity X9
Temperature X10
◦ Time-based trajectory features
Filter Temperature F1, F2, F3, F4
Cast temp F5, F6, F7
Coolant temperature F8, F9
Mold level F10, F11, F12, F13
Coolant flow etc
Dist Trough Metal level etc
Mold Controller(5) etc
Integrating Static Data (1/batch) with Time-based batch trajectories through feature extraction
MaxCASTTemp( L>4)
F6
MinCASTTemp( L>4)
F7
MaxCASTTemp( L<4)
F5
MaxFiltTemp( L<4)
F1
MaxFiltTemp( L>4
F2
Init_FiltTemp( L<4)
F3
MinFiltTemp( L>4)
F4
PhaseTwo Analytics
MaxCoolTemp
F9 MinCoolTemp
F8
Lvl Slope1
F12
Lvl Slope2
F11
Lvl Max
F10
Lvl Final
F13
Filte
r te
mp
C
ast
Tem
p
Co
ola
nt te
mp
M
old
Level
Model Y = F(X1, X2, …X10, F1, F2,…,Fi)
“You Can Observe a Lot Just By Watching ” … Yogi Berra
Clean, merge, align, characterize your data … THEN EXPLORE IT!!!
Fit Y by X, time plots, Scatterplots, Graph builder, PCA
Case Study 3: Hidden Effects of Lubrication on Extrusion Process Friction
◦ Parts are extruded through a die and lubricant is sprayed on the part as it enters the die. Friction variation can occur and cause part surface damage & scrap. Lube components are measured but are difficult to control tightly. Lube effects are suspected qualitatively but never quantified
◦ A plant study was conducted. Key lube components were set at specific levels for periods of time with changes made from time period to time period.
◦ Data were “thrown over the fence” for analysis … ~9000 obs., ~30 predictors
◦ Research question #1: Do we see significant changes in mean measured die friction from time period to time period?
◦ Naïve answer (based on a Fit Y by X of the response versus test periods: Not much!
◦ Unusually high level of variability
But wait! There’s More! PhaseTwo Analytics
Tool Fri
cti
on
Closer Inspection/Visualization of The Data Reveals More
Plot of the response vs Time
Magnification reveals a repeating systematic downward trend
Further visual exploration reveals the culprit is Die Life.
• Die Life is the primary driver of the response • The effect of Die Life was known(qualitatively) but not communicated to
me • It’s effect obscures the effects of studied variables • If partitioning were used to model Die Friction, the tree would primarily
split on die life and miss other important variables • Solution??? “Regress out” the known die life effect
• A smoothing spline was used to regress out the effect • CompareTest Periods and the effects of other studied variables using Residuals
Comparisons are now clearer Facilitated follow-on modeling PhaseTwo
Analytics
Die
Fri
cti
on
Die
Fri
cti
on
Die
Fri
cti
on
Die
Fri
cti
on
Die
Fri
cti
on R
esid
s
Peril #3: Decision Trees Selecting Candidates that are Predictive But not Useful or Provide Incomplete Control Guidance
In manufacturing prediction is not good enough!
The end game in manufacturing is control
For control purposes, a predictor must be causative
Decision trees can select variables that are predictive but make no sense or are not useful to a process engineer from a control standpoint
Tree growth needs to be guided by process knowledge
JMP® permits guiding the partitioning process … enabling reality checks!!!
Decision trees may also miss important variables. They may be correlated with the top choice in a split, can share prediction/control capabilities but partitioning may ignore them
In non-manufacturing classification tree applications we’re not looking to control but to classify … even if the predictor isn’t “root cause” it may be good enough!
Common Issues: ◦ Nonsense correlations
◦ Non-actionable predictors
◦ Predictors that are actually responses … “After further review…”
◦ Important control variables are missed because of correlation between key predictors
PhaseTwo Analytics
Storks vs population in Oldenburg, Germany 1930-36
So, would getting rid of storks be an effective mechanism for population control ???
The reality is that increased population causes the increase in the number of Storks
Does watching TV increase life expectancy? "Televisions, Physicians, and Life Expectancy" in the _Journal of Statistics Education_ (Rossman 1994)
Data from a number of countries were gathered from The World Almanac and Book of Facts 1993
So, to improve life expectancy do we ship boat loads of TVs to those countries with low life expectancy ???
Nonsense Correlations
PhaseTwo Analytics
Sheet scrap was a >$1MM/yr problem
The problem was seasonal
Engineers weren’t sure if the cause was related to ingot casting, heat treatment or rolling
A swat team of metallurgists, process engineers and a data scientist was formed
Data were being recorded throughout the manufacturing flow path
Data collection, assembly, characterization/feature extraction was a HUGE task
Shortly after assembling & cleaning the data a quick partitioning exploration yielded the following:
Case Study 4: Sheet Scrap Integrating Process Knowledge With Partitioning
PhaseTwo Analytics
Occurrence rate ~10%
Partitioning Step 0
Partitioning Step 1: Problem Solution Emerges!
Although the root cause emerged quickly and required verification some additional exploration of the data took place that provided further discovery insights
PhaseTwo Analytics
Sheet Scrap (cont.)
Step 3 Over-ruled by the process engineers and MC1 is locked out Interesting … but can’t be implemented
Step 3.1 Tells us that the remainder of the problem would go away if dew point<63.3 BUT we can’t control dew point in a plant
Step 4 Also over-ruled by the process engineers and MC2 is locked out Step 4.1 Produced an interesting countermeasure
Step 5 Was a “what-if” scenario where we locked out the root cause to explore additional process countermeasures
Step 6 identified a temperature countermeasure that made total sense to our metallurgists!!!
PhaseTwo Analytics
Sheet Scrap Case Study Summary Points
1. Data assembly was a PAINFUL task!!!
2. Once data assembly was completed … discovery was FAST !!!! • This problem was ongoing for more than 2 years!!!
• Once the data were assembled discovery occurred within minutes!!!
3. Discovery was validated through a designed experiment
4. JMP® permitted control of the partitioning steps with the ability to override split selection based on engineering judgment and 1st principles • Nonsense correlations can be easily dismissed using JMP®
• Splits that are not practically useful (but perhaps interesting nonetheless) can be noted and then locked out
5. Interaction between the Data Scientist and Engineer during the partitioning process is critical to getting to root cause understanding
PhaseTwo Analytics
Case Study 5: Root Cause Discovery In Part Fabrication with Correlated Predictors
**Guiding The Discovery Process, Part II**
Key Learnings are:
◦ The Value of JMP® Visualization
◦ Must be attentive to correlated Predictors when partitioning
May be identified in “Candidate” list ….. pay attention!
Important (root cause) predictors can be missed
Partitioning can obscure their contribution
Only one may be selected as Predictive BUT More than One may actually be Needed for Control
◦ Engineering judgment/1st Principles Should guide Partitioning
◦ Complement Partitioning with Multivariate Methods
◦ Once the data are assembled, Discovery is FAST!!!
PhaseTwo Analytics
Problem: Dimensional Tolerance In A Part Fabrication Process
Process entails production of a highly engineered part
Processing irregularities produced thick and off tolerance parts (6% scrap and 50% rework)
Process data were in multiple, disparate databases that required extraction, alignment, characterization and cleaning
Attempts over many months by engineering to identify root cause using standard “one variable at a time” statistical tools were unsuccessful and time consuming
68 different process parameters studied against the product data
PhaseTwo Analytics
Initial Visual Exploration Critical part dimensions both with USL=3.0
PROBLEM LOCATION 6% Scrap 50% Rework
Different Location OK!
Problem improved over time but they didn’t understand why and were still incapable Furnace differences???
PhaseTwo Analytics
Guided Partitioning
The 1st Splits Appear to be Promising BUT Engineering Questioned The Validity and Usefulness of Some Predictors selected
This was a correlated response … NOT a predictor
This was interesting but is not actionable
V_WDY and S_KW_P were locked for the rest of the analysis
PhaseTwo Analytics
• Guided partitioning yielded actionable rules • The rules were easily implemented and the predicted results
were validated in production!!! • Partitioning required engineering guidance • Guidance provided a reality check on root cause • Yearly savings ~$300,000/yr. • Time to discovery … SEVERAL HOURS!!! • Return on investment VERY HIGH • Engineering spent months looking at data for a solution
Continue Partitioning With Guidance
PhaseTwo Analytics
Correlated Predictor Considerations
Candidates That Are Close Choices Require Scrutiny !!! 1. Initial group of “Candidates” were very close choices based on Logworth/SS 2. Be aware that minor error (e.g. measurement) would affect their ranking 3. The group is closely ranked because they’re strongly correlated (in this case) 4. Partitioning only selects one of them although all are important !!! 5. Because of this, careful scrutiny is advised and candidate selection based on engineering judgment 6. Diagnostics from a PLS suggests they may be influential IN TANDEM 7. This should factor into partitioning rule implementation with engineering input
Are SW_8 and W_7 worthy of closer consideration? PhaseTwo Analytics
A Case For Considering a Candidate Not Ranked 1st • W_8 and SW_8 were very close choices based on both Logworth and SS
• Can we be sure W_8 is more significant than SW_8? • Let’s examine what would have happened if W_8 were measured with a small amount of additional error
• Replace “W_8” with “W_8 err” where “W_8 err” is W_8 with .5% of error added • The Candidate list now reveals that SW_8 has moved into the 1 st ranking and W_8 is third
• Perhaps a case exists to give SW_8 consideration based on the sensitivity of the ranking to errors
PhaseTwo Analytics
PLS Useful In Vetting Influence of Predictors
• Augment a partitioning study with PLS modeling to guard against missing important influences • For the Part Fab data set a Correlation Loading Plot can be used to identify influential predictors • Correlated predictors will tend to cluster • The distance from the center to a predictors normal projection onto the diagonal line measures its influence
• PLS Validated Partitioning Contributors But Augmented The List & Suggests the Combined Influence of W & SW variables
• Engineering agreed that the correlation between the Ws and SWs needed to be maintained in implementation
PhaseTwo Analytics
There are Perils When Mining Manufacturing Data ◦ Data assembly
◦ Data with differing time scales… alignment & feature extraction
◦ Known effects, unknown to you & the value of visualization
◦ Prediction vs Control … getting to root cause & the need for engineering guidance (permitted by partitioning platform)
◦ Correlated predictor issues … important predictors can be missed
JMP® Tools & Platform Features help Mitigate the Perils
After data assembly discovery is fast It’s worth it!
PhaseTwo Analytics
QUESTIONS
PhaseTwo Analytics
