Upload
others
View
5
Download
1
Embed Size (px)
Citation preview
The Quality 4.0 Revolution: Reveal Hidden Insights Now With Data
Science and Machine LearningNicole M. Radziwill, PhD, MBA
ASQ Fellow & Editor, Software Quality ProfessionalQuality Practice Leader
Cameras on front of remote-controlled cars sampled at 20 images/second
Based on patterns in the photos, give car steering instructions
Goal: Keep them on the road up to max speed!
(AWS with multilayer neural net in Python/TensorFlow in Jupyter notebooks)
https://aws.amazon.com/blogs/machine-learning/build-an-autonomous-vehicle-on-aws-and-race-it-at-the-reinvent-robocar-rally/
“… a leading car manufacturer… was experiencing recurrent faults, and therefore high warranty costs, on the rear tail-lights. The engineers had been looking at the rear of the vehicle for the answer (and not succeeding) however the software quickly found the root cause of the fault to be located in the roof of the vehicle, a part of the car that had not even been investigated as a possible source. Sometimes prior experience, or being too close to a problem, can inhibit a solution if an old hypothesis is applied to a new problem.”
From http://www.qmtmag.com/display_eds.cfm?edno=9665074
“chimney”
Chen, X., Liu, F., Hou, Q., & Lu, Y. (2009, August). Industrial high-temperature radar and imaging technology in blast furnace burdendistribution monitoring process. In Electronic Measurement & Instruments, 2009. ICEMI'09. 9th Intl Conference on (pp. 1-599). IEEE.
From https://www.slideshare.net/WindowsDev/build-2017-p4005-microsoft-r-server-development-to-deployment-workflow
?
New Ways to Solve Old
Problems
You will:
1.Understand what Quality 4.0 is and the value propositions it presents.
2.Identify the types of problems to which you can apply these new methods in your organization.
3.Explain how these technologies support engagement and collaboration to promote a connected and empowered workforce.
4.Explore ways to incorporate data science and machine learning into your work, whether you are drowning in data or just starting your digital transformation.
Nicole RadziwillQuality Practice Lead, Intelex
Fellow, American Society for Quality (ASQ)CSSBB #11962 & CMQ/OE #9583Ph.D. Quality Systems, Indiana StateEditor, Software Quality Professional
Previously:•Project Manager, Solution Architect, Engagement Manager // Meteorological Research, Telecom, Manufacturing, Software, High Tech 1995-2002•Division Head, Software, Green Bank Observatory, 2002-2006•Assistant Director of End to End Operations, National Radio Astronomy Observatory, 2006-2009•Associate Professor of Data Science & Production Systems, James Madison University, 2009-2018; taught data science & applied machine learning from 2009-2018
1: Quality 4.0and value propositions enabled by data science
Quality as:1.0 - Inspection
2.0 - Design3.0 - Engagement
Quality Planning Quality Assurance/ Quality Control
Quality Improvement
• Establish quality goals• Understand organizational
context (mission, vision, values, strategy, position, capabilities)
• Understand business environment
• Identify customer & stakeholder needs – Voice of Customer (VoC)
• Design products/services to meet needs, implement design controls
• Design processes for consistent production & related controls
• Evaluate process performance
• Compare performance with quality goals
• Manage “quality events”and update controls
• Audits• Management reviews• Nonconformances• Incidents/near misses• Complaints
• Out-of-control production• Act to close the gaps
• Build capabilities (people and technical)
• Build infrastructure• Identify and justify needs
and gaps• Conduct improvement
activities and establish controls to maintain them
• Quick wins (PDCA)• Process improvement
(DMAIC, Lean)• Process design or redesign
(DMADV)• Business process mgmt
(BPM) or Robotic Process Automation (RPA)
• Confirm/Validate changesIntegrate lessons learned into
Quality Planning
Quality Events indicate that quality goals are not being met and action is needed
• Nonconforming product• Incidents/near misses• Customer complaints• Audit findings• Recalls/warranty calls• Deviations (from SOP)• Out-of-control Action Plans• Industry-specific events (e.g. MDRs)
Serious or systematic? CAPA
yes
not really
containment
Quality Controlsto prevent or correct unwanted or unexpected change stability and consistency
• Calibrations• Maintenance• Inspections• Sampling incoming parts• Process validation• Mistake-proofing• In-situ process monitoring• Environment monitoring• Professional testing/ competency
assessment• Training programs and reminders• Corrective actions taken• Information security/ network security
Why Quality Systems Work
From Kovach, J. V., & Fredendall, L. D. (2013). The influence of continuous improvement practices on learning: An empirical study. The Quality Management Journal, 20(4), 6.
AI/ML Can Enhance & AccelerateOrganizational Learning
People Intelligent SystemsAs people learn about the work environment, they update:
• policies, • procedures, • practices, and • heuristics
to improve the performance of people, projects, processes & the bottom line.
When AI/ML systems learn, they update how they:
• predict, • classify, • find patterns, and • identify important predictors
to improve the performance of people, projects, processes & the bottom line.
* Jim Duarte (ASQ TV) also noticed that the algorithms do what we’ve been used to doing as quality professionals… just iteratively, and much faster.
Quality as:1.0 - Inspection
2.0 - Design3.0 - Engagement
4.0 - DiscoveryAI/ML evolves data-driven decision making
to be self-aware & adapt to changing environments, circumstances, and
customer/stakeholder needs
@doug_laney
Data Science:•Predict•Identify Causal and Noncausal Relationships•Drive Value
Through:•Data Aggregation•Real-time pipelines•Dynamic Modeling
Learn
$30/GB
$~0/GB
From http://www.mkomo.com/cost-per-gigabyte
c/o 92 !Computer Science•Big Data infrastructure•Programming/data access•Deploying ML models•Streaming dataMath & Statistics•Machine learning (ML)•Model building and evaluation•Anomaly detection•Forecasting•Ensembles•Design of ExperimentsDomain Expertise
Scientific Method vs Data Science
Newman, M. E. (2003). The structure and function of complex networks. SIAM review, 45(2), 167-256.
1. Develop a hypothesis based on theory
2. Take a random, representative sample from the population
3. Compare characteristics to your prediction
4. Draw conclusions
1. Just look at the entire population of data and see what you find… we have the tools to do it and the data is cheap! Who needs theory or a hypothesis?
Traditional Data Science
Lazer, D., Kennedy, R., King, G., & Vespignani, A. (2014). The parable of Google Flu: traps in big data analysis. Science, 343(6176), 1203-1205.
‘ …GFT overlooksinformation that could be extracted by traditionalstatistical methods.’
‘ “Big data hubris” is the often implicit assumption that big data are a substitute for, rather than a supplement to, traditional data collection and analysis.’
What is Big Data?• Lots of it (volume)• It’s coming at you fast (velocity)• Different formats and sampling frequencies (variety)• Huge variations in data quality (veracity)• Different people/organizations produce it or own it (governance)• It could easily change or disappear (control)• There may be restrictions on how you use it (policy)
“… Big Data is anything bigger or more complex than what your organization is currently prepared to handle.”-- one of the world’s experts on cyberinfrastructure for Big Data at a National Science Foundation panel (2013)
data
From Waller, M. A., & Fawcett, S. E. (2013). Data science, predictive analytics, and big data: a revolution that will transform supply chain design and management. Journal of Business Logistics, 34(2), 77-84.
Radziwill, N. M. (2018, October). Let’s Get Digital: The many ways the fourth industrial revolution is reshaping the way we think about quality. Quality Progress, p. 24‐29. http://qualityprogress.com
Connectedness
Intelligence
Automation
Quality 4.0 = C I A
Degrees of Automation1. Human specifies process and computer only implements2. Computer assists by determining options3. Computer assists by determining options and suggests choice4. Computer assists by determining options and selects choice - human has
option to do it, or not5. Computer selects and implements options if human approves6. Computer selects option and automatically implements, but gives human
chance to stop the process flow7. Computer selects and implements options, then tells human about results8. Computer selects and implements options, but only tells human about
results if asked for report9. Computer selects and implements options, but only tells human about
results if asked, only reports some information10. Computer performs whole job; may or may not tell human
Increasing autonomy
From Sheridan, T. B., & Verplank, W. L. (1978). Human and computer control of undersea teleoperators. MASSACHUSETTS INST OF TECH CAMBRIDGE MAN-MACHINE SYSTEMS LAB.
2: Types of Problems& how to identify challenges that are well supported by DS/ML
From Carlson, R. O., Amirahmadi, F., & Hernandez, J. S. (2012). A primer on the cost of quality for improvement of laboratory and pathology specimen processes. American journal of clinical pathology, 138(3), 347-354.
Profit
Profit
From Carlson, R. O., Amirahmadi, F., & Hernandez, J. S. (2012). A primer on the cost of quality for improvement of laboratory and pathology specimen processes. American journal of clinical pathology, 138(3), 347-354.
Profit
ProfitProfit
Opportunity Capture
Value Vision for Quality 4.0
External Failure
Internal Failure
Appraisal
Error- and waste-free
service
Siloed/ Struggling
Quality 4.0 Ecosystem
Radziwill, N. M. (2018, October). Let’s Get Digital: The many ways the fourth industrial revolution is reshaping the way we think about quality. Quality Progress, p. 24‐29. http://qualityprogress.com
Machine Learning Tasks Data Science Tasks
1. Prediction2. Classification3. Pattern Identification4. Data Reduction5. Anomaly Detection6. Pathfinding
1. Frame the problem(s)2. Explore the data3. Build models4. Evaluate model performance5. Deploy models and real-time
processing pipelines6. Uncover high-value insights7. Tell stories8. Do it again… and again
From Duarte, J. (2017). Data Disruption. Quality Progress, 50(9), 20‐24.
“Data can take several forms, such as data at rest and data in motion. Data that are sitting in warehouses waiting to be analyzed are referred to as data at rest…
Data in motion (data analyzed in real time as the event occurs, such as click streams and sensors) flow through a connected device to a database on the receiving end for immediate analysis… used to fine tune the [predictive] models… and analytics at the edge… deploys information to control processes in real time.”
-- Jim Duarte
Granstedt Möller, E. (2017). The Use of Machine Learning in Industrial Quality Control. Thesis, KTH Vetenskap Och Konst (Stockholm, Sweden).
Radziwill, N. M. (2019). Value Propositions for Quality 4.0. Manuscript in review.
3: Engagement & Collaborationwith Quality 4.0
Radziwill, N. M. (2019). The Relationship Between Engagement and Quality Outcomes. Manuscript in review.
Execution
What is Engagement? A Review of ~120 Research Papers
Extension
Excitement
Community Building &
Trust
• Ensures alignment• Supports collective efforts• Achieves consistency• Deploys strategy broadly &
deeply
• Helps you do your job• Understand expectations• Understand process• Get into flow
• Drive out fear• Build safety and quality culture• Find meaning and purpose
Radziwill, N. M. (2018, October). Let’s Get Digital: The many ways the fourth industrial revolution is reshaping the way we think about quality. Quality Progress, p. 24‐29. http://qualityprogress.com
• Mobile Devices• Cloud Computing• IoT/IIoT• Integrated Systems• AR/VR for Training and
Documentation
• Trust• Data
Quality
• Do job better• Find “levers”• Ensure consistency &
alignment across org• Build safety & quality
culture by uncovering relationships
Bernd Leukert, Executive Board SAP, at https://news.sap.com/2018/01/industrie-4-0-why-openness-and-collaboration-make-all-the-difference/
4: Reveal Hidden Insights Nowincorporate DS/ML & ensure readiness for the future
I want my QMS/IMS to…• Help me make better decisions
about my processes• Audit itself and alert me when I
need to do something• Tell me where to focus my
resources for maximum added value• Based on risk assessments and
theory of constraints, tell me how should I prioritize work
• Alert me when an important changeoccurs at the systems level
DS in the Cloud: http://rstudio.cloud
Example to follow: https://qualityandinnovation.com/2015/07/14/a-simple-intro-to-bayesian-change-point-analysis/
> install.packages("changepoint")
Example 2: Wine Qualitylibrary(tidyverse)library(randomForest)
url <- "https://archive.ics.uci.edu/ml/machine-learning-databases/wine-quality/winequality-white.csv"wine <- as.tibble(read.delim(url,header=TRUE,sep=";"))
> wine# A tibble: 4,898 x 12
fixed.acidity volatile.acidity citric.acid residual.sugar chlorides free.sulfur.diox~ <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>17.00 0.270 0.360 20.7 0.0450 452 2 6.30 0.300 0.340 1.60 41
# ... with 4,888 more rows, and 5 more variables: density <dbl>, pH <dbl>, sulphates <dbl>, alcohol <dbl>,quality <int>
> wine$taste <- as.factor( ifelse(wine$quality <= 6,'bad','good') )
Build the ML Model> train <- wine[1:3000, ]> test <- wine[3001:4898, ]> ml.model <- randomForest(taste ~ . - quality, data=train)> print(ml.model)
Call:randomForest(formula = taste ~ . - quality, data = train)
Type of random forest: classificationNumber of trees: 500
No. of variables tried at each split: 3
OOB estimate of error rate: 9.7%Confusion matrix:
bad good class.errorbad 2276 83 0.0351844good 208 433 0.3244930
Check Performance of ML Model> pred <- predict(ml.model, newdata=test)> table(pred, test$taste)
pred bad goodbad 1233 191good 246 228
> ml.model$importanceMeanDecreaseGini
fixed.acidity 64.47684volatile.acidity 84.14794citric.acid 61.37304residual.sugar 91.95227chlorides 86.69503free.sulfur.dioxide 82.76855total.sulfur.dioxide 80.33434density 125.57005pH 81.95264sulphates 78.01374alcohol 169.45978
plot(ml.model, log="y") legend("topright",
colnames(ml.model$err.rate),col=1:4,cex=0.8,fill=1:4)
Drowning in Data Haven’t Started Yet
Set up data governance processes. Bust silos, encourage collaboration, continue systems integration.
Invest in cybersecurity: physical, behavioral, and technical
Be selective: archive only most critical information, or “compute at the edges” & store aggregated values
Put controls in place to avoid the Data Swamp/Graveyard problem
Digitize! Make data capture part of what’s expected in each work process
Practice good data hygiene: one observation per row, one variable per column
Keep learning key concepts, get your feet wet and play
Wait it out! Gartner says ~40% of data science tasks will be automated by 2020.
In a perfect world, my QMS would tell me…
From Kendall, K. (2017). The Increasing Importance of Risk Management in an Uncertain World. The Journal for Quality and Participation, 40(1), 4.
Integrated Management Systems for Environment, Health & Safety, and
Quality Management (EHSQ)
Established in 1992.
Worldwide client base.
More than 500 employees.
Over 1,200 global customers.
Over 1.5 million users.
Peer reviewed as best managed company.
Nicole RadziwillQuality Practice [email protected]
@nicoleradziwill
https://www.linkedin.com/in/nicoleradziwill/
Connectedness
Intelligence
Automation
Quality 4.0 is…
… for discovering insights into performance.