Measurement risk and the impact on your processes

Measurement Riskand the Impact on Your Processes

Webinar hosted and presented by

Better by Your Every Measure • 800.828.1470 • www.transcat.com


Agenda

What is Measurement Risk?

Where does Risk Creep into Your Process?

Where does Risk Creep into the Calibration Process?

Calibration Results: Impact on Your Process

Question and Answer Session


What is Measurement Risk?

Measurement Risk is the probability that measurements are incorrectly influencing the acceptance decision on your product.• This can impact your company in many ways:

– Acceptance of bad product, which can lead to negative results ranging from consumer dissatisfaction to fatalities and lawsuits.

– Rejection of good product, which can lead to negative results ranging from unnecessarily scrapped or reworked product to wasted resources performing superfluous impact analyses.

It is possible that some of your rework, scrap, recall or any problems with released product are the result of unbridled measurement risk.

How can you determine if measurement risk is a problem in your manufacturing process?

Let’s refer to this in the positive term of Measurement Assurance.



Measurement Assurance encompasses a number of assumptions in your

manufacturing process:

1. Product Acceptance You made the right decision about the acceptance or

rejection of your product (or a component of your

product) during the manufacturing and quality

acceptance process.

2. Basis of Decision The decision was based (partially or completely) on

quantitative information provided by one or more

measuring instruments.

3. Suitability for

Intended PurposeYou made the right choice in selecting instrument(s)

that are appropriate for the measurements in your

process.

4. Instrument

Application

The instrument(s) were used correctly in the

manufacturing process.



Measurement Assurance encompasses a number of assumptions (con’t):

5. Beginning of

Period Reliability

The calibration of the instrument prior to your decision about

the product indicated it met its performance expectations.

6. End of Period

Reliability

The subsequent calibration of the instrument following your

decision about the product indicates that it continues to meet

its performance expectations.

7. Calibration Process Both calibrations were executed correctly.

8. Non-Conformance

Review

If a calibration indicates that the instrument failed to meet its

performance expectations, your quality system reviewed any

impact to the decision(s) made about your product(s) that were

based on the quantitative values presented by the instrument.

9. NCR Accuracy This impact study was thorough and did not miss the mark on

determining any detrimental affect on product.

10. Risk mitigation Any negative result from the impact study was properly

mitigated to remove or reduce product risk.



Measurement Assurance

(Summary)

Reliability

Product Acceptance 1.00

Basis of Decision 1.00

Suitability for Intended Purpose 1.00

Instrument Application 1.00

Beginning of Period Reliability 1.00

End of Period Reliability 1.00

Calibration Process 1.00

Non-Conformance Review 1.00

NCR Accuracy 1.00

Risk Mitigation 1.00

• If a value from 0.00 to 1.00 is assigned to each of these factors, then a total reliability figure can be calculated to indicate the health status of your Measurement Assurance program.

• This is accomplished by multiplying the assigned values.

• If all factors have a reliability of 1.00, then overall Measurement Assurance has a reliability of 100%, indicating that no Measurement Risk is present.


Risk Category Ex. 1

Reliability

Ex. 2

Reliability

Ex. 3

Reliability

Product Acceptance 1.00 1.00 1.00

Basis of Decision 1.00 1.00 1.00

Suitability for Intended Purpose 1.00 1.00 1.00

Instrument Application 0.95 1.00 1.00

B.O.P. Reliability 1.00 1.00 1.00

E.O.P. Reliability 1.00 1.00 1.00

Calibration Process 1.00 0.50 1.00

NCR Review 1.00 1.00 0.00

NCR Accuracy 0.90 1.00 0.00

Risk Mitigation 1.00 1.00 0.00

Measurement Risk 0.855 0.50 0.00



Risk Category Ex. 1

Reliability

Ex. 2

Reliability

Ex. 3

Reliability








NCR Review 1.00 1.00 0.00






Risk Category Ex. 1

Reliability

Ex. 2

Reliability

Ex. 3

Reliability








NCR Review 1.00 1.00 0.00







Suitability for Intended Purpose (3) – This means that the

instrument selected:

• Measures the right parameter(s) (e.g., voltage, pressure,

temperature, length, et.al).

• Provides valid measurement results, meaning that the instrument is

accurate and stable enough to provide useful information.

• Can sustain the environmental conditions to which it must be

exposed while in use and still provide valid information to the user.

• Is user-friendly at a level commensurate with the

education/experience of the operator so that errors are not

introduced through improper use/application.



Instrument Application (4) – This means that the instrument is being used as

designed by the OEM.

• Incorrect use introduces errors into the measurement process, which will change

the decision of the outcome for the part/product.

SPRT

w/Indicator

Entering the wrong coefficients into the display, causing temperature

readings to be off (sometimes significantly).

Calipers Dropping the calipers, causing a nick or burr on the jaws to interfere

with the measurement.

Torque Wrench Operator error can introduce upwards of 10% error!

Dry Block

Calibrator

Using the wrong size insert for the temperature probe; Not

compensating for axial or radial uniformity errors; overloading the dry

block – all of these introduce errors into the perceived temperature

being generated by the dry block.

Oscilloscope Lack of understanding bandwidth constraints; using the instrument

above the frequency cutoff for bandwidth will provide low amplitude

readings, causing errors in your decisions about the product/test quality.



Beginning of Period Reliability (5) – Some food for thought:

• Did you order the instrument and put it right into production without

having it calibrated first?

• If you relied on the “production cert”, this is not always a guarantee

that the instrument performs within the OEM’s stated performance

specifications; often this is simply a statement that the instrument

went through an operation check or production test that only

guarantees it is functional.

• Was the initial OEM cal ISO 17025 accredited? Often this is not the

case.

• For recalibration of the instrument, was the instrument returned to

you in a condition that is ‘close’ to being OOT? How long will it

maintain this condition during the current usage period?



End of Period Reliability (6) – This is the most critical

information that must be evaluated in order to close the loop

on your measurement assurance program:

• Was the instrument found to be OOT?

• Was the instrument found to be In Tolerance but close to being

OOT?

• What impact does the lab’s uncertainty have on this condition?

• How do you apply this information to the use of the instrument

during its last usage period?



What are the measurement parameters, or functions, on

the instrument you are submitting for calibration?

Are there multiple ranges/values on the instrument?

What is the accuracy of these ranges/values?

Is the lab capable of performing calibrations for these

parameters/functions/ranges/values?

How does their measurement uncertainty compare to the

tolerances of the instrument? Will they be able to provide a

valid result that is meaningful?



What test points will they check on your instrument?

What tolerances do they apply? Are they the same as the

ones stated by the OEM?

If you have decided to use tolerances that are different than

those stated by the OEM, does your calibration provider

know this and apply them to your calibration?

Is the calibration technician competent to perform the

calibration? How would you know?

Is the calibration report easy or difficult to understand?



NCR Review: Whether or not your quality system mandates

it, this step can save you money, can prevent safety

problems from occurring, and can prevent negative

marketing resulting from bad product being released.

• Again, this evaluation is often under-executed resulting in the false

perception that there is no impact to the product.

• Transcat offers consulting in this area to help your engineers and

quality personnel to understand how to get the most out of an

impact evaluation.

• How many OOT-NCR evaluations have you had in the past 5

years?



NCR Review (continued):

• How many of these had a negative impact on your product?

• Answer one simple question: If the answer is zero or nearly zero,

then is this because you have such a well designed measurement

assurance program that it always prevents an impact to the

product?

• If it seems too good to be true, then you might consider the quality

of the evaluations being performed.

• Also, an external auditor might not believe you’ve designed a

system that is so good that you’ve never experienced a problem

with your measurements that affected product.


Risk Mitigation in Your Process

If you have discovered an OOT condition that may have

negatively influenced your decision about the

acceptance/rejection of product, consider this:

• If the OOT condition caused you to reject good product, then it is in

your best interest to determine how much this cost your company

in:

– scrapped product

– rework costs

– OOT-NCR evaluation costs


Risk Mitigation in Your Process

If you have discovered an OOT condition that may have

negatively influenced your decision about the

acceptance/rejection of product, consider this:

• If the OOT condition caused you to accept bad product, then it is in

your best interest to determine how much this cost your company

in:

– consumer perception of a troublesome or faulty product

– liability suits for safety issues resulting in property damage,

medical costs, or fatalities

• Determine root cause to reduce or prevent these costs from

recurring.


Measurement Assurance / Measurement Risk

In the end, running your business is not only about international, federal, customer, or internal requirements, policies, or procedures –it’s about making a safe, reliable, superior product that fills a need/desire in the market place and is profitable for your company.

Measurement Assurance should be designed to help you minimize risks in your decision making process about your product’s safety and quality.

Too often it is not fully implemented (all 10 categories), causing the reliability of your Measurement Assurance Program to lose value and become ineffective.

If you’re going to put the effort and money into only some parts of this program, it will likely cost you more elsewhere through rework, recall, or consumer perceptions.

Make your Measurement Assurance Program robust so that it works for you to keep the cost and safety dogs at bay!

Thank You for Joining Us!

800.828.1470

www.transcat.com

Questions?

Technology

Measurement risk and the impact on your processes