Lecture 15: Test Procedure in Engineering Designmech350/Lectures/MECH350-Lecture-15.pdf · A good test plan/procedure must be developed prior to testing. This plan should include

1

MECH 350Engineering Design I

University of VictoriaDept. of Mechanical Engineering

Lecture 15: Test Procedure in Engineering Design

© N. Dechev, University of Victoria

2

INTRO TO TESTINGDESIGN OF EXPERIMENTSDOCUMENTING TESTS

Outline:


Detailed Design-Detailed Analysis-Simulate & Optimize-Detail Specifications-Drawings, GD&T

3

Testing & Evaluation within the “General” Design Process


Identify Need-Talk with Client-Project Goals-Information Gathering

Conceptualization-Brainstorming-Drawing/Visualization-Functional Decomp.-Morphologic Chart

Preliminary Design & Planning-Prelim. Specifications-Prelim. Analysis-Decision Making-Gantt Charts & CPM

Report/Deliver-Oral Presentation-Client Feedback-Formal Design Report

Prototyping-Prototype Fabrication-Concept Verification

Testing/Evaluation-Evaluate Performance-Are Objectives Met?-Iterate Process Steps 2 - 7 as needed

Problem Definition-Problem Statement-Information Gathering-Design Objectives(quantifiable/measurable)

4

Testing is an essential part of the engineering design process.

Upon completion of the detailed design stage, the design team must evaluate the design to determine if the design objectives have been satisfied.

Introduction to Testing


5

Test results can take different forms:Qualitative (value statements, observations)

Yes/NoGreat, Good, Acceptable, Poor

Quantitative (numeric, measurable) Meters/second, Kilograms, Newtons Decibels, etc...

In MECH350, we are interested in the evaluation of the design objectives. Hence, your tests should generally produce ‘quantitative’ results.Quantification of test results (with proper testing) allows for designers and clients to agree that the design objectives have been met.

Introduction to Testing


6

This is the scientific method/process through which we acquire knowledge and build understanding on how nature works.

Formal Testing Methods: The Scientific Method


Hypothesis, Conjecture or Idea

Conduct Experiment- Procedure- Data Collection- Other Observations

Analysis of Data- Statistical Analysis- Report Results

Design of Experiment- Experiment Type- Variables- # of Samples

Discussion of, and Reflection on Results- Report Results- New Ideas!!!

7

An engineering experiment is used to test the performance of an engineered design, and is similar to the approach used in the scientific method.

Formal Testing Methods in Engineering Design: Design of Experiments


Define Testing Scope- Goal/Purpose- Expectations- Develop Test Plan

Conduct Experiment- Procedure- Data Collection- Other Observations

Analysis of Data- Statistical Analysis

Design of Experiment- Experiment Type- Variables- # of Samples

Discussion of Results- Test Report- Future Considerations

8

A good test plan/procedure must be developed prior to testing. This plan should include the following aspects:

Scope: Identify the system and goal/purpose of the testRelevant parameters for the test and justifyExpectations of the test (Hypothesis)

Administrative Details:Date and Location of TestingClient or Organization that has requested the testWho is conducting the test

Developing a Test Plan


9

Design of the Test (Also know as: Design of Experiment)Type of testing method and its significanceList of test apparatus and measurement equip, model #s, etc...Identify ‘dependant’ and ‘independent’ variables# of factors considered (single, double (factorial), etc...)Sampling procedure:

How are samples obtained?Number of samples

Detailed, step-by-step procedure to conduct the test.

Conduct/Perform the Test/ExperimentSafety precautionsData collection method (written, recorded, digital, etc...)Observation of external factors (temperature, wind, noise, vibration, etc...)

Developing a Test Plan (cont.)


10

Analysis of Test/Experiment Data

ConclusionsInterpretation of data analysis, and other observationsWritten test report

New Hypothesis, leading to re-design of test

Developing a Test Plan (cont.)


11

The subject of “Design of Experiments” is intimately related with statistics and statistical analysis, and is a major area of study. Hundreds of books, and courses are dedicated to the area.

Examples where “rigorous testing” is essential to demonstrate the performance of a consumer product:

____________________________________________________________

If you are the engineer in charge, how are you going to “demonstrate the product meets performance expectations”? How are you going to prove/demonstrate it to the world?Answer: --> Proper Design of Experiment

Design of Experiments for Engineers


12

Design of Experiments involves factors such as:

Formal HypothesisExperiment Type: Single-Factor, Factorial (multi-factor)Dependant VariableIndependent Variable# of SamplesRandomizationBlockingOrthogonalityStatistical SignificanceRegressionCovariance Analysis



13

Formal Hypothesis:A formal statement of what the experimenter expects to find in the data.For example: “Treatment of a material sample, will lower the percentage of sample failures under load.”

Null Hypothesis:A Hypothesis that is stated in a null form (which is more easily testable).For example: “There will be no difference in the percentage failures between those samples treated, and those that were not.”



14

Independent Variable:Is a controllable experimental variable. By changing/controlling the Independent Variable during an experiment, we seek to see if that contributes to a response in the Dependent Variable.

Dependent Variable (Response Variable):Is a characteristic outcome of an experiment on which information is to be gathered or obtained.The outcomes of a true experiment should be random, so a dependent variable is a random variable. There is a probability distribution associated with different kinds of random variables, such as: Normal, Poisson, Binomial, Chi-squared, Student t, Beta, Gamma, and many other distributions.



15

Full treatment of the subject of design of experiments is beyond the scope of MECH350, but it is vitally important in the testing phase of engineering design.

For further information, consider the following references:[1] C.R. Hicks, K.V. Turner, “Fundamental Concepts in the Design of Experiments”, Fifth Edition, Oxford University Press, 1999.Wikipedia, “Design of Experiments”, available online: http://en.wikipedia.org/wiki/Design_of_experiments

Design of Experiments


16

Good testing documentation serves to show others that your design has met the intended design objectives.

The test report should be written in such a way, as to allow an independent, third-party to review the test plan, the experimental design, the collected data, and the data analysis, and allow them to generally come to the same conclusions/test results that you did.

A good testing report is similar to a “laboratory report”, but includes extra elements (next page).

Test Reports for MECH350:


17

Elements of a Test Report:Scope (see page 9)Administrative Details (page 9)Design of Test (page 10)Procedure (step-by-step of how test is conducted)Data collected

Summary if lengthy, with raw data in Appendix.Other noteworthy observations (just state the facts and leave comments/discussion for later).

Analysis and Interpretation of DataSummary of Results and Discussion

Table format for this is helpful (see next page)Ideas for Further experiments or experimental improvementConclusion



18

Example of: Table Summarizing Results with DiscussionShould have four columns:

Original objective + it valueActual objective value from testingDeclaration of meeting objectiveExplanation/discussion of result



Objective (Target) Test Result Objective Met? DiscussionSize (L x W x H) La x Wa x Ha Yes Explain . . .Weight (10 lbs) 12 lbs No Explain . . .Noise (73 dB) 70 dB Yes Explain . . .Cost ($350) $250 No Explain . . .

Safety (described page x) (described on page y) Yes Explain . . .Etc. . .

Documents

Lecture 15: Test Procedure in Engineering Designmech350/Lectures/MECH350-Lecture-15.pdf · A good test plan/procedure must be developed prior to testing. This plan should include