Embed Size (px)
DESCRIPTION
Citation preview
UNIT - Lesson 1
Lesson Title: Project Management - NASA Style
Lesson Duration:
Standards: Students will: develop an understanding of the core concepts of technology (STL-2) develop abilities of technological design (AAAS-Standard E) understand and apply basic concepts of probability. (NCTM-17)
Benchmarks: Management is the process of planning, organizing, and controlling work.
(STL-2, EE) Write clear, step-by-step instructions for conducting investigations, operating
something, or following a procedure. (AAAS-65) Use tables, charts, and graphs in making arguments and claims in oral and
written presentations. (AAAS-65Q) Use simulations to construct empirical probability distributions. (NCTM-17, H)
Learning Objectives: Students will:1. identify, describe, and apply principles of project management used by NASA2. identify and describe significant ‘historical’ methods employed by NASA to
manage APOLLO missions (60s-70s)3. identify, describe, and apply NASA project planning methods4. identify, describe, and apply NASA project organization methods5. identify, describe, and apply NASA project scheduling or controlling methods6. participate in NASA management roles collaboratively7. use materials, tools, and equipment safely per NASA safety guidelines8. communicate information using NASA formats or styles.
Student Assessment Tools and/or Methods:Assessment Instrument - Engineering Design - 12 Steps
CategoryBelow Target At Target Above Target
Defining the Problem
Rephrases the problem with limited clarity.
Rephrases the problem clearly.
Rephrases the problem clearly and precisely.
BrainstormingContributes few or implausible ideas.
Contributes a plausible idea.
Contributes multiple plausible ideas.
Researching and Generating Ideas
Contributes ideas but without docu-mented research.Produces in-complete sketches.Does not present a concept.
Contributes one plausible idea based on documented research. Produces marginally accurate pictorial and ortho-graphic sketches of
Contributes multiple plausible ideas based on docu-mented research.Produces accurate pictorial and ortho-graphic sketches of
4/10/2023 1
design concepts. design concepts.
Identifying Criteria and Specifying Constraints
Does not restate the criteria clearly and fails to identify constraints.
Restates the criteria clearly and identifies several constraints.
Restates the criteria clearly and precisely and identifies many constraints.
Exploring Possibilities
Inadequately analyzes the pluses and minuses of a variety of possible solutions.
Satisfactorily analyzes the pluses and minuses of a variety of possible solutions.
Thoroughly analyzes the pluses and minuses of a variety of possible solutions.
Selecting an Approach
Selection of solution is not based on consideration of criteria and constraints.
Selects a promising solution based on criteria and constraints.
Selects a promising solution based on a thorough analysis criteria and constraints with high quality.
Developing a Design Proposal
Design proposal is inadequate lacking pertinent information.
Design proposal is adequate containing all pertinent elements.
Design proposal is accurate and comprehensive.
Making a Model or Prototype
Prototype meets the task criteria to a limited extent.
Prototype meets the task criteria.
Prototype meets the task criteria in insightful ways.
Testing and Evaluating the Design Using Specifications
Testing and evaluation processes are inadequate.
Testing and evaluation processes are adequate for refining the problem solution.
Testing processes are innovative.
Refining the Design
Refinement based on testing and evaluation is not evident.
Refinements made based on testing and evaluation results.
Significant improvement in the design is made based on prototype testing and evaluation.
Creating or Making It
Finished solution (product) fails to meet specifications.
Finished solution (product) meets specifications.
Finished solution (product) exceeds specifications.
Communicating Processes and Results
Solution presented with limited accuracy. Limited supporting evidence on how the solution meets the task criteria.
Solution presented accurately.Some supporting evidence on how the solution meets the task criteria.
Solution presented concisely with clarity and accuracy .Extensive supporting evidence on how the solution meets the task criteria.
CommentComment Comment Comment
4/10/2023 2
Rubric for Project Management
CategoryBelow Target At Target Above Target
PlanningLittle or no attempt at strategic or operational planning. Few, if any planning tools were used to accomplish financial, personnel, product performance, product forecasting, product life-cycle, or qualitative or quantitative analysis.
Effective use of strategic and operational planning for entire project. Appropriate planning tools were used to accomplish financial, personal, product analysis, forecasting, predictions using both qualitative and quantitative assessment strategies.
Outstanding examples of well developed planning tools for the entire project were evident and applied in both critical and creative ways. All areas of project management, including financial, personal, product development and analysis, forecasting and predicting using both qualitative and quantitative methods.
OrganizingThere was minimal attempt to generate an organizational structure related to the following areas: design teams, span of control, specific responsibilities, special-ization of work or tasks, work arrangements and relationships. Leadership roles were unclear or not identified appropriately. Leadership roles were poorly defined or not present at all.
Effective use of appropriate organiza-tional structures which addressed all critical areas: design teams, span of control, specific responsibilities, special-ization of work or tasks, work arrangements and relationships. Leadership roles were clearly defined and functioned effectively.
Outstanding organizational structure was employed through-out the project. Creative examples of organiza-tional tools were employed to direct all areas: design teams, span of control, specific responsibilities, special-ization of work or tasks, work arrangements and relationships. Exemplary leadership roles were exhibited.
ControllingActivities, strategies, and techniques reflective of contempo-rary project control were not evident. There was little attempt to establish industry standard practices to regulate, access and evaluate results for securing maximum productivity and reduce unacceptable perform-ance. There was very little attempt to establish and use accepted control techniques for people, process, or product.
Activities, strategies, and techniques reflective of contempo-rary project control were used effectively and appropriately. Industry standard practices to regulate, access work and evaluate results to secure maximum productivity and reduce unacceptable perform-ance were designed and used effectively throughout the project. This included the collection, analysis and storing of pertinent data, compare perform-
Activities, strategies, and techniques reflective of contempo-rary project control were used in critical and creative ways. Industry standard practices to regulate, access work and evaluate results to secure maximum productivity and reduce unacceptable perform-ance were designed and used to generate outstanding perform-ance data throughout the project. This included the collection, analysis and storing of
4/10/2023 3
ance against standards, generate detailed reports based on measurement tools. Limits of tolerance were established for people, process, and product with adequate clarity, but lacking some quality in detail and presenta-tion.
pertinent data, compare performance against standards, generate detailed reports based on measurement tools. Limits of tolerance were clearly established for people, process, and product.
CommentComment Comment Comment
Rubric for Quality Assurance
CategoryBelow Target At Target Above Target
Planning Little attempt at establishing critical processes and roles to ensure quality for all planning phases of the project. Critical areas such as quality standards for people, process and product were not clear in project planning, training to ensure quality for people, process, and product development, goal setting in all planning documents, quality councils, reporting timelines with milestones and the use of industry standard techniques such as, but not limited to; quality function deployment, criteria trees, needs metrics, milestone schedules, Gantt charts, house of quality charts.
Effective identification and use of critical processes and roles to ensure quality for all planning phases of the project. Critical areas such as quality standards for people, process and product were present and clear in project planning, training to ensure quality for people, process, and product development, goal setting in all planning documents, quality councils, reporting timelines with mile-stones and the use of industry standard techniques such as, but not limited to; quality function deployment, criteria trees, needs metrics, milestone schedules, Gantt charts, house of quality charts. Clearly defined roles and strategies were integrated into all planning and presented before project was initiated.
Outstanding identification and use of critical processes and roles to ensure quality for all planning phases of the project. Critical areas such as quality standards for people, process and product were present, clear and included extraordinary detail and focus in project planning, training to ensure quality for people, process, and product development, goal setting in all planning documents, quality councils, reporting timelines with mile-stones and the use of industry standard techniques such as, but not limited to; quality function deployment, criteria trees, needs metrics, milestone schedules, Gantt charts, house of quality charts. Clearly defined roles and strategies were integrated into all planning and presented before project was initiated.
Organizing The organization of work tasks, assign-ments, and responsibil-ities was unclear or
The organization of work tasks, assign-ments, and responsibil-ities was effectively
The organization of work tasks, assign-ments, and responsibil-ities was superbly
4/10/2023 4
poorly attempted. There was little evidence of quality assurance with respect to the deploy-ment of people to complete specific tasks related to product design and develop-ment, production processes, manage-ment processes and all industry standard techniques used to ensure structure and organization for the entire project.
accomplished. There was sufficient evidence of quality assurance with respect to the deploy-ment of people to complete specific tasks related to product design and develop-ment, production processes, manage-ment processes and all industry standard techniques used to ensure structure and organization for the entire project. Organization docu-ments were submitted in a timely fashion with adequate detail.
completed. There was significant and exemplary evidence of quality assurance with respect to the deploy-ment of people to complete specific tasks related to product design and develop-ment, production processes, manage-ment processes and all industry standard techniques used to ensure structure and organization for the entire project. Organization docu-ments were submitted in a timely fashion with outstanding detail and reflected the most contemporary industry practices for ensuring quality in all organiza-tion and structure.
Controlling Quality assurance techniques or strategies were unclear or simply not used to ensure any appropriate level of control with respect to people, process or product. There were very few instances or in some cases no attempt to employ industry standard practices to establish standards for controlling tasks, measuring perform-ance, evaluating performance and optimizing productivity for people, processes and product(s).
Quality assurance techniques or strategies were effectively designed and used to ensure appropriate levels of control with respect to people, process and product. There were numerous instances to employ industry standard practices for controlling tasks, measuring performance, evalu-ating performance and optimizing productivity for people, processes and product(s). Documentation was clearly developed and offered adequate evidence of appropriate control of people, processes and product design and develop-ment.
Quality assurance techniques or strategies were designed and used to in an exemplary fashion to ensure significantly high levels of control with respect to people, process and product. There were numerous creative instances to employ industry standard practices for controlling tasks, measuring performance, evalu-ating performance and optimizing productivity for people, processes and product(s). Documentation was clearly developed and offered exemplary evidence for the control of people, processes, and product design and development.
CommentComment Comment Comment
Assessment Instrument - Brief Constructed Response (BCR) - All Assigned TopicsCategory Below Target At Target Above Target
4/10/2023 5
Understanding Response demonstrates an implied, partial, or superficial understanding of the text and/or the question.
Response demonstrates an understanding of the text.
Response demonstrates an understanding of the complexities of the text at a deep level of understanding.
Focus Lacks transitional information to show the relationship of the support to the question.
Addresses the demands of the question.
Exceeds the demands of the question.
Use of Related Information
Uses minimal information from the text to clarify or extend meaning.
Uses some expressed or implied information from the text to clarify or extend meaning.
Effectively uses expressed or implied information from the text to clarify or extend meaning in all statements.
CommentComment Comment Comment
Assessment Instrument - Class Seminar for All Scheduled MeetingsCategory Below Target At Target Above Target
Participation Unacceptable interaction and participation with numerous inter-ruptions or off-topic discussions.
Adequate participation offering valuable comments at times, with only occasional inter-ruption of others
Active level of participation, offer-ing solid comments and ideas but not overbearing, allow-ing others to engage in discussion.
Contribution Rarely offers appropriate comments and seeks to disrupt the meeting.
Comments are appropriate and on topic, with some ideas of high value, enabling good dis-cussion of seminar topic.
Comments and ideas are of high value and enable more intense dis-cussion of seminar topic.
Cooperation Exhibited little courtesy towards others through inappropriate comments and behavior during meeting.
Appropriate level of courtesy towards others enabling good discussion on seminar topic and little disruption observed.
High level of courtesy towards others facilitating engaging and topical discussion on selected topics.
Topic Focus Not focused on topic and seeks to disrupt with inappropriate questions and comments through-out the meeting.
Comments usually on target and appropriate, including questions and topical discourse during
Comments are always focused on seminar topic, including questions and discourse with others during entire
4/10/2023 6
entire meeting. meeting.
CommentComment Comment Comment
Assessment Instrument - Business Letter - Various RequestsCategory Below Target At Target Above Target
Mechanics Header, salutation,and signaturecontain inaccurateinformation or areincomplete.
Header, salutation, and signatureaccurate.
Header, salutation, and signature in fullaccordance with thatspecified by acceptedbusiness format.
AuthorIdentity
Author ororganizationmissing.
Author identityand organizationstated.
The author hasproperly identifiedhim/her self and theorganization to which he/she belongs.
Content Purpose of letteris questionable.No direct actionor informationrequested.
Purpose of letter stated.
Purpose of letterclearly stated.Language is directand to the point. Noexcessive wordingpresent.
Grammar One major orseveral minorerrors present.
No errors. Flawless, with exceptional use of grammar.
ReferencedCommunication
(if applicable)
This letter followsanother. . ., whichrefers to previouscontact, but doesnot fully identifysuch.
This letter follows another. . ., which is identified by name, date, and context of previous communication.
This letter followsanother letter, phonecall, or e-mail, whichis clearly identifiedby name, date, andcontext of previouscommunication.
Comment Comment Comment Comment
Assessment Instrument - Extended Constructed Response (ECR) for All Assigned Topics
CategoryBelow Target At Target Above Target
Context and Argument
Context inappropriate.Argument unsatisfactory.
Context appropriate.Argument satisfactory.
Context appropriate.Argument satisfactory. Clearly stated thesis included.
Evidence Evidence is largely Ample and Abundant, relevant
4/10/2023 7
missing or generalized.
appropriate evidence provided.
specifics (names, events, legislation, court decisions, etc.) provided. Includes obscure, but important evidence. Thorough chronology.
Analysis Minimal analysis or fallacious reasoning.
Organizes argument and uses data to support conclusions.Recognizes causation, change, and continuity.
Well-reasoned cause and effect arguments.Fully explained conclusions.Refers to views of others.
HistoricalAccuracy
Many errors. May have a few errors. Mistakes may slightly hinder argument, but do not detract from the overall accuracy.
Virtually error free; minor mistakes do not compromise argument.
Thoroughness Covers question superficially. May not complete all tasks.
Covers entire question, but may be slightly imbalanced.
Covers all areas of question in approxi-mate proportions to their importance.
Presentation Inconsistent organization. Grammatical errors cloud argument to a major degree.
Uses clear language.Well organized.Contains few grammatical errors.
Uses clear, appropriate and precise language.Cohesive organiza-tion. Very few grammatical errors.
CommentComment Comment Comment
Assessment Instrument - Graphic Organizer- All Assigned TopicsCategory Below Target At Target Above Target
Arrangement of Concepts Main concept not
clearly identified; subconcepts don’t consistently branch from main idea.
Main concept easily identified; most subconcepts branch from main idea.
Main concept easily identified; sub-concepts branch appropriately from main idea.
Links and Linking Lines
Linking lines not always pointing in correct direction; linking words don’t clarify relationships between concepts; hyperlinks don’t
Most linking lines connect properly; most linking words accurately describe the relationship between concepts; most hyperlinks
Linking lines connect related terms/point in correct direction; linking words accurately describe relationship between
4/10/2023 8
function or fail to enhance the topic.
effectively used. concepts; hyperlinks effectively used.
Graphics Graphics used inappropriately and excessively; graphics poorly selected and don’t enhance the topic; some graphics are blurry and ill-placed.
Graphics used appropriately most of the time; most graphics selected enhance the topic, are of good quality, and are situated in logical places on the page.
Graphics used appropriately; greatly enhance the topic and aid in comprehension; are clear, crisp and well situated on the page.
ContentContains extraneous information; is not logically arranged; contains numerous spelling and grammatical errors.
Reflects most of the essential information; is generally logically arranged; concepts presented without too many excess words; only one misspelling or grammatical error.
Reflects essential information; is logically arranged; concepts succinctly presented; no misspellings or grammatical errors.High quality detail presented.
TextFont too small to read easily; more than four different fonts used; text amount is excessive for intended audience.
Most text is easy to read; uses no more than four different fonts; amount of text generally fits intended audience.
Easy to read/ appropriately sized; no more than three different fonts; amount of text is appropriate for intended audience; boldface used for emphasis. High quality text through-out document.
Design Cluttered design; low in visual appeal; requires a lot of scrolling to view entire diagram; choice of colors lacks visual appeal and impedes comprehension.
Design is fairly clean, with a few exceptions; diagram has visual appeal; four or fewer symbol shapes; fits page well; uses color effectively most of time.
Clean design; high visual appeal; four or fewer symbol shapes; fits page without a lot of scrolling; color used effectively for emphasis.
Knowledge Gained Student demon-strates a lack of knowledge about the content and the
Student can accurately answer most questions related to content
Student can accurately answer all questions related to content and the
4/10/2023 9
processes used to create the poster.
and the processes used to create the poster.
processes used to create the poster.
CommentComment Comment Comment
Assessment Instrument - Group Work - All Group Assignments
CategoryBelow Target At Target Above Target
ContributionsSeldom cooperative.Does little work.Rarely offers useful ideas.
Cooperative.Works at assign-ments.Usually offers useful ideas.
Always willing to help and do more.Does more than required.Routinely offers useful ideas.
CooperationRarely listens to, shares with, or supports the efforts of others.Often is not a good team member.
Usually listens to, shares with, and supports the efforts of others.Does not cause problems in the group.
Always listens to, shares with, and supports the efforts of others.Tries to keep people working together.
Focus on the Task Does not focus on the task and what needs to be done.Lets others do the work.
Focuses on the task and what needs to be done most of the time.
Almost always focused on the task and what needs to be done.Self-directed.
CommentComment Comment Comment
Assessment Instrument - Journal (Daily Entries Required)Category Below Target At Target Above Target
Organization Journal is sloppyand/or haphazardlyorganized.
Parts of the journal show organization,however some parts could be enhanced.
Journal contains achronological sectionas well as sections for sketches, reference sources, people, business contacts, etc.
Daily Entries Journal is missingmany daily entries.
Journal has daily entries with appropriate detail.
Details ofinformation gatheredand/or work accom-plished for each day is entered with high degree of quality and supplemental docu-mentation or verbage.
4/10/2023 10
Content Journal entries areinsufficiently descrip-tive to completelyrecreate the dailyaccomplishments.
Most information isdetailed, howevera few details may be missing.
Journal entries aresufficiently descriptive to completely recreatethe daily accomplish-ments.
ProperCitation ofJournals!Books!Videos!Websites
Some sources aremissing and othersources areincorrectly cited.
All information is documented and sources are correctly cited.
Has APA approveddocumented citations of all sources in thejournal and informa-tion is documented properly with high quality detail.
Drawingsand Sketches
Quantity ofsketches anddrawings areinsufficient toexplain the topic.
Sketches aredrawn explaining the topic adequately.
Journal containssketches anddrawings that arerelated to the topicand express what will be created.
ReferencingMaterials(Note Cards)
Note cards haveincomplete infor-mation and lack citations.
Note cards are adequately prepared.
Note cards containparaphrased infor-mation from source and cited reference with high level of detail.
PhoneConversationAbstracts
Missing information vital for calling backcontacts.
Most informationrequired is complete.
Phone conversations are documented for:contact, phonenumber, information,company, address,date.
BusinessContacts
Some contacts aremissing andinformation ismissing.
Has all contactswith sufficient detail.
Lists businesscontacts includingaddresses, phonenumbers, e-mails,company, faxnumbers, discussionexist exemplary detail.
CommentComment Comment Comment
Assessment Instrument - Multimedia Presentation for All Assigned Topics
CategoryBelow Target At Target Above Target
Content - Accuracy
Content confusing or contains more than one factual error.
Most content accurate but there is one piece of information that seems inaccurate.
All content throughout the presentation accurateNo factual errors.
Sequencing of No clear plan for the Most information is Information organ-
4/10/2023 11
Information organization of infor-mation.
organized in a clear, logical way.One slide or piece of information out of place.
ized in a clear, logical way. Easy to anticipate the next element.
Effectiveness Lacking several key elements and has inaccuracies.Completely inconsistent with driving question.
Consistent with driving question.Contains key elements.
Includes all material needed to give a good understanding of the topic.
Use of Graphics Graphics unattractive and detract from the content of the presentation.
A few graphics moderately done but all support the topic of the presentation.
All graphics attractive (size and colors) and support the topic of the presentation.
Text - Font Choice and Formatting
Difficult to read the text material.
Format carefully planned to enhance readability.
Formats (color, bold, italic) carefully planned to enhance readability and content.
Spelling and Grammar
More than 2 grammatical and/or spelling errors.
1-2 misspellings, but no grammatical errors.
No misspellings or grammatical errors.Excellent grammar!
Delivery Spoke a little faster or slower than necessary, or too quietly or loudly. Used unacceptable grammar. Failed to maintain eye-contact. Relied too much on their notes.
Spoke at a good rate. Volume appropriate.Good grammar.Maintained some eye-contact with audience.
Spoke at a good rate. Volume excellent for setting.Good grammar.Maintained eye-contact with audience.
CommentComment Comment Comment
Assessment Instrument - Oral Presentation for All Assigned Topics
CategoryBelow Target At Target Above Target
Organization Audience has difficulty following presentation because student jumps around.
Student presents information in logical sequence which audience can follow.
Student presents information in logical, interesting sequence which audience can follow.
Subject Knowledge Student is uncomfortable with information and is able to answer only rudimentary questions.
Student is at ease with expected answers to all questions, but fails to elaborate.
Student demonstrates full knowledge (more than required) by answering all class questions with
4/10/2023 12
explanations and elaboration.
Graphics Student occasionally uses graphics that rarely support text and presentation.
Student's graphics relate to text and presentation.
Student's graphics explain and reinforce screen text and presentation.
Mechanics Presentation has three or more misspellings and/or grammatical errors.
Presentation has no more than one misspelling and/or grammatical error.
Presentation has no misspellings or grammatical errors.
Eye Contact Student occasionally uses eye contact, but still reads most of report.
Student maintains eye contact most of the time but frequently returns to notes.
Student maintains eye contact with audience, seldom returning to notes.
Elocution Student's voice is low. Student incorrectly pronounces terms. Audience members have difficulty hearing presentation.
Student's voice is clear. Student pronounces most words correctly. Most audience members can hear presentation.
Student uses a clear voice and correct, precise pronunciation of terms so that all audience members can hear presentation.
Comment Comment Comment Comment
Assessment Instrument - Research Paper for All Assigned TopicsCategory Below Target At Target Above Target
Timeliness The student hassubmitted the finalversion of theresearch paper oneweek late.
The student hassubmitted the finalversion of theresearch paper on time.
The student has submitted the final version of the research paper early.
Content-Whole Paper
The student wasmissing multiplesections, orsections expectedto be separate werecombined.
The student ismissing nosections applicable to the research project.
The student's paper contains all content sections required by the teacher. Sections on Introduction, Identifying the Problem, InitialResearch, The Proposed Solution, Researching theSolution, Proto-typing, Testing, Evaluating test results and data, and a conclusion are most common.
4/10/2023 13
Content -Within EachSection
Large quantities ofinformation areplaced in sectionsinconsistent withthe purpose of thesection.
There are minoroccurrences whereinformation is placed in a section in which it does not belong, or sections do not refer to other sections where it would make sense to do so.
Each section of the paper contains the material appro-priate for that section.The sections are cross-referenced where necessary. No material is misplaced.Excellent content sections.
Organization Evidence oforganization, butwith serious flaws.Decisions discussed without backgroundinformation.
Within each section,material is presentedin a fairly wellorganized manner,but there are someitems which seemmisplaced. Eachsection still followsan outline.
Within each section, the material presented is very clearly organized.Discussion progresses along lines of logic.Readability is facilitated by crisp organization ofthoughts.
LanguageMechanics
Distractingly frequentoccurrences ofspelling, punctuation, and grammatical errors.
Minimal occurrences of spelling, punctuation, and grammatical errors.
Free of spelling,punctuation, andgrammatical errors.
TechnicalAccuracy
Numerous minortechnical flawsexist. Units areignored or misused.Overall credibilityof the author'sproficiency isdamaged.
Minor technicalflaws exist in words,units, or calculations, but these do not detract from the overall points made by the author.
The technical content of the paper is without flaw. Exacting care has been given to ensure facts, data, and results are stated in a technically correct manner. Numerical quantities are given proper units. Calculations are properly docu-mented and executed.
References Numerous minor orsome major errorsor omissions existin the referencingof others' work.
A few minor errorsor omissions exist inthe referencing ofothers' work.
Referenced material is annotated in proper AP A or MLA style consistently
4/10/2023 14
throughout. Works Cited page is in correct format.All items requiring citation are properly cited.
Appendices(if appropriate)
Some appendixInformation provided, but it is obviouslyincomplete.
Bulky amounts ofinformation ordrawings are included in the main body of the paper itself, rather than in the appendix.References missingor in error.
The paper includesAppropriate appendices tosupport the text of the paper. Appendices arereferenced in the text of the paper.
Comment Comment Comment Comment
Identify Resource Materials:Printed Materials -
Product Design and Development, Karl Ulrich, McGraw-Hill Design Concepts for Engineers, Mark Horenstein, Prentice Hall Fundamentals of Engineering Design, Barry Hyman, Prentice Hall Engineering Design, Rudolph Eggert, Prentice Hall Engineering by Design, Gerard Voland, Prentice Hall Engineering Design Methods, Nigel Cross, John Wiley Publisher Engineering Management, Challenges in the New Millennium, C.M. Chang,
Prentice Hall Managing Engineering and Technology, Daniel Babcock, Prentice Hall Design for Sixth Sigma, C. M. Creveling, Prentice Hall Project Risk Management, Bruce Barkley, McGraw-Hill Getting Started in Sixth Sigma, Michael Thomsett, John Wiley Publisher Engineering Robust Designs with Sixth Sigma, John Wang, Prentice Hall
Audiovisual Materials - NASA - 40 Years of Solutions CD/Video, Marshall Space Flight
Center, Technology Transfer Office, Huntsville, Alabama 35812 Introduction to Lean Manufacturing - Six Sigma
http://www.mfgeng.com
Print Materials - Product Design and Development, Karl Ulrich, McGraw-Hill Design Concepts for Engineers, Mark Horenstein, Prentice Hall Fundamentals of Engineering Design, Barry Hyman, Prentice Hall Engineering Design, Rudolph Eggert, Prentice Hall Engineering by Design, Gerard Voland, Prentice Hall Engineering Design Methods, Nigel Cross, John Wiley Publisher Engineering Management, Challenges in the New Millennium, C.M. Chang,
Prentice Hall
4/10/2023 15
Managing Engineering and Technology, Daniel Babcock, Prentice Hall Design for Sixth Sigma, C. M. Creveling, Prentice Hall Project Risk Management, Bruce Barkley, McGraw-Hill Getting Started in Sixth Sigma, Michael Thomsett, John Wiley Publisher Engineering Robust Designs with Sixth Sigma, John Wang, Prentice Hall
Audiovisual Materials -Introduction to Lean Manufacturing - Six Sigma
http://www.mfgeng.com
Internet Sites -Six Sigma
http://isixsigma.comhttp://isixsigma.com/library/content/c030317a.asp?action=printhttp://isixsigma.com/library/content/c050131a.asp?action=printhttp://jobs.isixsigma.com/sendmail.asp?ID=2072http://www.leanscm.net?Articles%20-%20September%2.../Six%20Sigma%20Management.html
Lean Management
http://childressconsulting.com/lean_management_system.htmhttp://maskell.com?LeanArticle.htmhttp://www.kiran.com/consulting_flm.asphttp://www.nwlean.net/home2.htmhttp://www.nwlean.net/article.htm
Project Management
http://www.ce.berkeley.edu/Courses/CE167/http://www.primavera.com/http://www.leeds.ac.uk/civil/pgopps/epm/epm.htmlhttp://construction.berkeley.eduhttp://www.aetsolar.com/Solar_Products_Services/Engineering_Project_ Management.htmhttp://www.supportengineering-online.com/discProjectMan.htmlhttp://ocw.mit.edu/OcwWeb/Mechanical-Engineering/2-96Manaagement-in-Enginee.../index.html
APPL’s project Management Development Processhttp://www.appl.nasa.gov/ask/issues/17/special/index.html
The Apollo 15 Flight Journalhttp://www.hq.nasa.gov/office/pao/History/ap15fj/index.htm
The Latest from the National Space Societyhttp://www.nss.org
4/10/2023 16
Working on the Moonhttp://www.thespacereview.com/article/436/1
Why Colonize the Moon First?http://www.universetoday.com/am/publish/why_moon_first.html?2132005
The ultimate public-private partnershiphttp://www.lasvegasmercury.com/2004/MERC-Jul-08-Thu-2004/24250261.html
The Space Settlement Initiativehttp://spacesettlement.org/
Purpose of the Lesson:Enable students to understand and apply the most contemporary NASA project management processes as part of the lunar exploration initiative.
Required Knowledge and Skills:Students should be able to graph linear, quadratic and exponential equations and construct or prepare charts and graphs with a variety of data points and other key information. Students should be able to use software applications (Office Suite) to prepare documents and spreadsheets as well as make highly informative, multi-media presentations. Students must be able to conduct a highly effective, efficient, and ethical Internet search. Students must be able to use tools, materials, and equipment safely or after review by instructor. In addition, students should be able to create mechanical drawings using CADD software.
Lesson: (5-E Model from Science)
Engagement:1. The instructor should lead a discussion with students on the issue of humans
living and working on the lunar surface. Usually, students are very interested in space travel and the many social, environmental, political and economic impacts that have been part of so many congressional and presidential debates as well as national television special productions. It seems everyone is curious about the ‘magic’ that NASA is able to perform with every mission or special project that involves humans safely traveling in space. It is suggested that students be presented with the following question:
Should humans be sent to live and work on the lunar surface or other planets?
It would be interesting to learn about the various student views on this topic. Charting student responses would be most beneficial and offer a way to review key points. Students should be challenged to state their opinion and offer as much supporting detail as possible to defend their point of view. A focus on social, environmental, economic, and political impacts might be worthy organizers for this discussion. The instructional team might offer the following
4/10/2023 17
NASA overview as an article to read by students. Once each student has read the article, students should be placed in small groups (2-3) and asked to analyze the various strategies presented in the article that explain exactly how NASA hopes to send humans once again to the moon safely and return them also. All groups will return as one class and each group will offer an analysis and their opinions or concerns regarding a return to the lunar surface- A good thing or not?
How We'll Get Back to the Moon
Before the end of the next decade, NASA astronauts will again explore the surface of the moon. And this time, we're going to stay, building outposts and paving the way for eventual journeys to Mars and beyond. There are echoes of the iconic images of the past, but it won't be your grandfather's moon shot.
This journey begins soon, with development of a new spaceship. Building on the best of Apollo and shuttle technology, NASA's creating a 21st century exploration system that will be affordable, reliable, versatile, and safe.
The centerpiece of this system is a new spacecraft designed to carry four astronauts to and from the moon, support up to six crewmembers on future missions to Mars, and deliver crew and supplies to the International Space Station.
The new crew vehicle will be shaped like an Apollo capsule, but it will be three times larger, allowing four astronauts to travel to the moon at a time.
The new spacecraft has solar panels to provide power, and both the capsule and the lunar lander use liquid methane in their engines. Why methane? NASA is thinking ahead, planning for a day when future astronauts can convert Martian atmospheric resources into methane fuel.
The new ship can be reused up to 10 times. After the craft parachutes to dry land (with a splashdown as a backup option), NASA can easily recover it, replace the heat shield and launch it again.
4/10/2023 18
Coupled with the new lunar lander, the system sends twice as many astronauts to the surface as Apollo, and they can stay longer, with the initial missions lasting four to seven days. And while Apollo was limited to landings along the moon's equator, the new ship carries enough propellant to land anywhere on the moon's surface. Once a lunar outpost is established, crews could remain on the lunar surface for up to six months. The spacecraft can also operate without a crew in lunar orbit, eliminating the need for one astronaut to stay behind while others explore the surface.
Safe and Reliable
The launch system that will get the crew off the ground builds on powerful, reliable shuttle propulsion elements. Astronauts will launch on a rocket made up of a single shuttle solid rocket booster, with a second stage powered by a shuttle main engine.
A second, heavy-lift system uses a pair of longer solid rocket boosters and five shuttle main engines to put up to 125 metric tons in orbit -- about one and a half times the weight of a shuttle orbiter. This versatile system will be used to carry cargo and to put the components needed to go to the moon and Mars into orbit. The heavy-lift rocket can be modified to carry crew as well.
Best of all, these launch systems are 10 times safer than the shuttle because of an escape rocket on top of the capsule that can quickly blast the crew away if launch problems develop. There's also little chance of damage from launch vehicle debris, since the capsule sits on top of the rocket.
The Flight Plan
In just five years, the new ship will begin to ferry crew and supplies to the International Space Station. Plans call for as many as six trips to the outpost a year. In the meantime, robotic missions will lay the groundwork for lunar exploration. In 2018, humans will return to the moon. Here's how a mission would unfold:
A heavy-lift rocket blasts off, carrying a lunar lander and a "departure stage" needed to leave Earth's orbit (below left). The crew launches separately (below, center), then docks their capsule with the lander and departure stage
4/10/2023 19
and heads for the moon (below, right).
Three days later, the crew goes into lunar orbit (below, left). The four astronauts climb into the lander, leaving the capsule to wait for them in orbit. After landing and exploring the surface for seven days, the crew blasts off in a portion of the lander (below, center), docks with the capsule and travels back to Earth. After a de-orbit burn, the service module is jettisoned, exposing the heat shield for the first time in the mission. The parachutes deploy, the heat shield is dropped and the capsule sets down on dry land (below, right).
'Into the Cosmos' With a minimum of two lunar missions per year, momentum will build quickly toward a permanent outpost. Crews will stay longer and learn to exploit the moon's resources, while landers make one way trips to deliver cargo. Eventually, the new system could rotate crews to and from a lunar outpost every six months.
Planners are already looking at the lunar south pole as a candidate for an outpost because of concentrations of hydrogen thought to be in the form of water ice, and an abundance of sunlight to provide power.
These plans give NASA a huge head start in getting to Mars. We will already have the heavy-lift system needed to get there, as well as a versatile crew capsule and propulsion systems that can make use of Martian resources. A lunar outpost just three days away from Earth will give us needed practice of "living off the land" away from our home planet, before making the longer trek to Mars.
As President Bush said when he announced the Vision for Space Exploration, "Humans are headed into the cosmos." Now we know how we'll get there.
4/10/2023 20
Students should be challenged to identify as many project management functions as possible. Using the graphic organizer below, have students complete a blank version and identify as many management functions as possible during a class brainstorming session led by the instructor.
4/10/2023 21
Once students have identified as many functions as possible, the instructor should assist with completing the chart by offering those items that were not identified by the students. Once a comprehensive list of all key management functions has been obtained, the instructor should require students to provide a brief definition for each of these using appropriate research techniques. Students should present their definitions as part of a class discussion with clarification of terms by the instructor.
It is important that students have an understanding for the historical development of project management. It is a process that has evolved over many years and morphed into a very sophisticated process of mathematically based techniques that also require the ability to understand people and performance motivation strategies. In effect, producing a blend of science and art in order to achieve an optimized control over numerous and diverse organizational functions.
In order for students to understand and appreciate the historical perspective for this subject, students should be challenged to research key people and events that have contributed to contemporary project management styles. Students should investigate the topics listed below and fill in the accomplishments section of this organizer:
4/10/2023 22
Historical Development of Management Theory and PracticesEra Persons or Events Accomplishments
Ancient Management Thoughts
The Great Wall in China, Pyramids of Egypt, monoliths on Easter Island, Mayan Temples in South America, Stonehenge in England
Involved management practices of coordination, control, and monitoring of many people over extended periods of time. (No records were available.)
Chinese Emperors (2350 B.C.) Practiced organizing, directing, and controlling.
Constitution of Chow (1100 B.C.) Organization chart for officials and craft specialization.
Persepolis in Persia (500 B.C.) Built the 2600-km Royal Road and set up message systems using horse riders.
Sun Tzu (500 B.C.) The art of war--planning and directing.Alexander the Great (336--332 B.C.) Practiced informal council with specific roles and
responsibilities to its members.India (321 B.C.) Practiced the concepts of government, commerce,
and custom.China (120 B.C.) Selected and classified officials by examinations
into nine specific grades.Production Practice (15th Century)
Arsenal of Venice (1436) Streamlined production process of outfitting ships, inventory control, standardization of specification, double-entry accounting, and cost control.
Industrial Revolution (18th Century)
Steam engine invented by James Watt (1769); other technological inventions
Factories are formed involving equipment and workers; destroyed the cottage industry in England; created problems related to child labor, poor living conditions for workers, crime, and brutality; induced the creation of factory layout planning, inventory control, production planning, work-flow analysis, and cost analysis.
Industrial Develop-ment in the United States (19th Century)
Railroads, textile mills, steel mills, and waterways were built
Charles Babbage (1792-1871) Advanced the concepts of division of labor, factory size optimization, profit-sharing scheme, method of observing manufactures, and the time-study method.
West Point Military Academy (1817) started teaching engineering and management; Norwich University (1819), Rensselear Polytechnic Institute (1823), Union College (1845), Harvard, Yale, and Michigan (1847)
Expansion of engineering and management education in the United States.
Morrill Land Grant Act (1862) Authorized federal land for each state to establish at least one college to teach "scientific and classical studies . . . agricultural and mechanical arts." The mechanical arts became engineering.
Formation of several associations: American Society of Engineering Education (1893), American Society of Mechanical Engineers (1880), and American Society of Civil Engineers (1982)
Promoted the exchange of best practices in engineering and management.
Scientific Manage-ment (20th Century)
Frederick Taylor (1856-1915) Pioneered the time-and-motion study to break down a complex job into elementary motions and find the most efficient procedure of doing the job. Taylor's study formed the corner stone of the discipline of industrial engineering.
Frank Gillbreth (1868-1924) and Lillian M. Gillbreth (1878-1972)
Pioneered the study of human factors in the workplace.
Gantt (1861-1919) Developed charts and graphed performance against time for project management.
Henri Fayol (1841-1925) Divided the industrial undertaking into six groups: technical (production), commercial (marketing), financial, security, accounting, and administrative activities (planning/forecasting, organization, command, coordination, and control).
Max Weber (1864-1920) Developed a model for rational and efficient organizations involving position charter, roles and responsibility, compensation policy, and others
Human Factors (20th Century)
Douglas M. McGregor (1906-1964) Developed Theory X and Theory Y.
William Ouchi (1943-) Developed Theory Z.
4/10/2023 23
Elton Mayo (1880-1945) and Fritz J. Roethlisberger (1898-1974)
Conducted extensive studies at Hawthorne Works near Cicero, Illinois, to study the impact of environmental, psychological, group factors, and other factors affecting workers productivity.
To complete engagement, students should be challenged to participate in the following task:
A collection of ball point pens (10) should be provided to production teams (4-5 students). Students should disassemble the pens and place all the same parts (springs, housings, button, plunger, etc.), in small cups or containers. Several objects can be used to accomplish this engagement activity. If ballpoint pens are not available, a simple flashlight can be used that offers multiple parts that can easily be disassembled and reassembled. This is also true for a simple penlight which in most cases has numerous and diverse parts. The images below provide assembly drawings showing such parts.
4/10/2023 24
4/10/2023 25
Exploded diagram of a handtorch.
Case
Button
Spring
Battery
Bulb
Cap
Assembly drawing of penlight showing standard and special-purpose components.
One early task for the group should be to analyze the device and generate a listing of all parts, with descriptions or a decomposition diagram as shown below for the penlight product:
Students should be required to organize themselves and reassemble the pens in the shortest time possible and ensure that all pens function properly. Student production teams should conduct several efficiency tests, producing key data that could impact organizational or management decisions.
The instructor should videotape the entire process as a class for future examination. Students should be required to plan and conduct appropriate efficiency tests, which should include all tables, charts, and graphs clearly showing results for such tests. It might be necessary for subassembly drawings and time charts to show the gains in efficiency with regard to disassembly and reassembly for the selected product.
4/10/2023 26
Product component decomposition diagram of a penlight.
Product
Subassembly A Subassembly B
Subassembly B1
Standardpart
Special-purpose part
Special purpose part
Standard part
Special-purpose part
Standard part
Special-purpose part
Product component decomposition diagram of a product having parts and subassemblies, both standard and special-purpose.
Each team should try to reach maximum efficiency with their production system. This should be presented as a class competition, with the winning team receiving some special extra credit or appropriate reward. Results should be posted for all students to examine. Students should document all special organizational techniques they used to create an efficient assembly process.
A final multi-media report and presentation should be made by each team clearly describing their final production method and organizational strategies. Students should also describe how they resolved critical production problems. This report should also include any key tables, charts, and graphs that help explain how the production team achieved their success. The instructor should lead a debriefing with the class to analyze, compare, and contrast each teams approach to this problem. Students should be required to compare and contrast each team’s approach using the new project management vocabulary acquired during the previous tasks in engagement.
Students should be required to prepare an Extended Constructed Response (ECR) report that explains, using management function terms, how they achieved improvement in production efficiency as a result of employing key strategies for organizing the work of the group. Final reports should be shared with the entire class. The instructor might lead a discussion as part of a closure component for this engagement. One goal is to see if students can identify major organizers for the work that was required to manage the entire process of disassembly and reassembly of the small scale product. Even with such a simple activity, there should have been significant gains in productivity and efficiency resulting from specific task or project management strategies. One thought is that the instructor should try to lead the group to such organizers reflected in recent project management literature and research such as the topics identified below:
Planning Organizing Controlling
The instructor should lead students in a final discussion on the topic of project management and the need to engineering organizations to participate in such endeavors in order to conduct work in an efficient way and remain competitive. The class should strive to write a final definition for project management that is comprehensive and helps students remain focused throughout this lesson. An example for such a definition is provided below. Project management is…
The systematic integration of technical, human, and financial resources to achieve goals and objectives.
4/10/2023 27
The image below is provided as an example for one way to represent the definition above. It might be a worthwhile activity for students to generate their own graphical representation for the definition determined by the instructor and students for each class.
Exploration:During engagement, students were left with a sense that organizing engineering work is a critical component for successful product development and overall competitiveness in the market. The simple definition and graphic representation activity offered students a way to understand in a rather simplistic sense what project management involves. However, it is very important that students explore in a more detailed way the intricate scope of authentic engineering project management.
In order to accomplish this, students should be challenged to investigate and generate a report with a detailed representation for all critical components within an engineering organization. This investigation should include contacting local or regional engineering companies to obtain examples of organizational representations. An example is provided below to guide this task.
4/10/2023 28
Interactions of functional groups in technology companies.
Corporate vice presidentand general manager
division D
Vice president
administration
Vicepresidentmarketing
Vice president
contracts and pricing
Vicepresidentfinance
Vice presidentresearch andengineering
Vicepresident
manufacturing
Vice president
quality control
Vice presidentproduct support
(logistics)
Technicalservices
Advancedproductplanning
Contractsmanagement
andsubcontracts
Budgetingand
analysis
Research Industrialengineering
Qualityassurance
Test andsupport
equipmentDraftingReproductionPublications,proposals, art
Library
Engineeringdesign Plant
engineeringand
maintenance
QualitycontrolMarketing
analysisGeneral
accountingSystem engineeringEngineering design Co
nceptual design
Preliminary
system design Det
ailed design
Design review
Technical dataPurchasing
Procurementqualitycontrol
Training andtraining
equipmentManagementservices
Manufacturingengineering
Policies andproceduresComputer support
Communications
CustomerfacilitiesProduction
operations
Engineeringsupport
Supply supportProduction
shopsReliabilityMaintainabilityHuman factors
Logistics engineeringOther support
Customer fieldservice
engineeringModel shop
BondingMachiningWeldingPainting
CalibrationOther shops
Industrial relations
Personneladministration
Modificationkits
Engineering andprototype model
development,test, and evaluation
Engineeringlaboratories
Engineering activities within a division of a large corporation.
This task can be accomplished as independent work or with small groups (2-3 students) depending on instructor preference. Students should present their findings with graphic examples to the entire class. Students should defend their analysis orally in a class seminar.
Students were able to address the concept of project management principles during engagement for this lesson by defining and describing some key principles associated with this topic. It is critical that students explore the key functions of engineering management. Students should be challenged to continue their research by investigating the following topics identified below. This task should be addressed by students in the following way: students should provide the following for each topic presented in the outline for ‘Functions of Engineering Management’:
Statement of definition or description of function elements Critical questions that need to be addressed by this function Practical example to illustrate each function Sources of all information must be sited.
Functions of Engineering Management:
1. Planning
Definition (Engineering Context)Articulating who will do what, how, where, when, and with which resources to enhance the effectiveness by providing focus and
4/10/2023 29
direction.
Types Strategic planning Operational planning
Tools for Planning SWOT analysis Financial what-if analysis and modeling Performance benchmarks Technology forecasting Product life-cycle analysis Qualitative and quantitative forecasting methods
Critical Questions (Examples) What are companies mission, vision, and value system? What business is the company to be in? What specific goals should the company accomplish? What new products or redesigned products should the company
offer? What core technologies should the company maintain, develop,
acquire, or utilize? What performance metrics are to be used for monitoring company
success? What is the most efficient way to accomplishing a project with known
objectives? What are the operational guidelines for performing specific work?
2. Organizing
Definition (engineering context)Arranging and relating work so that it can be done efficiently by appropriate people.
Legal Forms Sole Proprietorship Partnership Corporation
Types Line organization (business management, production, sales and
marketing, customer service) Staff organization (R&D, financial and accounting, information
technology procurement, legal affairs, public relations, facility engineering)
4/10/2023 30
Activities Develop an organizational structure Identify appropriate teams Identify span of control Identify specific responsibilities Identify specialization of work Identify work arrangement and relationships Delegating
Critical Questions (Examples) What work needs to be accomplished? Who should lead specific work? What organizational tools are available? What research data is available regarding organizational methods? How will work be organized? What leadership roles are needed? What is the relationship between departmental tasks?
3. Controlling
Definition (engineering context)Activities taken on by management to assess and regulate work in progress, evaluate results for securing maximum productivity, and reduce unacceptable performance.
Controlling Tasks Setting standards of performance Benchmarking (internal and external) Control methods/tools
Measuring Performance Collect, analyze, store, and report data Compare performance against established standards Generate reports Measurement tools (people, process, and product)
Evaluate PerformanceEstablish limits of toleranceNote deviations within tolerance limitsProvide recognition for good performancePerformance evaluation tools (people, process, and product)
Critical Questions (Examples) How should we measure the performance of employees?
4/10/2023 31
How should we correct poor performance of employees? How should we measure product performance? What standards should be used to establish performance measures? How should we measure efficiency of design process? How should we measure efficiency of production process? How do we correct performance in all areas? What are critical project control issues? How do we control product quality? How do we control employee performance?
Students should present their final report for the outline above with all appropriate supporting information; tables, charts, graphs, etc.
Managing Risk
One critical element in business management for engineering projects is analysis of ‘risk.’ In terms of project management, risk is defined as ‘uncertainty’ that has been defined. When estimating costs for engineering design or production costs, some costs are well defined, while others are not. Therefore, some projects in engineering are somewhat risk free, while others are not. Risk can also be described as a ‘measure of the potential variability of an outcome from its expected value’. Risks must be accounted for in projects. Risky events may be represented mathematically by the normal probability density function, which is defined by two parameters, standard deviation and mean. Besides the normal function, several other probability density functions may also be used to represent risky costs. These are represented by the images below:
Triangular probability density function. Normal probability density function.
4/10/2023 32
Beta probability density function. Poisson probability function.
The following discussion from NASA sources offers background information for the instructional team. One suggestion is that this review might be offered to students as assigned reading. Students could be assigned to small groups (2-3) and required to conduct an analysis of the major elements for Risk Management as performed by NASA and bring the groups together to share, discuss, and reach agreement on the key principles employed by NASA to ensure minimal risk with regards to aerospace engineering design work.
Continuous Risk Managementat NASA
Dr. Linda H. RosenbergUnisys @ NASA GSFC SATC
Bld 6 Code 300.1Greenbelt, MD 20771
v
Theodore HammerNASA GSFC
Bld 6 Code 302Greenbelt, MD 20771
Albert GalloUnisys @ NASA GSFC SATC
Bld 6 Code 300.1Greenbelt, MD 20771
Abstract
NPG 7120.5A, "NASA Program and Project Management Processes and Requirements" enacted in April, 1998, requires that "The program or project manager shall apply risk management principles…" The Software Assurance Technology Center (SATC) at NASA GSFC has been tasked with the responsibility for developing and
4/10/2023 33
teaching a systems level course for risk management that provides information on how to comply with this edict. This risk management structure of functions has been taught to projects at all NASA Centers and is being successfully implemented on many projects. The course was developed in conjunction with the Software Engineering Institute at Carnegie Mellon University, then tailored to the NASA systems community. This presentation will briefly discuss the six functions for risk management: (1) Identify the risks in a specific format; (2) Analyze the risk probability, impact/severity, and timeframe; (3) Plan the approach; (4) Track the risk through data compilation and analysis; (5) Control and monitor the risk; (6) Communicate and document the process and decisions.
Finally, the presentation will give project managers the information needed to implement Continuous Risk Management successfully at a cost they can afford.Introduction
Software risk management is important because it helps avoid disasters, rework, and overkill, but more importantly because it stimulates win-win situations. The objectives of software risk management are to identify, address, and eliminate software risk items before they become threats to success or major sources of rework. In general, good project managers are also good managers of risk. It makes good business sense for all software development projects to incorporate risk management as part of project management. NPG 7120.5A, the NASA guidebook for project managers, requires risk management applications and includes a section briefly discussing what should be included in a risk management plan. A course in continuous risk management was developed by the Software Engineering Institute at Carnegie Mellon University and has been adapted to NASA by the Software Assurance Technology Center (SATC) at NASA GSFC. The course was first taught in January, 1998, and has since been taught to over 300 students at all NASA centers.There are a number of definitions and uses for the term risk, but there is no universally accepted definition. What all definitions have in common is agreement that risk has two characteristics:
uncertainty: An event may or may not happen.loss: An event has unwanted consequences or losses.
Therefore, risk involves the likelihood that an undesirable event will occur, and the severity of the consequences of the event, should it occur. Risk management can: Identify potential problems and deal with them when it is easier and cheaper to
do so - before they are problems and before a crisis exists. Focus on the project’s objective and consciously look for things that may affect
quality throughout the production process. Allow the early identification of potential problems (the proactive approach) and
provide input into management decisions regarding resource allocation. Involve personnel at all levels of the project; focus their attention on a shared
product vision, and provide a mechanism for achieving it. Increase the chances of project success.
4/10/2023 34
At NASA, we focus on Continuous Risk Management that can be applied to any development process: hardware, software, systems, etc. It provides a disciplined environment for proactive decision making to: assess continually what could go wrong (risks) determine which risks are important to deal with implement strategies to deal with those risks assure, measure effectiveness of the implemented strategies
Risk management must not be allowed to become "shelfware". The process must be a part of regularly scheduled periodic product management. It requires identifying and managing risks routinely throughout all phases of the project's life. The paradigm shown in Figure 1 illustrates the set of continuous risk management functions throughout the life cycle of a project. These functions serve as the foundation for the application of continuous risk management. Each risk nominally goes through these functions sequentially, but the activity occurs continuously, concurrently, and iteratively. Risks are usually tracked in parallel while new risks are identified and analyzed, and the mitigation plan for one risk may yield another risk.
Figure 1: Continuous Risk Management Principle Functions
Continuous Risk Management Principle Functions
1 - Identify
The purpose of identification is to consider risks before they become problems and to incorporate this information into the project management process. Anyone in a project can identify risks to the project. Each individual has particular knowledge about various parts of a project. During Identify, uncertainties and issues about the project are transformed into distinct (tangible) risks that can be described and measured.
During this function, all risks are written with the same, two part format. The first part is the risk statement, written as a single statement concisely specifying the cause of the
4/10/2023 35
concern as well as its impact. The second part may contain additional supporting details in the form of a context.
The aim for a risk statement is that it be clear, concise, and sufficiently informative that the risk is easily understood. Risk statements in standard format must contain two parts: the condition and the consequence. The condition-consequence format provides a complete picture of the risk, which is critical during mitigation planning. It is read as follows:
given the <condition> there is a possibility that <consequence> will occur
The condition component focuses on what is currently causing concern; it must be something that is true or widely perceived to be true. This component provides information that is useful when determining how to mitigate a risk. The consequence component focuses on the intermediate and long-term impact of the risk. Understanding the depth and breadth of the impact is useful in determining how much time, resources, and effort should be allocated to the mitigation effort. A well-formed risk statement usually has only one condition, but may have more than one consequence.
Risk statements should avoid: abbreviations/acronyms that are not readily understood sweeping generalizations massive, irrelevant detail
Since the risk statement is to be concise, a context is added to provide enough additional information about the risk to ensure that the original intent of the risk can be understood by other personnel, particularly after time has passed. An effective context captures the what, when, where, how, and why of the risk by describing the circumstances, contributing factors, and related issues (background and additional information that are NOT in the risk statement).
A diagram of the complete risk statement and context are shown in Figure 2.
Figure 2: Risk Statement and Context
An example is shown in Figure 3. Note there is one condition and two consequences in the risk statement. The context explains why this is a risk.
4/10/2023 36
Figure 3: Example Risk Statement and Context
Risk identification depends heavily on both open communication and a forward-looking view to encourage all personnel to bring forward new risks and to plan beyond their immediate problems. Although individual contributions play a role in risk management, teamwork improves the chances of identifying new risks by allowing personnel to combine their knowledge and understanding of the project.
2 - Analyze
The purpose of Analyze is to convert the data into decision-making information. Analysis is a process of examining the risks in detail to determine the extent of the risks, how they relate to each other, and which ones are the most important. Analyzing risks has three basic activities: evaluating the attributes of the risks (impact, probability, and timeframe), classifying the risks, and prioritizing or ranking the risks.
Evaluating - The first step provides better understanding of the risk by qualifying the expected impact, probability, and timeframe of a risk. This involves establishing values for:
Impact: the loss or negative affect on the project should the risk occur
Probability: the likelihood the risk will occur
Timeframe: the period when you must take action in order to mitigate the risk
4/10/2023 37
Figure 4 demonstrates sample values that might be used to evaluate a risk's attributes
Attribute Value Description
Probability Very Likely (H)
Probable (M)
Improbable (L)
High chance of this risk occurring, thus becoming a problem > 70%
Risk like this may turn into a problem once in a while {30% < x < 70%}
Not much chance this will become a problem {0% < x < 30%}
Impact Catastrophic (H)
Critical (M)
Marginal (L)
Loss of system; unrecoverable failure of system operations; major damage to system; schedule slip causing launch date to be missed; cost overrun greater than 50% of budget
Minor system damage to system with recoverable operational capacity; cost overrun exceeding 10% (but less than 50% of planned cost
Minor system damage to project; recoverable loss of operational capacity; internal schedule slip that does not impact launch date cost overrun less than 10% of planned cost
Timeframe Near-term (N)
Mid-term (M)
Far-term (F)
Within 30 days
1 to 4 months from now
more than 4 months from now
NOTE: refers to when action must be taken
Figure 4: Sample Attribute Values
Classifying - The next step is to classify risks. There are several ways to classify or group risks. The ultimate purpose of classification is to understand the nature of the risks facing the project and to group any related risks so as to build more cost-effective mitigation plans. The process of classifying risks may reveal that two or more risks are equivalent - the statements of risk and context indicate that the subject of these risks is the same. Equivalent risks are therefore duplicate statements of the same risk and should be combined into one risk.
4/10/2023 38
Prioritize - The final step in the Analysis function is to prioritize the risks. The purpose is to sort through a large number of risks and determine which are most important and to separate out which risks should be dealt with first (the vital few risks) when allocating resources. This involves partitioning risks or groups of risks based on the "vital few" sense and ranking risks or sets of risks based on consistently applying an established set of criteria. No project has unlimited resources with which to mitigate risks. Thus, it is essential to determine consistently and efficiently which risks are most important and then to focus those limited resources on mitigating risks.
Conditions and priorities will change during a project, and this natural evolution can affect the important risks to a project–. Risk analysis must be a continual process. Analysis requires open communication so that prioritization and evaluation is accomplished using all known information. A forward-looking view enables personnel to consider long-range impacts of risks.
3 - Plan
Planning is the function of deciding what, if anything, should be done about a risk or set of related risks. In this function decisions and mitigation strategies are developed based on current knowledge of project risks.
The purpose of plan is to: make sure the consequences and the sources of the risk are known develop effective plans plan efficiently (only as much as needed or will be of benefit) produce, over time, the correct set of actions that minimize the impacts of risks
(cost and schedule) while maximizing opportunity and value plan important risks first
Figure 5 indicates the potential approaches to Risk Planning.
Figure 5: Planning Approaches
4/10/2023 39
There are four options to consider when planning for risks: 1. Research: establish a plan to research the risk(s) 2. Accept: decide to "accept" the risk(s) and document the rationale behind the
decision 3. Watch: monitor risk conditions for any indications of change in probability or
impact (tracking metrics must be established and documented) 4. Mitigate: allocate resources and assign actions in order to reduce the probability
or potential impact of risks. This can range from simple tasking to sweeping activities:
Action Items: a series of discrete tasks to mitigate risk
Task Plan: formal, well-documented and larger in scope
Dealing with risk is a continuous process of determining what to do with new concerns as they are identified and efficiently utilizing project resources. An integrated approach to management is needed to ensure mitigation actions do not conflict with project or team plans and goals. A shared product vision and global perspective are needed to create mitigation actions on the macro level to the benefit the project, customer and organization. The focus of risk planning is to be forward looking, to prevent risks from becoming problems. Teamwork and open communication enhance the planning process by increasing the amount of knowledge and expertise that can be applied to the development of mitigating actions.
4 - Track
Tracking is the process by which risk status data are acquired, compiled, and reported
The purpose of Track is to collect accurate, timely, and relevant risk information and to present it in a clear and easily understood manner to the appropriate people/group. Tracking is done by those responsible for monitoring "watched" or "mitigated" risks. Tracking status information become critical to performing the next function in the Continuous Risk Management paradigm, i.e. Control. Supporting information, such as schedule and budget variances, critical path changes, and project/performance indicators can be used as triggers, thresholds, and risk- or plan-specific measures where appropriate.
When a mitigation plan has been developed for a risk or risk set, both the mitigation plan and the risk attributes are tracked. Tracking the mitigation plan, or even a list of action items, will indicate whether the plan is being executed correctly and/or on schedule. Tracking any changes in the risk attributes will indicate whether the mitigation plan is reducing the impact or probability of the risk. In other words, tracking risk attributes gives an indication of how effective the mitigation plan is.
4/10/2023 40
Program and risk metrics provide decision makers with the information needed for making effective decisions. Normally program metrics are used to assess the cost and schedule of a program as well as the performance and quality of a product. Risk metrics are used to measure a risk’s attributes and assess the progress of a mitigation plan. They can also be used to help identify new risks.
Example: A program metric might look at the rate of module completion. If this metric indicates that the rate of completion is lower than expected, then a schedule risk should be identified.
Open communication regarding risk and mitigation status stimulates the project and risk management process. Tracking is a continuous process - current information about a risk status should be conveyed regularly to the rest of the project. Risk metrics provide decision makers with the information needed for making effective decisions.
5 - Control
The purpose of the Control function is to make informed, timely, and effective decisions regarding risks and their mitigation plans. It is the process that takes in tracking status information and decides exactly what to do based on the reported data. Controlling risks involves analyzing the status reports, deciding how to proceed, and then implementing those decisions.
Decision makers need to know 1) when or whether there is a significant change in risk attributes and 2) the effectiveness of mitigation plans within the context of project needs and constraints. The goal is to obtain a clear understanding of the current status of each risk and mitigation plan relative to the project and then to make decisions based on that understanding. Tracking data is used to ensure that project risks continue to be managed effectively and to determine how to proceed with project risks. Options include: Replan - A new or modified plan is required when the threshold value has been
exceeded, analysis of the indicators shows that the action plan is not working, or an unexpected adverse trend is discovered.
Close the risk - A closed risk is one that no longer exists or is no longer cost effective to track as a risk. This occurs when: the probability falls below a defined threshold, impact lies below a defined threshold, or the risk has become a problem and is tracked.
Invoke a contingency plan - A contingency plan is invoked when a trigger has been exceeded or some other related action needs to be taken.
Continue tracking and executing the current plan - No additional action is taken when analysis of the tracking data indicates that all is going as expected or project personnel decide to continue tracking the risk or mitigation plan as before.
4/10/2023 41
Open communication is important for effective feedback and decision making - a critical aspect of Control. Risk control is also enhanced through integrated management - combining it with routine project management activities enables comprehensive project decision making.
6 - Communication & Documentation
The purpose of Communicate and Document is for all personnel to understand the project’s risks and mitigation alternatives as well as risk data and to make effective choices within the constraints of the project. Communication and Documentation are essential to the success of all other functions within the paradigm and is critical for managing risks.
Identify: In risk identification, risk statements are communicated.
Analyze: In analysis, project personnel communicate information about impact, probability, and timeframe attributes. Risk classification involves grouping risk information communicated by individuals.
Plan: During planning, action plans are developed and communicated to project personnel.
Track: Reports designed to communicate data to decision-makers are compiled during tracking.
Control: The decisions made during control must be communicated and recorded to project personnel.
For effective risk management, an organization must have open communication and formal documentation. Communication of risk information is often difficult because the concept of risk comprises two subjects that people don’t normally deal well with: probability and negative consequences.
Not only Continuous Risk Management, but the project as a whole are in jeopardy when the environment is not based on open communication. No one has better insight into risks than project personnel, and management needs that input. Experienced managers know that the free flow of information can make or break any project. Open communication requires: Encouraging free-flowing information at and between all project levels Enabling forma, informal and impromptu communication Using consensus-based processes that value the individual voice, bringing
unique knowledge and insight to identifying and managing risks.
4/10/2023 42
NASA Risk Management Course
Risk is a daily reality on all projects, and Continuous Risk Management should become just as routine. It should be ongoing and comfortable and neither imposed nor forgotten. Like any good habit, it should seamlessly fit into the daily work. During the course taught at NASA, various tools and methods are demonstrated that will work for any project. The key is to adhere to the principles, perform the functions, and adapt the practice to fit the project's needs. Continuous risk management is not "one size fits all". To be effective, tailoring is needed. Tailoring occurs when organizations adapt the processes, select methods and tools which best fit their project management practice and their organizational culture. Following the principles of the continuous risk management is the key to successful tailoring.
With this in mind, the Continuous Risk Management course for NASA was tailored to 2 days. The first day is lecture, covering all material with some exercises applying the methods and tools. This is a very intense day since there is a lot of information to absorb. The second day is devoted to a "project" workshop. In most classes, personnel from one or two projects attend the lecture then split for the workshop (Classes are limited to 20 students.) The workshop is done in small groups, periodically these groups come together to review what each group has chosen to work on. (It is interesting as the instructor to observe the similarities in their results.) Depending on the audience, there are two possible workshops, one for management and the other for the implementation team.
The workshop for management starts by compiling the project information needed for the risk management plan. This starts with getting the functional organizational chart, identifying key meetings where risk management activities should take place, and identifying key personnel. The methods and tools to be used are then selected, and the criteria for the attributes probability, impact and timeframe are defined. This usually takes 2-3 hours. A shortened version of the implementation workshop described below is then applied.
The implementation workshop starts by identifying risks to the project based on everyone's knowledge. Phrases are used, with brainstorming to get a list of over 20 potential risks. It is stressed that if it is a problem now, it is not a risk. From this list 5 risks are identified as those the group feels they can do something about and would like to work on. The risks are then written using the correct format of condition and consequence. The risk context is discussed but not written. Using these 5 risks and the attribute definitions from management, the risks are classified and prioritized. A mitigation plan for the top risk is developed, data for tracking is identified and presentation formats discussed. Depending on time, 2 or 3 risks are processed through this cycle so that the attendees not only feel comfortable with the process, they have some risks specific to their project that they can start working on. Based on course feedback, it is believed the workshop is the key to the success of this training.
4/10/2023 43
When a class is not all from the same project, either the group is told to make up a project based on common experience, or use a project that many of them are familiar with. The second option is encouraged so real work is actually accomplished although it only benefits a few of the attendees.
After completion of the course, students should: Understand the concepts and principles of Continuous Risk Management and
how to apply them Possess basic risk management skills for each function of the risk management
paradigm Be able to use the key methods and tools Be able to tailor CRM to a project or organization
Implementation
Three steps should be considered by projects when implementing risk management. First, project risk management should be arranged. The training in itself is not important, it is what the training does for the project. The training helps the project to see how a formal process can be used to manage risks, but more importantly facilitate communication and initial brainstorming among project personnel. Second, the project should adopt tools that they are familiar with to aid in the tracking of risks and communication of risk status. The key is that the project use tools that they know how to use and that they will use. Lastly, the risk management process needs to be integrated into the normal project management process. Risk management must become the normal way of doing project business. This will ensure that rather than a separate process requiring extra overhead, risk management is ingrained in the project. This will lead to a cost-effective implementation within the project.
Conclusion
Most project managers agree that risk management works, but the difficulty lies in actually implementing it, even when required to do so. The risk management plan is often hastily written and then thrown in a corner to gather dust. In addition to the course, one of the steps NASA has taken is to establish a risk management web site (http://satc.gsfc.nasa.gov) that contains sample risk management plans and a schedule of classes. Much time is spent discussing with managers the benefits of taking a formal training course, which is more than recovered by a project when all team members are working toward common goals in a coordinated manner.
"Continuous Risk Management at NASA" was presented at the Applied Software Measurement / Software Management Conference, February 1999, San Jose, California.
4/10/2023 44
Students should be challenged to investigate the concept of risk management in engineering and offer a definition and description of ways in which this is addressed quantitatively and qualitatively. Students should also offer an authentic engineering example from their research showing the use of at least one input distribution function presented in the images above.
In addition, students should analyze the two functions presented below and the cost estimation chart for a capital project. What results can be described?
Normal probability density function for total project cost. Cumulative distribution function for total project cost.
Beta probability density function.
The following are presented as examples of results that can be offered by student analysis:
The most likely total project cost is $5,136,000. There is an 80% probability that the project cost will exceed $5,100,000. There is a 20% probability that the project cost will exceed $5,170,000. The maximum project cost is $5,250,000. The minimum project cost is $4,989.71.
Students can work in teams (2-3) to address the research topic and the analysis of the capital project risk sample and present their work as part of a final report to the entire class.
Students should identify at least six specific examples of how 'Risk Manage-ment' was accomplished by NASA with Apollo missions.
4/10/2023 45
Another key management function is the use of evaluation matrices to perform job performance evaluations. In order to introduce this concept to students, the rating scale for cartoon heroes is presented below. Students should be challenged to identify the correct cartoon hero for each rating. This humorous example is provided to guide student thinking and research to locate several ‘authentic’ examples of personnel rating systems used in engineering organizations. Samples from Internet searches as well as local or regional businesses should be gathered by student groups and presented as part of class seminar.
Rating Scale for Cartoon HeroesPerformance
FactorsFar Exceeds
RequirementsExceeds
RequirementsMeets
RequirementsNeeds
ImprovementDoes Not MeetRequirements
Quality Leaps tall buildings with a single bound
Must take a running start to leap over tall buildings
Can leap only over short buildings
Crashes into buildings when attempting to leap over them
Cannot recognize buildings at all
Timeliness Is faster than a speeding bullet
Is as fast as a speeding bullet
Not quite as fast as a speeding bullet
Would you believe a slow bullet
Wounds self with bullet when attempting to shoot
Initiative Is stronger than a locomotive
Is stronger than a bull elephant
Is stronger than a bull
Takes bull by the horns
Shoots the bull
Ability Walks on water consistently
Walks on water in emergencies
Washes with water
Drinks water Has water on the knee
Communications Talks with God Talks with angels
Talks to himself Argues with himself
Loses those arguments
Another industry standard tool for project management is the use of PERT and Gantt charting methods. These are very useful and accepted flow diagrams that illustrate exactly what tools and best practices are going to be used within each phase of technology development or product design. Students should be challenged to investigate both of these management tools and provide the following information:
Definition Software tools available to generate these two tools (Microsoft Project,
Microsoft Excel, Crystal Ball Monte Carlo Simulator, etc.) Authentic examples for each Define terms: (critical path, total cycle-time, phase, Integrated Program
Plan (IPP), nominal control, statistical control, Upper and Lower Specification Limits (USL, LSL).
Describe what it means when PERT charts are linked in phase and show a completed example from engineering design project similar to the blank template provided below:
4/10/2023 46
PERT Charts Linked in Phase
Identifying the Critical Path
Below is an example of a Gantt chart depicting a product development process schedule. Gantt charts offer a detailed method for disciplined scheduling and tracking of work and resources enhancing efficiency and product success from design to production.
Students should locate examples of 'NASA' charts and identify as Gantt or Pert or 'Other" configuration and explain how it is being used.
4/10/2023 47
ID Task Name Duration Start Finish Prede%
Complete1 Stage 1 Systems Design and Requirements Definition 195 days Fri 3/3/00 Fri 12/1/00 33%2 Program management 136 days Fri 3/3/00 Mon 9/11/00 95%3 Detailed program schedule 8 days Fri 3/3/00 Fri 3/3/00 100%4 Manpower planning 6 days Fri 3/3/00 Fri 3/3/00 100%5 Schedule management review 4 days Tue 8/13/00 Fri 6/16/00 100%6 Program plan (internal) 2 days Mon 6/19/00 Tues 6/20/00 100%7 Schedule baseline established 1 day Mon 9/11/00 Mon 9/11/00 8, 5 0%8 System engineering 56 days Mon 8/28/00 Tue 11/14/00 60%
24 Reliability engineering 10 days Mon 9/11/00 Fri 9/22/00 0%27 Safety engineering 50 days Mon 9/25/00 Fri 12/1/00 0%32 Stage 2 Detailed Design 302 days Fri 11/19/99 Tue 1/16/01 44%33 Electronics design and development 71 days Mon 6/19/00 Tue 9/26/00 72%34 Attitude/revert panel redesign 16 days Tue 9/5/00 Tue 9/26/00 52%35 Mechanical specification 2 days Mon 9/11/00 Tue 9/12/00 100%36 Procure 2 wks Wed 9/13/00 Tue 9/26/00 35 100%37 Validate 11 days Tue 9/5/00 Tue 9/19/00 0%38 Tester design and fabricate 2 wks Tue 9/5/00 Mon 9/18/00 0%39 Evaluate panel 1 day Tue 9/19/00 Tue 9/19/00 38 0%40 RDR-1E/F 63 days Mon 6/19/00 Thu 9/14/00 87%41 Electrical design changes 1 wk Mon 6/19/00 Fri 6/23/00 100%42 Incorporate lightning/EMI mods 5 days Mon 6/26/00 Fri 6/30/00 41 100%43 Procure PWB 3 wks Wed 7/5/00 Tue 7/25/00 42 100%44 Build CCA 2 days Wed 8/23/00 Thu 8/24/00 100%45 Verification and test 1 day Mon 9/11/00 Mon 9/11/00 44 0%46 Integrate four CCAs in chassis 3 days Tue 9/12/00 Thu 9/14/00 45 0%47 Software requirements documentation 235 days Tue 2/1/00 Tue 12/26/00 27%
Tracking Gantt chart.
In 2004, President George Bush declared that our nation should revisit the mission to reach the moon and place humans there to live and work. It was this presidential charge that has provided direction for NASA’s work in the early 21st century.
However, our nation already visited the moon during 1969 and subsequently during the 1970’s. Students may not be aware of the APOLLO missions and the incredible history associated with one of the great NASA success stories.
In 1962, President John Kennedy made a similar charge that led our nation and NASA to plan, organize, and control sophisticated and diverse engineering work in order to transport humans to the lunar surface and return them safely. The APOLLO missions were very successful and are worthy research for all students interested in engineering.
It is suggested that students be divided into teams and each team assigned one of the APOLLO missions to investigate using numerous resources and prepare a multi-media presentation on each mission. Each group will become the ‘consultants’ for their respective APOLLO mission. Students should be required to compare and contrast each mission, articulating successes and any failures that may have occurred during each mission. These should be
4/10/2023
January 14, 2004 --Indeed it is the nature of humanity to explore beyond our horizons. Humanity explores in order to discover, discovers in order to gain new knowledge, and gains new knowledge to enhance the quality of life for itself. It has been by looking beyond current horizons that human civilization advances.
~ President George W. Bush
48
recorded and organized so that the entire class can review and discuss the overall APOLLO mission accomplishments. These presentations should be comprehensive and offer substantial detail that clearly articulates the scope of the specific mission, including, but not limited to the following: Astronauts involved Scientific experiments conducted Engineering Milestones achieved Length of mission (time, distance, etc.) Payload details (human and scientific cargo) Launch details (thrust, propulsion data, ratios) Lunar landings (details and activities on lunar surface) Major discoveries or revelations All critical mission dates (day by day review) Recovery operations (all details) Debriefing status (post-mission data) Mission failures (low-high status) New technologies demonstrated (per mission)
As part of project management review, students should read the following excerpt from NASA documents on project management and continue an investigation on how the APOLLO missions were managed with structural management details and processes. This sample description of key processes is just one example of many layers of project management employed by NASA currently. An interesting challenge for students is to compare and contrast management evolution from the 1960’s and 70’s APOLLO MISSIONS to with current techniques and methods-
These principles are listed here for clarity: (1) plan all work scope for the project to completion; (2) break down the project work scope into finite pieces that can be
assigned to a responsible person or organization for control of technical, schedule, and cost objectives;
(3) integrate project work scope, schedule, and cost objectives into a performance measurement baseline plan against which accomplishments may be measured and control changes to the baseline;
(4) use actual costs incurred and recorded in accomplishing the work performed;
(5) objectively assess accomplishments at the work performance level; (6) analyze significant variances from the plan, forecast impacts, and
prepare an estimate at completion based on performance to date and work to be performed; and
(7) incorporate Earned Value Management into the projects decision-making and review processes.
(Sample NASA Project Management Overview - 2005)
4/10/2023 49
Technology Readiness Levels is a key concept and process used by NASA as part of project management at all agencies. The following is provided as background for the instructional team and may be addressed via lecture/ discussion with students or as preferred by the instructional team.
TECHNOLOGY READINESS LEVELSJohn C. Mankins
Advanced Concepts OfficeOffice of Space Access and Technology
NASA
Introduction
Technology Readiness Levels (TRLs) are a systematic metric/measurement system that supports assessments of the maturity of a particular technology and the consistent comparison of maturity between different types of technology. The TRL approach has been used-on-and-off in NASA space technology planning for many years and was recently incorporated in the NASA Management Instruction (NMI7100) addressing integrated technology planning at NASA - Figure 1 (attached) provides a summary view of the technology maturation process model for NASA space activities for which the TRL’s were originally conceived; other process models may be used. However, to be most useful the general model must include:
(a) ‘basic’ research in new technologies and concepts (targeting identified goals, but not necessary specific systems),
(b) focused technology development addressing specific technologies for one or more potential identified applications,
(c) technology development and demonstration for each specific application before the beginning of full system development of that application,
(d) system development (through first unit fabrication), and (e) system ‘launch’ and operations.
Technology Readiness Levels Summary
TRL 1 Basic principlesTRL 2 Technology concept and/or application formulatedTRL 3 Analytical and experimental critical function and/or characteristic
proof-of-conceptTRL 4 Component and/or breadboard validation in laboratory
environmentTRL 5 Component and/or breadboard validation in relevant environmentTRL 6 System/subsystem model or prototype demonstration in a
relevant environment (ground or space)TRL 7 System prototype demonstration in a space environment
4/10/2023 50
TRL 8 Actual system completed and “flight qualified” through test and demonstration (ground and space)
TRL 9 Actual system “flight proven” through successful mission operations
Discussion of Each Level
The following paragraphs provide a descriptive discussion of each technology readiness level, including an example of the type of activities that would characterize each TRL.
TRL 1Basic principles observed and reported
This is the lowest “level” of technology maturation. At this level, scientific research begins to be translated into applied research and development. Examples might include studies of basic properties of materials (e.g., tensile strength as a function of temperature for a new fiber).
Cost to Achieve: Very Low ‘Unique’ Cost(investment cost is borne by scientist research programs)
TRL 2Technology concept and/or application formulated
Once basic physical principles are observed, then at the next level of maturation, practical applications of those characteristics can be ‘invented’ or identified. For example, following the observation of High Critical Temperature (HTC) superconductivity, potential applications of the new material for thin film devices (e.g., SIS mixers) and in instrument systems (e.g., telescope sensors can be defined). At this level, the application is still speculative: there is not experimental proof or detailed analysis to support the conjecture.
Cost to Achieve: Very Low ‘Unique’ Cost(investment cost is borne by scientist research programs)
TRL 3Analytical and experimental critical function and/or characteristic proof-of-concept
At this step in the maturation process, active research and development (R&D) is initiated. This must include both analytical studies to set the technology into an appropriate context and laboratory-based studies to physically validate that the analytical predictions are correct. These studies and experiments should constitute “proof-of-concept” validation of
4/10/2023 51
the applications/concepts formulated at TRL 2. For example, a concept for High Energy Density Matter (HEDM) propulsion might depend on slush or super-cooled hydrogen as a propellant: TRL 3 might be attained when the concept-enabling phase/temperature/pressure for the fluid was achieved in a laboratory.
Cost to Achieve: Low ‘Unique’ Cost(investment cost is borne by scientist research programs)
TRL 4Component and/or breadboard validation in laboratory environment
Following successful “proof-of-concept” work, basic technological elements must be integrated to establish that the “pieces” will work together to achieve concept-enabling levels of performance for a component and/or breadboard. This validation must be devised to support the concept that was formulated earlier, and should also be consistent with the requirements of potential system applications. The validation is relatively “low-fidelity” compared to the eventual system: it could be composed of ad hoc discrete components in a laboratory. For example, a TRL 4 demonstration of a new ‘fuzzy logic’ approach to avionics might consist of testing the algorithms in a partially computer-based, partially bench-top component (e.g., fiber optic gyros) demonstration in a controls lab using simulated vehicle inputs.
Cost to Achieve: Low-to-moderate ‘Unique’ Cost(investment will be technology specific, but probably several factors greater than investment required for TRL 3)
TRL 5Component and/or breadboard validation in relevant environment
At this, the fidelity of the component and/or breadboard being tested has to increase significantly. The basic technological elements must be integrated with reasonably realistic supporting elements so that the total applications (component-level, sub-system level, or system-level) can be tested in a ‘simulated’ or somewhat realistic environment. From one-to-several new technologies might be involved in the demonstration. For example, a new type of solar photovoltaic material promising higher efficiencies would at this level be used in an actual fabricated solar array ‘blanket’ that would be integrated with power supplies, supporting structure, etc., and tested in a thermal vacuum chamber with solar simulation capability.
Cost to Achieve: Moderate ‘Unique’ Cost
4/10/2023 52
(investment will be technology dependent, but likely to be several factors greater that cost to achieve TRL 4)
TRL 6System/subsystem model or prototype demonstration in a relevant environment (ground or space)
A major step in the level of fidelity of the technology demonstration follows the completion of TRL 5. At TRL 6, a representative model or prototype system or system - which would go well beyond ad hoc, ‘patch-cord’ or discrete component level bread boarding - would be tested in a relevant environment. At this level, if the only ‘relevant environment’ is the environment of space, and the model/prototype must be demonstrated in space. Of course, the demonstration should be successful to represent a true TRL 6. Not all technologies will undergo a TRL 6 demonstration: at this point the maturation step is driven more by assuring management confidence than by R&D requirements. The demonstration might represent an actual system application, or it might only be similar to the planned application, but using the same technologies. At this level, several-to-many new technologies might be integrated into the demonstration. For example, a innovative approach to high temperature/low mass radiators, involving liquid droplets and composite materials, would be demonstrated to TRL 6 by actually flying a working, sub-scale (but scaleable) model of the system on a Space Shuttle or International Space Station ‘pallet’. In this example, the reason space is the ‘relevant’ environment is that microgravity plus vacuum plus thermal environment effects will dictate the success/failure of the system - and the only way to validate the technology is in space.
Cost to Achieve: Technology and demonstration specific; a fraction of TRL 7 if on ground; nearly the same if space is required
TRL 7System prototype demonstration in a space environment
TRL 7 is a significant step beyond TRL 6, requiring an actual system prototype demonstration in a space environment. It has not always been implemented in the past. In this case, the prototype should be near or at the scale of the planned operations system and the demonstration must take place in space. The driving purposes for achieving this level of maturity are to assure system engineering and development management confidence (more than for purposes of technology R&D). Therefore, the demonstration must be of a prototype of that application. Not all technologies in all systems will go to this level. TRL 7 would normally only be performed in cases where the technology and/or subsystem application
4/10/2023 53
is mission critical and relatively high risk. Example: the Mars Pathfinder Rover is a TRL 7 technology demonstration for future Mars micro-rovers based on that system design. Example: X-vehicles are TRL 7, as are the demonstration projects planned in the New Millennium spacecraft program.
Cost to Achieve: Technology and demonstration specific, but a significant fraction of the cost of TRL 8 (investment = “Phase C/D to TFU” for demonstration system)
TRL 8Actual system completed and “flight qualified” through test and demon-stration (ground or space)
By definition, all technologies being applied in actual systems go through TRL 8. In almost all cases, this level is the end of true ‘system development’ for most technology elements. Example: this would include DDT&E through Theoretical First Unit (TFU) for a new reusable launch vehicle. This might include integration of new technology into an existing system. Example: loading and testing successfully a new control algorithm into the onboard computer on Hubble Space Telescope while in orbit.
Cost to Achieve: Mission specific; typically highest unique cost for a new technology (investment = “Phase C/D to TFU” for actual system)
TRL 9Actual system “flight proven” through successful mission operations
By definition, all technologies being applied in actual systems go through TRL 9. In almost all cases, the end of last ‘bug fixing’ aspects of true ‘system development’. For example, small fixes/changes to address problems found following launch (through ’30 days’ or some related date). This might include integration of new technology into an existing system (such operating a new artificial intelligence tool into operational mission control at JSC). This TRL does not include planned product improvement of ongoing or reusable systems. For example, a new engine for an existing RLV would not start at TRL 9: such ‘technology’ upgrades would start over at the appropriate level in the TRL system.
Cost to Achieve:
4/10/2023 54
Mission specific; less than cost of TRL 8 (e.g., cost of launch, plus 30 days of mission operations)
Two additional terms need to be introduced and explored by students in order to gain a comprehensive overview of NASA project management techniques. These terms are:
pull technology development push technology development
Pull technologies are developed in response to an immediate need. Given this urgency, the tendency is to avoid extensive dependence on innovation but rather to adapt existing, mature technologies by incorporating the minor modifications required for the application niche. By definition, the customer is fully prepared to cover all costs associated with the development, and the outcome has a high probability of success.
Push technologies, on the other hand, are ‘disruptive’ and are based almost exclusively on the innovator’s vision of customers’ perceived needs. On the plus side, this type of development carries the promise of a pioneering effort. The downside, however, is that most often the outcome has a low probability of success.
In a few cases, this high failure rate could be attributed to unforeseen, technical ‘fatal flaws’. Most of the failures are due to the inability of the technology development to cross the TRL gap to become a ‘pull’ technology. The causes to failure are peculiar to each case but, in general, are a combination of unenthusiastic customer perception. Impedance mismatch with customer needs, bad timing, insufficient niche development, and a lack of necessary technological support infrastructure, among other reasons.
The end result of the TRL gap is that the infusion of advanced technology is slowed, and in some cases, stopped. Therefore, the TRL gap problem can be reformulated as the challenge of how to efficiently transition push technologies into pull technologies.
With this as background, students should be challenged to investigate the interesting issue of push and pull technology development. One approach is to divide the class to address each topic and have them prepare a detailed presentation that clearly and accurately describes, with examples from NASA technology development how push-pull technology development is addressed.
NASA uses a system of assessing the status of technological development.
4/10/2023 55
This process is a key element for all project management and final determination of whether a technology or project moves further in terms of research or actual development or deployment on a mission. This process is referred to as TECHNOLOGY READINESS LEVEL or TRL. This is a key process and students should investigate further or be part of a class lecture/discussion on this topic. Background information is provided for the instructional team. The final instructional decision rests with the team in terms of how this information will be presented to students.
Students should be given the following document to read and critique and then after that reading, the instructional team should lead a class seminar designed to engage students further with respect to this critical process used by all NASA agencies:
Closing the TRL Gap
Thomas George and Robert Powers, NASA jet Propulsion Laboratory
In recent years, while accumulating a unique and extensive base of experience in spaceflight, NASA has had to face decreasing budgets. These factors have had a profound impact on new technology development for future space applications, and have resulted in a technology readiness level (TRL) gap in the development of advanced aerospace technologies.
NASA has pioneered the use of the TRL scale for assessing the maturity of a particular technology. The utility of the scale, which consists essentially of nine levels of technology development maturity, has led to its widespread use among other government agencies and in the commercial sector.
The TRL gap is not unique to NASA. Although described using different terminology (funding gap, valley of death, Darwinian sea, the wall between research and product), it exists in all industry sectors. Stated simply, it is the problem of efficiently transitioning a new technology from concept to viable product in the shortest possible time and at least cost. Although the solution proposed here is relevant for space technology development, parallels can be drawn to address similar issues in other sectors of industry.
A close analog of the TRL scale is the “S-curve” commonly used in industry. The S-curve assesses the maturity of a technology either in terms of value to the company or the price a new product can potentially command. It introduces a parameter not explicitly addressed by the TRL scale- the time required to develop the technology (plotted on the X-axis). However, considering development time alone does not adequately describe the technology development process. We propose to modify the X-axis parameter to an as-yet-undefined complex function of time and investment.
4/10/2023 56
Two more terms need introduction in order to fully understand the technology development process: “pull technology development” and “push technology development.”
Pull technologies are developed in response to an immediate need. Given this urgency, the tendency is to avoid extensive dependence on innovation but rather to adapt existing, mature technologies by incorporating the minor modifications required for the application niche. By definition, the customer is fully prepared to cover all costs associated with the development, and the outcome has a high probability of success.
Push technologies, on the other hand, are “disruptive,” and are based almost exclusively on the innovator’s vision of customers’ perceived needs. On the plus side, this type of development carries the promise of a pioneering effort. The downside, however, is that most often the outcome has a low probability of success.
In a few cases, this high failure rate could be attributed to unforeseen, technical “fatal flaws.” Most of the failures, however, are due to the inability of the technology development to cross the “TRL gap” to become a pull technology. The causes for failure are peculiar to each case but, in genera, are a combination of unenthusiastic customer perception, “impedance mismatch” with customer needs, subcritical investment, bad timing, insufficient niche development, and a lack of necessary technological support infrastructure, among others.
The end result of the TRL gap is that the infusion of advanced technology is slowed, and in some cases stopped. In light of this, the TRL gap problem can be reformulated as the challenge of how to efficiently transition push technologies into pull technologies.
Doing Things the Old Way
4/10/2023 57
The S-curve for technology development shows the value of a new product for a new product for a company plotted as a function of development.
Time
Val
ue
Time
Two historical technology developments, both of which have had an enormous impact on aerospace, were the result of long, painful development. One is electric ion propulsion, a revolutionary concept conceived in the 1930s but not fully implemented until 1998; another is GPS, which underwent many stops and starts its three-decade development.
An ion propulsion system generates thrust by accelerating electrically charged atoms out of an engine. Despite the tremendous potential savings in fuel mass and the greater specific thrust ion propulsion provides over chemical propulsion, the development of ion drives underwent a series of stops and starts throughout the 1950s and into the early 1990s.
A major demotivator was the fact that chemical propulsion already existed as a proven technology, and in the race to get a man on the Moon, electrical propulsion development was perceived as a distraction from the goal at hand. It was not until the Deep Space-1 mission in 1998 that ion propulsion became truly operational for space missions. Now, it has been base lined as the critical technology for rapid flights to Mars and beyond.
The Global Positioning System, considered one of the greatest inventions of humankind, is another technology with a long development history. The concept arose from pioneering work done in the 1960s on three-dimensional positioning, when the DOD developed the idea of global all-weather satellite positioning (the Transi and Timation programs). From these beginnings, however, GPS did not achieve full operation until 1995. Among the challenges holding back its rapid development were the lack of coordination among the various DOD agencies, competing initiatives, low and unstable funding, and the lack of a well-defined cost/benefit argument.
Ultimately, the successful development of the GPS system required the establishment of a joint initiative among the Navy, the Army, and the Applied Physics Laboratory. Even then, it took several years for the group to establish the specifications for the system. Today, although the GPS system was developed to satisfy specific DOD positioning requirements, there has been an explosion in the number and diversity of its scientific and commercial applications, many probably not imagined by the original developers.
Why a New Approach Now?
It is recognized within the aerospace sector that a combination of risk-averse conservatism and lack of sufficient technology development funding has reduced the insertion of new technologies to a trickle. NASA and the U.S. aerospace industry at large are suffering from a broken technology development pipeline, which threatens to eliminate our competitive edge and
4/10/2023 58
with it our leadership in the aerospace sector. Clearly, the existing technology development process is unsuitable for the realities of today’s environment and the constraints imposed on the infusion of new technologies.
NASA’s technology development process is similar to that employed by most commercial firms, in which a Darwinian selection approach is applied to technology development and infusion. At the low TRL development stage, an attempt is made to increase the number of innovative ideas in hopes that there will ultimately be one or two productive concepts. The implicit assumption is that overcoming the low probability of success for push technologies requires the “seeding” of multiple concepts. It is questionable whether this is an efficient technology development paradigm. There are two key problems with this approach:
The overall resources allocated to low TRL development are severely limited. Therefore, by increasing one’s initial portfolio, one has further stretched these resources over an increased number of projects, driving each to be further sub-critically funded and thereby raising the probability of failure for each.
There is no one to “pick up the ball” once a technology has reached a mid-TRL stage. The link has not been made with potential customers and sponsors, who can carry the technology forward across the TRL gap.
Crossing the TRL Divide
Among the root causes of the TRL gap is the major divide in the mindsets of developers working in the high and low TRL areas. Individuals in the high TRL arena are accustomed to a relatively well-defined work environment in which the objective is to construct and demonstrate the technology at a system level. Technology development is generally straightforward, with few surprises. Low TRL folks, on the other hand, excel in an environment in which very little is precise. Chaos, serendipity, and ingenuity are woven intimately into the fabric of their work day.
The primary challenge for NASA sponsors seeking to increase the efficiency of the technology “harvesting” process is to bring these communities together and create the most continuous technology development pipeline possible. The answer may lie in the creation of a TRL Maturation Team (TMT), composed of representatives from the high and low TRL communities. Given the current divide between the two, this may only happen via a NASA-inspired “shotgun marriage,” with strong management oversight, at least in the early stages.
4/10/2023 59
The TMT concept is not original-the commercial sector has adopted “technology champion” and “transition team” concepts in an attempt to improve the efficiency of the technology maturation process. Ideally, the proposed TMT would be composed of the inventor of the technology, the high TRL system developer, reliability and testing personnel, and the end user. From an implementation perspective, the team should be formed at the early stages of low TRL development, essentially immediately after a new concept has been selected for funding.
STUDENTS SHOULD CREATE THEIR OWN TRL SCALE FOR USE WITH ASSIGNED PROJECTS IN THIS UNIT.
TMTs are created after funding decisions for low TRL concepts have been made to avoid coloring the initial technology selection process in any way with high TRL pragmatism. Instead, the selection process should be left
4/10/2023 60
With this modified S-curve, the new product value is plotted as a function of both time and investment. Also seen are the equivalences with NASA’s TRL scale.
Fn (time, investment)
Val
ue
TRL SCALE (Abbreviated)
TRL 1 Basic principles observed and reportedTRL 2 Technology concept and/or application formulatedTRL 3 Analytical and experimental critical function and/or
characteristic proof-of conceptTRL 4 Component and/or breadboard validation in the laboratory
environmentTRL 5 Component and/or breadboard validation in the relevant
environmentTRL 6 System/subsystem model or prototype demonstration in the
relevant environment (ground or space)TRL 7 System prototype demonstration in a space environmentTRL 8 Actual system completed and flight-qualified through test and
demonstration (ground or flight)TRL 9 Actual system “flight-proven” through successful mission
operations
pretty much in its current form, wherein a competitive process is employed in which revolutionary or breakthrough concepts are selected by a panel of low TRL developers. Maintaining the status quo ensures that “diamonds in the rough” are not rejected.
Even as a selection process can be expected to gather truly innovative concepts, the subsequent investment can also be made more efficient. By funding each project at the level required for a successful outcome, we can ensure that we are getting the best “bang” for our limited new technology development dollars.
The shotgun marriage of the low and high TRL points of view within the TMT could prove to be mutually beneficial. During the low TRL development phase, high TRL team members play essentially an advisory role, guiding the inventor away from technological dead ends that could stop the technology from transitioning to the system level. Design changes are far cheaper and more cost-effective at low TRL than after the technology has matured in a direction that is not well aligned with the end application.
In return, the high TRL members become intimately acquainted with the emerging technology and its various nuances, so that they can anticipate the challenges they will have to face during high TRL development. The TMT’s role becomes increasingly important once the proof-of-concept for the technology has been successfully demonstrated.The TMT assumes a critical role during the mid-TRL phase of the technology development. This is the crucial juncture in the development cycle - the point at which the TRL gap manifests itself. The reason mid-TRL development is such a dreaded phase is that a successful transition to high TRL depends on factors coming together in the correct order. The ultimate objective here is to change the character of the technology from “push” to “pull” - create a customer demand.
In NASA terms, this means convincing mission mangers or principal investigators that the technology has significant advantages over the state-of-the-art. Just as in the commercial sector, the conversion process involves first adapting the technology to a sequential series of niche applications and technology demonstrations (as in the electric ion propulsion case) until the demonstrated successes combine to create a demand for the technology. Successful transition to high TRL is achieved when new missions are designed around the technology rather than the other way around. This is an expensive, time-consuming process and one in which the skills of the high TRL members of the team come to the forefront.
Selling the Customers
A TMT - like mechanism may prove to be a significant improvement over the
4/10/2023 61
status quo and offers program managers, and ultimately the agency, a means to efficiently transition a larger number of new technologies within the same budgetary constraints. In the current technology development environment, it is incumbent on the inventor to develop the necessary skills to transition a technology across the TRL gap. With the proposed TMT approach, the burden of maturing the technology is shared by the TMT as a whole.
Quite often, the hardest task the inventor faces is convincing potential customers and high TRL developers that the new technology will provide significant benefits over the state-of-the-art. The vicious cycle - inadequate results to attract funding to generate the necessary results - is an all-too-common occurrence in the low TRL world.
The TMT-based technology development process, if implemented well, should result in a smooth transition through the various TRL levels. The success of the process rests primarily on the higher TRL “champions” for the technology. By making them an integral part of the TMT from the beginning, they will need no further convincing on the merits of the new technology. Within the TMT mechanism a smooth transition in “leading roles” is assured. During the low TRL development phase, the inventor’s role is ascendant; as the technology achieves higher maturity, member possessing the appropriate level of marketing and customer interface skills required for the further advancement assume leadership roles.
BENEFITS OF THE TMT APPROACH
* Creates a smoothly functioning “technology development pipeline.”* Minimizes the inefficiencies that currently exist in the transitioning of new
technologies to mission applications by:
Reducing waste of low TRL investment, (increasing the probability of successful development outcomes)
Enabling low-cost design changes at the low TRL development stage to make the technology more compatible with the ultimate system implementation
Reducing “impedance mismatch” between low TRL technologies and high TRL end applications
Shortening the total life cycle for new technology development, including avoiding stops and starts
Avoiding additional investment and loss of infusion time at the high TRL stage by cash-strapped flight projects attempting to adapt the technology to fit within the mission constraints.
* Establishes effective advocacy for the technology, beginning at the low TRL development stage.
Students should conclude the reading of this article and class seminar discussion led by the instructional team with an ECR written statement that requires each student to describe the benefits of the TRL process.
4/10/2023 62
Explanation:1. Students should be challenged to address the following questions in both
oral and written formats. Written statements should be brief constructed responses with oral responses given during class seminar or class discussion settings:
What are the major project management approaches used currently by diverse organizations and businesses?
How does NASA plan, organize, and control all of the work required to complete key missions or projects?
Why should NASA be interested in measuring or determining the maturity of a product or system?
2. Students will also offer their opinions with supporting information during discussions on selected topics presented by the instructor. One example is the critical engagement question:
Should humans be sent to live and work on the lunar surface or other planets?
Responses should be presented orally as part of a class seminar or class discussion and if appropriate as a brief constructed response before a class discussion is initiated so that students will be prepared to address the key question.
3. TRL benefits analysis after reading of TRL article from authors at the Jet Propulsion laboratory as an ECR statement.
4. Project management review of management functions identified and discussed via BCR statement.
5. Multi-media presentation for the engagement task of organizing teams to assemble pens, penlights or flashlight components - review of team approach.
6. Multi-media presentation on selected APOLLO mission by student teams -management processes employed to control work functions and quality.
7. Analysis of "Risk Management" process in seminar.
Extension:1. Students will be challenged to design and develop their own version of a
TRL scale which will be used to evaluate and manage future projects as part of several Engineering Design projects required later in this unit. Since NASA pioneered this scale and has used this system to assess the ‘maturity’ of a particular technology, it is essential that students fully understand how the scale was developed mathematically and its overall value for accurate technology assessment. The utility of the scale, which at NASA consists of nine levels of technology development maturity, has led to its widespread use among other government agencies as well as the commercial sector.
4/10/2023 63
A simple way to present a TRL feature is the use of a close analog of this scale, the S-curve. The S-curve assesses the maturity of a technology either in terms of value to the company or the price a new product can command. It introduces a parameter not explicitly addressed by the TRL scale - the time required to develop the technology (plotted on the X-axis). However, considering development time alone does not adequately describe the technology development process. At NASA, things are a bit more complex with respect to final determination of product maturity.
A proposal is to modify the X-axis parameter to an as-yet-undefined complex function of time and investment. The S-curve image below should be reviewed by students and discussed in terms of it’s mathematical value and construction:
2. In preparation for fully understanding how the development time of new technologies or powerful systems for aerospace purposes can extend for many years, students should be challenged to investigate and report on two outstanding examples of technological development that have made significant contributions to space travel. One is electric ion propulsion - a revolutionary concept conceived in the 1930’s but not fully implemented until 1998. The other is GPS (Global Positioning System), which underwent many stops and starts in a three-decade development. These two examples will provide a strong background for students as they attempt to realize how complicated the process is for a new technology to be appropriately ‘mature’ and ready for integration into a full scale aerospace project with valuable and widespread use.
This research could be accomplished via independent or small group work as determined by the instructor. One key feature of this research should include an explanation of ‘why’ such technologies required long development time periods. A discussion of the social, political, geopolitical, economic, and environmental impacts might be most worthwhile.
Additional topics could also be explored that provide more quality examples
4/10/2023 64
The S-curve for technology development shows the value of a new product for a new product for a company plotted as a function of development.
Time
Val
ue
of varying degrees of technological development from concept to application such as, but not limited to: Fission Rockets Solar Sails Stirling RTG Generators Fuel Cells Food Products (Space Travel) Lunar Rover Space Suit Designs Robotic Exploration (Surface) Radio Communications
Evaluation: Rubrics for the following are provided:1. Students should be challenged to address the following questions in both
oral and written formats. Written statements should be brief constructed responses with oral responses given during class seminar or class discussion settings: What are the major project management approaches used currently
by diverse organizations and businesses? How does NASA plan, organize, and control all of the work required
to complete key missions or projects? Why should NASA be interested in measuring or determining the
maturity of a product or system?
2. Students will also offer their opinions with supporting information during discussions on selected topics presented by the instructor. One example is the critical engagement question:
Should humans be sent to live and work on the lunar surface or other planets?
Responses should be presented orally as part of a class seminar or class discussion and if appropriate as a brief constructed response before a class discussion is initiated so that students will be prepared to address the key question.
3. TRL benefits analysis after reading of TRL article from authors at the Jet Propulsion laboratory as an ECR statement.
4. Project management review of management functions identified and discussed via BCR statement.
5. TRL benefits analysis after reading of TRL article from authors at the Jet Propulsion laboratory as an ECR statement.
6. Project management review of management functions identified and discussed via BCR statement.
7. Multi-media presentation for the engagement task of organizing teams to assemble pens, penlights or flashlight components - review of team approach.
4/10/2023 65
8. Multi-media presentation on selected APOLLO mission by student teams -management processes employed to control work functions and quality.
9. Risk management analysis and discussion in class seminar.
Enrichment Activities:1. Students should contact one of the NASA departments that conducts work
that is interesting to them and request information about how projects are managed in that specific department. This request should include examples of how work or tasks are planned, organized, and controlled. Using these three terms should help obtain detailed information about such activities at that office. This contact should be made via a formal business letter that is highly professional per the Business Letter Rubric.
2. Software development is a rapidly increasing need for NASA projects. How does project management for software development differ from other projects? The following offered as background information on this highly controlled and monitored process. Students should be challenged to investigate further and based on interest, present a detailed multi-media presentation on this unique process.
3. Students should be challenged to write a simple program using current languages (visual basic, Java, etc.) that will require the NASA software development process to be used. Students will use the provided background information and their additional research to design, monitor, test, and evaluate a final software product that performs a critical 'mission' function. Some suggestions are provided below: Program that monitors 'temperature' changes in a space suit Program that monitors 'heart rate' of lunar crew in space suite Program that monitors 'perspiration; in a space suite Program that monitors 'personal body temperature' of lunar crew in
space suit.
These are just suggestions for authentic basic programs that could be written by student teams to showcase their understanding and application of the value of integrated software and hardware systems to 'monitor' critical functions of the lunar crew.
It is a great opportunity for students to 'integrate' sensor technology through well designed programming to perform these important data analysis functions as part of human physiology and health of the lunar crew. Students show present their final program with 'sensor' integration as a formal presentation. There should be a clear review of how the NASA 'ISO' project monitoring was accomplished.
ISDProject Monitoring & Control
(PMC)-SOFTWARE DEVELOPMENT
4/10/2023 66
Number: 580-PC-012-01 Approved By: (signature)Effective Date: February 1, 2005 Name: Joe HennessyExpiration Date: February 1, 2009 Title: Chief, ISD
Responsible Office: 580/Information Systems Division (ISD)
Asset Type: Process
Title: Project Monitoring & Control (PMC) PAL Number: 1.4
Purpose This document establishes the process for Project Monitoring and Control (PMC) for all ISD mission software.
PMC is performed to provide understanding and insight into the project’s progress so that appropriate corrective actions can be taken when the project’s performance deviates significantly from the plan. Aspects of a project’s progress include interfaces to other organizations, deliverables, schedules, cost, effort, risk, reviews, verification, validation, and amount of supporting services. Planned management of these aspects is captured in one or more software and/or system plans.
Scope This document provides the basic PMC process and requirements for the life cycle of mission software.
Context Diagram
Software Project Management Processes
Roles and Responsibilities
Product Development Lead (PDL):
Responsible for project safety, cost, schedule, and technical performance
4/10/2023 67
ProjectFormulation
Process
ProjectStartupProcess
ProjectPlanningProcess
ProjectMonitoring &
Control Process
ProjectCloseoutProcess
Develops a cooperative and performance-oriented team Ensures that products and services from the project meet
customer needs
Development Team Lead (DTL)
Responsible for products produced by the team Produces consolidated status reports from the teamReview Team
Responsible for review of designated products, project progress, and project specific areas
Software Developer
Produces product elements and related status reports on work progress
Usage Scenario Primary Usage Scenario:
This process starts as soon as the project starts. The process is ongoing during the whole project life cycle.
Inputs Primary Usage Scenario:
Base lined SMP/PP and subsidiary plans Established development environment Initial progress tracking worksheet Project status information Technical review materials
Review packages Change Requests Requests For Action (RFAs) Review Item Dispositions (RIDS) Impact Analysis (for Requirements changes), etc.
4/10/2023 68
Entry Criteria Primary Usage Scenario:This process starts as soon as the project starts. Ideally, one should have access to the following inputs as a minimum for PMC startup:
Baselined Software Management Plan/Product Plan (SMP/PP) and subsidiary plans
Initial progress tracking worksheet
Exit Criteria Primary Usage Scenario:Only two events can end the operation of this process:
The project stops when an Abort/Suspend order has been issued.
OR The project reaches “End-of-Mission”.
Outputs Primary Usage Scenario:
Project Status Reports Issues Lessons Learned PMC Risk Information Requests for Action Review Item Dispositions
OR None, if Abort/Suspend order is received.
Major Tasks The PDL shall perform continuously: Monitor project activities and resources Monitor work products and project data Monitor software acquisition Monitor commitments
The PDL shall perform as needed: Manage corrective actions Generate reports and review progress Conduct milestone reviews Document lessons learned
4/10/2023 69
Task 1 Monitor Project Activities, Resources, and Personnel (PDL)
a) Monitor progress against the schedule by periodically measuring actual completion of activities and milestones. Compare this progress against the planned documented
schedule. Identify significant deviations and trends.
b) Monitor the project’s cost and effort by periodically measuring actual cost and effort expended by project staff. Compare the cost and effort to the planned documented
estimates. Identify significant deviations and trends.
c) Monitor resources provided and used by the project. Compare the resources to the planned documented
estimates. Identify significant deviations and trends.
d) Monitor documented risks in the context of the project’s current status and circumstances.
If project circumstances change which could give rise to new risk(s), then send relevant information (PMC Risk Information) to the ISD Software Risk Identification sub-process.
Revise the documentation on risks as additional information becomes available to incorporate changes. Communicate risk status to those affected. See ISD Software Risk Monitoring
e) Monitor project personnel training schedule by periodically measuring the progress of scheduled training.
Compare actual training against the planned documented training
Identify and document significant deviations and trends.GUIDANCE: Monitoring typically involves measuring the actual values of the SMP/PP (i.e., completion rate of software elements, resource utilization, etc.), comparing actual values to the estimates in the plan, and identifying significant deviations. Examples of resources, Task 1c, include:
Development and test environment Safety and security environments Network capacity Manpower usage and training Processor (CPU) and memory usage Process usage and improvement Facility development
4/10/2023 70
Task 2 Monitor Work Products and Project Data (PDL)
a) Monitor the project’s work products and tasks by periodically measuring the actual characteristics of the work products and task, e.g., size, complexity, quality, security, etc. Compare the actual characteristics and the changes to the
characteristics to estimates documented in the SMP/PP. Identify significant deviations and trends.
b) Monitor data management activities against the description in the SMP/PP on a periodic basis. Identify significant issues and their potential
impacts. Document the results from the data
management monitoring.GUIDANCE: Some examples of project data covered includes: source code and related files, meeting minutes, metrics, project documentation, telemetry, test scripts, input data, science data, safety and security backup plans, etc.
Task 3 Monitor Software Acquisitions (PDL)
Monitor the project’s acquisition of software by periodically performing the following as needed:a) Reviewing the needs of the project for new acquisitions of
software.b) Initiate Requests-for-Proposal (RFPs) to satisfy identified needs.c) Prepare or update contracts to acquire software.d) Monitor suppliers for compliance with contract provisions, on-
time software delivery, and quality of software to be delivered.e) Monitor acceptance of software that is delivered to assure full
compliance with acceptance processes and quality requirements.
GUIDANCE: See the Software Acquisition process for details.Task 4 Monitor Commitments (PDL)
a) Monitor internal and external commitments against the plan.b) Monitor the status of stakeholder involvement against the plan.c) Identify those commitments that have not been satisfied or those that are at significant risk of not being satisfied.d) Document the results of these reviews.GUIDANCE: Some examples of these types of commitments include:
Deliverables Interface Control Documents (ICDs) Interface Requirements Documents (IRDs) Engineering Test Units (ETUs) Requests for Action (RFAs)
4/10/2023 71
Review Item Dispositions (RIDs) Simulator availability Staff from other organizations
Once the stakeholders are identified and the extent of their environment within the project is specified in the SMP/PP, that involvement must be monitored to ensure that the appropriate stakeholders.
Task 5 Generate Reports and Review Progress (PDL)
a) Gather issues for analysis developed during previous tasks or input from other processes.
b) Analyze issues to determine need for corrective action. Document the analysis and appropriate actions needed to address the identified issues.
c) Review and get agreement with the relevant stakeholders on the actions to be taken and the priority to be assigned for their
completion.d) Negotiate changes to internal and external commitments.e) Monitor the corrective actions for completion.f) Analyze results of corrective actions to determine their
effectiveness. If previous corrective actions did not produce the desired result, then return to step b) above to rework the issues involved.
GUIDANCE: Issues are collected from reviews and the execution of other processes. Examples of issues to be gathered include:
Issues discovered through performing product development, maintenance, reviews, execution of other processes, verification and validation activities.
Significant deviations in schedule, cost, staffing, quality, product size, requirements, risk, etc. from the estimates in the SMP/PP.
Commitments (internal and external) that have not been satisfied.
Significant changes in risk status. Data access, collection, privacy, safety, and security issues. Stakeholder representation or involvement issues. Change requests, impact analysis (for requirements
changes) Requests for action Review item dispositions
4/10/2023 72
Task 6 Generate Reports and Review Progress (PDL)
a) Assemble project measures and the identified significant deviations and trends from what was planned in the SMP/PP.
b) Use the data to generate Metrics Reports.c) Produce project status report (that includes the Metrics data).d) Communicate status on assigned activities and work products to relevant stakeholders, including project, line management, and
the project team.e) Review the results of collecting and analyzing measures for
controlling the project with relevant stakeholders.f) Identify and document significant issues and action items
resulting from these project progress reviews.g) Track action items and issue resolution to closure.GUIDANCE: Data that should be included in the report: progress tracking data, schedule, risk, cost, effort, software error rates including severity, testing results, deficiency report (DR) summary, issues, etc.
Examples of progress reviews include the following: Reviews with the project team Reviews with project management and suppliers Reviews with line management Reviews with customers and end users
Stakeholders include managers, project team, customers, end users, suppliers, and others affected within the organization. Include these stakeholders in reviews as appropriate.
Task 7 Conduct Milestone Reviews (PDL)
a) Conduct the reviews at meaningful points in the project’s schedule with relevant stakeholders.
b) Review the commitments, plan, status, and risks of the project.c) Collect and document significant issues and their impacts as
Review Item Dispositions (RIDs) and Requests For Action (RFAs).
d) Assign RIDs/RFAs for corrective action to the appropriate process.
e) Track action items and issues to closure.GUIDANCE: Milestone reviews are planned during project planning and are typically formal reviews.
Software Requirements Review (SRR) Software Preliminary Design Review (PDR) Software Critical Design Review (CDR) Acceptance Test Readiness Review (ATRR) Operational Readiness Review (ORR
4/10/2023 73
See checklists at http://software.gsfc.nasa.gov/process.htm for details.
Stakeholders include managers, staff members, customers, end users, suppliers, and others affected within the organization. Include these stakeholders in milestone reviews as appropriate.
Task 7d example: “Assign RIDs/RFAs for corrective action”, Requirements change RIDs/RFAs would be assigned to the Requirements Management process.
Corrective action is required when the issue may prevent the project from meeting its objectives if left unresolved. Examples of potential actions include the following:
Modifying the statement of work Modifying the requirements Revising estimates and plans Renegotiating commitments Adding resources Changing appropriate processes Revising project risks
Task 8 Document Lessons Learned (PDL)
a) Collect and document issues that are found to have had a significant positive or negative impact on the project. If possible provide a suggestion for improvement to processes.b) Submit these significant issues (Lessons Learned) to the GSFC Engineering Process group for distribution to relevant
stakeholders.GUIDANCE: Lessons Learned are those significant issues encountered during a project that have affected, either positively or negatively, the schedule, cost, effort, staffing, quality, product size, requirements, risk, required resources, commitments, training, stakeholder involvement, processes, etc.
To view previous Lessons Learned or to submit a new one goes to the GSFC website http://software.gsfc.nasa.gov/lessons.htm .
4/10/2023 74
Measures Recommended Measures: Resource use (planned versus actual) Commitments (both internal and external) Project risks (status of current as well as possible new) Training for project personnel (planned versus actual) Stakeholder Involvement (planned versus actual)
Required Measures are found in “In-House Development and Maintenance of Software Products”, GPG 8700.5, at http://gdms.gsfc.nasa.gov/gdms
Schedule (planned versus actual) Budget (cost and effort) Product size Product error information
Tools and Templates Name Description
Action Item Tracking Tool
Tracking and maintaining status
Earned Value Tool Excel-based workbook tool available athttp://software.gsfc.nasa.gov/process.cfm
Microsoft Project Tool Tool for tracking schedule available as COTS software from Microsoft Corp.
Risk Tracking Tool Tracking and maintaining status
Training Course Name Description
Earned Value Earned Value strategies and methods for the first time user of the Excel-based workbook tool. For more details see: http://software.gsfc.nasa.gov/ training.htm
Foundations of Project Management
FPM provides interesting and relevant instruction of the methodologies, techniques, terms and guidelines used to manage cost, schedules and technical aspects through the life cycle of a project. The course is invaluable for project control and support personnel who need a better grasp of the project world. For more details see:http://ohr.gsfc.nasa.gov/DevGuide/Home.htm
4/10/2023 75
Training(continued)
QuantitativeSoftwareManagement
A two-day course developed a FPL and taught by Jairus Hihn and Bill Decker. Contains GSFC-specific information. Course materials include lecture presentations, tools, spreadsheets, and supporting information. Individual presentations, tools, etc. can be accessed from the web page address provided. For more details see: http:/ /software.gsfc.nasa.gov/training.htm
Risk Management
This 2-day course familiarizes the student with the fundamentals of Continuous Risk Management (CRM) and provides for interactive learning through the use of various methods and tools and a hypothetical space flight project case study. The second day is dedicated to organization-specific activities that will: 1) establish and organization-specific risk baseline; 2) practice the functions of CRM paradigm; 3) promote teambuilding and a more cohesive work environment; 4) provide risk information that can be acted on immediately upon completion of the course. Emphasis can be placed on the creation of Risk Management Plan as deemed necessary by each organization. For more details see: http://ohr.gsfc.nasa.gov/DevGuide/ Home.htm
SoftwareProject Management
The Software Project Management Course is a 5-day, residential, intermediate project management course targeted at those interested in increasing their knowledge of systems and software. Attendees should have some experience in managing projects. The course provides an overview of project management and associated topics. Classroom activities are augmented by hands-on workshops and group projects (e.g., project management plans, earned value, risk management, cost/schedule/technical performance monitoring). For more details see: http://software. gsfc.nasa.gov/training.htm
4/10/2023 76
Training(continued)
Technical Manager’s Training
The TMT is a 6-day residential program that focuses on presenting a high level overview of how work gets done in the Goddard environment. The Course Objectives are to:a) Learn about the Life Cycle of a project within the Goddard environmentb) Get familiar with principles of good Project Management, (How to plan, organize,
implement, and control technical projects) andc) Learn principles of how to increase
effectiveness within work teams through collaborative team participation.
There is a two-hour orientation at Goddard Greenbelt, 6 full days at Wallops and an hour and a half wrap-up session the following week in Goddard Greenbelt. The course begins on a Sunday and ends on a Friday. Developmental activities begin on the bus ride to Wallops.
References Glossary: http://software.gsfc.nasa.gov/glossary.cfm
Defines common terms used in ISD processes ETVX Diagram: Link to the ETVX diagram for this process Process Asset Library:
http://software.gsfc.nasa.gov/process.cfm Library of all ISD process descriptions
In-House Development and Maintenance of Software Products, GPG 8700.5, at http://gdms.gsfc.nasa.gov/gdms
NASA Software Engineering Requirements, NPR 7150.x, at http://gdms.gsfc.nasa.gov/gdms
The latest versions of the following can be found in the Process Asset Library at http://software.gsfc.nasa.gov/process.cfm ISD Software Risk Identification ISD Software Risk Monitoring and Control Software Requirements Review (SRR) Checklist Software Preliminary Design Review (PDR) Checklist Software Critical Design Review (CDR) Checklist Acceptance Test Readiness Review (ATRR) Checklist Operational Readiness Review (ORR) Checklist
Students will apply 'NASA' Project Management techniques to plan, organize, and control extension activities in Lessons 2-6. Specifically, they should apply their version of a 'TRL' Readiness Assessment.
4/10/2023 77
Classroom - Laboratory Preparation:There needs to be a formal presentation area with overhead projector, computer projector, and screen. There should be an area for group work with large tables that can be reorganized to accommodate small or large group work. There must be a computer lab with full Internet capability and CADD software. In addition, there needs to be a comprehensive fabrication laboratory that includes numerous and diverse power equipment allowing all types of materials to be processed in order to generate final products as part of the design brief challenge.
Tools/Materials Equipment: Computers (a standard for all lessons in this course) Computer projector (a standard for this course) Screen (a standard for this course) Overhead projector (a standard for this course) Paper supplies, scissors, graph paper, chart paper, markers CADD lab with appropriate software program (suggested solid works or similar
3-D development software) Modeling materials (clay, Styrofoam, plastic, paper, cardboard, glue guns, etc.) Numerous and diverse fabrication equipment (woods, metals, plastics) Numerous and diverse materials for final product creation (modeling
techniques)
Laboratory/Classroom Safety and Conduct:Students should follow prescribed program and school safety rules. It is also assumed that every teacher will establish appropriate rules of conduct and a management system to ensure high performance behavior and interaction with peers by all students.
A clear set of consequences and rewards should be defined, reviewed, and maintained throughout the school year. Specific rubrics for independent and group work are modeled in many lessons in this course guide.
During this course, many lessons will suggest that students assume numerous, diverse, and authentic roles found in the engineering field. This technique offers students the opportunity to learn about ‘conduct’ becoming to a professional in engineering, science, or mathematics. It is highly recommended that each instructor offer as much job-task structure as possible so that students can experience the demanding and highly responsible nature of engineering careers.
During this unit, all efforts by the instructional team should be made to ensure that students are engaged and challenged to function in an authentic NASA management system. Students should have clearly defined roles and responsibilities reflective of the most current NASA project management techniques used during the lunar exploration initiative.
4/10/2023 78
If possible, students should be allowed to assume a wide variety of management and labor positions throughout the entire course. The structure and other more realistic options for such an approach can be found in the Project Management unit. In fact, some instructors actually assign salaries reflective of the industry and connect some aspects of assessment to a professional review for salary increases based on job-performance. Again, this enables students to better understand the issue of connecting ability, performance, work ethics, and attitude to actual career evaluation standards and processes via classroom activities. The more authentic the environment and class procedures, student gains in understanding how engineers perform their work will be greatly enhanced.
The use of actual NASA ‘personnel’ charting or NASA ‘project-task’ charting for all group work as well as other simulations found in the course offers powerful, authentic experiences helping to prepare students for the rigor and challenge of engineering work accomplished within the numerous NASA research and development facilities. In this unit, students should assume roles and responsibilities consistent with NASA project management and quality control processes as they plan, organize, and control the required instructional tasks associated with living and working on the lunar surface.
Students should be challenged to investigate and report on the following management processes employed by NASA agencies:
Foundations of Project Management (FPM) Quantitative Software Management Risk Management Software Project Management
NASA resources for these programs are: http://oht,hdgv.nsds.gov/DevGuide/Home.htm http://software.gstc.nasa.gov/training.htm
KS/bdr/O1441/4/06
4/10/2023 79
