PM Template Project Review

8/3/2019 PM Template Project Review

1/74

PROJECT REVIEW

Version Number 1.0


2/74

VERSION HISTORY

Version #

Implemented

By

RevisionDate

ApprovedBy

ApprovalDate

Reason

1.0

Page 2 of 74


3/74

TABLE OF CONTENTS1 PURPOSE ................................................... ....................................................... ......................... . 4

2 SCORING STEPS AND ANALYSIS ..................................................... ................................... 4

2.1 Deliverables Scoring .......................................................................................................... 42.2 Exit Criteria Scoring ........................................................................................................... 72.3 Questions Scoring ............................................................................................................... 72.4 Known Issues/Risk Scoring ................................................................................................ 72.5 Summary Scoring Analysis ................................................................................................ 8

3 REVIEW FORMS ..................................................... ............................................ ........ ....... ...... 8

3.1 Review - Initiation Phase .................................................................................................... 93.2 Review - Concept Phase ................................................................................................... 133.3 Review - Planning Phase ................................................................................................. 203.4 Review - Requirements Analysis Phase ........................................................................... 283.5 Review - Design Phase .................................................................................................... 343.6 Review - Development Phase ........................................................................................... 423.7 Review - Test Phase ........................................................................................................ 493.8 Review - Implementation Phase ...................................................................................... 543.9 Review - Operations & Maintenance Phase .................................................................... 623.10 Review - Disposition Phase ........................................................................................... 70

Page 3 of 74


4/74

1 PURPOSE The purpose of these templates is to recommend the structure andscoring of each of the [stages of project review]. The evaluation formsand scoring analysis are included.

2 SCORING STEPS AND ANALYSISScoring of each of the ten Phases is based on four components:

Deliverables Exit Criteria Compliance Known Issues/Risk Questions

2.1 DELIVERABLES SCORING

Deliverables for each Phase of the PLC are outlined in the PLC

Framework Overview Document. Projects must complete each of therequired deliverables unless otherwise agreed to and tailored in theProject Process Agreement.

Prior to the initiation of the [stages of project review]. reviewers shouldread the content of each deliverable in the Phase and score eachbetween 1-3 on the following criteria:

Completeness1=incomplete deliverable or deliverable does not exist2=deliverable needs to be more detailed3= deliverable is complete

Accuracy1=deliverable information is not accurate or isinconsistent2=deliverable needs to be more detailed3=deliverable is accurate

Adequacy1= deliverable does not follow HHS best practices

2=deliverable needs to be more detailed3=deliverable is adequate and meets the definedpurpose for which it was designed and follows OPDIVor HHS best practices

It is suggested that if this process is adopted, projects receiving a totalscore of 3 on an individual deliverable (a rating of 1 for each criteriaevaluated) may be recommended for discontinuation. Projects receivingtotal scores on individual deliverables of 4 - 8 can be approved with the

Page 4 of 74


5/74

condition of improvement of the deliverable(s). If each of thedeliverables in a phase receives criteria scores of 9 the project can berecommended for approval. Please note that [stages of project review].approval(s) can be impacted by Exit Criteria, Reviewer Questions andKnown Project Risks and Issues.

Page 5 of 74


6/74

Project A:

Deliverable Completeness

Accuracy Adequacy Total Rating

BusinessCase

2 2 1 5

Final Project

Charter

3 3 3 9

PreliminaryProjectManagementPlan

2 3 3 9

In the example above, Project A has a score of 5 for the Business Case and 9for the Project Charter and Project Management Plan. The project will berecommended for Stage Gate approval with conditions.

Project B:



BusinessCase

1 1 1 3

Final ProjectCharter

1 1 1 3

Preliminary

ProjectManagementPlan

1 1 1 3

In the example above, Project B has a score of 3 for each deliverable. Thisproject will be recommended for discontinuation.

Project C:



BusinessCase

3 3 3 9

Final ProjectCharter

3 3 3 9

PreliminaryProjectManagementPlan

3 3 3 9

Page 6 of 74


7/74


8/74

TechnologyStrategicSecurityPrivacyProject Resources

The project can use the risks identified in the Stage Gate Review tosupplement their risk planning and vice-versa.

Risk impact is scored as high, medium, or low. If the identified risk willhave a large impact on the project largely increasing the cost of theproject, then the impact is high. Probability of Occurrence is also scoredas high, medium, or low. If the risk will happen, then the probability ishigh.If the project scores high impact/high probability on more than one risk,the project management team may want to consider not recommendingproject approval without risk planning.

2.5 SUMMARY SCORING ANALYSIS

If the project team adopts this scoring process it is suggested that if aproject scores 3 on all deliverables and scores NP in all other categories,it will be recommended to the IT Governance Board that the project isdiscontinued. If a project scores a 9 on all deliverables and score P in allother categories, it will be recommended for approval and will pass on tothe next Phase of the project. All other combinations of score will beApproved with Conditions. Approval with conditions requires the projectteam to establish a process for maintaining oversight of the project toensure conditions are met. The project team may require issueresolution by the PM before approving continuation, and is responsiblefor discontinuing any project which fails to resolve serious issues.

3 REVIEW FORMS The following are suggested forms for the review process. They can bemodified to suit any project requirements.

Page 8 of 74


9/74

3.1 REVIEW - INITIATION PHASE

Project:Reviewing Body:Date of Review:

Name of Reviewer:Role in Review:

Initiation Phase Review The Initiation Review considers whether the Business Needs Statement justifies proceeding to the Concept Phase for development of a full BusinessCase and preliminary Project Management Plan.Responsibilities

Business Owner Responsibilities in Initiation Review The Business Owner is the principal authority on matters regarding theexpression of business needs, the interpretation of functional requirementslanguage, and the mediation of issues regarding the priority, scope anddomain of business requirements. The Business Owner must understandwhat constitutes a requirement and must take ownership of therequirements and input and output. The Business Owner champions theproposed investment to the IT governance body to gain approval.

Critical Partner Responsibilities in Initiation Review

Enterprise Architecture: Validate alignment of the Business Needsstatement with the Enterprise Architecture. Determine if thepreliminary enterprise architecture review reveals any duplication orinterferes, contradicts, or can leverage another existing or proposedinvestment.

Security: Determine if the Business Needs Statement contains anypotential security concerns.

Budget: Determine if the Business Needs Statement ensures thatadequate financial resources are available.

CPIC: Verify that the initial scope of the project will adequately

address requirements specified in the Business Needs Statement.Performance: Ensure that Risk Tolerance levels are established. This

function is performed by the Business Owner.

Stage DeliverablesPlease rate the deliverables for this Stage from 1 (Poor) to 3 (Excellent)

Deliverable Name Completeness (1-3)

Accuracy (1-

3)

Adequacy

(1-3)

Comments

Page 9 of 74


10/74

Business NeedsStatement

Mandatory Exit Criteria The objective is to determine if this investment proposal is worth pursuing.[Is there a good chance that the investment will be approved and funded?Does this investment proposal warrant investing in the development of abusiness case and preliminary project management plan?]

Exit Criteria Pass (P)/Not Pass

(NP)

Comment

A BusinessOwner has beenidentified andconfirmed.[Someone whowill champion

the investment,defines thebusiness needsand investmentrequirements,and securesfunding).

Approval of thisinvestment ishighly probable.

The decision isbased on thefollowingfactors:acceptablerisk/return;high-prioritybusinessneed/mandate;and no morepreferablealternative(use/modifyexistingapplication, notaddressablethroughbusinessprocessreengineeringor other non-IT

Page 10 of 74


11/74

solution).

Investmentdescription issufficient topermitdevelopment of an acceptablebusiness caseand preliminaryprojectmanagementplan.

Suggested Questions The Business Owner and Critical Partners will fill in their Role next to relevantquestions and comment on observations.

Role Question Comment

Has the Business Owner defined thebusiness need?

Is the Business Owner aware of his/her rolethroughout the lifecycle of the project?

Have the stakeholders been identified andinformed of the Business Needs Statementfor the potential project?

Has the goal and scope of the project beendescribed?

Has the business risk of executing or notexecuting the project been described?

What is the risk tolerance level of thestakeholders?

Has Enterprise Architecture conducted apreliminary review of the business need?

Is the Business Needs Statement soundand consistent with the EnterpriseArchitecture?

Does this Business Needs Statementaddress a PMA goal?

Does the Business Needs Statementsupport the OPDIV strategic goals andobjectives?

Page 11 of 74


12/74

Is there a Rough Order of Magnitude oncost and schedule in the Business NeedsStatement?

Does the proposed business need satisfy acapability gap?

Known Issues/RisksRisk

DescriptionArea of Risk

(Communication, Cost,Quality , Schedule,

Scope)

Impact(High, Medium, or Low)

Probability of Occurrence (High,Medium, or Low)

Summary ScoringDeliverables

(Total Score between3-6)

Exit Criteria(P or NP)

Questions(Subjective P or NP))

Risk (Number of High

Impact/HighProbability)

(2 or more = NP)

Recommendations

Approval Level (check one) Explanations, Caveats or ConditionsApprove

Approve withConditions

Discontinue Project

Governance Forward:Forwarded to:

Signature: Date:PrintName:

Title:

Page 12 of 74


13/74

3.2 REVIEW - CONCEPT PHASE

Project:Name of IT Governance Body:Date of Review:


Concept Phase Review The Project Selection Review (PSR) is a formal inspection of a proposed ITproject by the IT governance organization to determine if it is a sound,viable, and worthy of funding, support and inclusion in the organizations ITInvestment Portfolio. This Stage Gate Review is one of the four that cannotbe delegated by the IT governance organization.

ResponsibilitiesBusiness Owner Responsibilities in the Review

The Business Owner is responsible for ensuring that adequate financial andbusiness process resources are made available to support the investmentonce approved. Responsibility may include the designation of the ProjectManager.IT Governance Organization Responsibilities in the Review

The IT governance organization conducts the Project Selection Review.Project Manager Responsibilities in the Review

The Project Manager develops the Business Case and preliminary ProjectManagement Plan.Critical Partners Responsibilities in the Review

Critical Partners review and comment on the Business Case and participatein the Project Selection Review.

Enterprise Architecture: that the business processes are modeled insufficient detail.

Security: Conclude that all applicable security and privacy standardshave been considered in sufficient detail as part of the Business Case.

Verify that a high level security analysis and a preliminary riskassessment are complete and justify proceeding to the Planning Phase.

Acquisition: Ascertain if a preliminary Acquisition Plan that isappropriate to the level of the requirements definition is part of theBusiness Case, and includes performance-based acquisitions. Verifythat the overall acquisition plan includes consideration of internalversus external acquisition, re-use, the use of commercial off-the-shelf technologies, and, if requests for information are necessary, howcontracting work will be divided, and expected contract types.

Page 13 of 74


14/74

Budget: Establish that the Business Case includes a financing andbudgeting plan and that there is sufficient requirements detail tosupport the detailed cost and schedule estimates needed during thePlanning and Requirements Analysis Phases.

HR: Determine the probability and/or impact of any anticipatedworkforce disruptions has been reviewed and make certain the needfor staffing classifications.

Performance: Ensure that the approval of the performance baselinesis completed. Determine that appropriate potential performance goalsare established as part of the Business Case. Conclude that therequired authority and project structural foundation is in place.

Stage DeliverablesPlease rate the deliverables for this Stage from 1 (Poor) to 3 (Excellent)Deliverable Name Complete

ness (1-3)Accuracy(1-3)

Adequacy(1-3)

Comments

Business Case w/components(Final)

Project Charter(Final)

ProjectManagement Planw/components(Preliminary)

Mandatory Exit Criteria: The objective is to determine if the project has been clearly defined and hasthe supporting organizational structure to proceed with full planning.Exit Criteria Pass (P)/

Not Pass(NP)

Comment

The scope of the project hasbeenadequatelydescribed in theBusiness Caseand that thehigh levelrequirementsmeet thebusiness need.

The projectorganizational

Page 14 of 74


15/74

structure isscaled tosupport theproject and theproject managerand the projectteam are

qualified[OrganizationalMappingssupport projectcommunicationneeds.]

The PreliminaryProjectManagementPlan adequatelydefines how theproject will beexecuted,monitored andcontrolled andincludes highlevel estimatesof the baselines.

The high levelanalysisdemonstrates

that theoutcomes willbe aligned withthe TargetEnterpriseArchitecture.

All applicablesecurity andprivacystandards havebeen consideredin sufficientdetail as part of the BusinessCase. FIPS-199categorizationand an initialassessment of systemaccreditation

Page 15 of 74


16/74

boundary areestablished.

Suggested QuestionsReviewers will fill in their Role next to relevant questions and comment onobservations

Role Question CommentWhat are the key objectives of the project?

How is success measured?

Does the business case identify high levelrequirements?

Does the business case rest on a detailedgap analysis which validates theopportunity to improve businessaccomplishments or correct a deficiency

related to a business need?Is the business case missing any keyacquisition-related items (e.g., costs forhardware, software, and serviceacquisitions)?

Does the Business Case details thebusiness need and expected performanceoutcomes?

Has an Alternatives Analysis been done tosupport the Business Case thatincorporates recommendations by the IPTof a specific solution?

Have business processes been modeled toa sufficient level given the phase of theproject/investment?

Have assumptions and constraints beenidentified with respect to each consideredalternative?

Does the Project Charter give adequate

authority to the Project Manager to executethe project?

Does this project require special planningconsiderations (constraints), or subsidiaryplanning documents?

Has the approach to Risk Managementbeen tailored to suit the scale of theproject?

Page 16 of 74


17/74

Have risks been identified for each high-level of the WBS?

Have the risks been evaluated andassessed?

Has the Basis of Estimate been riskadjusted?

Have triggers for risks been identified?Does the preliminary acquisition planinclude performance based acquisitions?

Will there be a Change Control Board? Whowill it include?

Has an internal (government) configurationmanagement process been developed?

Have high level requirements beendeveloped?

Has the role of Line of Business Sponsorbeen considered in the CommunicationsPlan?

Has a system or process been developedor identified to manage the project andtechnical documentation of the project(Configuration Artifacts)?

Does the initial WBS and Schedule have atleast three levels and do all activities havedependencies?

Are high level WBS nouns and activitiesverbs?

Has an initial basis of estimate beenprepared for each alternative underconsideration?

Have potential performance goals beenidentified as a part of the business case?

Has a records disposition schedule beenconsidered?

Are there any anticipated potentialworkforce disruptions, Labor Relations orEmployee Relations issues associated withthe project/investment?

Are there any staffing classification issuessuch as new position descriptions, grades,etc that are associated with thisproject/investment?

Page 17 of 74


18/74

Are there any potential workforce planningissues such as employee development andtraining, staffing levels, filling skill gapswith contractors, and/or A-76 activitiesassociated with this project/investment?

Have the applicable security and privacystandards been considered as a part of thebusiness case?Is the basis of estimate realistic andthoughtfully prepared?

Is the proposed project/investmentincluded in the target enterprisearchitecture and the EA transitionstrategy?

Have a preliminary Acquisition Plan beendeveloped that is appropriate to the levelof requirements defined in the BusinessCase?



(Communication, Cost,Quality, Schedule, Scope)

Impact(High, Medium, or

Low)








(2 or more = NP)

RecommendationsApproval Level (check one) Explanations, Caveats or Conditions

Approve


Discontinue Project

Page 18 of 74


19/74


Signature: Date:

PrintName:

Title:

Page 19 of 74


20/74

3.3 REVIEW - PLANNING PHASE



Project Baseline Review The Project Baseline Review (PBR) is a formal inspection of the entire projectand performance measurement baseline initially developed for the IT project.

The PBR is conducted to obtain management approval that the scope, costand schedule that have been established for the project are adequatelydocumented and that the project management strategy is appropriate formoving the project forward in the life cycle. Upon successful completion of this review, the Project Management Plan is officially baselined.

The PBR includes review of the budget, risk, and user requirements for theinvestment. Emphasis should be on the total cost of ownership and not justdevelopment or acquisition costs. Support and training issues may becomevery important from this perspective.

ResponsibilitiesBusiness Owner Responsibilities

The Business Owner is responsible for authorizing and ensuring that thefunding and resources are in place to support the project.

IT Governance Organization Responsibilities in PlanningDuring the Project Baseline Review, the IT governance organizationexamines whether scope, cost and schedule that have been established forthe project are adequately documented and that the project managementstrategy is appropriate for moving the project forward in the life cycle.Project Manager Responsibilities in Planning

The Project Manager is responsible and accountable for the successfulexecution of the Planning Phase. The Project Manager is responsible forleading the Integrated Project Team that accomplishes the Phase activitiesand deliverables.Integrated Project Team Responsibilities in Planning

The Integrated Project Team members (regardless of the organization of permanent assignment) are responsible for accomplishing assigned tasks asdirected by the Project Manager.Critical Partners Responsibilities in PlanningCritical Partners assess completeness of Planning Phase activities, robustnessof the plans for the next life cycle phase, availability of resources to executethe next phase, and acceptability of the acquisition risk of entering the next

Page 20 of 74


21/74

phase. For applicable projects, this assessment also includes the readiness toaward any major contracting efforts needed to execute the next phase.

Enterprise Architecture: Conclude that compliance with EnterpriseArchitecture has been maintained.

Security: Review the PMP Risk Management Plan accuratelyestablishes that the security and privacy requirements have beenidentified and planned for.

Acquisition: Make certain that acquisition activities to obtaincontractor support have been completed in compliance with theProject Management Plan. Confirm that detailed activities andtimelines for preparing acquisition documents, selecting vendors, andawarding contracts are developed.

Budget: Determine if there is a realistic budget to accomplish allplanned work and that the Total Cost of Ownership has beenevaluated.

Finance: Ensure that planning for financial management issues has

been properly addressed and that interactions with financial systemsare planned in compliance with financial standards and regulations.

HR: Find out if required staff development has been documented andplanned.

Section 508: Verify that Applicable Section 508 standards areidentified and planned for

Performance: Ensure that expected performance benefits are fullydefined, that business product deliverables are well-planned, and thatfunding and resources are allocated.



Adequacy(1-3)

Comments

ProjectManagement Planw/components(Final)

Privacy ImpactAssessment(Final)

Project ProcessAgreement (Final)

Mandatory Exit Criteria: The objective is to determine if the project has finalized project planning anddefined initial baselines and requirements to permit outside validation.

Page 21 of 74


22/74

Exit Criteria Pass (P)/Not Pass(NP)

Comment

The full scope of the project hasbeenadequatelydescribed in theBusiness Caseand the highlevelrequirementsmeet thebusiness need.

The ProjectManagementPlan is fullyscaled and

details all theappropriatecomponentsthat address theneeds of theproject. Thisincludes thedefinition of appropriatelyscaled reviewsand deliverables

All Deliverableshave beendefined.

The AcquisitionPlan has beenapproved by theContractingOfficer andthere isobligated

money forcontractawards. Allapplicablecontract clauseshave beenconsidered.

The risk limits of the Business

Page 22 of 74


23/74

Owner havebeen definedand risks of highest impacthave beensufficientlyaddressed with

eithermitigation orcontingencyplans.

Variances frombaselines havebeen identifiedand mitigated.[Cost andschedulevariances andscope changesare identified,significantvariances areexplained, andCorrectiveAction Plans(CAPs) orrebaselinerequests are inplace asappropriate.]Investmentbaselines havebeen reviewedand revised asappropriate.[Should thisinvestmentcontinue as-is,be modified, or

be terminatedbased oncurrentknowledge?]

The ProjectManagementPlan andcomponentplans have been

Page 23 of 74


24/74


25/74

with external systems and projects.

Does the schedule appear to beachievable, realistic and address all areasthat need to be included in the project?

Have performance goals been establishedand a monitoring mechanism implemented

to assure goals are achieved?Do the performance goals align with thepurpose of the project/investment asdocumented in the performance gapaddressed in the Business Case?

Does the reporting period cover the lifecycle of the project/investment?

Are performance measures outcome-based, or where appropriate, output-based,and related to the performance gaps theproject/investment is designed to fulfill?

Are performance measures stated asmeasures and are they SMART?

Is there sufficient number of annual goalsto provide an adequate view of theproject/investment performance?

Have any anticipated potential workforcedisruptions, Labor Relations or EmployeeRelations issues associated with the

project/investment been planned for?Have any staffing classification issues suchas new position descriptions, grades, etcthat are associated with thisproject/investment been planned for?

Have contractor security procedures beendeveloped?

Have the applicable security and privacystandards been identified and planned for?

Has identifying and assessing security andprivacy risks been incorporated into theoverall risk management planning?

Does it appear that an increase in securityfunding is needed to remediate IT securityweaknesses?

Is it clear when the resources need to starton the project?

Page 25 of 74


26/74

What needs to be done to bring newresources onto the team?

What tools do new resources require?

What are the tipping points for the projectgoing off track? Is the communicationchannel open to the sponsor in this case?

Is the WBS based on deliverables or tasks?Are the estimate assumptions clear and upfront?

Are the applicable Section 508 standardsidentified and planned for?

Are there any difficulties in meeting 508standards anticipated that do not havemitigation measures planned?

Has identifying and assessing thevulnerability and impact of being noncompliant with Section 508 been includedin the overall risk management planning?

Have any weaknesses been identified bythe agency or IG that have not beremediated? Have they been incorporatedinto the plan of action and milestoneprocess?





Low)








(2 or more = NP)

Recommendations

Page 26 of 74


27/74

Approval Level (check one) Explanations, Caveats or Conditions

Approve


Discontinue Project



Title:

Page 27 of 74


28/74

3.4 REVIEW - REQUIREMENTS ANALYSIS PHASE



Requirements Analysis Review The Requirements Analysis review considers whether the project shouldproceed to the Design Phase.

ResponsibilitiesBusiness Owner Responsibilities in Requirements Analysis Review

The Business Owner participates in the development and elicitation of bothfunctional and non-functional requirements.End User Responsibilities in Requirements Analysis Review.

The End Users participate in the development of detail of functionalrequirement and provide input into non-functional requirements.Project Manager Responsibilities in Requirements Analysis Review

The Project Manager is responsible and accountable for the successfulplanning and execution of the Requirements Analysis Phase. The ProjectManager is responsible for leading the Integrated Project Team thataccomplishes the Phase tasks and deliverables.Integrated Project Team Responsibilities in Requirements AnalysisReview

The Integrated Project Team members (regardless of the organization of permanent assignment) are responsible for accomplishing assigned tasks asdirected by the Project Manager.Contracting Officer Responsibilities in Requirements AnalysisReview

The Contracting Officer is responsible and accountable for preparingsolicitation documents under the guidance of the Project Manager and Headof Contracting Activity.Critical Partners Responsibilities in Requirements Analysis Review

The Critical Partners provide oversight, advice and counsel to the ProjectManager to ensure that the Requirements Document addresses relevantstandards. Additionally, Critical Partners provide information, judgments, andrecommendations during the Requirements Review.

Enterprise Architecture : Find out if requirements provide a suitablebasis for subsequent design activities and all service components havebeen appropriately identified. Determine if technologies and other

Page 28 of 74


29/74

requirements are consistent with the Enterprise Architecture. Identifyrelevant technical and/or service standards that will apply to orconstrain solution design and development activities.

Security: Ensure that an assessment of the required security controlshas been completed and determine if requirements reflect alignmentwith established security standards.

Acquisition : Review acquisition planning to ensure it includesnecessary requirements analysis, alternatives analysis, andprocurement and contract award plans. Ensure that there is sufficientinformation to make management decisions and evaluate vendorproposals.

Budget: Ascertain if requirements are in accord with investment-levelcost baselines established at the end of the Planning Phase or a formalchange to the Investment Baselines has been requested.

Finance: Determine if financial management requirements are inaccordance with requirements established at the end of the PlanningPhase or a formal change to the Investment Baselines has beenrequested.Performance: Determine if the requirements are in accordance withinvestment-level performance baselines established at the end of thePlanning Phase or a formal change to the Investment Baselines hasbeen requested.

Stage DeliverablesPlease rate the deliverables for this Stage from 1 (Poor) to 3 (Excellent)Deliverable Name Completene

ss (1-3)Accuracy (1-3)

Adequacy(1-3)

Comments

RequirementsDocumentw/components(Final)

Mandatory Exit Criteria: The objective is to determine if the project requirements have been definedsufficiently to be translated into the Business Product.Exit Criteria Pass (P)/

Not Pass(NP)

Comment

The initial testplan is defined.

Requirementshave beengrouped andsufficientlydetailed so that

Page 29 of 74


30/74

they can betested once theproduct isdeveloped.

Process andData Models aredefinedadequately forproduct design.

Variances frombaselines havebeen identifiedand mitigated.[Cost andschedulevariances andscope changes

are identified,significantvariances areexplained, andCorrectiveAction Plans(CAPs) orrebaselinerequests are inplace asappropriate.]

Investmentbaselines havebeen reviewedand revised asappropriate.[Should thisinvestmentcontinue as-is,be modified, orbe terminatedbased oncurrentknowledge?]

The ProjectManagementPlan andcomponentplans have beenreviewed andappropriately

Page 30 of 74


31/74

updated. [Thisincludes RiskManagement,Acquisition Plan,ChangeManagement,Configuration

Management,ProjectCategorization,RequirementsManagement,CommunicationPlan,WBS/Schedule,IV&V Planning,QualityAssurance,

RecordsManagement,Staff DevelopmentPlan andSecurityApproach.]

Suggested QuestionsReviewers will fill in their Role next to relevant questions and comment on

observations Role Question Comment

Are meetings conducted with the EndUsers to elicit requirements?

Have the major stakeholders provided thebusiness requirements?

Has there been agreement by allstakeholders and the business owner onthe requirements?

What is the single most importantrequirement for the project?

Can the business requirements be groupedinto critical, major, minor, and nice-to-havecategories?

Are there any requirements that appearcontradictory, ambiguous or unclear?

Page 31 of 74


32/74

Is there enough detail in the businessrequirements for an analyst to write atechnical specification?

What has been done to ensure thatrequirements are complete?

What has been done to determine theaccuracy of the requirements?

Are the requirements detailed enough andwith enough specificity enough to bemeasurable?

What is the quality assurance process forthe business requirements?

Are the requirements testable?

Are requirements suitable for subsequentdesign activities?

Has the assessment of required securitycontrols been completed?

Do the requirements have sufficientinformation to ensure that acquisitionmanagement decisions and vendorproposal evaluations can take place?





Low)








(2 or more = NP)

Page 32 of 74


33/74


Approve


Discontinue Project


Signature: Date:

PrintName:

Title:

Page 33 of 74


34/74

3.5 REVIEW - DESIGN PHASE



Preliminary Design Review The Preliminary Design Review (PDR) is a formal inspection of the high-levelarchitectural design of an automated system, its software and externalinterfaces, which is conducted to achieve agreement and confidence that thedesign satisfies the functional and non-functional requirements and is inconformance with the enterprise architecture. Overall project status,proposed technical solutions, evolving software products, associateddocumentation, and capacity estimates are reviewed to determinecompleteness and consistency with design standards, to raise and resolveany technical and/or project-related issues, and to identify and mitigateproject, technical, security, and/or business risks affecting continued detaileddesign and subsequent development, testing, implementation, andoperations & maintenance activities.ResponsibilitiesBusiness Owner Responsibility in Design

The Business Owner may participate in the Preliminary Design Review.IT Governance Organization Responsibility

The IT governance organization conducts the Preliminary Design Review toachieve agreement and confidence that the design satisfies the functionaland non-functional requirements and is in conformance with the enterprisearchitecture.Project Manager Responsibility

The Project Manager is responsible and accountable for the successfulexecution of the Design Phase. The Project Manager is responsible forleading the team that accomplishes the phase activities and deliverables.Integrated Project Team Responsibility

The Integrated Project Team members (regardless of the organization of permanent assignment) are responsible for accomplishing assigned tasks asdirected by the Project Manager.Contracting Officer Responsibility

The Contracting Officer is responsible and accountable for preparingsolicitation documents under the guidance of the Project Manager and Headof Contracting Activity.Critical Partners Responsibility

Page 34 of 74


35/74

The Critical Partners participate in a Design Review to ensure compliancewith policies in their respective areas and to make any necessary tradeoff decisions if conflicting goals have arisen during the Design.

Enterprise Architecture: Conduct a formal review of the high-levelarchitectural design to achieve confidence that the design satisfies thesystem requirements, is in conformance with the EnterpriseArchitecture and prescribed design standards.

Acquisition: Verify that contracts are being fulfilled according toaward or approved changes.

Budget: Guarantee that the budget is sufficient to meet the needs of the project. Determine if project business risks are identified andmitigation plans are made.

Finance: Guarantee that estimates of project expenses have beenupdated to reflect actual costs and estimates for future phases.Determine if project business risks are identified and mitigation plansare made.

HR: Confirm that issues related to staffing, workforce, or other HRareas have been addressed.

Performance: Determine if project technical risks are identified andmitigation plans are made. Verify that performance goals are agreedupon.


ness (1-3)Accuracy

(1-3)

Adequacy

(1-3)

Comments

Design Documentwith components(Final)

Computer MatchAgreement (Final)

Test Plan (Final)

ContingencyDisaster

Recovery Plan(Final Draft)

System of RecordNotice (FinalDraft)

Mandatory Exit Criteria:

Page 35 of 74


36/74

The objective is to determine if the project has finalized project planning anddefined initial baselines and requirements to permit outside validation.Exit Criteria Pass (P)/

Not Pass(NP)

Comment

No outstandingconcerns amongstakeholdersregarding designadequacy orfeasibility.

Design isadequatelydocumented toallow effectiveand efficientdevelopment.

Contingency/Disaster Recoveryplans areadequatelydocumented toprovide clearprocedures andresponsibilities

SecurityDocuments areas complete andaccurate aspossible.

Variances frombaselines havebeen identifiedand mitigated.[Cost andschedulevariances andscope changes

are identified,significantvariances areexplained, andCorrective ActionPlans (CAPs) orrebaselinerequests are inplace asappropriate.]

Page 36 of 74


37/74

Investmentbaselines havebeen reviewedand revised asappropriate.[Should thisinvestment

continue as-is,be modified, orbe terminatedbased on currentknowledge?]

The ProjectManagementPlan andcomponent planshave beenreviewed andappropriatelyupdated. [Thisincludes RiskManagement,Acquisition Plan,ChangeManagement,ConfigurationManagement,ProjectCategorization,RequirementsManagement,CommunicationPlan,WBS/Schedule,IV&V Planning,QualityAssurance,RecordsManagement,Staff DevelopmentPlan andSecurityApproach.]


Page 37 of 74


38/74


Has a formal review of the high-levelarchitectural design been conducted?

Does the Design Document provide anoverview of the entire hardware andsoftware architecture and data design,

including specifications for externalinterfaces?

Does the design include all lower-leveldetailed design specifications of theBusiness Product, such as general systemcharacteristics, the logical and physicaldata model, user interfaces, and businessrules?

Has the Requirements Traceability Matrixbeen updated to describe how the system

design will satisfy the functional, business,security, and technical specifications in theRequirements Document?

Does the design define the releasestrategy in sufficient detail?

Has the design addressed data conversionissues at the appropriate level?

Has the interface control beendocumented?

Has the design considered the impact of capacity (e.g., database, hardware)requirements on the implementation?

Have the needs for user, system,maintenance, operations, and businesstraining and/or documentation beenconsidered in the design?

Does the design of the system incorporateSection 508 standards?

Have all stakeholders, including the end-

user community, been kept informedand/or consulted as appropriate during theDesign Phase?

Does the design introduce the need tomodify the Acquisition Plan?

Given the proposed design, will the budgetbe sufficient to meet the needs of theproject completion?

Page 38 of 74


39/74

Does the design align with the EnterpriseArchitecture Technical Reference Model?

Will the design facilitate theaccomplishment of performance metrics?

Are measurement indicators tailored andshow clear line of sight to specific BRM lineof business or sub-functions?

Do any of the approved change requestsfor the project require modification in cost,schedule, scope, or resources?

If required, has a Computer MatchAgreement been finalized to establishconditions, safeguards, and procedures fordisclosing data?

Are the types of tests, the acceptancecriteria for those tests, and the manner of

testing defined in sufficient detail?Does the test plan define all the types of tests (unit, functional, integration, system,security, performance (load and stress),user acceptance, and/or independentverification) that are to be carried out?

Does the test plan describe the roles andresponsibilities of individuals involved inthe testing process and the traceabilitymatrix?

Are the resources needed for the hardwareand software environments documented inthe test plan?

Are all other elements relevant to testplanning and execution described in detail?

Does the Test Plan include detailed TestCase Specifications that describe thepurpose and manner of each specific test,the required inputs and expected resultsfor the test, step-by-step procedures forexecuting the test, and the Pass/Not Passcriteria for determining acceptance?

Have applicable test cases been developedto address Section 508 Standards?

Are security control test completion datesfor all systems associated with thisproject/investment within the last 365

Page 39 of 74


40/74

days?

Does the Contingency/Disaster RecoveryPlan include complete descriptions of thestrategy and courses of action if there is aloss of use of the established businessproduct (e.g., system) due to factors suchas natural disasters or system or securityfailures?

Does the recovery strategy meet statedrecovery time and recovery pointobjectives?

Are backup procedures and responsibilitieswell-designed and fully documented?

Are post-disaster recovery proceduresincluded in the design?

Have contingency/disaster recovery plansfor all systems associated with thisproject/investment been tested within thelast 365 days?





Low)








(2 or more = NP)


Approve


Discontinue Project

Page 40 of 74


41/74


Signature: Date:Print

Name: Title:

Page 41 of 74


42/74


43/74

Deliverable Name Completeness (1-3)

Accuracy (1-3)

Adequacy(1-3)

Comments

Test Plan (Final)

BusinessProduct (Final)

Operations&MaintenanceManual (FinalDraft)

System SecurityPlan (Final Draft)

Security RiskAssessment

Training Plan(Final Draft)

TrainingMaterials (FinalDraft)

User Manual(Final Draft)

Mandatory Exit Criteria: The objective is to determine if the code and/or other deliverables needed tobuild the Business Product have been completed within cost, schedule, and

scope guidelines.Exit Criteria Pass (P)/

Not Pass(NP)

Comment

Business Productsatisfies therequirementsestablished andrefined during theRequirements andDesign Phases.

Test Plan ensuresthat all test caseswill be adequatelyevaluated andexecuted, andsystem tested toensurerequirements aremet.

Page 43 of 74


44/74

Security plans andrisk assessmentsare complete andin compliancewith regulatoryrequirements.

Variances frombaselines havebeen identifiedand mitigated.[Cost andschedulevariances andscope changesare identified,significantvariances areexplained, andCorrective ActionPlans (CAPs) orrebaselinerequests are inplace asappropriate.]

Investmentbaselines havebeen reviewedand revised as

appropriate.[Should thisinvestmentcontinue as-is, bemodified, or beterminated basedon currentknowledge?]

The ProjectManagement Planand componentplans have beenreviewed andappropriatelyupdated. [Thisincludes RiskManagement,Acquisition Plan,ChangeManagement,

Page 44 of 74


45/74

ConfigurationManagement,ProjectCategorization,RequirementsManagement,Communication

Plan,WBS/Schedule,IV&V Planning,QualityAssurance,RecordsManagement,Staff Development Planand SecurityApproach.]

Suggested QuestionsReviewers will fill in their Role next to relevant questions and comment onobservations Role Question Comment

Have the types of tests, the acceptancecriteria for those tests, and the manner of testing been finalized?

Has test files and/or test data been

developed?Have the Test Plan and Test Cases beenfinalized?

Are new custom-software programsdeveloped, new databases build and/orsoftware components integrated?

Has the developer placed code or otherdeliverables under configuration controland has change control been performed, asneeded?

Have unit and integration testing beenperformed by the developer with testresults appropriately documented?

Has the developer ensured that allcomponents of the system functioncorrectly and interface properly with othercomponents?

Page 45 of 74


46/74


47/74

original source code, the binaryexecutable, and the data repository (ies)?

If this is a software development effort, hasthe developer transformed the logicalinformation documented in the designphase and transformed it into source code?

Have necessary infrastructure andassociated products been acquired,configured, and integrated?

Does the software Business Product alsoinclude a Version Description Documentthat identifies and describes allconfiguration items that comprise aspecific build or release of the BusinessProduct?

Have the requirements identified forSection 508 compliance been incorporatedinto the system?

As a result of the Development activities,do any of approved change requests forthe project require modification in cost,schedule, scope, resources, or acquisitionplanning?

Has static code analysis been performed toidentify security vulnerabilities?

Has the Validation Readiness Review been

conducted to provide assurance that thesoftware that is about to enter systemtesting has completed a thoroughunit/module/software integration test?





Low)


Page 47 of 74


48/74






Impact/HighProbability)(2 or more = NP)


Approve


Discontinue Project



Title:

Page 48 of 74


49/74

3.7 REVIEW - TEST PHASE



Test Phase Review The Test Stage Gate Review evaluates whether the project should proceed tothe Implementation Phase.

ResponsibilitiesProject Manager Responsibilities in Test Phase Review

The Project Manager is responsible and accountable for the successfulexecution of the Test Phase. The Project Manager is responsible for leadingthe Integrated Project Team that accomplishes the Test Phase activities anddeliverables.Test and Evaluation Team Responsibilities in Test Phase Review

The Test and Evaluation Team is responsible for Business Product testing anddocumentation of test results.Users Responsibilities in Test Phase ReviewSelected users may be required to participate in testing.Critical Partners Responsibilities in Test Phase Review

The Critical Partners review test procedures and outcomes in their areas.

Security: Check that the validation tests confirm the security of theBusiness Product. Penetration tests and vulnerability scans areexecuted, documented, and any failed components are reworked.

Acquisition: Determine if changes are reviewed to determine if anycontract modifications are necessary.

Finance: Conclude that Changes are reviewed to determine thefinancial impact.

Performance: Determine if Measurement indicators support theperformance measures agreed upon and validation tests confirm theperformance measures. Ensure that system functionality is performingas stated and is able to achieve performance goals.



Adequacy(1-3)

Comments

Page 49 of 74


50/74

ImplementationPlan (Final)

Test Reports(Final)

Mandatory Exit Criteria:

The objective is to determine if the test processes have been executedaccording to plan and whether the tests verify that the implementation of the Business Product will be successful.Exit Criteria Pass (P)/

Not Pass(NP)

Comment

Test plan ensuresthat test caseswill be executedto make certainthat requirementsare met.

Testing of theBusiness Productsupports thedecision to moveto theImplementationPhase.

ImplementationPlan providesdetailedinformation onthe move of theBusiness Productinto production.

Variances frombaselines havebeen identifiedand mitigated.[Cost and

schedulevariances andscope changesare identified,significantvariances areexplained, andCorrective ActionPlans (CAPs) orrebaseline

Page 50 of 74


51/74

requests are inplace asappropriate.]

Investmentbaselines havebeen reviewedand revised asappropriate.[Should thisinvestmentcontinue as-is, bemodified, or beterminated basedon currentknowledge?]

The ProjectManagement Plan

components havebeen reviewedand appropriatelyupdated. [Thisincludes RiskManagement,Acquisition Plan,ChangeManagement,ConfigurationManagement,

ProjectCategorization,RequirementsManagement,CommunicationPlan,WBS/Schedule,IV&V Planning,QualityAssurance,Records

Management,Staff Development Planand SecurityApproach.]


Page 51 of 74


52/74


Has the final Implementation Plan beendeveloped?

Does the Implementation Plan describehow the business product will be installed,deployed, and transitioned into theoperational environment?

As a result of the Test activities and thedevelopment of the Implementation Plan,do any of approved change requests forthe project require modification in cost,schedule, scope, resources, or acquisitionplanning?

Has acceptance testing been completedand do the outcomes verify readiness fortraining and implementation?

Was a summary report created at the endof the test phases that completelydocuments the overall test results,including summarizing the test activitiesand describing variances?

Was the identification of unexpectedproblems and/or defects that wereencountered included?

Was the validity of Performance Metricsevaluated?

Were any applicable additional testsconducted to validate documentation,training, contingency plans, disasterrecovery, and installation?





Low)








(2 or more = NP)

Page 52 of 74


53/74


54/74

3.8 REVIEW - IMPLEMENTATION PHASE



Operational Readiness Review The Operational Readiness Review (ORR) is a formal inspection conducted todetermine if the final IT solution or automated system/application that hasbeen developed or acquired, tested, and implemented is ready for releaseinto the production environment for sustained operations and maintenancesupport. The IT governance organization cannot delegate this review.

ResponsibilitiesProject Manager Responsibilities Implementation Review The Project Manager is responsible and accountable for the successfulexecution of the Implementation Phase. The Project Manager is responsiblefor leading the Integrated Project Team that accomplishes theImplementation Phase activities and deliverables.IT Governance Organization Responsibilities in ImplementationReview

The IT governance organization conducts the Operational Readiness Review.Integrated Project Team Responsibilities in Implementation Review

The Integrated Project Team members (regardless of the organization of permanent assignment) are responsible for accomplishing assigned tasks asdirected by the Project Manager.Critical Partners Responsibilities in Implementation Review

The Critical Partners provide oversight, advice and counsel to the ProjectManager on the conduct and requirements of the Implementation Phase.Additionally, they provide information, judgments, and recommendations tothe Business Owner and IT governance organization during investmentreviews and in support of Investment Baselines.

Enterprise Architecture: Confirm that approved change requests are

compliant with the Enterprise Architecture.Security: Determine if the Authority to Operate, including the SystemCertification and Accreditation, is complete and System of RecordNotice is published.

Acquisition: Guarantee that the contracts are being fulfilled accordingto award or approved changes and completed contracts are closedappropriately.

Page 54 of 74


55/74

Budget: Ascertain if change requests are reviewed to determine if anew financial analysis is required.

Finance: Ascertain if actual expenses are in accordance with thebudget plan.

HR : Find if issues related to staffing, workforce, or other HR areas havebeen addressed.

Performance: Confirm that the completed Business Product isoperating as expected and is positioned to meet performance targets.



Adequacy(1-3)

Comments

Authority toOperate with

components(Final)

System of RecordNotice (Final)

Business Product(Final)

ProjectCompletionReport (Final)

Service LevelAgreements (SLA)and/orMemorandum(s)of Understanding(MOU)

Contingency/Disaster RecoveryPlan (Final)

Operations &

MaintenanceManual (Final)

System SecurityPlan (Final)

Security RiskAssessment(Final)

Training Plan

Page 55 of 74


56/74

(Final)

Training Materials(Final)

User Manual(Final)

Mandatory Exit Criteria: The objective is to determine if the project has finalized implementation.Exit Criteria Pass (P)/

Not Pass(NP)

Comment

BusinessProduct readyfor productionservice andnotification of

the new solutionis provided toall users andstaff who areaffected.

No outstandingconcerns amongstakeholdersregardingimplementation.

Security andauthorization tooperatedocuments arecomplete andthe system isconsideredCertified andAccredited

Variances frombaselines havebeen identifiedand mitigated.[Cost andschedulevariances andscope changesare identified,significantvariances are

Page 56 of 74


57/74

explained, andCorrectiveAction Plans(CAPs) orrebaselinerequests are inplace as

appropriate.]Investmentbaselines havebeen reviewedand revised asappropriate.[Should thisinvestmentcontinue as-is,be modified, orbe terminatedbased oncurrentknowledge?]

The ProjectManagementPlan andcomponentplans have beenreviewed andappropriately

updated. [Thisincludes RiskManagement,Acquisition Plan,ChangeManagement,ConfigurationManagement,ProjectCategorization,Requirements

Management,CommunicationPlan,WBS/Schedule,IV&V Planning,QualityAssurance,RecordsManagement,Staff

Page 57 of 74


58/74

DevelopmentPlan andSecurityApproach.]


Has a System Certification, includingmanagement, operational, and technicalsecurity certifications, ensures compliancewith information security requirementsbeen successfully completed?

Has formal documentation of Section 508Certification or Exception been completed?

Has a System Certification, including bothsecurity and technical certifications, thatensures compliance with securityrequirements been successfullycompleted?

Have required corrective actions beeninitiated on any outstanding documents?

Are all required Service Level Agreement(s)(SLAs) and Memorandum(s) of Understanding (MOU) fully executed and ineffect, specifying each party'srequirements, responsibilities and period of performance including performanceguarantees?

Has the Operations & Maintenance Manualbeen updated based on results from the

Test Phase?

Has the Systems Security Plan beenfinalized to describe the security controls,as defined by the National Institute of Standards and Technology that aredesigned and implemented within thesystem?

Does the Training Plan adequately describethe goals, learning objectives, andactivities of the information that has beenimplemented?

Have the Training Materials been reviewed

Page 58 of 74


59/74

and updated to include complete andaccurate documentation on thedeployment of the Business Product?

Has the Training Program been executed?

Does the Security Risk Assessment providea formal risk assessment including theanalysis of the security functionalrequirements and the identification of theprotection requirements?

Does the risk assessment include theidentification of all threats to andvulnerabilities in the information system;the potential impact that a loss of confidentiality, integrity, or availabilitywould have and the identification andanalysis of security controls?

Has the User Manual been updated basedon the results of the Test Phase?

Does the Business Product that resultsfrom the development and test effortssatisfy the established requirements?

Have necessary infrastructure andassociated products been acquired,configured, and integrated?

Is the Version Description Document thatidentifies and describes all configuration

items that comprise a specific build orrelease of the Business Product in use?

Have all necessary data conversion stepsbeen completed?

Have any specified periods of paralleloperation been completed successfully?

Have all stakeholders been notified of theimplementation, including information onthe schedule, the benefits, the changes,and the impact on end-users?

Have code attack simulations usingautomated scans and penetration testingon pre-production servers been carriedout?

As a result of the Development activities,do any of approved change requests forthe project require modification in cost,schedule, scope, resources, or acquisition

Page 59 of 74


60/74

planning?

Has an accurate Project Completion Reportthat describes any differences betweenproposed and actual accomplishments,documents lessons learned, provides astatus of funds, and provides anexplanation of any open-ended actionitems, along with a certification of conditional or final closeout of thedevelopment project, been developed andhave the processes been implemented?

Has the Contingency/Disaster RecoveryPlan been updated based on results fromthe Development and Test Phases?

Are backup procedures and responsibilitieswell-designed and up-to-date?





Low)








(2 or more = NP)

Recommendations


Approve


Discontinue Project

Governance Forward:

Page 60 of 74


61/74

Forwarded to:


Title:

Page 61 of 74


62/74


63/74

by the users when questions or problems occur with the daily operations of the system. Help Desk personnel need to maintain a level of proficiency withthe Business Product.Operations or Operators Responsibilities in the Operations &Maintenance Phase Review (turn on/off systems, start tasks, backupetc)For many mainframe systems, an operator provides technical support for a

program. The operator performs scheduled backup, performs maintenanceduring downtime and is responsible to ensure the system is online andavailable for users. Operators may be involved with issuing user IDs or loginnames and passwords for the system.Customers Responsibilities in the Operations & Maintenance PhaseReview

The customer needs to be able to share with the project manager the needfor improvements or the existence of problems. Some users live with asituation or problem because they feel they must. Customers may feel thatchange will be slow or disruptive. Some feel the need to create work-arounds. A customer has the responsibility to report problems, makerecommendations for changes to a system, and contribute to OperationalAnalyses.Program Analysts or Programmer Responsibilities in the Operations& Maintenance Phase Review Interprets user requirements, designs and writes the code for specializedprograms. User changes, improvements, enhancements may be discussedin Joint Application Design sessions. Analyzes programs for errors, debugsthe program and tests program design.Configuration Control Board Responsibilities in the Operations &Maintenance Phase Review:

A board of individuals may be convened to approve recommendations forchanges and improvements to the Business Product. This group may bechartered. The charter should outline what should be brought before thegroup for consideration and approval. The board may issue a ChangeDirective.Users Group or Team Responsibilities in the Operations &Maintenance Phase ReviewA group of computer users who share knowledge they have gainedconcerning a program or system. They usually meet to exchangeinformation, share programs and can provide expert knowledge for a system

under consideration for change.Contract Manager Responsibilities in the Operations & MaintenancePhase Review

The Contract Manager has many responsibilities when a contract has beenawarded for maintenance of a program. The Contract Manager should havea certificate of training for completion of a Contracting Officers TechnicalRepresentative (COTR) course. The Contract Managers main role is to makesure that the interests of the Contracting Office are protected and that no

Page 63 of 74


64/74

modifications are made to the contract without permission from theContracting Office.Data Administrator Responsibilities in the Operations &Maintenance Phase ReviewPerforms tasks which ensure accurate and valid data are entered into theBusiness Product. Sometimes this person creates the information systemsdatabase, maintains the databases security and develops plans for disaster

recovery. The data administrator may be called upon to create queries andreports for a variety of user requests. The data administratorsresponsibilities include maintaining the databases data dictionary. The datadictionary provides a description of each field in the database, the fieldcharacteristics and what data is maintained with the field.Telecommunications Analyst and Network System Analyst Responsibilities in the Operations & Maintenance Phase ReviewPlans, installs, configures, upgrades, and maintains networks as needed. If the investment requires it, they ensure that external communications andconnectivity are available.Information Systems Security Officer (ISSO) Responsibilities in theOperations & Maintenance Phase Review

The ISSO has a requirement to review system change requests, review andin some cases coordinate the Change Impact Assessments, participate in theConfiguration Control Board process, and conduct and report changes thatmay be made that affect the security posture of the systemCritical Partners Responsibilities in the Operations & MaintenancePhase Review

The Critical Partners provide oversight, advice and counsel to the ProjectManager during the Operations and Maintenance Phase.

Enterprise Architecture: Confirm that the business product is beingoperated in accord with Enterprise Architecture guidelines.

Security : Determine if the Authority to Operate, System Certificationand Accreditation and Privacy Impact Assessments are reviewed andupdated at the appropriate times for continued operation. Ensure thatSecurity documents are updated as necessary in response tocontinuous testing and monitoring. Confirm that system backups,physical security, contingency planning, and continuous securitymonitoring and testing are operated in accord with established securitycontrols.

Acquisition: Guarantee that contracts are being fulfilled according toaward or approved changes.

Budget: Determine if modification requests include appropriate justification and cost benefit analysis.

Finance: Ascertain if actual expenses are in accordance with thebudget plan.

Performance: Confirm service level objectives are being met and thatperformance measurements and system logs are being maintained.Determine that modifications needed to resolve errors or performance

Page 64 of 74


65/74

problems are made in accord with change control procedures. Ensurethat annual Operational Analysis is performed to evaluate systemperformance and user satisfaction to verify that risk and performancegoals are under control.



Adequacy(1-3)

Comments

AnnualOperationalAnalysis (Final)

Disposition Plans(Final)

Continued

Authority toOperation (Final)

Privacy ImpactAssessment(Final)

Mandatory Exit Criteria: The objective is to verify that the Business Product is managed andsupported in a robust production environment and to determine whether theBusiness Product is still cost-effective to operate or if it should be retired.Exit Criteria Pass (P)/

Not Pass(NP)

Comment

Annual review of the operationprovides aframework fordeciding whatenhancements ormodifications areneeded orwhether thebusiness productshould bereplaced ordisposed of.

Documentationand the trainingprograms includeinput from

Page 65 of 74


66/74

stakeholders.

Variances frombaselines havebeen identifiedand mitigated.[Cost andschedulevariances andscope changesare identified,significantvariances areexplained, andCorrective ActionPlans (CAPs) orrebaselinerequests are inplace asappropriate.]Investmentbaselines havebeen reviewedand revised asappropriate.[Should thisinvestmentcontinue as-is, bemodified, or be

terminated basedon currentknowledge?]

The ProjectManagement Planand componentplans have beenreviewed andappropriatelyupdated. [Thisincludes RiskManagement,Acquisition Plan,ChangeManagement,ConfigurationManagement,ProjectCategorization,Requirements

Page 66 of 74


67/74

Management,CommunicationPlan,WBS/Schedule,IV&V Planning,QualityAssurance,

RecordsManagement,Staff Development Planand SecurityApproach.]



Does the plan for the Annual Operationalassessment include a review of the CPICevaluation and the performance metricsduring operation to determine whether thebusiness product is meeting original userrequirements and any new requirements orchanges?

Does the plan provide a means to analyzealternatives for deciding on new functional

enhancements and/or modifications to thebusiness product, or the need to dispose of or replace the business product altogether?

Does the annual System Re-Certificationprovide a comprehensive re-evaluation of the management, operational, andtechnical security controls implemented foran information system to ensure that thesystem is continuing to operate at anacceptable risk level?

Is the annual System Re-Accreditation up-to-date, fully documenting the officialmanagement decision to authorizecontinued operation of an informationsystem?

Has the annual Operational Analysisadequately evaluated system performance,user satisfaction with the system,adaptability to changing business needs,

Page 67 of 74


68/74

and new technologies that might improvethe system?

Do ongoing change requests incorporatethe requirements for Section 508?

Do any of the approved change requestsfor this project/investment require a

modification in the financial analysis?Do measurement indicators support theperformance measures agreed upon?

Are contingency plan test dates for allsystems associated with thisproject/investment within the last 365days?

Is continuous security monitoring of selected controls conducted on an ongoingbasis to ensure that maintenance patchesand enhancements have not introducedany vulnerabilities?

Have the Operations Manual, BusinessCase Analysis, and Contingency/DisasterRecovery Plan been updated as required?

Is there a well-developed Disposition Planthat addresses how the components of theoperating Business Product will be handledat the completion of operations to ensureproper disposition of all the componentsand to avoid disruption of the individualsand/or any other Business Productsimpacted by the disposition?

Does the plan include methods for thedeliberate and systematicdecommissioning of the Business Productwith appropriate consideration of recordsmanagement?

Does the privacy impact assessment (PIA)for all systems associated with thisproject/investment include information onany changes?





Low)


Page 68 of 74


69/74







(2 or more = NP)


Approve

Approve with

ConditionsDiscontinue Project



Title:

Page 69 of 74


70/74

3.10 REVIEW - DISPOSITION PHASE



Disposition Phase ReviewA Disposition Review is conducted to ensure that a system/application orother IT situation has been completely and appropriately disposed, therebyending the lifecycle of the IT project.

This phase-end review shall be conducted again within six months afterretirement of the system. The Disposition Review Report also documents the

lessons learned from the shutdown and archiving of the terminated system.

ResponsibilitiesProject Manager Responsibilities in the Disposition Phase ReviewAuthors the Disposition Plan and ensures that all aspects of the DispositionPlan are followed. The Disposition Plan should outline all roles andresponsibilities for all actions related to the close down and archive of thesystem.Technical Support or Vendor Support Responsibilities in theDisposition Phase Review

The Disposition Plan may call for the Technical Support Personnel to sendsystem related hardware to a warehouse or may reassign equipment to anew or replacement system. Technical Support Personnel or Operators mayperform the cutoff of users access per instructions from the SecurityManager. Technical Support personnel may assist with the archive of theInformation Systems data.Data Administrator Responsibilities in the Disposition Phase Review

The Disposition Plan may direct that only certain Business Product data bearchived. The Data Administrator would identify the data and assisttechnical personnel with the actual archive process. The Data Administratormay be involved with identifying data which due to its sensitive nature mustbe destroyed. They would also be involved with identifying and migratingdata to a new or replacement Business Product.User Services (Training & Help Desk) Responsibilities in theDisposition Phase ReviewUser Services includes training, telecommunications, and Help Deskpersonnel. The training component coordinates and schedules thedevelopment and delivery of all training and facilitates the development of systems training methods and materials. In this phase, User Services may

Page 70 of 74


71/74


72/74

3) (1-3)

Project Archives

Mandatory Exit Criteria: The objective is to have an orderly shutdown of the Business Productoperation.

Exit Criteria Pass (P)/Not Pass(NP)

Comment

Data archiving,security, and dataand systemsmigrations arecomplete.

If appropriate, has

the migration of data and thefunction to a newsystem been well-planned.

Final phase-endreview has beenconducted.

The ProjectManagement Plan

and componentplans have beenreviewed andappropriatelyupdated. [Thisincludes RiskManagement,Acquisition Plan,ChangeManagement,Configuration

Management,ProjectCategorization,RequirementsManagement,CommunicationPlan,WBS/Schedule,IV&V Planning,Quality

Page 72 of 74


73/74

Assurance,RecordsManagement,Staff Development Planand SecurityApproach.]


Are the Project Archives that preserve vitalinformation, including both documentationof project execution and the data from theproduction system, appropriatelypreserved?

Is Lessons Learned included in the ProjectArchives?

Have security objectives, including securedata and system transfer, sanitization anddisposal of media, been accomplished?

Has the Disposition Plan, including theorderly breakdown of the system, itscomponents and the data within, beenfollowed?

Has a final phase-end review beenconducted after the system retirement toascertain if the system and data have beencompletely and appropriately disposed of?

Are completed contracts closedappropriately?




Impact

(High, Medium, orLow)







Impact/High

Page 73 of 74


74/74

Probability)(2 or more = NP)

Recommendations


Approve


Discontinue Project



Title: