Systems Engineering – A Systems Engineering – A Global PerspectiveGlobal Perspective

Systems Engineering Systems Engineering Seminar SeriesSeminar Series

Dinesh Verma, Ph.D.Professor and Associate DeanStevens Institute of Technology

dverma@stevens.edu

Underlying “TONE” for this Underlying “TONE” for this PresentationPresentation

IdealisticIdealistic or Realistic or Realistic(Actually, Cynical)

The Contextual Setting for this The Contextual Setting for this PresentationPresentation

““Rear View Looking” or “Forward Looking” Rear View Looking” or “Forward Looking”

Systems Engineering – ExpectationSystems Engineering – Expectation

Successful implementation of proven,

disciplined systems engineering

processes results in a total system

solution that is:

Robust to changing technical, production, and

operating conditions;

Adaptive to the needs of the users; and

Balanced among the multiple requirements,

design considerations, design constraints, and

program budgets.

3

Customer Related Input:• Isolation from real “user”

• Customer requirements and (even) identity not clear

• Customer doesn’t know what they want

• Scope creep; Undocumented system scope and functionality

• User/buyer too distant

• Don’t understand the customer value system

Management Related Input:• Executive management doesn’t buy in

• Lack of teamwork

• Program Managers not empowered

• Program manager and capture managers are different

• Unstable funding stream; Lack of upper management support

Organizational/Cultural Input (Some Perceptions):• SEA only adds to the Project Cost

• SEA often seen as an “outside” team or “project reviewer” role

Some Inhibitors to Good Systems Engineering:Based on a survey of IT architects and project managers

4

We would like you to We would like you to build us a lawn build us a lawn mower please!mower please!

Deploying Systems Engineering within a Deploying Systems Engineering within a Commercial Global Leader: Commercial Global Leader: Some Some ResultsResults

Systems Engineering Has Been Applied to Systems Engineering Has Been Applied to Both Internal and Commercial Accounts Both Internal and Commercial Accounts

20012001200020002nd qtr2nd qtr1st qtr1st qtr 4th qtr4th qtr3rd qtr3rd qtr 2nd qtr2nd qtr1st qtr1st qtr 4th qtr4th qtr3rd qtr3rd qtr

1st project uses SE principlesSE organization introduced

SE Reviews / Scorecards introduced Directive to use SE on projects >$1M

‘Fundamentals of SE’ course introduced1st commercial account uses SE

Directive to use SE on projects > $500KFormal SE dept created

20032003200220022nd qtr2nd qtr1st qtr1st qtr 4th qtr4th qtr3rd qtr3rd qtr 2nd qtr2nd qtr1st qtr1st qtr 4th qtr4th qtr3rd qtr3rd qtr

13 projects using SE

SEI introduces CMMI 1.1‘SE Design’ Class introduced

SE deliverables templates provided

SE team grows to 14

17 completed and over 50 active projects using SEOver 230 trained in SE Fundamentals

SE team grows to 3012 completed and 13 active projects using SE

SE process integration - AMS MS

SE process updated for CMMI

SE process integration - GS Method

Systems Engineering Process defines Systems Engineering Process defines deliverables and a series of Reviews (Part I)deliverables and a series of Reviews (Part I)

Need / OpportunityIdentification

Detailed Design

Component Architecture

ConceptualSystem

Specification

CustomerCustomerBaselineBaseline

SystemSystemBaselineBaseline

Architecture/ComponentArchitecture/ComponentBaselineBaseline

DesignDesignBaselineBaseline

SystemsSystemsRequirementsRequirementsReview (SRR)Review (SRR)

PreliminaryPreliminaryDesign Design Review (PDR)Review (PDR)

CriticalCriticalDesignDesignReview CDR)Review CDR)

Customer Provided Systems Engineering Provided Component Developer Provided

Business Business RequirementsRequirementsReview (BRR)Review (BRR)

BusinessBusinessRequire.Require.Specs.Specs.

SystemsSystemsReq’mentReq’mentSpecsSpecs

RTVMRTVMSystemSystemLevelLevelArchitect.Architect.

ComponentComponentLevelLevelArchitectureArchitecture

TestTestArchitectureArchitecture

ComponentComponentDesignDesign

Component Component Test PlanTest Plan

ComponentComponentReq’ment Req’ment SpecsSpecs

ComponentComponentRTVMRTVM

Systems Engineering Process defines Systems Engineering Process defines deliverables and a series of Reviews (Part II)deliverables and a series of Reviews (Part II)

TestTestBaselineBaseline

ProductionProductionBaselineBaseline

DesignDesignBaselineBaseline

TestTestReadinessReadinessReview (TRR)Review (TRR)

Production Production Readiness Readiness Review (PRR)Review (PRR)

Customer Provided Systems Engineering Provided Component Developer Provided

CDRCDR

New ProductionSystem

Test and ProductionSystem Update

Development

System System Test Test DataData

Test Test Traceability Traceability Matrix.Matrix.

Move toMove toProd.Prod.PlanPlan

Data Data MigrationMigrationPlanPlan

Detail Design

Comp.Comp.DesignDesign

Comp. Comp. Test PlanTest Plan

System Test Provided Service Delivery / Managed Ops Provided

DeploymentDeploymentPlanPlan

System System Test Plan /Test Plan /Test CasesTest Cases

System System TestTestStrategyStrategy

ReleaseReleaseContentContent

ISM delivered 5% under budget ISM delivered 5% under budget and with higher quality in production and with higher quality in production

The charts here are based on data collected from a recent study analyzing project defects by type and phase. Here ISM defects by phase is compared to 46 similarly sized projects not utilizing SE.

Total defect counts for non-SE projects exhibited 53.4% of total project defects during the Test Phase of the project. On ISM defects were detected earlier in the project life-cycle. In fact 56% of ISM detects were detected in Plan Phase.

The chart on the left illustrates the cost implications of early defect detection as found with ISM 2.0.

In effect ISM 2.0 expended 2.4 times less than what would have normally been required for the non-SE oriented projects compared to in the study.

IGA Metrics show 8% cost avoidance when IGA Metrics show 8% cost avoidance when comparing SE&A projects to non-SE&A comparing SE&A projects to non-SE&A projectsprojects

Cumulative Costs to Repair Requirement and Design Defects

$0

$200,000

$400,000

$600,000

$800,000

$1,000,000

$1,200,000

$1,400,000

$1,600,000

Req Design Build Test Install

Phase Detected

Accu

mu

late

d C

osts

SE&A non SE&A

= 15% of $10.7MM Baseline

= 7% of $10.7MM Baseline

= 8% Cost Avoidance

Similar Initiatives Underway at…Similar Initiatives Underway at…

11

R&D spend per product launched is much R&D spend per product launched is much higher than key competitorshigher than key competitors Cost per product program is much higher

than major competitors

high R&D spend as % of sales

absolute spend >2* any other

not many more products launched than other players with much less R&D spend

(benchmarking takes into account broader scope and uses comparable definition of what is a product)

Higher sales volumes per product keep R&D costs per unit shipped in line

no clear advantage in terms of product competitiveness

higher volumes come from other resources: global reach; brand’ strong channels; logistics

No economies of scale from R&D despite spend more than twice any other player

R&D spend per device launched (all devices, 2003, $ millions)

0 10 20 30 40 50 60

LG Electronics

Sony Ericsson

Siemens

Samsung

Motorola

Nokia

R&D per device launched, $ millions

R&D spend per device launched vs. volume of devices shipped; bubble size is R&D spend per device LAUNCHED (2003)

NokiaMotorola

Samsung

Siemens

Sony Ericsson

LG Electronics

0

5

10

15

20

25

30

0 25 50 75 100 125 150 175 200Volume of devices SHIPPED (2003, millions)

R&

D s

pend

per

dev

ice

SH

IPP

ED

(20

03,

$ m

illio

ns)

Despite much higher spend, products now Despite much higher spend, products now lag best competitors on key dimensionslag best competitors on key dimensions

Zero = about same as competitors, 1 = 1 standard deviation better than competitors, and so onCompetitors: Motorola, Panasonic, Sagem, Samsung, Sharp, Siemens and SonyEricsson

1

1

1

0

0

0

0

0

0

0

0

-1

-1

-2

-3 -2 -1 0 1 2 3

Menu system

Ease of name search

Ease of messaging

Technical reliability

Resistance to scratches

Quality of reception

Features/ functions

Durability/ build quality

Display

Buttons and keypad

Battery performance

Size & weight

Price

Design/ style/ appearance

Nokia versusAverage of Competitors

2

2

2

1

2

2

1

1

0

0

1

0

0

0

-3 -2 -1 0 1 2 3

Menu system

Ease of name search

Ease of messaging

Technical reliability

Resistance to scratches

Quality of reception

Features/ functions

Durability/ build quality

Display

Buttons and keypad

Battery performance

Size & weight

Price

Design/ style/ appearance

Nokia versusBest Competitors

Competitorsnow have better designs

Nokia still leads on ease of `phone

user

Key Concerns Identified – SymptomsKey Concerns Identified – Symptoms

Asset management and technology management is a weakness Architecture portfolio management is extremely poor

Design consistency is missing The norm seems to be reactive design versus proactive design

Architecture studies, tradeoffs, and evolution could be better planned (architecture archeology!) Architecture ownership is not clear, and often not executed Too many product/platform variants, each requiring special testing and support Reactive redesign during implementation – We don’t know the performance thresholds of our

architectures Architecture is often the result of implementation rather than the driver Too much discretion at the lowest implementation levels

– Example: Data and image formats Software quality could be better – expensive rework

currently required Requirements are not managed (end to end)

Competing and conflicting positions in external forums Requirements management (traceability) is often poor Interface management is poor Not enough linkage to business requirements

R&D Productivity ImpactedR&D Productivity ImpactedInnovation ImpactedInnovation ImpactedMorale ImpactedMorale Impacted

If these are symptoms, If these are symptoms, then what are the then what are the causes?causes?

I wish I had tools to I wish I had tools to help my help my management team management team realize the realize the importance of importance of architecting… short-architecting… short-term-ism gets in the term-ism gets in the way… our headaches way… our headaches today were born 1.5 today were born 1.5 to 2 years ago… to 2 years ago…

Sometimes I feel Sometimes I feel like I am in a like I am in a different different company than company than them… Business them… Business priorities don’t priorities don’t seem to be linked seem to be linked to Technology to Technology priorities”priorities”

Resulting SituationResulting Situation

Systems and devices often developed and tested through significant individual efforts and human networks – under significant schedule pressure Existing processes are often uniquely (one time use)

tailored/used Reuse is a myth in some cases, and getting in the way of

innovation in other cases (because of serious schedule pressures)

Processes and methodologies (and even language) are personality dependent, with no time for documentation (competency development?)

In the absence of clear organizational authority and accountability, personality based authority and accountability has evolved in the current situation (e.g., Roles and responsibilities between BGs and TPs)

CHALLENGE: Scale up NOKIA R&D efficiency (Time to Market) and CHALLENGE: Scale up NOKIA R&D efficiency (Time to Market) and innovation (Feature Leadership), without scaling up in size and innovation (Feature Leadership), without scaling up in size and compromising qualitycompromising quality Architectural thinking can contributeArchitectural thinking can contribute

Theory versus Theory versus (Virtual)(Virtual) Reality… Reality…Primary Reasons for Dysfunctional Behavior – My Primary Reasons for Dysfunctional Behavior – My OpinionOpinion

Confusion between “What you NEED” versus “What you WANT” Also called Gold-Plating

It is the moral duty of a systems engineer to articulate the resulting cost and schedule delta

Confusion with regard to the SYSTEM BOUNDARY This is more difficult for legacy systems with undocumented and

implied interfaces; and even more so for “network-centric systems” and “SoS”

Confusion (?) with regard to fidelity between the technical project scope and its allocated budget and schedule

The result is cynicism and complacency, along with other negative behavioral patterns

Lack of Leadership

0943-98Innovation Associates

Holistic Thinking versus Local Thinking…Holistic Thinking versus Local Thinking…

17

A few words about competency A few words about competency development…development…

Discipline Centric Discipline Centric Systems Engineering Systems Engineering Programs:Programs: These are These are programs where the major is programs where the major is designated only as Systems designated only as Systems Engineering Engineering

Domain Centric Systems Domain Centric Systems Engineering Programs:Engineering Programs: These are programs where the These are programs where the major is designated as X and major is designated as X and Systems Engineering; or Systems Engineering; or Systems and X Engineering.Systems and X Engineering.

In this case, the most common In this case, the most common instances of “X” Engineering are:instances of “X” Engineering are:

Industrial EngineeringIndustrial Engineering

Manufacturing EngineeringManufacturing Engineering

Electrical EngineeringElectrical Engineering

Management EngineeringManagement Engineering

Computer EngineeringComputer Engineering

Academic Perspective: Discipline and Academic Perspective: Discipline and Domain Centric Systems Engineering Domain Centric Systems Engineering ProgramsPrograms

The primary source of this data is: Fabrycky, W.J., “Systems Engineering Education in the United States”, Proceedings, Conference on Systems Integration (CSI), Stevens Institute of Technology, New Jersey, March 2003.

Air Force Institute of Technology Air Force Institute of Technology

California State UniversityCalifornia State University

Colorado School of MinesColorado School of Mines

Cornell UniversityCornell University

George Mason UniversityGeorge Mason University

George Washington UniversityGeorge Washington University

Iowa State UniversityIowa State University

Johns Hopkins UniversityJohns Hopkins University

National Technological UniversityNational Technological University

Naval Postgraduate SchoolNaval Postgraduate School

Oakland UniversityOakland University

Polytechnic University - FarmingdalePolytechnic University - Farmingdale

Portland State UniversityPortland State University

Purdue UniversityPurdue University

Rochester Institute of TechnologyRochester Institute of Technology

Southern Methodist UniversitySouthern Methodist University

Stevens Institute of TechnologyStevens Institute of Technology

University of Alabama - HuntsvilleUniversity of Alabama - Huntsville

University of ArizonaUniversity of Arizona

University of IdahoUniversity of Idaho

University of Illinois at Urbana-ChampaignUniversity of Illinois at Urbana-Champaign

University of Maryland University of Maryland

University of MassachusettsUniversity of Massachusetts

University of MinnesotaUniversity of Minnesota

University of Missouri - RollaUniversity of Missouri - Rolla

University of PennsylvaniaUniversity of Pennsylvania

University of Rhode IslandUniversity of Rhode Island

University of Southern CaliforniaUniversity of Southern California

University of VirginiaUniversity of Virginia

VPI and State UniversityVPI and State University

Academic Perspective: Universities with Academic Perspective: Universities with Discipline Centric Systems Engineering Discipline Centric Systems Engineering Programs - 30Programs - 30

The list in the primary reference contains 35 records. Four of these referred to Universities with only an Undergraduate Program in Systems Engineering, and one to a University with only a Doctoral Program in Systems Engineering.

Auburn University

Boston University

California State University - Fullerton

Case Western Reserve University

Georgia Tech

Massachusetts Institute of Technology

New Jersey Institute of Technology

North Carolina A and T University

Northeastern University

Ohio State University

Ohio University

Polytechnic University

Purdue University

Rensselear Polytechnic University

Rutgers, The State University

San Jose State University

Stanford University

Texas Tech University

University of Alabama - Huntsville

University of Arizona

University of Central Florida

University of Connecticut

University of Florida

University of Houston

University of Illinois

University of Memphis

University of Michigan Ann Arbor

University of Michigan-Dearborn

University of Minnesota

University of Pittsburgh

University of Rhode Island

University of South Florida

University of Southern California

University of Southern Colorado

University of St. Thomas

Virginia Tech

Wichita State University

Youngstown State University

Academic Perspective: Universities with Domain Academic Perspective: Universities with Domain Centric Systems Engineering Programs - 38Centric Systems Engineering Programs - 38

3030++3838

~15~15

Available…Available… Relevant…Relevant…

Industry/Government Perspective: Industry/Government Perspective: Systems Engineering Education and Systems Engineering Education and TrainingTraining

Definition of Relevant:Definition of Relevant:1.1.Relevance of the curriculum – orientation Relevance of the curriculum – orientation

to DoD projects and programsto DoD projects and programs2.2.Portability and flexibility of the delivery Portability and flexibility of the delivery

format – distributed organizationsformat – distributed organizations3.3.Hybrid – credit and continuing educationHybrid – credit and continuing education

Basis:Basis:The Boeing SE Education Program (50 > 15 > 6 > 2)The Boeing SE Education Program (50 > 15 > 6 > 2)A DoD Component SE Education Program (80 > 25 > A DoD Component SE Education Program (80 > 25 >

11 > 2)11 > 2)

??3030++3838

~15~15

Available…Available… Relevant…Relevant… Critical Mass…Critical Mass…

Industry/Government Perspective: Industry/Government Perspective: Systems Engineering Education and Systems Engineering Education and TrainingTraining

Definition of Critical Definition of Critical Mass:Mass:

1.1.Number of Tenured Number of Tenured or Tenure Track or Tenure Track FacultyFaculty

2.2.Number of Faculty Number of Faculty with DoD/Aerospace with DoD/Aerospace Relevant Relevant Project/Program Project/Program ExperienceExperience

3.3.Bench StrengthBench Strength

……Why?Why?

??3030++3838

~15~15

Available…Available… Relevant…Relevant… Critical Mass…Critical Mass…

Industry/Government Perspective: Industry/Government Perspective: Systems Engineering Education and Systems Engineering Education and TrainingTraining

Definition of Critical Definition of Critical Mass:Mass:

1.1.Number of Tenured Number of Tenured or Tenure Track or Tenure Track FacultyFaculty

2.2.Number of Faculty Number of Faculty with DoD/Aerospace with DoD/Aerospace Relevant Relevant Project/Program Project/Program ExperienceExperience

3.3.Bench StrengthBench Strength

It is my opinion that we do not have a critical mass in graduate SE It is my opinion that we do not have a critical mass in graduate SE education in the US…education in the US…

However, we do have a very mature base and recognition within industry However, we do have a very mature base and recognition within industry and government to facilitate the building of this critical mass…and government to facilitate the building of this critical mass…

Systems Engineering Education: Relevant graduate level courses,

anchored with academic rigor and yet sensitive to current system development and integration challenges.

Flexible delivery formats – online/distance; modular

Flexible for part-time course loads

– Full time leave of absence for educational purposes is a thing of the past…

Systems Engineering Training: Relevant and focused training courses

on specific subjects of immediate relevance

Industry/Government Expectations: Industry/Government Expectations: Systems Engineering Education, Training, Systems Engineering Education, Training, EducationEducation

Systems Engineering Research:» Development of an SE Toolkit

(Templates, Metrics, etc.) of immediate utility within an organization Domain centric

Specific purpose of enhancing development efficiency and effectiveness

Usage centric

» Architectural assessment frameworks Open systems (a much abused

phrase)

Domain centric

» Others…In an environment of high demand and scarce In an environment of high demand and scarce supply, there are many experts… but who sets supply, there are many experts… but who sets the minimum thresholds on relevance and the minimum thresholds on relevance and quality?quality?

Crossing the boundaries… the Crossing the boundaries… the “Open Academic Model”“Open Academic Model”

Blurring the boundary between academia and industry/governmentBringing a “fresh” perspective to Bringing a “fresh” perspective to industry/government in an executable form – a industry/government in an executable form – a specific method, tool, heuristic, templatespecific method, tool, heuristic, template

Bringing industry/government “reality” into academia in a Bringing industry/government “reality” into academia in a researchable or usable form – a problem statement, a researchable or usable form – a problem statement, a specific challenge, guest instructors, heuristics, case studiesspecific challenge, guest instructors, heuristics, case studies

SD

OE-6

51

SD

OE-6

51

SD

OE-7

80

SD

OE-7

80

SD

OE-6

55

SD

OE-6

55

SD

OE-6

06

SD

OE-6

06

Decide Decide program program projectsprojects

First QuarterFirst Quarter Second QuarterSecond Quarter Third QuarterThird Quarter Fourth QuarterFourth Quarter

Equivalent of 20% facilitation support throughout 2004Equivalent of 20% facilitation support throughout 2004

The Challenge from Ralph Nelson at IBM Global ServicesThe Challenge from Ralph Nelson at IBM Global Services

Wrap-up: Essential Elements of a Wrap-up: Essential Elements of a Systems Engineering Program Systems Engineering Program

Leadership Policy with Executive Measurements

Investment to develop the process, templates, education, mentoring

Process and tools Defined Process

Templates

Skilled SEs – Core group of SEs with 15 years experience on major programs within relevant domains

Certification Program Education

Experience

Examination

Ongoing Process Improvement