Upload
gwendolyn-peters
View
222
Download
1
Tags:
Embed Size (px)
Citation preview
Modeling Project Organizations:Virtual Prototypes
and Virtual Experiments
CEE214 Fall 1999Raymond E. Levitt
Elements of Model Development
Representation Reasoning User Interfaces System Interfaces Validation
OUTLINE
BackgroundWhy simulate organizations?Why start with project organizations?Why use “information processing” as the
central modeling framework? VDT Model Concepts and Evolution VDT Applications to Date
Using VDT models as “virtual prototypes”Using VDT models as “virtual experiments”
The Big Ideas
1. Validated analysis tools are central to design; they distinguish real design from trial-and-error experimentation!
2. In the same way that physical-science-based analysis tools help engineers design bridges, airplanes, semiconductors, pharmaceuticals, etc., social-science-based analysis tools can help managers design their organizations systematically
3. A small number of validated organizational analysis tools are already being used by managers to design organizations for projects, programs, and enterprises (SimVision, OrgCon… )
4. Validated organizational analysis tools also allow researchers to conduct new kinds of virtual computational experiments
Steps in a Formal Design Process
START… Set Design Goals
Design• Synthesize: Develop a candidate design solution
• Analyze: Predict candidate solution’s performance
• Evaluate: Compare predicted performance vs. goals
Iterate Designs
Relax Design Goals
…TERMINATE—Success or Failure?
“Project Org’n Design” vs. Trial and Error
0%
100%
Level of InfluenceLevel of Influence
Conceptual Design
Detailed Design and Implementation
Closeout &Operations
Expenditure of Funds Expenditure of Funds
OutcomeKnowledgeProject Design
OutcomePredictions
Modeling and Simulation of Organizations Bridges the Micro Macro Theory Gap
Organization macro-theory
Organization macro-experience
Sociology/Economics/Political Science
Organization micro-theory
Organizationmicro-experience
Cognitive andSocial Psychology
Agent micro-behavior
Agent-BasedSimu-lation
Organization micro-theory
Organizationmicro-experience
Cognitive andSocial Psychology
Emergent simulation macro-behavior
Raymond E. LevittJohn C. Kunz
Department of Civil EngineeringCenter for Integrated Facility Engineering
The Virtual Design Team (VDT):A language and tools for modeling and simulating “virtual organizations” executing work processes
SCOPE: Product Development Organizations? I. Practical Motivation—High Economic Importance
Product lifecycles and market windows are shrinking for many consumer and industrial products,
To accelerate time-to-market, complex, highly interdependent work processes must be executed concurrently
This causes an exponential increase in the amount of needed coordination work and rework
Insufficient “information processing capacity” emerges as a leading cause of “failure” in product development organizations
Extant theory and tools cannot help managers to predict whether, when and where their organizations will fail due to information processing overload
Why Study Product Development Organizations? II. Theoretical Motivation—Perfect fit for IP
frameworkTa
sk In
terd
epen
denc
e
Goal Incongruency/Ambiguity
Task Routineness
Fast Track Projects are Information-Intensive
High performance,complex product has high level of inter-
dependency betweenits subsystems
Fast-track schedule triggers unplanned coordination and rework for project
organization
Process OrganizationProduct
Project team must process large amount of information under
extremely tight time constraints
Using an “Information-Processing” Simulation Framework to Design Project Organizations
1. Model work process to determine “IP load” on organization arising from direct work, coordination and rework
2. Use simulation model as “virtual prototype” of real organization executing work process
3. Highlight predicted bottlenecks in IP capacity as likely loci of schedule and quality failures
4. Evaluate alternative “virtual prototypes” of work process and organization to test and select potential managerial interventions
What Affects Frequency of Exceptionsfor Workers and Managers in Projects?
For Workers:Task complexity relative to workers’ skillsTask interdependencyTask concurrency
For ManagersAbove factors, plus:
• Organizational span of control• Level of decentralization
OVERVIEW OF VDT REPRESENTATION & REASONING
VDT Info-Processing Abstraction Workers Process Information = “Direct Work”
TASK/ACTIVITY:(Volume of Information)
“ACTOR”:(Information Processor)
Coordination& Rework
Exceptions must be Processed by the Organization = “Hidden Work”
(Jay Galbraith, 1973)“Exception”
Direct Work is not Total Work! Total Work = Direct Work + Hidden Work
(Jay Galbraith, 1973)
Project Organization must also Coordinate and Supervise
+
“Exception”
Project ParticipantsPerform Assigned Tasks
Direct Work “Hidden Work”
Fast-Tracking Amplifies Hidden Work
REPRESENTATION
CPM to model processing of direct work Add two additional kinds of task
interdependence Add actor skill levels, application
experience Add organization structure, culture,
meetings …
Information Volume from Project Tasks & Dependencies:Direct Work, Communication Work and Rework
Start
Ready toExcavate
ChooseConstruction
Methods
Long LeadPurchasing
Apply Exc Permit
Seek ZoningVariance
Provide GMP
SelectSubconsultants
Arch Program
Select Key Subs
Estimate T ime
Define Scope
ProjectCoordination
Estimate Cost
Choose facadematerials
Choose Struct.System
GM PAccepted
DesignCoordination
Start
Ready toExcavate
ChooseConstruction
Methods
Long LeadPurchasing
Apply Exc Permit
Seek ZoningVariance
Provide GMP
SelectSubconsultants
Arch Program
Select Key Subs
Estimate T ime
Define Scope
ProjectCoordination
Estimate Cost
Choose facadematerials
Choose Struct.System
GM PAccepted
DesignCoordination
Start
Ready toExcavate
ChooseConstruction
Methods
Long LeadPurchasing
Apply Exc Permit
Seek ZoningVariance
Provide GMP
SelectSubconsultants
Arch Program
Select Key Subs
Estimate T ime
Define Scope
ProjectCoordination
Estimate Cost
Choose facadematerials
Choose Struct.System
GM PAccepted
DesignCoordination
Project Team Information Processing Capacity:# Actors, Skill Set, Experience, Structure, Policies
Client PM
Design PM Construction PM
ArchitecturalDesign Subteam
StructuralDesign Subteam
ProjectEngineers
ProcurementTeam
Client PM
Design PM Construction PM
ArchitecturalDesign Subteam
StructuralDesign Subteam
ProjectEngineers
ProcurementTeam
VDT Information Processing Model:Matching IP Capacity to IP Demand
Client PM
Design PM Construction PM
ArchitecturalDesign Subteam
StructuralDesign Subteam
ProjectEngineers
ProcurementTeam
Start
Ready toExcavate
ChooseConstruction
Methods
Long LeadPurchasing
Apply Exc Permit
Seek ZoningVariance
Provide GMP
SelectSubconsultants
Arch Program
Select Key Subs
Estimate T ime
Define Scope
ProjectCoordination
Estimate Cost
Choose facadematerials
1
1
1
Choose Struct.System
0.5
1
1.5
1
1.5
1
0.2
0.5
0.5
2
1
GM PAccepted
Design-Build Biotech Project
DesignCoordination
1
ProjectCoordination
M eeting
1
1 1
1
REASONING
Monte-Carlo discrete event simulation of information processing and communication
Used previously to model flow of physical work and materials through a supply chain
In VDT, direct work and hidden work are both simply quanta of information to be processed by humans and information processing/communication tools
Communicationsto other actors“Out tray”
Actor“In tray”
Communicationsfrom other actors
Direct Work
VDT Simulates Actors Working and Communicating
• Simulates:- every actor (team) & activity- work, errors, coordination, waiting, decisions, rework
• Produces:- “database” of actor and project behaviors/outcomes
• Simulates:- every actor (team) & activity- work, errors, coordination, waiting, decisions, rework
• Produces:- “database” of actor and project behaviors/outcomes
Performance Predictions Generated by VDT
Backlog
Quality
Schedule
Cost
ModelSimulation Results
StartFab Test& Deliver
DesignCoordination
DevelopSpecification
Insert Scan
Partition Chip
Gen Test Suite
PlaceRoute
FullChipSynth
Verify RTL
FloorPlanning
Write B1RTL
Sim GatesAssemble RTL
PhysVerifn
Verify B1RTL
Generate TestVectors
Synth_B1RTL
Activities
StartFab Test& Deliver
Project Lead
Marketing Team Chip ArchitectTest
Engineering StFoundry Lead
Logic DesignTeam 1
Foundry TestEngineer
Foundry LayoutEngineer
VerificationTeam
DesignCoordination
DevelopSpecification
Insert Scan
Partition Chip
Gen Test Suite
PlaceRoute
FullChipSynth
Verify RTL
FloorPlanning
Write B1RTL
Sim GatesAssemble RTL
PhysVerifn
Verify B1RTL
Generate TestVectors
Synth_B1RTL
Organization
StartFab Test& Deliver
Project Lead
Marketing Team Chip ArchitectTest
Engineering StFoundry Lead
Logic DesignTeam 1
Foundry TestEngineer
Foundry LayoutEngineer
VerificationTeam
DesignCoordination
DevelopSpecification
Insert Scan
Partition Chip
Gen Test Suite
PlaceRoute
FullChipSynth
Verify RTL
FloorPlanning
Write B1RTL
Sim GatesAssemble RTL
PhysVerifn
Verify B1RTL
Generate TestVectors
Synth_B1RTL
Communication
StartFab Test& Deliver
Project Lead
Marketing Team Chip ArchitectTest
Engineering StFoundry Lead
Logic DesignTeam 1
Foundry TestEngineer
Foundry LayoutEngineer
VerificationTeam
DesignCoordination
DevelopSpecification
Insert Scan
Partition Chip
Gen Test Suite
PlaceRoute
FullChipSynth
Verify RTL
FloorPlanning
Write B1RTL
Sim GatesAssemble RTL
PhysVerifn
Verify B1RTL
Generate TestVectors
Synth_B1RTL
Rework
StartFab Test& Deliver
Project Lead
Marketing Team Chip ArchitectTest
Engineering StFoundry Lead
Logic DesignTeam 1
Foundry TestEngineer
Foundry LayoutEngineer
VerificationTeam
DesignCoordination
DevelopSpecification
Insert Scan
Partition Chip
Gen Test Suite
PlaceRoute
FullChipSynth
Verify RTL
FloorPlanning
Write B1RTL
Sim GatesAssemble RTL
PhysVerifn
Verify B1RTL
Generate TestVectors
Synth_B1RTL
ManagementMeeting
Architecture TeamMeeting
Foundry TeamMeeting
1
1 1 1 1
1
1
1
1
1
1
Meetings
StartFab Test& Deliver
Project Lead
Marketing Team Chip ArchitectTest
Engineering StFoundry Lead
Logic DesignTeam 1
Foundry TestEngineer
Foundry LayoutEngineer
VerificationTeam
DesignCoordination
DevelopSpecification
Insert Scan
Partition Chip
Gen Test Suite
PlaceRoute
FullChipSynth
Verify RTL
FloorPlanning
Write B1RTL
Sim GatesAssemble RTL
PhysVerifn
Verify B1RTL
Generate TestVectors
Synth_B1RTL
0.8
4
4
4
4
1
1
1
1
1 1
1
1
1
1
1
Assignments 1
StartFab Test& Deliver
Project Lead
Marketing Team Chip ArchitectTest
Engineering StFoundry Lead
Logic DesignTeam 1
Foundry TestEngineer
Foundry LayoutEngineer
VerificationTeam
DesignCoordination
DevelopSpecification
Insert Scan
Partition Chip
Gen Test Suite
PlaceRoute
FullChipSynth
Verify RTL
FloorPlanning
Write B1RTL
Sim GatesAssemble RTL
PhysVerifn
Verify B1RTL
Generate TestVectors
Synth_B1RTL
0.8
4
4
4
4
1
1
1
1
1 1
1
1
1
1
ManagementMeeting
Architecture TeamMeeting
Foundry TeamMeeting
1
1 1 1 1
1
1
1
1
1
1
1
Reasoning Representation Usefulness
OrganizationMicro-experience
SimulationMicro-behaviors
EmergentSimulation
Macro-behaviors
OrganizationMicro-theory
OrganizationMacro-experience
ValidationElements
OrganizationMacro-theory
Ethnography
Internal Validity
Toy Problems
Intellective Experiments
Authenticity
Generalizability
Reproducability
Retrospective
Prospectivewith
Intervention
Natural
Validation Trajectory
VDT as “Virtual Prototype”
forOrganization
Design
CASE STUDY:The Lockheed Martin
Launch Vehicle
Lockheed Martin Launch Vehicle Organization
ProgramManager
SE & IProject
Manager
AvionicsProject
Manager
AvionicsProject
Manager
PropulsionProject
Manager
NG & CProject
Manager
Facilities& SE
EngineeringManager
BusinessDevelopment
Manager
OperationsManager
TestEngineering
Manager
StructuresProject
Manager
Organization of Avionics PDT
ST SubTeam
PDT Product Development Team
SE & I Systems Engineering and Integration
NG & C Navigation, Guidance and Control
SE Support Equipment
(Actor Name)Actor Not Representedin Avionics Model
(Actor Name)(# FTEs) Actor Represented
in Avionics Model
Power Dist.Panel
Sub Team (3)
Firing UnitSubTeam (3)
OperationalInterlocks
Sub Team (4)
Flight Boxes SubTeam
ProgramManager
SE & IProject
Manager
PropulsionProject
Manager
NG & CProject
Manager
Facilities& SE
EngineeringManager
BusinessDevelopment
Manager
OperationsManager
TestEngineering
Manager
StructuresProject
Manager
StructuresProject
Manager
Total: (15)
Off-shelf Department
Cables Contractor
Packaging Department
Electronics Department
Flight BoxesDepartment
Processor forPackage
SubTeam (5)
CablesSubTeam
(3)
Off-shelfSubTeam
(5)
PackagingSubTeam
(1.5)
ElectronicsParts
SubTeam (6)
AvionicsProject
Manager
Functional Guidance
Project Oversight- Monitoring-Exception Handling
Predecessor - Successorrelationship
Activity
Total Hours
(start)milestone
milestone(finish)Vehicle
AvionicsConcept
SystemIntegration and
Test
Identify PartsRequired
Search forVendors
ProcurementSupport
PrepareDocumentation
Printed WiringBoard Design
Printed WiringAssembly
EnclosureDesign
Top Assembly
Procurement
DevelopingSub-contracts
TeamingAgreements
RangeRequirements
VehicleInterconnect
Layout
DefineInterfaces
DetailedCable Drw.
Fabricate andTest Cables
DefineRequirements
NewEngineering
(Cables)
ReengineeringExperiences
ProductionEnhancements
Build and TestFlight Units
Bread BoardAnd Physical
Mockup
ApplyingExisting
Applications
Offshelf SubTeam (ST) (5)
Cables ST (3)
Flight Boxes ST (15)
ElectronicParts ST (6)
Packaging ST (1.5)
Total Work Volume: 29,234 hours
Simulated Duration: 4,611 hours
Project Manager (1)
8 hours 584 hours
848 hours
400 hours
904 hours
344 hours
296 hours 2,056 hours
592 hours
504 hours
840 hours 800 hours
448 hours
504 hours
4,800 hours2,504 hours
888 hours
752 hours 752 hours 752 hours 752 hours
752 hours 752 hours 752 hours 752 hours
1,200 hours 8 hours
Cables SubTeam (3 FTE)
Flight Boxes ST (15 FTE)
Electronic Parts SubTeam (6 FTE)
Packaging SubTeam (1.5 FTE)
PhysicalMockup
368 hours
AvionicsDrawings
2,952 hours
Activity Model for Avionics PDT
Offshelf SubTeam (5 FTE)
Activity Interdependency Chart for LMLV Avionics PDT
Note: Predecessor-successor relationships between activities are not shown in this chart.
Cables ST
VehicleAvionicsConcept
SystemIntegr’n and
Test
Identify PartsRequired
Search forVendors
ProcurementSupport
PrepareDocument’n
Printed WiringBoard Design
Printed WiringAssembly
EnclosureDesign
Top Assembly
Procurement
DevelopingSub-contracts
TeamingAgreements
RangeRequirements
VehicleInterconnect
Layout
DefineInterfaces
DetailedCable Drw.
Fabricate andTest Cables
DefineRequirements
NewEngineering
(Cables)
ReengineeringExperiences
ProductionEnhancements
Build and TestFlight Units
Bread BoardAnd Physical
Mockup
ApplyingExisting
Applications
Offshelf ST
Electronic Parts ST
PhysicalMockup
AvionicsDrawings
Failure-Dependantrelationship
SubTeam
Info-Exchangerelationship (2-way)
ST
Flight Boxes ST
Packaging ST
Vité Vité Model of LMLV Avionics Team
P ro jec tM an a g em en t
F in is h
1
1 5
1 5
1 5
1 5
1 5 15
55
55
5
3
3
3
3 3
1
1 1 1
1
1.5
11
1
1
5
S ta rt
P ro cu re m e n tS u p p ort
T ea m in gA g re e m e nt s
P rint e d W ir in gB o ard De sig n
De fin eR eq u ire m en t s Fa b ric a te a n d
Te s t C a ble s
P rin te d W ir in gA s se m b ly
N e w E n g in ee rin g
P h ys ic a l M o c k u p
R an g eRe q u irem en tsD ef in e In t erf a ce s
De ve lo p in gSu b c on t rac ts
P ro d uc tio nE n ha n ce m e n ts
A vio n ics D raw in g s
P ro cu re m e n t
S e a rc h f o rV e n d ors
P rep a reDo c um en t at ion
Id en t ify P art sR eq u ire d
R e -e n g in ee rin gE xp e rie n c es
B u ild a n d T es tF ligh t U nit s
S ys te m In t eg ra tion& Te s t
V eh ic le A vio n icsC on c ep t
E nc lo su re d e sig n
B re a d B oa rd a n dP h ys ica l M oc ku p
V e h ic leI n te rco n n ec tLa yo u t
A p p ly E xis tin gA p p lic a tion s
To p A s s em b ly
D et aile d Ca b leD raw in g s
Av io n ics P ro jec tM an a g er
O ff S he lf S u bT ea mCa b les S u b -te am
E le ct ron ic P a rtsS u b -te a m
Flig h t B ox S u b -t ea m
P ac ka g in g S u b -t ea m
E n g in ee rin gM an a g er
Avion ic s Project - Sce nario 1
Notes: No meetings
Communicationsto other actors“Out tray”
Actor“In tray”
Communicationsfrom other actors
Direct Work
VDT Simulates Actors Working and Communicating
• Simulates:- every actor (team) & activity- work, coordination, errors, decisions, rework
• Produces:- “database” of project behaviors/outcomes
VitéVité Gantt Chart for LMLV Avionics Team
LMLV Project: Actors’ Backlogs
LMLV Project: Non-Completed Communications
Guiding Managerial Interventions: VDT “What-if Analysis” of LMLV Avionics Design Team
0%
1%
2%
3%
4%
5%
6%
7%
8%
9%
10%
Increase Cable Subteam’s capacityfrom 3-5 engineers
Replace Cable Subteammembers with 3 moreexperienced engineers
Cost
Duration
Except's
Bet
ter
Wo
rse
Design Fast-Track Project Organizations: Some Example Applications
Reduced time to market for complex manufacturing facilities
Facilitated roll-out of new wireless telecom infrastructure across multiple regions
Developed best practices template toaccelerate factory start-ups
Identified & corrected subcontractor management problem that would have delayed project 4 mo.
Helped to meet ship milestone date required to close sale with largest customer
Aligned goals and accelerated rollout of innovative consumer product by 3 mo.
Identified and mitigated critical quality risks to accelerate rollout of new server product
Helped to define scope, schedule and organization for strategic IT projects
Chronology of VDT SimVision®
Steps in the Maturation of a COM&S FrameworkResearch at Stanford U. by Levitt, Jin, Kunz, et. al.Research at Stanford U. by Levitt, Jin, Kunz, et. al. • • •• • •
Concept/methodology development
Simulation tool development
30+ validating case studies
Mature technology
Mature technology
Exclusive License
Exclusive License
Commercial Product DevelopmentCommercial Product Development
1987 1997 1998 1999• • • • • •
1996
Vité Corp. Formed to Commercialize Technology
Vité Corp. Formed to Commercialize Technology
• • • • • •
Consulting & AnalysisConsulting & Analysis
Product OfferingsProduct Offerings
Ongoing VDT ResearchOngoing VDT Research • • •• • •
Maturation of Modeling User Interface
SYSTEM ELEMENTS/ INTERFACES OF VDT/SimVision® SOFTWARE
Vité Simulation Engine
Project Database
Analysis Tools
Model Builder/Viewer
S ta rt
Fa b, T e s ta n d D e liv er
D e v e lopS p ec i fic a tio n
W r ite _ B 1 R T L
V e ri fy _ B 1 R T L
Fu ll C hi pS y n th S im _ G a te s
F loo rp la nn in g P la c e _ R o ute P hy s V e r ifn
A s s em b le _ R T L
V e r ify _ R T L
In s e rt Sc a n
D e s i gn_ C oo rd ina ti on
P a rti t ion Ch ip
S y nth_ B 1 R T L
G en e ra te T e s tV e c tor s
G en _ Te s tS ui te
P r oje c t_ Le a d
C h ip_ A rc hi te c t
Lo gi cD e si gn Te a m 1
T es t_ E ng in e er in g_ S tM a r ke ti ng Te a m
Fo un dr y _ Le a d
Fo un dry _T e s t_ E ng rF oun dr y _ La y o ut_ E ng r
V e r if ic a ti on T ea m
4
1
4
1
1 1
1
4
1
1
1
1
0 .9
1
4
1
ASIC Developm ent ProjectNotes: Baseline Scenario - Project completes la te, wi th quality prob lems
0
2
4
6
8
10
12
14
16
Jan Feb Mar Apr May Jun Jul Aug Sep
Bac
klo
g (
day
s)
Actor BacklogS001 - Baseline
Chip_ArchitectFoundry_Layout_EngrFoundry_LeadFoundry_Test_EngrLogicDesignTeam1MarketingTeamProject_LeadTest_Engineering_StVerification Team
XMLXM L
Virtual Organizational Experiments
Modeling and Simulation of Organizations Bridges the Micro Macro Theory Gap
Organization macro-theory
Organization macro-experience
Sociology/Economics/Political Science
Organization micro-theory
Organizationmicro-experience
Cognitive andSocial Psychology
Agent micro-behavior
Agent-BasedSimu-lation
Organization micro-theory
Organizationmicro-experience
Cognitive andSocial Psychology
Emergent simulation macro-behavior
Research Modalities in Engineering Science — (Pre-1960s)
EmpiricalData• Inputs• Outputs
Physical ScaleModels• Inputs• Outputs• Empirical scaling rules
Theory• Physics• Chemistry• Biology (generally expressed
as sets of linear or differential eq’s.)
Limitations of Physical Scale Models
Costly and time-consuming to build Required skilled physical model builders (often built by
model shop technicians—not scientists)
Slow and costly to modifyScientists could not adapt models rapidly to react to
surprising data or to test new insightsCalibration against real world data took decades
Results needed to be interpreted with care Many important effects do not scale linearly
Physical ScaleModels• Inputs• Outputs• Empirical scaling rules
Research Modalities in Engineering Science — (Post-1960s)
EmpiricalData• Inputs• Outputs
Computational Modeling & Simulation• Inputs• Outputs• Limiting modeling
assumptions
Theory• Physics• Chemistry• Biology
How CM&S Affected Engineering Science and Practice
Rapidly declining time & cost to build and change models Rapidly declining time & cost to build and change models Two orders of magnitude improvement Two orders of magnitude improvement “disruptive” changes “disruptive” changes
Progress of Engineering Science Dramatically AcceleratedProgress of Engineering Science Dramatically Accelerated Could rapidly modify models to test & refine theory iterativelyCould rapidly modify models to test & refine theory iteratively ““Regress” micro-modeling assumptions against meso/macro dataRegress” micro-modeling assumptions against meso/macro data
Engineering practice made huge leaps forwardEngineering practice made huge leaps forward ““Real-time” prediction now feasible for even complex problemsReal-time” prediction now feasible for even complex problems Wider range of mathematically indeterminate problems can be solvedWider range of mathematically indeterminate problems can be solved Computational modeling now part of standard BS/MS curriculaComputational modeling now part of standard BS/MS curricula
Model results still need to be interpreted with great care! Model results still need to be interpreted with great care! Violations of assumptions can be catastrophic —e.g. Sleipner platform)Violations of assumptions can be catastrophic —e.g. Sleipner platform)
Research Modalities in Organizational Science—Pre-1970s
Empirical Data from Natural Experiments• Micro/Meso/Macro Inputs• Micro/Meso/Macro Outputs• Span 1 or, at most 2, levels
Empirical Data from SyntheticExperiments• Micro/Meso Inputs• Micro/Meso Outputs
Theory• Sociology• Psychology• Economics (usually expressed in words & diagrams;
sometimes in mathematical or computational models)
Limitations of Synthetic Experiments
Modest time and cost to design and perform experiments
Validation, calibration against real world data is difficult Individual motivation and context are very difficult to replicate Many effects do not scale linearly
Ethical concerns nowadays preclude many kinds of experiments previously conducted on human subjects
No links between micro-behaviors and macro outcomes Micro inputs and outputs cannot generally be related to, or reconciled
with, macro data or even macro-theory Result: Discipline-Based “Islands of Theorizing”
Empirical Data from SyntheticExperiments• Micro/Meso Inputs• Micro/Meso Outputs
Research Modalities in Organizational Science — Post ~1970
Empirical Data from Natural Experiments• Micro/Meso/Macro Inputs• Micro/Meso/Macro Outputs
Computational Modeling & Simulation• Micro/Meso/Macro Inputs• Micro/Meso/Macro Outputs• Nested models link micro-
macro data and theories
Theory• Psychology• Sociology• Economics
Computational Virtual Experiments
CM&S of organizations now beginning to be used to replace some kinds of natural or synthetic social experiments
A validated “emulation” model can be viewed as an “organizational test bench” for theorem - proving experiments
CMOT Journal has already published several papers of this type(Wong & Burton, Carroll & Burton, …)
How Computational Modeling is Affecting Organizational Science and Mgt. Practice
Rapidly declining time and cost to start generating Rapidly declining time and cost to start generating validated predictions for a modeled systemvalidated predictions for a modeled system
Organization Science poised to make huge leaps forwardOrganization Science poised to make huge leaps forward Can now rapidly modify models to test & refine theory iterativelyCan now rapidly modify models to test & refine theory iteratively Can “regress” micro-modeling assumptions against meso/macro dataCan “regress” micro-modeling assumptions against meso/macro data Validated models beginning to serve as “virtual synthetic experiments”Validated models beginning to serve as “virtual synthetic experiments”
Org. Design practice starting to incorporate CM&SOrg. Design practice starting to incorporate CM&S Rapid feedback develops “management judgment” by inductionRapid feedback develops “management judgment” by induction Enabling “flight simulation of alternatives” and “extreme collaboration”Enabling “flight simulation of alternatives” and “extreme collaboration”
CM&S Entering Mainstream Education and ResearchCM&S Entering Mainstream Education and Research Computational modeling & simulation now taught as part of PhD/MS/(BS)Computational modeling & simulation now taught as part of PhD/MS/(BS) MIT launching a MIT launching a Center for Computational PoliticsCenter for Computational Politics
Model results still need to be interpreted with care Model results still need to be interpreted with care Contingency theory says Contingency theory says context matters greatlycontext matters greatly!! Differences in task, technology, … must be taken into accountDifferences in task, technology, … must be taken into account
Using a Calibrated “Emulation” Model to Conduct Virtual Experiments
StartFinish
Project Manager
Subteam Lead Subteam Lead
Task 4
Task 3
Task 2
Task 1
SL 1 Task
PM Task
SL 2 Task
Task 5
Task 6
Task 7
Task 8
Subteam 1 Subteam 2 Subteam 3 Subteam 4 Subteam 5 Subteam 6 Subteam 7 Subteam 8
0
100
200
300
400
500
600
700
800
0 0.1 0.2 0.3 0.4
Exception Probability
Indi
rect
Wor
k (Da
ys)
DR 1
DR 2
DR 3
DR 4
30
100
600
0.1 0.3
Error Rate
Ind
ire
ct
Wo
rk
(Da
ys
)
Laminar Transition Turbulent
Exception Rate
Commercialization of COM&S Tools
Progress of CM&S of Organizations
1950 1960 1970 1980 1990 2000
CM&S in Engineering ScienceCM&S in Engineering Science
CM&S in Organization ScienceCM&S in Organization Science
First use by leading edge consultants
First taught at MS level in multiple universities
Commercial SW — Routinely used in practice
2010
Where is COM&S Going? Games
From action games to Sims®
Analysis tools for many kinds of planningFrom military, intelligence, to other public agencies (e.g.,
building plans, health care, transportation)Commercial (department stores, arenas, …)
Analysis Tools for Corporate Decision MakingFrom project design to enterprise designOrganizational aspects of M&A evaluationOrganizational aspects of supply chain optimization
Analysis Tools for Personal Decision MakingEvaluating your fit with a prospective employer Evaluating compatibility with a marriage partner, …
Using VDT as “Virtual Experiment”
Example: Explore relationship between centralization of decision making and time taken to complete complex taskUse Contingent design and run multiple “virtual
experiments” varying centralization for:1. High Task Uncertainty
1.1 With High Skill for all Actors1.2 With Low Skill for all Actors
2. Low Task Uncertainty2.2 With High Skill for all Actors2.2 With Low Skill for all Actors
Current Research with UndergraduatesSimulate Virtual Organizations to Search for the
“Edge of Chaos”
00.05
0.10.15
0.2
0
0.05
0.1
0.15
0.2
1
1.5
2
2.5
3
3.5
4
VFP external
VFP internal
Schedule Quality Ratio
3.5-4
3-3.5
2.5-3
2-2.5
1.5-2
1-1.5
Figure A. The parallel project with two dependency links; schedule quality ratio for different VFP values.
00.05
0.10.15
0.2
0
0.05
0.1
0.15
0.2
1
1.5
2
2.5
3
3.5
4
4.5
VFP external
VFP internal
Schedule Quality Ratio
4-4.5
3.5-4
3-3.5
2.5-3
2-2.5
1.5-2
1-1.5
Figure B. The parallel project with six dependency links; schedule quality ratio for different VFP values.
Closing in on an “Organizational Reynolds Number”
30
100
600
0.1 0.3
Error Rate
Laminar Transition Turbulent
0.2
E / C + 0.25 * Log(r) = 0.25
EVOLUTION OF SCOPE OF VDT
Trajectory of VDT Research Scope
Organizational Flexibility Low High
Predictable
Unpredictable
Task Predictability
Nonroutine Projects
Routine Projects
Service/Maintenance
Work
Communities of Practice
99-03: Lambert/ Buettner
Model More Complex Social BehaviorsModel More Complex Social Behaviors
Model More Innovative Tasks
Model More Innovative Tasks
Model More Dynamic Organizational FormsModel More Dynamic Organizational Forms
VDT Scope Trajectory: Routine Projects to Non-Routine Communities of Practice
Model More Effects of Communication/
Collaboration Tools
Model More Effects of Communication/
Collaboration Tools
97-01: Miller
96-03: Fridsma/Cheng
95-99: Thomsen/Kish
90-94: Cohen/ Christiansen
… behind the Virtual Design Team Faculty Collaborators
James March (SU: Ed., Sociology, GSB) John Kunz (SU: CIFE) Yan Jin (USC: ME) Clifford Nass (SU: Comm.) Richard Burton (Duke: Business) Martin Fischer (SU: CEE) Bernardo Huberman (Xerox PARC: Physics) Peter Glynn (SU: MS&E) Pam Hinds (SU: MS&E) Noah Mark (SU: Sociology) Dianne Bailey (SU: MS&E) Borge Obel (Odense U: Business School) Kathleen Carley (CMU: CS) Nosh Contractor (UIUC: Comm.) Andrea Hollingshead (UIUC: Psych) Janet Fulk (USC: Business School) Peter Monge (USC: Comm.) Douglass North (Wash. U: Econ., NL) Steve Barley (SU: MS&E) John Koza (SU: CS)
Students Geoff Cohen Tore Christiansen Jan Thomsen Douglas Fridsma Gaye Oralkan Yul Kwon John Chachere Per Björnsson William Hewlett, III Jolin Salazar Kish Carol Cheng-Cain Walid Nasrallah Roxanne Zolin Monique Lambert Archis Ghate Sam Miller Ray Buettner Mike Fyall Alfonso Pulido Ashwin Mahalingam Michael Murray Bijan Khosraviani Ryan Orr Tamaki Horii Laleh Haghshenass
The Real Team Behind the Virtual Design Team:Undergraduate Research Assistants to Date
Diane Newman BS German StudiesMS, Ph.D, MIT, Civil & Environmental Eng’g.
Professor of Geomicrobiology, Cal Tech
Yul Kwon BS Symbolic SystemsJ.D., Yale Law SchoolEnrolled in Ph.D. in Public Policy, Harvard
Corporate Attorney, Wilson Sonsini, Goodrich & Rosati
William C. Hewlett, III BS, Symbolic SystemsMS, Computer Science
PC game developer
Mike Fyall Senior, MSERecruited to Goldman Sachs
Summer internships with Vité management consultants
Jason Glickman Senior MSECoTerminal MS Student, MSE
Summer internships with MSD Investments
Jason Powers Senior MSECoTerminal MS Student, MSE
Summer internship with Vité management consultants
Tarmigan Casebolt Sophomore, MSE ?
Ongoing Research on Organization Design Institutional Complexity in Global Projects
Existing project organization modeling and simulation tools address engineering projects, whose participants have similar goals, values, culture, norms & technologies
Coordination Complexity
Production Costs
Coordi-nation CostsInstitutional
Complexity
Institutional Costs Global projects to develop
infrastructure, eco-sustainability, health care delivery and education encounter conflicting goals, values, norms, cultures and technologies