Subjective Multidimensional Workload Index for Distributed Teams:

Development Program for the Team Subjective Assessment of Workload (T-SAW)

HFE DoD TAG, 21 May 2014, APG, MD

Sandro Scielzo, Jennifer Riley, and Fleet DavisSA Technologies, Inc.

Marietta, GA

Shannon ScielzoUniversity of Texas at Arlington

UTA

SBIR DATA RIGHTSContract No.: W911QX-11-C-0059

SA Technologies, Inc.3750 Palladian Village Drive, Building 600, Marietta, GA 30066

Expiration of SBIR Data Rights Period: 10 September 2019, subject to SBIR Policy Directive of 24 September 2002

Proprietary

UTA

The military has an urgent need for conceptualizing and operationalizing the construct of team workload to create a domain-independent subjective scale for teams

OSD

Proprietary

1• Problem Space and the T-SAW Solution

2• A Unified Theoretical Approach

3• Phase I Model and Item Development

4• Phase II Validation: T-SAW First Iteration

5• Phase II Validation: Abridged T-SAW

6• T-SAW Diagnosticity and Applicability

UTA

Proprietary

Problem SpaceUTA

Problem Significance No consensus on how workload,

much less team workload, should be characterized and operationalized

No clear theoretical framework leading to a satisfactory operationalization of the construct

No validated subjective workload measures for teams

Team Workload Limitations · Omit the critical steps of item

development· Limited studies at the team-level· Little understanding regarding

individual, contextual, and team-level antecedents· No comprehensive theoretical model· Lack of validity· Limited applicability · Flawed assumptions regarding team

member awareness of other team members’ workload

Team Workload Limitations · Omit the critical steps of item

development· Limited studies at the team-level· Little understanding regarding

individual, contextual, and team-level antecedents· No comprehensive theoretical model· Lack of validity· Limited applicability · Flawed assumptions regarding team

member awareness of other team members’ workload

Problem / Solution Unified Theory Phase I Development Phase II Year 1 Phase II Year 2 T-SAW Applicability

Proprietary

T-SAW Solution

Develop the first fully (a) validated, (b) domain-independent, (c) diagnostic, and (d) prescriptive subjective workload measure, sensitive to different team configurations, from intact co-located teams, to ad-hoc distributed teams

Our Solution

Our Team: Complementary Strengths

SA Technologies- Workload/performance theory- VBS2 scenario development- NUWC collaboration- USMA partnership

University of Texas at Arlington- Scale development/validation- Psychometric theory- Access to general population and

ROTC students

Problem / Solution Unified Theory Phase I Development Phase II Year 1 Phase II Year 2 T-SAW Applicability

Proprietary

Unified Theoretical Approach

• Existing Workload Frameworks are Insufficient

• MRT is cognitive-centric• How to account for full coverage

of individual and team workload-related factors?

• Affective, Behavioral, Cognitive, and Team + Theoretical Framework

• Ensures full coverage of individual and team workload spectrum

• Ensures proper construct classification under the ABC+ framework

UTA

Team Workload

ABC Factors

Affective

Behavioral

Cognitive

Team+ Dimensions

Context & Resources

Team task

Team composition

Comprehensive ABC+ Theoretical Framework

Problem / Solution Unified Theory Phase I Development Phase II Year 1 Phase II Year 2 T-SAW Applicability

Proprietary

Phase I Model and Item DevelopmentProblem / Solution Unified Theory Phase I Development Phase II Year 1 Phase II Year 2 T-SAW Applicability

Proprietary

Phase I Model and Item Development

Measurement Model

• Manageable Number of Dimensions• Items generation for T-SAW constructs• Testable framework

• Diagnostic Capability• Pinpoint high workload areas• Team workload profile

• Prescriptive Training Capability• Trainable competencies• Non-trainable traits / dispositions

• Team + Component• Team workload moderators• Further enhance diagnosticity

• Item Development• 500+ item pool• Initial card sort and bias analyses

Problem / Solution Unified Theory Phase I Development Phase II Year 1 Phase II Year 2 T-SAW Applicability

Proprietary

Phase II Validation: T-SAW First Iteration

Validate a domain-independent T-SAW, by down-selecting best behaving items that are predictive of performance using participant samples from the general population and other environments. Provide T-SAW in paper/pencil and electronic format.

Year 1 Goal

Validation Plan and Methods

SA Technologies- Develop research protocols- Develop SAR VBS2 scenarios

for civilians- Gather data from NUWC team

study- Produce full T-SAW metric set

with GEMS presenter

University of Texas at Arlington- Perform bias analysis and card

sort and reduce item pool- Administer T-SAW in scenario

anchoring paradigm- Execute experimental protocols

with general population- Refine T-SAW item set

Problem / Solution Unified Theory Phase I Development Phase II Year 1 Phase II Year 2 T-SAW Applicability

Proprietary

Phase II Validation: T-SAW First Iteration

UTA

SAR – Experiment Design and Setup

• 4 VBS2 SAR Scenarios• IVs

– Number of victims (within-subjects)

– Visual noise (between-subjects)

• DVs– Real time metrics– AAR metrics– Communications

Problem / Solution Unified Theory Phase I Development Phase II Year 1 Phase II Year 2 T-SAW Applicability

Proprietary

Phase II Validation: T-SAW First Iteration

UTA

NUWC– Experiment Design and Setup

• NUWC Experiment – Artemis starship

simulator– 3-members team

(captain, helm, weapons)– 4 teams

• IV– Collocated vs. distributed

team• DVs

– Mission performance– Communications

Problem / Solution Unified Theory Phase I Development Phase II Year 1 Phase II Year 2 T-SAW Applicability

Proprietary

Phase II Validation: T-SAW First Iteration

UTA

Correlational Analyses

• 618 items analyzed– Best behaving items matched against ‘archetype’ items by model

dimensions

Problem / Solution Unified Theory Phase I Development Phase II Year 1 Phase II Year 2 T-SAW Applicability

Proprietary

• Experimental Efforts– 100s of data points collected between

UTA and NUWC– Validation of model dimensions

• Initial T-SAW measure (domain independent)– PDF file and electronic version– Full T-SAW and T-SAW for simulated

environments (no physical items)• Validation support from NUWC

– Team data from simulation environment• Novel approach for further T-SAW

development– T-SAW core with branching items based

on team characteristics

Phase II Validation: T-SAW First IterationProblem / Solution Unified Theory Phase I Development Phase II Year 1 Phase II Year 2 T-SAW Applicability

Proprietary

Phase II Validation: Abridged T-SAW

Validate Abridged Army T-SAW , by using USMA and UTA ROTC Cadets. Determine utility of “dynamic” T-SAW with item branching. Develop T-SAW scoring sheet with automated diagnostic visualizations. Provide T-SAW in paper/pencil and electronic format.

Year 2 Goal

Validation Plan and Methods

SA Technologies- Develop research protocols- Develop SAR VBS2 scenarios

for Army population- Gather data from USMA and UTA

ROTC on team study- Produce Abridged T-SAW metric

and diagnostic visualizations

University of Texas at Arlington- Develop Abridged T-SAW- Administer Abridged T-SAW in

scenario anchoring paradigm- Execute experimental protocols

with general population- Develop scoring algorithms

Problem / Solution Unified Theory Phase I Development Phase II Year 1 Phase II Year 2 T-SAW Applicability

Proprietary

Phase II Validation: Abridged T-SAW

Year 2– Developed Abridged ‘Dynamic’ T-SAW

• Item branching based on core item responses• Completion between 30 and 90 seconds (faster

than NASA-TLX)

– USMA Card Sort and Bias analysis• Validated appropriateness of T-SAW items for

Army domain

– Team Study: SAR Scenarios• Entire pool of UTA ROTC Cadets• Presence of armed civilians

– Team Indices and algorithms• ABC+ Indices and scoring algorithms• Indices predictive of performance

– Diagnostic scores and visualizations• Automated diagnostic information

Problem / Solution Unified Theory Phase I Development Phase II Year 1 Phase II Year 2 T-SAW Applicability

Proprietary

UTA

Many thanks to:

Contact Description

COL. James Ness

United States Military Academy (USMA). James is an Academy Professor and Director of the Engineering Psychology program in the Department of Behavioral Sciences and Leadership at the US Military Academy, West Point. He facilitated T-SAW validation support with Cadet involvement.

Dr. Jason Wong Navy Undersea Warfare Center (NUWC). Jason is Human Factors Scientist at NUWC and Technical Area Manager for ONR. He was involved in the refinement of the first T-SAW iteration and its administration within simulated team environment

CPT. James Anderson

University of Texas at Arlington Army ROTC. James is part of Cadre and Battalion Executive office. Supervised and facilitated recruitment of UTA ROTC cadets for team laboratory experiment and team land navigation

Phase II Validation: Abridged T-SAWProblem / Solution Unified Theory Phase I Development Phase II Year 1 Phase II Year 2 T-SAW Applicability

Proprietary

UTAT-SAW Diagnosticity and Applicability

T-SAW INDEX: 15.21 HIGHEST INDEXED FACTOR: Inability to plan BIGGEST BOTTLENECK: Environmental constraints

RANK INDEX1 26.502 25.003 22.504 19.425 17.426 12.837 4.678 4.339 Lack of motivation 4.25

TOP 3 FACTORS TO ADDRESS

RANK BOTTLENECK INDEX1 Environmental constraints 45.172 Poor communication devices 37.583 Resource limitation 30.674 Lack of team supervision 30.085 Lack of team cohesion 26.086 Inexperience with tasks 25.007 Inability to provide backup 24.928 Dependency on your team 18.089 Role confusion 17.00

TOP 3 BOTTLENECKS TO ADDRESS

Lack of motivation

PAGE 1/1

Inability to concentrateInability to monitorPhysical exhaustionPhysical abilityNegative affectivity

Report for Squad Alpha Involved in Search And Rescue on 11 May 2014

SUMMARY OF TEAM WORKLOAD FACTORS IMPACT ON PERFORMANCE

FACTORInability to planLack of communicationStress

Lack of communication YES 4 Your team's Lack of communication has a high impact on performance, and it can be improved with training. To improve this factor's impact you must reduce the team workload related to its 4 bottleneck(s)

Stress YES 4 Your team's Stress has a high impact on performance, and it can be improved with training. To improve this factor's impact you must reduce the team workload related to its 4 bottleneck(s)

FACTOR TRAINABLE BOTTLENECKS DIAGNOSISInability to plan YES 5 Your team's Inability to plan has a high impact on performance, and it can be improved with training. To

improve this factor's impact you must reduce the team workload related to its 5 bottleneck(s)

DIAGNOSISA high index on environmental constraints will typically indicate issues with weather, terrain, visibility, and noise. Some aspects of environmental constraints can be mediated when training under the same environmental circumstancesPoor communication devices indicates that the communication system(s) used is(are) not ideal. Situation awareness and shared situation awareness training can improve communications. Litany training is also advisableResource limitations indicate that team members have limited supplies, gear items, or other task-critical items. This may be an indication that team members are not managing resource well, or that the mission exceeds resource availability

Poor communication devices 5

Resource limitation 3

SUMMARY OF TEAM WORKLOAD BOTTLENECKS IMPACT ON PERFORMANCE

BOTTLENECK ON # FACTORSEnvironmental constraints 6

TEAM WORKLOAD FACTORS AND BOTTLENECKS MATRIX

FACTOR/BOTTLENECKInability to planLack of communicationStressInability to concentrate

Poor comm. Res. limit No team sup. No team coh. Inexp. tasks No backup Depend team Role conf.

Inability to monitorPhysical exhaustionPhysical abilityNegative affectivity

Env. const.1

1

0

0 01 1 1 0 0 0 0 0

1 0 1 0 0 00

1 0 1 0 0 0 0 0 0

0 00 0 0 0 1 0 0 0

1 0 0 0 0 00

1 1 1 0 0 0 0 0 0

0 00 0 0 0 0 0 0 0

0 0 0 0 0 00

0 0 0 0 0 0 0 0 0

Army Abridged

Team - Subjective Assessment of Workload

Minimal Moderate High Very High Critical

0 10 20 30 40 50 60 70 80 90 100

0 10 20 30 40 50 60 70 80 90 100

Minimal Moderate High Very High Critical

4.3

0 10 20 30 40 50 60 70 80 90 100

Lack of motivation Index

22.5

0 10 20 30 40 50 60 70 80 90 100

Stress Index

0 25.0 100

NO DATA

Inexperience with tasks

0%

Inability to provide backup

18%

Dependency on your team to

complete your tasks0%

Role confusion

0%

Poor communication

devices30%

Resource limitation

24%

Environmental constraints

28%

Lack of team supervision

0%

Lack of team cohesion

0%

LACK OF COMMUNICATION BOTTLENECKS

• Goal and Advantages– Rapid data visualization

(e.g., AAR)– Powerful diagnostics– No SW required other than

Microsoft Excel• Excel Workbook

– Input team parameters– Input team averages – Visualize team workload

indices– Print diagnostic summary

T-SAW Algorithms and Visualizations

Problem / Solution Unified Theory Phase I Development Phase II Year 1 Phase II Year 2 T-SAW Applicability

Proprietary

UTAT-SAW Diagnosticity and Applicability

Team Name:

Affective ConstructsCore Item Score (team average)

Team + (follow up items)Team + Score (team average)

STRESS = 7.50 Inexperience with tasks 6

Inability to provide backup 5

Dependency on your team to complete your tasks

3

Role confusion 4

Poor communication devices 8

Resource limitation 4

Environmental constraints 6

Lack of team supervision 7

Lack of team cohesion 4

Squad Alpha

• Input Sheet– Team name, date,

event type– Team averages for 9

ABC factors and corresponding Team + dimensions

• Team Averages– Compute manually OR– Copy from

SurveyMonkey descriptives

Problem / Solution Unified Theory Phase I Development Phase II Year 1 Phase II Year 2 T-SAW Applicability

Proprietary

UTAT-SAW Diagnosticity and Applicability

Affective ConstructsCore Item Score (team

average)Team + (follow up items)

Team + Score (team average)

Core*Team+ Bottlenecks Sub Index

Stress Boundaries (exeeding boudaries correlates with decreased team performance) Stress High = 4 Index min = 0

Cut Off = 5 Stress Low = 1 Index max = 100

4

STRESS = Inexperience with tasks 6 45 45

Inability to provide backup 5 37.5 0

Dependency on your team to complete your tasks

3 22.5 0

Role confusion 4 30 0

Poor communication devices

8 60 60

Resource limitation 4 30 0

Environmental constraints 6 45 45

Lack of team supervision 7 52.5 52.5

Lack of team cohesion 4 30 0

22.50

7.50

• Algorithms– Automatically

computed– Protected formulas– Standardized 0-100

Indices across ABC and Team+ factors

• Indices based on– Factor relationship

with performance– Factor relationship

with Team+ factors

Problem / Solution Unified Theory Phase I Development Phase II Year 1 Phase II Year 2 T-SAW Applicability

Proprietary

UTAT-SAW Diagnosticity and Applicability

• ABC visualizations– 9 ABC factors figures– Color-coded

background indicating performance impact

• Team+ Bottlenecks– Bottleneck breakdown

by factor with factor index

26.5

0 10 20 30 40 50 60 70 80 90 100

Inability to plan Index

17.4

0 10 20 30 40 50 60 70 80 90 100

Inability to monitor Index

19.4

0 10 20 30 40 50 60 70 80 90 100

Processing ability Index

0 25.0 100

NO DATA

Inexperience with tasks

0%

Inability to provide backup

18%

Dependency on your team to

complete your tasks0%

Role confusion

0%

Poor communication

devices30%

Resource limitation

24%

Environmental constraints

28%

Lack of team supervision

0%

Lack of team cohesion

0%

LACK OF COMMUNICATION BOTTLENECKS

Indices Visualizations

Problem / Solution Unified Theory Phase I Development Phase II Year 1 Phase II Year 2 T-SAW Applicability

Proprietary

UTA

T-SAW INDEX: 15.21 HIGHEST INDEXED FACTOR: Inability to plan BIGGEST BOTTLENECK: Environmental constraints

RANK INDEX1 26.502 25.003 22.504 19.425 17.426 12.837 4.678 4.339 Lack of motivation 4.25

TOP 3 FACTORS TO ADDRESS

RANK BOTTLENECK INDEX1 Environmental constraints 45.172 Poor communication devices 37.583 Resource limitation 30.674 Lack of team supervision 30.085 Lack of team cohesion 26.086 Inexperience with tasks 25.007 Inability to provide backup 24.928 Dependency on your team 18.089 Role confusion 17.00

TOP 3 BOTTLENECKS TO ADDRESS

Lack of motivation

PAGE 1/1

Inability to concentrateInability to monitorPhysical exhaustionPhysical abilityNegative affectivity

Report for Squad Alpha Involved in Search And Rescue on 11 May 2014

SUMMARY OF TEAM WORKLOAD FACTORS IMPACT ON PERFORMANCE

FACTORInability to planLack of communicationStress

Lack of communication YES 4 Your team's Lack of communication has a high impact on performance, and it can be improved with training. To improve this factor's impact you must reduce the team workload related to its 4 bottleneck(s)

Stress YES 4 Your team's Stress has a high impact on performance, and it can be improved with training. To improve this factor's impact you must reduce the team workload related to its 4 bottleneck(s)

FACTOR TRAINABLE BOTTLENECKS DIAGNOSISInability to plan YES 5 Your team's Inability to plan has a high impact on performance, and it can be improved with training. To

improve this factor's impact you must reduce the team workload related to its 5 bottleneck(s)

DIAGNOSISA high index on environmental constraints will typically indicate issues with weather, terrain, visibility, and noise. Some aspects of environmental constraints can be mediated when training under the same environmental circumstancesPoor communication devices indicates that the communication system(s) used is(are) not ideal. Situation awareness and shared situation awareness training can improve communications. Litany training is also advisableResource limitations indicate that team members have limited supplies, gear items, or other task-critical items. This may be an indication that team members are not managing resource well, or that the mission exceeds resource availability

Poor communication devices 5

Resource limitation 3

SUMMARY OF TEAM WORKLOAD BOTTLENECKS IMPACT ON PERFORMANCE

BOTTLENECK ON # FACTORSEnvironmental constraints 6

TEAM WORKLOAD FACTORS AND BOTTLENECKS MATRIX

FACTOR/BOTTLENECKInability to planLack of communicationStressInability to concentrate

Poor comm. Res. limit No team sup. No team coh. Inexp. tasks No backup Depend team Role conf.

Inability to monitorPhysical exhaustionPhysical abilityNegative affectivity

Env. const.1

1

0

0 0

1 1 1 0 0 0 0 0

1 0 1 0 0 0

0

1 0 1 0 0 0 0 0 0

0 0

0 0 0 0 1 0 0 0

1 0 0 0 0 0

0

1 1 1 0 0 0 0 0 0

0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0

0

0 0 0 0 0 0 0 0 0

Army Abridged

Team - Subjective Assessment of Workload

Minimal Moderate High Very High Critical

0 10 20 30 40 50 60 70 80 90 100

0 10 20 30 40 50 60 70 80 90 100

Minimal Moderate High Very High Critical

Custom header

Sorted ABC factors

Top 3 factors with diagnostic table

Sorted Team + “bottleneck” factors

Top 3 bottlenecks with diagnostic table

Factors / bottlenecks matrix

Print-out format

T-SAW INDEX: 15.21 HIGHEST INDEXED FACTOR: Inability to plan BIGGEST BOTTLENECK: Environmental constraints

Report for Squad Alpha Involved in Search And Rescue on 11 May 2014

RANK INDEX1 26.502 25.003 22.504 19.425 17.426 12.837 4.678 4.339 Lack of motivation 4.25

TOP 3 FACTORS TO ADDRESS

Inability to concentrateInability to monitorPhysical exhaustionPhysical abilityNegative affectivity

SUMMARY OF TEAM WORKLOAD FACTORS IMPACT ON PERFORMANCE

FACTORInability to planLack of communicationStress

0 10 20 30 40 50 60 70 80 90 100

Minimal Moderate High Very High Critical

TOP 3 FACTORS TO ADDRESS

Lack of communication YES 4 Your team's Lack of communication has a high impact on performance, and it can be improved with training. To improve this factor's impact you must reduce the team workload related to its 4 bottleneck(s)

Stress YES 4 Your team's Stress has a high impact on performance, and it can be improved with training. To improve this factor's impact you must reduce the team workload related to its 4 bottleneck(s)

FACTOR TRAINABLE BOTTLENECKS DIAGNOSISInability to plan YES 5 Your team's Inability to plan has a high impact on performance, and it can be improved with training. To

improve this factor's impact you must reduce the team workload related to its 5 bottleneck(s)

0 10 20 30 40 50 60 70 80 90 100

RANK BOTTLENECK INDEX1 Environmental constraints 45.172 Poor communication devices 37.583 Resource limitation 30.674 Lack of team supervision 30.085 Lack of team cohesion 26.086 Inexperience with tasks 25.007 Inability to provide backup 24.928 Dependency on your team 18.089 Role confusion 17.00

TOP 3 BOTTLENECKS TO ADDRESS

SUMMARY OF TEAM WORKLOAD BOTTLENECKS IMPACT ON PERFORMANCE

Minimal Moderate High Very High Critical

0 10 20 30 40 50 60 70 80 90 100

TOP 3 BOTTLENECKS TO ADDRESSDIAGNOSIS

A high index on environmental constraints will typically indicate issues with weather, terrain, visibility, and noise. Some aspects of environmental constraints can be mediated when training under the same environmental circumstancesPoor communication devices indicates that the communication system(s) used is(are) not ideal. Situation awareness and shared situation awareness training can improve communications. Litany training is also advisableResource limitations indicate that team members have limited supplies, gear items, or other task-critical items. This may be an indication that team members are not managing resource well, or that the mission exceeds resource availability

Poor communication devices 5

Resource limitation 3

BOTTLENECK ON # FACTORSEnvironmental constraints 6

0 10 20 30 40 50 60 70 80 90 100

Lack of motivation 0 0 0 0 0 0 0 0 0

0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0

0

1 1 1 0 0 0 0 0 0

0 0

0 0 0 0 1 0 0 0

1 0 0 0 0 0

0Inability to monitorPhysical exhaustionPhysical abilityNegative affectivity

Env. const.1

1

0

0

1 1 1 0 0 0 0 0

1 0 1 0 0 0

1

TEAM WORKLOAD FACTORS AND BOTTLENECKS MATRIX

FACTOR/BOTTLENECKInability to planLack of communicationStressInability to concentrate

Poor comm. Res. limit No team sup. No team coh. Inexp. tasks No backup Depend team Role conf.0

0

0 1 0 0 0 0 0 0

Proprietary

• Military Applicability• Acquisition programs

• HSI lifecycle

• Manpower analyses support

• Any team research

• Industry Applicability• Medical teams

• Sport teams

• Organizations

T-SAW Diagnosticity and Applicability Problem / Solution Unified Theory Phase I Development Phase II Year 1 Phase II Year 2 T-SAW Applicability

Proprietary

UTA

This research was supported by:

Transition Description

Office of the Secretary of Defense (OSD).

OSD sponsored project under Small Business Incentive Research grant W911QX-11-C-0059

United States Army Research Laboratory

Many thanks to Ms. Katrina May (former Technical Point of Contact), who advised on the scientific merit of the research, validation studies, and aided us in their quest to find transition partners within the military

Many thanks to Dr. Daniel Cassenti, who took over TPOC responsibilities and facilitated the project completion and liaison with other DoD entities

Finally, we thank Dr. Donald Headley for his support throughout this project

Acknowledgments

T-SAW Research TeamUTA

Fleet DavisResearch Associate

Skills & Specialties:-Developing MOPs and MOEs-Developing live performance measures for training and operational environments- Conducting Cognitive Task Analysis-Situation Awareness Measurement-Developing training content for complex skill acquisition

YearsGeneral Experience: 10Specialized: 10

Sandro ScielzoResearch Associate

Skills & Specialties:-Metrics and Scale Development-Development of User-CenteredMultimedia Interfaces to SupportTraining and Decision-Making-Perceptual Discrimination in Complex Visual Search Task Environments-Situation Awareness Measurement-Human/Robot Team Interactions

YearsGeneral Experience: 12Specialized: 8

Jennifer RileyPrincipal ResearchAssociate

Skills & Specialties:- Training for situation

awareness in complex and dynamic domains

- Human-automation interaction

- Human-computer interface design

- Development of real-time Situation Awareness measures for virtual environment training

- Human-interaction with unmanned systems

YearsGeneral Experience: 16Specialized: 12

Shannon ScielzoAssistant Professor, UTADirector, TMT lab

Skills & Specialties:-Psychometric theory-Statistics and research design-Scale development-Dyadic and team communications-Computer-mediated communications-Virtual team processes-Training Needs Analysis-Performance evaluation and assessment

YearsGeneral Experience: 10Specialized: 8

QuestionsUTA