1 Best of 2008: Research findings you may have missed. Ryan Watkins, Ph.D. George Washington University etl

1

Best of 2008: Research findings you may have missed.

Ryan Watkins, Ph.D.George Washington University

www.gwu.edu/~etlwww.ryanrwatkins.com

2

Session Objectives

Participants will be able to… Summarize leading research articles from

multiple disciplines; Translate research finding into useful HPT

applications; and Apply research findings in their daily practice

of HPT.

3

“Best of 2008”???

Step 1: Surveyed 20 researchers and faculty Both ISPI and non-ISPI members Varied disciplines

HRD, OD, ISD, Management, Psychology

No specific criteria for “best”

Step 2: Reviewed diverse journals from 2008

Step 3: Based on initial input, sent preliminary list of articles back to researchers and faculty

Step 4: Used their input to create a final list.

4

Reminders

Only a summary of each research article is being provided, please read the whole article if you are interested in the details.

Presenting them in alphabetical order, not order of importance.

To maintain copyright I can not provide copies of the full articles.

The “HPT Implications” are my applications of the findings, you may have your own and the person sitting next to you may see different applications.

5

Article One

Title: An Examination of the Relationship Among Structure, Trust, and Conflict Management Styles in Virtual Teams

Authors: Xiaojing Liu, Richard J. Magjuka, and Seung-hee Lee

Publication: Performance Improvement Quarterly (an ISPI publication)

6

Article One – Liu, Magjuka & Lee

Purpose: Examine “the roles of team structure, trust, and conflict variables in the effectiveness of virtual teamwork.”

Sample Hypotheses: – Nonhierarchical structures outperform hierarchical structures on

their virtual team performance in complex task environments.– Trust will be positively associated with virtual team

performance.– A collaboration conflict management style in virtual teams is

positively associated with virtual team satisfaction..– Hierarchical teams have a higher level of trust than

nonhierarchical teams.

7


Variables: A 22-item questionnaire contained three dimensions: trust, conflict management resolution, and teamwork satisfaction.

Methods: – Students enrolled in a capstone management course as part of

an online MBA program at a large Midwestern university.– Subjects were separated into project groups of four to six people

to work on an online simulation project.– Hierarchical (17 teams) and non-hierarchical (21 teams) team

structures were assigned to the teams.– N = ~208 participants (some were dropped for teams not

following the instructions, but the final N was not given).

8


Findings: – Team structure had a significant effect on team performance (with

nonhierarchical leading to greater performance).– Team structure did not have a significant effect on team

satisfaction.– Trust had a significant effect on team satisfaction, but not

performance.– Collaborative conflict management had a significant effect on team

satisfaction. HPT Implications:

– In virtual teams, structure seems to have a greater influence on team performance, while social processes (i.e., team trust and conflict management) has greater influence on member satisfaction.

– “Even in a virtual team that is formed for a short time, it is worthwhile to facilitate ‘‘swift’’ relationship building through proactive communicative actions and to develop shared goals…”

9

Article Two

Title: Feedback Acceptance in Developmental Assessment Centers: The role of feedback message, participant personality, and affective response to the feedback session

Authors: Suzanne T. Bell and Winfred Arthur Jr.

Publication: Journal of Organizational Behavior

10

Article Two – Bell and Arthur

Purpose: – Examine the role of the affective reaction to the feedback session

in mediating the relationship between the assessor ratings and feedback acceptance.

Hypotheses: – The relationship between assessor ratings and feedback

acceptance will be mediated by the affective reaction to the feedback session.

– Participant self-ratings will moderate the relationship between assessor ratings and feedback acceptance.

– Three additional hypotheses related to the moderating role of personality characteristics (extroversion, emotional stability, and agreeableness).

11


Variables: – Assessor ratings, affective reaction to the feedback session, and

feedback acceptance.– Participants’ self-ratings of their performance.– Participant personality characteristics of extraversion, emotional

stability, and agreeableness as moderators. Methods:

– Two organizations: First: Career development program intended to help mid-level

managers develop their leadership skills. Second: Professional school of public administration which was

training individuals for leadership and managerial positions in government or public service.

– N = 141 participants, working with 35 trained assessors.

12


Findings: – Assessor ratings were positively related to feedback acceptance

and this relationship was partially mediated by the affective reaction to the feedback session.

Participant’s self-ratings, extroversion, and emotional stability did not serve as a moderator of this relationship.

Participant’s agreeableness (good-natured, flexible, trusting, cooperative, and tolerant dispositions) did serve as a moderator.

HPT Implications: – Performance feedback is essential to improving performance,

and the context in which that feedback is given is very important to acceptance.

– Build rapport before giving feedback (for example, light conversation before and after).

13

Article Three

Title: Implications of Domain-General ‘‘Psychological Support Skills’’ for Transfer of Skill and Acquisition of Expertise

Authors: David W. Eccles and Paul J. Feltovich

Publication: Performance Improvement Quarterly (an ISPI publication)

14

Article Three -- Eccles & Feltovich

Purpose: – Little research has been directed at identifying

psychological skills that (1) support learning and performance and (2) are domain-general and thus can be applied in a variety of novel learning and performance settings.

15


Variables: – Psychological Support Skills

Domain-general skills that support meta-cognition. Mental imagery and rehearsal skills, self-talk management skills, goal-setting

skills, relaxation skills, and planning, preparing, and organizing skills.– Meta-Cognition

Ability to regulate one’s psychological state in order to attain and subsequently maintain a state conducive to learning and performance.

Sustain a high level of self-confidence, sustain a high level of motivation, control anxiety and cope with stress, control concentration and attention, and organize the learning environment and the learners’ time and energy.

– Learning and Performance

Methods:– Model development (early stages).

16


HPT Implications: – Experts typically prepare over a 10-year period that

includes upward of 10,000 hours of practice.– Transfer of domain-specific skills gets more

challenging as performance gets more complex (i.e., tasks have fewer similarities).

– Developing domain-general psychological support skills may be a good complement to meta-cognitive skills and “weak” (i.e., non-task specific) problem-solving methods for improving performance.

17

Article Four

Title: Improving the quality of evaluation participation: a meta-evaluation

Authors: Darlene Russ-Eft and Hallie Preskill Publication: Human Resource Development

International

18

Article Four -- Russ-Eft & Preskill

Purpose: – Describe the successes and failures in obtaining desired

participation as of one organization attempted to implement several beta-test evaluations of a newly designed training program.

Sample Research Questions: – Why did some clients not fulfill their commitment to complete all

of the evaluation’s requirements?– What kind of incentives would have made clients more likely to

participate fully in the evaluation?– For those clients who participated more fully in the evaluation

process, what factors contributed to their participation?

19


Variables: – A newly designed customer service program that replaced two

older customer service programs the organization used to offer. – Pre-test, post-test, post-course reaction survey, 3 to 4 weeks

post-course follow-up survey and telephone interviews.

Methods:– Eight organizations (received free training in return), all of which

has very low response rates to post-training surveys.– Meta-evaluation (i.e., an evaluation of an evaluation) using a

case study method.– Ten 45-minute telephone interviews were conducted with 24

individuals.

20


Findings: – Individual incentives probably would not have helped,

free training was enough.– Sample reasons for none response:

They were too busy. They left the organization. They changed jobs within the organization. The office was shut down. A lack of manager follow.

21


HPT Implications: – Respondents must know that they have the “authority”

to respond for the organization. Key sponsors within an organization can help communicate this authority.

– Respondents and their organization must have the “capacity” to participate fully in the beta-test (including having time available to participate in all aspects of the evaluation).

– The “motivation” to participate in the evaluation must be built, including “what is in it for them” and the “importance of their input”.

22

Reflection Activity

Identify one “take away” from the five research articles previously presented

Discuss your “take away” with the person sitting next to you

23

Article Five

Title: Institutional Economics and Human Resource Development

Authors: Greg G. Wang and Rita L. Dobbs Publication: Advances in Developing Human

Resources

24

Article Five – Wang & Dobbs

Purpose: – Expand the “knowledge of institutional economics and

its foundational role and direct applicability to HRD research and practice.”

– Institutional economics… is the study of relationships between the role of human

institutions (i.e., organizations) and the related economic behaviors.

considers the complex interactions of various institutions. For example, individuals, organizations, governments, socio-culture and history, rather than a simple demand–supply equation.

25


Focus: – Orthodoxy neoclassical economics focuses on the allocation of

scarce resources among alternative ends. It emphasizes equilibria, where the solutions to individual and firm maximization problems are reached.

– Alternatively key assumptions of institutional economics include:

1. Institutions involve the interaction of agents with crucial information feedbacks.

2. Institutions have common conceptions and routines.3. Institutions sustain and are sustained by shared conceptions and

expectations.4. Institutions have relatively durable, self-reinforcing, and persistent

qualities.5. Institutions incorporate values and processes of normative evaluation.

26


Assumptions:

Assumption of Institutional Assumptions of Neoclassical

Labor market

Imperfect labor market with underemployment as an outcome

Labor market is competitive and reaches full employment in the long-run

Jobs Institutional rules and customs allocate employees’ jobs within unequal job and training structures

Job allocation is by market clearing prices or wages

HRD HRD increases the value of employee to the organization, and the interventions is embedded in the work process as well as taught by coworkers

Training is differentiated by types of skills, i.e., specific vs. general, which determines who pays for the investment and who receives the return

27


Assumptions:

Assumption of Institutional Assumptions of Neoclassical

Labor Market Focus

Internal labor market – promotion, OJT, career ladders, internal hirers, training, and other factors are accounted for decisions.

External Labor Market - pricing, allocating, hiring, training, and other decisions are controlled directly by economic variables.

Learning Learning is a developmental and reconstitutive process. Learning typically takes place through and within social structures and involves adaptation to changing circumstances. Such adaptations mean the reconstitution of the individuals involved.

Learning is the cumulative discovery of preexisting blueprint information, stimulus and response, or the updating of subjective probability estimates in light of incoming data.

28


HPT Implications: – Both Institutional and Neoclassical economic theories

are at play in most organizations, and their application plays an important role in the application of HPT and HRD in organizations.

– We should be familiar with both, and understand the consequences of each as they applied in organizations (especially in these economic conditions).

– Realize that we may be coming an Institutional Economics perspective but others in our organizations may have a Neoclassical perspective.

29

Article Six

Title: Investigating the Success of Decision Making Processes

Authors: Paul C. Nutt Publication: Journal of Management Studies

30

Article Six -- Nutt

Purpose: – Effectiveness of two decision making processes…– Idea-imposition which “caters to the interests of

powerful stakeholders by looking for ready-made solutions they can support. A ready-made solution allows the DM to visualize a course of action and its ramifications before commitments are made.”

– Discovery (i.e., HPT) which “gathers intelligence about needs, specify desired results, uncover options, evaluate options according to their benefits, and implement by working with interest groups and their perceptions to install the most beneficial option.”

31

Article Six -- Nutt

Sample Hypotheses: – The prospect of success increases when decision

makers follow a discovery process compared to an idea-imposition process.

– The prospect of success declines when decision makers use an emergent opportunity process, compared to a discovery process.

– For each of the contingencies considered, such as high and low urgency, the prospect of success will be greater when decision makers follow a discovery process, compared to the three other processes.

32

Article Six -- Nutt

Variables: – Discovery

and the “emergent opportunity” variation.– Idea-imposition

and the “redevelopment” variation.– Frequency of use in relation to success.– Controlling for context and decision type.

33

Article Six -- Nutt

Methods:– Analyzed 202 decisions about: financing, internal

operations/controls, personnel policy, marketing, buildings, technologies, and reorganizations.

– Profit and non-profit; top and mid-level managers– Interviews with decision makers used to determine

actions, and decision making process.– Indicators of effectiveness and efficiency are used to

measure success, with at least 2 raters on each decision.

34

Article Six -- Nutt

Findings:– Discovery had far more success than idea imposition.– Decision makers are drawn to idea-imposition

because it seems timely and pragmatic, the results however indicate little timeliness, low quality, and very low adoption rates compared to discovery.

– Discovery produced far better outcomes, with 90% sustained adoptions, 85% complete adoptions, rated as good to excellent, completed in nearly half the time.

– Emergent opportunity processes fell a little short of discovery in effectiveness, and were inefficient.

35

Article Six -- Nutt

HPT Implications: – Decision made using a systematic HPT process that

begins with a needs assessment (without a solution) are more likely to produce desired results, results are more likely to be maintained, and results will be achieved more efficiently.

– It is tempting, but do not abandon your systematic processes once you think a good solution is identified (i.e., emergent opportunity), this is less effective and significantly delays results.

36

Article Seven

Title: Long work hours: a social identity perspective on meta-analysis data

Authors: Thomas Ng and Daniel Feldman Publication: Journal of Organizational Behavior

37

Article Seven -- Ng & Feldman

Purpose: – Understand the factors that contribute to longer work weeks and

the relationships among hours worked, job performance, and employee well-being.

Sample Hypotheses: – Job security, organizational support, job autonomy, opportunities

for learning, organizational tenure, job level, current salary, number of promotions, career satisfaction, educational level, general work experience, international work experience, and social networking are each positively related to employees’ hours worked.

– Hours worked are positively related to job stress, mental strain, physical health problems, work injuries, and use of tobacco, alcohol, and drugs.

38


Methods:– Meta-analysis.– 20 years of journal articles and 7 dissertations.– Organizational science, applied psychology, and

sociology journals.– 199 relevant articles, containing a total of 222

independent samples.

39


Findings:– Average number of hours worked was 42.4 (SD is 10.2 hours).– Work hours were positively related to job stress and mental

strain.– Once a very high state of stress is reached, each new hour work

creates marginally less additional stress.– One additional work hour in an already jammed week may

exponentially increase work–family conflict.– The relationship of career satisfaction and educational level with

long work hours were stronger for women than for men.– With some exceptions, the results here provide some support for

the position that working long hours is negatively associated with work effectiveness and work attitudes.

40


HPT Implications: – Long work hours can increase stress, mental strain,

and family-to-work conflict.– The factors leading to long work hours are numerous,

thus if we want to manage work hours (increasing or decreasing) you will have to account for multiple variables.

41

Article Eight

Title: Organizational diversity, integration and performance

Authors: Andrew H. Van De Ven, Russel W. Rogers, John P. Bechara, And Kangyong Sun

Publication: Journal of Organizational Behavior

42

Article Eight -- Van De Ven, et. al

Purpose: – Examine how encouraging diverse perspectives (for

example, around a change) influences the relationship of varied models of organizing on performance.

Hypothesis: – Integrative behavior moderates the relationship

between organizational diversity in perspectives and performance.

43


Variables: – Integrative behavior as the degree to which employees perceive

that their organization encourages different perspectives and synthesizes those perspectives in ongoing day-to-day activities (e.g., openness to ideas and change involvement).

– Models of organizing: System/Bureaucracy Model; Market Model; Profession Model; and Community Model.

– Performance: Clinic productivity, net income, and patient satisfaction.

Methods: – 37 primary healthcare clinics go through restructuring changes.– Surveys and secondary-data analysis.

44


Findings: – Although the evidence is not as strong, results show that the main effect

of diverse organizing models on performance is positive.– Diverse mental models for organizing among employees was positively

related to performance, and this relationship was substantially stronger when integration processes for managing diversity are taken into account.

HPT Implications: – Processes that involve employees in decision making, encourage open

debate and integrative methods of conflict resolution, and diversity of perspectives are associated with higher performance.

– Do not focus solely on developing a single vision of change, but rather engage people with alternative perspectives.

– Supports Framing Organizations by Bolman and Soft Systems Theory by Checkland.

45

Reflection Activity

Identify one “take away” from the five research articles previously presented

Discuss your “take away” with the person sitting next to you

46

Article Nine

Title: Performance Implications Of Delayed Competitive Responses: Evidence From The U.S. Retail Industry

Authors: Jens L. Boyd And Rudi K. F. Bresser

Publication: Strategic Management Journal

47

Article Nine – Boyd and Bresser

Purpose: – Contribute to ongoing efforts to develop theory on

first mover and follower advantages by revisiting the consequences of fast versus delayed competitive responses.

Research Question: – What is the nature of the response timing-

performance relationship?

48


Hypotheses:– Curvilinear relationship between response delay and

responder performance (i.e., on average, there will be lower success for fast responders, higher success for responders with intermediate delays, and lower success for late responders).

– Linear relationship between response delay and first mover performance (i.e., as response delays increase, the impact on the performance of first mover decreases).

49


Variables: – Response delay (number of working days between action and

reaction).– Stock price effects (first mover and responder).– Control variables of imitation, tactical actions, firm size, financial

performance, age, industry concentration, and industry sales growth.

Methods: – The 17 largest U.S. department stores from 1994 to 2000.– Structured content analysis of 982 press articles.– 370 competitive moves including 105 responses.

50


Findings: – Most responses were imitations, indicating that many responders

intend to copy to catch up, potentially further escalating competition.

– As responders move too quickly or too slowly, this impacts their performance.

– The performance effects of response delays for responders and first movers do not mirror each other.

HPT Implications: – Be aware of the risk of responding too fast to competition.– Response timing requires a balance between the risks of

premature entry against missed opportunity.

51

Article Ten

Title: Task knowledge overlap and knowledge variety: the role of advice network structures and impact on group effectiveness

Authors: Sze-Sze Wong Publication: Journal of Organizational

Behavior

52

Article Ten -- Wong

Purpose: – Study 1 examines whether advice networks and

group knowledge affect subsequent group performance.

– Study 2 was conducted on 40 knowledge-intensive work groups from three organizations.

53

Article Ten -- Wong

Sample Hypotheses: – The higher the knowledge overlap and variety in a

group, the higher the group effectiveness.– The higher the density of the internal advice network

(i.e., the more each group member seeks advice from other members), the higher the degree of knowledge overlap in a group.

– The higher the centralization of the internal advice network (i.e., a few key influential group members), the lower the degree of knowledge variety in a group.

54

Article Ten -- Wong

Variables: – Knowledge variety is the range of different types of task

knowledge in a group.– Knowledge overlap is the extent to which group members have

common task knowledge– Internal and external social advice network structures (rather

than social or friendship networks).

Methods: – Study 1: Sample of 80 undergraduate students groups involved

in a complex business simulation task.– Study 2: 50 work groups from a hospital, an industrial diversified

firm, and a high-technology firm.

55

Article Ten -- Wong

Findings: – Knowledge overlap was not significantly associated

with group effectiveness in both studies.– Knowledge variety was positively associated with

group effectiveness in both studies.– Internal network centralization was negatively

associated with knowledge variety in both studies.– Internal network centralization was positively related to

group effectiveness in Study 1, but not Study 2.

56

Article Ten -- Wong

HPT Implications: – Promote knowledge breadth for groups involved in

knowledge-intensive work (such a cross-disciplinary teams).

– Foster ties with many different individuals, both within and outside of your group, to promote knowledge variety.

– Network with external groups.

57

Article Eleven (BONUS)

Title: The Effect of Performance Support and Training as Performance Interventions

Authors: Frank Nguyen & James Klein Publication: Performance Improvement

Quarterly (an ISPI publication)

58

Article Eleven -- Nguyen & Klein

Research Questions: – What combination of performance support and

training maximizes user performance?– What combination of performance support and

training do users prefer?– Do users access performance support more if they

have not had training prior to task performance?– What combination of performance support and

training minimizes the time to complete a task?

59


Variables: – EPSS, EPSS + web-based training, and web-based

training (with no partial-task practice in web-based training).

– User performance on the task; user attitudes; use of EPSS; time in training; and time to complete task.

Methods: – Focus on use of tax preparation software.– Posttest-only control group design, using assessments

and interviews.– 78 participants from various organizations.

60


Findings: – Training & EPSS and EPSS-only had significantly higher scores

on the task than training-only.– Time on task

EPSS-only = 26 minutes Training & EPSS = 31 minutes Training only = 1.5 hour

HPT Implications: – Adding performance support to training can significantly increase

performance.– EPSS may be sufficient by itself in many situations, though the

addition of training demonstrated some attitudinal benefits.– Finding the right “mix” is key.

61

Reflection Activity

Identify one “take away” from the four research articles previously presented.

Discuss your “take away” with the person sitting next to you.

62

Survey Was Sent To…

• Richard Clark (USC)• Richard Swanson (Univ. Texas, Tyler)• Darlene Russ-Eft (Oregon State)• Tim Hatcher (NCSU)• Rob Branch (UGA)• Jim Klein (ASU)• Steven Condly (UCF)• Jan Visser • Jeroen Van Merrienboer• Gordan Rowland (Ithica)

•Jim Altshuld (OSU)•Harold Stolovich•Ingrid Guerra-Lopez (Wayne)•John Wedman (Missouri)•Scott Schaffer (Purdue)•Maria Cseh (GWU)•Julia Storberg-Walker (NCSU)•Hillary Leigh•Tony Marker (BSU)•Yonnie Chyung (BSU)

63

Best of 2008: Research findings you may have missed.

Ryan Watkins, Ph.D.George Washington University

www.gwu.edu/~etlwww.ryanrwatkins.com

Documents

1 Best of 2008: Research findings you may have missed. Ryan Watkins, Ph.D. George Washington University etl