Evaluating the Effectiveness of a Career Certificate Program for International Students

Jessamyn Perlus, B.A., Julia Panke Makela, Ph.D., Daniel Scholes, Ph.D., Un Yeong Park, M.Ed., Gaeun Seo, M.S., & Kevin Hoff, B.A.The Career Center, University of Illinois at Urbana-Champaign

The Career Center Research Team

We work at the intersection of research and practice

• Four team members (1 full time, 3 PhD students)

• Located within a practice environment

• Advising and leading a cross-functional Assessment Team

• Exploring research questions, with opportunities for immediate application to practice

2

International (INTL) Students in the U.S.

• US: The world’s TOP education destination for INTL students• Number of International students studying in the U.S. increased continuously

from 526,670 during the 2005/06 academic year to 854,639 in the 2014/15academic year

526,670 541,324 567,039605,015 623,119 647,246 679,338

724,725780,055

854,639

2005/06 2006/07 2007/08 2008/09 2009/10 2010/11 2011/12 2012/13 2013/14 2014/15

Number of International Students Studying in U.S. Colleges and Universities

+ 62%

Data Source: Institute of International Education, 2015

3

INTL Students’ Unique Career Challenges

Career challenges to obtaining employment in the U.S.• Cultural adjustment and language barriers (Lin & Flores, 2011; Lopez,

2002; Reynolds & Constantine, 2007)• Complicated employment regulations (e.g., CPT, OPT, H1BVisa) (Kisch,

2015; Lin & Flores, 2011)

Career development responses to challenges• Challenges have been shown to lower career aspirations or limit

consideration of options (Lopez, 2002; Reynolds & Constantine, 2007)• International students express strong needs for assistance with career

exploration and the job search process (Shen & Herr, 2004; Spencer-Rodgers, 2000)

• A growing number of higher education career services offices have added dedicated programs/staff to serve international students (NCDA, 2015)

4

Study Purpose

The literature acknowledges:

• Growing population of international students

• The unique challenges that these students face

• The development of programs and services to address their needs

Yet, what do we know about the outcomes of those programs? How well do career services programs meet the needs of international students?

5

Career Intervention OutcomesMeta-Analytic Studies

• Consistently demonstrate moderate effect sizes “career interventions are effective” in relation to a wide variety of career choice outcomes…

• E.g., career maturity, career decision-making self-efficacy, choice certainty, choice satisfaction, congruence, decidedness, indecision, vocational identity

(E.g., Baker & Popowizc, 1983; Brown & Ryan-Krane, 2000; Brown et al., 2003; Oliver & Spokane, 1988; Whiston et al., 1998)

Critical Intervention Components

• Written exercises

• Individualized interpretation and feedback

• World of work information

• Modeling opportunities

• Attention to building support for choices within one’s social network

(Brown & Ryan Krane, 2000)

6

Career Certificate – International (CCI)

• Innovative 9-week career certificate program dedicated to helping international students become a more competitive job/internship candidate in the U.S.

• Includes weekly workshops, peer mentoring, and participation in a variety of career services throughout one academic semester

• Offered by The Career Center at Illinois (TCC) since Fall 2012

7

Method

• Data collected first and last weeks of 9-week CCI program

• 3 data sources

– Indirect Measure:

• Career Search Efficacy Scale (CSES)

– Direct Measures:

• Resumes

• Written networking introduction statements

8

Career Search Self-Efficacy

Self-efficacy

Individuals’ beliefs concerning his or her capability to successfully perform a certain task or behavior (Bandura, 1977, 1986)

“Can I do this?”

Career Search Efficacy Scale (CSES, Solberg et al., 1994)

Focus on career/job search related domains

9

Method - Career Search Efficacy Scale (CSES)

• 4 subscales: Job Search Efficacy, Interviewing Efficacy, Personal Exploration Efficacy, & Networking Efficacy

• 1 new subscale: “International Student Job Search Efficacy”

• Analysis: Paired samples t-tests to examine change over time

1. Discuss CPT/OPT options with an employer

2. Communicate effectively with an employer in English

3. Identify employers more friendly to international students

4.Utilize social media like LinkedIn to identify and connect with individuals who can support international students' career development

5.Identify and utilize career resources dedicated to international students

10

Method – Document Analysis

• Documents were blind-reviewed by trained reviewers

– Resume uses established rubric (Rooney et al., 2012)

– Networking script rubric designed for this study

• Analysis: Paired samples t-tests to examine change over time

11

Participants

• 49 students completed the Spring 2015 program; 43 (88%) participated in the evaluation study

• Student Response Rates:

– 31 (63%) completed self-reported pre- and post-assessment of Career Search Efficacy Scale (CSES)

– 43 (88%) submitted pre- and post-resumes

– 40 (82%) submitted pre- and post-elevator pitches

• Countries Represented: Bangladesh, China, Ecuador, Indonesia, Malaysia, Mongolia, and South Korea

12

Demographics of Program Participants

Freshman13%

Sophomore16%

Junior26%

Graduate35%

Missing10%

CLASS STANDING

Male29%

Female61%

Missing10%

GENDER

Heterosexual68%

Bisexual3%

Prefer not to respond

16%

Missing13%

SEXUAL IDENTITY

84%

3%

3%10%

RACIAL / ETHNIC IDENTITY

Asian Hispanic or Latino Prefer not to respond Missing

13

Results – Career Search Efficacy

5.7 6.05.4 5.7 6.1

5.3

7.7 7.8 7.6 7.7 7.6 7.9

0

1

2

3

4

5

6

7

8

9

10

***OverallCareer SearchEfficacy Scale

***Job SearchEfficacy

***InterviewEfficacy

***NetworkingEfficacy

***PersonalExploration

Efficacy

***Int’l Student Job Search

Efficacy

Career Search Efficacy Scale

Pretest Posttest

14

*** p < .001

Results – Resumes

2.5

2.2

2.5

1.4

3.6

2.9

2.3

2.8

2.4

2.7

2.42.6

1.5

3.8

3.0

2.6

2.9 2.9

1

1.5

2

2.5

3

3.5

4

**Overall *30 SecondReview

PresentationFormat

Spelling/Grammar

*Heading Education *DescribedExperience

OptionalSections

*PositiveProfessional

Language

Pretest Posttest

15

* p < .05, ** p < .01, *** p < .001

1 =

Inco

mp

lete

, 2 =

Dev

elo

pin

g, 3

= P

rofi

cien

t, 4

= P

rofe

ssio

nal

Results – Networking Scripts

2.4

2.62.5

2.6

2.3

1.9

3.5

1.5

2.4

2.6

2.82.9

2.8

2.3

1.7

3.6

1.7

2.8

1

1.5

2

2.5

3

3.5

4

Overall Hook *DesribedExperiences

Goals/Aspirations

Assess Fitwith

Company

Closing Length Grammar &SentenceStructure

*Clear Story& PositiveLanguage

Pretest Posttest

16

* p < .05

1 =

Inco

mp

lete

, 2 =

Dev

elo

pin

g, 3

= P

rofi

cien

t, 4

= P

rofe

ssio

nal

Summary of Findings

• Significant increases in all domains of Career Search Efficacy Scale (CSES) indicate that the CCI program is making a difference in students’ beliefs in their ability to successfully engage in career search tasks.

• Significant increases in quality on written resumes and elevator pitches, particularly on rubric subscales that are targeted by the program (e.g., 30 Second Review, Described Experience, Positive Professional Language, Clear Story).

17

Implications for Practice

• Share Successes– Communicate successful aspects of the program to colleagues and

future participants

• Modifications to CCI Program– Strengthen “closings” on networking scripts

– Recognize program boundaries

• Provide list of common grammatical mistakes that international students make and direct participants to resources on campus for assistance

• New Resources: Networking Script Rubric– Created a version to use in staff training and student workshops

– Updated research version to clarify criteria to improve interrater reliability

18

Implications for Research

• What about this intervention makes a difference? How can it be improved?

• Can relationships be observed between programs for international students (the intervention) and:

– Developmental outcomes, e.g. reduced influence of perceived barriers on career aspirations

– Process outcomes, e.g. increased knowledge and strategies for completing career development tasks

– Summative outcomes, e.g. obtained employment

– Comparison outcomes, e.g. vs. domestic students, country of origin

19

Future Directions

Qualitative Project in Progress• Grant from Midwest ACE to conduct

interviews of CCI alums• How students perceive each factor of

the program contributed to their experience and learning: – Mentoring? – Peer relationships built with other

participants and volunteers? – Skills practiced? – Resources or information gained

during the workshops?– What could have helped students

make even greater leaps forward?

20

Resources from The Career Center

• Makela, J. P., Scholes, D. R., Perlus, J. G., Seo, G., & Park, U. (2016). Outcomes of the Career Certificate-International Program, Report. Available upon request.

• INTL Connect Website: http://www.intlconnect.illinois.edu/tccservices

21

References (Page 1 of 2)

22

• Baker, S. B., & Popowicz, C. L. (1983). Meta-analysis as a strategy for evaluating effects of career education interventions. Vocational Guidance Quarterly, 31, 178–186.

• Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice-Hall.

• Bandura, A. (1997). Self-efficacy: The exercise of control. New York: Freeman.

• Brown, S. D., & Ryan Krane, N. E. (2000). Four (or five) sessions and a cloud of dust: Old assumptions and new observations about career counseling. In S. D. Brown & R. W. Lent (Eds.), Handbook of counseling psychology (3rd ed., pp. 740–766). New York, NY: John Wiley & Sons.

• Brown, S. D., Ryan Krane, N. E., Brecheisen, J., Castelino, P., Budisin, I., Miller, M., & Edens, L. (2003). Critical ingredients of career choice interventions: More analysis and new hypotheses. Journal of Vocational Behavior, 62, 411-428. doi:10.1016/S0001-8791(02)00052-0.

• Institute of International Education. (20142015). International student enrollment trends, 1948/49-20134/154. Open Doors Report on International Educational Exchange. Retrieved from http://www.iie.org/opendoors

• Kisch, M. (2015). Helping international students navigate career options. International Educator, 24(3), 66-69. Retrieved from http://www.nafsa.org/_/File/_/ie_mayjun15_fsa.pdf

• Lin, Y. & Flores, L. Y. (2011). Job search self-efficacy of East Asian international graduate students. Journal of Career Development, 40(3), 186-202. doi:10.1177/0894845311418655.

• Lopez, A. E. (2002). Factors influencing career decision-making and post-secondary career choices of a cohort group of female Cuban immigrants. Doctoral dissertation. Retrieved from ProQuest. 304788595.

References (Page 2 of 2)

• NCDA International Student Work Group (2015). Resources for partnering with international students. National Career Development Association (NCDA). Retrieved from http://ncda.org/aws/NCDA/asset_manager/get_file/64588/resources_for_partnering_with_international_students.pdf

• Oliver, L. W., & Spokane, A. R. (1988). Career-intervention outcome: What contributes to client gain? Journal of Counseling Psychology, 35(4), 447–462.

• Reynolds, A. L., & Constantine, M. G. (2O07). Cultural adjustment difficulties and career development of international college students. Journal of Career Assessment, 15(3), 338–350. doi: 10.1177/1069072707301218.

• Rooney, G. S., Pipkins, K., DiVietro, M. & Sun, H. (2012). Fall 2012 resume review evaluation findings.[Technical Report]. Champaign, IL: The Career Center, University of Illinois at Urbana-Champaign.

• Shen, Y. J., & Herr, E. L. (2004). Career placement concerns of international graduate students: A qualitative study. Journal of Career Development, 31(1), 15-29. doi: 10.1023/B:JOCD.0000036703.83885.5d.

• Solberg, Good, G. E., Nord, D., Holm, C., Hohner, R., Zima, N., Heffernan, M., & Malen, A. (1994). Assessing career search expectations: Development and validation of the Career Search Efficacy Scale. Journal of Career Assessment, 2(2), 111-123. doi: 10.1177/106907279400200202.

• Spencer-Rodgers, J., & Cortijo, A. (1998). An assessment of the career development needs of international students. Journal of College Student Development, 39(5), 509–513. Retrieved from http://eric.ed.gov/?id=EJ581220.

• Whiston, S. C., Sexton, T. L., & Lasoff, D. L. (1998). Career-intervention outcome: A replication and extension of Oliver and Spokane. Journal of Counseling Psychology, 45, 150-165.

23

Thank You

• Un Yeong Park and Tina Yeo for their coordination of the CCI program Fall 2015

• Assessment Committee for their feedback (Brian Neighbors, Taisha Mikell, Amanda Cox)

• Midwest ACE for their generous support of our future research in this area

24

25

26

27

28