UIJ AvisJackson Copyedit

Running head: STUDENT AFFECT 1

Student Affect During an HBCU Summer Research Program

Avis D. Jackson, Eric Pickering Boorman,

Farin Kamangar, Christine F. Hohmann

Morgan State University

Avis D. Jackson, Research Associate, ASCEND Center for Biomedical Research,

Morgan State University; Eric Pickering Boorman, Psychometrics Program, Department of

Psychology, Morgan State University; Farin Kamangar, Principal Investigator, ASCEND Center

for Biomedical Research, and Professor and Associate Dean for Research, School of Computer,

Mathematical, and Natural Sciences, Morgan State University; Christine F. Hohmann, Research

Enrichment Core Director, ASCEND Center for Biomedical Research, and Professor of Biology,

School of Computer, Mathematical, and Natural Sciences, Morgan State University.

Acknowledgements: This research is supported by funding from the NIH Diversity

Program Consortium BUILD Award (UL1MD009605/RL5MD009590/TL4MD009635). We

would like to thank Ms. Gillian Silver for her help with editing this manuscript.

Correspondence concerning this article should be addressed to Avis Jackson, Ph.D.,

ASCEND Center for Biomedical Research, Morgan State University, Carnegie Hall G64,1700

East Cold Spring Lane, Baltimore, Maryland, 21251. Email: [email protected]


Abstract

The popularity and effectiveness of intensive summer research programs to increase student self-efficacy is known. The Summer Research Institute (SRI) training experience, as part of undergraduate student training in Morgan State University’s NIH BUILD program, uses an entrepreneurial approach to prepare students for careers in health-related research. Bandura’s self-efficacy theory’s (1977) four antecedents are represented in the SRI curriculum, which provides multiple opportunities for mastery experiences and for moments of roused feelings. These occurrences are accompanied by extensive multilayered mentoring, where the mentors provide verbal encouragement, and facilitate various modes of academic, psychosocial and institutional support. To our knowledge, student affect over time has not been tested to assess impact or program effectiveness in a summer research training program. This study is based on the qualitative assessment of bi-weekly journals of 28 students in the SRI. The practice of students consistently writing journals is aligned with the scaffolded knowledge integration framework proposed by Linn (1995). The journals were reviewed for their cumulative affective content and change over time. The students responded as expected with positive and negative affect throughout the program and ended with overwhelmingly positive affect with their concluding presentations. Using Linguistic Inquiry Word Count (LIWC), a text analysis program, we matched the fluctuations in affect to activities during the program and interpreted the changes for program assessment. This type of analysis opens a window into student affective responses to training components that, to our knowledge, have not been widely used for research training programs of this kind.

Keywords: research training programs, NIH BUILD, emotion, LIWC, cooperative learning, self-efficacy


Introduction

Intensive summer research internship programs, sometimes described as research “boot

camps,” have become a mainstay in the landscape of undergraduate student research training

(Ashley, Cooper, Cala, & Brownel, 2017; Ghee, Keels, Collins, Neal-Spence, & Baker, 2016;

Lopato, 2007). These experiences have been shown to be quite effective in increasing student

science interest, persistence, identity, and self-efficacy, which are all key components of

increasing student success, college retention and completion (Ashley et al. 2017; Ghee, et al.

2016; Lopato, 2007). These findings across analyses of many projects have led to their adoption

across a spectrum of training programs. The common denominator of research “boot camps” is

that they are residential programs, expecting students to work with each other and on their

research experiences exclusively for a continuous stretch of 8 to 10 weeks, and they usually end

with a capstone experience such as an oral or poster presentation of the summer’s research.

The ASCEND Summer Research Institute (SRI) at Morgan State University (MSU) is the

initial component of a two-year scholarship program sponsored by the National Institutes of

Health Building Infrastructure Building Infrastructure Leading to Diversity (NIH BUILD)

program at MSU (Kamangar et al., 2017). Approximately 30 sophomore and junior STEM and

social/behavioral sciences students enter the SRI each summer, and 20 of these students are then

selected to become ASCEND Scholars, which is a two-year program that starts in the fall

subsequent to the SRI. The ASCEND SRI, like other internship programs referenced above, is a

residential, research-intensive training model. Unlike most other programs, however, the SRI

uses a group-based, cooperative learning model to acquire research skills in health sciences.

Instead of training in individual faculty laboratories, students proceed through a “backward


design” curriculum that teaches them research principles, provides hands-on experience with

research techniques, and culminates in students designing group-generated research proposals

(Handelsman et al., 2004). These research proposals are a high-stakes capstone experience for

the students, because they factor into their selection for the ASCEND Scholars program.

In STEM research programs for underrepresented minorities (URMs), science identity

increases and positively influences persistence (Estrada-Hollenbeck et al. 2011; Linn et al. 2015;

Robinson et al. 2016). The students’ belief in their capacity to respond well and perform over

time, or self-efficacy, is formed from their experiences. It strengthens their response to, and

mediates, anxiety while matriculating through their majors and programs (Bandura, 1977;

Estrada-Hollenbeck et al. 2011; Robinson et al. 2016). STEM self-efficacy positively influences

persistence in STEM (Chemers et al. 2011). Further, self-efficacy and science identity builds as

students experience successes, failures, and positive mentored feedback while constructing skills

(Estrada-Hollenbeck et al. 2011; Lent, 2007; Robinson et al. 2016).

Bandura’s self-efficacy theory (1977) posits four antecedents for self-efficacy:

performance accomplishments (mastery experiences), vicarious experience (such as role models

or mentors), verbal persuasion (encouragement), and emotional arousal (positive or negative

affect). Self-efficacy is widely referenced in research programs as integral to student success and

to continuance in STEM fields through graduate school and into careers (Chemers et al. 2011;

Estrada-Hollenbeck et al. 2011; Frantz et al. 2017). The four origins of self-efficacy undergird

social cognitive career theory (SCCT; Lent, Brown, & Hackett, 1994). SCCT is also used as the

theoretical framework to explain URM STEM persistence (Estrada-Hollenbeck et al. 2011;

Frantz et al. 2017).


The four antecedents to self-efficacy can be interpreted as specifically represented and

subsequently witnessed in the curriculum of the ASCEND SRI. The “boot camp” format

provides multiple opportunities for mastery experiences and for moments of roused feelings.

These occurrences are accompanied by extensive multilayered mentoring, where the mentors

provide verbal encouragement and facilitate various modes of academic, psychosocial and

institutional support. The students respond with positive and negative affect throughout the

program and end with what we believe is overwhelmingly positive affect in their concluding

presentations.

Journaling allows students to consider their experiences: what they have learned, and

how these thoughts fit into their understanding of how the world functions (Linn et al. 2015).

Linn further notes that the practice of students writing journals is aligned with a scaffolded

knowledge integration framework (Linn,1995), which provides a structure to assist students with

developing inquiry skills, and to help them refine, clarify and weigh acquired knowledge to

advance in and integrate into their field (Linn et al. 2015). Thus, analyses of student journals can

provide a window into the students’ affectual experience as they master the science content.

The purpose of this study was to examine changes in emotional arousal (positive and

negative affect) during the course of ASCEND SRI, measured through qualitative assessment of

required bi-weekly journal entries that the students are asked to maintain throughout the SRI.

These journals are reviewed for their cumulative affective content and change over time. We

match the fluctuations in affect to activities during the program and interpret the changes for

program assessment. We hypothesized that positive and negative changes in student affect would

relate to program activities and self-efficacy. To our knowledge, student affect over time has not


been tested through a summer research training program to assess either its impact or program

effectiveness.

Methods

IRB approval was included under the overall ASCEND NIH BUILD grant. Twenty-eight

students completed journals reflecting on their experiences in the SRI 2017 program. The journal

entries were open-ended, allowing students the freedom to discuss any topics the students felt

were relevant to their experiences in the program, building on previous journal entries. Journals

were completed and submitted electronically, two times per week, with the exceptions of the first

and final weeks during which only one journal entry was completed.

Data were entered into Linguistic Inquiry Word Count (LIWC), a computer program

designed to quantify various aspects of written text such as emotional content and cognitive

content. LIWC analyzes the individual words participants wrote and codes the words based upon

the LIWC internal dictionary. The LIWC internal dictionary determines the proportion of

emotional content, cognitive content, and other linguistic categories (Pennebaker, Boyd, Jordan,

& Blackburn, 2015). LIWC variables selected for further inquiry in this study comprised: Total

Affect, Anger, Anxiety, Negative Emotion, Positive Emotion, and Sadness. These variables

represent all of the emotional content variables that LIWC produces, and all of these variables

were chosen for the study. By analyzing them, we can determine particular emotional feelings or

trends within the SRI program.

Individual journal entries were extracted into Microsoft Word files without alteration of

the original documents. Documents were named to reflect a participant identification number as


well as a code to represent the journal entry number. These data were then entered into LIWC,

which analyzed each individual journal entry that a student produced. All students’ LIWC scores

were combined into a single data file and arranged by the file codenames such that each entry

corresponds to one student’s LIWC scores across the 14 weeks. Data were screened to identify

any participants with outlier scores on any of the six LIWC emotion variables in any of the 14

journal entries. Outliers were removed so as to not skew any individual or combined results.

Results were similar regardless of whether outliers were included or removed.

Results

A series of Repeated Measures ANOVA analyses was conducted to determine how each

emotional content variable changed over the 14 journal entries. As shown in Table 1 and Table 2,

most variables (Total Affect, Anxiety, Negative Emotion, and Positive Emotion) displayed

statistically significant changes across time. These trends are best conceptualized by their orders

or the sum of the exponents (linear is x1, quadratic is x2, cubic is x3, 4th order is x4, etc.). These

exponents reflect how many turning points, or points where the curve changes from positive to

negative or from negative to positive, are seen. The number of turning points can be determined

by subtracting the order of the trend by one (e.g., a linear trend has zero turning points, a

quadratic trend has one turning point, a cubic trend has two turning points, a 4th order trend has

three turning points). While many changes were lower order (e.g., linear, quadratic), the trends

with the highest effect sizes are higher order trends (e.g., 5th order, 6th order, 7th order, 8th order,

and 10th order trends). These higher order trends alternated between positive and negative values

repeatedly, as determined by the number of turning points, throughout the eight-week period.

Some variables did not display significant trends (Anger, Sadness); rather, these qualities of the


written text stayed at a near-zero level throughout the SRI program. Results of this analysis are

illustrated in Figure 1.

When comparing these non-linear trends to the timeline of the SRI program, we

discovered that peaks and valleys corresponded closely to specific training modules in the SRI

curriculum, suggesting that these training experiences precipitated the affective changes seen

(See Figure 1). Verbal indicators of negative emotion and anxiety peaked around week five of

the eight-week experience. This point in the training represents the journal entry after the first,

longest and most complicated wet-lab module, which includes multiple revisions to their group

reports, group presentations, and multiple feedback sessions. The introduction of the second lab-

module, with an all-day seminar that includes presentations from multiple researcher/professors,

followed closely on that experience, and on its heels was an all-day trip to a partner institution

that included lab visits and presentations and interaction with students from other programs. The

following three journal records (See Figure 1 records 6, 7, 8) show decreasing levels of both

negative emotions and anxiety, and these emotions remain relatively low throughout the

remainder of the program. We interpret these trends as indication of the students’ acclimation to

the program and its requirements, while simultaneously recognizing their capabilities and

consequently responding with lower anxiety and negativity in their written expression. These

results are the type of responses one might expect based on Bandura’s Self-Efficacy theory

(1977).

The indicator of positive emotion drops to its lowest level after the third journal entry,

which is at the beginning of the first module discussed above, and modulates up and down after

this point. It maintains a largely positive trajectory, reaching its highest level at the end of the

SRI, at the time that students were preparing their capstone research proposals. We interpret this


result as further evidence of students’ increasing self-efficacy in their abilities to appreciate and

begin to incorporate the skills they have acquired while successfully meeting the challenges

provided by the program.

We have added to our LIWC journal analyses another set of data generated in the

evaluation of our ASCEND SRI program that provides complimentary information. The

Attitudes Toward Research Scale (ATR; Papanastasiou, 2005) is designed to measure student

attitudes towards a basic research methods course. The 30-question scale uses a five-point self-

report Likert scale with 1=Strongly Disagree, 2=Disagree, 3=Neutral, 4=Agree, 5=Strongly

Agree. Sample items include “I enjoy research” and “I use research in my daily life.” The ATR

contains five subscales including Research Anxiety and Positive Attitudes Toward Research.

Comparison of pre-post responses of two of these ATR subscales provides convergent

validity to our interpretations of students’ journal entries. The Research Anxiety subscale pre-

post paired t-test results show significant reduction in anxiety (t(26)= 2.109, p<.05). Further, the

Positive Attitudes Toward Research subscale showed consistent high levels (pre M=4.12,

SD=.53; post M=4.19, SD=.56; see Figure 2).

Discussion and Conclusion

The SRI is a high-intensity and very challenging experience for participants. We

anticipated students’ experience and expression of anxiety resulting sometimes in negative

emotions during the course of training. We were surprised that the highest levels of such

negative affect correlated with wet-bench research experiences, because students in previous

cohorts have asked us for more of such training experiences. In this particular cohort, that may


have been the first time in their academic careers that many of the students were introduced to

wet-bench research. The high level of positive emotions and low level of anxiety in latter weeks

of the SRI, as participants define research questions and subsequently prepare their selected

group proposals, indicate that students have gained a sense of mastery. The following verbatim

quotes from students illustrate this:

It is amazing to see how far everyone of us have come from the first day of

not knowing how to properly achieve literature review to forming our own

research proposal. (Student 1)

I appreciate all the support and contributions to proposal development

and the effort that everyone puts in to make the program beneficial to everyone….

I appreciate being able to step outside of my comfort zone a little. As our final

presentation date approaches, I have become anxious yet excited. (Student 2)

The instructors have trained us to start thinking for ourselves, and to push

our ideas. … This summer showed me that I can be more than I am right now. It

showed me that I don’t have to be scared to speak up and make myself visible.

I’ve learned to not be scared to take risks, and to stop withdrawing. (Student 3)

Overall, the training experience leaves students with a mentored and

supported sense of accomplishment in the preparation and public presentation of

their formal, high-stakes group presentations. As anticipated, this analysis of the

SRI demonstrates the fourth antecedent—emotional arousal—of Bandura’s Self-

Efficacy theory (1977).


To our knowledge, this is the first time a qualitative word count analysis has been used to

assess student experiences of positive and negative affect in a summer research training program

within the framework of Bandura’s Self-Efficacy theory. Future studies will subject the journal

entries to more fine-grained, manually-coded analysis of student responses to explore the four

antecedents for self-efficacy and other specific concepts of attitudes towards research that have

been raised in the literature. Important to the Understanding Interventions community is that our

data suggest that this may be a good approach to gauge student responses to training

components. As a summative review of program components, the assessment is valuable for

insight into specific affective impacts across time that may be obscured with simple pre-post

assessment design.


References

Ashley, M., Cooper, K.M., Cala, J.M. and Brownel, S.E. (2017) Building Better Bridges into STEM: A Synthesis of 25 Years of Literature on STEM Summer Bridge Programs. CBE Life Science Education, 16(3):1-13.

Bandura, A. (1977). Self-efficacy: Toward a Unifying Theory of Behavioral Change.

Psychological Review 84, 191-215. Chemers, M. M., Zurbriggen, E. L., Syed, M., Goza, B. K., & Bearman, S. (2011). The Role of

Efficacy and Identity in Science Career Commitment Among Underrepresented Minority Students. Journal of Social Issues, 67(3), 469-491. doi:10.1111/j.1540-4560.2011.01710.x

Estrada-Hollenbeck, M., Woodcock, A., Hernandez, P. R., & Schultz, P. W. (2011). Toward a

Model of Social Influence that Explains Minority Student Integration into the Scientific Community. J Educ Psychol, 103(1), 206-222. doi:10.1037/a0020743

Frantz, K. J., Demetrikopoulos, M. K., Britner, S. L., Carruth, L. L., Williams, B. A., Pecore, J.

L., . . . Goode, C. T. (2017). A Comparison of Internal Dispositions and Career Trajectories after Collaborative versus Apprenticed Research Experiences for Undergraduates. CBE Life Sci Educ, 16(1). doi:10.1187/cbe.16-06-0206

Ghee, M., Keels, M., Collins, D., Neal-Spence, C. and Baker, E. (2016), Fine-Tuning Summer

Research Programs to Promote Underrepresented Students’ Persistence in the STEM Pathway. CBE Life Science Education, 15(28): 1-11.

Handelsman, J., Ebert-May, D., Beichner, R., Bruns, P., Chang, A., DeHaan, R., . . . Wood, W.

B. (2004). Education. Scientific teaching. Science, 304(5670), 521-522. doi:10.1126/science.1096022

Kamangar, F., Silver, G., Hohmann, C., Hughes-Darden, C., Turner-Musa, J., Haines, R. T.,

Jackson A., Aguila H. N., Sheikhattari, P. (2017). An entrepreneurial training model to enhance undergraduate training in biomedical research. BMC proceedings,11(Suppl 12), 69-77.


Lent, R. W., Brown, S. D., & Hackett, G. (1994). Toward a unifying social cognitive theory of career and academic interest, choice, and performance. Journal of Vocational Behavior, 45(1), 79-122.

Linn, M. C. (1995). Designing computer learning environments for engineering and computer science: The scaffolded knowledge integration framework. Journal of Science Education and Technology, 4(2), 103-126. doi:10.1007/bf02214052

Linn, M. C., Palmer, E., Baranger, A., Gerard, E., & Stone, E. (2015). Education. Undergraduate

research experiences: impacts and opportunities. Science, 347(6222), 1261757. doi:10.1126/science.1261757

Lopato, D., (2007), Undergraduate research experiences support science career decisions and

active learning. CBE Life Science Education, 6 (4): 297-306. Pennebaker, J.W., Boyd, R.L., Jordan, K., & Blackburn, K. (2015). The development and

psychometric properties of LIWC 2015. Austin, TX: University of Texas at Austin. DOI: 10.15781/T29G6Z

Richardson, J. T. (2011). Eta squared and partial eta squared as measures of effect size in

educational research. Educational Research Review, 6(2), 135-147. doi: 10.1016/j.edurev.2010.12.001

Robinson, S., Neergaard, H., Tanggaard, L., & Krueger, N. F. (2016). New horizons in

entrepreneurship education: from teacher-led to student-centered learning. Education + Training, 58(7/8), 661-683. doi:10.1108/ET-03-2016-0048

Figure 1. Linguistic Inquiry Word Count (LIWC) variations in emotional variables across 14

journal entries. Twice weekly journal entries started at the end of the 1st week (entry 1) in the

SRI and concluded, after students completed their research proposals (14). Text boxes and

arrows mark key SRI activities that coincide with changes in affect.

Figure 2. Pre- and post-test results of two Attitudes Toward Research Scale (ATR)

(Papanastasiou, 2005) subscales indicated significantly reduced scores in the research anxiety

subscale and improved scores in the positive attitudes toward research subscale.

*p < .05

Table 1

Table of Repeated Measures ANOVA Results

LIWC Variable

Statistical Test

Degrees of Freedom Effect

Degrees of Freedom Error

F Partial eta square

Observed power

Total Affect Sphericity Assumed

13 78 6.09*

0.50 1.00

Anger Sphericity Assumed

13 65 0.72

0.13 0.39

Anxiety Sphericity Assumed

13 91 9.32***

0.57 1.00

Negative Emotion

Sphericity Assumed

12 91 6.74***

0.49 1.00

Positive Emotion

Sphericity Assumed

13 91 4.78***

0.41 1.00

Sad Sphericity Assumed

13 91 0.85

0.11 0.48

Note. * indicated p < .05. ** indicated p < .01. *** indicated p < .001.

Table 2

Table of Linear and Nonlinear Trends across LIWC Emotion Variables

LIWC Variable Trend Degrees of Freedom Effect

Degrees of Freedom Error

F Partial eta square

Total Affect Cubic 1 6 7.97*

0.57

4th order 1 6 8.02*

0.57

5th order 1 6 7.50*

0.55

8th order 1 6 37.97***

0.86

10th order 1 6 8.27*

0.58

Anxiety Linear 1 7 9.33*

0.57

Quadratic 1 7 25.55***

0.79

Cubic 1 7 14.28**

0.67

5th order 1 7 28.40***

0.80

6th order 1 7 12.58**

0.64

7th order 1 7 13.80**

0.66

8th order 1 7 9.00*

0.56

Negative Emotion

Linear 1 7 5.96*

0.46

Quadratic 1 7 18.50**

0.73

5th order 1 7 31.50***

0.82

7th order 1 7 9.80*

0.58

8th order 1 7 13.45**

0.66

Positive Emotion

Quadratic 1 7 11.92*

0.63

Cubic 1 7 10.52*

0.60

4th order 1 7 20.62**

0.75

8th order 1 7 14.27**

0.67

10th order 1 7 6.84*

0.49

Note. Sad and anger not included as these trends were not significant. * indicated p < .05. ** indicated p < .01. *** indicated p < .001.