34
RUNNING HEAD: MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS The [Mis] Perceptions of Graduate Student Research Skills . Hurst, M., Feldon, D., Maher, M., Gilmore, J., Timmerman, B., Strickland, D, and Stiegelmeyer, C., Paper presented at the Annual Meeting of the American Educational Research Association, Denver, CO April 30 May 4, 2010 The work reported in this paper is supported in part by a grant of the National Science Foundation (NSF-0723686) to David Feldon, Briana Timmerman, Stephen Thompson, Jed Lyons, and Michelle Maher under the REESE program. The views in this paper are those of the authors and do not necessarily represent the views of the supporting funding agency

The [Mis] Perceptions of Graduate Student Research Skills ......MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 1 Abstract The mismatch between the academic preparation and realistic

  • Upload
    others

  • View
    1

  • Download
    0

Embed Size (px)

Citation preview

Page 1: The [Mis] Perceptions of Graduate Student Research Skills ......MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 1 Abstract The mismatch between the academic preparation and realistic

RUNNING HEAD: MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS

The [Mis] Perceptions of Graduate Student Research Skills

.

Hurst, M., Feldon, D., Maher, M., Gilmore, J., Timmerman, B.,

Strickland, D, and Stiegelmeyer, C.,

Paper presented at the Annual Meeting of the American Educational Research Association,

Denver, CO April 30 – May 4, 2010

The work reported in this paper is supported in part by a grant of the National Science

Foundation (NSF-0723686) to David Feldon, Briana Timmerman, Stephen Thompson, Jed

Lyons, and Michelle Maher under the REESE program. The views in this paper are those of the

authors and do not necessarily represent the views of the supporting funding agency

Page 2: The [Mis] Perceptions of Graduate Student Research Skills ......MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 1 Abstract The mismatch between the academic preparation and realistic

MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 1

Abstract

The mismatch between the academic preparation and realistic career prospects of

graduate students has been highlighted frequently in education literature (Golde &

Dore, 2001; Nyquist et al., 1999; Nyquist & Woodford 2000). Minimal attention,

however, has been placed on the mismatch between how graduate students and

their mentors perceive the students‘ research skills and the students‘ actual

research skills (Leggett et al., 2004). Using a mixed-method approach, this study

investigates the level of agreement between faculty mentors and their students‘

self-perceptions of research skills, knowledge, and dispositions. Further, it

compares both faculty and student assessments against independent ratings of the

students‘ work products. Results indicate that faculty mentors and their students

address non-overlapping facets of developing knowledge and skills at least 80%

of the time. However, when they do address the same issues, they agree less than

two thirds of the time. Neither faculty mentors‘ nor students‘ assessments

predicted strength or weakness on performance-based measures of research

proficiency at levels better than chance for most categories. Further, despite the

common assumption that mentors play a large role in students‘ research skill

development, collaboration with mentors is cited less frequently by graduate

students compared to other academic and social experiences. Implications for

graduate mentorship as both a facet of educational practice and as an area of

research are discussed.

Page 3: The [Mis] Perceptions of Graduate Student Research Skills ......MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 1 Abstract The mismatch between the academic preparation and realistic

MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 2

Problem and Review of Literature

The growing mismatch between the training and preparation of graduate students for

future careers has fueled an interest among researchers, leading to recommendations for

improving graduate student education (Golde & Dore, 2001, Nyquist et al., 1999; Nyquist &

Woodford 2000). Recommendations include enhanced mentoring practices and increased

opportunities for students to gain skills in areas beyond the scope of their direct research mentor

(Bieber & Worley, 2006; Gaff, 2002). Commonly, however, the faculty mentor role in graduate

student preparation is considered fundamental to the development of research skills (Carnegie

Initiative on the Doctorate, 2001; Mervis, 2000; Pearson & Brew, 2002).

Through guidance, collaboration, and role modeling, faculty mentorship is associated

with students‘ increased self-efficacy for research and research productivity (Paglis, Green, &

Bauer, 2006). Such mentoring relationships are commonly believed to be directly beneficial for

graduate students‘ performance and skill development (Austin, 2002; Clark et al., 2000; Green &

Bauer; 1995). A direct causal relationship is often assumed, despite frequent acknowledgements

of the importance of peer mentorship and support from postdoctoral researchers in a research lab

other than the faculty mentor (e.g., Delamont & Atkinson, 2001; Delamont, Atkinson, & Parry,

1997; Sweitzer, 2009). Further, it is assumed that the explanations provided by the mentor are

foundational to students‘ skill development, despite concerns about the accuracy of self-reported

problem-solving strategies provided by experts (Author, 2007, in press; Nickerson, 1999).

Students consistently report the impacts their mentors have on their motivation and their

decisions to continue in or withdraw from graduate programs (Lovitts & Nelson, 2000).

However, few studies characterize or empirically test the effects of interactions between mentors

and mentees that focus on the development and assessment of research skills.

Page 4: The [Mis] Perceptions of Graduate Student Research Skills ......MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 1 Abstract The mismatch between the academic preparation and realistic

MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 3

The current study builds upon a previous effort (Author et al., 2009) that identified and

defined research skills and dispositions described by graduate students and valued by faculty

mentors. Results of the previous study revealed misalignments among what faculty mentors

value in a researcher and reported research skills of their students. Many attributes and ‗soft

skills‘ such as confidence, integrity, ability to learn from failures, and, of particular note to

successful interdisciplinary efforts that define many current research investigations, collaborative

skills, were deemed as important, and perhaps more so, than research skills (Author et al., 2009).

This study investigates agreement between student mentees‘ and their faculty mentors‘

perceptions of the students‘ developing research skills, knowledge, and relevant personal

dispositions in science, technology, engineering, and mathematics (STEM) disciplines. Further,

these perspectives are compared against independently scored measures of research skills.

Theoretical Perspective

A commonly held belief among researchers in academia is that mentoring relationships

are beneficial for graduate students‘ performance and skill development (Austin, 2002; Clark, et

al., 2000; Green & Bauer; 1995). Moreover, graduate student instruction in conducting research

typically relies heavily on individual faculty mentors to provide mentorship and teaching of

research skills (Carnegie Initiative on the Doctorate, 2001; Mervis, 2000). These cognitive

apprenticeships guided by a mentor provide a mechanism through which graduate students

acquire research expertise, with the ultimate goal of the student becoming an independent

researcher (Pearson & Brew, 2002). A question arises, however, regarding the accuracy with

which mentors can convey their knowledge and an accurate perception of student research skills

which will in turn benefit and increase the knowledge of research skills among graduate students.

Page 5: The [Mis] Perceptions of Graduate Student Research Skills ......MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 1 Abstract The mismatch between the academic preparation and realistic

MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 4

It is commonly understood that the mentor-mentee relationship is reciprocal in nature.

Specifically, in order for a positive and productive relationship to exist, a student must receive

clear expectations regarding the development of research skills from his or her mentor and also

assume responsibility for the development of those skills (Carnegie Initiative on the Doctorate,

2001; Hon Kam, 1997; Mervis, 2000). While performing authentic academic tasks, students

must receive explicit mentorship to cultivate and clarify necessary skills needed to become a

professional researcher capable of adapting to new scholarly challenges (Mervis, 2000; Pearson

& Brew, 2002, Willison & O‘Regan, 2007). Evidence suggests that meaningful research comes

from the development of research skills among students and the guidance of research mentors

(Willison & O‘Regan, 2007). ―Articulating coaching and mentoring as tasks for supervisors to

engage in allows for clearer differentiation of responsibilities and elaboration of what is involved

in helping students learn the craft of research‖ (Pearson & Brew, 2002, p.140).

While research skill development is critical to graduate students‘ research and academic

experience, students themselves are often not fully aware of their own skills and abilities needed

to become a professional researcher (Lovitts, 2007) and often have incorrect assumptions about

their actual skills. Previous research indicates that faculty mentors typically have clear

expectations of their mentees regarding their attainment of knowledge, skills and dispositions

(Kardash, 2000). However, Leggett and colleagues (2004) report that these goals are seldom

articulated effectively to students by their mentors. There is also evidence of a misalignment

between students‘ expectations of the role of the mentor and their own roles as developing

researchers (Lagowski & Vick, 1995). In addition, mentors often hold perceptions of a student‘s

research skills that differ from that student‘s actual skills. These perceptions may be influenced

Page 6: The [Mis] Perceptions of Graduate Student Research Skills ......MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 1 Abstract The mismatch between the academic preparation and realistic

MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 5

by personality characteristics and types of experiences with the graduate student, and can impede

judgment of accurate skill assessment (Johnson, 2002).

Some guidelines for identifying research skills are available. A review of research

literature on criteria used by professional referees identifies several common qualities of

manuscripts and grant proposals that indicate competent and worthwhile science has occurred

(Cicchetti, 1991; Marsh & Ball, 1989; Marsh & Bazeley, 1999; Marsh & Bell, 1981; Petty,

Fleming, & Fabrigar, 1999). Additionally, Mullins and Kiley (1998, p.4) identified a list of

―generic‖ or ―employable‖ research skills, including (a) good communication/ presentation

skills, b) the ability to use fundamental and technical knowledge to applied systems, (c)

intellectual property management skills, (d) skills in the scientific method and linkage to the

broad context, and (e) good laboratory practice. The use of research proposals and scientific

assessments are also accurate indicators of skills, but often mentors receive relatively minimal

format training on mentoring practices and rely on their observations and interactions with their

graduate students to assess research skills as opposed to an objective assessment (Johnson,

2002).

Research literature identifying alignment or mismatch between student and mentor

perceptions of key research skills in graduate school is relatively rare particularly studies that

draw student and mentor participants from across the disciplinary spectrum. This study

addresses this gap in the literature by posing the following questions:

1. What is the level of agreement between faculty mentors‘ and student mentees'

perceptions of the mentees‘ research skills, knowledge, and related dispositions?

2. What is the relationship between those perceptions and student mentees‘ actual skill

type and level as measured through a performance-based skills assessment?

3. What types of experiences do graduate students frequently cite as contributing to the

development of research skills?

Page 7: The [Mis] Perceptions of Graduate Student Research Skills ......MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 1 Abstract The mismatch between the academic preparation and realistic

MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 6

Methods

This study uses a multi-method approach to identify the alignment or misalignment of

perceived versus actual research skills. To pursue study objectives, qualitative data were

gathered from doctoral students and their respective faculty mentors through their participation

in individual semi-structured interviews. All interviews were tape-recorded and transcribed to

identify emergent research skill levels. Quantitative data collected included a scientific

reasoning assessment (Lawson‘s Test of Scientific Reasoning) and written research proposals.

Research proposals were assessed via a modified form of a previously tested rubric for assessing

scientific reasoning skills in written laboratory reports (Timmerman, Johnson, & Payne, 2007).

The current study is part of a larger ongoing project investigating factors affecting the

development of science and engineering graduate student research skills. The primary thrust of

the larger project is to assess what happens to graduate students‘ own research skills when they

teach in a classroom in an inquiry based fashion. The premise is that mentoring and teaching of

research skills to others will cause reflection and meta-cognition and consequential accelerated

improvement in graduate students' own research skills.

Data Source

Participants used for this analysis were recruited from a large, public research university

in the Southeastern United States and were enrolled as full-time graduate students in research-

oriented programs. As part of their participation, they gave consent for researchers to conduct

confidential interviews with their faculty mentors. In all, the sample consisted of 58 students and

their respective faculty mentors (n=54).A review of mentors' faculty ranks revealed that the

distribution was roughly even across the assistant, associate, and full professor levels (15

Page 8: The [Mis] Perceptions of Graduate Student Research Skills ......MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 1 Abstract The mismatch between the academic preparation and realistic

MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 7

assistant, 15 associate, and 24 full). Table 1 displays the programmatic affiliation and year for

each of the 58 students.

[INSERT TABLE 1 ABOUT HERE]

Data Collection

Students and their mentors each participated in one-on-one, semi-structured interviews

with researchers early in the fall and late in the spring of an academic year. During the

interviews, students were asked to describe their strengths, weaknesses, and overall identity as

researchers. Mentors were asked to describe their participating students‘ research strengths and

weaknesses. Faculty mentors were also asked to describe what they valued in a researcher in

general.

To gain an authentic, performance-based measure of research skill, student participants

submitted written research proposals for projects in their respective areas of interest early in the

fall term and revised them for resubmission in late spring of the same academic year.

Data Analysis

Interviews: To compile interview data, transcripts were analyzed using the constant

comparison approach (Glesne & Peshkin, 1992). Members of the research team continuously

compared statements regarding mentee skills, knowledge representations, and research

dispositions both within and across mentor and mentee groups to identify emergent themes.

Perceptions recorded in the interview transcripts were classified into categories and

evaluated as indicating strength or weakness in that category. Redundant codes and

positive/negative characterizations of the same traits or skills were consolidated (e.g., ―analytic

Page 9: The [Mis] Perceptions of Graduate Student Research Skills ......MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 1 Abstract The mismatch between the academic preparation and realistic

MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 8

skills‖ and ―data analysis‖ codes were merged into a single code, ―data analysis‖; ―focused‖ and

―unfocused‖ were coded as ―focus– strength‖ and ―focus-weakness,‖ respectively).A full list of

codes, organized by classification as disposition, skill, and knowledge is provided in Table 2.

[INSERT TABLE 2 ABOUT HERE]

Research proposals. Research proposals were assessed using a modified form of a

previously validated rubric for evaluating scientific research skills in written laboratory reports

(Author et al., 2007; Author et al., in press) and has been applied in other work (e.g., Author et

al., in press). Proposals were received from participants electronically, checked for plagiarism

using SafeAssign™ and assigned to raters based on subject matter. Raters possessed graduate

degrees in relevant STEM disciplines and had attained an inter-rater reliability above r = 0.80

when coding student written responses. At least two raters scored each proposal, and any

discrepant scores were resolved by discussion until consensus was reached (cf.Johnson, Penny,

& Gordon, 2000; Johnson, Penny, Gordon, Shumate, & Fisher, 2005).

Integrated analysis. Following transcript coding and research proposal scoring, analysis

of the data occurred in three phases. First, frequency analyses were conducted for codes

identified from paired mentor and mentee interviews. Due to mentor scheduling constraints, 45

of 54 mentors were interviewed in the fall, and 40 of 54 were interviewed in the spring.

Instances in which both members of a mentor-mentee pair discussed the same attribute as either

a strength or weakness were identified and agreement or disagreement was noted.

Second, the student attributes as identified in both the mentor and mentee interviews were

crosstabulated with the individual elements of the mentee‘s rated proposal score to reflect the

Page 10: The [Mis] Perceptions of Graduate Student Research Skills ......MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 1 Abstract The mismatch between the academic preparation and realistic

MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 9

frequency with which interview-identified strengths and weaknesses aligned with rubric scores

above or below the sample mean.

Third, the extent to which interview data predicted performance data was assessed using

the Fisher Exact test. Frequencies within the 2 x 2 contingency table (interview-based

strength/weakness x rubric element scores above/below sample mean) were assessed against a

distribution with equal counts in all cells. Thus, a significant p-value indicated that interview-

based assessments were significantly better or worse than chance at predicting performance on a

given facet of the proposal. For the purposes of this study, a nontraditional significance

threshold value of p < 0.5 was used. The generally low frequency of any given code in the

dataset limits statistical power, which would otherwise generate an unacceptably high likelihood

of Type II error rates (Schunn & Anderson, 1999). This is particularly important given the weak

evidence for experts' perceptions of learners' abilities as accurate in the literature (e.g., Hinds,

1999).

All analyses were performed separately for the fall and spring data sets to determine

stability and the extent to which additional time influenced the alignment of assessments.

Results

Study results are presented in the order of the analytic steps detailed above. Together,

they represent a systematic investigation into the degree of alignment between faculty mentors'

and their graduate mentees' assessments of the students' research skill development. Further,

they reflect the degree to which either group can successfully predict the mentees' performance

on written research proposals evaluated against a rubric. The first section presents the results of

paired mentor and mentee response analysis conducted to identify alignment between mentor and

Page 11: The [Mis] Perceptions of Graduate Student Research Skills ......MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 1 Abstract The mismatch between the academic preparation and realistic

MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 10

mentee in terms of perceived strengths and weaknesses. The second section compares the

responses of faculty mentors and their student mentees to performance-based measures of their

emerging skills as assessed by multiple blind reviewers. The third section results from the

application of a measure of stability are presented as fall and spring data are compared. Lastly,

the final section identifies types of experiences graduate students cite as impacting their research

skill development.

Comparison of Code Alignment in Mentor-Mentee Responses

Analysis of mentor-mentee response alignment indicated that most mentor-mentee

interview responses did not overlap. In the fall, only 9 out of 45 (19%) mentor-mentee responses

aligned. In the spring, only 4 out of 40 (10%) aligned.

In 8 of the 13 pairs (62%), when a disposition, skill, or conceptual knowledge was

observed independently by both mentors and mentees, the pair agreed on whether the mentioned

trait constituted a strength or a weakness of the mentee. For example, in one case of agreement

between a student and his mentor, the student indicated that a research strength of his was

implementing experiments in the field:

I think the implementation phase (is a strength), plus I enjoy that. So I would design an

experiment and I would be sitting here and planning out what I would do. I‘m pretty

excited about that. Of course I have to collaborate with a lot of different people to make

sure what I have here will actually work. What I find that I‘m very good at is having it

written down on paper and then going out into the field and actually execute what I have

and what I‘m supposed to do. To me, my strongest point is actually the implementation

of the experimental design that I prepare.

Page 12: The [Mis] Perceptions of Graduate Student Research Skills ......MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 1 Abstract The mismatch between the academic preparation and realistic

MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 11

The student‘s advisor agreed with the student‘s assessment of research strengths, stating:

He is not afraid to tackle anything in the field at all. He has gone down to Belize with

just a boxful of equipment for three months and it works, and that is not something that

everyone can do. He is extremely independent, and he is not afraid to make contacts with

other people, and to seek out collaborators and to take on personal responsibility for

anything in his path, so he is really an independent operator.

In five pairs, however, a closer look revealed discordant understandings of mentee

development as a researcher. For example, a student described his strengths as a researcher by

stating:

I‘m good at asking questions when I don‘t know the answers. Pretty practical I think.

Pretty big overall, I have a good idea of what I want to do. I‘m not lost. I‘m pretty self-

motivated. I think I have the ability to be able to do the research on my own. I just need

someone to answer the questions that I occasionally have.

However, his advisor‘s comments countered this student‘s self-assessment:

I think he is a classic case of he has been trained by books to answer questions that are

based on those books and when you give him a more amorphous problem, something

more tiger-headed, it is a real challenge. He becomes very defensive. And that goes to

Page 13: The [Mis] Perceptions of Graduate Student Research Skills ......MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 1 Abstract The mismatch between the academic preparation and realistic

MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 12

shut him down, I think. And he is unable to open his mind at that point to the problem at

hand.

In another misaligned case, a student stated, ―I don‘t feel really comfortable with me

being pretty much on my own right now. But right now I really describe myself as a weak

researcher who is striving to become a stronger one.‖ In contrast, her advisor observed that ―She

understands fully the concepts that she does need to design and complete a study. So, I would

say she is strong.‖

A Test of the Predictive Power of Perceptions

To determine the predictive power of mentors' and mentees' perceptions of mentee

research skills, the identified strengths and weaknesses reported by the faculty and students were

crosstabulated with elements of each mentee‘s rated proposal scores (see Table 3).

[INSERT TABLE 3 ABOUT HERE]

When compared to the sample population's mean scores, mentors‘ perceptions of

mentees‘ research skills were not always accurate predictors of their mentee‘s demonstrated

abilities. During the fall, 44% (47 of 108) of mentors‘ assessments of their mentees‘ research

skills were inconsistent with the mentees‘ norm-referenced performance. These misalignments

reflected mentors‘ perceptions of strength that corresponded to below average performance (n =

24) and their perceptions of weakness that corresponded to above average performance (n = 23)

on the written research proposals. Alignment between mentors‘ perceptions and mentees'

Page 14: The [Mis] Perceptions of Graduate Student Research Skills ......MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 1 Abstract The mismatch between the academic preparation and realistic

MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 13

performance diminished in the spring, with mentors failing to predict performance 64% (34 of

53) of the time, with 19 instances of perceived strengths aligning with below average

performance and 15 instances of perceived weakness aligning with above average performance

(see Table 4).

[INSERT TABLE 4 ABOUT HERE]

Mentees' self-assessments were similarly inaccurate (see Table 5). In the fall semester,

mentees‘ perceptions of their research skills did not accurately predict their norm-referenced

performance 56% of the time (56 of 100 instances). Perceived strengths corresponded with

below average performance in 22 cases, and perceived weaknesses corresponded with above

average performance in 34 cases. Mentees' perception-skill alignment fell slightly to 46% (23 of

50 instances) in the spring. Mentees were more likely to underestimate their skills in the fall

(n=34 underestimations versus n=22 overestimations) and more likely to overestimate in the

spring (n=19 overestimations versus n=4 underestimations). Sixty percent of the time mentees

cited a research weakness, but when compared to actual research skills, they scored above the

population mean for those areas.

[INSERT TABLE 5ABOUT HERE]

Finally, when graduate students were asked about the types of experiences that

contributed to the development or increase of their research skills, approximately 40% cited

collaboration with their mentor. A larger majority cited experiences that did not involve a

mentor, such as trial and error in a lab, reviewing literature and observing other researchers or

Page 15: The [Mis] Perceptions of Graduate Student Research Skills ......MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 1 Abstract The mismatch between the academic preparation and realistic

MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 14

graduate students in the lab, and collaborating with their peers. These results, coupled with the

mismatch between perception and actual skill level indicates a need for increased collaboration

between the student and mentor.

[INSERT TABLE 6 ABOUT HERE]

Conclusions

Faculty mentors are thought to play a key role in defining expectations for research skill

acquisition (Kardash, 2000), and to serve as role models for their graduate mentees (Paglis et al.,

2006). However, findings from the current study suggest that mentors‘ insights do not reliably

map onto their mentees‘ demonstrated abilities. Interpretation of the data indicates that mentors

often demonstrate a fundamental lack of understanding of their mentees‘ progress as developing

researchers, particularly in the critical first years of graduate education. Their perceptions of

their mentees rarely match their mentees‘ own perception of themselves as researchers. More

tellingly, when mentor perceptions are matched against an empirical measure of mentee research

skills, roughly one of every two mentors‘ perceptions are incorrect. This finding contradicts the

standard perceptions of mentorship‘s effects on the development of research skills.

Research in the field of science education suggests that students whose learning is not

closely monitored are more likely to acquire misconceptions that impact both their theoretical

understanding of key phenomena and their scientific problem-solving skills (Lohman, 1986;

Schwartz & Bransford, 1998). Unfortunately, once such misconceptions are acquired, they are

almost impossible to change without substantial effort being invested on both the part of the

student and a willing mentor (Bargh & Ferguson, 2000; Chinn & Brewer, 1993; Thorley &

Stofflet, 1996). It is therefore possible that those who "just don't get it" are held back by

Page 16: The [Mis] Perceptions of Graduate Student Research Skills ......MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 1 Abstract The mismatch between the academic preparation and realistic

MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 15

incorrect or maladaptive knowledge that they acquired during the benign neglect of the first

couple years of their graduate training.

Increased interaction and collaboration between the graduate student and research mentor

may close the gap of perception versus actual skills level and provide the mentor with increased

opportunity to see the student engage in objective tasks in which the mentor can determine a

more accurate research skill level and provide the student with guidance to increase the research

skills

Significance

The current study is of educational significance in many ways. To create an effective

educational setting that addresses the cognitive and social demands of learning, the mentor must

be aware of the research skills a graduate student actually possesses. The study also has

implications for graduate mentor guidance. As doctoral education is characterized by cognitive

apprenticeship with a faculty mentor (Florence & Yore, 2004; Golde & Dore, 2001), the quality

of doctoral mentoring is of critical importance to the successful scholarly doctoral student

development.

Page 17: The [Mis] Perceptions of Graduate Student Research Skills ......MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 1 Abstract The mismatch between the academic preparation and realistic

MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 16

References

Austin, A. E. (2002). Preparing the next generation of faculty: Graduate school as socialization

to the academic career. The Journal of Higher Education, 73(1), 94-122.

Bargh, J. A. & Ferguson, M. J. (2000). Beyond behaviorism: On the automaticity of higher

mental processes. Psychological Bulletin, 126(6), 925-945.

Bieber, J. & Worley, L. (2006). Conceptualizing the academic life: graduate student‘s

perspectives. Journal of Higher Education, 77 (6), 1009-1035.

Carnegie Initiative on the Doctorate. (2001). Overview of Doctoral Educational Studies and

Reports: 1990 - Present. Stanford, CA: The Carnegie Foundation for the Advancement of

Teaching.

Chinn, C. A., & Brewer, W. F. (1993). The role of anomalous data in knowledge acquisition: A

theoretical framework and implications for science instruction. Review of Educational

Research, 63, 1-49.

Cicchetti, D. V. (1991). The reliability of peer review for manuscript and grant submissions: A

cross-disciplinary investigation. Behavioral and Brain Sciences, 14, 119-135.

Clark, R., Harden, S., & Johnson, W. (2000). Mentor relationships in clinical psychology

doctoral training: Results of a national survey. Teaching of Psychology, 27, 262–268.

Cox, M. & Andriot, A. (2009). Mentor and undergraduate student comparisons of students‘

research skills. Journal of STEM Education, 10 (1 & 2), 31-39.

Davis, G., & Fiske, P. (2001). The 2000 national doctoral program survey. University of

Missouri-Columbia: National Association of Graduate-Professional Students.

Author. (2007). Implications of research on expertise for curriculum and pedagogy.

Educational Psychology Review, 19(2), 91-110.

Author. (in press). Do expert researchers tell it like it is? A microgenetic analysis of research

strategies and self-report accuracy. Instructional Science.

Florence, M., & Yore, L. (2004). Learning to write like a scientist: Coauthoring as an

enculturation task. Journal of Research in Science Teaching, 41, 637-668

Gaff, J. G. (2002). The disconnect between graduate education and the realities of faculty work:

A review of recent research. Liberal Education, 88(3), 6-13.

Page 18: The [Mis] Perceptions of Graduate Student Research Skills ......MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 1 Abstract The mismatch between the academic preparation and realistic

MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 17

Golde, C. & Dore, T. (2001). At cross purposes: What the experiences of doctoral students

reveal about doctoral education (www.phd-survey.org). Philadelphia, PA: A report

prepared for The Pew Charitable Trusts.

Glesne, C. & Peshkin, A. (1992). Becoming qualitative researchers: An introduction. New York:

Longman.

Green, S. G., & Bauer, T. N. (1995). Supervisory mentoring by mentors: Relationships

with doctoral student potential, productivity, and commitment. Personnel Psychology,

48(3), 537–562.

Hinds, P. J. (1999). The curse of expertise: The effects of expertise and debiasing methods on

predictions of novice performance. Journal of Experimental Psychology: Applied, 5(2),

205-221.

Hodkinson, P. (2004) Research as a form of work: Expertise, community and methodological

objectivity. British Educational Research Journal, 30(1), 9-26.

Hon Kam, B. (1997). Style and quality in research supervision: the supervisor dependency

factor. Higher Education, 34, 81-103.

Johnson, R. L., Penny, J., Gordon, B., Shumate, S. R., & Fisher, S. P. (2005). Resolving score

differences in the rating of writing samples: Does discussion improve the accuracy of

scores? Language Assessment Quarterly: An International Journal, 2(2), 117-146.

Johnson, W. (2002). The intentional mentor: strategies and guidelines for the practice of

mentoring. Professional Psychology: Research and Practice, 33 (1), 88-96.

Kardash, C. (2000). Evaluation of an undergraduate research experiences: perceptions of

undergraduate interns and their faculty mentors. Journal of Educational Psychology, 92

(1), 191-201.

Lagowski, J. & Vick, J. (1995). Faculty as mentors. New Direction for Teaching and Learning,

(62), 79-85.

Leggett, M., Kinnear, A., Boyce, M. & Bennett, I. (2004). Student and staff perceptions of the

importance of generic skills in science. Higher Education Research & Development,

23(3), 295-312.

Lohman, D. F. (1986). Predicting mathemathantic effects in the teaching of higher-order

thinking skills. Educational Psychologist, 21(3), 191-208.

Lovitts, B. E. (2005). Being a good course-taker is not enough: A theoretical perspective on the

transition to independent research. Studies in Higher Education, 30(2), 137-154.

Page 19: The [Mis] Perceptions of Graduate Student Research Skills ......MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 1 Abstract The mismatch between the academic preparation and realistic

MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 18

Lovitts, B. E. (2007). Making the implicit explicit: Creating performance expectations for the

dissertation. Sterling, VA: Stylus Publishing.

Lovitts, B., & Nelson, C. (2000). The hidden crisis in graduate education: Attrition from

Ph.D programs. Academe, 86, 44-51.

Manathunga, C. (2005). The development of research supervision: ‖Turning the light on a

private space‖. The International Journal for Academic Development, 10(1), 17-30.

Mervis, J. (2000). Graduate educators struggle to grade themselves. Science, 287(5453), 568-

570.

Mullins, G. & Kiley, M. (1998) Quality in postgraduate research: the changing agenda, in:

M. Kiley & G. Mullins (Eds) Quality in Postgraduate Research: managing the new agenda.

Proceedings of the 1998 Quality in Postgraduate Research Conference, pp. 1–13 (Adelaide,

Advisory Centre for University Education, University of Adelaide).

Nickerson, R. S. (1999). How we know—and sometimes misjudge—what others know:

Imputing one‘s own knowledge to others. Psychological Bulletin, 125(6), 737-159.

Nyquist J., Manning L., Wulff, D. H., Austin, A. E., Sprague, J., Fraser, P. K., Calcagono, C., &

Woodford, B. (1999). On the road to becoming a professor: The graduate student

experience. Change, 31, 18–27.

Nyquist, J., & Woodford, B. (2000). Re-envisioning the PhD: What concerns do we have?

Seattle, WA: University of Washington, Center for Instructional Development and

Research.

Paglis, L., Green, S., & Bauer, T. (2006). Does mentor mentoring add value? A longitudinal

study of mentoring and doctoral student outcomes. Research in Higher Education, 47(4),

451-476.

Pearson, M. & Brew, A. (2002). Research training and supervision. Development Studies in

Higher Education, 27(2), 135-150.

Petty, R. E., Fleming, M. A., & Fabrigar, L. R. (1999). The Review Process at PSPB: Correlates

of Interreviewer Agreement and Manuscript Acceptance. Personality and Social Psychology

Bulletin, 25(2), 188-203.

Schwartz, D., & Bransford, J. D. (1998). A time for telling. Cognition and Instruction, 16(4),

475-522.

Sweitzer, V. (2009). Towards a theory of doctoral student professional identity development: a

developmental networks approach. The Journal of Higher Education, 80 (1), 1-33.

Page 20: The [Mis] Perceptions of Graduate Student Research Skills ......MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 1 Abstract The mismatch between the academic preparation and realistic

MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 19

Thorley, N., & Stofflet, R. (1996). Representation of the conceptual change model in science

teacher education. Science Education, 80, 317-339.

Author et al. (2007). Development of a universal rubric for assessing students' science inquiry

skills. Paper presented at the National Association of Research in Science Teaching.

Author et al. (in press). Development of a ‗universal‘ rubric for assessing students‘ scientific

reasoning skills using scientific writing. Assessment and Evaluation in Higher

Education.

Willison, J. & O‘Regan, K. (2007). Commonly known, commonly not known, totally unframed:

a framework for students becoming researchers. Higher Education Research &

Development, 26(4), 393-409.

Page 21: The [Mis] Perceptions of Graduate Student Research Skills ......MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 1 Abstract The mismatch between the academic preparation and realistic

MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 20

Table 1

Graduate Student Participants by Programmatic Affiliation and Year in Program

Discipline 1st Year 2nd Year 3rd Year 4th Year or

Beyond Total

Education 0 1 2 0 3

Engineering 7 5 2 3 17

Math 5 0 1 1 7

Science 25 3 2 1 31

Total 37 9 7 5 58

Page 22: The [Mis] Perceptions of Graduate Student Research Skills ......MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 1 Abstract The mismatch between the academic preparation and realistic

MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 21

Table 2.

Codes Used to Designate Disposition, Skill, and Knowledge

Disposition Skill Knowledge

Aggressive Communication Big Picture/Context

Collaborative Computer skills Conceptual Knowledge

Confident Data Analysis Terminology

Creative Defining Research Problem Theoretical Framework

Critical Thinking Field work

Enthusiastic Literature Review

Experienced Math/Statistics Skills

Focused Operating Lab Equipment

Hard Working Research Design

Independent

Inquisitive

Intuitive

Meticulous

Organized/Time Management

Persistent/Motivated

Page 23: The [Mis] Perceptions of Graduate Student Research Skills ......MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 1 Abstract The mismatch between the academic preparation and realistic

MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 22

Table 3.

Comparison of Proposal Rubric Categories to Mentor-Mentee Strengths and Weaknesses in Disposition, Skill and Knowledge

Perceived Strengths and Weaknesses

Analysis

Skills

Big

Picture

Conceptual

Knowledge

Critical

Thinking

Data

Analysis

Defining

Problem

Literature

Review

Math/Stat

Skills

Research

Methods

Proposal Rating Categories

Introduction/Context Writer provides a clear sense of what is

known and what gaps exist in our

knowledge. Background information is

accurate, relevant and provides a clear

rationale for the objectives.

X X

X

Hypothesis Testable Research questions and expected findings

are clearly stated and plausible/testable

using proposed methods. Note For

experimental studies or when otherwise

appropriate, specific hypotheses should be

stated. As appropriate, plausible alternative

explanations / mechanisms / hypotheses

should be explained and the proposed

research design will allow investigators to

distinguish among them.

X

X

X

X

Methods-Validity/Reliability Appropriate controls and/or mechanisms to

ensure validity and reliability are present

and explained. Degree of replication /

sample size is explained and appropriate

for the research area.

X

X

X

Methods-Exp. Design Data collection plan / Experimental design

is likely to produce salient and fruitful

X

X

X

Page 24: The [Mis] Perceptions of Graduate Student Research Skills ......MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 1 Abstract The mismatch between the academic preparation and realistic

MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 23

results (i.e. addresses the research

objectives posed).

Results-Data Selection Data produced by the research will be

comprehensive, informative, accurate and

relevant to the questions / hypotheses

posed.

If actual data are presented, use these

criteria Data produced by the research will

be comprehensive, accurate and relevant.

X

X

Results-Data Analysis Proposed interpretive framework and/or

statistical methods are appropriate for

research objectives. Rationale for the

choice of methods is explained clearly.

Expected evidence for data‘s validity,

reliability, and or statistical significance

(as appropriate to the proposed study) are

indicated.

X X X

X

Discussions/Conclusion Conclusion could be clearly and logically

drawn from predicted data. A logical chain

of reasoning from hypothesis/ design

element to predicted data to conclusions

are clearly and persuasively explained.

X

X X X X X

Broader Impacts How well does the activity advance

discovery and understanding while

promoting teaching, training, and learning?

How well does the proposed activity

broaden the participation of

underrepresented groups? What may be the

benefits of the proposed activity to

society?

X X

X

Page 25: The [Mis] Perceptions of Graduate Student Research Skills ......MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 1 Abstract The mismatch between the academic preparation and realistic

MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 24

Limitations/Significance of

Findings Limitations of findings and remaining

questions to be answered in relation to the

phenomenon of interest are discussed.

Alternative explanations of the predicted

data are considered and weighted against

conclusions. How this study relates to

other knowledge in the field is clearly

discussed.

X

X

Primary Literature Use Relevant literature is reasonably complete

and present in both the

Introduction/context and Discussion

sections. Use of the literature demonstrates

the intellectual merit of the proposed

research and specifies how it relates to

other work in the field. Citations follow an

accepted format for the field and are

accurate. (Please indicate the citation style

you are using in the literature cited

section.) .

X

X

Page 26: The [Mis] Perceptions of Graduate Student Research Skills ......MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 1 Abstract The mismatch between the academic preparation and realistic

MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 25

Table 4.

Comparison of Proposal Rubric Categories to Mentor-Mentee Strengths and Weaknesses in Disposition, Skill and Knowledge

Fall

Semester Introduction: Context Testable Hypothesis Validity/Reliability Experimental Design Data Selection Data Presentation

Faculty Student Faculty Student Faculty Student Faculty Student Faculty Student Faculty Student

CountFisher's

ExactCount

Fisher's

ExactCount

Fisher's

ExactCount

Fisher's

ExactCount

Fisher's

ExactCount

Fisher's

ExactCount

Fisher's

ExactCount

Fisher's

ExactCount

Fisher's

ExactCount

Fisher's

ExactCount

Fisher's

ExactCount

Fisher's

Exact

Literature

Review

Strength3 0.999 4 0.429* 3 0.429* 3 0.464* 1 0.999 3 0.464* 1 0.486* 3 0.999 1 0.999 2 0.999 2 0.399* 2 0.999 Above Mean

0 2 0 3 1 3 2 3 2 4 0 4 At Mean or Below

Weakness3 0 2 0 1 0 3 1 2 1 1 0 Above Mean

1 2 2 2 3 2 1 1 2 1 3 2 At Mean or Below

Analytical

Skills

Strength1 0.999 0 0.999 0 0.999 1 0.999 1 0.999 1 0.999 Above Mean

1 2 2 1 1 1 At Mean or Below

Weakness0 0 0 0 0 0 Above Mean

1 1 1 1 1 1 At Mean or Below

Conceptual

Knowledge

Strength1 0.999 0 0.999 1 0.999 1 0.375* 1 0.429* 0 0.999 1 0.999 0 0.999 0 0.999 0 0.999 0 0.999 0 0.999 Above Mean

0 1 0 0 0 1 0 1 1 1 1 1 At Mean or Below

Weakness3 3 2 2 1 1 3 4 2 2 1 2 Above Mean

1 4 2 5 3 6 1 3 2 5 3 5 At Mean or Below

Critical

Thinking

Strength1 0.999 1 0.999 1 0.999 1 0.999 0 0.999 2 0.333* Above Mean

1 1 1 1 2 0 At Mean or Below

Weakness0 0 0 0 0 0 Above Mean

1 1 1 1 1 1 At Mean or Below

Big Picture

Strength2 0.999 1 0.999 2 0.999 2 0.999 2 0.999 1 0.999 2 0.999 1 0.999 2 0.999 2 0.999 1 0.999 2 0.999 Above Mean

0 2 0 1 0 2 0 2 0 1 1 1 At Mean or Below

Weakness2 3 2 3 1 3 2 3 1 2 1 3 Above Mean

0 2 0 2 1 2 0 2 1 3 1 2 At Mean or Below

Research

Methods

Strength1 0.999 1 0.999 1 0.999 1 0.999 1 0.999 0 0.333* Above Mean

1 1 0 1 1 2 At Mean or Below

Weakness1 1 0 1 0 1 Above Mean

0 0 1 0 1 0 At Mean or Below

Defining

the Problem

Strength0 0.999 1 0.999 1 0.999 1 0.999 1 0.999 0 0.999 Above Mean

1 0 0 0 0 1 At Mean or Below

Weakness0 0 1 0 0 0 Above Mean

1 1 0 1 1 1 At Mean or Below

Data

Interpret.

Strength1 0.999 0 0.333* 0 0.333* 1 0.999 0 0.333* 0 0.333*Above Mean

0 1 1 0 1 1 At Mean or Below

Weakness2 2 2 2 2 2 Above Mean

0 0 0 0 0 0 At Mean or Below

Page 27: The [Mis] Perceptions of Graduate Student Research Skills ......MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 1 Abstract The mismatch between the academic preparation and realistic

MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 26

Page 28: The [Mis] Perceptions of Graduate Student Research Skills ......MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 1 Abstract The mismatch between the academic preparation and realistic

MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 27

Table 4. Continued

Comparison of Proposal Rubric Categories to Mentor-Mentee Strengths and Weaknesses in Disposition, Skill and Knowledge-Fall Semester

Page 29: The [Mis] Perceptions of Graduate Student Research Skills ......MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 1 Abstract The mismatch between the academic preparation and realistic

MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 28

Fall

Semester Data Analysis Valid Conclusions Broader Implications Limitations/Significance Primary Literature Total

Faculty Student Faculty Student Faculty Student Faculty Student Faculty Student Faculty Student

CountFisher's

ExactCount

Fisher's

ExactCount

Fisher's

ExactCount

Fisher's

ExactCount

Fisher's

ExactCount

Fisher's

ExactCount

Fisher's

ExactCount

Fisher's

ExactCount

Fisher's

ExactCount

Fisher's

ExactCount

Fisher's

ExactCount

Fisher's

Exact

Literature

Review

Strength1 0.999 1 0.999 1 0.999 2 0.999 1 0.999 2 0.429* 1 0.333* 4 0.429* 3 0.999 2 0.429* 2 0.486* 2 0.999 Above Mean

1 5 2 4 2 4 1 2 0 4 1 4 At Mean or Below

Weakness2 0 1 0 2 2 0 0 3 1 1 0 Above Mean

2 2 3 2 2 0 4 2 1 0 3 2 At Mean or Below

Analytical

Skills

Strength0 0.999 1 0.999 0 0.999 1 0.999 1 0.999 0 0.999 Above Mean

2 1 2 1 1 2 At Mean or Below

Weakness0 0 0 0 1 0 Above Mean

1 1 1 1 0 1 At Mean or Below

Conceptual

Knowledge

Strength0 0.999 0 0.999 0 0.999 1 0.999 0 0.999 0 0.999 0 0.999 0 0.999 0 0.399* 1 0.999 0 0.999 Above Mean

1 1 1 0 1 1 1 1 1 0 1 At Mean or Below

Weakness2 1 2 4 2 3 0 2 3 3 1 Above Mean

2 6 2 3 2 4 4 5 1 4 3 At Mean or Below

Critical

Thinking

Strength0 0.999 2 0.333* 2 0.333* 2 0.333* 0 0.333* 1 0.999 Above Mean

2 0 0 0 2 1 At Mean or Below

Weakness0 0 0 0 1 0 Above Mean

1 1 1 1 0 1 At Mean or Below

Big Picture

Strength2 0.999 1 0.999 1 0.999 2 0.999 2 0.333* 3 0.196* 0 0.999 1 0.107* 2 0.999 1 0.999 2 0.999 1 0.999 Above Mean

0 2 1 1 0 0 1 2 0 2 0 2 At Mean or Below

Weakness2 3 1 3 0 2 1 5 2 3 1 3 Above Mean

0 2 1 2 2 3 1 0 0 2 1 2 At Mean or Below

Research

Methods

Strength0 0.999 0 0.333* 0 0.333* 0 0.999 1 0.999 0 0.333* Above Mean

1 2 2 2 1 2 At Mean or Below

Weakness0 1 1 0 1 1 Above Mean

1 0 0 1 0 0 At Mean or Below

Defining

the Problem

Strength1 0.999 0 0.999 0 0.999 0 0.999 1 0.999 1 0.999 Above Mean

0 1 1 1 0 0 At Mean or Below

Weakness0 0 1 1 0 0 Above Mean

1 1 0 0 1 1 At Mean or Below

Data

Interpret.

Strength0 0.333* 1 0.999 0 0.333* 1 0.999 0 0.333* 1 0.999 Above Mean

1 0 1 0 1 0 At Mean or Below

Weakness2 2 2 1 2 2 Above Mean

0 0 0 1 0 0 At Mean or Below

Page 30: The [Mis] Perceptions of Graduate Student Research Skills ......MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 1 Abstract The mismatch between the academic preparation and realistic

MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 29

Table 5.

Comparison of Proposal Rubric Categories to Mentor-Mentee Strengths and Weaknesses in Disposition, Skill and Knowledge- Spring Semester

Page 31: The [Mis] Perceptions of Graduate Student Research Skills ......MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 1 Abstract The mismatch between the academic preparation and realistic

MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 30

Spring

SemesterIntroduction: Context Testable Hypothesis Validity/Reliability Experimental Design Data Selection Data Presentation

Faculty Student Faculty Student Faculty Student Faculty Student Faculty Student Faculty Student

CountFisher's

ExactCount

Fisher's

ExactCount

Fisher's

ExactCount

Fisher's

ExactCount

Fisher's

ExactCount

Fisher's

ExactCount

Fisher's

ExactCount

Fisher's

ExactCount

Fisher's

ExactCount

Fisher's

ExactCount

Fisher's

ExactCount

Fisher's

Exact

Literature

Review

Strength 0 0.999 4 0.999 0 0.999 2 0.429* 1 0.999 2 0.429* 0 0.999 2 0.999 0 0.999 2 0.999 0 0.999 3 0.999 Above Mean

1 2 1 4 0 4 1 4 1 4 1 3 At Mean or Below

Weakness1 1 1 1 1 1 1 0 1 0 1 0 Above Mean

1 0 1 0 1 0 1 1 1 1 1 1 At Mean or Below

Conceptual

Knowledge

Strength1 0.999 0 0.999 0 0.999 0 0.999 0 0.999 0 0.999 Above Mean

1 2 2 2 2 2 At Mean or Below

Weakness1 1 0 1 1 1 Above Mean

1 1 2 1 1 1 At Mean or Below

Big Picture

Strength0 0.999 2 0.999 0 0.999 2 0.999 0 0.999 1 0.999 0 0.999 2 0.999 0 0.399* 1 0.999 0 0.199* 1 0.999 Above Mean

1 1 1 1 1 2 1 1 1 2 1 2 At Mean or Below

Weakness1 0 2 0 1 0 2 0 3 0 4 1 Above Mean

3 1 2 1 3 1 2 1 1 1 0 0 At Mean or Below

Math/

Statistics

Strength 0 0.999 0 0.099* 0 0.099* 0 0.399* 0 0.999 1 0.999Above Mean

2 2 2 2 2 1 At Mean or Below

Weakness0 3 3 2 1 1 Above Mean

3 0 0 1 2 2 At Mean or Below

Data

Interpret.

Strength0 0.999 0 0.999 0 0.999 0 0.999 0 0.999 0 0.999 Above Mean

1 1 1 1 1 1 At Mean or Below

Weakness0 0 0 0 0 1 Above Mean

1 1 1 1 1 0 At Mean or Below

Page 32: The [Mis] Perceptions of Graduate Student Research Skills ......MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 1 Abstract The mismatch between the academic preparation and realistic

MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 31

Table 5. Continued

Comparison of Proposal Rubric Categories to Mentor-Mentee Strengths and Weaknesses in Disposition, Skill and Knowledge- Spring Semester

Page 33: The [Mis] Perceptions of Graduate Student Research Skills ......MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 1 Abstract The mismatch between the academic preparation and realistic

MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 32

Spring

SemesterData Analysis Valid Conclusions Broader Implications Limitations/Significance Primary Literature Total

Faculty Student Faculty Student Faculty Student Faculty Student Faculty Student Faculty Student

CountFisher's

ExactCount

Fisher's

ExactCount

Fisher's

ExactCount

Fisher's

ExactCount

Fisher's

ExactCount

Fisher's

ExactCount

Fisher's

ExactCount

Fisher's

ExactCount

Fisher's

ExactCount

Fisher's

ExactCount

Fisher's

ExactCount

Fisher's

Exact

Literature

Review

Strength 0 0.999 0 0.999 0 0.999 2 0.999 0 0.999 4 0.999 0 0.999 2 0.999 0 0.999 4 0.999 1 0.999 2 0.999 Above Mean

1 6 1 4 1 2 1 4 1 2 0 4 At Mean or Below

Weakness1 0 0 0 1 1 0 0 1 1 1 0 Above Mean

1 1 2 1 1 0 2 1 1 0 1 1 At Mean or Below

Conceptual

Knowledge

Strength0 0.999 0 0.999 0 0.999 0 0.999 0 0.999 0 0.999 Above Mean

2 2 2 2 2 2 At Mean or Below

Weakness1 0 1 0 0 0 Above Mean

1 2 1 2 2 2 At Mean or Below

Big Picture

Strength0 0.999 1 0.999 0 0.999 3 0.250* 0 0.999 1 0.999 0 0.999 1 0.999 0 0.999 2 0.999 0 0.999 1 0.999 Above Mean

1 2 1 0 1 2 1 2 1 1 1 2 At Mean or Below

Weakness1 0 1 0 1 0 1 0 1 0 1 0 Above Mean

3 1 3 1 3 1 3 1 3 1 3 1 At Mean or Below

Math/

Statistics

Strength 0 0.099* 0 0.999 0 0.399* 0 0.999 0 0.999 0 0.399*Above Mean

2 2 2 2 2 2 At Mean or Below

Weakness3 1 2 1 1 2 Above Mean

0 2 1 2 2 1 At Mean or Below

Data

Interpret.

Strength0 0.999 0 0.999 0 0.999 1 0.999 1 0.999 0 0.999 Above Mean

1 1 1 0 0 1 At Mean or Below

Weakness0 0 0 0 0 0 Above Mean

1 1 1 1 1 1 At Mean or Below

Page 34: The [Mis] Perceptions of Graduate Student Research Skills ......MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 1 Abstract The mismatch between the academic preparation and realistic

MENTOR PERCEPTIONS AND STUDENT RESEARCH SKILLS 33

Table 6.

Cited Experiences Leading to a Perceived Improvement in Research Skills

Experience Type

Number of

Instances

Collaboration with mentor 22

Collaboration with non-mentor (i.e. peers, post doctoral students) 7

Independent scholarly work (i.e. reading journal articles,

coursework) 11

Trial and error 29

Presenting at or attending conferences 16