Translating research experiences to employability skills: using evidence to make a convincing case

ASPA-ASCEPT 2015 Hobart, Tasmania

Translating research experiences

into employability skills: using evidence to make a

convincing case.

Kirsten Zimbardi & Kay ColthorpeEducational Research Unit, School of Biomedical Science,

University of Queensland, St Lucia, QLD


Science education for the 21st century workplace

Students need to develop the skills to deal with novel, complex, unstructured problems

The Boyer Commission (1998) Reinventing undergraduate education: a blueprint for America’s research universities. National Research Council (2003) BIO2010: Transforming undergraduate education for future research biologists President’s Council of Advisors on Science and Technology (2012) Engage to Excel: Producing one million additional college gradates with degrees in STEM

Students need to learn to ‘think like a scientist’


What is “scientific thinking”?aka scientific reasoning?

1. Existing knowledge used to build a hypothesis• Previous evidence from scientific literature

2. Hypothesis includes a specific cause and specific measurable outcome

3. Methods appropriate to test hypothesis

4. Results analysed and presented in relation to hypothesis

5. Findings interpreted in relation to existing evidence and knowledge• Dealing with unexpected findings

Dunbar and Fugelsang(2006) The Cambridge Handbook of Thinking and Reasoning, pp 705, 708


Australian Learning Teaching Academic StandardsThreshold Learning Outcomes

Inquiry and problem solving (TLO3)Critically analyse and solve scientific problems by:

• Gathering, synthesising and critically evaluating information from a range of sources

• Designing and planning an investigation

• Selecting and applying practical and/or theoretical techniques or tools in order to conduct and investigation

• Collecting, accurately recording, interpreting and drawing conclusions from scientific data

Jones, Yates & Kelder (2011) Learning & Teaching Academic Standards Project: Science


Nature of scientific knowledgea threshold learning concept in biology

Understanding science3 (TLO1)Demonstrate a coherent understanding of science by• Articulating the methods of science and explaining why current scientific

knowledge is both contestable and testable by further inquiry

1. Taylor et al (2011) Threshold Concepts in Biology; 2. Zimbardi, Meyer et al (2014) NAIRTL; 3. Jones, Yates & Kelder (2011) Learning & Teaching Academic Standards Project: Science;

Hypothesis formulation is a threshold concept in the biology1,2

Essential to developing disciplinary expertise & habits of mind


Undergraduate research experiences & inquiry-based curricula

Embedding research experience in undergraduate curricula1

• Apprenticeships & Industry placements• Inquiry & Methods courses

Vertically-integrated, inquiry-based practical curricula2,3

1. Zimbardi & Myatt (2014) Studies in HEd; 2. Zimbardi et al (2013) Adv Phys Ed; 3. Kirkup & Johnson (2013) Good practice guide: Threshold Learning Outcome 3 - Inquiry & Problem Solving

Year 1 Sem 2

Yaer 2 Sem 1

Year 2 Sem 2

Year 3 Sem 1

Year 3 Sem 2

Scaffolded Guided choice

Extendedchoice

Cutting edgemethods

Professionalisation


What does the development of scientific thinking skills look like – in assessment?

Students’ scientific writing (reports)Early writing tended to use definitive terminology.Later reports more often contained more speculative language.

Student 2:Yr 2 Sem 1: “The results of this study confirmed the hypothesis...”

Yr 3 Sem 2: “This result suggests that...is most likely.”

Student 5:Yr 2 Sem 1: “This study has proven that...”

Yr 3 Sem 2: “...there is no certainty that an association is present, however, the trend observed is of interest.”

Colthorpe, Zimbardi et al (in press) IJISME


Evidence of scientific thinking developmentDealing with unexpected results

Student 4:Yr 2 Sem 2: “As such the lack of effect observed in the current study may be a result of the relatively low intensity and short duration for which subjects exercised.”

Yr 3 Sem 2: “No change in the expression of LEP-R in the kidneys of rats exposed to either vehicle or STZ was found (Figure 1)...Alternatively LEP-R regulation in hypoleptinemic conditions may be via changes in the localisation of the receptor. Soluble LEP-R is able to bind circulating leptin and delay degradation (Huang, et al., 2001) as well as inhibit leptin signalling (Schaab, et al., 2012). It is thus possible, that in conditions of hypoleptinemia, there may be a proportional decrease in soluble LEP-R without any change in total receptor expression.”

Colthorpe, Zimbardi et al (in press) IJISME


The development of scientific thinking skills – in action…

UQ T&L Fellowship 2012• Videos of 4 groups of students in a series of 3

inquiry-practical classes • 3 interviews with each student across semester• Students annotated their videos highlighting

inquiry skills in action

Zimbardi et al (in press) IJISME


Critical points of variation in students’ ‘lived experience’

5 students working together in the same groupsame experiment, same results, very different interpretations & learning outcomes è phenomenographical study

Curriculum interacts with students' prior experience & expectations

Critical ‘stuck’ pointsScientific knowledge is:

• concrete & complete• NOT tentative, contestable & ‘gappy’


The development of scientific thinking skills – in students’ own words

Meta-cognitive awareness of learning strategies and difficulties improves academic performance1

Meta-cognitive assessment tasks that prompt students to:• think about their learning gains and strategies2

• translate educational experiences into workplace scenarios

Behavioural questionse.g. “Tell me about a time when you successfully used your scientific problem skills”

Hypothetical questionse.g “Suggest a potential approach for investigating this issue…”

1. Hattie (2009) Visible learning; 2. Colthorpe et al (in press) J. Learning Analytics


Hints in the questions

• Deal with complex, unstructured, novel problems

• Break the problem into testable chunks

• Use an evidence-based approach to the solution

• Arrive at the best solution at a given time


Methods

3rd year (final semester) students from 2013 - 2015Mix of science/professional career paths

1-2 Meta-cognitive tasks during semester Behavioural question (2014: n = 78)Hypothetical questions (2013-2015: n = 399)

2-3 scenarios – near transfer to far transfer

Inductive thematic analysisQualities of convincing answersCritical stuck points


Evidencing scientific problem solving- the devil is in the detail

No transfer: I think that most situations in a lab require scientific inquiry

Parroting back the question: The solution, I found, was to take science as it is and only in sizeable, digestible chunks that I can deal with seperate from the rest of everything.

More detail in setup, but still parroting back the question: When I was doing work experience in Tanzania, I was exposed to a myriad of medical mysteries, ranging from fascinating, to simply disturbing. Each new patient I was introduced to brought with them a problem that required a number of doctors, medical students and volunteers to put their heads together [complex, collaboration] and come up with a solution. For me though, I have had minimal exposure to much of the pathologies of third world medicine [novel], and my biggest problem was maintaining a vertical stance, and not offending the patients by my fainting/vomiting/absence. However, I reminded myself of skills I have acquired through practical attendance at UQ. My rather new ability to objectively observe a situation, ask the right questions and cross-check facts with others and literature made me feel as though my presence in the hospital was actually beneficial for all involved.

Still lacking specific details – what & how?


Too many details derail the relevanceIn BIOM2012 systems physiology, a course at UQ, during the practical component, we conducted an experiement to examine the effects of caffeine on skeletal muscle. Although we had reviewed prior research, and it had found a significant effect, we did not find any significant results in our data. Consequently we re-read the research we had found and had to go searching for further articles which may tell us as to why we did not observe significant results. Through further research we discovered that the dosage that we gave in the practical was insufficient to observe the positive effects of caffeine on exercise performance. This information enabled us to effectively explain our results and suggest future avenues of research.

Completing a 3rd year microbiology subject at the University of Queensland me and my research group were ask to determine experimentally if similarities between the metabolomic profile of brewing yeasts can be used to infer their genetic relatedness. We established a number of cultures of 4 separate yeast strains in the same media [controls] which would be analysed via GC-MS to produce a metabolomicprofile. Rapid PCR [methods] was used to determine if there were any obvious genetic differences. The genome of the yeast strains were sequenced and mapped to the laboratory yeast strain [standards] to identify genetic differences. Also a gene implicated [?] in the production of a specific metabolic was analysed in each yeast strain (both metabolomic profile and protien translation) to identifiy any differences.

Lack of explicit links/transfer to workplace


Workplace transfer with specific hypothesis, methods and tentative solution

Coming from a nursing background, I was once with an elderly male patient who was admitted for repeated falls. These falls happened almost exclusively in the morning. This patient had multiple co-morbidities, however did suffer from long standing hypertension [identify/narrow problem]. After conversing with the patient for a space, and reviewing his medication chart [evidence], it came to my attention that almost his entire regimen of anti-hypertensive medications (there were at least 4) were prescribed in the morning [hypothesis]. I took this man's blood pressure in regular intervals (after these morning medications were administered) standing up for approximately 5 minutes and noticed a considerable [quantity] decrease in his systolic pressure [evidence]. Additionally, he became symptomatic when standing [sufficient/necessary test]. It had become obvious that his symptoms were related to this systolic decrease when standing (postural hypotension), and so I discussed this with the medical registrar and suggested [tentative, collaborative] at least a portion of his anti-hypertensive medications be changed to night time just before sleeping, to eliminate these symptoms in the morning.


Evidence-based predictive modeling- in the workplace, linked to education

An example of such a situation was during my time as acting store manager when I had to compile an ordering sheet to order stock for the next week. I understand this may seem like a situation that really doesn't use science inquiry skills but I believe that my studies in science had provided the thinking skills necessary to overcome this problem. Initially I began with some very backgroundthinking, akin to beginning a scientific investigation, it was going to be school holidays and it was predicted to rain, meaning it was going to be busier than usual and thus I needed to order more stock than we had ordered in the previous week, but I didn't know how much [narrowing problem]. I decided that if I was going to know how much to order, I had to know how much stock we currently had in the first place. From there, I compared my count of stock to my manager's from the week before and put a baseline order that matched up with what she had put based on the amount of stock. From there I looked at product sales from previous Saturdays, as I knew that school holidays usually do the same sales per day as a Saturday [equivalent model] and found which products we had been selling the most of and made a note to order more of those than usual [specific]. From there I also took in to consideration the fact that it was going to rain so I decided to order 2 extra of everything to ensure we didn't run out because we could just freeze escess stock. In the end my ordering was sufficient [final check on success] and I believe it was my knowledge in scientific process and analysing of data to make conclusions that allowed me to overcome this problem.


Conclusions & Implications

Meta-cognitive tasks are an easy/efficient way to prompt students to: • Practice articulating their answers in a low stakes situation• Reflect on their learning gains and strategies• Think about how to evidence ‘soft skills’

Good articulation of answers to interview questions• Use specific, detailed, relevant evidence• Include explicit links showing relevance to question/context • Probably requires practice for many students

• may benefit from seeing analyses like these


Thank youFunding

UQ T&L FellowshipUQ SoTL Early Career Grant

Thank you to all of our students

Kirsten [email protected]