School of Medical Sciences

Patsie: Lead facilitator: ePortfolios pedagogy/rationale/skills in the Medical Sciences and beyond

Jia-Lin: Lead Career learning and ePortfolios, Data/statistics / spokesperson

Thuan: Project manager/ spokesperson - Aligning Assessment Project implementation across disciplines, 2015

Annie: Project manager/ spokesperson - Fellowship implementation across programs, 2016

Richard, Trevor, Nicole, Cristan, Thomas, Julian, Sue:Discipline Specific Experts: will lead table discussions and discuss their course and task specific approaches

Stories and Strategies from UNSW Australia

Collaborative, Program-wide Alignment of Assessments and ePortfolios to Build Teamwork Skills and Reflective Practice for

Medical Sciences Students:

ePortfolios Australia: 29/09/16 Workshop 2 Room 2174, 2-4:30pm

Title: Collaborative, Program-wide Alignment of Assessments and ePortfolios to Build Teamwork Skills and Reflective Practice for Medical Science Students: Stories and Strategies from UNSW Australia

Themes: Working across disciplines, building teamwork skills, scaffolding reflective practice, ePortfolios for future use beyond university , extracting data from ePortfolios

Schedule• Workshop overview: 15 min • Group discussion: 30 min• Group presentation: 30 min (5 min per group/table)• Discussion: Design, Development, Implementation (30 mins) • Q & A (30 min)• Summary (15 min)

AIMS: Workshop

• To engage colleagues in thinking about how to align assessment tasks across programs and disciplines

• Implementation of ePortfolio use linked to assessment tasks

• Discuss the idea of ePortfolios, rubrics and data to recognise attainment of skills

ePORTFOLIO / Reflective Blogging

Select

Collect

ReflectCurate

Present

Self appraisal

Gathering occupational information

Goal setting and selection

Planning and problem solving

Self awareness

Decision making

Opportunity awareness

Transition learning

Yang et al, 2014Polly et al., 2015

UNSW – Institutional Priorities for Graduates

Global Citizens

Scholars

Leaders Professionals

UNSW, Strategic Intent, 2013UNSW, Key Priorities, 2012-2016

UNSW Strategy, 2025

The ‘Why’

P. Polly et al, 2013

Understanding oftheir discipline in itsInterdisciplinary context

Capable of initiating and embracing innovation and change

Capable of ethical, self-directed practice and independent lifelong learning

Graduates who are culturally adept and capable of respecting diversity

2:00-3:15pm

Workshop Part 1

2:10-2:40pmGroup discussion: 30 min

• What is your understanding of a student ePortfolio?• What is your experience in ePortfolio study and/or practice?• What are 3 important issues in current ePortfolio study and

practice from your points of view?• What are your 3 suggestions improving ePortfolio study and

practice in educational institutes in Australia?

2:40-3:15pm (35 min, 5 min/table)Group feedbackePortfolio definition

Three important issues

Resolving issues

3:15-3:30pm

Break

3:30-4:30pm

Workshop Part 2

3:30-4:00pm (30 min) What have we done in resolving the issues? UNSW Stories and strategiesPATSIE: Overview (3-5mins)PATSIE: Design: Top down from school incorporating teamwork skill development and ePortfolio in curriculum of science program

PATSIE: Development• Teamwork, Modified AAC&U Rubric• ePorfolio, template questions, reflection rubricJIA-LIN: Career development learning, CDMSE survey

Implementation: THUAN (APP; 3mins): Moodle workshop, self-/peer assessment, WordPressANNIE (TF; 3mins): Moodle as an LMS, central data collection

JIA-LIN (3mins): Statistical analysis, ePortfolio reflection analysis (N Vivo)

Discipline Expert Stories: 3 mins each – suggestion, assessment strategy linked to ePortfolio slide, data slideTREVOR: Neuro Physiology - 3rd Year RICHARD: Neuro Physiology – 2nd Year NICOLE: Pharmacology – 3rd Year THOMAS: Anatomy - 3rd Year CRISTAN: Pathology – 2nd Year SUE: Medical Science – 1st Year

ISSUE and WHY we use ePortfolio

• Students find teamwork activities challenging

• Assessment tasks coupled with reflective practice are critical for building professional skills such as teamwork

• Address graduate attributes to launch into professional skills and career learning

• Capturing, tracking and extracting data from ePortfolios is a challenge

Long

itud

inal

, Pro

gram

-wid

e us

e of

eP

ortf

olio

ProfessionalReadiness

Pathways Further studies Research Industry

Year 1

Year 2

Year 3

Year 3

Year 4

CareerAwareness

ePortfolio Use in the Medical SciencesCross-Discipline Approaches – WordPress 2015, Moodle 2016

Skills Building

Anatomy Pathology Pharmacology Physiology

Transverse, Cross-context and cross-discipline

UNSW L&T Innovation and Development Grant, Round 2 2014Aligning assessment to track development of graduate capabilities, professional skills and life-longlearning in the biomedical sciences

UNSW Ethics ApprovalHC15134

P. Polly et al, 2014

Identity

Narrative

Employability

Teacher Involvement – Medical Science Program

Aligning Assessments

P. Polly et al, 2014

ePortfolio: a high impact practice in higher education:

Statistical analysis and ePortfolio assessmentJia Lin Yang

ePortfolio Forum Workshop28 September 2016

ePortfolios impact on learning: Statistical analysis

• Scholarship practice (educational publications)• Hypothesis testing and errors• Descriptive and inferential statistics• Selection of correct statistical analysis methods (Yang et al.

2010 eSTATS-CHOICE: http://dev.adaptiveelearning.com/demos/DTv3/)

ePortfolio assessment- https://teaching.unsw.edu.au/printpdf/535

• ePortfolio – digitised collection of artefacts (Lorenzo & Ittelson, 2005)

• Software (eg. NVivo)• Rubrics (Different focuses)• Others (eg. Badging)

Rubric - https://www2.uwstout.edu/content/profdev/rubrics/eportfoliorubric.html

• Selection of Artefacts • Descriptive Text • Reflective Commentary • Citations • Navigation • Usability and Accessibility: Text Elements, Layout, and

Colour• Writing Conventions • Multimedia Elements (Optional)

© COPYRIGHT 2001-2016 Joan Vandervelde All Rights Reserved. University of WisconsinRevised: Monday, January 11, 2016

NVivo - http://www.qsrinternational.com/nvivo-product

NVivo is software that supports qualitative and mixed methods research. It’s designed to help you organize, analyze and find insights in unstructured, or qualitative data like: interviews, open-ended survey responses, articles, social media and web content.

A systemic approach:• Converting data into digital transcripts• Coding/recoding• Constant comparing• Clustering• Storing• Facilitating the connection• Pilot inductive and interpretive analysis• Within/across cases• Commonalities and exceptionalities

Badging

A badge is a visual representation of an accomplishment, achievement or skill acquisition—more granular than a formal degree, but helps to make incremental learning more visible.

(http://www.educause.edu/badging)

A digital badge is a validated indicator of accomplishment, skill, quality, or interest that can be earned in many learning environments. Open digital badging makes it easy for anyone to issue, earn, and display badges across the web—through an infrastructure that uses shared and open technical technical standards.

(https://www.hastac.org/initiatives/digital-badges)

Career Development Learning across Year 3 Science Courses

Jia Lin Yang

Research article: Jia-Lin Yang, Patsie Polly, Thomas Fath, Nicole Jones, John Power.2016. Int J Science Math Tech Learning. 23(3), 33-46. http://ijlsmtl.cgpublisher.com/product/pub.266/prod.111

Pilot study: Educational interventions on career development learning (2012, PATH3208)

Yang et al. Int J Adult, Commun Prof Learning Vol 22 (1), 2015, pp 1-17.

Table 4: Results of CDSME Survey Pre- and Post the Study Intervention Total (n=32) Female (n=21) Male (n=11)

mean SD p mean SD p mean SD p

Self-appraisal pre 18.1 3.1 0.014 18.3 2.6 0.019 17.8 3.9 0.240

post 19.8 2.6 19.8 2.0 19.7 3.6

Occupational information

pre 15.9 2.8 0.001 16.2 2.4 0.002 15.4 3.5 0.122

post 18.2 3.1 18.7 3.0 17.3 3.1

Goal selection pre 18.4 3.1 0.507 18.9 3.4 0.794 17.6 2.5 0.438

post 18.8 2.6 19.1 2.4 18.3 3.0

Planning pre 17.0 2.9 0.002 17.8 2.7 0.048 15.4 2.6 0.009

post 19.2 3.1 19.3 3.0 18.9 3.4

Problem solving

pre 17.1 3.0 0.016 17.6 3.4 0.162 16.1 1.9 0.024

post 18.3 2.5 18.5 2.8 18.1 1.9

Source(s): Yang, Coleman, Das and Hawkins, 2013

Aim1. To enable more students in Stage 3 courses at the School of

Medical Sciences to be engaged in a career development learning process in order to develop career awareness, employability and professional skills.

2. To perform a controlled study in large sample populations.

1. There would be no improvement in CDMSE scales within individual courses, pre- and post-course.

2. There would be no difference in variation of CDMSE scales in CDL intervention or control group.

Null Hypotheses

CDL intervention

CDLtutorials

personal/group counselling

Professional knowledge and skills

CDMSE survey

CDL assessm

ent

Career and reflective ePortfolio

Learning centred ePortfolio

CDMSE Survey

Gathering occupational information: 1, 10, 15, 19, 23 Goal selection: 2, 6, 11, 16, 20 Planning: 3, 7, 12, 21, 24 Self-appraisal: 5, 9, 14, 18, 22 Problem solving: 4, 8, 13, 17, 25

(Betz & Taylor, 1983; Scott & Ciani, 2008)

2013 Project Design

Pre/post-course CDMSE Survey

CDL intervention(n=135)

ANAT3212 (n=11)

PATH3208(n=37)

PHAR3202

(n=35)

PHAR3101

(n=52)

Non-CDL intervention control

NEOR3221(n=30)

Results 1: CDMSE scales within individual courses

occupational info

goal selection

planning

self appraisal

problem solving

0 5 10 15 20 25

post-course pre-course

ANAT3212 (n=11)

occupational info

goal selection

planning

self appraisal

problem solving

0 5 10 15 20 25

post-course pre-course

PATH3208 (n=37)

occupational info

goal selection

planning

self appraisal

problem solving

16 16.5 17 17.5 18 18.5 19 19.5

post-course pre-course

PHAR3101 (n=52)

occupational info

goal selection

planning

self appraisal

problem solving

0 5 10 15 20 25

post-course pre-course

PHAR3202 (n=35)

occupational info

goal selection

planning

self appraisal

problem solving

15 15.5 16 16.5 17 17.5 18 18.5 19

post-course pre-course

NUER3221 (n=30)Control group

* p<0.05, ** p<0.01

**

Result 2: CDMSE scales obtained pre- and post-course for CDL intervention (n=135) and control (n=30) groups

**

* p<0.05, ** p<0.01

Conclusion

• The introduction of this pedagogy and model has proved successful in evidencing graduate learning outcomes for improving student confidence in CDMSE.

• This study supports further extension of this approach

to senior undergraduate courses at UNSW and other universities.

• Using ePortfolio to deliver CDL will result in directive and structural reflection by learners.

4:00- 4:20pm Q & Aworkshop: leaders and facilitators

4:20-4:30pmPatsie: Summary of Workshop Outcomes

Acknowledgements and OutcomesUNSW Australia funding• 2015 : SEF#4 1x LNT Innovation and Development• 2016 : SEF#3 UNSW Teaching Fellowship

Presentations• 2015 Yang et al, MedEd Forum, Faculty of Medicine, Dec• 2015 Yang, Jia-Lin, and Patsie Polly. ‘Career development learning across year 3 science courses – Pedagogy and

curriculum: Improving student self-efficacy in employability skills’ 22nd Learners Conference, University of St Paulo, Madrid.

• 2016 Polly et al, AAEEBL International ePortfolio Conference, Boston, US, Jul• 2016 Polly et al, ePIC, International ePortfolio and Identity Conference, Invited Panelist, Invited Speaker, Bologna, Oct

Published Journal • Naumann…Polly et al, (2016). Virtual Patient Consultations and the use of ePortfolios to support student learning of

integrated professional skills. (Focus on Health Professional Education, under final review). • J.L. Yang, P. Polly, T. Fath, N. Jones, J. Power (2016). ISA Model and Integrative Career Development Learning in Year 3

Science Courses. The International Journal of Science, Mathematics and Technology Learning (Vol 23, Issue, 3, p33-46).

Book Chapters • P. Polly, et al (2016). ePortfolios, Assessment and Professional Skills in the Medical Sciences at UNSW Australia. In

ePortfolios in Australian Universities. Springer.• P. Polly, et al (2015). Creative teaching, learning and assessment in Medical Science: ePortfolios to support skills

development in scientists beyond just knowing their own discipline content. Capturing Creativity through Creative teaching. The Learner Series.