Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
VC’s L&T Forum 2013
Integration of Research Literacy and Skills in the Undergraduate Curricula
using the RSD Framework: Implementation across Four Different
Disciplines
Dr Jito Vanualailai (Director of Research, Research Office)
Ms Naolah Pitia (MG101 Coordinator, FBE)
Ms Jackie-Anne I Ete (UU204 Assistant Lecturer, FALE)
Dr Sereima Naisilisili (ED153 Coordinator, FALE)
Dr Shaneel Chandra (CH101 Coordinator, FSTE)
Ms Heena Lal (RSD Coordinator, Research Office)
Copyright © Jito Vanualailai, Heena Lal, Naolah Pitia, Jackie-Anne I Ete, Sereima Naisilisili, Shaneel Chandra, The University
of the South Pacific, 2013. Available under Creative Commons Attribution NonCommercial-ShareAlike 3.0 Unported License
Implementation of RSD Framework at USP
Purpose of RSD
To embed research literacy and skills in an explicit, coherent and systematic manner from 100-level to 300-level/ final year courses in undergraduate programmes and also at the postgraduate level.
The RSD Framework
• Five Facets: Refer to research literacy and skills
• Six Levels of Student Autonomy: Autonomy in research
• Research literacy and skills is now the 8th Graduate Attribute.
Link to RSD Framework – simplified version
Timeline for RSD
Implementation at USP
Year 2013 Year 2014 Year 2015
Embed RSD in UU courses
Introduce RSD in 100-
level discipline based
courses
Continue to embed RSD
in selected 100-level
courses
Introduce RSD in 200-
level discipline based
courses
Continue to embed RSD
in selected 200-level
courses
Implement and embed
RSD in core 300-level
courses
Introduce RSD in core
postgraduate courses
Continue to introduce
and embed RSD in core
postgraduate courses
Continue to embed RSD
in core postgraduate
courses
• RSD Framework rolled out at USP in 2012 - ‘UU100 Communication and Information Literacy’ and ‘UU114 English for Academic Purposes’, UU200 Ethics and Governance’. RSD implemented in UU204 in 2013.
What does RSD Implementation in courses
entail?
RSD is integrated in courses using as assessment based approach. This entails revision or modification of assignments and development of marking rubrics underpinned by the RSD framework.
• In this symposium, the RSD marking rubrics developed in three 100-level discipline based courses and one 200-level generic course during Semester 1 2013 and experiences of RSD implementation will be presented by course coordinators.
Courses: MG101 ‘Introduction to Management’, UU204 ‘Pacific Worlds’, ED153 ‘School and Society’, CH101 ‘Chemical Principles’.
RSD Implementation in MG101 Introduction to
Management MG101 Research Assignment Choose only one of the specific area given below to do your research assignment on:
1. Use the research sources you have in your disposal (e.g. literatures, articles, journals, and
book etc.; survey questionnaire, structured or semi structured interviews, case studies
etc.) to collect and compile information/data you require to write a report on the current
developments in terms of progress for minority groups and women in the workplace;
using the following questions as your guidelines:
a. What are some facts on the current developments in terms of progress for minority
groups and women in the workplace? Gather some informational facts/data from
countries like Australia, New Zealand, Fiji and other Pacific Island countries
b. What are the benefits and challenges of diversity at workplace?
c. What are “best” managers doing , in relation to diversity at workplace?
d. What lessons of diversity have you learned from this study?
OR
2. Analyze survey reports (or conduct a survey yourself) on how glass ceiling may affect the
careers of women and minority groups in various organization setting, and draw out
implications of such on the progress of women and other minority groups’ professionally and
personally.
MG101 RSD Assignment Marking Rubric
MG101 Experiences: Benefits and Challenges
Benefits: • Integrated the
assessment of more than one learning outcome.
• Allows for a more systematic marking (marker-friendly, less student complaints).
• Provide students with more understanding of the scope and standard of quality expected of their research assignments.
Challenges:
• Understanding the use of the RSD framework.
• Developing RSD assessment rubric that is simple and relevant to the development of the target group.
• Effective integration of the assessment of other learning outcomes.
• Applying the assessment rubrics to other assignments.
MG101 Recommendations
1. Revision of marking rubrics for simplification to both students and teaching staff.
2. Workshop training at School level:
• Learning outcomes at different levels as per year of study.
• Training on how to design an effective integrated assessment rubric based on RSD framework.
• Understanding the application of the assessment in other assignments or assessment work.
• Monitoring student development and remedial effort needed.
UU204 Letter to the Navigator & Vaka Crew
Assignment 1: Essentially a student’s introduction of him/herself to their online crew/ vaka-mates, mapping themselves, their ‘identities’ in relation to the readings, research and writings they encounter under Unit 1 themes - ‘Naming and Mapping Oceania’.
UU204 RSD Assignment Marking Rubric
Fig 2 & 3: Navigator
Bera Cama and youth
aboard Camakau,
Korova, Suva.
Picture courtesy of
Leota-Ete
UU204 Experiences with RSD
Prior to RSD
• NO additional research was required. Students were restricted to the use of UU204 course material which we provided as well as knowledge of own oral histories/stories.
• NO marking rubrics were available to students. (Except Steven Dutch’s ‘What is an ‘A’?’ guidelines for one assessment)
After RSD
• Oral histories/stories supported by scholarly literature.
• Students learn about source credibility and reliability (Lots of Wikipedia/tourist sites referenced at the beginning of semester).
• Transparent marking processes. Precise, concise, visually appealing rubrics.
UU204: Challenges in RSD Implementation
• Linear
• Phrasing/wording
• Preparation? (Defrosting)
• Connectivity in the region to online library databases.
RSD Implementation in ED153 Education and
Society Assignment 3: Research
Weighting: 15%
Due date: 24th May, 2013
• Carry out a research to find out people’s attitude about integrating Pacific culture in education. Your study may focus on specific aspects of your culture (e.g. cultural values, gendering process, traditional parenting styles, traditional models of discipline, traditional food preservation methods, traditional music/dance, etc.). Are there ways of merging culture and education?
• The skills section in your ED153 Course Book clearly provides the details required for this assignment. Follow the instructions in order to complete the research and your report.
ED153 Background: FALE Student Learning
Support Help in unpacking the question and process into researchable
components
Example
• Q: What are you writing about? Ans: attitudes
• Q: What attitudes?
• Ans: attitudes about integrating culture in education
• Q: Whose attitudes?
• Ans: youths/ parents/ women/ men
• Q: What/Which youths/ parents/ women/ men?
• Ans: Youths etc. in my/your community
• Q. Which aspect of culture?
• Ans: music/dance/values etc.
• Why?
ED153: My experience with RSD
• March symposium at the Holiday Inn
• Research assignment was analyzed against the RSD framework
• 207 students for Face-to-face and Print modes
• Research assignment and rubrics were already given to students
• I had no prior experience with RSD
• So focused on marking rubrics
ED153 Original Marking Rubric for Assignment 3
(15 marks) Aspect Criteria /15m
Introduction No introduction, aim not stated or wrong 0
Vague, introduction fails to describe the background/ outline of work undertaken 1
Aim is accurate and clearly stated adequate description of background/ outlines
of work undertaken
2
Exceptional/clear comprehensive description of aims, background and outline of
work
3
Methodology/
materials
No mention of methods/materials used 0
Some mention of methods/materials used 1
Methods and materials are almost complete and generally accurate 2
Accurate and complete account of methods/materials used 3
Results & Discussion No analysis and results given or totally incorrect 0
Incorrect analysis of data, results are incomplete and vaguely presented 1
Adequate analysis of data, results are accurate, some discussion of results
attempted
2
Analysis of data, results, and discussions are accurate and complete 3
Analysis of data, results and discussions are accurate and complete with
suitable illustrations
6
Conclusion No conclusion or wrong conclusion 0
Conclusion is partly correct but incomplete 2
Conclusion is correct, complete, related to the aim of the research and
suggestions for improvement of research are given
3
ED153 Marking Rubric Reorganised using RSD
Research
feature
Elements Excellent - 4 Very good - 3 Good -2 Unsatisfactory - 1
to 0
MARK
Embark and
clarify
Coverage
Introduction
All terms of reference
met
Detailed analysis
Purpose and
background of study
exceptionally clear ,
supported by literature
Terms of reference
met.
Details and analysis
adequate
Purpose is generally
clear and meaningful
with some literature
Most terms met.
Some
background and
purpose of study
mentioned,
weakly supported
Below standard
Weak with several
missing elements of
background and
purpose
Find and
generate
Methodology
and data
collection
Clearly discussed and
detailed with
appropriate data
Clear with some
appropriate data.
Generally
described lacking
details.
Some appropriate
data
Ineffective
methodology.
Unclear and
inappropriate
Organise and
manage
Appropriate
format
presentation of
results
Well planned, logical
sequence. All major
sections included.
Thorough with clear,
and properly
sequenced
presentation of data &
subheadings
Planned report,
logical sequence ;
major sections
included, data
correctly integrated
in to texts –
meaningful and easy
to follow
Adequate, mostly
clear discussions,
data
interpretations are
quite weak
Data not integrated.
Little organisation
Research
feature
Elements Excellent - 4 Very good - 3 Good -2 Unsatisfactory - 1
to 0
MARK
Analyse and
synthesise
Discussion of
results
Full, clear and
sophisticated.
Accurate
interpretations
well supported
Clear and detailed
discussion of data
with sound
interpretation
Adequate, mostly
clear discussion of
data
Discussion is weak.
Thin interpretations
of data
Evaluate and
reflect
Conclusions Well considered
and strong. Highly
relevant. Well
supported by data
analysis.
Relevant and
mostly clear.
Supported by data
and analysis
Relevant but lacks
depth. Supporting
evidence unclear
Questionable
conclusion.
Lacking support from
data analysis
Communicate
and apply
Grammar
Accurate writing
convention
Sentence fluency
and word choices
of high level
Strong & specific
words used to
convey meaning
Generally uses
appropriate words.
Sentences are
grammatical but
are sometimes stiff
or choppy.
Contains
occasional spelling
errors
Some errors in
grammar and word
choices. Some
distracting spelling
errors that need
proof reading
spelling errors,
punctuation,
typographical errors
–overly distracting
Presentation* +1 mark : assignments must be typed with proper font etc
TOTAL out of 25
ED153 Marking Rubric Reorganised using RSD
ED153: Results
• Gained a better & more holistic view of the features of research
• Redistributed weighting to reflect specific phases & process of research as reflected in the report
• Important features of research were brought to the forefront of the rubric. E.g. communication
• Marks carried more meaning at a glance
• Complemented the comments within the assignments
• Students were generally satisfied, judging from the almost zero rate of complaints about marking
RSD Implementation in CH101 Chemical Principles
CH101 RSD Assignment
The objective of the experiment was to determine the amount of calcium carbonate in eggshells, a routine but very relevant analysis. The goal of this experiment was to, among the learning of the method:
• To provide a "feel" of real time analysis of common food/items to students
• To also impart a sense of multiple analysis, which is a core component of scientific research
• CH101 RSD Marking Rubric for Lab Report
CH101: Student Feedback
Strongly
Agree
Somewhat
Agree
Somewhat
Diasgree
Strongly
Disagree Undecided
1 2 3 4 0
QUESTION Score Rationale
I have a complete understanding of the Rubrics 2 General "feel"
The Rubrics are too demanding for us as first year students 2 Issues such as adjustment
The Rubrics make writing labs easier 2 Are the rubrics helping?
I generally have difficulty in writing scientific reports 2 Baseline for lab writing skills
The Rubrics tell me what the Instructors expect of me in write-ups 1 Are the rubrics guiding?
I have never referred to the Rubrics at all in my Practical report writing 3 Disinclined?
Having Rubrics in labs has resulted in higher marks for me 2 Improvement noted?
I have difficulty in identifying/sourcing the information required by the Rubrics 2 Are the rubrics too complex?
The Rubrics have helped me improve my research and report writing skills to some extent 2 Is any perceived progress evident?
I prefer no Rubrics as my marks used to be higher without Rubrics 3 Failed experiment by instructors?
I can easily satisfy the requirements for the Above Benchmark category in my lab reports 2 A self-review
I do not like Rubrics as all I want is a pass in my practical reports 3 Generally content with pass
I get more valuable feedback when using Rubrics 2 A desire to improve
I can only manage a lab report that satisfies the quality of Benchmark and/or below it 2 Rubrics setting the bar too high?
I love the University style of laboratory classes 2 General enthusiasm - bias?
I need more time to adjust in writing up lab reports using the Rubrics 2 I can do it, but give me time!
The timeframe given (2 weeks) is not sufficient enough for me to closely follow the Rubrics 2 Time constraints?
CH101: Observations and Possibilities
• Previously marks in practical reports ranges between 14-18.5.
• Students appeared to have understood and followed the rubrics quite well and scored between 15-17
– (though there was scope for improvement particularly in the discussion of their results), there were a handful of students who wrote quite brilliantly and the amount of effort they had put in was evident.
• Key places where students still need to build on is critical thinking and discussion of results (justification of their findings with scientific reasoning)
Conclusions and Way Forward
• It can be seen that the RSD framework has been
contextualized to develop marking rubrics for assignments in different disciplines, providing a common platform to assess research literacy and skills across 4 different disciplines.
MG101 ‘Introduction to Management’, UU204 ‘Pacific Worlds’, ED153 ‘School and Society’, CH101 ‘Chemical Principles’.
Conclusions and Way Forward
Benefits/ Advantages reported:
• RSD framework allows for explicit articulation of research components and assessment of research literacy and skills that were not previously assessed in the course. (MG101, UU204 – credibility of sources, ED153 – communication, CH101- objectives and research questions).
• RSD based rubrics allowed for explicit communication of assignment expectations to students. Less student queries noted on marked assignments – MG101.
• More consistent marking across assignments reported when using the RSD based marking rubrics in MG101.
• Use of RSD based marking rubric allowed for a more systematic marking and subsequent reduction in marking time – MG101, possibly in ED153.
• Allowed for assessment of other USP Graduate Outcomes - Communication, Critical Thinking, Creativity. (Refer to Matrix)
Conclusions and Way Forward
Challenges:
• ‘Defrosting’ the research facets to students.
Challenge for students to understand the ‘facet’ expectations, and to relate facets to marking criteria.
• Wording used in rubrics (UU204, MG101).
Conclusions and Way Forward
Recommendations/ Way Forward
• Gradual introduction and assessment of RSD facets from the beginning of the semester (e.g CH101). Allow students time to get familiar with the expectations of RSD rubrics.
• Work on ‘defrosting’ of RSD facets and expectations of marking criteria, for example in tutorials.
• Simplification and fine tuning of RSD marking rubrics (UU204, MG101).
• To plan for further workshops on integration of RSD in rubrics.
• Expand on monitoring and evaluation as implementation progresses.
• Moving forward, use assessments at the 100-level as baseline for developing assessments at 200-level and 300-level to enhance student level of autonomy in research.
THANK-YOU!
Questions and Comments