Embed Size (px)
Citation preview
Institute of Teaching and Learning – Academic partnering to promote excellence in learning and teaching
Designing Assessments (in Engineering)
Stuart Palmer
http://commons.wikimedia.org/wiki/File:Ready_for_final_exam_at_Norwegian_University_of_Science_and_Technology.jpg
#deakeng
Institute of Teaching and Learning – Academic partnering to promote excellence in learning and teaching
#deakeng
SETU 'assessment' items (T2 2010)SETU item Mean S.D. Agr. Dis. Diff.
Uni 3. The workload in this unit was manageable. 3.96 0.92 77.6 7.7 70.0
4. Requirements in this unit were clear. 3.91 1.04 74.7 11.1 63.6
5. The teaching staff gave me helpful feedback. 3.90 1.06 71.7 10.6 61.1
Fac 3. The workload in this unit was manageable. 3.94 0.94 76.3 7.9 68.3
4. Requirements in this unit were clear. 3.90 1.06 73.7 11.4 62.3
5. The teaching staff gave me helpful feedback. 3.81 1.12 67.7 13.2 54.5
Eng 3. The workload in this unit was manageable. 3.74 1.01 68.4 11.5 56.9
4. Requirements for this unit were clear. 3.74 1.12 68.2 14.1 54.1
5. The teaching staff gave me helpful feedback. 3.47 1.27 56.0 22.8 33.2
Institute of Teaching and Learning – Academic partnering to promote excellence in learning and teaching
#deakeng
Institute of Teaching and Learning – Academic partnering to promote excellence in learning and teaching
#deakeng
Five contemporary assessment issues in higher education
?http://commons.wikimedia.org/w/index.php?title=File:Thomas_Young-An_Essay_on_the_Cohesion_of_Fluids.pdf&page=1&uselang=en-gb
Institute of Teaching and Learning – Academic partnering to promote excellence in learning and teaching
#deakeng
Five contemporary assessment issues in higher education
1. Capturing the potential of online assessment
2. Designing efficient and effective assessment for large classes
3. Responding to plagiarism and developing policies to foster academic honesty
4. Using assessment to guide effective group work
5. Recognising the needs of students unfamiliar with assessment practices in Australian higher education
Institute of Teaching and Learning – Academic partnering to promote excellence in learning and teaching
#deakeng
Differing perspectives on assessment
James, R., McInnis, C. & Devlin, M. (2002). Assessing Learning in Australian Universities. Melbourne: CSHE. Page 8.
Institute of Teaching and Learning – Academic partnering to promote excellence in learning and teaching
#deakeng
Purpose(s) of assessment
?http://commons.wikimedia.org/wiki/File:Belt_Problem_v2.svg?uselang=en-gb
Institute of Teaching and Learning – Academic partnering to promote excellence in learning and teaching
#deakeng
Purpose(s) of assessment – broadly:
Formative (for learning)•guide and encourage effective approaches to learning•assist students to learn•ensuring students can identify high quality work•provide feedback on student performance
Summative (of learning)•validly and reliably measure expected learning outcomes•define and protect academic and professional standards
Institute of Teaching and Learning – Academic partnering to promote excellence in learning and teaching
#deakeng
Where does assessment fit in?
Jolly, L. (2001). Graduate Attributes Fact Sheet 1.10 Implementing Graduate Attributes. Brisbane: University of Queensland, The Value Added Career Start Program. Page 3.
Institute of Teaching and Learning – Academic partnering to promote excellence in learning and teaching
#deakeng
Program mappingBachelor of Engineering – Mechanical major
Year 1 SEB121 SEP101 SIT199 SED102 SEE103 SEM111 SIT194 SIT172
Year 2 SIT294 SEM212 SEM218 Elective SEB323 SEM223 SED202 SEM314
Year 3 SEE321 SEM322 SED302 Elective SEB322 SEE434 SEM422 SEM327
Year 4 SEJ441 SEM405 SEM413 Elective SEJ442 SEB421 SED402 Elective
SEP490
Written communication Mechanical design Problem solving
Program objectives
Institute of Teaching and Learning – Academic partnering to promote excellence in learning and teaching
#deakeng
Bloom's taxonomy
Knowledge
Understanding
Application
Analysis
Synthesis
Evaluation
RecognitionRecallFactsSubject matter
Grasp meaningOrder and groupInfer causesTranslate context
Use informationUse equipmentSolve problemsUse methods
Discern patternsOrganise partsIdentify partsSimplify
Combine ideas to create an original productCreative thinking
DiscriminateThink criticallyMake judgementArgue
DefineDescribeFindTabulate
ClassifyCompareExplainSummarise
CalculateDemonstrateEstimateReport
AnalyseCorrelateQuantifyInterpret
AdaptDesignFormulateHypothesise
ArgueChooseJustifyRecommend
?http://www.tedi.uq.edu.au/downloads/bloom.pdf
Institute of Teaching and Learning – Academic partnering to promote excellence in learning and teaching
#deakeng
Learning objectives in program core units
KnowledgeUnderstandingApplicationAnalysisSynthesisEvaluation
Institute of Teaching and Learning – Academic partnering to promote excellence in learning and teaching
#deakeng
MAE406 – Business and Financial EconometricsLearning objectives
On completion of this unit students should be able to:
1. Link the economic theory and real world data and establish a proper econometric model based on the economic and financial concepts;
2. Estimate an econometric model with the most ideal methodology;
3. Interpret the estimated results; and4. Formulate policies based on results
obtained from the econometric analysis.
Assessment name Weight Due date2 Brief
description1Linked learning objective(s)
Test(s) 20% Refer to DSOShort questions covering earlier chapters in the textbook.
1,2
Assignment 30% 13 May 20113000 words – econometric analysis and modelling
1-3
Examination 50% Examination period
It is your responsibility to determine the date, time and location for the examination from Student Connect. Three-hour; closed-book
1-4
Institute of Teaching and Learning – Academic partnering to promote excellence in learning and teaching
#deakeng
Eng. PBL unitfrom Aus uni.
Institute of Teaching and Learning – Academic partnering to promote excellence in learning and teaching
#deakeng
Assessment criteria / Rubrics
From Deakin’s Assessment procedure:“… a tool used to articulate expected standards at various levels of performance in assessment. The criteria for each level of performance (from high distinction through to fail) are explicitly described in clear language to assist students to understand and interpret these descriptions.”
Institute of Teaching and Learning – Academic partnering to promote excellence in learning and teaching
#deakeng
Civil Engrubric
http://www.adelaide.edu.au/erga/publications/maier_2ndyr.pdf
Institute of Teaching and Learning – Academic partnering to promote excellence in learning and teaching
#deakeng
In summary …
Jolly, L. (2001). Graduate Attributes Fact Sheet 1.10 Implementing Graduate Attributes. Brisbane: University of Queensland, The Value Added Career Start Program. Page 3.
Institute of Teaching and Learning – Academic partnering to promote excellence in learning and teaching
#deakeng
Continuous formative assessment
Put existing weekly problem sets and/or review questions on DSO as online quizzes
Computer can ‘mark’ and provide student feedback
Student usage and scores are recorded
Could be ‘taken into account’ if student requests or needs any form of special consideration later
Institute of Teaching and Learning – Academic partnering to promote excellence in learning and teaching
#deakeng
Self-and-peerassessment
http://spark.uts.edu.au
http://tectra.it.uts.edu.au
Institute of Teaching and Learning – Academic partnering to promote excellence in learning and teaching
#deakeng
Multi-choice quizzes
HNN112 - Principles of Nursing CareContentThis unit is the first of two in the Clinical Studies in Nursing: Year 1 cluster. In this unit the notions of person-centred care, the role of nursing in the interdisciplinary team, evidence-based practice, and quality improvement approaches in health care are explored. To achieve this, opportunities are provided for students to explore the role of nurses in promoting healing environments for patients and their families. Key concepts to be addressed include assessment and management of patient function and risk, and promotion of patient comfort. An orienting framework will be provided to enable students to understand nursing skill development throughout the curriculum. Students will have the opportunity to further develop and integrate knowledge and skills learned during a one (1) week clinical placement.AssessmentOnline tests 3 x 20 minute (equivalent 1000 words) 40%, examination (2 hours) 60%, laboratory skills (equivalent 500 words) hurdle requirement, clinical assessment: 40 hours pass/fail
http://www.dreamstime.com/item.php?imageid=7299038
Institute of Teaching and Learning – Academic partnering to promote excellence in learning and teaching
#deakeng
Project-basedlearning
Typically involves:• solution of a problem or completion of a task requiring students to complete a number
of educational activities that drive learning;• generally, students work in teams to complete a project;• the project is non-trivial and often multidisciplinary in nature, requiring work over an
extended period of time;• normally, the project involves the development of a concrete artefact – a design, a
model, a thesis, a computer simulation, etc.;• the culmination of the project is often a written report and/or oral presentation
describing the project methods and the final product; and• teaching staff take an advisory rather than authoritarian role.
Institute of Teaching and Learning – Academic partnering to promote excellence in learning and teaching
#deakeng
Student portfolios
1. Previously used hardcopy portfolio?2. Previously used electronic portfolio?3. Aware of the concept of graduate attributes?4. Aware that Engineers Australia has a list of graduate attributes?
5. Aware that Deakin University has a list of graduate attributes?6. Link between study & assessment, and development of GAs?7. Understand the purpose of a student professional portfolio?
14.6% 14.6%
45.8%
52.1%
33.3%
66.7%
43.8%
100.0% 96.0% 94.0% 94.0% 96.0%
0%
20%
40%
60%
80%
100%
1 2 3 4 5 6 7
% A
gre
em
en
t
Pre-test
Post-test
Institute of Teaching and Learning – Academic partnering to promote excellence in learning and teaching
#deakeng
ERA 'A' journal publications
Palmer, S., Holt, D. and Bray, S. (2008), Does the discussion help? The impact of a formally assessed online discussion on final student results, British Journal of Educational Technology, v39, n5, pp. 847-858.
Palmer, S., Bray, S. and Hall W. (2008), What is the on-campus experience? Engineering student study and work, International Journal of Engineering Education, v24, n4, pp. 802-810.
Palmer, S. (2004), Authenticity in assessment: reflecting undergraduate study and professional practice, European Journal of Engineering Education, v29, n2, pp. 193-202.
Palmer, S. (2002), An Evaluation of Undergraduate Engineering Management Studies, International Journal of Engineering Education, v18, n3, pp 321-330.
Palmer, S. (2001), Engineering flexible teaching and learning in engineering education, European Journal of Engineering Education, v26, n1, pp 1-13.
Institute of Teaching and Learning – Academic partnering to promote excellence in learning and teaching
#deakeng
ERA 'A' Eng Ed publication opportunities
A* Journal of Engineering Education
A European Journal of Engineering Education
A International Journal of Engineering Education
A IEEE Frontiers in Education conference
A SEFI Research on Engineering Education Symposium
Institute of Teaching and Learning – Academic partnering to promote excellence in learning and teaching
#deakeng
STALGS 'assessment' projects
Enhancing student engagement: strategies to improve student-staff interactions with particular reference to assessment feedback
Fair Assessment and Reflective Learning: A cross-faculty online self-and-peer-assessment tool for teamwork assignments piloted in architecture and business communications
Developing a framework for discipline-contextualised graduate attributes in the professional field of engineering: enhancing student achievement of Deakin's graduate attributes
Innovative, relevant and responsive assessment practices in higher education
https://staff.deakin.edu.au/teaching-learning/grants/stalgs.php
Institute of Teaching and Learning – Academic partnering to promote excellence in learning and teaching
#deakeng
Assessment resources
ITL PD modules:http://www.deakin.edu.au/itl/pd/tl-modules/assessment/index.php
ATN assessment conference:http://www.iml.uts.edu.au/atnassessment/index.html
CSHE report:http://www.cshe.unimelb.edu.au/assessinglearning/index.html
Boud / ALTC / UTS assessment futures project:http://www.iml.uts.edu.au/assessment-futures/
The Higher Education Academy Engineering Subject Centrehttp://www.engsc.ac.uk/home
Institute of Teaching and Learning – Academic partnering to promote excellence in learning and teaching
#deakeng
Thank you for your time
