Final
Integrated
Summative
Assessment Dr Linda Meyer
Presentation for the
SABPP
Background to the Final
Integrated Summative
Assessment
Current vs. QCTO requirements,
Historic Experience & Context,
SETAs, Professional Bodies /
Statutory Bodies (EAAB - PDE)
Future, Provider vs. Assessment
Quality Partners (AQP)
requirements.
SABPP – FISA
FISA Requirement -contained in the SABPP ETQA Assessment And Moderation Policy.
“In the case of full HR. qualifications (including HR. Learnerships), the assessment cycle also includes a Final Integrated Summative Assessment (FISA) against the exit level outcomes of the HR. qualification”.
Occupational Qualifications
Occupational qualifications are a
feature of the revised NQF and are
designed to address skills needs in
the labour market.
They will replace legacy
occupational qualifications such as
those for the trades and work-
focused unit standards-based qualifications.
Assessment is defined as..
A structured process for gathering
evidence and making judgments about an individual’s performance in
relation to registered national
standards and qualifications (SAQA,
2001: 16).
Assessments
• Formative assessment is “assessment that takes place during the process of learning and teaching”, with the purpose of supporting learning. (SAQA, 2001: 26)
Formative
• Summative assessment is used to make a “judgement about [learner] achievement” that is used at a particular point (usually at the end) of a learning programme to measure progress in terms of the requirements of national standards and qualifications so that credits can be awarded. (SAQA, 2001: 26)
Summative
• A form of assessment which permits the learner to demonstrate applied competence and which uses a range of formative and summative assessment methods. NSB Regulations (SA, 1998:4)
Integrated
Integrated Assessment:
Integrated assessment is meaningful if there
are clear relationships between the
purpose statement, exit level outcomes
and integrated assessment of this
Qualification.
In addition to the competence assessed to
achieve the Unit Standards:
learners must demonstrate that they can
achieve the outcomes in an integrated manner.
Integrated Assessment
… should [assess] the ability to
combine key foundational, practical
and reflexive competence with some
critical cross-field outcomes and
apply these in a practical context
for a defined purpose. The context
should be relevant to real life
application (SAQA/CIDA, 2003: 62).
Guidelines for Integrated
Assessment
Assessment approaches must confirm a
learner’s ability to integrate knowledge.
Assessments must focus on a learners’
ability to demonstrate applied
knowledge (applied competence).
According to the NQF - evidence of
applied competence is the learners’
ability to integrate concepts, ideas and
actions in authentic, real-life contexts.
Guidelines for Integrated Assessment Assessing a number of outcomes
together;
Assessing a number of assessment criteria
together;
Assessing a number of unit standards
together;
Using a combination of assessment
methods and instruments for an
outcome/outcomes;
Guidelines for Integrated
Assessment
Collecting naturally occurring
evidence (such as in a workplace
setting);
Acquiring evidence from other
sources such as supervisor’s reports,
testimonials, portfolios of work
previously done, logbooks,
journals, etc. (SAQA, 2001: 55).
Guidelines for Integrated
Assessment
Integrated assessment should offer an opportunity to demonstrate the depth and breadth of learning at all stages and in a variety of ways throughout the learning programme.
Assessments are important moments in the course of learning programmes.
Guidelines for Integrated
Assessment
“However, educators should guard
against over-assessment where each
outcome “(or worse, each assessment
criterion) [is assessed separately
resulting in] hundreds of little fragmented
meaningless assessments of the check-
list type, taking up valuable learner and
educator time without anything of value
being learnt” (LGWSETA, 2004: 13).
Under the QCTO system ..
Key quality assurance processes are
articulated as:
Monitoring and support of learner
progress during programme
implementation ; and
Final integrated summative
assessment of learners for
occupational competence.
Final Integrated Summative
Assessment
The FISA refers to the process of making judgments about achievement on a qualification/learnership.
This is carried out when a learner is ready to be assessed at the end of a learnership or full qualification.
The FISA must be set by a constituent assessor and moderated by a constituent moderator.
Final Integrated Summative
Assessment
FISA should not weigh more than
continuous assessment of
providers,
The objective for the FISA is to
confirm the standard across providers and across qualifications
- STANDARDISATION
Final Integrated Summative
Assessment
Database of sample assessment
instruments & tasks are included,
Examples of good integrated
assessments are combined,
Formats are designed for recording
evidence e.g. templates, log-
books.
Final Integrated Summative
Assessment
A FISA for the qualification focuses on the extent to which a learner can demonstrate applied competence.
Applied competence, in terms of the NQF is evidenced through learner’s ability to integrate concepts, ideas and actions in authentic, real-life contexts and is expressed as practical, foundational and reflexive competence.
Final Integrated Summative
Assessment Practical competence - demonstrated
ability to perform a set of tasks and actions in authentic contexts;
Foundational competence - demonstrated understanding of what we are doing and why we are doing it;
Reflexive competence - demonstrated ability to integrate our performances with our understanding so that we are able to adapt to changed circumstances and explain the reason behind these adaptations.
Final Integrated Summative
Assessment More than one FISA instrument should be
developed.
The moderator must ensure that the assessment instrument is aligned with the purpose and rationale of the qualification and that it complies with the applicable SAQA level descriptor.
Draft assessment instruments must fall within the ambit and in accordance with the guideline document.
Blooms and SAQA level descriptors considered.
Bloom’s Taxonomy 1. Knowledge: remembering of previously learned
material; recall (facts or whole theories); bringing to mind.
Terms: defines, describes, identifies, lists, matches, names.
2. Comprehension: grasping the meaning of material; interpreting (explaining or summarizing); predicting outcome and effects (estimating future trends).
Terms: convert, defend, distinguish, estimate, explain, generalize, rewrite.
3. Application: ability to use learned material in a new situation; apply rules, laws, methods, theories.
Terms: changes, computes, demonstrates, operates, shows, uses, solves.
Bloom’s Taxonomy 4. Analysis questions (taking apart the known)
a. use - graph, survey, diagram, chart, questionnaire, report.... b. observed behavior - classify, categorize, dissect, advertise, survey.
5. Synthesis (putting things together in another way) a. use - article, radio show, video, puppet show, inventions, poetry, short story... b. observed behavior - combine, invent, compose, hypothesis, create, produce, write.
6. Evaluation (judging outcomes) a. use - letters, group with discussion panel, court trial, survey, self-evaluation, value, allusions... b. observed behavior - judge, debate, evaluating, editorialize, recommend.
Design FISA
FISA Instrument is aligned;
Multiple versions;
Security;
Fit for Purpose (Blooms, Level descriptors
ect).
Technical specifications;
Validity, Authentic, Current, Reliable and
Sufficient ;
Final Integrated Summative
Assessment
The Moderator must ensure that the FISA instrument is aligned with the purpose and rationale of the qualification, and that it complies with the applicable SAQA level descriptor.
The FISA must be of such a length that a well-prepared learner will be able to answer it comfortably within the time allocated, with reasonable time remaining for revision.
Final Integrated Summative
Assessment Standard of the paper (validity, reliability and
fairness)
Questions of various types e.g. Multiple Choice; Questions, Case studies, paragraphs, data response, essay, etc;
Questions from which candidates are to choose – are they of equal difficulty level;
Correct distribution in terms of cognitive levels (Bloom’s Taxonomy);
Overall: how does the standard of the assessment instrument compare in relation to other qualifications/s assessment instruments and previous assessment instruments.
Final Integrated Summative
Assessment
Intellectually challenging and allowing for creative responses from candidates;
Suitability of examples and illustrations;
Relationship between weighting allocation, degree of difficulty and time allocation;
Technical Criteria
Cover page with all relevant details such as time, unit standards and instructions to candidates
Clarity of instructions to candidates
Lay out: learner friendly
Correct numbering
Technical Criteria
Mark/weighting allocation clearly indicated;
Quality of illustrations, graphs, tables etc must be print ready
Complete memorandum with model answers with :
Mark/weight allocation and provision for alternatives
Registered Unit Standards and Qualifications
Relevance to registered Qualification, i.e. unit standards, specific outcomes, assessment criteria
Levels of questions
Coverage of unit standards
Weighting and spread of contents
Conceptual construct
example
What conceptual constructs - The
assessment instrument must deal with:
e.g. – reasoning ability ability to communicate
ability to translate from verbal to symbolic
ability to compare and contrast
ability to see causal relationship
ability to express an argument clearly
Cognitive skills
Are these constructs representative of
the best and latest developments in the training of this knowledge field?
Are the questions challenging and allowing for creative responses from candidates?
Suggested application of cognitive levels for an NQF Level 4 qualification is: 10% knowledge 20% comprehension 40% application 30% analysis, synthesis, evaluation
Language and bias
Correct terminology;
Appropriate language register – for the
level of the learner;
Avoidance of gender, race, cultural,
provincial bias;
Clear and unambiguous specification
of instructions within questions e.g. list,
describe;
Moderation Is there evidence that the paper
has been moderated?
Quality, standard and relevance
of input from moderator.
Alignment Grid
The availability of an alignment grid
showing the alignment of the
instrument with the purpose,
rational, assessment criteria and
NQF level descriptor of the
qualification.
Focus on ELO’s
Overall impression
Fairness of the assessment instrument as
whole,
Will the assessment instrument as a
whole assess the achievement of the
purpose of the qualification and the
exit level outcomes,
Recommendations for improvement,
Monitoring and Review.
FISA construct example:
Knowledge Assessment;
PoE (completed by the learner);
Workplace based project;
Expert Practitioner Panel interview
and presentation;
Focus on Work Integrated Learning
Practice.
Thank you www.cimap.co.za
The Chartered Institute for the Management of Assessment
Practice (CIMAP)
A registered NPO