Competency Assessment: Designing Timely, Practical, and Holistic Assessments at Ontario Colleges

October 20, 2008

CIITE Overview

Colleges Integrating Immigrants to Employment (CIITE)

Vision: Internationally trained immigrants have access to programs and services in the Ontario college system that build on their qualifications and expedite their securing employment commensurate with their skills and knowledge

Objective: To improve the support structures and services colleges provide to skilled immigrants.

Focus: The development and implementation of systemic solutions to address barriers to immigrants’ rapid entry into the workplace.

Result: Improved services for skilled immigrants

Supported by the Ministry of Citizenship and Immigration of the Government of Ontario

Developed, led, and implemented by Ontario colleges

A multi-phased strategy for modifying and refining college systems and programs

CIITE Overview

CIITE History

Phase 1: December 2003 – December 2004: Identify barriers ITIs face in

the Ontario college system

Phase 2: September 2005 – December 2007: Develop and test selected

immigrant focused practices and services in the college system

Phase 3.1: April 2008 - March 2009: Implement immigrant focused practices

and services in the Ontario college system

Phase 3.1 Projects

April 2008 - March 2009: Project Implementation o Pilot Credential Recognition (REE)o Implement ITI Advising Serviceso Implement Language Benchmarking Training o Prepare Competency Assessment Models for Rollouto ITI Data Collectiono Develop and Implement Self-Assessment and Upgrade Employment Serviceso Develop and deliver Cultural Intelligence Training and Change Managemento Flexible/Modular Delivery Working Group

For more information on CIITE Projects visit www.ciite.ca

Competency Assessment - Addressing the Gap

CIITE Phase 2 identified the need for a consistent and standardized approach in order to maximize the recognition of education and experience and competency assessment was consequently funded as an additional project in Phase 2 (2007).

What does CIITE mean by Competency Assessment?

Competency assessment is a process whereby the competencies of an individual are

evaluated in relation to a particular occupation, task, or process (i.e., “what does a person

know and can they apply their knowledge and skills to a particular standard?”).

Competency assessment may be used for two purposes. The first is the assessment of prior

learning to determine which components of a particular college program are lacking for

graduation (competency assessment for education). The second potential application is the

provision of a credential to accompany formal degrees, diplomas, and certificates which

attests to practical abilities (competency assessment for employment).

What does CIITE mean by Competency Assessment? (cont’d)

The CIITE Competency Assessment Project design was based on two key assumptions of what is required to conduct viable, sustainable assessments of skills and experience in Ontario colleges.

First, college competency assessments should be replicable across the Ontario college system so that ITIs can access assessment accordingly, and colleges can affordably offer appropriate assessment.

Second, competency assessments should be based on program and professional requirements.

Distinguishing PLAR and Competency Assessment

PLAR

Competency Assessment

PurposeIdentify, assess, and recognize skills, knowledge, and professional behaviour that cannot be assessed in formal credentials based on the learning outcomes of college courses

Identify, assess, and recognize skills, knowledge, and professional behaviour based on the learning outcomes of college programs and professional requirements

Demonstrate a candidate’s capability for professional employment in Canada (regulated professions)

End resultReceive a grade for a college course. Grades for PLAR courses are included on college transcripts towards the achievement of a college diploma/degree

Receive credit for a college program or, if all competencies are demonstrated, be given a “Letter of Achievement”

Identify any skills, knowledge, and/or professional behaviour that are useful for employment

Identify any skills, knowledge, and/or professional behaviour that require further development

Competency Assessment Project

The Competency Assessment Project was funded near the end of CIITE Phase 2 (2007).

CIITE is currently in Phase 3, which began in April 2008.

• Phase 2 Competency Assessment :

• Project Timeline: February 2007 – December 2007

• Participants: Algonquin, Boreal, CAPLA, Centennial, Conestoga, CRTO, Fanshawe, George Brown, Humber, La Cité, Niagara, OACETT, Seneca, St Clair, and Sheridan

• Key deliverable:– Develop outlines of models that will assess the knowledge and skills of ITIs by

program rather than on a course-by-course basis in Respiratory Therapy (regulated) and Mechanical Engineering Technologist (unregulated)

Results of Phase 2 Competency Assessment

Results: A Competency Assessment model (see handout) and recommendations resulting from project work and research. The recommendations are listed below:

1. Pilot the design of two competency assessment models in a regulated and unregulated profession.

Competency assessment models have been designed for both a regulated (Respiratory Therapy) and unregulated (Mechanical Engineering Technologist) profession. In order to prepare to pilot these models, colleges need to validate a List of Competencies for two pilots and develop a project plan for implementation of the pilots in selected colleges. Preliminary work has also been done to establish the List of Competencies to be tested in a pilot .

 

2. Increase the ability of colleges to deliver flexible programs.

Competency assessment allows an ITI to demonstrate their skills and knowledge in a holistic way, in order to recognize these skills and knowledge and to make informed decisions about areas that require gap training. If competency assessment is to be successful in Ontario colleges, colleges must be able to deliver their programs in a flexible manner that allows an ITI to take only the courses s/he needs to meet program and/or professional standards.  

Phase 2 Results Cont’d.

3. Develop college capacity to provide methods of recognition that correspond to flexible programming.

Colleges currently have residency requirements that require a certain percentage of credits be taken at a college in order for that institution to grant a credential. These requirements are a barrier to ITIs who only require specific areas of development and may not wish to enrol in a college program. Colleges should re-examine residency requirements in order to provide a credential to ITIs who successfully complete a competency assessment.

4. Examine the opportunity to develop regional centres of assessment.• Competency assessment is an expensive process and needs to be sustainable. Rather than preparing all

colleges to complete competency assessments in each profession, regional centres of assessment should be designated. These could be coordinated through a central competency assessment office that would provide both colleges and candidates with a central point of administration. Assessment should be available in both official languages.

•  

5. Colleges undergo discussions at all levels (faculty, management, etc.) with regard to the issues, implications, and opportunities for competency assessment in Ontario colleges.

• The implementation of competency assessment in the Ontario college system provides se. Colleges need to engage in discussion in order to make informed decisions on the adoption of competency assessment.

Phase 3.1 Overview

The competency assessment activity in Phase 3.1 is developing, validating

and piloting the model in two occupations: Respiratory Therapy (RT) and

Mechanical Engineering Technology (MET). Teams of experts in each

profession have developed the professions’ ‘List of Competencies’ to be

tested, as well as assessment tools. These will be validated by employers,

regulatory bodies, and/or professional associations before being piloted in the

colleges in the last three months of Phase 3.1.

Phase 3.1 Participants

MET – Algonquin, Conestoga, Fanshawe, George Brown, Georgian, Humber, St. Clair, Sheridan and OACETT in an advisory capacity

RT – Algonquin, La Cité, Fanshawe and the CRTO in an advisory capacity

Activity Objectives

o Validate standards for assessment in Respiratory Therapy (RT) and Mechanical Engineering Technology (MET)

o Create assessment tools for competency assessment in RT and METo Develop workplan and prepare for pilots at 3 colleges in RT and 8 in METo Establish a Working Group that will ultimately develop an implementation strategy. •  

In the last 3 months:o Launch on a pilot basis the competency assessments in up to 6 colleges for each of

RT and MET.o Implement strong measurement and evaluation processes and systems, including full

tracking of activity, costs and outcomes.

Overview of RT Process

Update on Respiratory Therapy Competency Assessment

The Respiratory Therapy project has worked closely with the regulatory body, the Council of Respiratory Therapists of Ontario (CRTO) in ensuring that the checklist of competencies meets the most up-to-date regulations and policies.

Currently, the team is developing the assessment tools. The assessment will have three levels of testing: written, simulated, and clinical. A candidate has to be successful at each level of simulation in order to move on to the next.

The pilot is set to begin in January 2009.

Challenges in Respiratory Therapy

o Challenge: Success of candidates - As the competency assessment aims to test the qualifications necessary to practice Respiratory Therapy(RT) in Ontario there is concern about the success of candidates. Respiratory Therapy is primarily a North American profession and many of the internationally trained applicants will be individuals who are switching professions (e.g. Anesthesiology Assistants, Intensive Care specialist, etc.). There is concern that there won’t be many individuals in the pilot who pass the written test. Action: Searching for pilot participants with relevant experience

o Challenge: Cost of assessment for candidate – A general concern that should be kept in mind when developing the competency assessment is the direct financial costs of taking the assessment, the cost of travel and having to take time off work, securing child care, etc.

Update on Mechanical Engineering Technology

o The team has determined 14 competencies which cover the skills and

knowledge that are fundamental for an MET. OACETT has recognized

the competencies as valid.

o The team is currently in the process of developing the assessment

tools to be piloted in January 2009.

Competency Assessment Process

Stage 1Information/Advising

Stage 2Preparation of Required Documents

Stage 3 Submission of Required Documents

Stage 4Analysis

of EducationStage 5

Develop a Plan

Stage 6Competency Assessment

•Candidate acquires information on applying for credit for previous learning and/or college competency assessment through websites, information packages, or advising services

Candidate prepares all relevant evidence requirements

Candidate submits all necessary documents to the college, which may include: resume, supporting documents (evaluated transcripts, course outlines, certificates, licenses, etc.) and language test scores.

The information is placed into the Record of Education and Experience (REE).

•A knowledgeable Assessor is appointed by the college.

•Assessor examines the information in the REE.

•Candidate meets with Assessor and they determine a plan of assessment for competencies where there is lack of documentation (no course outline or skill learned on the job).

•Candidate completes required competency assessments

Demonstrate the knowledge of Statics, Dynamics & Strength of

Material

MechanicalEngineeringTechnology

11

22

33

Demonstrate the ability to create solid models & assemble them using CAD software

77

99

Demonstrate the knowledge of Fluid mechanics & Thermodynamics

66

88

1010

Demonstrate the knowledge of basic computer skills

Demonstrate communication skills and personal skills

Demonstrate the knowledge of Statistics and Quality Control

ManufacturingProcesses

QualityControl

Electric/ElectronicFundamentals

Demonstrate the knowledge of Hydraulics & Pneumatics

44

55

1111

1212

1313

1414

Communication(English/French)

Engineering Drawing- Theory

CAD Software Skills- Solid Modeling

Fluid /Thermodynamics

Engineering Materials

Solid Mechanics

Machine ShopPractices

Hydraulics/Pneumatics

Machine Design

Demonstrate the knowledge of mathematics (up to Basic Calculus)

Understand & Interpret CSA engineering drawing

Demonstrate the knowledge of manufacturing processes

Demonstrate the knowledge of engineering materials

Apply the knowledge of machines & tools and use them to machine a part

Demonstrate the knowledge of electrical/electronic fundamentals

Demonstrate the knowledge of Machine design

Mathematics

Computer Skills& Applications

Outcomes of Competency Assessment MET

• The LOA will detail which skills were demonstrated and how and refer the candidate to further training to fill gaps

• (Depending on the gaps, ITIs may be eligible to earn a diploma at the college. For example, if ITIs have to take enough courses to meet the residency requirement at the college, they may be eligible for a diploma.)

Referral

Competency

AssessmentLetter of

Achievement

Statement of Equivalence

• In the LoA it will state that the candidate’s skills and knowledge are equivalent to the competencies a student would acquire in a college Mechanical Engineering Technologist (MET) program.

• CIITE is currently in discussion with OACETT to ensure the individuals who successfully complete the competency assessment are eligible for OACETT certification.

Challenges in MET

Length of the assessment – The assessment should be as timely as possible. If an individual has to take the entire competency assessment( such as in the case of a refugee unable to produce transcripts or course outlines) they would currently have to undergo approximately 40 hours of assessment. Action: The team is currently revising the assessments to make this time shorter, while maintaining the quality of the assessment.

Cost of assessment for candidate – A general concern that should be kept in mind when developing the competency assessment is the direct financial costs of taking the assessment, the cost of travel and having to take time off work, securing child care, etc.

Success of the candidate – There is a concern that METs who have been in the workforce for many years won’t be able to remember some of the competencies learned in a college program. Action: The team is developing the assessment questions to be as “high level” as possible to help alleviate this issue.

College Program Learning Outcomes verses On-the-Job Acquired Competencies – The competency assessment will test outcomes of the college program/ competencies needed for certification from OACETT. It will not test all of the skills and knowledge someone would develop in the workforce over a number of years. Should we test mid-career and senior-career skills? The next phase of development could include direct involvement with industry to develop job competencies.

Discussion questions

o Should there be separate “list of competencies” for ITIs with mid/senior level experience?

o What are the implications for roll-out of competency assessment in Ontario colleges (cost, access to qualified Assessors, tools)? Is there opportunity to collaborate with other organizations to improve access to competency assessment for ITIs?

Contact: Rebecca Carnevale - Project ManagerCarnevale@collegeconnect.on.caT: 416.351.7530 x. 3200