Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006,

Panel on Competency-based LearningI2LOR2006 Conference, Montreal, Canada

10 November 2006,


10 November 2006,

Demetrios G SampsonAdvanced e-Services for the Knowledge Society Research Unit (A.S.K.)

Informatics & Telematics Institute (I.T.I.)

The Centre of Research and Technology Hellas (C.E.R.T.H.)

http://www.ask4research.info

TenCompetence The European Network for

lifelong competence development

This work is licenced. This work is licenced under the Creative Commons Attribution-NoDerivs-NonCommercial. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nd-nc/1.0/

or send a letter to Creative Commons, 559 Nathan Abbott Way, Stanford, California 94305, USA.


10 November 2006,

Outline (1/2)

• Competence-based Learning: The need for a new generation of Learning Services

• The TenCompetence Approach– Competence Development Lifecycle– The TenC Cycle for Lifelong Competence

Development– TenC Competence Domain Model


10 November 2006,

Outline (2/2)

• WP6 Learning Activities & Units of Learning– TenC key problems tackled in WP6– Overall WP6 Objectives– WP6 Contributions to TenCompetence Domain Model– Overall WP6 Activities

• WG-A (IMS LD Authoring)• WG-B (Assessment Model)• WG-C (IMS LD Run-time)

• WP8 Task 1: Competence Observatory– TenC key problems tackled in WP8 T1– Scope of the TenC Competence Observatory– Targeted User Groups


10 November 2006,

TenCompetence Project facts• Duration: December 2005-November 2009 • An Integrated Project funded under the IST

Programme, Sixth Framework Programme, European Commission

• Estimated Budget: 14MEuro total with 8,5 MEuro public funding

• 13 partners from 9 different countries– Open Universiteit Nederland, University of Bolton,

CERTH, INSEAD, Universitat Hannover, Universitat Pompeu Fabra, Universiteit van Amsterdam, Sofia University

– Software de Base, LogicaCMG, GIUNTI, Stichting SURF, Synergetics


10 November 2006,

Central TenCompetence Problem

• The knowledge society demands continuous competence development and management at the individual-, group- and organizational level

• Competence development and management for individuals, groups and organizations are distinct fields with their own approaches and tooling

• Integrated support for informal and formal learning is missing


10 November 2006,

TenCompetence Goal

TENCompetence will support networks of individuals, groups and organisations in Europe in lifelong competence development by establishing the most appropriate technical and organisational infrastructure, using open source standards-based, sustainable and innovative technology


10 November 2006,

Aspects to be integrated

• Knowledge Resource Sharing & Management

• Learning Activities & Units of Learning

• Competence Development Programmes

• Networks for Lifelong Competence Development


10 November 2006,

Integration Levels


10 November 2006,

The TenCompetence Approach

• Support– Individuals– groups – organizations

• In– Lifelong competence development

• By– Pedagogical Models– Organizational Processes– Technological infrastructure

• Using– Open-source, standards-based technology

[TenCompetence DoW pp. 5, 10]


10 November 2006,

Competence Development Lifecycle

1. Organizational Competence Map

2. Competence Assessment

3. Strategies

Developm

ent4.

Rea

sses

s

Identification of Job Roles and relevant competencies

1. Assess individual’s competencies.

2. Gap Analysis between existing competencies and the required competencies for a specific job role.

Design Competence Development

Programmes to minimize the identified

gaps

Continuous Performance Monitoring and Assessment

to confirm improvement.


10 November 2006,

TenC Cycle for Lifelong Competence Development

CompetenceProgramme

Units ofLearning

KnowledgeResources

populated with

as a synthesis of

OrganizationsNetwork



3. Strategies

Developm

ent

4. R

eass

ess

1. Specify relevant Job Roles2. Define required competencies for each

job role3. Assess the competencies of their

employees4. Define and deliver competence

development programmes5. Monitor their employees performance

CompetenciesDescription

CompetenciesAssessment

GapAnalysis

PerformanceAssessment

design a



3. Strategies

Developm

ent

4. R

eass

ess

IndividualsNetwork

1. Join community of relevant Job Role2. Assess own competencies for the

specific Job Role3. Search/Find relevant Competence

Development Programme4. Share experiences with other members

of the network

execute

WP5

WP6

WP7


10 November 2006,

TenC Core Functional Services

Knowledge Resources

Units of Learning

Competence DevelopmentProgrammes

Networks for Lifelong

CompetenceDevelopment

WP 8

WP 7

WP 6

WP 5

refer to

use

populated with


10 November 2006,

Key role for 'competences'• The concept of competence can bridge the world

of education, training, knowledge management,human resource management & informal learning

• Many definitions, problem for system development• - Competence: Effective performance in a domain

at different levels of proficiency- Competency: Undefined (we prefer to avoid the

term and use more specific terms when needed, eg, a skill)

• We use a stipulative definition and model in TENCompetence to avoid endless discussions about the concept


10 November 2006,

TenC Competence

Domain Model

[Rob Koper, TenC Domain Model v1.0,

19 June 2006]


10 November 2006,

TenC WorkPackages WP 1

WP 2

WP 3

WP 4

WP 10

WP 5

WP 6

WP 7

WP 8

WP 9


10 November 2006,

Issues to be tackled per Work Package

WP Issue3 • integrate isolated models and tools for competence development into a common, easy to use

infrastructure

4 • run pilots to ensure the validity and viability of the approach

5 • develop software for the effective support of users who create, store, use and exchange knowledge resources within a learning network

6 • develop new innovative pedagogical approaches and assessment models for lifelong competence development

• develop software for the effective support of users who create, store, use and exchange learning activities and units of learning within a learning network

7 • develop software for the effective support of users who create, store, use and exchange competence development programmes within a learning network

8 • develop new innovative organisational models for lifelong competence development

9 • deliver training programs to learn users how to work with the infrastructure, and to train instructors and companies (specifically SMEs) to deliver services using the infrastructure

10 • build a growing network of associated partners to ensure large-scale use in Europe


10 November 2006,

Work Package 6

Learning Activities & Units of Learning


10 November 2006,

TenC key problems tackled in WP6

TenC Issue #2: For individuals, groups and organisations in Europe, it is still hard to get an overview of all possible formal and informal knowledge resources, units of learning, programmes and learning networks that are available, and to identify the most appropriate for their needs.

WP6 Specific Objective:Research and develop innovative, standards-based methods and tools for the exchange and quality rating of formal and informal learning activities and units of learning.

TenC Issue #4: For an organisation in Europe it is still hard to assess the competencies of applicants, employees and learners who have studied and worked in a variety of settings.

WP6 Specific Objective:Develop a formal specification model and supporting tools that combines new assessment types and the ones included in the IMS QTI (providing input for standards development).

TenC Issue #5: Current e-learning and knowledge management environments provide too little effective support to the users in their various tasks.

WP6 Specific Objective:Research and develop innovative, standards-based methods and tools for the creation, storage and use of formal and informal learning activities and units of learning.


10 November 2006,

Overall WP6 Objectives

Research and develop innovative, standards-based

methods and tools for the creation, storage, use, exchange

and quality rating of formal and informal learning activities

and units of learning, including methods and tools for the

assessment of the learning process and learning outcomes.


10 November 2006,

WP6 Contributions to TenCompetence Domain Model (1/2)

WP6 related items


10 November 2006,

WP6 Contributions to TenCompetence Domain Model (2/2)

Component B Component A

Component C

Assessment ComponentPhase 1: Specification Definition and proof-of-concept toolPhase 2: Specification Elaboration and TenC Assessment ToolPhase 3: Preparation for International Specification

Authoring ComponentPhase 1: Specification of IMS LD Authoring ToolPhase 2: TenC Authoring Tool v1.0 (implementing IMS LD)Phase 3: TenC Authoring Tool v2.0 (implementing IMS LD + the new Assessment Specification)

Run-time ComponentPhase 1: Connector Protocol DefinitionPhase 2: TenC IMS LD Run-time Engine (implementing the protocol) and Protocol ElaborationPhase 3: TenC Run-time Engine (implementing IMS LD + the protocol + new Assessment Specification)

Working Group BWorking Group A

Working Group C


10 November 2006,

WP6: WG-A (IMS LD Authoring) ActivitiesIssue to be tackled:

Although there are a number of existing IMS LD Authoring Tools (such as Reload LD Editor, CooperAuthor), in TenCompetence a higher-level graphical Learning Flow tool is needed to enable practitioners visualize and assemble Units of Learning easily. The design paradigm of this tool should be closer to the common practice on designing pedagogical scenarios, rather than to the XML-based structure of the IMS Learning Design specification.

Main activities for Working Group A:a) Definition of an algorithm for machine (automatic) transformation of practitioners’ workflow

design paradigms to the lower XML language of IMS LD and vice versa

b) Architectural design and implementation of the TenC IMS LD Authoring Tool prototype

c) Creation of a set of examples to be used for testing the TenC IMS LD Authoring Tool

d) Planning and execution of a usability evaluation study (in respect to the provided flexibility for learning designers and practitioners to define their own pedagogical scenarios) of the TenC IMS LD Authoring Tool


10 November 2006,

WP6: WG-B (Assessment Model) Activities

Issue to be tackled:Assessment in lifelong competence development is a complex process with many actors, which can and needs to be supported in a variety of ways. The scope of the TenC Assessment Specification is to provide the means for defining assessment processes, as an internal part of the design process of a Unit of Learning, by combining new types of assessment (e.g. 360-degree assessment, portfolio assessment) with the ones already included in the IMS QTI specification.

Main activities for Working Group B:a) Definition of an Assessment Specification and integration with the IMS Learning Design

specificationb) Development of a proof-of-concept Assessment Authoring Tool for demonstrating the use of

the TenC Assessment Specification and its integration with IMS QTIc) Creation of a set of examples to be used for validating the TenC Assessment Specificationd) Planning and execution of validation testing of the TenC Assessment Specification via the use

of the Assessment Authoring Tool


10 November 2006,

WP6: WG-C (IMS LD Run-time) ActivitiesIssue to be tackled:

The range of services available in the IMS Learning Design specification is restricted. The current structure for a service in the IMS LD specification essentially maps an open set of roles to a fixed set of service roles, limiting the ability of IMS LD Run-time engines to connect to external communication and collaboration services (e.g Instant Messaging, Chat services).

Main activities for Working Group C:

a) Definition of a specification to connect communication and collaboration services to IMS LD Runtime Engines

b) Enhancement of existing IMS LD Run-time Engines (namely, Reload Player and Coopercore) to support the TenC Services Connection Specification

c) Creation of a set of examples to be used for validating the TenC Services Connection Specification

d) Planning and execution of validation testing of the TenC Services Connection Specification via the use of the enhanced IMS LD Runtime Engines


10 November 2006,

Work Package 8 – Task 1

Competence Observatory


10 November 2006,

TenC key problems tackled in WP8 T1

TenC Issue #4: For an organization in Europe it is still hard to assess the competencies of applicants, employees and learners who have studied and worked in a variety of settings. To this end TENCompetence will provide models and software tools to assess the competencies of individuals, groups and organizations in an exchangeable way. An interoperable mechanism to express and exchange learners’ competencies will be developed, especially for informally acquired knowledge and competencies. Core aspects of such a mechanism are methods and technologies for interoperable person and group portfolio's and personal development plans, and interoperable competence definitions.

WP8 Task 1 Specific Objective:Develop a Competence Observatory to monitor and capture the competences that have to be acquired in different professional and academic fields, in an interoperable way following the current state of art specifications namely IMS RDCEO and HR-XML.


10 November 2006,

TenC Competence

Domain Model

[Rob Koper, TenC Domain Model v1.0,

19 June 2006]

WP8 T1 Competence Observatory


10 November 2006,

TenC Competence Observatory Scope

• Monitor and capture the competencies that have to be acquired in different professional and academic fields

• Provide the infrastructure for specifying competencies in an interoperable way

• Provide the communication tools, which will allow competence experts and practitioners to discuss and exchange opinions about competence-related issues


10 November 2006,

TenC Competence Targeted User Groups

• The targeted groups of the TENC Competence Observatory are:– Enterprises or organizations who are interested in defining job

profiles for their employees– Individuals who have a specific profession and they want to

assess their proficiency level by comparing their own competencies with reference ones for their profession

• Organizations and Individuals can be seen as “Communities related with specific job roles and they want to share experiences related to required competencies for a specific job role”

• The TenC Competence Observatory needs to support Communities of Professions


10 November 2006,

Main Functionalities

Information Services Communication Tools Resource Management

News Search Engines Forum Chat Wiki Links Resource Library

• Competence related functionalities:– Searching specific competencies, and exporting them

as XML files– Creating competencies descriptions– Editing/updating competencies descriptions

• Basic functionalities:


10 November 2006,

Using the TenC Competence Observatory


10 November 2006,

Review of Competence Models and Observatory Tools


10 November 2006,

Definition of Competency and Competency Models

• A Competency can be defined as “A set of skills, related knowledge and attributes that allow an individual to perform a task or an activity within a specific function or job” [1]

• The Competency Description Models define an information model for describing, referencing, and exchanging definitions of competencies.

• Two existing specifications for describing competencies.

– IMS RDCEO specification from e-learning domain– HR-XML specification from Human Resources domain

[1] United Nations Industrial Development Organization, “UNIDO Competencies”, 2002, Available Online at: http://www.unido.org/userfiles/timminsk/UNIDO-CompetencyModel-Part1.pdf, Last retrieved in 13 April 2006


10 November 2006,

IMS RDCEO Key Elements (1)• Identifier: A globally unique label that identifies this definition of competency or

educational objective. The “Identifier” element consists of two other sub-elements: “Catalog” and “Entry”.

• Title: A short name for this competency or educational objective. The “Title” may be repeated in multiple languages.

• Description: A narrative description of the competency or educational objective. The “Description” may be repeated in multiple languages.

• Definition: A structured description that provides a more complete definition of the competency or educational objective, using a collection of statements that determine a competency or an educational objective. Typically, such models define a competency or educational objective in terms of a “statement, conditions, criteria”, “proficiency, criteria, indicators”, “standards, performance indicators, outcomes”, “abilities, basic skills, content, process”, and similar sets of statements. The “Definition” consists of two other sub-elements: “Model Source” and “Statement”.

• Metadata: Optional meta-data record that further describe the RDCEO. The meta-data records must be conform to IEEE 1484-12.1-2002 (IEEE Learning Object Metadata).


10 November 2006,

IMS RDCEO Key Elements (2)


10 November 2006,

Disadvantages of IMS RDCEO

• How to represent the level of a competency? The IMS-RDCEO specification supports the representation of a competency level, within the “title” element. The information stored within this element is in a narrative format, thus, it is not machine understandable and limits the scope of interoperability among different systems.

• How to represent the grading scale of a competency? The IMS-RDCEO specification does not provide a way to represent the “grading scale” of a competency, thus, provides limited support for the assessment of complex competencies.

• How to represent the success threshold of a competency? The IMS-RDCEO specification does not support the definition of a “success threshold” for a competency. Therefore, a learning system cannot interpret the existence of a competence.

• How to describe complex competencies in an interoperable way? The IMS-RDCEO already supports the definition of complex competencies (that is, any competency consisting of other – simple or complex – competencies) through the use of “metadata/relation” element. However, it does not provide a way to represent the weighting factor of sub-competencies when assessing a complex one, thus, provides limited support for the assessment of complex competencies.


10 November 2006,

HR-XML Key Elements (1)• Name: A short name for the related competency.• Description: A narrative description of the competency.• Required: A boolean used to indicate whether the CompetencyEvidence is mandatory

for a particular position or given context.• CompetencyId: An identification code assigned to identify or classify the competency.• TaxonomyId: A code that identifies the taxonomy of the competency.• CompetencyEvidence: A text label that is used to capture information to substantiate

the existence, sufficiency, or level of a Competency. CompetencyEvidence might include test results, reports, performance appraisals, evaluations, certificates, licenses, or a record of direct observation, such as a report given by a former supervisor or other employment reference.

• CompetencyWeight: A text label that allows the capture of information on the relative importance of the Competency.

• Competency: Competencies can be recursive. A competency may include other competencies. One competency might be decomposed into several component competencies, each of which might be separately measurable.

• UserArea: Personal information about the individual that holds the defined competency.


10 November 2006,

HR-XML Key Elements (2)


10 November 2006,

Disadvantages of HR-XML• How to represent the level of a competency? The HR-XML specification supports

the representation of the proficiency level of a competency, within the “CompetencyWeight” element. This element also permits the capture of the minimum and the maximum value for the proficiency level of the relative competency.

• How to represent the grading scale of a competency? The HR-XML specification does not provide a way to represent the “grading scale” of a competency, thus, provides limited support for the assessment of complex competencies.

• How to represent the success threshold of a competency? The HR-XML specification does not support the definition of a “success threshold” for a competency. Therefore, a learning system cannot interpret the existence of a competence.

• How to describe complex competencies in an interoperable way? The HR-XML supports the definition of complex competencies through the use of “Competency” element. However, it does not provide a way to represent the weighting factor of sub-competencies when assessing a complex one, thus, provides limited support for the assessment of complex competencies.


10 November 2006,

Mapping of IMS-RDCEO and HR-XML (1)

IMS RDCEO HR-XML

Statement Text

Statement ID

Statement Token

Statement Name

Statement

Entry

Catalog

Identification

Title

Description

Definition

Model Source

IdOwner

Competency

Id

Description

Name

CompetencyId

IdOwner

Id

Description

Name

CompetencyId

UserArea

CompetencyWeight

CompetencyEvidence

TaxonomyId


10 November 2006,

Mapping of IMS-RDCEO and HR-XML (2)

IMS RDCEO HR-XML

Metadata (IEEE LOM)

CompetencyEvidence

CompetencyWeight

UserArea

IdOwner

Id

TaxonomyId

Description

Classification/TaxonPath/

Source

Classification/TaxonPath/

Taxon/Id

Classification/Description

User Profile (IMS LIP)


10 November 2006,

ASK-CDM Key Elements (1)• Identifier: A globally unique label that identifies this definition of

competency or educational objective. The “Identifier” element consists of two other sub-elements: “Catalog” and “Entry”.

• Title: A short name for this competency. The “Title” may be repeated in multiple languages.

• Description: A narrative description of the competency or educational objective. The “Description” may be repeated in multiple languages.

• ProficiencyLevel: The proficiency level of the competency. There may be multiple instances of this category.

• Classification: This category describes where this competency falls within a particular classification system. To define multiple classifications, there may be multiple instances of this category.

• Relation: Competencies can be recursive. A competency may refer to other competencies.


10 November 2006,

ASK-CDM Key Elements (2)


10 November 2006,

Case Study: The Europass

• Europass has recently adopted by the European Parliament, in order to help citizens to convey their qualifications and competences in an effective way, by providing a comprehensive tool for users based on an accessible electronic format. The Europass consists of five documents:– Europass CV– Europass Language Passport– Europass Mobility– Europass Diploma Supplement– Europass Certificate Supplement


10 November 2006,

www.tencompetence.org


10 November 2006,

Presenter’s Contact Information

Advanced e-Services for the Knowledge Society Research Unit (A.S.K.)

Informatics & Telematics Institute (I.T.I.)The Centre of Recearch and Technology Hellas (C.E.R.T.H.)

Postal Address: 42 Arkadias & Taygetou Str., GR-15234, Chalandri, Athens, Greece

TEL: +30-210-6839916/17, +30-210-414.2766 FAX: +30-210-6896082, +30-210-414.2767

URL: http://www.ask4research.info

Documents

Panel on Competency-based Learning I2LOR2006 Conference, Montreal, Canada 10 November 2006,