Lifelong Competence Development: On the Advantages of Formal

Competence-Performance Modeling

Michael D. Kickmeier-Rust, Dietrich Albert, & Christina SteinerCognitive Science Section, Department of Psychology

University of Graz, Austria

Learning Networks for Lifelong Competence Development

March 30 – 31, 2006, Sofia, Bulgaria

Lifelong Competence Development

- Lifelong competence development is undoubtedly an important and ambitious aim for the information and knowledge society

- This presentation intends to introduce and motivate Knowledge Space Theory and the Competence-Performance Approach as tools

to facilitate the development and assessment of competencies over

time

Introduction:Competence vs. Competency

- Competence

- vague

- broad and intangible

- complex vs. simple

- hard to be modeled on a formal basis

- Competency

- small and unique

- still intangible

- easy to be modeled on a formal basis

Introduction:Competence vs. Performance

- Often the terms competence/y and performance are mixed-up

- Often it is assumed that competencies could directly be observed or assessed

- Inflation of competencies because different assessment methods measure different competencies or sets of competencies

- e.g., maintain an aircraft, write a scientific article, pass a certain school exam

- Incomparability of different assessment methods

- e.g., school exam vs. assessment on the job

Introduction:Competence vs. Performance

- The American Heritage Dictionary of the English Language states:

“Competence means the state or quality of being adequately or well qualified; a specific range of skill, knowledge or ability”

- This and many other definitions have in common that they describe competence as

- abstract

- latent

- not directly observable

Introduction:Competence vs. Performance

- Chomsky (1965) distinguished latent competence and observable performance in linguistic theory

- Today, this distinction between competence and performance has a much wider application e.g., in the field of knowledge and learning psychology

- competence is an unobservable quality or ability

- performance is the observable behavior in specific situations (e.g., an exam), which is determined by one specific competency or by a set of competencies

Why distinguish competence and performance?

- Example 1: Exam in trigonometry Students might be allowed to use a mathematical formulary and a pocket calculator

(1) If two students master a certain task of the exam, can we conclude that these students do have the same competencies

with regard to the task?

- Student 1 might have the necessary competencies to master the task without using the formulary

- Student 2 maybe mastered the task only by chance, incidentally choosing the right formula from the formulary

Why distinguish competence and performance?

- Example 2: Exam in trigonometry Students might be allowed to use a mathematical formulary and a pocket calculator

(2) If three students fail in a certain task, can we conclude that these students lack the same competencies?

- Student 1 might lack the competence to fully understand the task and its formulation

- Student 2 might fully understand the task and also might be able to choose the right formula, but maybe this student is not able to use a required function of the calculator

- Student 3 might have the necessary competencies to master the entire

task but might have problems to concentrate on the tasks during an exam

Lifelong learning?

- In terms of tracking and assessing lifelong competence development

- we should make sure to measure competencies independent from assessment methods

- refer to probably standardized competencies

- refer to defined developmental / learning paths

Knowledge Space Theory

- A well-elaborated theory that may help to achieve these goals is Knowledge Space Theory by Doignon & Falmagne (1985, 1999) and its extensions

- KST provides a set-theoretic framework to organize and model the knowledge / competencies in a given domain of knowledge by utilizing Surmise Relations, which establish Knowledge Spaces

- KST in its initial form is only a behavoristic approach focusing on problems (e.g., test tasks), which can be mastered or not

- From mastering a certain problem, KST allows to assume the mastering of other problems and from failing in a certain problem, KST allows to assume a failing in other problems

Knowledge Space Theory

- Example: Five problems from the domain “basic algebra”:

- a: addition

- b: subtraction

- c: multiplication

- d: division

- e: simple linear equations

Q = {a, b, c, d, e}

Knowledge Space Theory

- Example: Prerequisite Relation for the domain Q = {a, b, c, d, e}

Knowledge Space Theory

- Example: We can establish a Knowledge Space, which does not contain all of the 25 possible Knowledge States but only 8

Knowledge Space Theory

- Advantages:

- Reduction of the number of possible Knowledge States and definition of meaningful learning paths

- Mathematical properties:

- reflexive

- transitive

- anti-symmetric

Competence-Performance Approach

- KST is purely behavioristic focusing on observable performance

- CPA (Korrosy, 1997, 1999) is an extension of KST, which distinguishes latent competencies and observable performance

- We have a set E of abstract competencies that are relevant for a domain

- The Competence State is the collection of a person’s competencies

- As in KST, Prerequisite Relations are described on the set of competencies establishing a competence structure C, which contains all possible competence states

Competence-Performance Approach

- Example: Four competencies from the domain “basic algebra”:

- A: addition

- B: subtraction

- C: multiplication

- D: division

E = {A, B, C, D}

Competence-Performance Approach

- Example: Prerequisite Relation for the competencies in the domain E = {A,B,C,D}

Competence-Performance Approach

- Example: We can establish a Competence Space, which does not contain all of the 24 possible Competence States but only 7

Competence-Performance Approach

- Unfortunately, we cannot observe this…

Representation and Interpretation Functions enable to map test items / tasks to the competencies

We can determine a person’s Competence State

We can determine the required competencies to master a specific task

No 1-to-1 mapping required

Competence-Performance Approach

- Example: Two tasks from the domain “basic algebra”:

- a: multiplication problem

- b: solving linear equations

Q = {a, b}

Representation Function

Problem Competence State

a {A, B, C}

b {A, B, C, D}

Competence-Performance Approach

- Example:

- Solved a but not b

- Solved a and b

- Solved not a and not b

- Solved b and not a

Advantages

- Modeling a domain of knowledge on a formal basis

- Referring to clearly defined and unique competencies

- Mapping different assessment methods to the same set of competencies

- Efficient adaptive testing

- Efficient adaptive teaching

- Modeling of individual learning path

- Computable

So what about errors?

Careless errors

Lucky Guesses

- Besides the deterministic approach there are also probabilistic approaches

Applications

- Adaptive, personalized eLearning (RATH, APeLS, EASEL, iCLASS, ELeGI, ELEKTRA)

- Organizational competencies / knowledge (Know-Center)

- Research tool (e.g., in child development)

Applications

- Commercial eLearning platform ALEKS (www.aleks.com)

Image courtesy of ALEKS CorporationSanta Ana, CA, USA

THANK YOU