711:Intelligent Tutoring

SystemsWeek 1 – Introduction

2

Today Introductions Some overviews…

• Course objectives and course structure• Syllabus and assignments• Intelligent tutoring systems

Reading discussion• Why ITSs?• What this course will cover and what it won’t cover

Teams and topics Create your first interface and behavior graph

3

Introductions Your name Department Research interest What’s your motivation for taking this course? What do you expect to learn?

4

OVERVIEWS

5

Course objectives Relate learning sciences theory to practice

• You’ll learn about research-based principles about how to design effective instruction

• You’ll create an ITS that makes use of these principles

Work in a domain of your choice• Think about how the assignments can be useful within

your own discipline of interest• Think about how you can make the assignments

useful for your own research

6

Course structure Part I (between classes): Read about some

research-based principles about learning and instruction• Reflect and plan how you would apply these

principles in designing your own ITS (design posts)

Part II (in class): Critical discussion of principles Part III (in class and at home): Putting the

principles into practice• Work on your ITS in class• Finish what you did not complete in class at home

Part IV (final): Your own ITS!

7

Syllabus…

8

Assignments Always:

• Readings• Design posts• Comment on others’ design posts• Finish implementations that you did not complete in

class

5/6:• Final presentations• Final write-ups

9

INTRODUCTION TO INTELLIGENT TUTORING SYSTEMS(my only “lecture” this semester)

10

Outline What is ‘a tutor?’

• Some examples Use of CTAT to author tutors

• Motivation• Common features: What CTAT can do• Demos• Examples of projects that have used CTAT

Evidence of authoring efficiency with CTAT

11

President Obama on ITSs“[W]e will devote more than three percent of our GDP to

research and development. …. Just think what this will allow us to accomplish: solar cells as cheap as paint, and green buildings that produce all of the energy they consume; learning software as effective as a personal tutor; prosthetics so advanced that you could play the piano again; an expansion of the frontiers of human knowledge about ourselves and world the around us. We can do this.”

http://my.barackobama.com/page/community/post/amyhamblin/gGxW3n

12

Cognitive Tutor Algebra

Use graphs, graphics calculator

Analyze real world problem scenarios

Use table, spreadsheet

Use equations, symbolic calculator

Tracked by knowledge tracing

Model tracing to provide context-sensitive Instruction

13

Cognitive Tutor Geometry

14

The nested loop of conventional teaching

For each chapter in curriculum Read chapter For each exercise, solve it Teacher gives feedback on all solutions at once Take a test on chapter

VanLehn, K. (2006). The behavior of tutoring systems. International Journal of Artificial Intelligence in Education, 16(3), 227-265.

15

Nested loops of computer-assisted instruction

For each chapter in curriculum Read chapter For each exercise

• Attempt answer• Get feedback & hints on answer; try again• If mastery is reached, exit loop

Take a test on chapter

VanLehn, K. (2006). The behavior of tutoring systems. International Journal of Artificial Intelligence in Education, 16(3), 227-265.

16

The nested loops of ITSs For each chapter in curriculum Read chapter For each exercise

• For each step in solutiono Student attempts stepo Get feedback & hints on step; try again

• If mastery is reached, exit loop Take a test on chapter

VanLehn, K. (2006). The behavior of tutoring systems. International Journal of Artificial Intelligence in Education, 16(3), 227-265.

17

Common Tutor Features1. Problem-type-specific interfaces with multiple representational

tools, designed to make thinking steps visible (Anderson, Corbett, Koedinger, & Pelletier, 1995);

2. Correctness feedback on problem-solving steps

3. Feedback messages for commonly-occurring errors;

4. On-demand hints about what to do next, available at any point during problem solving;

5. Background materials such as a Glossary, online textbook, worked examples, etc.

6. An open learner model (dubbed “skill meter”) that displays the system’s estimates of an individual student’s skill mastery

7. Individualized problem selection based on each student’s performance with the tutor

18

Design/Development Process

As important as the “tutor features” is grounding the design (and re-design) process in a strong understanding of student thinking of learning:• Use of cognitive task analysis up front to inform initial

design• Use of student data (e.g., tutor log data in the

DataShop or results from in vivo studies) to inform redesign

19

Demo 1 Fractions Tutor

20

Feedback Studies in LISP Tutor (Corbett & Anderson, 1991)

65432100

250

500

750

1000

1250

1500

Immediate Feedback

Error Flagging

Demand FeedbackNo Feedback

Tutor Lesson

Time to Complete Programming

Problems in LISP Tutor

Immediate Feedback Vs

Student-Controlled Feedback

21

What does it mean to say that tutors are “adaptive?”

22

Demo 2 ChemTutor

23

CTAT : easier and faster tutor development!

Cognitive Tutors: • Large student learning gains as a result of detailed cognitive modeling• ~200 dev hours per hour of instruction (Koedinger et al., 1997)• Requires PhD level cog scientists and AI programmers

Development costs of instructional technology are, in general, quite high• E.g., ~300 dev hours per hour of instruction for Computer Aided

Instruction (Murray, 1999)

Solution: Easy to use Cognitive Tutor Authoring Tools (CTAT)

Murray, T. (1999). Authoring Intelligent Tutoring Systems: An Analysis of the state of the art. The International Journal of Artificial Intelligence in Education, 10, 98-129.

Koedinger, K. R., Anderson, J. R., Hadley, W. H., & Mark, M. A. (1997). Intelligent tutoring goes to school in the big city. The International Journal of Artificial Intelligence in Education, 8, 30-43.

25

Tutors supported by CTAT Cognitive Tutors

• Difficult to build; for programmers• Uses rule-based cognitive model to guide students• General for a class of problems

Example-Tracing Tutors• Novel ITS technology• Much easier to build; for non-programmers• Use generalized examples to guide students• Programming by demonstration• One problem (or so) at a time

26

Building an example-tracing tutor

1. Decide on educational objectives

2. Cognitive Task Analysis

3. Design and create a user interface for the tutor

4. Demonstrate correct and incorrect behavior (i.e., create a behavior graph)

• Alternative strategies, anticipated errors

5. Generalize and annotate the behavior graph• Add formulas, ordering constraints• Add hints and error messages• Label steps with knowledge components

6. Test the tutor

7. (Optional) Use template-based Mass Production to create (near)-isomorphic behavior graphs

8. Deliver on the web - import problem set into LMS (TutorShop)

27

CTAT’s utility Over 500 CTAT users in summer schools, courses,

workshops, research, and tutor development projects• Domains: mathematics, chemistry, genetics, French culture, Chinese, ESL,

thermodynamics• At least 44 research studies built tutors and deployed in real educational settings

In the past two years• CTAT was downloaded 6,600 times• the CTAT website drew over 2.9M hits from 164k unique visitors

28

Other CTAT topics Techniques to make example-tracing tutors

flexible• Multiple paths, formulas, ordering constraints

Techniques to batch-author isomorphic and near-isomorphic problems• Mass Production combined Excel

Evidence of authoring efficiency for example-tracing tutors

29

READING DISCUSSION

30

Why Intelligent Tutoring Systems?

… will cover

• Model tracing• Knowledge components• Mass production• Learning sciences

principles for instructional design

… will not cover

• Knowledge tracing• Production rules• DataShop and tutor log

data analysis• Empirical evaluations

What this course…

32

YOUR FIRST OWN ITS

33

Individual or Group Work

Come up with 3 tasks that could be implemented in a tutoring system

Tutor problems should:• Include multiple steps• Lend themselves to creating multiple versions that

increase in difficulty

Sketch out your ideas on paper

34

Practice Session Flash

Create a simple interface for one of your problems

Use at least the following components:• CommShell• CommTextInput• Done button

35

Practice Session CTAT

Create a simple behavior graph for one of your problems

Steps:• Create start state• Demonstrate solutions• Annotate solution steps:

o Incorrect stepso Hint messages

Test the tutor

36

FOR NEXT WEEK

37

Tasks before next week

Finish interface and behavior graph for your problems

Do the assigned readings Post on Moodle (Sunday, 11:59pm) how you

plan to conduct an expert cognitive task analysis, apply the subgoaling principle to your tutor problem, and how you plan to come up with a sequence of tutor problems for your task

Comment on others’ posts