Upload
-
View
131
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Citation preview
Intelligent Tutoring(Student-Problem Table)
Timothy K. Shih
Outline
• Purpose of ITS (Intelligent Tutoring System) and ATS (Adaptive Testing System)
• Evaluation Methods• Student-Problem Chart• Item Respond Theory
Purpose• ITS (Intelligent Tutoring System)
– Students learns in different pace– Can not provide individualized lecture– Intelligent Tutoring: students learn in different
topology– Tutorial Generation: remedial lectures after tests
• ATS (Adaptive Testing System)– Traditional test methods are not flexible– Item Response Theory: a relatively modern test
method– Different students use different number of questions
and different questions– Instance hints to students during test
Evaluation Methods
• Quality of Distance Education (timely course contents, presentation quality, etc.)
• Type of Evaluations– Student to instructor: teaching evaluation– Student/instructor to content: content evaluation– Instructor to student: performance evaluation– Administrator to program: DL program evaluation
InstructionObjective
ReadinessEvaluation
InstructionActivity
InstructionEvaluation
Feedback
• Is the objective satisfied?• Considering the background of students, is the
course material too difficult?• Which part of instruction was not explained clearly
by the instructor?• Which student is not working hard?
The General Model of Instruction
• Placement evaluation: The evaluation assesses how a new student or a new instructor is adapted to the new environment (to test both prerequisite skill and course objective).
• Formative evaluation: During the process of instruction, informal quizzes are given to students. The outcome of evaluation are feedback to both the instructor and the students.
• Diagnostic evaluation: The identification of persistent or recurring learning difficulties that are left unresolved by the standard correction perspective of formative evaluation.
• Summative evaluation: Grades are given to the students to certify the students’ global level of knowledge on the topic taught.
• Self-evaluation: A mix of formative and diagnostic evaluations, which is taken by the students to evaluate whether to face summative evaluation.
• Self-regulation
Evaluation Methods
• Norm-referenced evaluation: The credit of an individual is compared to the average credit of a group to justify the performance of the individual.
• Criterion-referenced evaluation: A standard terminal behavior is established. The credit of an individual is compared to the standard.
Evaluation Strategies
Mastery Learning
Formative Assessment
Repeated Training
Course Unit A
FormativeAssessment A
FormativeAssessment B
Course Unit B
Pass ?
Pass ?
Training Preparation
Yes
Yes
No
No
Student-Problem Chart
• The Student-Problem Chart Analysis Theory (S-P Table) [Sato 1975]
• Caution Indices for Students and Items [Sato 1975]
• The Student-Navigation Chart (S-N Table)
• Caution Indices for Students and Course Units
• Cross References of S-P Table and S-N Table
An Example of Student-Problem Chart
S
P
2 3 7 4 9 1 6 5 10 8 grades percentage Student
caution index
Determine
classification
7 1 1 1 1 1 1 1 1 1 1 10 100% 0.00 A
5 1 1 1 1 1 1 1 1 1 0 9 90% 0.00 A
9 1 1 1 1 1 0 1 1 0 1 8 80% 0.57 A’
4 1 1 1 1 1 1 1 0 0 0 7 70% 0.00 B
10 1 1 1 0 1 0 1 0 1 0 6 60% 0.50 B’
2 1 1 0 1 1 0 1 0 0 0 6 60% 0.50 B’
14 1 1 1 0 1 0 1 0 0 0 5 50% 0.00 B
1 1 1 1 0 1 0 0 0 1 0 5 50% 0.40 B
13 1 1 0 1 0 0 1 0 0 1 5 50% 0.70 B’
6 1 0 0 1 0 1 0 1 1 0 5 50% 1.00 B’
15 1 1 1 1 0 1 0 0 0 0 4 40% 0.40 C
11 1 0 0 0 0 1 0 1 0 1 4 40% 1.20 C’
3 1 0 1 0 1 0 0 0 0 0 3 30% 0.33 C
8 0 0 1 0 0 1 0 0 0 0 2 20% 0.71 C’
12 0 1 0 0 0 0 0 0 0 0 1 10% 0.25 C
numbers 12 11 10 9 8 8 7 6 5 4 80
Correct
percentage
80 73 67 60 53 53 47 40 33 27
Problem
Caution index
0.10 0.35 0.55 0.15 0.15 0.75 0.00 0.29 0.38 0.55
Determine
classification
A A A’ A A A’ B B B B’
Problem Caution Index Diagnosis
Problem Caution Index
Percentage of students who answer questions correctly
A A’
B B’
0.50 1.000
50%
100%
Student Caution Index Diagnosis
Student Caution Index
Percentage of student score
A A’
B B’
0.50 1.000
50%
100%
C C’
75%
Student-Navigation: The Original Table
網頁課程
學習者 1 2 3 4 5 6 7 8 9 10
精熟
學習分數 學習
總分
1 1 2 1 2 2 2 2 2 2 3 8 19
2 2 2 3 3 2 3 3 3 3 3 10 27
3 2 1 2 2 1 1 3 1 1 3 5 17
4 1 1 3 1 1 1 1 3 3 2 4 17
5 1 3 1 3 1 1 1 2 2 3 5 18
6 1 2 1 2 1 2 1 2 1 3 5 16
7 1 1 2 2 2 2 1 2 3 2 7 18
8 2 2 2 3 2 3 1 2 2 2 9 21
9 2 2 3 3 3 3 3 2 3 2 10 26
10 1 1 3 3 2 1 3 2 3 2 7 21
11 1 2 3 3 2 3 2 2 2 3 9 23
12 1 3 1 2 1 2 2 2 1 3 6 18
13 1 1 1 1 2 2 2 2 2 2 6 16
14 1 1 1 1 1 1 1 1 3 2 2 13
15 1 1 1 1 1 1 1 1 1 1 0 10
16 1 1 2 1 2 2 1 1 3 1 4 15
17 1 3 2 2 2 1 1 2 3 2 7 19
18 1 1 2 3 2 2 3 2 2 3 8 21
19 1 1 1 1 1 1 1 3 1 1 1 12
20 1 1 3 1 1 1 3 1 1 2 3 15
精熟
瀏覽分數 4 9 12 13 11 11 10 15 14 17 18.1
瀏覽總分 24 32 38 40 32 35 36 38 42 45 36.2
Sorted by Mastery Learning Score網頁課程
學習者 1 2 3 4 5 6 7 8 9 10
精熟
學習分數 學習
總分
2 2 2 3 3 2 3 3 3 3 3 10 27
9 2 2 3 3 3 3 3 2 3 2 10 26
11 1 2 3 3 2 3 2 2 2 3 9 23
8 2 2 2 3 2 3 1 2 2 2 9 21
18 1 1 2 3 2 2 3 2 2 3 8 21
1 1 2 1 2 2 2 2 2 2 3 8 19
10 1 1 3 3 2 1 3 2 3 2 7 21
17 1 3 2 2 2 1 1 2 3 2 7 19
7 1 1 2 2 2 2 1 2 3 2 7 18
12 1 3 1 2 1 2 2 2 1 3 6 18
13 1 1 1 1 2 2 2 2 2 2 6 16
5 1 3 1 3 1 1 1 2 2 3 5 18
3 2 1 2 2 1 1 3 1 1 3 5 17
6 1 2 1 2 1 2 1 2 1 3 5 16
4 1 1 3 1 1 1 1 3 3 2 4 17
16 1 1 2 1 2 2 1 1 3 1 4 15
20 1 1 3 1 1 1 3 1 1 2 3 15
14 1 1 1 1 1 1 1 1 3 2 2 13
19 1 1 1 1 1 1 1 3 1 1 1 12
15 1 1 1 1 1 1 1 1 1 1 0 10
精熟
瀏覽分數 4 9 12 13 11 1 10 15 14 17 18.1
瀏覽總分 24 32 38 40 32 35 36 38 41 45 36.2
Sorted by Mastery Navigation Score網頁課程
學習者 10 8 9 4 3 6 5 7 2 1
精熟
學習分數 學習
總分
2 3 3 3 3 3 3 2 3 2 2 10 27
9 2 2 3 3 3 3 3 3 2 2 10 26
11 3 2 2 3 3 3 2 2 2 1 9 23
8 2 2 2 3 2 3 2 1 2 2 9 21
18 3 2 2 3 2 2 2 3 1 1 8 21
1 3 2 2 2 1 2 2 2 2 1 8 19
10 2 2 3 3 3 1 2 3 1 1 7 21
17 2 2 3 2 2 1 2 1 3 1 7 19
7 2 2 3 2 2 2 2 1 1 1 7 18
12 3 2 1 2 1 2 1 2 3 1 6 18
13 2 2 2 1 1 2 2 2 1 1 6 16
5 3 2 2 3 1 1 1 1 3 1 5 18
3 3 1 1 2 2 1 1 3 1 2 5 17
6 3 2 1 2 1 2 1 1 2 1 5 16
4 2 3 3 1 3 1 1 1 1 1 4 17
16 1 1 3 1 2 2 2 1 1 1 4 15
20 2 1 1 1 3 1 1 3 1 1 3 15
14 2 1 3 1 1 1 1 1 1 1 2 13
19 1 3 1 1 1 1 1 1 1 1 1 12
15 1 1 1 1 1 1 1 1 1 1 0 10
精熟
瀏覽分數 17 15 14 13 12 11 11 10 9 4 18.1
瀏覽總分 44 37 40 39 37 34 31 35 31 23 36.2
Student-Navigation Table網頁課程
學習者 10 8 9 4 3 6 5 7 2 1
精熟
學習分數 學習
總分
2 3 3 3 3 3 3 2 3 2 2 10 27
9 2 2 3 3 3 3 3 3 2 2 10 26
11 3 2 2 3 3 3 2 2 2 1 9 23
8 2 2 2 3 2 3 2 1 2 2 9 21
18 3 2 2 3 2 2 2 3 1 1 8 21
1 3 2 2 2 1 2 2 2 2 1 8 19
10 2 2 3 3 3 1 2 3 1 1 7 21
17 2 2 3 2 2 1 2 1 3 1 7 19
7 2 2 3 2 2 2 2 1 1 1 7 18
12 3 2 1 2 1 2 1 2 3 1 6 18
13 2 2 2 1 1 2 2 2 1 1 6 16
5 3 2 2 3 1 1 1 1 3 1 5 18
3 3 1 1 2 2 1 1 3 1 2 5 17
6 3 2 1 2 1 2 1 1 2 1 5 16
4 2 3 3 1 3 1 1 1 1 1 4 17
16 1 1 2 1 2 2 2 1 1 1 4 15
20 2 1 1 1 3 1 1 3 1 1 3 15
14 2 1 3 1 1 1 1 1 1 1 2 13
19 1 3 1 1 1 1 1 1 1 1 1 12
15 1 1 1 1 1 1 1 1 1 1 0 10
精熟
瀏覽分數 17 15 14 13 12 11 11 10 9 4 18.1
瀏覽總分 45 38 42 40 38 35 32 36 32 24 36.1
S Curve
N Curve
Student-Navigation Table with Indices網頁課程
學習者 10 8 9 4 3 6 5 7 2 1
精熟
學習分數
學習
總分
平均
學習次數
學習者
注意係數
判定
類別
2 3 3 3 3 3 3 2 3 2 2 10 27 2.7 0 A
9 2 2 3 3 3 3 3 3 2 2 10 26 2.6 0.73 A’
11 3 2 2 3 3 3 2 2 2 1 9 23 2.3 0.22 B
8 2 2 2 3 2 3 2 1 2 2 9 21 2.1 0.88 B’
18 3 2 2 3 2 2 2 3 1 1 8 21 2.1 0.22 B
1 3 2 2 2 1 2 2 2 2 1 8 19 1.9 0.24 C
10 2 2 3 3 3 1 2 3 1 1 7 21 2.1 0.22 B
17 2 2 3 2 2 1 2 1 3 1 7 19 1.9 0.56 C’
7 2 2 3 2 2 2 2 1 1 1 7 18 1.8 0.03 C
12 3 2 1 2 1 2 1 2 3 1 6 18 1.8 0.64 C’
13 2 2 2 1 1 2 2 2 1 1 6 16 1.6 0.20 C
5 3 2 2 3 1 1 1 1 3 1 5 18 1.8 0.32 C
3 3 1 1 2 2 1 1 3 1 2 5 17 1.7 0.56 C’
6 3 2 1 2 1 2 1 1 2 1 5 16 1.6 0.25 C
4 2 3 3 1 3 1 1 1 1 1 4 17 1.7 0.22 C
16 1 1 2 1 2 2 2 1 1 1 4 15 1.5 0.58 C’
20 2 1 1 1 3 1 1 3 1 1 3 15 1.5 0.45 C
14 2 1 3 1 1 1 1 1 1 1 2 13 1.3 0.36 C
19 1 3 1 1 1 1 1 1 1 1 1 12 1.2 0.46 C
15 1 1 1 1 1 1 1 1 1 1 0 10 1 0 C
精熟瀏覽分數 17 15 14 13 12 11 11 10 9 4 18.1
瀏覽總分 45 38 42 40 38 35 32 36 32 24 36.1
平均瀏覽次數 2.25 1.9 2.1 2 1.9 1.75 1.6 1.8 1.6 1.2
網頁注意係數 0.39 0.53 0.52 0.02 0.26 0.11 0.34 0.17 0.54 0.25
判定類別 A B’ A’ A B B B B B’ B
Course Node Caution Index Diagnosis
B
0
A A’
B’
Course Node Caution Index
0.50 1.00
2
3Average number of navigation to course nodes
1
Student Navigation Caution Index Diagnosis
0
A A’
B B’
Student Navigation Caution Index
0.50 1.00
2
2.5Average of student navigation
1
C C’
Cross Reference of S-P and S-N Tables
S-N S-P Observation
A A Excellent students, with stable learning progress
A A’ Good students, but careless from time to time
A B Good students, but do not work hard enough
A B’ Good students, but careless and do not work hard enough
A C Good students, but due to some reasons, do not perform well. Need special attention
A C’
• How do we develop Web courses with an easy to access style? – Course Patterns
• How do we evaluate student learning performance? – Navigation, Tests
• How can we help them from the evaluation? – SPC Table
• How can computer help them? – Tutorials
Timothy K. Shih, Nigel Lin, Jianhua Ma, and Runhe Huang (U. of Aizu, Japan)
A Course Development and Student Assessment System
• Objectives – Mastery Learning– Formative Evaluation– Norm-referenced Evaluation
• Methods and Systems– Course Development with Patterns– Instruction and Assessment Tools– Individualized Course Content Generation
Research Motivation
• Students lost in a complicated navigation graph
• A text book has its fixed structure• Course Pattern = Structure Pattern +
Content Pattern• Only one or two course patterns are used in
a virtual university
Course Development with Patterns
• Pattern definition language• Limited number of sections in a course unit• Small section size• Fixed basic structure for Structure Pattern
(i.e., tree, linked list, set)• Fixed basic structure for Content Pattern
(i.e., text, picture, Pop-up Quiz, Question)
Course Pattern Construction
Reusable Course Unit
Reusable Session
Course Unit
Session
A Graphical User Interface is used to construct structure patterns.
Example: Tree and Linked Lists
Web Course Structure Pattern
Example: Tree, Linked Lists, and Sets
Web Course Structure Pattern (Cont.)
Formal Definition of Course PatternsStructurePattern := RCU StructurePRCU := Null | CourseUnitName @CourseUnitName := ASCIIStructureP := Session | TreePattern | ListPattern | SetPattern TreePattern := Session #< TreeP >TreeP := Null | StructurePattern TreePListPattern := #[ ListP ]ListP := Null | StructurePattern ListPSetPattern := #{ SetP }SetP := Null | StructurePattern SetP
Session := SessionName SessionContent Pop-upQuizzes SessionQuestionsSessionName := ASCIISessionContent := WebDocumentPop-upQuizzes := Null | Pop-upQuiz Pop-upQuizzesPop-upQuiz := YesNoQuestion Constraints | MultipleChoiceQuestion Constraints | FillInBlankQuestion ConstraintsSessionQuestions := Null | SessionQuestion SessionQuestionsSessionQuestion := YesNoQuestion | MultipleChoiceQuestion | FillInBlankQuestion
Constraints := Trigger | not Constraints | Constraints and Constraints | Constraints or Constraints | Constraints xor ConstraintsTrigger := TimeTrigger | ObjectNumberMetTriger | SpecificObjectMetTrigerTimeTrigger := TimeVisited >= IntegerObjectNumberMetTriger := NoOfObjectsVisited >= IntegerSpecificObjectMetTriger := CurrentMObject = MObjectYesNoQuestion := WebDocumentMultipleChoiceQuestion := WebDocumentFillInBlankQuestion := WebDocument
Note: Null is a null object @ represents the structure pattern is a reusable course unit # is an integer which represents the number of internal objectsSpecial Variables: TimeVisited CurrentMObject NoOfObjectsVisitedDefinitions omitted are: Integer is an integer ASCII is an ASCII string MObject is a multimedia object WebDocument is an HTML document
Database Schema Generation
• The Structure Pattern Table: representation of tree, list and set
• The Content Pattern Table: representation of object types and object coordinates
• The Quiz Table: representation of test content and constraints
• The Question Table: representation of assignment content
• Purpose: Easy to use and trace objects• Components
– Fixed Patterns– Text, Picture, Audio, and Video for Course
Materials– Pop-Up Quizzes (real-time interaction)– Assignments (as homework and references of
exams)• A Graphical User Interface is used to
construct content patterns
Web Course Content Pattern
• The Web Course Pattern Tools: Pattern tools are used to help the construction of Web course patterns.
• The Web Navigation Patrol: The patrol is a mobile agent program which run on client sites. The agent collects navigation messages of Web interaction, which are used in the analysis model of student behavior.
• The Performance Analysis Model: This analysis model is based on some instruction theory from the educational literature, such as the SP-table mechanism with the computation of student caution indices.
• The Progressive Learning State Machine Constructor: The constructor generates intelligent tutorials from the course material provided by the instructor.
Instruction and Assessment Tools
Pattern DDL Program
Pattern GUI
Pattern Specification
Web Course Design Editor
Web CoursePattern
Database
Curriculum Development Committee
Web CourseContentDatabase
Web Browser with Web Navigation Patrol
WebNavigationDatabase
Progressive LearningState Machine
PerformanceAnalysis Model
Pattern Formal Syntax
Students Instructors
Pattern Design
MessageSpecification File
DDL Statements
Course Pattern
Course Content
Course Content
Course Pattern
Web Navigation Records
PerformanceRecords
NavigationRecords
BNF Grammar
IntelligentTutorial
InteractionCourse Pattern
Course Design
Analysis
An Instruction Design & Delivery System
• The Student-Problem Chart Analysis Theory (S-P Table) [Sato]
• Caution Indices for Students and Items [Sato]• The Student-Navigation Chart (S-N Table) [Shih]• Caution Indices for Students and Course Units
[Shih]• Cross References of S-P Table and S-N Table
[Shih]• The Student-Problem-Course Table (SPC Table)
[Shih]
Models of Learning Assessment
S
P
2 3 7 4 9 1 6 5 10 8 grades percentage Student
caution index
Determine
classification
7 1 1 1 1 1 1 1 1 1 1 10 100% 0.00 A
5 1 1 1 1 1 1 1 1 1 0 9 90% 0.00 A
9 1 1 1 1 1 0 1 1 0 1 8 80% 0.57 A’
4 1 1 1 1 1 1 1 0 0 0 7 70% 0.00 B
10 1 1 1 0 1 0 1 0 1 0 6 60% 0.50 B’
2 1 1 0 1 1 0 1 0 0 0 6 60% 0.50 B’
14 1 1 1 0 1 0 1 0 0 0 5 50% 0.00 B
1 1 1 1 0 1 0 0 0 1 0 5 50% 0.40 B
13 1 1 0 1 0 0 1 0 0 1 5 50% 0.70 B’
6 1 0 0 1 0 1 0 1 1 0 5 50% 1.00 B’
15 1 1 1 1 0 1 0 0 0 0 4 40% 0.40 C
11 1 0 0 0 0 1 0 1 0 1 4 40% 1.20 C’
3 1 0 1 0 1 0 0 0 0 0 3 30% 0.33 C
8 0 0 1 0 0 1 0 0 0 0 2 20% 0.71 C’
12 0 1 0 0 0 0 0 0 0 0 1 10% 0.25 C
numbers 12 11 10 9 8 8 7 6 5 4 80
Correct
percentage
80 73 67 60 53 53 47 40 33 27
Problem
Caution index
0.10 0.35 0.55 0.15 0.15 0.75 0.00 0.29 0.38 0.55
Determine
classification
A A A’ A A A’ B B B B’
The Student-Problem Chart
The Student-Problem-Course (SPC) Table
View A: Score View B: Navigation View C: Test
Each view has two caution indices
The Revised Model – the SPC Table
The CSa index (caution index of student w.r.t. problem): indicates the outcome of exam. Students of a high index need special attention. On-line tutorial can be generated for these students.
The CPa index (caution index of problem w.r.t. student): indicates the quality of problems. Problems of a high index value can be re-designed.
The CSb index (caution index of student w.r.t. course unit navigation): indicates the degree of student navigation. High value means the student either works too hard (too much navigation), or too lazy (low navigation). On-line tests can be generated for those lazy students.
The CCb index (caution index of course unit w.r.t. student navigation): indicates which course unit is less visited and less effective. Course content can be revised or access paths can be re-constructed.
The CPc index (caution index of problem w.r.t. course unit): each problem is selected for an exam from a unique course unit.
The CCc index (caution index of course unit w.r.t. problem): indicate which course unit does not have a problem chosen in an exam. Problems of this course unit can be selected in the next exam of a higher priority.
The Six Caution Indices
The Student-Course Table
Course Unit Number
Stu
den
t Nu
mb
er
1 2 3 4 5 6 7 8 910 11 12 13 14 TCN CSb
1 NF1,1 NF1,2 NF1,j NF1,n TCN1 CSb1
2 NF2,1 NF2,2 NF2,n TCN2 CSb2
3 NF3,1 NF3,n TCN3 CSb3
4
5
6 NFi,j NFi,n TCNi CSbi
7
8
9
10 NFm,1 NFm,n TCNm CSbm
TSN TSN1 TSN2 TSNj TSNn
CCb CCb1 CCb2 CCbj CCbn
Caution Indices Computation
Average of Course Navigation = (Σi = 1 m TCN i ) / m =
Average of Student Navigation = (Σj = 1 n TSN j ) / n =
where TCN is the Total Course Navigation
and TSN is the Total Student Navigation
TCN i = Σj = 1 n NF i,j
TSN j = Σi = 1 m NF i,j
CSb i = 1 – (Σj = 1 n (NF i, j) (TSN j) )– (TCN i) / (Σj = 1
TCN i TSN j - (TCN i) )
CCb j = 1 – (Σi = 1 m (NF i, j) (TCN i) ) – (TSN j) / (Σi = 1
TSN j TCN i - (TSN j) )
Caution Indices Computation
• The Assessment Criteria of Mastery Learning Navigation– The frequency of the mouse movement on
course objects– The duration of course navigation– The activation on op-up quizzes
• Navigation Factor: to evaluate how hard a student is working on the Web course material
Learning Effect Function
If the degree of frequency or duration is less than the minimal threshold, the elementary message does not count.
If the degree of frequency or duration is greater the maximal threshold, it is counted as the maximal value.
Effective navigation frequency =
Σ Web object Effective Frequency / Σ Web object Actual Frequency
Effective navigation duration =
Effective Duration / Actual Duration
Navigation Frequency and Duration
• Fixed Time Activation (assigned by the instructor to enforce the student)
• Criterion-based Activation (no time limitation, activated on buttons to decide pass/no pass, Mastery Learning)
• Popup Quizzes are brought up by the system at the navigation
Effectiveness of quiz interaction =
Number of quizzes answered correctly / Total number of quizzes
Activation on Pop-up Quizzes
Navigation Factor = Effective navigation frequency * W1+
Effective navigation duration * W2 +
Effectiveness of quiz interaction * W3
where W1 + W2 + W3 = 1.0, and W1 > 0.0, W2 > 0.0, W3 > 0.0
Thus, 0.0 <= Navigation Factor <= 1.0
Mastery Learning Navigation Factor
• Web document structure (physical links)
• Web knowledge structure (logical links in a traversal)
• Knowledge Structure can represent the dependency relations among course units, from the perspective of an instructor
Web Document and Knowledge Structures
• Based on Test Results• Based on Student Navigation Factors• Based on Test Problem Used• Based on Web Knowledge Structure, or
Dependency Relations among course units
• The generated lectures are state machines which include new physical links
• Automatic lecture generation can be proceeded dynamically
Automatic Lecture Generation
• Tutorial on Missing Course Content
• Based on Test Score• Based on Navigation Factor
Let alpha_1 be the max threshold of caution index CSa
Let beta_1 be the min threshold of navigation factor Fb
TutorialSetInitiation
Input: SPC table
Output: TutorialInitSet for each student
Algorithm:
For each student S with CSa > alpha_1
For each factor Fa in View A of S, where Fa = 0
Find the factor Fc in View C w.r.t. Fa, where Fc = 'Y'
Find the Factor Fb in View B w.r.t. Fc
If Fb < beta_1 then
Find course C w.r.t. Fb
Assert course C into the TutorialInitSet for student S
Automatic Tutorial Generation
• On-Line Test for Lazy Student• Based on Student Caution Index• Based on Navigation Factor
Let alpha_2 be the max threshold of caution index CSb
TestSetInitiation
Input: SPC Table
Output: TestInitSet for each student
Algorithm:
For each student S with CSb > alpha_2
For each factor Fb in View B, where Fb < beta_1
Find course C w.r.t. Fb
Select problems of course C
Assert problems to the TestInitSet for student S
Automatic Test Generation
• Find Inappropriate Test Problem• Based on Test Problem
Let alpha_3 be the max threshold of caution index CPa
ProblemRefinement
Input: SPC Table
Output: problem update
Algorithm:
For each problem P with CPa > alpha_3
Find the factor Fc in View C w.r.t. problem P, Fc = 'Y'
Find course C w.r.t. Fc
Refine problem P in course C
Automatic Problem Refinement Indication
• Find Misleading Course Content
• Based on Course Content
Let alpha_4 be the max threshold of caution index CCb
Let alpha_5 be the min threshold of caution index CCc
CourseRefinement
Input: SPC Table
Output: course update
Algorithm:
For each course C with CCb > alpha_4
Refine content of course C and its associated Web structure
Find the corresponding caution index CCc of course C
If CCc < alpha_5
suggest to use problems in course C for the test
Automatic Course Refinement Indication
Implementation Technology
• Windows NT Server, IIS (Internet Information Server)
• ASP (DHTML, VB Script, ActiveX, CSS Component)
• MS SQL Server
• Student Management Tool– Manages
General Information of Students
– Shows Study Records of Students
System Implementation
• Course Structure and Tutorial Management Tool– Show Course
Structure– Manage Tutorial
Designed by Instructors
System Implementation
• Course Unit Design Tool– Manage General
Information of Course Units
– Allow Teacher to Design Course Unit with 5 Types of Course Unit Objects
• Text• Picture• Button• Sound• Video
System Implementation
• Problem Management Tool– Manage Problem
Database– 2 Types of Problems
• Assignments• Pop-up Quizzes
– Allow Teachers to Design 3 Types of Problems
• Multiple Choice• True or False• Fill-in-Blank
System Implementation
• Exam Tool– Allow Teacher to
Hold On-Line Exam in 3 Steps• Set up Exam
Time• Select Course
Units as an Exam Scope
• Select Problems in the Exam Scope
System Implementation
• Assessment Tool– Create SPC Table
Based on Exam– 3 Views of SPC
Table• View A: Score• View B: Navigation• View C: Exam
System Implementation
• On-Line Learning Tool– Allow Students to
Study Course Unit via Web Browser
– 3 Agents Running under the System• Course Unit
Object Navigation Tracking Agent
• Course Unit Study Record Agent
• Pop-up Quiz Agent
System Implementation
• On-Line Exam Tool– Allow
Student to Attend On-Line Exam
– Also Works on PDA
System Implementation
• On-Line Tutorial Tool– Instructor’s Tutorial
• Allow student to view pre-designed tutorials
– Student’s Tutorial• Each student has
his own tutorial generated according to SPC table
System Implementation
DemoSystem Demo
Summary of ITS
• SP Table was designed before Web technology
• Navigation factors can be tracked• Pop-up Quizzes can be used• Learning Topology can be pre-defined
• Automatically generate individualized tutorial (and Web site)
Other Intelligent Technologies
• Automatic FAQ reply• Automatic grading of chat room
participation• Analysis of collaborative learning
behavior• Semantic Web for content searching
Summary
• Automatic tutorial generation• Toward automatic test generation
• Semantics• Educational Professionals