Improve Instruction

8/13/2019 Improve Instruction

1/28

Improving and Assessing

Instructional Effectiveness

Theodore Frick

Department of Instructional Systems Technology

School of Education

Indiana University Bloomington

Invited Lecture to IUB Medical Sciences FacultyOctober 14, 2002
http://education.indiana.edu/~frick/http://education.indiana.edu/~ist/http://education.indiana.edu/http://www.iub.edu/http://www.iub.edu/http://education.indiana.edu/http://education.indiana.edu/~ist/http://education.indiana.edu/~ist/http://education.indiana.edu/~frick/


2/28

October 14, 2002Improving and Assessing Instructional

Effectiveness -- Ted Frick 2

Overview

Confounding of instructional

effectiveness by other factors

Types of knowledge about education

Type 2, 3 and 4 examples

Recommendations


3/28



The Logic of Effectiveness

Before instruction After instruction Conclusion

1. Student nonmastery Student mastery Instruction appears to be

effective enough, but

2. Student nonmastery Student nonmastery Instruction apparently was noteffective enough

3. Student mastery Student mastery Can't tell if instruction is

effective, since student had

achieved the goal before

instruction began

4. Student mastery Student nonmastery ??? Something wrong with the

content - factual errors,

incorrect procedures, bad

models ??? Bad luck ???


4/28



Confounding Factors

1. Learning occurs without intentional

instruction.

2. Learning occurs despite poor

instruction.

3. Were not the only teachers a student

has.

4. Results of instruction may occur LONG

afterwards.


5/28



Conflation of Research Methods and

Outcomes

In addition to the confounding problem in

determining effectiveness of instruction,

there has been a lot of debate in

education about inquiry methodse.g.,quantitative vs. qualitative.

Continuing problem of bridging research

and practice in education


6/28



Consider this Debate

Imagine for a moment three carpenters arguing aboutwhich tool is best.

Quanta:"Hammers and nails are clearly superior."

Qualia:"I disagree. Screwdrivers and screws aremuch more effective."

Performa:"You're both wrong. Saws are best forcutting wood."

Quanta:"Who said anything about cutting wood? I

thought we were talking about fastening woodtogether."

Qualia:"Right. Who needs saws?"

Performa:"I do. I need to cut this board in half."


7/28



Premise

The kind of knowledge about

education that we create through

disciplined inquiry determines whatresearch methods are appropriate and

useful.


8/28



Types of Educology: Knowledge

about Education

Methodology of Theory Building(Elizabeth

Steiner, 1988) (link)

Non-axiological knowledgewhat is

Axiological knowledgepertaining to values: Instrumental value: good forwhat is effective

Intrinsic value: good in itselfwhat is worthwhile

The Dependability of Behavioral

Measurements: Theory of Generalizability forScores and Profiles(Lee Cronbach, et al.,

1972)
http://www.indiana.edu/~tedfrick/steiner/http://www.indiana.edu/~tedfrick/steiner/


9/28



Six Types of Knowledge (Frick)

Purpose of Inquiry

Scope of Know ledge

Unique Generalizable

What is? 1 2What is effective? 3 4

What is worthwhile? 5 6

See Frick: R690 Syllabus for further details
http://education.indiana.edu/~frick/r690/http://education.indiana.edu/~frick/r690/


10/28




Purpose of Inquiry

Scope of Knowledge




Example: Type 1 Knowledge Claim

Elizabeth Steiner is an educational philosopher.


11/28


12/28




Purpose of Inquiry

Scope of Knowledge





The DISTAR program for teaching arithmetic and reading to

elementary students works.


13/28

October 14, 2002 Improving and Assessing InstructionalEffectiveness -- Ted Frick 13


Purpose of Inquiry

Scope of Knowledge





Instruction is most effective when it is problem-based, activates priorlearner knowledge, demonstrates what is to be learned, provides

opportunities for guided practice, and encourages integration with

everyday life. (Merrill, 2001)


14/28



Purpose of Inquiry

Scope of Knowledge





The corporal punishment policy in Houston schools is a bad

policy.


15/28



Purpose of Inquiry

Scope of Knowledge





Teachers and students should respect each other.


16/28


Do not confuse ends and means

The kind of knowledge about education that wecreate through disciplined inquiry determineswhat research methods are appropriate anduseful.

Outcome of research is knowledge. Inquiry methods are a means to that end.

Criteria for evaluating adequacy of researchmethodology are NOT the same for eachknowledge typee.g., criteria for statistical inference to a broad population do not

apply to Types 1, 3, and 5;

effectiveness not of concern in Types 1, 2, 5, and 6, butmain concern in 3 and 4.


17/28


18/28


19/28


Type 2: Academic Learning Time

Allocated Time

Engaged Time

Task Success

ALT is positively correlated with academic

achievement in the same content areas (Fisher,

et al., 1976; Rieth & Frick, 1983; Berliner; 1985).


20/28



Purpose of Inquiry

Scope of Knowledge




Type 3: Evaluation of a particular instructional

program or product


21/28


Type 3: Program/Product

EvaluationKirkpatricks Levels

Techniques for Evaluating Training

Programs -- Donald Kirkpatrick (1959)

1. Reaction (satisfaction)

2. Learning (achievement)

3. Behavior (transfer to real context)

4. Results (impact on organization/context)

More on Kirkpatricks levelsfrom Encyclopedia of Educational Technology
http://www.astd.org/CMS/templates/index.html?template_id=1&articleid=20840http://www.astd.org/CMS/templates/index.html?template_id=1&articleid=20840http://coe.sdsu.edu/eet/Articles/k4levels/index.htmhttp://coe.sdsu.edu/eet/Articles/k4levels/index.htmhttp://www.astd.org/CMS/templates/index.html?template_id=1&articleid=20840http://www.astd.org/CMS/templates/index.html?template_id=1&articleid=20840


22/28



Purpose of Inquiry

Scope of Knowledge




Type 4 Example: how to create effective

instructional productsdesign theory


23/28


Type 4: Effective Web Instruction: An

Inquiry-Based ProcessFrick & Boling

More
http://www.indiana.edu/~r547/syllabus.htmlhttp://www.indiana.edu/~r547/syllabus.html


24/28


Type 4: First Principles of

InstructionDavid Merrill

Problem

Activation

DemonstrationApplication

Integration


25/28


Type 4: First Principles of

Instruction (contd)

Learning is facilitated when:

1. Learners are engaged in solving real-world

problems.

2. Existing knowledge is activated as a foundationfor new knowledge.

3. New knowledge is demonstrated to the learner.

4. New knowledge is applied by the learner.

5. New knowledge is integrated into the learnersworld. (Merrill, 2001, p. 2)

Does your instruction rate 5 stars? A rating scale
http://www.id2.usu.edu/Papers/5FirstPrinciples.PDFhttp://www.id2.usu.edu/5Star/FiveStarRating.PDFhttp://www.id2.usu.edu/5Star/FiveStarRating.PDFhttp://www.id2.usu.edu/Papers/5FirstPrinciples.PDF


26/28



Purpose of Inquiry

Scope of Knowledge




Type 3: Indiana University physician education

program


27/28


Recommendations

In physician education, focus on Type 3

knowledge:

Apply Merrills 5 principles as criteria for

design. See video(requires RealPlayer).

Use Kirkpatricks 4 levels of evaluation for

your programs at IU.
http://www.id2.usu.edu/5Star/5starins.ramhttp://www.id2.usu.edu/5Star/5starins.ram


28/28


Questions?

This presentation is available at:

http://www.indiana.edu/~tedfrick/improveinstruction.ppt

Contact: [email protected]
http://www.indiana.edu/~tedfrick/improveinstruction.pptmailto:[email protected]:[email protected]://www.indiana.edu/~tedfrick/improveinstruction.ppt

Documents

Improve Instruction