Instructor II - MFRI

Instructor II

Maryland Fire and Rescue Institute University of Maryland

Steven T. Edwards, Director

Fall 2012

Copyright© 2012 by the Maryland Fire and Rescue Institute. All rights reserved. No part of this book may by copied or reproduced in any form or by any means without written permission of the Maryland Fire and Rescue Institute.

Note-Taking Guide

The Maryland Fire and Rescue Institute of the University of Maryland is the State’s compre-hensive training and education system for all emergency services. The Institute plans, researches, develops, and delivers quality programs to enhance the ability of emergency service providers to protect life, the environment, and property.

Lesson 1-2Training Evolution Supervision

Student Performance Objective

Given information from lecture, discussion, and

reading materials, the student will be able to

describe the supervision of training evolutions.

MGMT 210-PPT-1-2-1

p g

Overview

The Safety Challenge

Using ICS to Supervise Training

Environmental Issues at Training Evolutions

MGMT 210-PPT-1-2-2


Accident Investigation


Even in a controlled environment emergency services training is inherently dangerous.

Providing realistic training situations while

MGMT 210-PPT-1-2-3

g gproviding the maximum level of safety is a challenge.

Organizations, administrators and instructors share the responsibility for meeting this challenge.

Organizational Support

Changes in organizational policy can reduce the number of training accidents.

Trends and Hazards in Firefighter Training (2003)

MGMT 210-PPT-1-2-4

e ds a d a a ds e g e a g ( 003) Recommendations

Follow guidelines, procedures, and training and safety standards

Conduct live burn

evolutions in a variety of

structure types

Train to recognize the

visual and physical clues to impending

danger

Have a risk management

plan

Administrative Responsibilities

Provide adequate personal protective equipment

Maintain apparatus and equipment

Ensure that safety equipment is properly installed

MGMT 210-PPT-1-2-5

Ensure that safety equipment is properly installed and operating

Provide policies and procedures for the safe use of apparatus and equipment

Address the fitness, health and wellness of personnel

Unsafe Behavior

Many injuries and fatalities are the result of unsafe behavior

Instructors should recognize and address:

MGMT 210-PPT-1-2-6

Instructors should recognize and address:Improper AttitudesComplacencyLack of Knowledge or SkillPhysical Limitations

Hazard and Risk Analysis

Hazard and risk analysis identifies potential problem areas and is the

MGMT 210-PPT-1-2-7

foundation for the risk management plan.

Potential hazards are prioritized based on frequency and severity.


Instructors should have a working knowledge of incident command

MGMT 210-PPT-1-2-8

systems.

ICS positions should be filled based on the scope of the training evolution.

The Training Plan

Every lesson requires a lesson plan.

The training plan should contain strategic goals, tactical objectives and support

MGMT 210-PPT-1-2-9

g , j pprequirements.

Training Evolution Evaluation

Evaluates student skills and learningEvaluates practical training evolutionDetermines safety problems that need to be

MGMT 210-PPT-1-2-10

correctedTrains students in the post-incident critique processEvaluates the instructor's supervisory and teaching skills

Environmental Issues

The EPA regulates activities that affect the nation’s water, atmosphere and soil.

Instructors and administrators must be familiar

MGMT 210-PPT-1-2-11

Instructors and administrators must be familiar with federal, state and local regulations.

Instructors must ensure that training does not pose an environmental threat.


Report the accident according to organizational policies

Answer any questions asked by investigators

MGMT 210-PPT-1-2-12

y q y g

Complete appropriate forms and reports

Decide whether or not the training evolution can continue

Obtain witness statements




d ib h i i f i i l i

MGMT 210-PPT-1-2-13

describe the supervision of training evolutions.

Review




MGMT 210-PPT-1-2-14



MGMT 210-PPT-2-1-0

Lesson 2-1Course and Curriculum Development


Given information from lecture, discussion, and reading materials, the student will be able to discuss course and curriculum development.

MGMT 210-PPT-2-1-1

discuss course and curriculum development.

Overview

The Four-Step Development ModelIdentifying Training NeedsDesigning a Course or Curriculum

MGMT 210-PPT-2-1-2

Designing a Course or CurriculumImplementing the Course or CurriculumEvaluating the Course or CurriculumEvaluating Testing InstrumentsCourse or Curriculum Revision

Course and Curriculum Development

MGMT 210-PPT-2-1-3

The Four-Step Development Model

Identify Training NeedsDesign a Course or CurriculumImplement the Course or Curriculum

MGMT 210-PPT-2-1-4

Implement the Course or CurriculumEvaluate the Course or Curriculum

Identifying Training Needs

Changes in the operating environment necessitate changes in the training programs offered.

State/provincial or federal mandates

MGMT 210-PPT-2-1-5

/pRequests from stakeholdersProgram evaluationUnexpected eventsChanges in national standards

Identifying Training Needs

Needs AnalysisJob Performance RequirementsTask Analysis

MGMT 210-PPT-2-1-6

Task AnalysisCost/Benefit Analysis

Needs Analysis

Defines current levels of knowledge, skills and attitudes.

Determines whether a need exists and

MGMT 210-PPT-2-1-7

Determines whether a need exists and indicates how training, equipment or procedural change could meet the need.

Needs Analysis

Identifies specific individuals or groups that need the training, equipment or procedural change.

d f h d h h d d

MGMT 210-PPT-2-1-8

Identifies a method to achieve the desired levels of knowledge, skills and attitudes.

Needs Analysis

Operational reports Injury records Accident reports

MGMT 210-PPT-2-1-9

p

Results of promotional

exams

Personnel, operation, and training records

NFPA®

standards

Determining Individual Needs

PretestsAgency Training HistoryPrerequisite Training and Skills

MGMT 210-PPT-2-1-10

Prerequisite Training and SkillsTraining Records

Job Performance Requirements

JPRs specify the knowledge and skills required to perform a specific job.

Instructors must understand the JPRs in order

MGMT 210-PPT-2-1-11

Instructors must understand the JPRs in order to effectively and efficiently develop and present a course or curriculum.

Professional Qualification Standards

Professional qualification standards are often written in the JPR format.

The standard defines and describes the job

MGMT 210-PPT-2-1-12

The standard defines and describes the job, giving instructors and planners an end point around which to design and deliver a training program.

Task Analysis

A task analysis is a detailed review of each physical task that is performed as part of a skill or set of job functions.

MGMT 210-PPT-2-1-13

Conducting a task analysis is the best way for instructors to understand what needs to be included in a training program.

Organizational Models

SequentialCause and EffectModel-Based

MGMT 210-PPT-2-1-14

Cognitive

Complete Classroom Activity 2-1-1

Create a Task Analysis

MGMT 210-PPT-2-1-15

Cost/Benefit Analysis

Each hour of training time is the result of many more hours of development.

The cost of internally developing training

MGMT 210-PPT-2-1-16

The cost of internally developing training should be compared to the cost of obtaining training from an external resource.

Course Design

Group Similar Learning Objectives

Develop Learning Objectives

Identify a Course Goal

MGMT 210-PPT-2-1-17

Create Evaluation Instruments

Create a Course Outline

Sequence Lessons into a Course

Develop Lesson Plans

Group Similar Learning Objectives

Curriculum Design

Identify a Curriculum GoalIdentify Courses for the CurriculumSequence Courses in the Curriculum

MGMT 210-PPT-2-1-18

Sequence Courses in the Curriculum

Implementing the Course

Acquire funding and facilitiesDetermine instructor and student time requirementsFind or train qualified instructors

MGMT 210-PPT-2-1-19

Find or train qualified instructorsCreate training aids and materials

Formative Evaluation

Reviews by peers or supervisors

Consultations with subject-matter experts

Reviews by prospective instructors

MGMT 210-PPT-2-1-20

Reviews by prospective instructors

Any other steps that helps to ensure the quality of the course or curriculum

Implementation Actions

Obtain final course approvalAssemble, create or select training aidsSchedule facilities and equipment

MGMT 210-PPT-2-1-21

q pSelect qualified instructorsEstablish records system or databaseSchedule and announce the course or curriculumSelect studentsPresent a pilot version of the course

The Pilot Course

The Pilot Course

Is the first implementation of a course

Should be taught by experienced instructors

MGMT 210-PPT-2-1-22

Should be taught by experienced instructors

Evaluates the effectiveness of the course or curriculum

Is evaluated based on feedback from instructors, observers, students and student performance

Evaluate the Course or Curriculum

Did students meet learning objectives and course or curriculum goals?

Was training conducted as designed and

MGMT 210-PPT-2-1-23

Was training conducted as designed and within the resources allocated?

Evaluating Course Materials

Did the materials support the objectives?

Were the materials relevant or applicable to job requirements?

MGMT 210-PPT-2-1-24

job requirements?

Evaluating Instructor Performance

Instructors should be evaluated periodically by more experienced instructors.

Student input is also an important component

MGMT 210-PPT-2-1-25

Student input is also an important component of instructor evaluation.

Evaluating Testing Instruments

The purpose of test analysis is to improve the teaching/learning process.

Performing test analysis provides instructors

MGMT 210-PPT-2-1-26

Performing test analysis provides instructors, test developers and organizations with information on test validity and reliability.

Test Validity and Reliability

ValidityThe degree to which a test measures what it was designed to measure.

MGMT 210-PPT-2-1-27

ReliabilityThe consistency of test scores from one measurement to another.

Test Result Analysis

Test Result Analysis analyzes the average scores and indicates the overall difficulty of the examination

MGMT 210-PPT-2-1-28

Test Item Analysis

Test item analysis allows instructors to use systematic methods to assess the quality of an item on a test.

MGMT 210-PPT-2-1-29

Specific measures of quality include:Difficulty IndexDiscrimination IndexDistracter Analysis

Test Revision

Remove poor test items and recalculate the score.

Review the test analysis adjust the test items

MGMT 210-PPT-2-1-30

Review the test analysis, adjust the test items and give the test again.

Teach the lesson again and give the test again.

Course or Curriculum Revision

Evaluation of training programs may determine the need for course or curriculum revision.

Changes in operational standards

MGMT 210-PPT-2-1-31

Changes in operational standards, departmental protocols or new technologies may necessitate revision.

Periodic evaluations of all courses and curricula should be scheduled.


Given information from lecture, discussion, and reading materials, the student will be able to discuss course and curriculum development.

MGMT 210-PPT-2-1-32

discuss course and curriculum development.

Review

The Four-Step Development ModelIdentifying Training NeedsDesigning a Course or Curriculum

MGMT 210-PPT-2-1-33

Designing a Course or CurriculumImplementing the Course or CurriculumEvaluating the Course or CurriculumEvaluating Testing InstrumentsCourse or Curriculum Revision

MGMT 210-PPT-3-1-0

Lesson 3-1Lesson Plan Development – Part I


Given information from lecture, discussion, and reading materials, the student will be able to

discuss lesson plan development.

MGMT 210-PPT-3-1-1

Overview

Laws of LearningLesson Plan Creation

MGMT 210-PPT-3-1-2

Laws of Learning

Readiness

Exercise

Effect

MGMT 210-PPT-3-1-3

Disuse

Association

Recency

Primacy

Intensity

Lesson Plan Creation

Identify the expected learning outcomes.Divide learning outcomes into tasks. Divide tasks into steps.

MGMT 210-PPT-3-1-4

Divide tasks into steps. Write learning objectives for each task.


The First Steps of the Lesson Plan

MGMT 210-PPT-3-1-5


Write test items or design practical assessments for each lesson objective. Determine the sequence in which you will present

MGMT 210-PPT-3-1-6

the material. Conduct research to become more familiar with the topic. Write the outline.


Lesson Outlines and PowerPoint

MGMT 210-PPT-3-1-7


Develop lesson activities. Identify and develop training aids to support instruction.

MGMT 210-PPT-3-1-8

Write or develop the planning components of the lesson plan. Write ancillary and reference material.Write the lesson summary.

Eliminating Bias

Design lesson plans so that a wide variety of students can learn from the materials.

Adhere very closely to the source material

MGMT 210-PPT-3-1-9

y ythat is being taught.

Be very careful in the wording used.

Learning Objectives

Learning ObjectivesAre specific statements that describe desired learning results.

MGMT 210-PPT-3-1-10

Focus on specific, measureable results of instruction.

Represent the knowledge, skills, and attitudes students should acquire by the conclusion of the lesson.

Learning Objectives

Learning ObjectivesAre the foundation for instructional designHelp instructors select content and develop

MGMT 210-PPT-3-1-11

appropriate strategyProvide a basis for measuring and evaluating studentsInform students of expected performanceAllow instructors to be flexible in teaching

The Mager Model

Performance (Behavior) StatementConditions DescriptionStandards Criteria

MGMT 210-PPT-3-1-12

Standards Criteria

General Guidelines

Avoid ambiguous terminology; use action verbs.

Make learning objectives short and focused

MGMT 210-PPT-3-1-13

Make learning objectives short and focused on a single result.

Make learning objectives specific, objective and measurable.

Cognitive Levels of Learning

RememberUnderstandApply

MGMT 210-PPT-3-1-14

ApplyAnalyzeEvaluateCreate


The Cognitive Domain

MGMT 210-PPT-3-1-15

Psychomotor Levels of Learning

Observation

Imitation

MGMT 210-PPT-3-1-16

Adaptation

Performance

Perfection

Affective Domain

Determine in advance the desired change in attitude and plan into the lesson.

Emphasize the correct behavior

MGMT 210-PPT-3-1-17

Emphasize the correct behavior.

Use words such as confident or satisfied.

Affective Domain

Remember that

Affective changes in attitude likely develop alongside cognitive and psychomotor

MGMT 210-PPT-3-1-18

learning.

Learning objectives in the affective domain are difficult to measure objectively.

Lesson Outline Development

After establishing learning objectives, the development of an outline is next in the lesson planning process.

MGMT 210-PPT-3-1-19

Organize the learning objectives into the order that the instructor should teach.

Remember that a lesson outline is not enough for developing a course for another instructor.

Instructional Method Selection

Consider which instructional methods best match the learning objectives.

Employ a number of different instructional

MGMT 210-PPT-3-1-20

Employ a number of different instructional methods when creating lessons in order to reach a variety of learning styles

Lesson Activity Development

MGMT 210-PPT-3-1-21

Whole Group

Discussion

Small Group

DiscussionCase Study Role Play


Creating Learning Activities

MGMT 210-PPT-3-1-22




MGMT 210-PPT-3-1-23

Review

Laws of LearningLesson Plan Creation

MGMT 210-PPT-3-1-24

MGMT 210-PPT-4-1-0

Lesson 4-1Lesson Plan Development – Part II




MGMT 210-PPT-4-1-1

Overview

Ancillary ComponentsLesson Plan ModificationLesson Plan First Use

MGMT 210-PPT-4-1-2

Lesson Plan First UseLesson Plan Evaluation and Revision

Ancillary Components

Enable students to apply, study, and practice the lesson content.

Provide students with background or resource

MGMT 210-PPT-4-1-3

Provide students with background or resource information that is not readily available.

Serve as important instructional adjuncts and helpful guides to reinforce learning.

Information Sheets

Are handouts that address a topic not covered in the course textbook or other course materials

MGMT 210-PPT-4-1-4

Are usually created when:Information is unavailable to studentsResearch on the student’s part is extensive and time-consumingInformation is not available in any text

Skills Sheets

Are appropriate for tasks that require both psychomotor skills and cognitive knowledge.

Divide a task into operational steps critical

MGMT 210-PPT-4-1-5

Divide a task into operational steps, critical criteria, and the key points or steps for completion.

Can be used by students as guides during unsupervised practice.

Worksheets

Are completed during class.

Provide students opportunities to apply rules, analyze and evaluate objects and situations or

MGMT 210-PPT-4-1-6

analyze and evaluate objects and situations, or use multiple skills while completing activities.

Must support the learning objectives.

Study Sheets

Explain the specific areas students will need to study before an exam or certification test.

May be distributed by instructors for students to

MGMT 210-PPT-4-1-7

May be distributed by instructors for students to use during instruction or for students to use as self study aides.

May or may not include a practice test to measure understanding.

Assignment Sheets

Contain information about a specific activity or project that the student is expected to complete without supervision.

MGMT 210-PPT-4-1-8

Activity may occur within the class period or outside of class.

Differ from worksheets in that the assignment is required and will be graded.

Audiovisual Components

Show abstract concepts through the use of charts or diagrams.

Aid memory through the use of eye-catching,

MGMT 210-PPT-4-1-9

y g y g,humorous, or colorful images.

Illustrate real environments using plans, maps, photographs or videos.

Reinforce key points through use of quotes, tables, or figures.


Tie complex ideas together through diagrams, outlines, and headings.

Compare information through the use of charts

MGMT 210-PPT-4-1-10

Compare information through the use of charts and graphs.

Introduce the lesson through the use of a title slide or image.


Illustrate a process through the use of artwork, photographs, or cutaway models.

Define terms or words through the use of

MGMT 210-PPT-4-1-11

Define terms or words through the use of glossaries or examples.

Provide motivation through images, logos, or slogans.

Creating PowerPoint Slides

Keep visual aids simple and easy to read.

Take the time to ensure that all slides look professional and convey their messages

MGMT 210-PPT-4-1-12

professional and convey their messages effectively.

Use typefaces or fonts that are consistent, easy to read, and large enough to read at a distance.


Make text concise.

Create one heading for each slide or image.

MGMT 210-PPT-4-1-13

Keep the backgrounds simple.


Use a background color that contrasts with the color of the text.

Use transition effects sparingly because they

MGMT 210-PPT-4-1-14

Use transition effects sparingly because they can overpower the message.

Use graphs, charts, photographs, and clipart to create interest.


Use parallel structure on each slide or image.

Use one style of transition.

MGMT 210-PPT-4-1-15

Provide handouts of some slides or images.

Lesson Plan Modification

Lessons are modified

To ensure thorough coverage of the course material

MGMT 210-PPT-4-1-16

To include updated information or changes

To address particular groups of students

Lesson Plan Modification

The steps for modifying a lesson plan are the same as steps for creating a lesson plan.

The first step is to identify the learning

MGMT 210-PPT-4-1-17

The first step is to identify the learning outcomes.

Always keep records of modifications to the lesson plan.

Lesson Plan First Use

Instructors should teach from a newly developed lesson plan while evaluating their work in a real world situation.

MGMT 210-PPT-4-1-18

Items that appear effective on paper do not always meet expectations in practice.

Instructors should take notes on what was and was not effective to provide feedback for evaluation.

Lesson Plan Evaluation

Evaluations can help course planners determine how effective a lesson plan has been.

MGMT 210-PPT-4-1-19

Course evaluations evaluate the students, instructor, and lesson plan.

A thorough course evaluation is based on input from students, instructors, and administrators.

Lesson Plan Evaluation

Step 1: Review student test/course scores

Step 2: Review the instructor and course evaluations

MGMT 210-PPT-4-1-20

evaluations

Step 3: Review the lesson plan and determine whether the learning objectives are clear, concise, and attainable

Lesson Plan Revision

Evaluation should provide a definite list of revisions to be made.

Revisions should be reviewed by other

MGMT 210-PPT-4-1-21

Revisions should be reviewed by other instructors, the training agency, administrators, or other experts.

The revised lesson plan should be evaluated after it has been presented for the first time.



discuss lesson plan development

MGMT 210-PPT-4-1-22


Review

Ancillary ComponentsLesson Plan ModificationLesson Plan First Use

MGMT 210-PPT-4-1-23

Lesson Plan First UseLesson Plan Evaluation and Revision

MGMT 210-PPT-6-1-0

Lesson 6-1Test Item Construction – Part I




describe the process of test item construction.

MGMT 210-PPT-6-1-1

p

Overview

Test Instruments

Common Considerations For All Tests

Test Formatting

MGMT 210-PPT-6-1-2

Test Formatting

Test Instruments

Domains of Learning

Level of Learning

Difficulty of the Course

MGMT 210-PPT-6-1-3

Difficulty of the Course

Types of Test Instruments

Written

MGMT 210-PPT-6-1-4

Oral

Performance

Common Considerations

Specific Learning Objectives

Test Formatting

Difficulty of Items

Test Instructions

Ti R i

MGMT 210-PPT-6-1-5

Test Formatting

Arrangement of Items

Level of Cognition

Time Requirements

Test Formatting

Provide space for Student Name and Date

Provide a Test Title

Number Tests

MGMT 210-PPT-6-1-6

Number Tests

Provide Clear Instructions

Provide a Sample Test Item

Test Formatting

Number Items Consecutively

Single Space within Items

Double Space between Items

MGMT 210-PPT-6-1-7

Double Space between Items

State the Point Value

Use Common Terminology

Test Item Arrangement

Test Items

Arrange in a logical order

Group by learning domain outcome or

MGMT 210-PPT-6-1-8

Group by learning domain outcome or type of item

Word items so as not to reveal the correct answer

Level of Cognition

Test items should evaluate

the student’s ability at the

Remember

Understand

MGMT 210-PPT-6-1-9

level that corresponds to

the learning objective.

Apply

Analyze

Evaluate

Create

Test Difficulty

The actual determination of test difficulty does not occur until the test has been taken and scored.

MGMT 210-PPT-6-1-10

Test difficulty is determined by students' performance on individual questions

Tests are appropriately difficult if most of the students answered between 65 and 85 percent of the questions correctly

Test Instructions

Purpose of the test

Method and means for recording answers

Suggestion to guess when undecided on an

MGMT 210-PPT-6-1-11

Suggestion to guess when undecided on an answer

Amount of time available to complete the test

Time Requirements

Problem solving, analyze, create, or evaluate level questions

Short-answer

MGMT 210-PPT-6-1-12

30 to 60

seconds 30 to 60 seconds

Essay

60 seconds for each major point that students must include

Time Requirements

Know the average time it takes a student who knows the material to complete each type of question.

MGMT 210-PPT-6-1-13

Use these estimates to calculate how much time students will need to complete the test.

When time is limited, use sampling to select test items.

Test Bias

Ensure that test questions reflect the materials being tested

Questions associated with bias should be

MGMT 210-PPT-6-1-14

revised

Bias is generally limited to use of regional jargon and differences in terminology

Terminology on the test should reflect the terminology of the students

Written Tests

Written tests measure students’ understanding and retention of technical information

MGMT 210-PPT-6-1-15

information.

Test items are classified as objective or subjective.

Written Test Items

Multiple Choice Short-Answer

Objective Subjective

MGMT 210-PPT-6-1-16

True-False

Matching

Essay

Interpretive Exercise

Multiple Choice Test Items

A multiple-choice test item consists of either a question or an incomplete statement (stem) and a list of choices.

MGMT 210-PPT-6-1-17

Multiple Choice Test Guidelines

Write the stem in the form of a direct question or an incomplete sentence.

Make the statement clear and brief.

MGMT 210-PPT-6-1-18

Use positive questions, but if negative statements are used, emphasize negative words consistently.

Provide at least three plausible, attractive distracters


Phrase choices so that they are parallel with each other and grammatically consistent.

Place correct answers in varied positions.

MGMT 210-PPT-6-1-19

p

Place each choice on a separate, indented line and in a single column.

Begin responses with capital letters when the stem is a complete question.


Begin responses with lowercase letters when the stem is an incomplete sentence.

Do not include choices that are obviously

MGMT 210-PPT-6-1-20

ywrong or intended to be humorous.

Make sure that stems and alternatives do not give clues as to the correct response.

Make all alternatives the same length.


Avoid using “all of the above” or “none of the above”

Do not test trivial information

MGMT 210-PPT-6-1-21

Use correct grammar and punctuation

Multiple Choice Disadvantages

Multiple choice is not well suited to measure some cognitive skills.

It can be very difficult to construct multiple-

MGMT 210-PPT-6-1-22

It can be very difficult to construct multiple-choice test that include different difficulty level test items.

Multiple Choice Disadvantages

It can be difficult to create enough appropriate and plausible distracters for each item.

MGMT 210-PPT-6-1-23

Students who do not know the material may still be able to guess the answer.

True-False Test Items

True statements should be based on facts.

False statements should be created based on common misconceptions of the facts

MGMT 210-PPT-6-1-24

common misconceptions of the facts.

Modified true-false items ask the student to explain why an item is false or to rewrite the item to make it true.

True-False Test Guidelines

Provide clear instructions

Create enough test items to provide reliable results

MGMT 210-PPT-6-1-25

Distribute true and false items randomly

Avoid determiners


Avoid creating items that could trick or mislead students into making a mistake

Avoid double-negative items

MGMT 210-PPT-6-1-26

g

Avoid using personal pronouns such as “you”

Do not use test items that test trivia or obscure facts


Develop test items that require students to think about what they have learned

Avoid unusually long or short test items,

MGMT 210-PPT-6-1-27

y g ,because the length may be a clue

Create brief, simply stated items that deal with a single concept

Avoid quoting information word-for-word from the text

Matching Test Items

Make two parallel columns of words,

phrases, images or a

MGMT 210-PPT-6-1-28

combination of them.

Matching Test Guidelines

Avoid placing prompts and lists on different pages

Separate matching sections into sets of 5

MGMT 210-PPT-6-1-29

p gproblems and responses

Consider preparing one more response than there are prompts

Arrange problem statements and responses into two columns

Matching Test Guidelines

Arrange responses alphabetically, chronologically, or numerically

State whether a response may be used once,

MGMT 210-PPT-6-1-30

p y ,more than once, or not at all

Avoid giving clues to answers

Do not include answers that are obviously wrong

Short Answer Test Items

Write a question for which students must provide a correct

MGMT 210-PPT-6-1-31

response.

Short Answer Test Guidelines

Create short, concise, and direct statements

Avoid long, involved statements with a string of blanks to fill

MGMT 210-PPT-6-1-32

Start with a direct question and change it to an incomplete statement

Make sure that the desired response is a key point in the lesson.

Short Answer Test Guidelines

Arrange the statement in order to place the blanks at or near the end of the sentence

Avoid statements that call for answers with

MGMT 210-PPT-6-1-33

more than one word, phrase, or number

Eliminate unnecessary clues

Write a rubric or detailed answer sheet

Essay Test Items

Essay tests require students to demonstrate the ability to:

Analyze a topic

MGMT 210-PPT-6-1-34

y p

Create a solution to a problem

Evaluate a system or process

Essay Test Items

Essay tests

Eliminate guessing

Test higher cognitive levels

MGMT 210-PPT-6-1-35

Test higher cognitive levels

Are time consuming to administer and to grade

Essay Test Guidelines

Choose essay topics that reflect key aspects of the course material

Create a rubric that establishes clear scoring

MGMT 210-PPT-6-1-36

gguidelines

Provide clear instructions that define how students should respond

Provide sufficient time for students to respond to all questions

Interpretive Exercises

Interpretive Exercises

Consist of introductory material followed by a series of test items.

MGMT 210-PPT-6-1-37

Measure higher level cognitive processes.

Interpretive Exercise Guidelines

Make sure that all introductory material relates to key learning objectives

Apply relevant guidelines for effective item

MGMT 210-PPT-6-1-38

Apply relevant guidelines for effective item construction

Use test items that require the same type of performance that is listed in the test specifications

Interpretive Exercise Guidelines

Create introductory material that is unfamiliar

Make sure that the introductory material does not give away answers to any test items

MGMT 210-PPT-6-1-39

g y y

Make sure that students must read the introductory material to be able to answer test items

Provide enough test items to effectively measure students’ understanding of the material

Oral Tests

Oral tests may consist of open and/or closed questions.

When designing and conducting oral tests

MGMT 210-PPT-6-1-40

When designing and conducting oral tests consider

Development

Validation

Evaluation

Performance Tests

Performance Tests

Measure a student’s proficiency in performing any task involving a psychomotor objective.

MGMT 210-PPT-6-1-41

y g p y j

May be based on a speed standard, a quality standard, or a combination of both standards.

Should be conducted in a controlled environment.

Performance Test Guidelines

Specify performance objectives to be measured

Rate students against a standard, not against

MGMT 210-PPT-6-1-42

g , gthe performance of other students

Provide written instructions that clearly explain the test situation

Try a new performance test on other instructors before administering it to students

Performance Test Guidelines

Use more than one test evaluator

Follow established procedures when administering the test

MGMT 210-PPT-6-1-43

g

Make a score distribution chart after tests have been administered

Rotate team members to every position for team evaluation ratings

Performance Test Advantages

Validity Reliability Skills-Based

MGMT 210-PPT-6-1-44

Student Motivation

Sense of Accomplishment Job Related




describe the process of test item construction

MGMT 210-PPT-6-1-45

describe the process of test item construction.

Review

Test Instruments

Common Considerations For All Tests

Test Formatting

MGMT 210-PPT-6-1-46

Test Formatting

MGMT 210-PPT-7-1-0

Lesson 7-1Test Item Construction – Part II




describe the process of test item construction, including

MGMT 210-PPT-7-1-1

p , gtest planning and scoring.

Overview

Test Planning

Test Scoring Method Selection

MGMT 210-PPT-7-1-2

Test Planning

Determine test purpose and classification

Define learning objectives or learning outcomes

MGMT 210-PPT-7-1-3

Construct appropriate test items

Test Planning

Should the test assess student performance against

a set criterion or rank individual performance

against other students?

MGMT 210-PPT-7-1-4

against other students?

Test Purpose and Classification

Criterion-Referenced TestsCompare performance against appropriate minimum standards

MGMT 210-PPT-7-1-5

Norm-Referenced TestsRate student performance compared to other students


Prescriptive (Pretest)Determines readiness for instruction or placement in appropriate level

MGMT 210-PPT-7-1-6

Formative (Progress)Measures improved progress or problems hampering progress

Summative (Comprehensive)Rates terminal performance


CognitiveTechnical knowledge retention and recall

Psychomotor

MGMT 210-PPT-7-1-7

PsychomotorManipulative skills

AffectiveBehavioral changes in attitude, values or beliefs

Identifying Learning Objectives

Learning objectives reflect the learning outcomes

The test should be based upon the specific

MGMT 210-PPT-7-1-8

The test should be based upon the specific learning or behavioral objectives

These are specific, detailed, and measurable

Constructing Appropriate Test Items

Select proper level of test item difficulty

Determine appropriate number of test items

Eliminate language and comprehension barriers

MGMT 210-PPT-7-1-9

Eliminate language and comprehension barriers

Avoid giving clues to test answers

Match test items to learning objectives

Ensure test usability

Ensure validity and reliability

Selecting Test Item Difficulty

Test item difficulty is determined by whether the test is norm-referenced or criterion-referenced.

Criterion-referenced test items should be designed

MGMT 210-PPT-7-1-10

so that knowledgeable students should be able to pass the test.

Norm-referenced tests items should create a wide spread of scores so that students can be ranked against one another.

Determining Number of Test Items

The number of test items depends on the purpose of the

test, types of test items and desired level of reliability.

MGMT 210-PPT-7-1-11

Guideline Approach

Mathematical Approach

Eliminating Language Barriers

Use words that students would use during training or on the job

Avoid test items that include the following:

MGMT 210-PPT-7-1-12

gHigher reading level than the students possessLengthy, complex, or unclear sentencesVague directionsUnclear graphic materialsObsolete words or terms

Avoiding Giving Clues

Word associations that give away the answer

Plural or singular verbs or use of the words aor an

MGMT 210-PPT-7-1-13

or an

Words that make some answers more likely (such as sometimes) or less likely (such as always or never)

Correct answers that are consistently placed in the same location

Avoiding Giving Clues

Correct answers that are consistently longer than distractorsWords and materials copied verbatim from the

MGMT 210-PPT-7-1-14

textbook

Stereotypical answersTest items that give the answer to other test items

Ensuring Test Usability

Easy to administer

Easy to take

Appropriate length

MGMT 210-PPT-7-1-15

Appropriate length

Cost-effective

Contains multiple testing instruments

Easy to score

Ensuring Validity and Reliability

ValidityTest items should require students to display specific knowledge of the appropriate learning bj i

MGMT 210-PPT-7-1-16

objectives

ReliabilityClear instructionsClear, well-written test itemsSpecific scoring criteria


Select a representative sample of learning objectives

Select enough test items to represent the skills

MGMT 210-PPT-7-1-17

Select enough test items to represent the skills required in the learning objective

Select test-item formats that reduce the potential for guessing


Use only the number of test items that an average student can complete in the available time

MGMT 210-PPT-7-1-18

Determine methods to maintain positive student attitudes toward testing

No test should be biased towards one group of individuals


Written TestsA point value is assigned to each question

Oral Tests

MGMT 210-PPT-7-1-19

Scoring sheets indicate the proper responses and how many points those responses are worth

Performance TestsScoring sheets are created with mandatory tasks; those tasks should be indicated as pass/fail

Overcoming Subjectivity

Train instructors or test evaluators in the steps of the skills to be tested

Develop checklists for each tested skill and

MGMT 210-PPT-7-1-20

Develop checklists for each tested skill and use them for scoring students’ performances




describe the process of test construction including test

MGMT 210-PPT-7-1-21

describe the process of test construction, including test planning and scoring.

Review

Test Planning


MGMT 210-PPT-7-1-22

MGMT 210-PPT-7-2-0

Lesson 7-2Instructor and Course Evaluation




discuss instructor and course evaluations

MGMT 210-PPT-7-2-1

discuss instructor and course evaluations.

Overview

Supervisory Instructor Evaluations

Course Evaluations

Findings from Evaluations

MGMT 210-PPT-7-2-2



Evaluation of instructors is based on observations of both supervisors and students

Many fire and emergency services

MGMT 210-PPT-7-2-3

y g yorganizations have systematic personnel evaluation programs

Instructors must be familiar with the elements of a personnel evaluation program

Personnel Evaluation Program Elements

Forms Guidelines

MGMT 210-PPT-7-2-4

Processes Findings

Supervisory Evaluation Forms

A supervisory evaluation

form should address the

skills and behaviors an

INSERT SCANNED PICUTRE OF THE MICRB EVALUATION FORM HERE

MGMT 210-PPT-7-2-5

skills and behaviors an

instructor should possess.

Evaluation Form Elements

Classroom management considerations

Effective interaction with students

Proper use of communication techniques

MGMT 210-PPT-7-2-6

Proper use of communication techniques

Communication of all learning objectives

Appropriate use of questions

Use of appropriate instructional methods

Performance Evaluation Process

Instructors are evaluated both formally and informally

Informal evaluations occur often and are

MGMT 210-PPT-7-2-7

Informal evaluations occur often and are based on the observation of the instructors in their work environment

Formal evaluations use observations and student evaluations to generate comments and suggestions

Guidelines for Formal Evaluations

Train supervisors properly

Apply in a timely manner

State criteria clearly

MGMT 210-PPT-7-2-8

State criteria clearly

Ensure standards are not discriminatory

Maintain consistency

Maintain thorough records

Ensure objectivity

Guidelines for Formal Evaluations

Always provide positive feedback before offering constructive suggestions for improvement

MGMT 210-PPT-7-2-9

Instructors performing effectively should be praised in front of students and peers

Suggestions for improvements should be made in private

Stages of Instructor Evaluations

Before Evaluation

MGMT 210-PPT-7-2-10

Performance Review

Observation

Accountability After the Evaluation

Supervisors must hold instructors accountable for making improvements

Inexperienced or underperforming instructors

MGMT 210-PPT-7-2-11

Inexperienced or underperforming instructors may benefit from working with a mentor

Instructors unable to meet performance standards may need to be reassigned to non-teaching duties

Course Evaluations

The most immediate form of feedback an instructor receives after a course are student evaluations

MGMT 210-PPT-7-2-12

Students should be asked to evaluate both the instructor and the course

Factors Affecting Student Responses

Background knowledge of the subject

Reason for attending the class

MGMT 210-PPT-7-2-13

Personality of the instructor or student

Preconceived ideas held by the student

Frustration or pressure felt by the student

Instructor Characteristics Evaluated

Preparation Presentation Skills

Knowledge of the Subject

MGMT 210-PPT-7-2-14

Interest and Enthusiasm for

the Subject

Ability to answer students’

questionsInteraction with

students

Instructor Characteristics Evaluated

Concern for Students

Instructor Strengths & Weaknesses

Time Management

Skills

MGMT 210-PPT-7-2-15

Weaknesses Skills

Whether the student would choose to take another class with the instructor

Student Evaluations of Instructors

Student evaluations are most accurate when completed during class and collected immediately

Student anonymity is crucial during the evaluation

MGMT 210-PPT-7-2-16

Student anonymity is crucial during the evaluation process

Completed student surveys should be gathered and forwarded to the instructor’s supervisor

Student Course Evaluations

Reaction

Knowledge

Skill

Environment

Safety

M i l

MGMT 210-PPT-7-2-17

Skills

Attitudes

Materials

Results

Student Course Evaluations

Answers to questions should be given on a continuum

There should always be space for open

MGMT 210-PPT-7-2-18

y p presponse

Students should not be required to sign the form


Identify instructor strengths and weaknesses

Provide an opportunity for supervisors to craft a response to those strengths and

MGMT 210-PPT-7-2-19

p gweaknesses.

Instructor Strengths

Strengths should be cultivated and used to the benefit of the instructor, organization, and the community

MGMT 210-PPT-7-2-20

Helping an instructor build upon strengths is one of the best ways to build the instructor’s confidence

Instructor Weaknesses

Counseling, coaching, and mentoring are useful when helping an instructor improve teaching performance

MGMT 210-PPT-7-2-21

Supervisors should work with instructors to determine steps necessary to overcome weakness

Retraining, professional counseling, or disciplinary actions may be necessary

Evaluating Instructor Performance


MGMT 210-PPT-7-2-22

Complete Classroom Activity 7 2 1

Add Video to slide

MGMT 210-PPT-7-2-23

Reviewing Course Evaluations


MGMT 210-PPT-7-2-24




discuss instructor and course evaluations.

MGMT 210-PPT-7-2-25

Review


Course Evaluations


MGMT 210-PPT-7-2-26


MGMT 210-PPT-8-1-0

Lesson 8-1Supervisory and Administrative Duties



reading materials, the student will be able to discuss

supervisory and administrative duties related

MGMT 210-PPT-8-1-1

p y

to the fire instructor.

Overview

Supervising Other Instructors

Scheduling Resources and Instructional Delivery

Formulating Budget Needs

MGMT 210-PPT-8-1-2


The Purchasing Process

Keeping Training Records

Performing Basic Research


The number of instructors who are directly supervised varies according to the following factors

Size of the Supervisor’s

MGMT 210-PPT-8-1-3

training division or department

pposition and

duties

Abilities of the instructors

Complexity of the specific training environment or

learning scenario


Encourage instructors to participate

Delegate or involve instructors in planning

R h j d f i

MGMT 210-PPT-8-1-4

Respect the judgment of instructors

Teach, enforce, and follow health and safety rules


Coach and mentor

Show consideration for diversity among instructors

MGMT 210-PPT-8-1-5

Acknowledge instructors’ accomplishments

Treat each instructor fairly and equitably


Keep accurate records

Keep lines of communication open

Build and maintain a professional work

MGMT 210-PPT-8-1-6

Build and maintain a professional work environment

Resolve conflicts between instructors and students


Level II Instructors should

remember that they lead by

example and must adhere to

MGMT 210-PPT-8-1-7

example and must adhere to

a standard of ethical,

moral, and legal behavior.

Establish and Communicate Goals

Objectives and timelines can be communicated through

Group meetings where information is shared and

MGMT 210-PPT-8-1-8

discussed

Written task sheets, including personalized assignments and deadlines

Supervisors should involve instructors in the process

Promote Professional Development

It is important that instructors maintain and develop professional skills

Instructors benefit from having a supervisor

MGMT 210-PPT-8-1-9

Instructors benefit from having a supervisor who actively recommends or provides opportunities for development

Empower Instructors

Instructors should feel that

they have authority over

MGMT 210-PPT-8-1-10

their own jobs

Celebrate Accomplishments

When objectives are met or

significant accomplishments are

MGMT 210-PPT-8-1-11

reached a supervisor should

celebrate the accomplishment as

soon as possible

Offer Incentives for Performance

Make public acknowledgements of accomplishments

Hold group gatherings or parties to encourage

MGMT 210-PPT-8-1-12

Hold group gatherings or parties to encourage unit cohesiveness

Make positive statements about the skills and abilities of instructors

Make appropriate comments on performance evaluation

Resolve Conflicts

Focus on the conflict, not the individuals involved

Speak to the parties of a dispute individually and privately

MGMT 210-PPT-8-1-13

privately

Allow the parties of the dispute to express themselves freely

Make decisions about the conflict based upon policies and procedures

Maintain Positive Example/Attitude

A supervisor should always express a positive attitude toward all personnel

MGMT 210-PPT-8-1-14

Attitude should be reflected through personal actions that set an example for others

Scheduling Resources and Training

Instructors are often responsible for scheduling resources and instructional delivery

MGMT 210-PPT-8-1-15

Training resources include all elements needed to present a course or curriculum

Creating a Schedule

Assess factors that affect scheduling

Determine needs

Determine requirements

MGMT 210-PPT-8-1-16

Determine requirements

Determine availability

Creating a Schedule

Coordinate training

Create a schedule

Publish the schedule

MGMT 210-PPT-8-1-17

Publish the schedule

Revise the schedule as needed

Factors that Affect Scheduling

Training Requirements

Physical Resources

Instructor Availability

Minimum Staffing Levels

Budgetary Considerations

MGMT 210-PPT-8-1-18

Instructor Availability

Student Availability Environment

Determine Needs

Conduct a needs analysis to determine the types of

services that are currently delivered and compare

to the services that are likely to be needed in the future

MGMT 210-PPT-8-1-19

Factors that Affect Need

Equipment or Technology

Personnel

MGMT 210-PPT-8-1-20

Legal Concerns

Observations at Emergency Incidents

Requests from the Public

Determine Needs

Any training that a supervisor considers scheduling should be based on priority levels

Priority 1: Training that is mandated by federal,

MGMT 210-PPT-8-1-21

y g y ,state, and provincial agencies

Priority 2: Training that is required to meet goals determined by the organization and jurisdiction

Priority 3: Training that is not mandated or required but would benefit the agency or service

Determine Requirements

Specific topics to be taught

Certification or testing criteria

The minimum number of contact or teaching

MGMT 210-PPT-8-1-22

The minimum number of contact or teaching hours

Determine Availability

Supervisors should determine the availability of instructors and facilities

Training should be scheduled when students are

MGMT 210-PPT-8-1-23

Training should be scheduled when students are most likely to be able to attend

Coordinate Training

Supervisors should

contact other fire and

emergency services

MGMT 210-PPT-8-1-24

g y

organizations to see if

joint training sessions

can be scheduled

Create a Schedule

Create a draft schedule based on needs, requirements and availability

Include alternative dates and remain as flexible

MGMT 210-PPT-8-1-25

Include alternative dates and remain as flexible as possible

Distribute a table or chart of courses, dates and instructor assignments to the teaching staff

Publish the Schedule

Distribute at all worksites and facilities

Send to a predetermined e-mail list

MGMT 210-PPT-8-1-26

Post to the training organization’s website

Manage using computer calendar programs

Revise the Schedule

Both short- and long-term training schedules may require revisions

Schedule changes must be communicated to

MGMT 210-PPT-8-1-27

Schedule changes must be communicated to the teaching staff and members of the organization

Reasons to Revise the Schedule

Instructor availability conflicts

Inclement weather

Lack of funds

MGMT 210-PPT-8-1-28

Lack of funds

Lack of equipment or materials

Reasons to Revise the Schedule

Lack of facilities

Unforeseen situations

Changes in the amount of time required to

MGMT 210-PPT-8-1-29

Changes in the amount of time required to present courses

Creation of newly mandated courses


Supervisors may be responsible for formulating the budgetary needs for training

Requests should provide an accurate estimate

MGMT 210-PPT-8-1-30

Requests should provide an accurate estimate of costs and justification for the expense

Typical Operating Expenses

Instructor Pay

Equipment

Course Materials

Audiovisuals

Travel Expenses

Training Ground

MGMT 210-PPT-8-1-31

Course Materials

Student Manuals

Training Ground Expenses

Capital Budget Requests

Sometimes a course will require a large, one-time purchase

These requests usually require greater

MGMT 210-PPT-8-1-32

justification

Organizations may have special procedures for capital budget requests

Agency Budget Policies

Supervisors must

become familiar

with their agency’s

MGMT 210-PPT-8-1-33

with their agency s

budgetary policies

Sources of Funding

All governmental jurisdictions require revenue in order to provide necessary services

Revenue typically comes from a combination

MGMT 210-PPT-8-1-34

Revenue typically comes from a combination of property, sales and income taxes

Other sources of public funding include trust funds, bond sales, gifts, and fundraising

Sources of Funding

Many organizations

supplement their

general budgets with

MGMT 210-PPT-8-1-35

general budgets with

grants and charitable

contributions

Budget Request Justification

Financial history

Actual cost of materials

Thi d t l ti

Contractual requirements

New programs or services

T i i fi t

MGMT 210-PPT-8-1-36

Third party evaluations

Training mandates

Training refinements

Injury reports and fire losses


Supervisors

May be responsible for purchasing materials or equipment specific to the organization’s training

MGMT 210-PPT-8-1-37

needs

Must ensure that the organization receives the exact materials that were budgeted for/ordered

Must keep purchasing expenses within the approved budget

Determining Funding Sources

The first step in the purchasing process is to determine sources of funding

Primary funding sources include operating

MGMT 210-PPT-8-1-38

Primary funding sources include operating funds, capital funds, grants and leases or lease/purchases

Determining Purchasing Needs

Purchasing needs should have been determined during the budget process

Actions for when the time comes to purchase

MGMT 210-PPT-8-1-39

Actions for when the time comes to purchase:Review the standards/regulationsReview current resourcesDetermine the amount of funds available

Contacting Vendors

Supervisors should contact

available vendors for the

resources they need to purchase

MGMT 210-PPT-8-1-40

resources they need to purchase

Purchase Orders

To release funds for purchase, a supervisor will complete a purchase order

The purchase order includes

MGMT 210-PPT-8-1-41

The purchase order includesWhat resources are being purchased and the costThe name of the entity from which resources are being purchasedAuthorization and account numbersAppropriate signatures to approve the request


Supervisors may be

involved in developing

and retaining training

MGMT 210-PPT-8-1-42

and retaining training

records or record

keeping systems

Training Information

Course Name

Dates and Hours

Instructor Name(s)

Topics Taught

Lesson Plan and Materials

MGMT 210-PPT-8-1-43

( )

Student Attendance Record

Evaluation Scores

Course Evaluations


Required Training and Certifications

Missed Training

Performance Deficiencies

MGMT 210-PPT-8-1-44

Performance Deficiencies

Recommended Remedial Training

Other Information Deemed Appropriate


Instructors collect

information for some or all

types of training

Daily Training

Company Level Training

Organizational Training

MGMT 210-PPT-8-1-45

types of trainingOrganizational Training

Individual Self-Study

Individual Training

Special Training

Degrees and Certificates Degrees and Certificates Attained

Record Management Systems

Retain documents so that authorized parties may review them when necessary

Help determine if required mandates are being

MGMT 210-PPT-8-1-46

Help determine if required mandates are being met and identify gaps in training

Provide information to help administrators plan and schedule

Provide a degree of protection against lawsuits

Record Auditing Procedures

Record auditing is the process by which a document can be traced back to the individual who created it

MGMT 210-PPT-8-1-47

The process is made easier by including Name of the individual who completed the recordNames of other individuals who contributed information Dates and times that the records were completed

Legal Requirements for Records

Retention length of records and reports

Privacy of records and reports

MGMT 210-PPT-8-1-48

Public access to records and reports

Open records act exemptions


Research benefits training

programs in many ways.

MGMT 210-PPT-8-1-49

Information gathered can

be used to support a

proposal or to answer

open-ended questions.

Basic Steps for Conducting Research

Identify the topic

List all similar topics

MGMT 210-PPT-8-1-50

List types of data that may support the topic

List the sources for each information type

Read and analyze the gathered information

Data Collection

Expert OpinionsTrendsModels

AnalysesIndustry StandardsLegal Requirements

MGMT 210-PPT-8-1-51

Similar SituationsStatisticsExamples

g qRecommended PracticesSources of Supplemental Funds

Information Sources

Internet

Government Agencies

Professional Organizations

Standards Organizations

MGMT 210-PPT-8-1-52

Libraries

Education Institutions

Vendors/Manufacturers

Nonprofit Organizations

Level of Source Material

MGMT 210-PPT-8-1-53

Primary Secondary Tertiary

Quality of Source Material

Credibility/Authority

Accuracy

Reasonableness

MGMT 210-PPT-8-1-54

Reasonableness

Support

Review

Reference Material Citations

Citations are used in research papers, books, articles, and electronic media to indicate the source of quotes, statistics, and other information

MGMT 210-PPT-8-1-55

that is not original to the writer

Two popular citation styles used in the fire and emergency services are Chicago and APA



reading materials, the student will be able to discuss

supervisory and administrative duties related

MGMT 210-PPT-8-1-56

p y

to the fire instructor.

Review


Scheduling Resources and Instructional Delivery


MGMT 210-PPT-8-1-57





MGMT 210-PPT-9-1-0

Lesson 9-1Training Program Evaluation




di i i l i

MGMT 210-PPT-9-1-1

discuss training program evaluation.

Overview

Evaluation Methodology

Evaluation Plans

MGMT 210-PPT-9-1-2

Evaluation Results

Types of Evaluation

Goal-Based Evaluations

MGMT 210-PPT-9-1-3

Process-Based Evaluations

Outcome-Based Evaluations


Set criteria by identifying the problems or issues that must be addressed

Determine what data is currently available for

MGMT 210-PPT-9-1-4

Determine what data is currently available for evaluation

Determine what new data will have to be collected for evaluation


Identify the evaluation model that will be most effective

Determine the amount of time necessary to

MGMT 210-PPT-9-1-5

Determine the amount of time necessary to complete the evaluation

Identify the personnel who will be responsible for making the evaluation


Conduct the evaluation

Analyze the results of the evaluation and d l l i b d h l

MGMT 210-PPT-9-1-6

develop conclusions based on the results

Establish multiple sets of reasonable actions to take based on the results


Prepare a specialized final report for each audience that will be affected by the outcome of the evaluations

MGMT 210-PPT-9-1-7

of the evaluations

Develop revision strategies and priorities

Categories of Evaluation

Qualitative Quantitative

MGMT 210-PPT-9-1-8

Qualitative (subjective)

Quantitative (objective)

Evaluation Plans

Contain the step-by-step process for the various types of evaluations used by the jurisdiction

MGMT 210-PPT-9-1-9

Ensure that the process is conducted in the same manner each time it is implemented

Kirkpatrick’s Evaluation Model

Results

B h i

MGMT 210-PPT-9-1-10

Behavior

Learning

Reaction

NFA Evaluation Model

MGMT 210-PPT-9-1-11

Retrospective Evaluations

Encompass overall effectivenessIdentify the need for new programsProvide a comparison with the past outcomes

MGMT 210-PPT-9-1-12

Provide a comparison with the past outcomesDocument the achievements of the programDocument students’ habitsRely on evidence

Observation of Instructors

Observation can be used to gather evidence during individual classes of a course

MGMT 210-PPT-9-1-13

With feedback received during instruction, instructors can change or modify instructional methods to meet the needs of the class

Supervisor Surveys

Completed by company officers, supervisors, and division managers within the organization

MGMT 210-PPT-9-1-14

Constructed to assess how a student has applied the learned knowledge, skills, or behavioral habits while on duty

Student Learning Surveys

Determine how

students are using the

MGMT 210-PPT-9-1-15

information learned

Organizational Evaluations

Are intended to compare an organization’s mission statement with its output

MGMT 210-PPT-9-1-16

May be conducted internally, by an outside agency, or by a third party

Help the leadership of the department improve services and adjust to potential changes

Performance Measurement

Used to

evaluate success

MGMT 210-PPT-9-1-17

of training

programs and

to justify budgets

Course and Design Evaluation

Instructors must always be alert to ways of improving or updating instruction

MGMT 210-PPT-9-1-18

Instructional content and methods must remain flexible in order to serve intended purpose

Test scores are only one form of evidence to support or refute the success of the learning process

Course and Design Evaluation

Test scores are only one form of evidence to support or refute the success of the learning process

MGMT 210-PPT-9-1-19

process

Data collection must be accurate

Instructors must plan the approach by thinking through and answering questions

Evaluation Results

Determine areas for improvement based on student outcomes

MGMT 210-PPT-9-1-20

Document and report results as appropriate

Identify actions to correct deficiencies

Customer Approval Rating

Organizational rating

that adds a

MGMT 210-PPT-9-1-21

quantifiable ranking to

qualitative evaluations


Leadership Problem Solving

MGMT 210-PPT-9-1-22




di i i l i

MGMT 210-PPT-9-1-23

discuss training program evaluation.

Review


Evaluation Plans

MGMT 210-PPT-9-1-24

Evaluation Results

MGMT 210-PPT-9-2-0

Lesson 9-2Training Program Administration – Part I



discuss training program administration.

MGMT 210-PPT-9-2-1

g p g

Overview

Record-Keeping SystemsDisclosure of InformationDevelopment of Forms

MGMT 210-PPT-9-2-2

pAgency Level Needs AnalysisDevelopment of Training Policies, Procedures, and Guidelines

Record Keeping Systems

Encompass a wide range of information-organizing skills

Statistical analysis

MGMT 210-PPT-9-2-3

Strategic handing of data Evaluations to establish trends and relationships

Are used to formulate training policies, mandate safe practices, or change procedures


Manual SystemsInvolve physical collection, organization, storage, preservation and disposal of hard copy records

d t i l

MGMT 210-PPT-9-2-4

and materials

Automated SystemsElectronic storage of data in password-protected computer databases


Any record keeping system must perform the following functions

MGMT 210-PPT-9-2-5

Organize Records Index Records

Allow only authorized staff to access records

Ensure that records can be easily found


Record-keeping systems consist of more than the software applications or

People

Policies

Procedures

MGMT 210-PPT-9-2-6

software applications or filing systems designed to manage the records

Procedures

Tools

Technology

Training and Education

Maintenance


Instructors must ensure that components work together and that the system functions according to agency policies

MGMT 210-PPT-9-2-7

Consult applicable standards for record-keeping as a model for record-keeping policies

Disclosure of Information

Disclosure is a legal term referring to the act of giving out information

Some records can be released with no

MGMT 210-PPT-9-2-8

Some records can be released with no authorization

Documents for investigationInstructional materials

FERPA states that test records cannot be disclosed without consent from the student

Development of Forms

Standardized forms ensure that information is consistent and complete

MGMT 210-PPT-9-2-9

Computers make the creation of forms very easy

Forms should have fields that are consistent with other organizational forms

Agency Level Needs Analysis

Level III instructors identify and address needs projected for the future

The process may involve other agencies that can

MGMT 210-PPT-9-2-10

p y gprovide training, facilities or equipment when funding is limited

Interested employees should be included in the process

Conducting Committee Meetings

Schedule meetings at appropriate intervals

Invite those most relevant to the project or analysis

MGMT 210-PPT-9-2-11

Always have an agenda

Report findings to higher ranking members of the jurisdiction

Data Organization and Interpretation

The most important outcome of interpretation is that the information

Tells a story

MGMT 210-PPT-9-2-12

Supports a proposalAnswers an analysis question

Interpreted data can be correlated with changes in training policies or courses from the previous year

Training Policies

A policy is a guiding principle that organizations develop, adopt, and use as a foundation for decision-making

MGMT 210-PPT-9-2-13

Documents must contain information that is current and appropriate

Effective Training Policies

Are written using language that is simple, concise and respectful

Are adopted through a process that provides

MGMT 210-PPT-9-2-14

p g p preasoned feedback

Are explicitly supported by the organization’s administration and training manager

Procedures and Guidelines

ProcedureIdentifies the steps that must be taken to fulfill the intent of a policy and is written to support a

li

MGMT 210-PPT-9-2-15

policy

GuidelineIdentifies a general philosophyMay be part of a policy or exist independently

Needs Determination

Level III instructors must know the steps to identify a need for new policy, procedure, or guideline

MGMT 210-PPT-9-2-16

Develop a Draft Document

Determine whether a policy, procedure, or guideline is the most appropriate for the issue or incident

Always select the least restrictive type of document

MGMT 210-PPT-9-2-17

Use similar existing policies, procedures, or guidelines as references when possible

Follow agency or national models for format, terminology, and organization

Develop a Draft Document

Include the date the document becomes effective if adopted

List the person or unit responsible for managing the li d id li

MGMT 210-PPT-9-2-18

new policy, procedure, or guideline

Conduct research on the legality of the new policy, procedure, or guideline

Submit the draft document to the organization’s legal counsel for review

Submit Draft for Review

Provide personnel an opportunity to respond with feedback and input on any document that affects them

MGMT 210-PPT-9-2-19

Provide a comment period for the draft document and review all comments

Evaluate comments and amend the draft document as necessary

Adopt Policy

The appropriate manager or administrator endorses the document

When a broader policy affects the entire

MGMT 210-PPT-9-2-20

p yorganization, the endorsement must come from the organization’s chief executive officer

An endorsement signals to organizational personnel that the policy, procedure, or guideline has official sanction

Publish Document

Anyone affected by the policy, procedure, or guideline must be

MGMT 210-PPT-9-2-21

ginformed of the change.

Face-to-face meetings may be necessary when

the changes are significant.

Implement Document’s Contents

Improper implementation of policies, procedures and guidelines is the primary cause for noncompliance

MGMT 210-PPT-9-2-22

Personnel must know the reason for the change, understand the benefits of the change, and accept the change as an improvement

Implementation must be consistent, fair, and documented

Evaluate Effectiveness

New policies must be monitored for effectiveness

If observations indicate that the new

MGMT 210-PPT-9-2-23

requirement does not provide the necessary change, it should be evaluated.

A periodic review should be performed




MGMT 210-PPT-9-2-24


Review

Record-Keeping SystemsDisclosure of InformationDevelopment of Forms

MGMT 210-PPT-9-2-25

Development of FormsAgency Level Needs AnalysisDevelopment of Training Policies, Procedures, and Guidelines

MGMT 210-PPT-10-1-0

Lesson 10-1Training Program Administration – Part II



discuss training program administration

MGMT 210-PPT-10-1-1


Overview

Standards that Influence TrainingEquipment Purchasing PoliciesHuman Resources Management

MGMT 210-PPT-10-1-2

Human Resources ManagementPersonnel Evaluations

Standards that Influence Training

Training managers often make decisions based on standards

Adopting a standard gives it formal authority

MGMT 210-PPT-10-1-3

Adopting a standard gives it formal authority in the organization and allows the training manager to enforce the requirements set forth in the standard

Equipment Purchasing Policies

A bid process is conducted to ensure the organization is getting a good product at a reasonable price

MGMT 210-PPT-10-1-4

The bid process begins with performing research about the capital resource in question

Equipment Purchasing Policies

Survey other jurisdictions

Review standards and regulations

MGMT 210-PPT-10-1-5

Review industry trends

Compare various products

Determine equipment compatibility

Developing a Request for Proposal

An RFP defines the needs of the organization and allows manufacturers or their authorized distributors to decide whether they can meet purchasing specifications

MGMT 210-PPT-10-1-6

decide whether they can meet purchasing specifications


The RFP must include:Specific schedule outlineBid dates and delivery dates

MGMT 210-PPT-10-1-7

yProvisions for supplying equipment for scheduled evaluationsTraining dates for benefit of maintenance technicians and instructors


The RFP process reduces the number of bidders that are capable of meeting the purchasing specifications

MGMT 210-PPT-10-1-8


The common reasons a RFP is rejected include:Failure to meet delivery deadlines

Inability to provide the required performance

MGMT 210-PPT-10-1-9

y p q pbonds

Lack of established financial support to complete the contract

Documented history of contract violations


Instructors should also review any product data made available to them

Features

Durability

MGMT 210-PPT-10-1-10

Lifecycle Cost

Maintenance Requirements

Infrastructure

Create Purchasing Specifications

Vendor attendance at pre-bid meetingsWarrantiesLiability of performance bonds

MGMT 210-PPT-10-1-11

Liability of performance bondsSpecified delivery timesPayment schedulesFinancial statements

Bid Evaluation and Contract Award

Instructors evaluate and score the qualified bids based on the original purchasing specifications

MGMT 210-PPT-10-1-12

A recommendation is made about which bidder should be awarded a purchase contract


Creating Purchase Specifications

MGMT 210-PPT-10-1-13

Human Resources Management

Level III instructors have fewer teaching responsibilities and more supervisory duties

These duties include certain human resource

MGMT 210-PPT-10-1-14

These duties include certain human resource responsibilities

Selecting instructional staffEnsuring instructors maintain qualificationsAdvertising positions

Instructional Staff Selection

Selecting competent instructional staff is a key management function

Selection criteria will vary depending on the

MGMT 210-PPT-10-1-15

Selection criteria will vary depending on the instructor role

Instructor Qualifications

Instructors tend to specialize in certain areas

Instructors must have credibility with the personnel being trained

MGMT 210-PPT-10-1-16

personnel being trained

Instructors must be effective communicators

Instructors must be qualified and certified

Position Advertising

The Level III Instructor must advertise or market the position to prospective candidates

An organization must advertise policies or

MGMT 210-PPT-10-1-17

An organization must advertise policies or other requirements that limit the number of applicants

Personnel Evaluations

Personnel evaluations

Begin with an initial meeting between the Level III Instructor and the new staff

MGMT 210-PPT-10-1-18

Establish performance standards, expectations of the training division, and a probationary period

May include praise for meeting expectations

Make new instructors aware of 360-degree feedback

Personnel Evaluations

Performance evaluations ask for feedback from other instructors about the new hire’s performance

MGMT 210-PPT-10-1-19

Analyze the information from other instructors and form a plan to improve any behaviors that need correction


The Mock Interview

MGMT 210-PPT-10-1-20




MGMT 210-PPT-10-1-21

Review

Standards that Influence TrainingEquipment Purchasing PoliciesHuman Resources Management

MGMT 210-PPT-10-1-22

Human Resources ManagementPersonnel Evaluations