Formative Evaluation Findings Formative Evaluation Findings fromfrom

Teachers on the GoTeachers on the GoPilot Project: A PT3 InitiativePilot Project: A PT3 Initiative

QuickTime™ and aGraphics decompressor

are needed to see this picture.

SITE 2004

Stephanie Stigler Bales, Ph.D.Stephanie Stigler Bales, Ph.D.Center for Urban StudiesCenter for Urban StudiesWayne State UniversityWayne State UniversityStephaniebales@wayne.eduStephaniebales@wayne.edu

Marie Colombo, M.A.Marie Colombo, M.A.Center for Urban StudiesCenter for Urban StudiesWayne State UniversityWayne State Universitym.colombo@wayne.edum.colombo@wayne.edu

Presenters

Mary Waker, Ed.D.Mary Waker, Ed.D.College of EducationCollege of EducationWayne State UniversityWayne State Universitym.waker@wayne.edum.waker@wayne.edu

Donna CarrollDonna CarrollCollege of EducationCollege of EducationWayne State UniversityWayne State Universitydecarroll@wayne.edudecarroll@wayne.edu

Wayne State UniversityWayne State UniversityUrban university Urban university located in Detroit, MIlocated in Detroit, MIServes primarily Serves primarily commuter studentscommuter studentsMajority of the 34,000 Majority of the 34,000 students attend students attend classes partclasses part--time, time, balancing school & balancing school & workwork

““Teachers on the Go” Teachers on the Go” ProgramProgram

Pilot laptop & ePilot laptop & e--portfolio programportfolio programCreated in response to:Created in response to:

Instructors’ lack of technology resources and Instructors’ lack of technology resources and trainingtrainingStudents’ lagging technology skills and Students’ lagging technology skills and anxiety about technology useanxiety about technology use

EvaluationEvaluation

Center for Urban StudiesCenter for Urban StudiesPT3 EvaluatorsPT3 Evaluators

Purpose of initial formative evaluation Purpose of initial formative evaluation Evaluate technology integration in laptop Evaluate technology integration in laptop coursescoursesEvaluate technology training & support needs Evaluate technology training & support needs for students and facultyfor students and faculty

Overview of Laptop ClassesOverview of Laptop Classes

5 courses from WSU’s College of 5 courses from WSU’s College of Education, Teacher Education DivisionEducation, Teacher Education DivisionAll students & instructors were given All students & instructors were given

AppleApple iBooks iBooks Course specific softwareCourse specific softwareTechnology training & supportTechnology training & support

2 classes also created e2 classes also created e--portfoliosportfolios

Evaluation PlanEvaluation PlanClassroom Classroom observations observations Focus groups with Focus groups with students and students and instructorsinstructorsStudent online preStudent online pre--and postand post--surveysurvey

Classroom ObservationsClassroom ObservationsTo better understand technology activities in To better understand technology activities in laptop classroomslaptop classroomsTo develop valid data collection instruments (i.e., To develop valid data collection instruments (i.e., focus group scripts, student postfocus group scripts, student post--survey)survey)Observations:Observations:

Instructors & studentsInstructors & studentsType of technology used Type of technology used Collection of syllabiCollection of syllabiMeet with instructorsMeet with instructors

Focus GroupsFocus GroupsStudents:Students:

1 focus group per 1 focus group per classclass5 focus groups, 39 5 focus groups, 39 students totalstudents totalDuration: 1 hourDuration: 1 hourStudent volunteers Student volunteers participatedparticipatedBefore, during, or after Before, during, or after classclass

Instructors:Instructors:1 focus group with 5 1 focus group with 5 instructorsinstructors

Student SurveysStudent SurveysPrior technology Prior technology experienceexperienceLaptop use during & Laptop use during & outside classoutside classSatisfaction with technical Satisfaction with technical training/supporttraining/supportChallenges experiencedChallenges experiencedComputer competencyComputer competencyOutcomes experiencedOutcomes experiencedRecommended Recommended technology changestechnology changes

FindingsFindings

Impact of Technology: Impact of Technology: StudentsStudents

Students' Pre- and Post-Survey Reports of Being "Competent/Very Competent" with

Specific Technology

48%

26%

71%

30%

22%

54%

81%

47%

92%

63%

36%

83%

0% 20% 40% 60% 80% 100%

Computer operatingsystem

Desktop publishing

Presentation software

Online research

Dataorganizing/analyzing

Online coursemanagement system

Post (n=54)Pre (n=99)

Technology Integration Technology Integration DuringDuring Class TimeClass Time

Percent of Ways Students Used Laptops in Class (n=48)

48%

46%

79%

90%

67%

81%

19%

0% 20% 40% 60% 80% 100%

Note taking

Presentations/peer teaching

Using software

Using the internet

Small group work

Individual work

Taking an exam

Technology Integration Technology Integration OutsideOutsideClass Time for Laptop CoursesClass Time for Laptop Courses

How Students Used Laptops Outside of Class(n=46)

28%

2%

83%

20%

78%

48%

83%

78%

0% 20% 40% 60% 80% 100%

Creating e-portfolio

Designing a website

Using the internet

Using iMovie

Using software

Preparing presentation

E-mailing students/ instructor

Typing assignments/papers

Technology Integration in Technology Integration in KK--12 12 ClassroomsClassrooms

Percent of How Students Used Laptops in K-12 Classrooms

(n=14)

29%

43%

79%

21%

50%

36%

0%

0% 20% 40% 60% 80% 100%

Observations/note taking

Presentations

Creating lesson plans

Showing cooperating teacher how to usecomputer/software

Having students use laptop/software

Downloading pictures of class

Making iMovie of class

Training & Support: Training & Support: Student Satisfaction RatingsStudent Satisfaction Ratings

Percent of Student Satisfaction with Technical Training/SupportAgree/Strongly Agree

36%

41%

14%

14%

56%

17%

65%

71%

0% 10% 20% 30% 40% 50% 60% 70% 80%

There was enough training on how to use theMac. (n=53)

Training sessions/workshops have NOT beenuseful to me for this course. (n=52)

Had to learn to use the laptops/software onown through “trial and error.” (n=52)

COE technology staff is patient with me when Ihave a question. (n=52)

I feel lost in this course because I have not hadenough tech support/training. (n=51)

Training/support has helped me to learn how touse the software for this course. (n=52)

I feel the COE staff is NOT available enough formy computer needs. (n=52)

There has been enough training/support to helpme with my e-portfolio. (n=12)

Students’ Challenges with Students’ Challenges with TechnologyTechnology

Percent of Challenges Students Experienced (n=49)

35%

33%

31%

37%

12%

27%

43%

27%

6%

45%

4%

18%

12%

18%

0% 10% 20% 30% 40% 50%

Using the Ma c

Us ing the so ftwa re /progra m s

Try ing to le a rn bothte chno logy & course conte nt

Printing

Se nding/re ce iv ing e -m a ils

Sa v ing to d isk /ha rd dr iv e

Conne cting to the inte rne t

La ptop be ing too s low (note nough m e m ory )

Us ing Bla ck boa rd

Incom pa tib ility be twe e n Ma ca nd o the r com pute rs

Ne e ding to re p la ce la ptop

Not ha v ing a cce ss to ne e de de quipm e nt (e .g ., sca nne r,

d ig ita l ca m e ra , pr inte r)

Not e nough tra ining/supportto do a ss ignm e nts

F e e ling o v e rwhe lm e d by thea m ount o f work inv o lv e d

Recommendations for Laptop Recommendations for Laptop ClassesClasses

Students' Suggested Changes for Future PT3 Courses(n=53)

25%

72%

63%

21%

43%

32%

4%

26%

0% 20% 40% 60% 80%

Inform students ahead of time that laptops willbe used in this course.

Allow students to keep laptops untilgraduation.

Provide better access to printers.

Provide more memory on laptops.

Provide more basic Mac/laptop training.

Provide more training in using the software forthis course.

Provide more support from COE staff.

Incorporate technology into more COE classes.

NETSNETS--TT

Meeting the StandardsMeeting the Standards

Technology Operations & Technology Operations & ConceptsConcepts

Teachers demonstrate a sound understanding of technology operations & concepts

Introducing New Concepts

TechnologyTechnology--enhanced enhanced EnvironmentsEnvironments

Teachers plan & design effective learning environments & experiences supported by technology

Planning & Designing Learning Planning & Designing Learning Environments & Experiences Environments & Experiences

Learning EnvironmentsLearning Environments

Teachers implement curriculum plans that include methods and strategies for applying technology to

maximize student learning

Teaching, Learning, & the Teaching, Learning, & the CurriculumCurriculum

New Strategies in the New Strategies in the ClassroomClassroom

Literacy StrategyLiteracy Strategy

Teachers apply technology to facilitate a variety of effective assessment & evaluation strategies

Assessment & EvaluationAssessment & Evaluation

Assessment & Evaluation Strategies

Teachers use technology to enhance their productivity & professional practice

Productivity & Professional Productivity & Professional PracticePractice

Collaboration & ResourcesCollaboration & Resources

Professional DevelopmentProfessional Development

Teachers understand the social, ethical, legal, and human issues surrounding the use of technology in PK-12 schools

and apply those principles in practice

Social, Ethical, Legal & Human Social, Ethical, Legal & Human IssuesIssues

Digital EquityDigital Equity

Diverse Learning ActivitiesDiverse Learning Activities

Our graduates…Our graduates…

The Effective Urban Educator: Reflective, The Effective Urban Educator: Reflective, Innovative Innovative And And

CommittedCommitted To DiversityTo Diversity

Thank You!Thank You!