Formative Evaluation Findings from Teachers on the Go Pilot Project

  • View
    214

  • Download
    2

Embed Size (px)

Text of Formative Evaluation Findings from Teachers on the Go Pilot Project

  • Formative Evaluation Findings Formative Evaluation Findings fromfrom

    Teachers on the GoTeachers on the GoPilot Project: A PT3 InitiativePilot Project: A PT3 Initiative

    QuickTime and aGraphics decompressor

    are needed to see this picture.

    SITE 2004

  • Stephanie Stigler Bales, Ph.D.Stephanie Stigler Bales, Ph.D.Center for Urban StudiesCenter for Urban StudiesWayne State UniversityWayne State UniversityStephaniebales@wayne.eduStephaniebales@wayne.edu

    Marie Colombo, M.A.Marie Colombo, M.A.Center for Urban StudiesCenter for Urban StudiesWayne State UniversityWayne State Universitym.colombo@wayne.edum.colombo@wayne.edu

    Presenters

    Mary Waker, Ed.D.Mary Waker, Ed.D.College of EducationCollege of EducationWayne State UniversityWayne State Universitym.waker@wayne.edum.waker@wayne.edu

    Donna CarrollDonna CarrollCollege of EducationCollege of EducationWayne State UniversityWayne State Universitydecarroll@wayne.edudecarroll@wayne.edu

  • Wayne State UniversityWayne State UniversityUrban university Urban university located in Detroit, MIlocated in Detroit, MIServes primarily Serves primarily commuter studentscommuter studentsMajority of the 34,000 Majority of the 34,000 students attend students attend classes partclasses part--time, time, balancing school & balancing school & workwork

  • Teachers on the Go Teachers on the Go ProgramProgram

    Pilot laptop & ePilot laptop & e--portfolio programportfolio programCreated in response to:Created in response to:

    Instructors lack of technology resources and Instructors lack of technology resources and trainingtrainingStudents lagging technology skills and Students lagging technology skills and anxiety about technology useanxiety about technology use

  • EvaluationEvaluation

    Center for Urban StudiesCenter for Urban StudiesPT3 EvaluatorsPT3 Evaluators

    Purpose of initial formative evaluation Purpose of initial formative evaluation Evaluate technology integration in laptop Evaluate technology integration in laptop coursescoursesEvaluate technology training & support needs Evaluate technology training & support needs for students and facultyfor students and faculty

  • Overview of Laptop ClassesOverview of Laptop Classes

    5 courses from WSUs College of 5 courses from WSUs College of Education, Teacher Education DivisionEducation, Teacher Education DivisionAll students & instructors were given All students & instructors were given

    AppleApple iBooks iBooks Course specific softwareCourse specific softwareTechnology training & supportTechnology training & support

    2 classes also created e2 classes also created e--portfoliosportfolios

  • Evaluation PlanEvaluation PlanClassroom Classroom observations observations Focus groups with Focus groups with students and students and instructorsinstructorsStudent online preStudent online pre--and postand post--surveysurvey

  • Classroom ObservationsClassroom ObservationsTo better understand technology activities in To better understand technology activities in laptop classroomslaptop classroomsTo develop valid data collection instruments (i.e., To develop valid data collection instruments (i.e., focus group scripts, student postfocus group scripts, student post--survey)survey)Observations:Observations:

    Instructors & studentsInstructors & studentsType of technology used Type of technology used Collection of syllabiCollection of syllabiMeet with instructorsMeet with instructors

  • Focus GroupsFocus GroupsStudents:Students:

    1 focus group per 1 focus group per classclass5 focus groups, 39 5 focus groups, 39 students totalstudents totalDuration: 1 hourDuration: 1 hourStudent volunteers Student volunteers participatedparticipatedBefore, during, or after Before, during, or after classclass

    Instructors:Instructors:1 focus group with 5 1 focus group with 5 instructorsinstructors

  • Student SurveysStudent SurveysPrior technology Prior technology experienceexperienceLaptop use during & Laptop use during & outside classoutside classSatisfaction with technical Satisfaction with technical training/supporttraining/supportChallenges experiencedChallenges experiencedComputer competencyComputer competencyOutcomes experiencedOutcomes experiencedRecommended Recommended technology changestechnology changes

  • FindingsFindings

  • Impact of Technology: Impact of Technology: StudentsStudents

    Students' Pre- and Post-Survey Reports of Being "Competent/Very Competent" with

    Specific Technology

    48%

    26%

    71%

    30%

    22%

    54%

    81%

    47%

    92%

    63%

    36%

    83%

    0% 20% 40% 60% 80% 100%

    Computer operatingsystem

    Desktop publishing

    Presentation software

    Online research

    Dataorganizing/analyzing

    Online coursemanagement system

    Post (n=54)Pre (n=99)

  • Technology Integration Technology Integration DuringDuring Class TimeClass Time

    Percent of Ways Students Used Laptops in Class (n=48)

    48%

    46%

    79%

    90%

    67%

    81%

    19%

    0% 20% 40% 60% 80% 100%

    Note taking

    Presentations/peer teaching

    Using software

    Using the internet

    Small group work

    Individual work

    Taking an exam

  • Technology Integration Technology Integration OutsideOutsideClass Time for Laptop CoursesClass Time for Laptop Courses

    How Students Used Laptops Outside of Class(n=46)

    28%

    2%

    83%

    20%

    78%

    48%

    83%

    78%

    0% 20% 40% 60% 80% 100%

    Creating e-portfolio

    Designing a website

    Using the internet

    Using iMovie

    Using software

    Preparing presentation

    E-mailing students/ instructor

    Typing assignments/papers

  • Technology Integration in Technology Integration in KK--12 12 ClassroomsClassrooms

    Percent of How Students Used Laptops in K-12 Classrooms

    (n=14)

    29%

    43%

    79%

    21%

    50%

    36%

    0%

    0% 20% 40% 60% 80% 100%

    Observations/note taking

    Presentations

    Creating lesson plans

    Showing cooperating teacher how to usecomputer/software

    Having students use laptop/software

    Downloading pictures of class

    Making iMovie of class

  • Training & Support: Training & Support: Student Satisfaction RatingsStudent Satisfaction Ratings

    Percent of Student Satisfaction with Technical Training/SupportAgree/Strongly Agree

    36%

    41%

    14%

    14%

    56%

    17%

    65%

    71%

    0% 10% 20% 30% 40% 50% 60% 70% 80%

    There was enough training on how to use theMac. (n=53)

    Training sessions/workshops have NOT beenuseful to me for this course. (n=52)

    Had to learn to use the laptops/software onown through trial and error. (n=52)

    COE technology staff is patient with me when Ihave a question. (n=52)

    I feel lost in this course because I have not hadenough tech support/training. (n=51)

    Training/support has helped me to learn how touse the software for this course. (n=52)

    I feel the COE staff is NOT available enough formy computer needs. (n=52)

    There has been enough training/support to helpme with my e-portfolio. (n=12)

  • Students Challenges with Students Challenges with TechnologyTechnology

    Percent of Challenges Students Experienced (n=49)

    35%

    33%

    31%

    37%

    12%

    27%

    43%

    27%

    6%

    45%

    4%

    18%

    12%

    18%

    0% 10% 20% 30% 40% 50%

    Using the Ma c

    Us ing the so ftwa re /progra m s

    Try ing to le a rn bothte chno logy & course conte nt

    Printing

    Se nding/re ce iv ing e -m a ils

    Sa v ing to d isk /ha rd dr iv e

    Conne cting to the inte rne t

    La ptop be ing too s low (note nough m e m ory )

    Us ing Bla ck boa rd

    Incom pa tib ility be twe e n Ma ca nd o the r com pute rs

    Ne e ding to re p la ce la ptop

    Not ha v ing a cce ss to ne e de de quipm e nt (e .g ., sca nne r,

    d ig ita l ca m e ra , pr inte r)

    Not e nough tra ining/supportto do a ss ignm e nts

    F e e ling o v e rwhe lm e d by thea m ount o f work inv o lv e d

  • Recommendations for Laptop Recommendations for Laptop ClassesClasses

    Students' Suggested Changes for Future PT3 Courses(n=53)

    25%

    72%

    63%

    21%

    43%

    32%

    4%

    26%

    0% 20% 40% 60% 80%

    Inform students ahead of time that laptops willbe used in this course.

    Allow students to keep laptops untilgraduation.

    Provide better access to printers.

    Provide more memory on laptops.

    Provide more basic Mac/laptop training.

    Provide more training in using the software forthis course.

    Provide more support from COE staff.

    Incorporate technology into more COE classes.

  • NETSNETS--TT

    Meeting the StandardsMeeting the Standards

  • Technology Operations & Technology Operations & ConceptsConcepts

    Teachers demonstrate a sound understanding of technology operations & concepts

  • Introducing New Concepts

  • TechnologyTechnology--enhanced enhanced EnvironmentsEnvironments

  • Teachers plan & design effective learning environments & experiences supported by technology

    Planning & Designing Learning Planning & Designing Learning Environments & Experiences Environments & Experiences

  • Learning EnvironmentsLearning Environments

  • Teachers implement curriculum plans that include methods and strategies for applying technology to