Comparing Faculty Perceptions of their teaching with classroom observations

By Erika Steele, Donna Turner, Dennis Sunal, and Cynthia SunalComparing Faculty Perceptions of their teaching with classroom observations

Supported by the National Science Foundation Number ESI 0554594

IntroductionAbout NSEUSCollaborative Partners:University of AlabamaSan Diego State UniversityKansas State UniversityThe content of this paper was developed under the National Science Foundation Grant TPC 0554594. However, the content does not necessarily represent the policy of NSF and should not be assumed as an endorsement by the Federal Government.National Study of Education in Undergraduate Science

3About NSEUS5-year project focuses on critical needs in teaching undergraduate science to all diverse majors with emphasis on the preparation and long-term development of pre-service, undergraduate, K-6 teachers of science.

The goal is to investigate the impact on these students of undergraduate, standards-based, reform entry-level science courses developed by faculty that are different from traditional courses.

GoalsInvestigate the impact of reformed entry level, standards-based science courses on undergraduate students

Characterize levels of reform in undergraduate science coursework

Determine short- and long-term impacts of undergraduate science courses on elementary teachers.

Research SampleAlaska Pacific UniversityAustin Peay State UniversityCalifornia State Polytechnical University at PomonaCalifornia State University at ChicoEastern Connecticut State UniversityEdgewood CollegeEmporia State UniversityFort Hays State UniversityGeorge Fox UniversityGuilford CollegeHampton UniversityIndiana UniversityJarvis Christian CollegeLewis and Clark CollegeNorth Carolina Central University Northern Michigan UniversityNorthwest Nazarene University Oklahoma State University OKPortland State UniversitySt. Augustine's CollegeSan Francisco State UniversitySouth Carolina State UniversitySoutheastern Oklahoma State UniversitySouthern Connecticut State UniversityStephen F Austin State UniversityUniversity of AlabamaUniversity of Arkansas University of Incarnate WordUniversity of Oklahoma OKUniversity of Texas at Tyler Wheeling Jesuit UniversityWinona State University

Research Participant Pilot StudyCarnegie Designation of Sample Institutions

Reformed Course CharacteristicsIncorporates the following:Reflects national science standardsEmphasizes student-centered activitiesUtilizes inquiry-based pedagogyBuilds on students prior knowledgeIncorporates interdisciplinary learning and collaborative approaches

8Undergraduate Science TeachingPedagogy in traditional coursesPedagogy in reform courses

Undergraduate Science Teaching

CoRe and PaPeRCoRe and PaPeRModified by Glenda Ogletree

CoRe and PaPeRModifications to Glenda Ogletrees rubric for observations of undergraduate science teaching

NOVA Faculty Member 210Teaching for 19 years at 4 year university.No courses in education or professional development for improving teaching other than the NOVA workshop when applying for grant and GLOBE workshop.The NOVA workshop sparked interest in pedagogy and is interested in more professional development to improve teaching. The instructor is the only one in the department using innovative techniques for teaching. Believes that science is important for all students.Uses short lectures, and relies more on hands on experience to teach content.Very passionate about teaching and student learning.

CoPa: NOVA Faculty Member (210)NOVA faculty tended to use hands-on activities, used alternative assessmenttended to be more aware of pedagogy, and had clear goals for student learning with ideas on how to get them accomplished.This one works so well, that I am trying to change my other classes to not lecture so much because they get the totally glazed over look. They dont seem interested at all. And this class is completely different. You are taking people who have no interest in it at all in the first place and at the end of each unit I actually ask them to write a little reflectionThis helps me assess what Im doing to. If I know something is not working by reading these, I can change what Im doing. (210)

Non-NOVA Faculty Member (420)16 years teaching at 4 year University. Prior to that, taught for 2 years at the high-school and community college level.Took education courses in order to be certified as a high school teacher. Has had no professional development for improving but believes that teaching skills were developed while working as a tutor.Does not like eduspeakHas not really thought about his teaching or student learning.Believes that science majors are more dedicated to learningApproach to covering subjects is to have extensive lectures.Instructor does present a very caring and respectful attitude in the classroom.

CoPa: Non-NOVA faculty members (420)Non-NOVA faculty tended to use lecture, referred to the textbook as a major resource, used traditional assessment, and were unaware or did not implement pedagogical knowledge in the classroom.You have to follow a logical sequence like the one in the Campbell textbook that we use. I try to have zeal and to draw the students out. Droning on just puts them to sleep. I try to have a hook to get their attention. Even during lectures, I use examples and reflections on what I am presenting. (420)

SummaryYear Two Pilot Study DataObservations and InterviewsQualitatively, Instructors teaching non-NOVA courses at the score significantly lower than instructors teaching NOVA science courses NOVA undergraduate instructors Used more active learning strategiesPoised questions to assess student prior knowledgeLectured lessRelied less on the textbookUsed less cookbook labsCollaborated with others about their teaching moreWere more aware of pedagogical strategies

18Year Two Pilot Study DataObservations and InterviewsNon-NOVA Undergraduate instructorsRelied heavily on lecturesStudents were passive in their learningUsed low-level questioningTo pace lectureRhetoricallyConsidered cook book labs hands on experienceCollaborated less with others about their pedagogyMay have been aware of good teaching practices, but did not implement them in their classroomsTended to think students were not interested in science.

Year Two Pilot Study DataObservations and InterviewsBoth NOVA and Non-NOVA Undergraduate instructorsHad passion for the subject they were teaching in interviewsWanted students to leave with a better understanding of the nature of scienceCared about their students performance in the class

Effect of Reformed TeachingShort term and Long TermStudent InterviewsNon-NOVANOVAMaterial is presented over my head.Purpose of the lesson is unclear.Lesson has no applicability to my future career.Teacher doesnt explain well.I need help with the problems.I dont like science and this course does not make me like science.Material is only presented one way.I didnt like science before, but I like science now.Class material is presented in a way I can understand.Lesson can be used when I teach elementary school, even if I have to modify some of the material.I can learn the material.I can teach science..Material is presented in several ways.Student Interviews: Common ThemesHands-on is the best way to present material.Make the content relevant.Make the content practical and connect to students lives.Experiences in elementary and high school were generally dismal: read the book and answer the questions at the back of the book; know the definitions.Im not a math person.

Short Term Effects: Draw A Scientist Test

ABCDStudent 1005 Pre-InstructionStudent 1005 Post-InstructionStudent 1003 Pre-InstructionStudent 1003 Post-Instruction

Pre-Instruction both students a stereotypical drawing of nerdy white men in a lab coatAfter instruction both students showed less stereotypical drawings. Student 1005 developed the idea that science can occur anywhere. Student 1003 indicated that anyone can do science.Draw a Scientist TestPre-Test StatisticsPilot Study Institution 2ClassSample NumberMeanStandard Deviation2101710.92.92202911.532.46T-TestFSigtSig (two-tailed)0.0040.950.7110.6920.4810.4921Equal variance assumed2Equal variance not assumedPost-test StatisticsPilot Study Institution 2ClassSample NumberMeanStandard Deviation210258.364.041205610.714.22T-TestFSigtSig (two-tailed)10.570.953.4214.0320.0010.0021Equal variance assumed2Equal variance not assumedPre-Test Statistics Pilot Study Institution 1ClassSample NumberMeanStandard Deviation1103812.972.91207912.172.46T-TestFSigtSig (two-tailed)0.110.740.7110.6920.4810.521Equal variance assumed2Equal variance not assumedPost-Test Statistics Pilot Study Institution 1ClassSample NumberMeanStandard Deviation1103312.162.641205610.712.53T-TestFSigtSig (two-tailed)0.1860.6671.601.580.1110.1221Equal variance assumed2Equal variance not assumed25Long term effects: RTOP In-Service teachersReformed Teaching Observation Protocol (RTOP) discriminated elementary science classroom lessons of teachers who graduated from reform & traditional courses.

Future DirectionsFuture Direction: Year 3 ActivitiesFinalize site visits to collect data for a sample of up to 15 national institutions, 30 undergraduate science courses, 30 faculty, and 90 elementary inservice teachers.Analyze data collected and provide interim summaries of progress.Analyze data collecting and processing protocols to optimize data collection and analyses.28Future Directions: Year 3 Preliminary ResultsProfessional development targeted specifically for the teaching of scienceActual activities that can be implementedRole modeling science teaching-undergraduate coursesSite-based conversations29Chart12133118112

Sheet1Alaska Pacific University AKMA-IAustin Peay State University TNMA-ICalifornia State Polytechnical UniversityPomona CAMA-ICalifornia State UniversityChico CAMA-IEastern Connecticut State University CTMA-IEdgewood College WIMA-IEmporia State University KSMA-IFort Hays State University KSMA-IGeorge Fox Univeristy ORMA-IGuilford College NCBA-LAHampton University VAMA-IIndiana University INDR-INTJarvis Christian College TXBA-GENLewis and Clark College IDBA-GENNorth Carolina Central University NCMA-I (HBCU)Northern Michigan University MIMA-INorthwest Nazarene UniversityMA-IIOklahoma State University OKDR-EXTPortland State University ORDR-INTSt. Augustine's College NCMA-I (HBCU)San Francisco State University CAMA-ISouth Carolina State University SCDR-INT (HBCU)Southeastern Oklahoma State University OKMA-ISouthern Connecticut State University CTMA-IStephen F Austin State University TXMA-IUnivesrity of Alabama ALDR-EXTUniversity of Arkansas ARDR-EXTUniversity of Incarnate Word TXMA-IUniversity of Oklahoma OKDR-INTUniversity of Texas at Tyler TXMA-IWheeling Jesuit University WVMA-IWinona State University MNMA-IBA-GEN2BA-LA1DR-EXT3DR-INT3DR-INT ( HBCU)1MA-I18MA-I ( Tribal)1MA-II1MA-II ( HBCU)2

Sheet2InstitutionCarnegie DesignationDesignationAlabama State UniversityMA-IPrivateAlaska Pacific UniversityMA-IPrivateAllen UniversityBA-GENPublicAustin Peay State UniversityMA-IPrivateBaker UniversityMA-IPrivateBaylor UiversityDR-INTPrivateBennett CollegeBA-LAPrivate (HBCU)Black Hills State UniversityBA-GENPublicBowling Green State UniversityDR-INTPublicBrescia UniversityBA-GENPrivateCalifornia State Poly. Univ.-Pom.MA-IPublicCalifornia State Polytechnical University PomonaMA-IPublicCalifornia State University ChicoMA-IPublicCalifornia State University-FullertonMA-IPublicCalifronia State Univeristy-NorthridgeMA-IPublicCedar CrestBA-GENPrivateCentral State UniversityBA-GENPublic (HBCU)Claflin UniversityBA-GENPublicClemson CollegeDR-EXTPublicCleveland State UniversityDR-INTPublicColumbus State UniversityMA-IPublicCrichton CollegeBA-GENPrivateDoane CollegeMA-IIPrivateDrake UniversityMA-IPrivateEastern Connecticut State UniversityMA-IPublicEdgewood CollegeMA-IPrivateElizabeth City State UniversityBA-GENPublic (HBCU)Elon UniversityMA-IPrivateEmporia State UniversityMA-IPublicFairmount State CollegeBA-GENPublicFayetteville State UniversityMA-IPublic (HBCU)Florida International UniversityDR-EXTPublicFort Hays State UniversityMA-IPublicGeorge Fox UniveristyMA-IPrivateGonzaga UniversityMA-IPrivateGuilford CollegeBA-LAPrivateHampton UniversityMA-IPrivate (HBCU)Henderson State UniversityMA-IPublicIndiana UniversityDR-INTPublicJackson State UniversityMA-IPublic (HBCU)Jarvis Christian CollegeBA-GENPrivateKansas State UniversityDR-EXTPublicKentucky State UniversityMA-IIPublicKutztown UniversityMA-IPublicLane CollegeBA-LAPrivate (HBCU)Lewis and Clark CollegeBA-GENPublicLouisiana Tech UniversityDR-INTPublicMedgar Evers CollegeBA-GENPublic (HBCU)MercerMA-IPrivateMichigan State UniversityDR-EXTPublicMiddle Tennessee State UniversityDR-INTPublicMorehead State UniversityMA-IPublicMorris Brown CollegeBA-GENPrivate (HBCU)North Carolina Central UniversityMA-IPublic (HBCU)Northern Michigan UniversityMA-IPublicNorthwest Nazarene UniversityMA-IIPrivateNorthwestern State University of LouisianaMA-IPublicOklahoma State UniversityDR-EXTPublicPortland State UniversityDR-INTPublicRoger Williams UniversityBA-GENPrivateSaint Mary's UniversityMA-IPrivateSan Francisco State UniversityMA-IPublicSojourner Douglass CollegeBA-SPECIPrivate (HBCU)South Carolina State UniversityDR-INTPublicSoutheastern Oklahoma State UniversityMA-IPublicSouthern Connecticut State UniversityMA-IPublicSpelman CollegeBA-LAPrivate (HBCU)St. Augustine CollegeBA-LAPrivateSt. John FisherMA-IIPrivateState University of New York @ OswegoMA-IPublicStephen F Austin State UniversityMA-IPublicSusquehanna UniversityBA-LAPrivateTexas Woman's UniversityDR-INTPrivateTexarkana CollegeAA/DR-INTPublicTexas A&M University CommerceDR-EXTPublicTrinity CollegeBA-GENPrivateTuskegee UniversityMA-IPrivate (HBCU)University of Alabama (A Consortium Lead)DR-EXTPublicUniversity of Alabama (Grantee)DR-EXTPublicUniversity of ArkansasDR-EXTPublicUniversity of ColoradoDR-EXTPublicUniversity of DaytonDR-INTPrivateUniversity of IdahoDR-EXTPublicUniversity of Incarnate WordMA-IPrivateUniversity Of Maryland, Eastern ShoreMA-IPublic (HBCU)University of Minnesota, Twin CitiesDR-EXTPublicUniversity of Missouri-ColumbiaDR-EXTPublicUniversity of New Mexico GallupBA-LAPublicUniversity of North AlabamaMA-IPublicUniversity of North Carolina--CharlotteMA-IPublicUniversity of OklahomaDR-INTPublicUniversity of South FloridaDR-EXTPublicUniversity Of Tennessee @ ChattanoogaMA-IPublicUniversity of Texas at TylerMA-IPublicUniversity of the Virgin IslandsMA-IIPublic (HBCU)University of ToledoDR-EXTPublicUniversity of Wisconsin La CrosseMA-IPublicUniversity of Wisconsin--SuperiorMA-IPublicUnivesrity of AlabamaDR-EXTPublicValdosta State UniversityMA-IPublicWestern Kentucky UniversityMA-IPublicWheeling Jesuit UniversityMA-IPrivateWheelock CollegeMA-IPrivateWhitworth CollegeMA-IPrivateWinona State UniversityMA-IPublic

Sheet3