lable at ScienceDirect

Teaching and Teacher Education 26 (2010) 863e870

Contents lists avai

Teaching and Teacher Education

journal homepage: www.elsevier .com/locate/ tate

Evidence of impact: Transforming teacher education with preparingtomorrow's teachers to teach with technology (PT3) grants

Drew Polly a,*, Clif Mims b, Craig E. Shepherd d, Fethi Inan c

aUniversity of North Carolina at Charlotte, Department of Reading and Elementary Education, COED 367, 9201 University City Blvd., Charlotte, NC 28223, USAbUniversity of Memphis, USAc Texas Tech University, USAdUniversity of Wyoming, USA

a r t i c l e i n f o

Article history:Received 14 February 2009Received in revised form3 September 2009Accepted 27 October 2009

Keywords:Teacher educationTechnology integrationMentorshipProfessional development

* Corresponding author. Tel.: þ1 704 687 8700; faxE-mail address: abpolly@uncc.edu (D. Polly).

0742-051X/$ e see front matter Published by Elseviedoi:10.1016/j.tate.2009.10.024

a b s t r a c t

This article uses the framework of technological pedagogical content knowledge (TPACK; Mishra &Koehler, 2006) to analyze findings across projects from the U.S. Department of Education's PreparingTomorrow's Teachers to Use Technology (PT3) initiative. Approaches such as mentoring methods coursefaculty, teachers, and creating technology-rich instructional materials were associated with increases inpreservice teachers' technological knowledge and their frequency of technology-rich instruction duringfield experiences. Lastly, the authors provide implications and directions for future analyses of technologyintegration efforts.

Published by Elsevier Ltd.

1. Background

There is wide contention that technology use can improvestudent learning in schools. Both educational researchers (e.g.,Bransford, Brown, & Cocking, 2000; Chandra & Lloyd, 2008; Davis,Preston, & Sahln, 2008; Earle, 2002; Lawless & Pellegrino, 2007;Roschelle, Pea, Hoadley, Gordin, & Means, 2001; Schacter, 1999)and educational organizations (Departamento de Inform�atica naEducacx~ao a Distancia, 2002; International Society for Technology inEducation [ISTE], 2007, 2000; Ministry of Education, 2004; Ministryof National Education, 2005; United Kingdom Department forChildren, Schools and Families, 2009) have committed to supportingstudent learning through the effective use of technologies inschools.. However, while technology can improve student achieve-ment in K-12 schools (ages 5e18), questions continue to emergeabout whether teachers are prepared to effectively integrate tech-nology in their classroom. We view integration as instances inwhich teachers and/or learners use technology as a tool to supportthe learning process. Many initiatives around the world have beenadvanced in order to develop skills related to technology integration(e.g., Chen, 2000; Lawless & Pellegrino, 2007; Preston & Cuthell,2007). In the past decade,many teacher educationprogrammeshaveattempted to develop preservice teachers' technology integration

: þ1 704 687 3749.

r Ltd.

skills through an introductory course in educational technology.Nearly every preservice teacher takes an educational technologycourse (Andersson, 2006; Persichitte, Tharp, & Caffarella, 1997),these classes improve preservice teachers' technological skillsto enhance productivity, such as making newsletters, grade books,seating charts and PowerPoint presentations to present content(Polly & Shepherd, 2007;Mims, Polly, Shepherd, & Inan, 2006;Wang,2002). However, teachers' technological skills do not result in theeffective use of technology in elementary, middle and secondaryschools in ways that are likely to impact student learning (Beyer-bach, Walsh, & Vannatta, 2001; Hoel, 2005; Wang, 2002). Educa-tional technology courses are typically disconnected from methodscourses and provide only basic technology skills (Carter, 1997;Friedman & Kajder, 2006; Kay, 2006; Moursund & Bielefeldt, 1999).Without experiences in methods courses and field experiences towitness firsthand how technology can be effectively integrated intoK-12 schools, preservice teachers are left with technology skills, butlittle idea about how to implement them into their classroom(Andersson, 2006; Beyerbach et al., 2001; Wang, 2002).

The United States Department of Education's Preparing Tomor-rows Teachers to Teach with Technology (PT3) initiative providedmillions of dollars to universities, K-12 schools, state departments ofeducation and other educational agencies to better prepare preserviceand inservice teachers to integrate technology in K-12 settings.However, limited data-based outcomes have been disseminated fromthese projects. In this paper, we share the findings of a document

D. Polly et al. / Teaching and Teacher Education 26 (2010) 863e870864

analysis of peer-reviewed journal articles and evaluation reports fromvarious PT3 projects. We start by providing an overview of techno-logical pedagogical and content knowledge (TPACK), our frameworkfor this study. Lastly, we discuss the implications and issues involvedwith mining the findings of projects, such as PT3 projects.

2. Theoretical framework: technological pedagogicaland content knowledge

This study is grounded in the framework of technological peda-gogical and content knowledge (TPACK; cf., Koehler, Mishra, & Yahya,2007;Mishra&Koehler, 2006;Niess,2005;Pierson,2001). Fig.1 showsMishraandKoehler's (2006)TPACKframeworkandthevariousaspectsof knowledge encompassed within. TPACK builds off of Schulman'sconcept of pedagogical content knowledge (Schulman, 1987).

Koehler et al. (2007, p. 743) describe technology, pedagogy andcontent as:

Content (C) is the subject matter that is to be learned/taught.The content to be covered in high-school social studies oralgebra is very different from the content to be covered ina graduate course on computer science or art history.Technology (T) broadly encompasses standard technologies suchas books and chalk and blackboard, as well as more advancedtechnologies such as the Internet and digital video, and thedifferent modalities they provide for representing information.Pedagogy (P) includes the process and practice or methods ofteaching and learning, including the purpose(s), values, tech-niques or methods used to teach, and strategies for evaluatingstudent learning.

Extending beyond just the three isolated aspects of knowledge,TPACK accounts for numerous intersections of technology, contentand pedagogy with each other. Thus, in order to effectively integratetechnology into their classroom, teachers must be knowledgeableabout the relationships between technology and content- howtechnology can be used to support the learning of specific content,technology and pedagogy- how specific pedagogies best support theuse of technology, and content and pedagogy- how specific

Fig. 1. Framework of technological pedagogical and content knowledge (TPACK:Koehler & Mishra, 2008).

pedagogies facilitate learning of specific content. Further, the centercircle recognizes that teachers must possess knowledge about theintersection between technology, pedagogy as well as content thatthey are teaching.

TPACK has been described in various documents (Koehler & Mis-hra, 2008;Mishra & Koehler, 2006; Niess, 2005; Polly & Brantley-Dias,2009) as a framework used to explain and describe teachers' knowl-edge and skills related to technology integration. Further, in priorstudies (e.g., Hofer & Swan, 2008; Koehler et al., 2007; Mishra &Koehler, 2006; Niess, 2005) TPACK has been used as a framework fordata analysis as researchers have analyzed data in light of the variousaspects of TPACK. In that spirit, TPACK provided us with a frameworkto help make sense of the findings reported from PT3 initiatives thatwere designed to influence teacher education courses and fieldexperiences.

3. Methodology

This article shares the findings from a systematic documentanalysis (Hodder, 2000) of peer-reviewed journal articles, projectevaluation reports and peer-reviewed conference papers pertain-ing to PT3 projects that began in 2000e2001. While another roundof PT3 projects were funded in 2005e2006, only the first roundof PT3 projects were held accountable to turn in annual and end-of-project reports. The U.S. Department of Education disbanded thePT3 initiative after they funded more projects in 2003e2004. Thus,access to documentation related to PT3 initiatives is only availablefor the projects funded in 2000e2001.

Based on our literature review and interest in exploring both theapproaches used and the impact of the PT3 initiatives in teachereducation programmes, our work was primarily guided by thequestions:

1) What approaches were commonly used in PT3 initiatives?2) What were the reported outcomes of these approaches used in

PT3 initiatives?

Document analysis was used since the rich amount of infor-mation that was located in project reports and refereed journalarticles (Guba & Lincoln, 1982). Document analysis is a systematicprocess that is used as a technique for gathering retrospective data(Hodder, 2000). In his report for the National Institute of Education,Garman (1982) cited document analysis as a vital tool for researchand evaluation, because of the stepwise nature of tracking, miningand synthesizing information located in the documents.

3.1. Document selection

In order to narrow the scope of the study, this exploratory, quali-tative inquiry (Patton, 2002) focused on PT3 projects that attemptedto transform preservice education courses and field experiences.Since theU.S. Department of Education decided to disintegrate the PT3initiative, obtaining documents related to projects' effectiveness wasdifficult. One hundred forty-eight PT3 project coordinators/principalinvestigators were invited via e-mail to share documents such asfinal project reports, conference proceedings and published articlesfor inclusion in our analysis. Through e-mail we obtained informationabout thirty-five projects from either project personnel. Somepersonnel responded,buteither refusedtoshare their informationwithus or reported that they could not locate any of the evaluation reports.

We also used ERIC to locate and identify refereed journal articlesusing combinations of keywords such as, PT3 project, technologyintegration, preservice teacher, teacher education and PT3 grant.This process yielded 15 more documents, 5 of them were aboutprojects we had obtained evaluation reports of, while 10 of them

D. Polly et al. / Teaching and Teacher Education 26 (2010) 863e870 865

were articles about projects that we did not have informationabout. Conference papers were only considered and used if theyprovided information about a project that we had either a peer-reviewed journal article or a project evaluation report from. Table 1shows our process of obtaining documents.

3.2. Data analysis

Based on our interest in examining the reported outcomes ofthese projects we developed a checklist to systematically evaluatethe rigor of both the evaluation reports and the referred journalarticles. The checklist included three components: methodology,findings, and alignment between implications, findings and meth-odology. The third category was developed after reading projectreports that described implications and next steps that werenot linked to or based on evidence presented in the report. On thechecklist, a point was awarded for each component if the standardwasmet. Project nameswere removed for confidentiality reasons. Inorder to establish inter-rater reliability among the researchers, thefour authors read and evaluated two project reports. There was 100percent agreement on both project reports.

After coming to consensus on the two project reports, theresearch team split the project documents among themselves.One rater evaluated each document. If there were questions, therater brought them back to the entire group and the document wasread and discussed by the entire research team. Documents wereonly used for analysis if they scored two out of three points on thechecklist. Based on the document evaluation, the authors deter-mined that 26 of the 35 PT3 projects reviewed were rigorousenough to be included in our analysis.

Modified inductive analysis was used to analyze the documents(Bogdan & Biklen, 1992). The authors used the analytical strategy ofopen coding to label passages of the documents and entered theminto a spreadsheet. Codes included both the approaches in theinitiatives as well as the reported outcomes. Once the data was putinto the spreadsheet, it was sorted by codes and subcodes werecreated. Through the process of meaning condensation, the authorssearched for patterns and central themes across PT3 projects (Cof-fey & Atkinson, 1996). These themes were rechecked in the originaldocuments by a differentmember of the research team and broughtbefore the researchers at analysis meetings. The data within everytheme was analyzed again by the researchers and refined throughthe use of an audit trail from the theme to the code to the associateddata sources. In the next section we share themes that were iden-tified through the data analysis process.

4. Findings

4.1. Mentoring university faculty

Numerous projects attempted to infuse more technology integra-tion into teacher education programmes through one-on-one

Table 1Summary of documents analyzed.

Type of documents Activity

Evaluation reports The research team e-mailed 147 project directors and/or Printo request evaluation reports.

Peer-reviewedjournal articles

The research team searched ERIC database using combinationtechnology integration, preservice teacher, teacher education anmore documents, 5 of them were about projects we had obtawhile 10 of them were articles about projects that they did n

Conference papers The project team included conference papers for projects thareports or peer-reviewed journal articles for.

mentoring approaches (Howland & Wedman, 2004; Nave, 2004;Schaffer & Richardson, 2004; Strudler, Archambault, Bendixen,Anderson, & Weiss, 2003). The most common approach useda combination of workshops with individualized mentoring betweenworkshop sessions (Brush et al., 2003; Ludwig & Booz, 2003; Schaffer& Richardson, 2004). Workshops primarily developed participatingfaculty's knowledge of technology and pedagogies associated withusing technology in university courses. Technology-savvy students(both graduate and undergraduate) typically met with individualfaculty members about their technology-related goals and needs.In some cases, meetings with mentors focused solely on the techno-logical skills, such as learninghow todesign anddevelop a PowerPointpresentation. In other meetings, mentors focused on TCK and TPACK,by building on faculty's technological knowledge (TK), and ways tointegrate specific technologies into their course (TPK) that theylearned inworkshops. Since faculty already had formed TK and TPK inthe workshops, mentors were more able to focus on addressingcontent needs and working on the overlap between technology,content and pedagogies.

4.1.1. Addressing barriers related to technological knowledge (TK)The method that projects used to mentor faculty varied greatly.

Some projects, such as the one at the University of Missouri (Univer-sity of Missouri, 2003), reported a shift in their project's instructionalfocus fromworkshops towards individualized mentoring:

Beginning the project, we provided some group sessions thatfocused on skill building . We redefined the learning sessionsas individualized tutoring sessions that were based on facultyneeds and goals. (p. 41)

Researchers at Boston University documented a dependenceon mentors (Nave, 2004) and a lack of independence related toTPACK. After meeting with graduate students, faculty attempted tointegrate technology in their courses. However, while teachingthe courses, faculty's inexperience using the technology (TK) andvarious technological glitches meant that they were not able toeffectively use technology during their courses. The Universityof Texas-Austin addressed some of faculty's problems related to TKby providing just-in-time and on-site support for methods courses(Resta, Petrosino, & Confrey, 2004). Faculty relied on this assistanceas another layer of support while incorporating technology-richactivities in their courses. University of Nevada-Las Vegas (UNLV)also offered just-in-time help provided by a variety of people,including the project coordinator, assistant coordinator, technologyintegration specialist, and student technology assistants (Strudleret al., 2003).

4.1.2. Reported outcomesThe evaluation of PT3 projects that used mentoring approaches

to develop TPACK and support technology integration in universitycourses found that these experiences improved faculty's techno-logical knowledge (TK), more frequent use of technology in

Number ofdocumentsobtained

Number of documentsmeeting criteriafor inclusion

cipal Investigators 35 26

s of key words, such as, PT3 project,d PT3 grant. This process yielded 15ined evaluation reports of,ot have information about.

11 11

t they also had either evaluation 5 5

D. Polly et al. / Teaching and Teacher Education 26 (2010) 863e870866

methods courses (TCK), and more instances of preservice teachersusing technology during their coursework (TK and TCK). Whilementorship approaches show promise in developing faculty andpreservice teachers' technology integration skills, mentorshiprequires more time and money than large-scale models of profes-sional development (Resta et al., 2004; Strudler et al., 2003).

4.2. Promoting TPACK of both preservice and inservice teachers

In some cases, PT3 projects attempted to develop K-12 teachers'TPACK by providing assistance with both planning and co-teachingtechnology-rich lessons. UNLV's teacher education programmeplaces over 500 student teachers and over 900 teacher candidatesin schools annually (Strudler et al., 2003). In order to maximizethe likelihood of preservice teachers' observing technology use inschools, the College of Education undertook the arduous task ofmatching teacher candidates up with technology-using teachersand working closely with schools that had large numbers of tech-nology-savvy K-12 teachers. While one of the project's goals wasto impact technology use in their entire district, project directorsdecided to focus their efforts on a limited number of schools inorder to maximize the project's impact on both the inservice andpreservice teachers. Teachers in participating schools attendedprofessional development workshops, received technology equip-ment and had access to individual help from graduate students,university faculty or school district technology personnel. Theseexperiences focused on improving teachers' technological knowl-edge (TK), using technology to address specific academic standards(TCK) and instructional practices that should be used with specifictechnologies (TPK).

4.2.1. Reported outcomesResearchers concluded that technology-rich field experiences

positively influenced teacher candidates' integration of technology;seventy-three percent of UNLV preservice teachers reported inte-grating technology in a manner in which K-12 students usedtechnology to meet an academic standard (TCK and TPACK). Ona scale ranging from zero to six, 28.1% of preservice teachersreported that they were at Stage 5 (n ¼ 43, 28.1%), “I think aboutthe computer as a tool to help me and am no longer concernedabout technical aspects”, on a Stages of Technology Adoptionsurvey (Strudler et al., 2003). Fifty-five teacher candidates (35.9%)rated themselves at Stage 6, which addresses all three aspects ofTPACK, “I can applywhat I know about technology in the classroom.I am able to use it as an instructional tool and integrate it in thecurriculum.”

Other projects reported difficulty finding technology-rich fieldexperiences for their preservice teachers. This barrier existed fordifferent reasons, including a lack of technology in K-12 schools(Brzycki & Dudt, 2005), schools' reluctance to participate in the PT3project (McMurray State University, 2004), and a lack of alignmentbetween how technology was used in teacher education pro-grammes and the K-12 schools (Boccia, 2003; Brush et al., 2001;Brzycki & Dudt, 2005). Most projects that attempted to improvepreservice teachers' field experiences reported a varied range inquality.

Analyses of project data concluded that preservice teacherswho observed and experienced technology integration in their fieldexperiences reported more positive attitudes towards technology(Bahr, Shaha, Farnsworth, Lewis, & Benson, 2004), more frequentuse of technology (TK) (Strudler et al., 2003), and more instances ofpreservice teachers teaching with technology to support learning(TCK) (Strudler et al., 2003; Wentworth, Waddoups, & Earle, 2004).Unfortunately, the lack of technology-rich field experiences limitedpreservice teachers' opportunities to learn.

4.3. Design of curricula materials

Various PT3 initiatives spent time and resources on the design ofcurricula materials (TCK). These materials included K-12 unitsand modules for teacher education courses.

4.3.1. K-12 curriculum redesignAt Brigham Young University, methods faculty collaborated with

curriculum and technology coordinators from a local school district,and inservice teachers to design technology-rich units that couldbe implemented in local schools (TCK). Faculty and K-12 teachersattended the same workshops and learned about WebQuests andhow to develop and implement technology-enhanced project-basedlearning (TPK). The curriculum design of the project was scaffoldedand gradually progressed from participants gathering ideas foreffective technology integration (TCK) (Year 1), designing curricula(TCK and TPACK) (Year 2), to developing ways to sustain the curric-ulum design collaboration after the project (TPACK) (Year 3). Projectstaff attributed the success of the curriculum design team to:a) allowing the collaborators to have a year to familiarize themselveswith each other and the idea of technology integration prior tobeginning the project (TCK), b) allowing collaborators to choose theirown design team based on content and grade level (TCK), andc) downscaling the size of the unit so that it was completed by theend of the summerworkshops (Graham, Culatta, Pratt, &West, 2004;Wentworth et al., 2004). While the multi-year partnership betweenfaculty and K-12 teachers started slowly, the project's initiativesweresustained and effective long-term. The researchers noted that whilemultiple factors influenced these initiatives, participants reportedin both interviews and surveys that the duration of the relationshiphelped all stakeholders develop common goals for the project.

Other PT3 projects also created opportunities for preserviceteachers and K-12 teachers to collaborate on designing technology-rich curricula (TCK). In one case, preservice teachers at the Collegeof William & Mary, while taking the second of two educationaltechnology courses, collaborated with K-12 teachers to designtechnology-rich lessons (TCK). The preservice teachers served therole as the technology experts, with a lot of TK and TPK, whilethe K-12 teachers served as the content experts (CK and PCK). Thepreservice teachers had just completed their educational technologycourse and were familiar with educational technologies, but lackedknowledge of the curriculum. Participants met after school to plantechnology-rich units, which were co-planned and co-taught byboth teachers and preservice teachers in classrooms (Richardson &Hannafin, 2005).

While some projects formally incorporated curricula design intothe project, most K-12 instructional materials were created on aninformal basis, where project staff co-planned and worked withK-12 teachers to design technology-rich lessons in the midst ofother project activities (Brush et al., 2001; Howland & Wedman,2004; Strudler et al., 2003; Vetter, Sologuk, & Stammen, 2001). Theinformal nature of this type of curricula design did not assessthe fidelity of implementation and the impact onparticipants.Whilethe idea of having preservice teachers, inservice teachers anduniversity faculty collaborate to create technology-rich curriculashows promise, there was no evidence from PT3 projects regardingthe nature of materials, the methods of implementation, and theimpact on student learning.

4.3.2. Creating teacher education instructional materialsNumerous PT3 projects used their funding to develop instruc-

tional materials for teacher education courses. Faculty at BrighamYoung University spent time learning about possible ways to inte-grate technology into their courses (TCK) and then designed andimplemented technology-rich units into methods courses (TPACK)

D. Polly et al. / Teaching and Teacher Education 26 (2010) 863e870 867

(Dyches, Smith, & Syal, 2004;Waddoups,Wentworth, & Earle, 2004).On post-course surveys, preservice teachers self-reported greatertechnological skills and more ideas on how to use technology withK-12 learners (TK, TCK and TPACK) (Wentworth et al., 2004).

Project staff with the InTime project (Krueger, Boboc, Smaldino,Cornish, & Callahan, 2004), at the University of Northern Iowa,created video-based examples of technology being used in K-12schools for the purpose of sharingwith preservice teachers. Further,one preservice teacher in the InTime project reported, “Because ofthese learning experiences, I have a better understanding of what itmeans to integrate technology into the classroom” (TPK) (Kruegeret al., 2004, p. 204). InTime faculty reported that 51% more preser-vice teachers were proficient in integrating technology into generallessons than prior to the project and that 42% of preservice teacherswere proficient at integrating technology into the curriculum(TPACK).

Collin County Community College project staff identified K-12Blue Ribbon Technology teachers and paid them to develop a 20-hour instructional module, which was used in the educationaltechnology course (Jenkins, 2005). Preservice teacherswent throughthe units and reported that the module gave them more concreteexamples of how to effectively use technology in elementary schools(TCK and TPACK).

4.4. The role of administrative support in developing knowledge

Administrative support in both teacher education programmesand local K-12 schools influenced the implementation of PT3projects. In teacher education programmes, administrators providedmatching funds and faculty release time as further incentives forparticipating in the project (Brzycki & Dudt, 2005; Wentworth et al.,2004). In one case, the Dean of the School of Education providedparticipating methods course faculty with travel funds to present atan educational technology conference, and even attended himself(Wentworth et al., 2004).

In addition to allocated resources, some projects benefited fromK-12 administrators whomade PT3 projects a priority, aligning otherinitiatives and teacher education reform efforts to the PT3 project(Brzycki & Dudt, 2005; Staples, Pugach, & Himes, 2005). However, insome cases, administrators were barriers to successful grant imple-mentation (Boccia, 2003; Hall, Fisher, Musanti, & Halquist, 2006).Problems occurred on account of a large turnover of grant personnel(Groves, 2005; Hall et al., 2006), or focusing on other priorities suchas NCATE accreditation (McMurray State University, 2004). Further,in some cases, K-12 administrators expressed an initial interest inparticipating, but once the grant was funded reported that the PT3project no longer aligned with their districts' goals (Hall et al., 2006;McMurry State University, 2005).

5. Discussion

Based on our analysis, there were numerous PT3 projects thatinvested project resources to develop preservice teachers' andfaculty members' TPACK. These approaches including both modi-fying both the experiences that preservice teachers participated in aswell as developing the TPACK of the individuals who facilitate theseexperiences, university faculty and inservice teachers. Preserviceteachers who observed or participated in technology integrationactivities in methods courses and/or K-12 field experiences tendedto report more positive attitudes towards technology integration(Bahr et al., 2004), more technological knowledge (TK) (Wentworthet al., 2004), more advanced placement on the technology integra-tion Stages of Adoption (TPACK) (Strudler et al., 2003), and moreimplementation of technology-rich instruction during field experi-ences (TPACK) (Krueger et al., 2004; Wentworth et al., 2004).

However, while technology-rich field experiences and methodscourses show promise to positively impact preservice teachers'TPACK, various questions warrant further examination.

5.1. Addressing the barriers of mentorship

One of the most frequently mentioned and most successfulapproaches reportedwas amentoring programmewhere project staffworkedwith individual facultyandK-12 teachers.While this approachmet participants' individual needs and was linked to successful tech-nology integration and TPACK, mentorship models reaches far fewereducators than workshops or other traditional forms of professionaldevelopment (Strudler et al., 2003). While mentorship has shownto positively increase educators' TPACK, there is a need to closelyexamine how to address issues related to scaling up mentorshipprogrammes. Research related to professional learning communities(DuFour& Eaker,1998;Hord, 2004) and communities of practice (Lave&Wenger,1991;Wenger,McDermott, & Snyder, 2002)may inform thedevelopment of communities focused on developing TPACK. Futurework should examine alternative forms of providing individualizedattention and mentorship to faculty and K-12 teachers. Possibilitiesinclude mentoring groups of individuals who need to develop TPACKin a common area (e.g., science education faculty, 2nd grade teachers)rather thanworking exclusively with individuals. Another potentiallyeffective approach includes preparing faculty or teachers to becomementors, similar to a train-the-trainer approach to professionaldevelopment.

5.2. Development of TPACK and beliefs

The successful integration of technology requires the under-standing of technology, pedagogies and content encompassedwithinthe TPACK framework (Mishra & Koehler, 2006). Teachers and facultymust be able to enact skills related to both using technologicalapplications and integrating technology in specific content areas(TK and TPACK) (Ertmer, 1999). Further, even when teachers possessadequate technological knowledge (TK) knowledge related to usingtechnology with students (TCK) and even knowledge related to theuse of technology and pedagogies in specific content areas (TPACK),there is no guarantee technology will be used effectively. K-12teachers and facultymust also subscribe to the belief that technologycan improve student learning (Ertmer, 2005). While the PT3 projectsdiscussed in this paper shared the development of faculty and K-12teachers' technology skills, there is little documentation about theimpact of PT3 or other technology-focused initiatives on participants'technology integration skills or their beliefs about technology inte-gration. In some cases, participants implemented technology as partof the grant; however, after the grant, it is unknown whether thosetechnology-related practices were still implemented.

Along similar lines, the long-term impact and sustainability ofthese projects is unknown. The writing of both the end-of-projectreports and journal articles occurred towards the conclusion of theprojects, and do not include data concerning whether or not facultyor K-12 teachers' technology integration practices changed one yearor a few years after the project ended. While some projects docu-mented participants' willingness to maintain the project activitiesafter the grant's completion (Wentworth et al., 2004), furtherdata is needed to confirm that claim. There is also a need for moreexamination about preservice teachers' technological skills as theyhave started their teaching career.

5.3. Alignment of SCDE and K-12 initiatives

Our findings indicate that partnerships between schools,colleges, and departments of education (SCDE) and K-12 schools

D. Polly et al. / Teaching and Teacher Education 26 (2010) 863e870868

influenced the implementation of PT3 projects. Projects that hadadministrative support and joint buy-in from both university andK-12 school personnel expressed more success with their initiatives(Strudler et al., 2003; Wentworth et al., 2004). However, adminis-trative turnover (Boccia, 2003; Hall et al., 2006) and differingpriorities between teacher education programmes and K-12 schools(McMurray State University, 2004) impeded project implementa-tion. While it seems intuitive that teacher education programmesand K-12 schools would have common goals for improving publiceducation, our findings indicate that that was not always the case.

The alignment of SCDE and K-12 schools related to technologyintegration ranges from more conceptual issues, such as a commonunderstanding of effective technology integration and beliefs aboutteaching and learning, to more concrete details, such as emphasizedinstructional practices (TPK) and access to specific technologies. Insome projects, technology was either not accessible or not used inK-12 schools (Bahr et al., 2004; Benson, Farnsworth, Bahr, Lewis, &Shaha, 2004;McMurray State University, 2004; Strudler et al., 2003).Through correlation studies, when preservice teachers experiencedtechnology integration in their educational technology course, butnot in K-12 schools, their views about the benefits of technologyintegration decreased as they spentmore time in schools (Bahr et al.,2004; Benson et al., 2004). The projects that reported substantialimpact on preservice teachers provided professional learningopportunities that addressed all components of TPACK by advancinga common belief about teaching and learning, and concrete exam-ples of how technology should be used with K-12 learners (TPACK)(Benson et al., 2004; Brush et al., 2001; Strudler et al., 2003). Futureinitiatives from SCDEs related to technology integration shouldconsider comprehensive approaches that address the needs ofmultiple stakeholders (e.g., faculty, K-12 teachers, preserviceteachers) and provide multiple modes of support, including work-shops, mentoring, and opportunities to co-design effective instruc-tional materials.

5.4. Limitations of the study

While this study presents empirically-based findings, specificlimitations must be considered when examining the impact ofPT3 projects. First, since PT3 projects were funded to developtechnology-rich experiences rather than conduct rigorous researchabout the impact of these experiences, there is a sparse amount ofresearch that has been published about the impact of PT3 projects.The primary goal of these PT3 grants was to design programmes tosupport the infusion of technology into courses in teacher educa-tion and arts and science as well as K-12 classrooms. The number ofavailable evaluation reports and lack of peer-reviewed journalarticles cautions the generalizability of these findings.

Second, most of the data collected is self-reported data that wascollected from surveys. While the reliability of survey data has beendebated (Buck Institute for Education, 2002; Mullens & Kasprzyk,1999), research studies and evaluation reports need to employmultiple data sources to provide a more robust view of how PT3projects have impacted each of the various stakeholders (Borko,2004). These limitations are addressed below as we provide impli-cations for future studies.

6. Implications for future work

6.1. Future directions for research

The research findings presented here are based on a documentanalysis related to PT3 projects. Future research studies related toboth large-scale projects and teacher education initiatives shouldextend beyond self-reported data, and analyze more objective data

from implementation, including classroom observations, videosof classroom teaching and artifacts of teacher and student work(Borko, 2004; Guskey, 2000). In particular, we suggest that futureresearch examines:

� The influence ofmentoring faculty, K-12 teachers andpreserviceteachers on both instructional practices and students' learning.Amajorityof theprojects focused on skill acquisition rather thanintegration. Specifically, there is a need to identify ways to scaleup programmes to develop TPACK of large numbers of facultywithout sacrificing the effectiveness of learning opportunities.

� Specific “best practices” in comprehensive initiatives that willlikely have the greatest impact on instructional practicesand student learning. Based on the findings presented here,preservice teachers benefited from the comprehensive focus oftechnology integration in methods courses and field experi-ences. Faculty benefited from allocated time to learn abouttechnology, plan technology-rich instruction and receive indi-vidualized support when needed.

� Focusing more on integration rather than skills. A majority ofthe PT3 projects focused their efforts on developing the tech-nological skills of preservice teachers and faculty (technolog-ical knowledge) rather than striving to develop knowledgerelated to the intersection of technology and pedagogy. Futureworkmust examine how to best develop TPACKwith a focus onintegrating technology rather than solely focusing on acquiringtechnology skills.

� The sustainability of long-term initiatives. As project funds aregone, research is needed to examine issues related to sustainingtechnology integration initiatives. Specifically, there is a need toidentify key characteristics of projects that are successfullysustained.

6.2. Implications for the TPACK framework

TPACK provided a framework for the analysis and sharing of ourfindings. It seems rather intuitive that effective teaching with tech-nology requires teachers to have knowledge of the technologies,pedagogies, content and all of the intersection of those three compo-nents. However, questions still remain in regards to how researcherscan validly and reliablymeasure TPACK and the various aspects of it. Inthis study, findings associated with PT3 projects. In this study, datasources were coded and organized according to the various compo-nents of TPACK, and the only associations that were made addressedwhether or not findings from PT3 manuscripts aligned with variousaspects of TPACK. Questions still remain about howmuch knowledgein each of the components of TPACK is needed to effectively plan andteach a technology-rich lesson. Further, building on the work done inmathematics education related to teachers' specialized knowledge(Hill, Ball, & Schilling, 2008), there is a need for reliable and valid waysto assess the TPACKof both preservice and inservice teachers. Buildingon teachers' need for specialized knowledge more work is needed toexaminewhat aspects of TPACKaremost critical to develop in teachersas well as what learning experiences facilitate the development of thevarious components of TPACK. These questions should be addressed infuture work.

7. Concluding remarks

This paper presents the findings from a document analysis ofartifacts related to the U.S. Department of Education's PreparingTomorrows Teachers to Teach with Technology (PT3) projects.Approaches to teacher and preservice teacher learning, such asindividualized mentorship, the development of curricula materials

D. Polly et al. / Teaching and Teacher Education 26 (2010) 863e870 869

and creating technology-rich field experiences are associated withgreater technological knowledge and skills, more use of technologyin methods courses and field experiences, and more frequent usesof technology with K-12 learners. Barriers, such as administrativesupport and the lack of alignment between teacher educationprogrammes and K-12 schools also existed. As teacher educationresearchers continue to examine how to develop preserviceteachers' technology integration skills, there are numerous prom-ises and issues worth considering.

Acknowledgements

The authors would like to thank Judy Duffield and Julie Moorefor their feedback on earlier versions of this manuscript.

References

Andersson, S. (2006). Newly qualified teachers' learning related to their use ofinformation and communication technology: a Swedish perspective. BritishJournal of Educational Technology, 37(5), 665e682.

Bahr, D. L., Shaha, S. H., Farnsworth, B. J., Lewis, V. K., & Benson, L. F. (2004).Preparing tomorrow's teachers to use technology: attitudinal impacts of tech-nology-supported field experience on preservice teacher candidates. Journal ofInstructional Psychology, 31(2), 88e97.

Benson, L. F., Farnsworth, B. J., Bahr, D. L., Lewis, V. K., & Shaha, S. H. (2004). Theimpact of training in technology assisted instruction on skills and attitudes ofpre-service teachers. Education, 124(4), 649e663.

Beyerbach, B., Walsh, C., & Vannatta, R. (2001). From teaching technology to usingtechnology to enhance student learning: preservice teachers' changing perceptionsof technology infusion. Journal of Technology and Teacher Education, 9, 105e127.

Boccia, J. (2003). Ready to Teach (RTT) project final report. Lowell, MA: University ofMassachusetts at Lowell.

Bogdan, R. C., & Biklen, S. K. (1992). Qualitative research for education: An intro-duction to theory and methods. Boston, MA: Allyn and Bacon.

Borko, H. (2004). Professional development and teacher learning: mapping theterrain. Educational Researcher, 33(8), 3e15.

Bransford, J. D., Brown, A., & Cocking, R. (Eds.). (2000). How people learn: Mind,brain, experience and school, expanded edition. Washington, DC: NationalAcademy Press.

Brush, T., Glazewski, K., Rutowski, K., Berg, K., Stromfors, C., Van-Nest, M. H., et al.(2003). Integrating technology into a field-based teacher training programme:the project. Journal of Educational Technology Research and Development, 51(1),57e72. PT3@ASU.

Brush, T., Igoe, A., Brinkerhoff, J., Glazewski, K., Ku, H., & Smith, T. C. (2001). Lessonsfrom the field: integrating technology into preservice teacher education. Journalof Computing in Teacher Education, 17(4), 16e20.

Brzycki, D., & Dudt, K. (2005). Overcoming barriers to technology use in teacherpreparation programs. Journal of Technology and Teacher Education, 13(4), 619e641.

Buck Institute for Education. (2002). Teaching with technology: a Statewideprofessional development program. Retrieved 12.11.05, from http://www.bie.org/files/twtfinal.pdf.

Carter, D. S. (1997). Information and communication technology in the professionalpractice of beginning teachers. Journal of Education for Teaching, 23, 3294e3296.

Chen. Z. (2000). To generalize information technology course in K-12 Schools.Retrieved 10.11.04, from http://www.edu.cn/20020327/3023659.shtml.

Chandra, V., & Lloyd, M. (2008). The methodological nettle: ICT and studentachievement. British Journal of Educational Technology, 39(6), 1087e1098.

Coffey, A., & Atkinson, P. (1996). Making sense of qualitative data: Complementaryresearch strategies. Thousand Oaks, CA: Sage.

Davis, N., Preston, C., & Sahln, I. (2008). Training teachers to use new technologiesimpacts multiple ecologies: evidence from a national initiative. British Journal ofEducational Technology, 39(4), 1e18.

Departamento de Inform�atica na Educacx~ao a Distancia. (2002). Relat�orio deatividades 1996/2002. Retrieved 20.12.03, from http://www.proinfo.gov.br/upload/img/relatorio_died.pdf.

DuFour, R., & Eaker, R. (1998). Professional learning communities at work: Best practicesfor enhancing student achievement. Bloomington, IN: National Educational Serviceand Alexandria, (VA: Association of Supervision and Curriculum Development).

Dyches, T. T., Smith, B. A., & Syal, S. (2004). Redesigning an introduction to specialeducation course by infusing technology. Computers in the Schools, 21, 59e72.

Earle, R. S. (2002). The integration of instructional technology into public education:promises and challenges. Educational Technology, 42(1), 5e13.

Ertmer, P. A. (1999). Addressing first- and second-order barriers to change: strat-egies for technology integration. Educational Technology Research and Develop-ment, 47(4), 47e61.

Ertmer, P. A. (2005). Teacher pedagogical beliefs: the final frontier in our quest fortechnology integration? Educational Technology Research and Development, 53(4), 25e39.

Friedman, A., & Kajder, S. (2006). Perceptions of beginning teacher educationstudents regarding educational technology. Journal of Computing in TeacherEducation, 22(4), 147e151.

Garman, K. (1982). Eastside, westside.An exercise in applying document analysistechniques in educational evaluation. (Northwest Regional Education Lab ReportSeries, No. 78). Washington, DC: National Institute of Education. (ERIC docu-ment Reproduction Service No. ED 231 872). Beginning of the paragraph- afterdocument analysis (Guba & Lincoln, 1982).

Graham, C., Culatta, R., Pratt, M., & West, R. (2004). Redesigning the teachereducation technology course to emphasize integration. Computers in theSchools, 21, 127e148.

Groves, J. (2005). Preparing tomorrow's teachers to use technology: A final report.Lakewood, NJ: Georgian Court University.

Guba, E. G., & Lincoln, Y. S. (1982). Effective evaluation. San Francisco: Jossey-BassPublishers.

Guskey, T. R. (2000). Evaluating professional development. Thousand Oaks, CA:Corwin Press.

Hall, L. D., Fisher, C., Musanti, C., & Halquist, D. (2006). What canwe learn from PT3?Tech Trends, 50(3), 25e31.

Hill, H., Ball, D. L., & Schilling, S. (2008). Unpacking “pedagogical content knowl-edge”: conceptualizing and measuring teachers' topic-specific knowledge ofstudents. Journal for Research in Mathematics Education, 39(4), 372e400.

Hodder, I. (2000). The Interpretation of documents and material Culture. InN. K. Denzin, & Y. S. Lincoln (Eds.), The handbook of qualitative research (2nd ed.).(pp. 155e175) Thousand Oaks, CA: Sage.

Hoel, T. L. (2005). F�rste�aret som ny lærer: et prosjekt basert p�a oppf�lging gjennome-post. [The first year as a new teacher. a project based on follow-up throughe-mail]. Nordisk Pedagogic. [Nordic Educational Research], 25(2), 95e107.

Hofer, M., & Swan, K. (2008). Technological pedagogical content knowledge fromthe ground level: a case study of a middle school digital documentary project.Journal of Research and Technology Education, 41(2), 179e200.

Hord, S. M. (2004). Learning together leading together: Changing schools throughprofessional learning communities. Oxford, OH: Teachers College Press.

Howland, J., & Wedman, J. (2004). A process model for faculty development:individualized technology learning. Journal of Technology and Teacher Education,12(2), 239e263.

International Society for Technology in Education. (2000). National educationaltechnology standards for teachers (NETS-T). Eugene, OR: Author.

International Society for Technology in Education. (2007). National educationaltechnology standards for students, second Edition (NETS-S). Eugene, OR: Author.Retrieved 17.09.07 from http://cnets.iste.org/inhouse/nets/cnets/students/pdf/NETS_for_Students_2007.pdf.

Jenkins, T. (2005). Preparing tomorrow's teachers to use technology: Final performancereport. Plano, TX: Collin County Community College.

Kay, R. H. (2006). Evaluating strategies used to incorporate technology intopreservice education: a review of the literature. Journal of Research on Tech-nology in Education, 38, 383e408.

Koehler, M. J., & Mishra, P. (2008). Introducing technological pedagogical knowl-edge. In AACTE. (Ed.), The Handbook of technological pedagogical contentknowledge for educators. Routledge/Taylor & Francis Group for the AmericanAssociation of Colleges of Teacher Education.

Koehler, M. J., Mishra, P., & Yahya, K. (2007). Tracing the development of teacherknowledge in a design seminar: integrating content, pedagogy, technology.Computers & Education, 49(3), 740e762.

Krueger, K., Boboc, M., Smaldino, S., Cornish, Y., & Callahan, W. (2004). InTime impactreport: what was InTime's effectiveness and impact on faculty and preserviceteachers? Journal of Technology and Teacher Education, 12(2), 185e210.

Lave, J., & Wenger, J. (1991). Situated learning: Legitimate peripheral participation(Learning in doing: Social, cognitive and computational perspectives). CambridgeUniversity Press.

Lawless, K. A., & Pellegrino, J. W. (2007). Professional development in integratingtechnology into teaching and learning: knowns, unknowns, and ways to pursuebetter questions and answers. Review of Educational Research, 77(4), 575.

Ludwig, M., & Booz, W. H. (2003). Teacher technology leaders. Washington, DC:George Washington University, Department of Teacher Preparation and SpecialEducation.

McMurray State University. (2004). McMurray State University Final Report.<http://soe.mcm.edu/pt3/evaldocs/FinalReport.pdf> Retrieved 6.11.2007.

Mims, C., Polly, D., Shepherd, C., & Inan, F. (2006). Examining PT3 projects designedto improve preservice education. Tech Trends, 50(3), 17e24.

Ministry of Education. (2004). China education reform and development. Ministry ofEducation. Retrieved25.08.09 fromhttp://www.edu.cn/20040107/3096922.shtml.

Ministry of National Education. (2005). Retrieved 25.09.09, from http://www.meb.gov.tr.

Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content knowledge:a new framework for teacher knowledge. Teachers College Record, 108(6),1017e1054.

Moursund, D., & Bielefeldt, T. (1999). Will new teachers be prepared to teach ina digital age? A national survey on information technology in teacher education.Santa Monica, CA: Milken Family Foundation.

Mullens, J. E., & Kasprzyk, D. (1999). Validating item responses on self-reportteacher surveys. Proceedings of the Survey Research Methods Section of theAmerican Statistical Association, 1999, 677e682.

Nave, B. (2004). Boston university preparing tomorrow's teachers for technology (BU-PT3) final summative report. Boston, MA: Boston University, School of Education.

D. Polly et al. / Teaching and Teacher Education 26 (2010) 863e870870

Niess, M. L. (2005). Preparing teachers to teach science and mathematics withtechnology: developing a technology pedagogical content knowledge. Teachingand Teacher Education, 21, 509e523.

Patton, M. Q. (2002). Qualitative research and evaluation methods (3rd ed.). Thou-sand Oaks, CA: Sage.

Persichitte, K. A., Tharp, D. D., & Caffarella, E. P. (1997). The use of technology byschools, colleges, and departments of education: Fall, 1996. Washington, DC:American Association of Colleges for Teacher Education.

Pierson, M. (2001). Technology integration practice as a function ofpedagogical expertise. Journal of Research on Computing in Education, 33(4),413e430.

Polly, D., & Brantley-Dias, L. (2009). TPACK: Where do we go now? Tech Trends, 53(5), 46e47.

Polly, D., & Shepherd, C. E. (2007). Preservice teachers' perceptions of appropriatetechnologies. In T. Kidd, & H. Song (Eds.), Handbook of research on instructionalsystems and technology (pp. 198e215).

Preston, C., & Cuthell, J. (2007). Education professionals' perspectives on ICT CPD: past,present and future. Centre for excellence in work-based learning for educationprofessionals, London: Institute of Education, University of London. Retrieved20.03.07, from http://www.wle.org.uk/.

Resta, P. E., Petrosino, A., & Confrey, J. (2004). Inventing new strategies for integratingtechnology into education: Project INSITE evaluation report. Austin, TX: Universityof Texas-Austin.

Richardson, K. R. & Hannafin, R. D. (2005, March). Making a technology grant work:Lessons from policy implementation research. Paper presented at the 2005Society for Information Technology in Teacher Education InternationalConference, Phoenix, AZ.

Roschelle, J., Pea, R., Hoadley, C., Gordin, D., & Means, B. (2001). Changing how andwhat children learn in schools with computer-based technologies. The Future ofChildren, 10(2), 76e101.

Schaffer, S., & Richardson, J. (2004). Supporting technology integration acrossthe teacher education system. Paper presented at the 2004 Society for

Information Technology in Teacher Education International Conference,Atlanta, GA.

Schacter, J. (1999). The impact of education technology on student achievement: Whatthe most current research has to say. Santa Monica, CA: Milken Exchange onEducation Technology.

Schulman, L. S. (1987). Knowledge and teaching: foundations of the new reform.Harvard Educational Review, 57(1), 1e22.

Staples, A., Pugach, M. C., & Himes, D. (2005). Rethinking the technology integrationchallenge: cases from three urban elementary schools. Journal of Research onTechnology in Education, 37(3), 285e311.

Strudler, N., Archambault, L., Bendixen, L., Anderson, D., & Weiss, R. (2003). ProjectTHREAD: technology helping restructure educational access and delivery.Educational Technology Research and Development, 51(3), 41e56.

United Kingdom Department of Children, Schools, and Families. (2009). ICT at keystages 1 and 2. Retrieved 25.08.09 from http://www.standards.dfes.gov.uk/schemes2/it/teaching?view ¼ get.

University of Missouri. (2003). MU Partnership for preparing tomorrow's teachers touse technology. Final Evaluation Report prepared for the U.S. Department ofEducation. Columbia, MO: Author.

Vetter, R., Sologuk, S., & Stammen, R. (2001). A collaborative approach for creatingcurriculum and instructional materials. Journal of Technology and TeacherEducation, 9(2), 199e210.

Waddoups, G. L., Wentworth, N., & Earle, R. (2004). Principles of technology inte-gration and curriculum development: a faculty design team approach.Computers in the Schools, 21, 15e23.

Wang, Y. (2002). When technology meets beliefs: preservice teachers' perception ofthe teacher's role in the classroom with computers. Journal of Research onTechnology in Teacher Education, 35, 150e161.

Wenger, E., McDermott, R., & Snyder, W. M. (2002). Cultivating communities ofpractice: A guide to managing knowledge. Boston: Harvard Business School Press.

Wentworth, N., Waddoups, G. L., & Earle, R. (2004). Technology integration intoa teacher education program. Computers in the Schools, 21, 1e14.