Development of an Instrument to Measure Preservice Teachers' Technology Skills, Technology Beliefs, and Technology Barriers

  • Published on

  • View

  • Download


  • Jl. of Technology and Teacher Education (2013) 21(2), 179-202

    Assessing Pre-service Teachers Information and Communication Technologies Knowledge

    Cindy KovaliK, Chia-ling Kuo, and aryn KarpinsKiKent State University, USA

    The impact of a redesigned educational technology course on preservice teachers knowledge and skills with regard to information and communications technology as defined by isTEs national Educational Technology standards for Teachers (nETs-T) was investigated in this study. Two changes were made to the course: (1) a commercially avail-able technology assessment aligned with isTEs nETs-T was administered, and (2) students completed observations of a technology-rich classroom. results indicated that the preservice teachers made significant progress in technology knowledge in all five standard areas of the nETs-T; how-ever, some performance indicators of the standards may not have been adequately addressed in the course. student reflec-tions from the observation experience provided evidence that preservice teachers found it helpful for their future teaching. although the majority of preservice teachers agreed that the observations were beneficial, a few preservice teachers had concerns about the integration of technology into lessons, es-pecially when trying to determine if technology integration is necessary, dealing with unexpected technical problems, managing the classroom, and assisting children with special needs.

    pre-service teachers need to gain the knowledge, skills, and attitudes in their education programs that will prepare them to effectively integrate

  • 180 Kovalik, Kuo, and Karpinski

    information and communication technologies (iCT) in their future class-rooms. one key to ensuring pre-service teachers have these skills is to in-clude well-designed technology-rich courses throughout teacher education programs that emphasize pedagogy as well as embedding and modeling ap-propriate technology knowledge, skills, and integration (Koc & Bakir, 2010; Koh, 2011; polly & Moore, 2008).

    a second key to ensuring pre-service teachers have strong iCT knowl-edge and skills is through adequate assessment strategies (stobaugh & Tas-sell, 2011). self-assessment surveys are often the instruments of choice for assessing pre-service and in-service teacher iCT knowledge and skills (at-kay, 2009; Brush, glazewski, & hew, 2008; hartshorne, Miller, & gretes, 2009; Minaidi & hlapanis, 2005); however, these types of surveys rely on self-determination of skills and knowledge rather than validated and reliable tests. Thus, few studies report on pre-service teachers iCT skills as mea-sured by a skills-based test.

    opportunities for pre-service teachers to observe effective technol-ogy integration in actual classrooms is a third key in providing pre-service teachers with the experiences needed to help them learn how to effective-ly teach with technology. although studies have found many pre-service teachers possess strong beliefs about the value of using technology for learning and feel confident in their technology skills, many of these pre-ser-vice teachers do not feel prepared to effectively integrate technology into their future classrooms (lei, 2009; polly & Moore, 2008). This feeling of being ill-equipped to effectively use technology with students may be the result of pre-service teachers having few opportunities to observe in-service teachers using technology effectively in actual K-12 classrooms.

    To ensure these three factors were part of the education program at our university, the education technology course that already contained technol-ogy-rich project-based assignments, was modified. The modifications in-cluded the addition of a commercially-available assessment instrument to measure pre-service teachers iCT knowledge and skills and the addition of required observations of classroom activities in a technology-rich class-room. Both the course and the assessment instrument were based on the in-ternational society for Technology in Educations (isTE) national Educa-tional Technology standards for Teachers (nETs-T). study results indicate that students made significant progress in their iCT knowledge and skills and that they recognized the value of observing in-service teachers using multiple technologies with K-12 students.

  • Assessing Pre-service Teachers Information and Communication 181

    Literature Review

    The dual nature of iCT within a teacher education program makes it complex and multifaceted. For not only do pre-service teachers need to gain content knowledge about the subjects they will teach and the methods and strategies to effectively teach those subjects, but future teachers also need to master both iCT knowledge for themselves and pedagogical knowledge about how to integrate technology in K-12 classrooms (Markauskaite, 2007). This combination of knowledge and skills affords teachers abilities to work effectively in helping their students attain both curriculum and iCT learning goals.

    Further, researchers have found that when teacher education courses effectively integrate technology within content areas, encourage collabora-tive efforts with in-service teachers, and include field experiences that in-clude specific technology uses, pre-service teachers are more predisposed to use technology in their future classrooms (Chen, 2010; Xiaodong, Mciner-ney, & Frechtling, 2010). Kim & hannafin (2011) addressed the perceived lack of observation opportunities by developing web-based case studies that helped pre-service teachers refine and strengthen their understanding of what it means to teach with technology. in another study, a combination of observations of technology use by university instructors and cooperating teachers and hands-on experiences with technology resulted in increased confidence in computer skills for pre-service teachers (Fleming, Motamedi, & May, 2007). Thus, high computer self-efficacy also may influence future integration of technology.

    one way that computer self-efficacy may be increased is through ef-fective interactions between classroom instructors and pre-service teach-ers in an educational technology course designed to teach technology skills (Koh & Frick, 2009). in the Koh and Frick study, pre-service teachers com-puter self-efficacy was supported and enhanced when instructors used effec-tive strategies including show and tell, progress checking, and inviting sug-gestions from students. The researchers hypothesized that the use of these types of strategies, by positively impacting pre-service teachers comfort and self-efficacy with technology, may also be a positive factor leading to increased technology integration when these pre-service teachers become teachers themselves (Koh & Frick, 2009).

    in a study by Teo (2009), significant relationships were found between pre-service teachers perception of their self-efficacy in both basic computer skills and using technology for pedagogical purposes, and how they intend-ed to use technology in their future classrooms. in other words, two factors,

  • 182 Kovalik, Kuo, and Karpinski

    namely, belief that one possesses good technology skills and that one has a good understanding of how technology can be used for pedagogical pur-poses, are significant predictors of how pre-service teachers intend to use technology with students (p. 13).

    regardless of how much interest pre-service teachers have in intending to integrate technology in their future classrooms, that interest is tempered by reservations they may have about the integration process, their limited skills in using collaborative tools such as wikis and blogs, their lack of ad-vanced computer skills, and their limited use of technology in their own K-12 experiences (lei, 2009). To address these issues, lei suggested pre-service teachers have more exposure to the use of a variety of technologies that can be used to support teaching and learning, including subject-specific technologies and assistive technologies. pre-service teachers may also ben-efit from knowledge of the conditions necessary for technology integration (lei, 2009), and how technology, pedagogy, and content knowledge can work together for meaningful learning (Mishra & Koehler, 2006).

    recognizing the need to teach and assess pre-service teachers basic and advanced computer skills, to relate those skills to specific content areas and K-12 curriculum standards, and to provide pre-service teachers with op-portunities to observe in-service teachers as they actively and effectively in-tegrate technology into K-12 classrooms, the required educational technolo-gy course at a large Midwestern university was modified to include a skills-based assessment and required observations in a technology-rich classroom.

    Research Question

    The purpose of this study was to assess the general impact of a rede-signed educational technology course. First, changes in teacher information and Communications Technology (iCT) knowledge were assessed via the administration of a survey based on the international society for Technol-ogy in Educations (isTE) national Educational Technology standards for Teachers (nETs-T). second, the impact of a redesigned educational tech-nology course was examined through classroom observations and reflective papers. The specific research question was: (1) how does a redesigned edu-cational technology course influence pre-service teacher iCT knowledge as defined by isTE-T standards?

  • Assessing Pre-service Teachers Information and Communication 183



    undergraduate pre-service teacher education students (N = 100) en-rolled in an educational technology course at a large, Midwestern univer-sity were recruited to participate in the study. There were 66 females and 34 males. The majority of participants were juniors (n = 48), followed by sophomores (n = 27), seniors (n = 23), and freshmen (n = 2). Finally, the largest group of majors were Early Childhood Education (n = 27), with inte-grated social studies (n = 12) and Mathematics (n = 11) as the next highest categories. other majors represented included special Education, physical Education, art Education, and intervention specialist.


    The Educational Technology Course. Educational Technology is an introductory technology course with the aim of increasing pre-service teach-ers technological competency and preparing them to use information and communication technology appropriately and effectively in their future classrooms. students enrolled in the Educational Technology course major in subjects across the disciplines and have had very different prior experi-ences with computer technology. Most of the students enrolled are in their second or third year of study. These students use technology for commu-nication purposes (e.g., cell phone, text messaging), personal pleasure, and networking (e.g., ipod, Facebook). however, most of these students have never thought about applying any of these technologies in the classroom or integrating technology into teaching, or they have reservations about how and when to use technology with students. Their ideas of teaching typically derive from the way they were taught, which may not have included much technology.

    To help students acquire necessary technology competencies, increase their knowledge of various technology tools, and connect technology to their proposed content areas, project-based learning is used in the course. Based on students proposed content areas and grade level(s), they design technology-integrated lesson plans and create teaching materials, including computer-based learning modules, videos, and WebQuests. in addition, stu-dents in the course identify web resources for their specific target learners and explore and study Web 2.0 tools that can be used in the classroom. an electronic portfolio is the culminating assignment for the course.

  • 184 Kovalik, Kuo, and Karpinski


    Simple Assessment. The measure of pre-service teacher technology knowledge used was the Teacher Technology proficiency assessment. The assessment was designed by simple K-12 (infosource, inc.) for classroom educators to ensure that teachers are effectively using technology to increase student achievement. licenses for the simple assessment instrument were purchased from simple K-12 (infosource, inc.), and, as a licensed product, we are unable to include samples of the web-based assessment. The assess-ment consists of a pretest and posttest (i.e., the same test given twice in one semester). Each test included the same, randomized 60 online, interactive, multiple-choice, performance-based questions aligned with the 2008 nETs-T. in terms of content validation, the test was developed with the help of educators in technology from all over the united states who reviewed the questions and gave their feedback. The test was revised based on the feed-back and given to the educators to review again. There were three rounds of review.

    Each question was identified as addressing specific nETs-T standards and their performance indicators. The questions involve demonstrating flu-ency in multiple productivity programs, presenting an understanding of incorporating digital tools to enhance teaching and learning, choosing the best technological tools to maximize content learning in local, global, and online communities, modeling legal and ethical use of digital information and technology, and indicating possible engagement in professional growth and leadership. The test provides instant feedback and detailed reporting of results. Twelve questions addressing each of the five standards comprise the test. grant funds were used to cover the cost of the test.


    Intervention. The educational technology course was redesigned to provide for the administration of a commercially-developed iCT assessment instrument and to provide students an opportunity to observe in-service teachers actively using technology with students in a technology-rich class-room environment, namely the aT&T Classroom housed on the university campus. The redesign effort consisted of two course modifications, (1) pre- and post-administration of Teacher Technology proficiency assessment, (2) observation in the aT&T Classroom coupled with a reflective paper assign-ment.

  • Assessing Pre-service Teachers Information and Communication 185

    Completing the Teacher Technology Proficiency Assessment. The pre-service teachers took the pretest during the first week of the semester (september, 2011) and the posttest at the end of the semester (december, 2011). The pre and posttests were the same, randomzied version of the Teacher Technology proficiency assessment. students were awarded credit based on the completion of the assessment instead of their actual test scores.

    Classroom Observation on Technology Integration. The aT&T classroom, which opened in spring 1998 on the main university campus, is a learning and research la...


View more >