Assess ESL Teacher Attitudes To Integrating Technology in Teaching

Assessing ESL Teacher Attitudes To Integrating Technology in Teaching

Troy CoxTESOL 2010, Boston

Handout: troycox.byu.edu, click on Conference Presentations

! English Language Center (ELC) at Brigham Young University (BYU)

! Teacher Training/Lab School for graduate students in Linguistics and English Language Department

! Intensive English Program for NNS wanting academic English preparation

Context

! Average teacher at the ELC has between 2 and 3 semesters teaching experience.

! Faculty Facts & Assumptions are in ELC Philosophy Document at

http://www.elc.byu.edu/teacher/philosophy/ELC%20Philosophy.pdf

Faculty

Technology Resources! 5 Computer Classrooms (77 machines)

! Labs used as classrooms for 5 hours of the day and open use for 5 hours a day

! 8 LCD projectors (5 portable, 3 in classrooms)

! AV (TVs, OHPs, CD Players, etc.)

! Prerequisites to teach at the ELC are:

! Methods class (1st semester)

! Student Teaching (2nd semester)

Requirements Needed to Teach

! Brigham Young University’s TESL MA was revised as a result of feedback received from…

! an academic review of the graduate programs in the Department of Linguistics and English Language conducted in 2006-2007

! the university reaccreditation process and the resulting discussions on the important role of learning outcomes

! a survey of graduates from BYU’s TESOL program.

M.A. TESOL Revision Background

What should the role of CALL be in the new program?

! BYU has always had great resources for students who are interested in CALL.

! The problem was that nothing systematic had been organized to ensure that all the students had CALL exposure.

!

! What should be taught?

! How should it be taught?

Two Questions

Needs Analysis

! Establish baseline of current technological knowledge

! Instrument

! Technology Integration Confidence Scale (TICS)http://www.brownelearning.org/tics/#5

! Based on National Education Technology Standards for Teachers (NETS-T)

National Education Technology Standards for

Teachers (NETS-T)I. Technology Operations and Concepts

II.Planning and Designing Learning Environments and Experiences

III.Teaching, Learning, and the Curriculum

IV. Assessment and Evaluation

V. Productivity and Professional Practice

VI. Social, Ethical, Legal, and Human Issues

I. Technology Operations and Concepts

A. Teachers demonstrate introductory knowledge, skills, and understanding of concepts related to technology (as described in the ISTE National Educational Standards for Students.)

B. Teachers demonstrate continual growth in technology knowledge and skills to stay abreast of current and emerging technologies.

II. Planning and Designing Learning Environments and Experiences

A. Teachers design developmentally appropriate learning opportunities that apply technology-enhanced instructional strategies to support the diverse needs of learners.

B. Teachers apply current research on teaching and learning with technology when planning learning environments and experiences.

C. Teachers identify and locate technology resources and evaluate them for accuracy and suitability.

D. Teachers plan for the management of technology resources within the context of learning activities.

E. Teachers plan strategies to manage student learning in a technology-enhanced environment.Teachers demonstrate introductory knowledge, skills, and understanding of concepts

III. Teaching, Learning, and the Curriculum

A. Teachers facilitate technology-enhanced experiences that address content standards and student technology standards.

B. Teachers use technology to support learner-centered strategies that address the diverse needs of students.

C. Teachers apply technology to develop students’ higher order skills and creativity.

D.Teachers manage student learning activities in a technology-enhanced environment.


A. Teachers apply technology in assessing student learning of subject matter using a variety of assessment techniques.

B. Teachers use technology resources to collect and analyze data, interpret results, and communicate findings to improve instructional practice and maximize student learning.

C. Teachers apply multiple methods of evaluation to determine students appropriate use of technology resources for learning, communication, and productivity.


A. Teachers use technology resources to engage in ongoing professional development and lifelong learning.

B. Teachers continually evaluate and reflect on professional practice to make informed decisions regarding the use of technology in support of student learning.

C. Teachers apply technology to increase productivity.

D.Teachers use technology to communicate and collaborate with peers, parents, and the larger community in order to nurture student learning..

VI. Social, Ethical, Legal, and Human Issues

A. Teachers model and teach legal and ethical practice related to technology use.

B. Teachers apply technology resources to enable and empower learners with diverse backgrounds, characteristics, and abilities.

C. Teachers identify and use technology resources that affirm diversity.

D. Teachers promote safe and healthy use of technology resources.

E. Teachers facilitate equitable access to technology resources for all students.

Technology Integration Confidence Scale (TICS)

! Advantages

! The survey doesn’t take very much time

! Many people are quite comfortable admitting technological confidence


! Disadvantages

! Often a mismatch between confidence and actual ability

! Written with K-12 perspective, so some items not applicable to future goals of all the TESOL MA students

Revised TICS

! Added a section on demographics

! Expanded questions related to Standard I:Technology Operations and Concepts.

! Added section on what Technological Knowledge students should have prior to admittance to an MA program vs. what they should have when they leave

! Added some open-ended response questions

InstrumentObjectives (NETS-T Standards & TESOL MA Needs Analysis) # of Items

Demographic DataTICS I Technology Operations and Concepts 8Added Section Ia Technology Operations and Concepts-Revised 14TICS II Planning and Designing Learning Environments and Experiences 7TICS III Teaching, Learning and Curriculum 5TICS IV Assessment and Evaluation 4TICS V Productivity and Professional Practice 5TICS VI Social, Ethical, Legal, and Human Issues 4

Likert VII TESOL MA Technology Prerequisites 18Likert VIII TESOL MA Technology Post Grad Skills 18

Opinion IX Qualitative Responses 2Total Items 85

Methods

! Survey emailed to…

! TESOL Faculty

! TESOL MA & Certificate Graduates

! Teachers in currently TESOL Graduate Programs

! Pre-Service Teachers in TESOL Graduate Programs

Findings

! Total Responses:% % 67

! Surveys completed:% 59

! Trimmed Mean:% % 33 minutes

! Duration Median:% 21 minutes

! Duration Mode:% % 10 - 14 minutes

DemographicsGender

N %Male 16 26%

Female 46 74%Total 62 100%

DemographicsAgeRespons

eN %

Under 20 0 0%20-24 7 11%25-29 25 40%30-34 11 18%35-39 3 5%40-44 3 5%45-49 3 5%50-54 6 10%55-59 1 2%

Over 60 3 5%

Total 62 100%

DemographicsExperience

Response N %Professor (TESOL Faculty) 8 10%

ESL Teacher (with Master's Degree or higher) 18 23%

ESL Teacher (with TESOL Certificate) 11 14%

ESL Teacher (current graduate student) 8 10%

Student Teacher 7 9%

Graduate Student in TESOL Program 18 23%

Undergraduate Student in TESOL Program 2 3%

Former ESL Teacher 3 4%

Other 2 3%

Total 77 100%


Results

ReliabilityDemographic Interactions

Instrument ReliabilityObjectives (NETS-T Standards & TESOL MA Needs Analysis) # of Items Cronbach’s Alpha

I Technology Operations and Concepts 8 0.90

Ia Technology Operations and Concepts -Revised 14 0.93

II Planning and Designing Learning Environments and Experiences 7 0.93

III Teaching, Learning and Curriculum 5 0.90

IV Assessment and Evaluation 4 0.85

V Productivity and Professional Practice 5 0.87

VI Social, Ethical, Legal, and Human Issues 4 0.81

VII TESOL MA Technology Prerequisites 18 0.89

VIII TESOL MA Technology Post Grad Skills 18 0.90

Overall 83 0.94

Is confidence level affected by any moderating variables?

! An ANOVA was run with Confidence Level as the Dependent Variable and Gender, Age, and Experience as Independent Variables.

Gender

Age

Experience

Source Type III Sum of Squares df Mean Square F Sig.

Corrected Model 26.031a 29.00 0.90 1.87 0.04Intercept 598.96 1.00 598.96 1249.01 0.00Gender 0.31 1.00 0.31 0.65 0.43Age 8.83 8.00 1.10 2.30 0.05Experience 5.74 3.00 1.91 3.99 0.02Gender * Age 3.12 4.00 0.78 1.63 0.19Gender * Experience 0.03 1.00 0.03 0.07 0.80Age * Experience 8.16 8.00 1.02 2.13 0.06Gender * Age * Experience 0.01 1.00 0.01 0.03 0.87Error 15.35 32.00 0.48Total 1485.85 62.00Corrected Total 41.38 61.00

a. R Squared = .723 (Adjusted R Squared = .350)

Discussion

! REMEMBER: Confidence does not mean competence.

! It’s good that Teacher Trainers feel more confident than the Students.

! Younger people seem to be more confident than the older ones, thus older learners may need more support.

! Even if confidence level isn’t significant, it would be good to keep in mind WHO you are teaching.


Results

Subsection and Item Analysis

TICS Subsections Least to Most Confident (Sections I-VI)

VI. Social Ethical Legal and Human Issues

III. Teaching Learning and Curriculum

Ia. Technology Operations and Concepts -Revised

II. Planning and Designing Learning...



I. Technology Operations and Concepts

4.0 4.5 5.0 5.5 6.0

N Mean

SD

59 4.49 0.94

61 4.50 1.07

62 4.60 1.06

61 4.64 1.04

60 4.93 0.94

60 5.31 0.71

63 5.39 0.78

Fairly confident

Quite confident

Completely confident

TICs Subsections Items

VI Social, Ethical, Legal, and Human Issues(N=59, Mean=4.49, SD=0.94)

IIITeaching, Learning and Curriculum(N=61, Mean=4.50, SD=1.07)

IaTechnology Operations and Concepts -Revised(N=62, Mean=4.60, SD=1.06)

IIPlanning and Designing Learning Environments and Experience(N=61, Mean=4.64, SD=1.04)

IV Assessment and Evaluation(N=60, Mean=4.93, SD=0.94)

VProductivity and Professional Practice(N=60, Mean=5.31, SD=0.71)

ITechnology Operations and Concepts(N=63, Mean=5.39, SD=0.78)

VI-31VI-32VI-33VI-30III-16III-19III-17III-18III-20

Ia-6a06Ia-6a11

Ia-6a04Ia-6a08Ia-6a10Ia-6a07Ia-6a05Ia-6a12Ia-6a14Ia-6a01Ia-6a02Ia-6a03Ia-6a13

Ia-6a09II-14II-15II-13II-10II-09II-11II-12IV-23IV-24IV-21IV-22V-26V-28V-27V-29V-25

I-8I-7I-2I-4I-6I-3I-1I-5

3.0 3.5 4.0 4.5 5.0 5.5 6.05.75.75.65.65.55.44.94.7 5.55.45.35.35.15.25.04.84.7 5.15.04.74.54.44.44.4 5.55.45.14.94.94.84.74.74.64.24.24.03.93.8 5.04.94.44.24.1 5.04.54.34.1

Fairly confident

Quite confident


Somewhat confident

TICs Subsections

VI

Social, Ethical, Legal, and Human IssuesN=59Mean=4.49SD=0.94)

VI-31

VI-32

VI-33

VI-30

3.0 3.5 4.0 4.5 5.0 5.5 6.0

5.0

4.5

4.3

4.1

Fairly confident

Quite confident


Somewhat confident

VI. Social Ethical Legal and Human Issues

(N=59, Mean=4.49, SD=0.94)

Question N Mean SD

VI-31. When some of your students do not have access to technology outside the classroom, how confident are you that you can appropriately, legally, and ethically lessen the effects of such unequal access?

59 4.10 1.20

VI-32. Because students are using the Internet and other technologies in school, they must be instructed how to stay safe while getting the most from these resources. How confident are you that you can model and teach safe usage of technology, including Internet safety?

59 4.32 1.32

VI-33. Technology can help students accomplish tasks, good or ill. For example, students can find images of rare historical artifacts, but they can also illegally obtain copyrighted materials online (such as music). Telecommunications technology can bring the world into your classroom, and allows students to text one another exam answers via cell phones. How confident are you that you can model and teach ethical and legal use of technology?

59 4.54 1.22

Discussion

! Make sure the curriculum includes strategies for…

! technology access for students.

! Internet safety.

! ethical and legal use of technology ESP. when in different country cultural contexts.

TICs Subsections

III

Teaching, Learning and CurriculumN=61Mean=4.50SD=1.07

III-16

III-19

III-17

III-18

III-20

3.0 3.5 4.0 4.5 5.0 5.5 6.0

5.0

4.9

4.4

4.2

4.1

Fairly confident

Quite confident


Somewhat confident

Question N Mean SD

III-16. Your principal promises full support for any technology that can be linked to the state’s core curriculum standards. How confident are you that you can find technologies that will help you meet these standards in your subject?

61 4.11 1.36

III-19. Due to a personal emergency, a fellow teacher asks you to teach his computer lab period during your preparation time this afternoon. How confident are you that you can make good use of the class time without the opportunity to plan?

61 4.21 1.33

III-17. Current educational practice stresses ‘higher order’ thinking skills such as analysis, synthesis, and evaluation. How confident are you that you can use technology to improve these skills in your students?

61 4.38 1.44

III. Teaching Learning and Curriculum Analysis

(N=61, Mean=4.50, SD=1.07)

Discussion


! matching technology to educational standards/objectives.

! adapting technology to any lesson plan.

! using technology for higher order thinking skills.

TICs Subsections

Ia

Technology Operations and Concepts -RevisedN=62Mean=4.60SD=1.06

Ia-6a06Ia-6a11

Ia-6a04Ia-6a08Ia-6a10Ia-6a07Ia-6a05Ia-6a12Ia-6a14Ia-6a01Ia-6a02Ia-6a03Ia-6a13

Ia-6a09

3.0 3.5 4.0 4.5 5.0 5.5 6.0

5.5

5.4

5.1

4.9

4.9

4.8

4.7

4.7

4.6

4.2

4.2

4.0

3.9

3.8

Fairly confident

Quite confident


Somewhat confident

Ia. Technology Operations and Concepts -Revised

(N=62, Mean=4.60, SD=1.06)

Question N Mean SDIa-6-6. You just got an mp3 player (e.g. iPod, Zune, etc). You want to subscribe to a podcast and have it load automatically to your mp3 player. How confident are you that you can accomplish this?

62 3.76 1.75

Ia-6-11. Your internet connection quit working. How confident are that you can identify and resolve problem so you can get on-line again? 61 3.85 1.53

Ia-6-4. You have a digital video camera and want to load some video on your computer, do some minor editing and burn it on a DVD. How confident are you that you can accomplish this?

62 3.95 1.66

I-6a8. A friend has just bought a new TV and is having difficulty attaching the peripherals (e.g. DVD player, stereo, video game system, etc.) How confident are you in your ability to help get it to work?

62 4.15 1.50

I-6a10. You just sent a document to the printer, and it didn't print. How confident are that you can identify and resolve the printer problem? 61 4.18 1.32

Discussion


! using peripherals (printers, digital cameras, mp3s, etc.) with computers

! incorporating multi-media devices

! networking

TICs Subsections

II

Planning and Designing Learning Environments and ExperienceN=61Mean=4.64SD=1.04

II-14

II-15

II-13

II-10

II-09

II-11

II-12

3.0 3.5 4.0 4.5 5.0 5.5 6.0

5.1

5.0

4.7

4.5

4.4

4.4

4.4

Fairly confident

Quite confident


Somewhat confident

Question N Mean SDII-15. Your district has allocated money to purchase educational technology products for your subject/grade. The board has asked for input to help them decide between two competing products. How confident are you that you can advise them on this purchase by evaluating the products for their suitability to your teaching situation?

61 4.36 1.239

II-14. A vice principal is upset that the new equipment that was donated to the school is not being used. She asks if you can demonstrate how to use it at the next inservice meeting. How confident are you that you can accomplish this task?

61 4.36 1.46

II-13. An educational software vendor gives a sales pitch to your department. How confident are you that you can evaluate their products for their suitability to your teaching environment?

61 4.44 1.285

II-10. A member of the PTA feels there is too much technology in the school and states that not all technologies are equally applicable to your classroom, and not all student learning goals are well suited for technology. How confident are you that you can effectively judge when and how to use technology to support your students’ learning?

61 4.52 1.273

II. Planning and Designing Learning Environments and Experience

(N=61, Mean=4.64, SD=1.04)

Discussion


! reviewing software from vendors.

! making purchasing decisions.

! performing inservice training for other teachers.

! justifying technology needs.

TESOL Program Pre-Requisite and Post-graduation

Requirements

Analysis

How important is it for TESOL graduate students to be able to CONFIDENTLY USE the following software packages/tools BEFORE THEY ARE ADMITTED? (N=59)

EmailWord Processing

Presentation Applications Spreadsheets

Online Collaboration ToolsConcordancing/Textual Analysis Software

Course Management SoftwareGraphics Applications

WikisDatabase Applications

PodcastsAudio Recoding/Editing Applications

Web Development ApplicationsVideo Sharing SitesInstant Messaging

BlogsSocial Networking Sites

Video Editing Applications

1 2 3 4 5 6

3.4

3.4

3.4

3.4

3.6

3.7

3.8

3.9

3.9

4.1

4.1

4.3

4.4

4.4

4.6

5.2

5.8

5.8

Somewhat Important

Very Important

Extremely Important

Somewhat Unimportant

Very Unimportant

Not at all important

Mean SD5.83 0.75.8 0.71

5.17 0.774.59 0.974.41 1.3414.36 2.054.29 1.16

4.1 1.554.05 1.5913.92 1.863.88 1.663.76 1.2363.68 1.7363.64 1.613.44 1.573.39 1.3263.37 1.483.36 1.54

Discussion! Create prerequisites (perhaps a test) that includes

! Email

! Word Processing

! Presentation Applications

! Spreadsheets

! Online Collaboration (perhaps)

! Technology Skills Assessment—Example

How important is it for TESOL graduate students to be able to CONFIDENTLY USE AND INTEGRATE IN THEIR TEACHING the following software packages/tools AFTER THEY GRADUATE? (N=59)

Word Processing Email

Presentation Applications Concordancing/Textual Analysis Software

Spreadsheets Course Management Software

PodcastsOnline Collaboration Tools

Graphics ApplicationsWikis

Database Applications Audio Recoding/Editing Applications

Video Sharing SitesVideo Editing Applications

Web Development Applications Blogs

Social Networking SitesInstant Messaging

1 2 3 4 5 6

3.9

4.2

4.3

4.4

4.6

4.6

4.8

4.8

4.9

4.9

4.9

5.0

5.1

5.2

5.3

5.5

5.7

5.8

Somewhat Important

Very Important

Extremely Important

Somewhat Unimportant

Very Unimportant

Not at all important

Mean SD5.78 0.7215.73 0.7625.47 0.8585.31 1.4895.24 1.0725.1 1.094

4.97 1.5084.93 1.0154.9 1.4234.85 1.4834.81 1.5814.75 1.3594.64 1.4714.56 1.5464.42 1.4044.32 1.2924.2 1.543.85 1.595

Skills to Teach (Post-Pre)Video Editing Applications

PodcastsVideo Sharing Sites

Audio Recoding/Editing ApplicationsConcordancing/Textual Analysis Software

BlogsDatabase Applications

Social Networking SitesCourse Management Software

Graphics ApplicationsWikis

Web Development Applications Spreadsheets

Online Collaboration Tools Instant Messaging

Presentation Applications Word Processing

Email

-0.10 0.16 0.420.68 0.94 1.2

0

-0.1

-0.0

0.3

0.4

0.5

0.6

0.8

0.8

0.8

0.8

0.8

0.9

0.9

1.0

1.0

1.0

1.1

1.2

Mean SD1.20 1.751.08 1.561.00 1.560.98 1.480.95 1.730.93 1.200.90 2.010.83 1.480.81 1.360.80 1.950.80 1.560.75 1.490.64 1.200.53 1.420.41 1.310.31 0.77-0.02 0.35-0.10 0.36

Discussion

! Create objectives for either specific classes or program-wide(e.g. portfolio) that include demonstration of using…

! Video Applications (Editing & Collaboration)

! Audio Applications (Editing & Podcasts)

! Concordancing/Textual Analysis

! Web use (Blogs, social networking, etc)

! Database applications

Would adding a class be sufficient?! Probably not.

! Most of the research shows that a single class isn’t very effect in training teachers in CALL.

What to teach?

Implementing the Needs Analysis

into a TESOL/Teacher Training

Program

Technological Pedagogical Content Knowledge: A Framework for Teacher Knowledge

TechnologicalPedagogical

ContentKnowledge

Content Pedagogy

Technology

Koehler, M. J. & Mishra, P. (2008) Technological Pedagogical Content Knowledge (PCK) for Educators

How to Teach

! There is insufficient time/credit hours to teach M.A. students the fundamentals of the Technological Knowledge & Concepts.

! Require students to demonstrate pre-requisite skills within the first semester of being admitted to the program.

! Opportunities will be provided for students to learn technology on their own throughout the program through problem-based learning projects.

How to Teach

! Technology should be integrated in every course

! Graduates needs to know how to use technology as a WILD card to accomplish whatever pedagogical content knowledge objective they are trying to meet.

! If a student only knows how to use one technology (e.g. chalkboard, OHP, LCD projector, computer lab, etc.), then they are woefully underprepared for future teaching environments

Qualitative Findings

! Realize there will be different levels of competence in Technology Knowledge

! I realize that most students under 35 will know these technologies well already. It is the rest of us who need to know them -- and may have a harder time doing so. Don't hold the rest back but offer a technology course for those of us who are "technologically challenged" but don't want to be.

Qualitative Findings! Don’t teach technology—teach technological application

! Teaching directly any technology has limited benefits because technology has such a short shelf-life. For example, there are plenty of teachers that may have spent hours of learning Hypercard as a graduate student but should or could that teacher still be using Hypercard today? Rather I think instruction should focus on problem solving skills, methods to stay current on how technology is currently being used, gauging the effectiveness of technology and its impact on learning and retention, and plenty of exposure to technology being used innovatively in the classroom through direct observation.

Documents

Assess ESL Teacher Attitudes To Integrating Technology in Teaching