Developing Instrument for Measuring The Qualities of Effective Teacher

• Kartika Yulianti, I Made Agus Ana Widiatmika, Riris Lumban Tobing

Sampoerna Foundation Teacher Institute Abstract

Teacher quality matters a great deal in terms of student learning. Therefore, teacher quality measurement is important. Researchers who study teacher quality put a great interest in investigating the quality of effective teacher. National issue in Indonesia regarding teachers is merely to improve the qualification of teachers through teacher certification. However, the certification more emphasizes on product which is portfolio rather than focusing on performance in teaching process. This research focused on developing a valid and reliable instrument to complement the portfolio. A number of policy makers and researchers have proposed that effectiveness, as determined by teachers’ contribution to student learning, should be an important component of assessing teacher quality. The purposes of this research are to (1) determine the components of teacher quality and (2) to apply the Teacher Quality Index (TQI) for recognizing the qualities of effective teacher. A survey involved 534 students in public and private high schools across Indonesia during May-June 2008 to identify their perceptions regarding components and variables in teacher effectiveness. The TQI was then applied for interviewing teachers to recognize their qualities of effectiveness. Keywords: Effective Teachers; Teacher Quality Index (TQI)

i

INTRODUCTION Background

The effectiveness of a teacher is often associated with its quality in terms of student

achievement. A large number of evidence suggests that schools can make a great difference in terms of student achievement, and a significant portion of that difference is attributable to teachers. Effective teachers are generally recognized as those who capable of inspiring significantly learning gains in their students when compared with their weaker colleagues.

Regulation of Ministry of National Education No.1, 2007 article 1 states that every teacher must fulfill the academic qualification standard and teacher competency. Teacher competency includes pedagogical, personal, social, and professional competencies while the qualifications are achieved by getting higher education certificate, namely bachelor (S1) or 4 year higher education (D4). National issue regarding teachers is to improve the quality of teachers through teacher certification. Teacher certifications aims to (1) ensure a teacher’s competency in carrying out their work as agents of learning and to realize the goals of national education; (2) to improve the process of learning and the quality of learning outcomes; and (3) to improve teacher professionalism. A teaching certificate is given to teachers who have fulfilled both academic and competency qualifications. Regulation of Ministry of National Education No. 18, 2007 states that teacher certification is to be done through an examination of competency in the form of a portfolio assessment. A portfolio is a physical document that describes the teacher’s experience and achievements attained during their teaching profession within a set time interval. There are ten portfolio components comprising (1) academic qualifications; (2) education and training; (3) teaching experience; (4) planning and carrying out the teaching and learning process; (5) assessment from superiors and superintendents; (6) academic achievements; (7) professional development work; (8) participation in scientific forums; (9) organizational involvement within educational or social subjects; as well as (10) awards relevant to education.

However, the certification hardly assesses the four teacher competencies (especially pedagogical and professional competencies) and improves teacher quality because the methodology more emphasizes on document portfolio rather than the process of teaching in the classroom. Lumbantobing (2006) noted that in elementary school level, Indonesian teachers more emphasize on expository teaching rather than problem solving teaching approach compared to Japanese teachers. This indicates that teacher performance should be assessed in the real classroom practice.

This research concern is to develop a valid and reliable instrument which can be used to assess the quality of effective teacher. There are five components of effective teachers in the Teacher Quality Index (TQI) used in this research. The main components are the Teacher as A Person, Classroom Management and Organization, Planning and Organizing for Instruction, Implementing Instruction, and Monitoring Student Progress and Potential. Using a valid and reliable instrument, the four teacher competencies, especially pedagogical and professional competencies in the real classroom can be assessed.

1

Objectives

The purposes of this research are: 1. to identify the components of Teacher Quality Index (TQI). 2. to apply the TQI instrument for recognizing the qualities of effective teachers.

Methodology The Development of the questionnaire

The first step in this study was developing a questionnaire from the Teacher Quality Index which was developed by Stronge and Hindman from Association for Supervision and Curriculum Development, Virginia, USA. The qualities of effective teachers were investigated through the series of statements in each category: teacher as a person, planning for instruction, classroom management and organization, implementing instruction, as well as monitoring student progress and potential.

Administration of survey Senior high school students were surveyed across Indonesia, including

Jakarta, Surabaya, Malang and Palembang. A total of 534 students from 11 public senior high schools and 9 private senior high schools participated in the survey. The questionnaires were administered in May 2008. A total of 528 questionnaires were completed.

Teacher Interview Teacher interviews were conducted in two cities, Jakarta and Denpasar.

Eight teachers participated in the interview. They were 2 permanent (civil servant/PNS) teachers with more than 20 years of teaching experience, 2 permanent teachers with 10-15 years of teaching experience, 2 non permanent teachers with more than 2 years of teaching experience, and 2 non permanent teacher with less than 2 years of teaching experience. Teachers were interviewed within five domains of TQI to recognize their qualities of effectiveness.

2

THEORITICAL FRAMEWORK Studies on Teacher Effectiveness

Research reveals that teacher quality matters a great deal in terms of student learning. It influences differences in student performance and close gap between traditionally poor students and their better-off peers. Teacher quality has been defined and measured in many ways. In addition to teacher contributions to student achievement, teacher quality may be evidenced by teachers who possess the following characteristics: qualification and experience to grade level and subject matter; high expectations for students, particularly those at risk for poor outcomes; creation of classroom environment that encourages all students to participate in worthwhile learning activities; desire to help students achieve at high levels; ability to motivate at risk students to come to school and participate in class; excellent skills in mentoring new teachers; and willingness to diligently work with students with special needs (Goe, 2007).

Researchers who study teacher quality also put a great interest in investigating teacher effectiveness. A number of policy makers and researchers have proposed that effectiveness, as determined by teachers’ contribution to student learning, should be an important component of assessing teacher quality. Specifically, differential teacher effectiveness is a strong determinant of differences in student learning, prevails over the effect of differences in class size and class heterogeneity (Darling-Hammond, 2000 in Anderson, L.W .2004, p.20). Students who are assigned to one ineffective teacher after another have significantly lower achievement and learning than those who are assigned to a sequence of several highly effective teachers (Sanders and Rivers, 1996)

Research offered a lot of definition of effective teacher. Collins (1990 in Markley, p.2) set up five criteria for an effective teacher: (a) is committed to students and learning, (b) knows the subject matter, (c) is responsible for managing students, (d) can think systematically about their own teaching practice, (e) is a member of the learning community. Anderson, L.W (2004) defines effective teachers as those who achieve the goals which they set for themselves or which they have set for them by others (e.g. ministries of education, legislators and other government officials, and school administrators. As a consequence of this definition is that effective teachers must possess the knowledge and skills needed to attain the goals, and must be able to use that knowledge and those skills appropriately if these goals are to be achieved.

Teacher Quality Index Although many studies suggest that instructional and management processes are

keys to effectiveness, many interview and survey responses about effective teaching accentuate the teacher’s affective characteristics, or social and emotional behaviors, more than pedagogical practice. The characteristics of the teacher as a person are difficult to measure, but have influence on student achievement.

Noddings (2005, in Stronge 2007) explained that a teacher’s happiness can affect the classroom climate and therefore affects students. Furthermore, Stronge (2007) listed teachers’ affective qualities or characteristics that relate to effectiveness, which are: caring, fairness and respect, interactions with students, enthusiasm, motivation, dedication to teaching, and reflective teaching.

3

Stronge (2007, p.39), states that “an effective teacher plans and prepares for the organization of the classroom with the same care and precision used to design a high-quality lesson”. Components of the organizational plan of a classroom include room arrangement, discipline, creating routines, and a plan to teach students how their learning environment is organized (Doyle (1986) established the essential prerequisite for effective classroom management is instructional strength in the implementation of: (a) time management procedures, such as appropriate pacing and well-planned transitions, (b) teaching functions such as attention to prerequisites, guided practice and systematic reviews, (c) effective academic feedback and monitoring skills.

While doing their jobs, teachers also prepare and plan objectives and activities on an hourly, daily, and weekly basis. The effective teacher put instruction in priority, a process that is accomplished partially through allocation of time. Research tells us that one key trait of effective teachers is their use of instructional strategies that work. Quality of instruction is about 15 to 20 times more influential in terms of learning growth than family background, income, race, gender, or other commonly recognized predictors (Leverett, 2006).

Effective teachers routinely use different strategies to ensure that concepts are well understood by students and combine various instructional techniques that involve individual, small group, and whole class instruction. This way, they are able to monitor and pace instruction based on the individual needs of students (Stronge 2007, p.68).

Progress monitoring is when teachers assess students’ academic performance on a regular basis for two purposes: to determine whether children are profiting appropriately from the typical instructional program and to build more effective programs for the children who benefit inadequately from typical instruction. When teachers use systematic progress monitoring to track their students progress in reading, mathematics, or spelling, they are better able to identify students in need of additional or different forms of instruction, they design stronger instructional programs, and their students achieve better (Fuchs, L.S & Fuchs, D, 2007). Effective teacher facilitate students’ understanding and knows how to assess their performance. They use a variety of formal and informal measures to monitor and assess their pupils ‘mastery of a concept or skill (Stronge 2007, p.123).

Teacher Quality Index (TQI) that is used in this research is designed to identify the key indicators of effectiveness in a teacher’s practice. There are five latent variables/components of TQI: the teacher as a person, classroom management and organization, planning and organizing for instruction, implementing instruction and monitoring student progress and potential. Each component has sub components and indicators.

4

RESULTS AND ANALYSIS

Validity and Reliability Tests To examine how good variables or indicators in representing or measuring the

latent variables, validity and reliability test should be conducted. Validity test is conducted by examining t value and λ value (influence significance or loading). For sample larger than 150, a variable has significant influence on the dependent variable with significant level 5%, if the calculated t value is higher than |1.96| and λ is higher than 0.30. Reliability test of a variable can be conducted by examining the squared multiple correlation (R2) of the variable.

TQI has 5 latent variables (components) with a total of 74 indicators: The teacher as person, Classroom management and organization, Planning and organizing for instruction, Implementing instruction and Monitoring student progress. The teacher as a person has 7 latent variables (sub components): caring, showing fairness and respect, interactions with students, enthusiasm, motivation, dedication to teaching, and reflective practice. Classroom management has 3 latent variables (sub components): classroom management, organization, and discipline of students. Planning and organizing for instruction has 2 latent variables (sub components): time allocation and instruction plans. Implementing instruction has 2 latent variables (sub components): instructional strategies, complexity and strategies. Monitoring student progress and potential has 3 latent indicators (sub components): homework, monitoring student progress, and responding to students’ needs and abilities.

To examine whether all indicators of TQI are representing the components and sub components, validity and reliability test were conducted using Confirmatory Factorial Analysis (CFA) with Lisrel.

Validity and Reliability of TQI Components and Their Indicators All latent variables (components) of TQI are statistically proven valid and have

significant influence, represented by λ > 0.30 and t > 1.96 and. All components are also reliable. Of all the five components, planning and organizing for instruction is the most reliable component, R2 = 0.88.

Each latent variable/component of TQI has indicators. The teacher as a person has 31 indicators. All indicators have significant influence and valid, represented by t > 1.96 and λ > 0.30, and also reliable, with the composite reliability = 0.6. The most reliable indicator is V9 (My teacher creates situations for all students to succeed), with R2 = 0.45.

Classroom management and organization has 12 reliable indicators, with composite reliability = 0.87. V43 (My teacher uses appropriate discipline measure) is the most reliable one, R2 = 0.45.

All indicators of Planning and organizing for instruction are reliable, with composite reliability = 0.86. The most reliable indicator is V50 (My teacher carefully links learning objectives and activities) is the most reliable one, R2 = 0.48.

5

Implementing instruction has 10 indicators which are proven reliable, with composite reliability = 0.6. Of all 10 indicators, V63 (My teacher varies question type to maintain interest) is the most reliable one, R2 = 0.54.

All indicators of Monitoring student progress and potential are reliable, with composite reliability = 0.88. Of all 10 indicators, V73 (My teacher assesses student progress) is the most reliable one, R2 = 0.52.

Figure 1: TQI and All Components

6

Validity and Reliability of TQI Sub Components and Their Indicators Each component of TQI has sub components. To examine how good indicators in

representing the sub components of TQI, the validity and reliability test were conducted. All sub components of the teacher as a person are statistically proven

valid and have significant influence to the component, represented by λ > 0.30 and t > 1.96. All sub components are also reliable. Interaction with students is the most reliable one, R2 = 0.83.

Caring as a sub component of The teacher as a person has 5 indicators. All indicators are valid, has significant influence on the component, and reliable, with the composite reliability = 0.77. The most reliable indicator is V3 (My teacher exhibits active listening), is the most reliable one, R2 = 0.83.

Showing fairness and respect has 5 indicators. All indicators are valid, has significant influence on the component, and reliable, with composite reliability = 0.78. V9 (My teacher creates situations for all students to succeed) is the most reliable one, R2 = 0.55.

All indicators of Interactions with students are valid, has significant influence on the component, and reliable with composite reliability = 0.77. V14 (My teacher values what students say) is the most reliable one, R2 = 0.52.

There are 2 indicators of Enthusiasm. Both are valid, has significant influence on the component, and reliable, with composite reliability = 0.61. V16 (My teacher takes pleasure in teaching) is more reliable than V17 (My teacher demonstrates involvement in learning activities outside school (R2 = 0.48).

Motivation has 4 indicators which are valid, has significant influence on the component, and reliable with composite reliability = 0.72. V20 (My teacher maintains students’ high quality work) is the most reliable one, R2 = 0.46.

Dedication to teaching has 6 indicators. All indicators are valid, has significant influence, and reliable, with composite reliability = 0.76. V25 (My teacher accepts responsibility for students outcomes) is the most reliable one, R2 = 0.42.

Reflective practice has 4 indicators which are valid, has significant influence, and reliable with composite reliability = 0.75. V31 (My teacher demonstrates high efficacy) is the most reliable one, R2 = 0.48.

From the validity and reliability test results of all indicators of the sub components of Teacher as a person, it is shown that all indicators are valid and reliable measure for the sub components. However, from the composite reliability value, it is seen that Enthusiasm is the least reliable one, but still reliable enough as a measure (composite reliability > 0.60).

All sub components of classroom management and organization are

statistically proven valid and have significant influence to the component, represented by λ > 0.30 and t > 1.96. All sub components are also reliable. Classroom management is the most reliable one, R2 = 0.96.

Classroom management as a sub component has 6 indicators. All indicators are proven valid, have significant influence, and reliable, with composite reliability = 0.75. V37 (My teacher encourages students’ attention) is the most reliable one, R2 = 0.44.

7

Organization has 3 indicators. All indicators are proven valid, have significance influence, and reliable with composite reliability = 0.70. V39 (My teacher systematically organizes the lesson) is the most reliable one, R2 = 0.54.

All indicators of Discipline of students are proven valid, have significance influence, and reliable with composite reliability = 0.75. V43 (My teacher uses appropriate discipline measure) is the most reliable one, R2 = 0.54.

From the validity and reliability test results of all indicators of the sub components of Classroom management and organization, it is shown that all indicators are valid and reliable measure for the sub components.

All sub components of Planning and organizing for instruction are

statistically proven valid and have significant influence to the component, represented by λ > 0.30 and t > 1.96. All sub components are also reliable.

Time allocation as a sub component has 6 indicators. All indicators are proven valid, have significant influence, and reliable, with composite reliability = 0.48. V46 (My teacher consistently applies the schedule) is the most reliable one, R2 = 0.48.

All indicators of instruction plans are proven valid, have significant influence, and reliable, with composite reliability = 0.80. V51 (My teacher organizes content for effective presentation) is the most reliable one, R2 = 0.51.

From the validity and reliability test results of all indicators of the sub components of Planning and organizing for instruction, it is shown that all indicators are valid and reliable measure for sub components.

All sub components of Implementing instruction are statistically proven

valid and have significant influence to the component, represented by λ > 0.30 and t > 1.96. All sub components are also reliable. Instructional strategies is the most reliable one, R2 = 0.87.

Instructional strategies as sub component has 4 indicators. All indicators are proven valid, have significant influence, and reliable, with composite reliability = 0.76. V56 (My teacher stresses meaningful conceptualization) is the most reliable one, R2 = 0.55.

Complexity has 3 indicators. All indicators are proven valid, have significant influence, and reliable, with composite reliability = 0.64. V61 (My teacher emphasizes higher-order thinking skills) is the most reliable one, R2 = 0.47.

Questioning has 3 indicators. All indicators are proven valid, have significant influence, and reliable, with composite reliability = 0.77. V63 (My teacher varies questions type to maintain students’ interest) is the most reliable one, R2 = 0.59.

From the validity and reliability test results of all indicators of the sub components of Implementing instruction, it is shown that all indicators are valid and reliable measure for sub components. However, Complexity is the least reliable, with composite reliability 0.64, but still reliable enough as a measure as the composite reliability > 0.60).

8

All sub components of Monitoring student progress and potential are statistically proven valid and have significant influence to the component, represented by λ > 0.30 and t > 1.96. All sub components are also reliable. Monitoring student progress is the most reliable one, R2 = 0.96.

Homework as a sub component has 3 indicators. All indicators are statistically proven valid, have significant influence to sub component, and reliable, with composite reliability 0.72. V65 (My teacher relates homework to student capacity) is the most reliable one, R2 = 0.56.

Monitoring student progress has 4 indicators. All indicators are statistically proven valid, have significant influence to sub component, and reliable, with composite reliability 0.79. V69 (My teacher anticipates students’ misconception) is the most reliable one, R2 = 0.52).

Responding student needs and abilities has 3 indicators. All indicators are statistically proven valid, have significant influence to sub component, and reliable, with composite reliability 0.76. V73 (My teacher monitors and assesses student progress) is the most reliable one, R2 = 0.64.

From the validity and reliability test results of all indicators of the sub components of Monitoring student progress and potential, it is shown that all indicators are significant and reliable measure for the sub components.

Teacher Interview After statistically proven that the instrument is valid and reliable, teacher interviews

were conducted. By using Teacher Quality Index (TQI), the interviewer or assessor would be able to gain information about a teacher that likely to be indicative of teacher effectiveness. In the interviews, teachers were asked about five domains of TQI that linked to quality indicators and assessed with a rubric that clarifies essential evidence of each indicator to ensure consistent response assessment.

1. Teacher as a person

Core Quality Prompt Novice Teachers < 3 years of teaching

experience

Experienced Teachers > 10 years of teaching

experience Enthusiasm and Motivation

What do you find most rewarding about the teaching profession?

“Teacher is a noble job and by being a teacher we can be a role model for our students.” (Cecep, part time Citizenship teacher with 1 year of teaching experience, Jakarta). “We always find something new as we need to be creative as a teacher.” (Era, part time English teacher with 2 years of teaching experience, Denpasar).

“What I like most being a teacher is when I see my students enthusiastic and successful. That is my pride and happiness.” (Bambang, full time Mathematics teacher with 25 years of teaching experience, Jakarta)

Interactions with students

How do you establish and maintain rapport with your students?

“I regard my students as my friends without forgetting ethics and school rules.” (Era, part time English teacher with 2 years of teaching experience,

“Teachers are like students’ parents at school. But, I show interest and give attention to my students not only at school but also outside school. Once my

9

Denpasar). student was lazy coming to school. I came to his home to wake him up. He then, quit his laziness and became more motivated.” (Bambang, full time Mathematics teacher with 25 years of teaching experience, Jakarta)

Reflective practice

If your student did not meet your expectations, despite planning and preparation, what did you do to help him achieve/meet your expectations?

“I teach several classes of grade X. When I found my strategies can’t help students meet the objectives, I altered strategies in the other classes. I also gave remedy to those who still didn’t meet the objective. (Era, part time English teacher with 2 years of teaching experience, Denpasar).

“I tried to re-teach in a slower pace and gave them remedy.” (Kosep, part time Sociology teacher with 15 years of teaching experience, and Bambang, full time Mathematics teacher with 25 years of teaching experience, Jakarta).

Students often start with personal qualities of their teacher before talking about subject matter knowledge and what the teacher did in the classroom when they are asked to describe the teacher who made the biggest impact on them. Stronge and Hindman (2006) stated that effective teachers feel optimistic about their profession, establish positive relationships with their students, and engage in reflective practice.

From the interview, it can be seen that teachers with more than 10 years of teaching experience are more likely to have the quality of Teacher as a person. However, there is no teacher involving other teachers in doing reflection to improve their teaching quality.

2. Planning for instruction Core Quality Prompt Novice Teachers

< 3 years of teaching experience

Experienced Teachers > 10 years of teaching

experience Planning

Think about a lesson you recently taught and describe how you planned for it.

“Before school year commences, we have teacher workshop to prepare lesson plans for 1 semester. But, I modified the lesson plans before entering the classroom when I found new ideas/resources which I thought better and more suitable than the old ones.” (Era, part time English teacher with 2 years of teaching experience, Denpasar). “Before teaching, I studied again the lesson plan that I already prepared before the school year begins, especially the learning objectives, competence standard and consulted with the national standard”

“I prepared lesson plans for every six months in advance, before the school year commences. I consulted the learning objectives with the national standard.” (Nyoman, full time Indonesian language teacher with 20 years of teaching experience, Denpasar).

10

(Cecep, part time citizenship teacher with 1 year of teaching experience, Jakarta).

Planning for technology use

How do you get teaching resources? Do you use technology during your instruction?

“Besides textbook, I use internet. I also provide students with worksheets.” (Sri, part time Biology teacher with 1.5 years of teaching experience). “Other than books, the resources are internet, newspaper, film.” (Cecep, part time citizenship teacher with 1 year of teaching experience, Jakarta).

“Besides textbook, I use media such as film, and power point slides. I also use internet as one of the resources.” (Kosep, part time Sociology teacher with 15 years of experience).

Before entering the classroom, an effective teacher makes the planning and preparation. The planning includes both long and short-term planning, which typically consist of reviewing state standards and educational materials such as textbooks, additional resources and Websites.

From the interview, we can see that there is no difference among novice and experienced teachers in planning the lessons. However, novice teachers seem to be more creative in modifying the lessons and finding resources.

3. Classroom management and organization Core Quality Prompt Novice Teachers

< 3 years of teaching experience

Experienced Teachers > 10 years of teaching

experience Classroom management

What do you do with your students during the first days of school year to establish a positive classroom environment.

“I establish classroom rules with my students. Besides that, we also have school rules and the rules are communicated to parents as well.” (Cecep, part time citizenship teacher with 1 year of teaching experience, Jakarta).

“In the first day of school year, I inform my classroom rules to the students. The rules should be agreed by all of them.” (Achmad, full time Physics teacher with 15 years of teaching experience, Jakarta).

Share with me how you handle a particular student’s behavior and what you did to address it.

“So far, I never had such problems.” (Sri, part time Biology teacher with 1.5 years of teaching experience, Denpasar).

“Once, I found students who I didn’t teach smoking. Although I didn’t teach him I still handled this problem. But, I didn’t push him or instantly give punishment. Instead, I talked to him first. Asked him about his problems. Then, I gave him advice and had him his promises. If he broke the rules, I told him that there would be consequences.” (Bambang, full time Mathematics teacher with 25 years of teaching experience, Jakarta).

11

“I talked to the student and communicate it with school counselor and parents.” (Kosep, part time Sociology teacher with 15 years of experience, Jakarta).

All teachers establish classroom rules which should be agreed by students. When having difficulty with a particular student’s behavior, teachers with more than 10 years of experience are more likely to handle it personally with the students.

4. Implementing instruction Core Quality Prompt Novice Teachers

< 3 years of teaching experience

Experienced Teachers > 10 years of teaching

experience Student engagement

Describe how you engage students in their learning.

“By delivering lessons that are related to daily life. My students also observe and do experiments, so that they don’t only learn theories.” (Sri, part time Biology teacher with 1.5 years of teaching experience, Denpasar). “By contextual learning, watching movies, observing reality.” (Cecep, part time Citizenship teacher with 1 year of teaching experience, Jakarta).

“I teach Sociology. To make the lessons more interesting, I often demonstrate to students. For example, while studying about Javanese culture, I danced in front of the class, and at the end of the lesson, students were asked to perform Javanese dance.” (Kosep, part time Sociology teacher with 15 years of teaching experience, Jakarta).

Instructional strategies

How do you accommodate students’ learning needs in the classroom?

“Besides modifying strategies, I often asked students with higher ability to assist their lower peers. I mixed students in groups.” (Era, part time English teacher with 2 years of teaching experience, Denpasar).

“I asked my students in computer club which they have advanced skills to assist, it’s like peer coaching.” (Udayana, full time ICT teacher with 15 years of teaching, Denpasar).

The area of implementing instructions speaks to the nuts and bolts of what occurs in the classroom. Effective teachers provide instruction in which actively engage students hands-on and minds-on. (Stronge and Hindman, 2006)

From the interview, it can be seen that regardless of year of teaching and status, in Implementing instruction novice teachers are more likely to use a variety of teaching strategies compared to experienced ones.

12

5. Monitoring student progress and potential Core Quality Prompt Novice Teachers

< 3 years of teaching experience

Experienced Teachers > 10 years of teaching

experience Responding to student needs and abilities

What kinds of assessment do you practice or give to your students?

“Besides written tests, I also assess students through projects. They have to make analysis and report of the projects.” (Cecep, part time Citizenship teacher with 1 year of teaching experience, Jakarta). “In English there are four skills. I assess my students through those skills. For example, in writing, I assessed how good they are in writing by asking them writing letter.” (Era, part time English teacher with 2 years of teaching experience, Denpasar).

“By giving them tests.” (Achmad, full time Physics teacher with 15 years of teaching experience, Jakarta). “I give them written test once a month.” (Bambang, full time Mathematics teacher with 25 years of teaching experience, Jakarta).

Monitoring student progress

How do you let students in your class know how well they are doing in class?

“By grading their work. If they haven’t met the minimum standard, I gave them remedy.” (Sri, part time Biology teacher with 1.5 years of teaching experience, Denpasar).

“By grading their work and giving them remedy if they don’t meet the minimum standard.” (Ahmad, full time Physics teacher with 15 years of teaching experience, Jakarta). “I involve my students to give feedback to their peer’s work. Because, sometimes what are good to me aren’t good to students. So, I also use peer assessment.” (Udayana, ICT teachers with 15 years of teaching experience, Denpasar).

Effective teachers monitor student learning through a variety of informal and formal assessments and offer timely verbal and non verbal feedback. (Cotton, 2000 in Stronge and Hindman, 2006).

It’s interesting to find that novice teachers are more likely to monitor student learning through different kinds of assessment. They don’t only give written tests to students but also asses students through project and performance.

Most teachers give feedback of student progress by grading. Only one teacher use peer assessment or involve students in providing feedback to students.

13

CONCLUSION AND RECOMMENDATION

Conclusion As mentioned in chapter one, the purposes of the research are (1) to identify the

components of Teacher Quality Index (TQI), (2) to apply the TQI instrument for recognizing the qualities of effective teachers.

Based on the purposes of the research and the results, several conclusions are given as follow:

1. There are five components of TQI that can be used to measure the quality of effective teachers in Indonesia. The five components are: The Teacher as a Person, Classroom Management and Organization, Planning and Organizing for Instruction, Implementing Instruction, and Monitoring Student Progress and Potential.

2. The TQI component of Teacher as a Person has 7 sub components (caring, shows fairness and respect, interactions with students, enthusiasm, motivation, and dedication to teaching). The TQI component of Classroom Management and Organization has 3 sub components (classroom management, organization, and discipline of students). The TQI component of Planning and Organizing for Instruction has 2 sub components (time allocation and instruction plans). The TQI component of Implementing Instruction has 3 sub components (instructional strategies, complexity and questioning). The TQI component of Monitoring Student Progress and Potential has 3 sub components (homework, monitoring student progress and responding to student needs and abilities). All the sub components significantly measured the components of TQI.

3. There are 74 variables significantly measured five components of TQI. (See the appendix): 31 variables of the teacher as a person, 12 variables of classroom management and organization, 11 variables of planning and organizing for instruction, 10 variables of implementing instruction and 10 variables of monitoring student progress and potential. The highly significant variables are: V31 (My teacher creates situations for all students to succeed), V43 (My teacher uses appropriate discipline measure), V50 (My teacher carefully links objectives and activities), V63 (My teacher varies question types to maintain interest), V73 (My teacher assesses student progress).

4. Teacher as a person

Teachers with more than 10 years of teaching experience are more likely to have the quality of Teacher as a person. However, there is no teacher involving other teachers in doing reflection to improve their teaching quality. Planning for instruction There is no difference among novice and experienced teachers in planning the lessons. However, novice teachers seem to be more creative in modifying the lessons and finding resources.

14

Classroom management and organization All teachers establish classroom rules which should be agreed by students. When having difficulty with a particular student’s behavior, teachers with more than 10 years of experience are more likely to handle it personally with the students. Implementing instruction From the interview, it can be seen that regardless of year of teaching and status, in Implementing instruction novice teachers are more likely to use a variety of teaching strategies compared to experienced ones. Monitoring student progress and potential It’s interesting to find that novice teachers are more likely to monitor student learning through different kinds of assessment. They don’t only give written tests to students but also asses students through project and performance.

Most teachers give feedback of student progress by grading. Only one teacher use peer assessment or involve students in providing feedback to students.

Recommendation

This research recommends the application of Teacher Quality Index (TQI) instrument for measuring the qualities of effective teacher as TQI considered valid and reliable for examining the qualities of effective teachers in Indonesia.

By using TQI, we can see in what qualities experienced and novice teachers differ. For example, teaching experience matters in shaping the quality of Teacher as a person: teachers with more than 10 years of teaching experience are more likely to have the quality of Teacher as a person. On the other side, through the interview we also gain the evidence that teaching experience doesn’t really matter in constructing the quality of Planning for instruction, Implementing Instruction and Monitoring student progress and potential. Novice teachers were even found more likely to have better qualities in these domains.

Interview results using TQI can also indicate in which aspects teachers need to be improved. For example, reflective practice as a way to improve the qualities of teacher should be implemented among teachers in Indonesia and the use of a variety of assessments and feedbacks for monitoring student progress and potential.

15

REFERENCES

Anderson, L.W. (2004). Increasing Teacher Effectiveness. Paris. UNESCO: International Institute for Educational Planning.

Bagozzi and Yi (1988). Theoritical Concepts, Measurement and Meaning. In Ghozali, I and Fuad (2005). Structural Equation Modeling: Teori, Konsep dan Aplikasi dengan Program LISREL 8.54. (p.321). Semarang: Badan Penerbit Universitas Diponegoro.

Cotton, K (2000). Research You Can Use to Improve Results. In Stronge, J & Hindman, J (2006). The Teacher Quality Index. Virginia, USA: Association for Supervision and Curriculum Development.

Collins, A. (1990). Transforming The Assessment of Teachers. In Markley. Defining The Effective Teacher: Current Arguments in Education. New Hampshire: White Mountains Regional School District.

Cruickshank, D.R., & Haefele, D. (2001, February). Good teachers, plural.Educational Leadership in Stronge & Hindman (2006). The Teacher Quality Index. Virginia, USA: Association for Supervision and Curriculum Development.

Darling-Hammond, L. (2000). Teacher Quality and Student Achievement. In Anderson, L.W. (2004). Increasing Teacher Effectiveness. (p.20). Paris: UNESCO, International Institute for Educational Planning.

Direktorat Pembinaan SMA, Direktorat Pembinaan SMA and Mandikdasmen (2007). Standar Kualifikasi Akademik dan Kompetensi Guru. [on-line] Available at:

http://www.dikmenum.go.id/dataapp/kurikulum Dirjen PMPTK (2007). Peningkatan Kualifikasi Guru. Dirjen PMPTK Depdiknas. [on-

line] Available at: http://www.pmptk.net/file/pedoman/1.%20Kualifikasi%20Guru.doc.

Doyle, W. (1986). Classroom Organization and Management. In The Research Literature: Classroom Management. [on-line] Available at: http://www.usu.edu/teachall/text/effective/research/class.pdf)

Fuchs, L.S and Fuchs, D. (2007). What is Scientifically-Based Research on Progress Monitoring? National Center on Student Progress Monitoring. [on-line] Available at: http://www.nichcy.org/toolkit/pdf/ScientificallyBasedResearch.pdf

Goe, Laura (2007). The Link between Teacher Quality and Student Outcomes: A Research Synthesis. Washington DC: National Comprehensive Center for Teacher Quality.

Leverett (2006). Instruction At the Core. Strategies for School System Leaders on District Level-Change. Vol. 12/Number 1/December 2006.

Lumbantobing, R (2006). A Comparative Study on Science Curriculum between Indonesia and Japan: Focus on the Process Skills in Elementary School. Hiroshima University.

Mandikdasmen (2008). Hasil UN Rata-rata Nasional Meningkat. [on-line] Available at: http://mandikdasmen.aptisi3.org/index.php?option=com_content&task=view&id=119&Itemid=11

16

Noddings, N (2005). What Does It Mean to Educate The Whole Child? In Stronge (2007). The Qualities of Effective Teachers. Virginia, USA: Association for Supervision and Curriculum Development.

Sanders, W.L.; Rivers, J.C. (1996). Cumulative and Residual Effects of Teachers on Future Student Academic Achievement. Knoxville: University of Tennessee Value-Added Research on Assessment Center.

Stronge, James (2007). The Qualities of Effective Teachers. Virginia, USA: Association for Supervision and Curriculum Development

Stronge, J & Hindman, J (2006). The Teacher Quality Index. Virginia, USA: Association for Supervision and Curriculum Development

17

Appendix: Teacher Quality Index

V My teacher: Strongly

agree Agree Neutral Disagree Strongly

disagree

Teacher as a person

Caring (T1)

1 care about students.

2 shows concern for students' emotional and physical well-being.

3 exhibits active listening.

4 displays interest in and concern about the students' lives outside school.

5 creates a supportive and warm classroom climate.

Show fairness and respect (T2)

6 responds to misbehavior on an individual level.

7 prevents situations in which a student loses peer respect.

8 treats students equally.

9 creates situations for all students to succeed.

10 shows respect to all students.

Interactions with students (T3)

11 maintains professional role while being friendly.

12 gives students responsibility.

13 knows students' interests both in and out of school.

14 values what students say.

15 values what students say.

Enthusiasm (T4)

16 takes pleasure in teaching.

17 demonstrates involvement in learning activities outside school.

Motivation (T5)

18 maintains high motivation.

19 returns student work in a timely manner.

20 put forward the quality of student performance.

21 provides students with meaningful feedback.

Dedication to teaching (T6)

22 possesses a positive attitude about life and teaching.

18

23 spends time outside of school to prepare.

24 participates in collegial activities.

25 accepts responsibility for students outcomes.

26 seeks professional development.

27 finds, implements, and shares new instructional strategies.

Reflective practice (T7)

28 knows areas of personal strengths and weaknesses.

29 uses reflection to improve teaching.

30 set high expectations for personal classroom performance.

31 demonstrates high efficacy.

Classroom Management and Organization

Classroom management (C1)

32 uses consistent and proactive discipline.

33 establishes routines for all daily tasks and needs.

34 balances variety and challenge in student activities.

35 multitasks.

36 anticipates potential problems.

37 focus students' attention.

Organization (C2)

38 handles routine tasks efficiently.

39 organizes lesson systematically.

40 organizes classroom space efficiently.

Discipline of students (C3)

41 responds to inappropriate behavior promptly.

42 reinforces and reiterates expectations for positive behavior.

43 uses appropriate disciplinary measures.

Planning and Organizing for Instruction

Time allocation (P1)

44 focuses classroom time on teaching and learning.

45 links instruction to students' real-life situations.

46 follows a consistent schedule and maintains procedures and routines.

47 handles administrative tasks quickly and efficiently.

48 prepare materials in advance.

49 limits disruption and interruptions.

19

Instruction plans (P2)

50 carefully links learning objectives and activities.

51 organizes content for effective presentation.

52 explores student understanding by asking questions.

53 considers student attention span and learning styles when designing lessons.

54 develops activities as appropriate for the content and the students.

Implementing Instruction

Instructional Strategies (I1)

55 employs different techniques and instructional strategies, such as hands-on learning.

56 stresses meaningful conceptualization.

57 suits instruction to students' achievement levels and needs.

58 uses a variety of grouping strategies.

Complexity (I2)

59 is concerned with having students learn and demonstrate understanding of meaning rather than memorization.

60 holds reading as a priority.

61 emphasizes higher-order thinking skills.

Questioning (I3)

62 asks questions that reflect type of content and goals of the lesson.

63 varies question type to maintain interest.

64 gives students chance to ask.

Monitoring Student Progress and Potential

Homework (M1)

65 relate homework with student capacity.

66 relate homework to the content under study.

67 grades, comments on, and discusses homework in class.

Monitoring Student Progress

20

(M2)

68 targets questions to lesson objectives.

69 thinks through likely misconceptions that may occur during instruction and monitors students for these misconceptions.

70 gives clear, specific, and timely feedback.

71 reteaches students who did not achieve mastery and offers tutoring to students who seek additional help.

Responding to Student Needs and Abilities

(M3)

72 monitors and assesses student progress.

73 uses data to make instructional decisions.

74 knows and understands students as individuals in terms of abillity, achievement, learning styles, and needs.

21

22

BIODATA Personal Detail Name : Kartika Yulianti Sex : Female Place/date of birth : Jakarta, July 26, 1976 Office Address : Sampoerna Strategic Square, North Tower, Fl. 18,

Jl Jend Sudirman Kav. 45, Jakarta 12930 Indonesia Phone number : 0812-8218126/+62-21-5772275 ext. 7524 E-mail Address : tika_hana@yahoo.com; kartika.yulianti@sampoernafoundation.org Work Experience At present : Program developer and facilitator at Sampoerna Foundation Teacher

Institute, Jakarta, Indonesia Education

- Bachelor degree: Chemistry Education, Faculty of Mathematics and Natural Sciences, State University of Jakarta, Indonesia

- Master of Science in Education, Rijksuniversiteit Groningen, majoring in Educational Effectiveness and School Improvement

Achievement

- STUNED scholarship from the Netherlands Education Centre, Jakarta, Indonesia in 2005

Research and Public Paper

- Undergraduate thesis: Xylitol and Its’ Usage to Avoid Teeth Carries - Master’s thesis: Learning Environment and Student Performance, an International

Comparative Study of Indonesia, Thailand, Korea, and Japan, A Secondary Analysis of PISA Results 2003

- Developing Instrument for Measuring The Qualities of Effective Teachers - Some paper presented at one or two day workshops in Indonesia

Others Reader/assessor of StuNed scholarship candidates at Netherlands Education Support Office (NESO) Indonesia in 2008