UNIVERSITI PUTRA MALAYSIA

DEVELOPMENT AND VALIDATION OF AN INSTRUMENT TO MEASURE SCIENCE TEACHERS’ INSTRUCTIONAL PREPAREDNESS IN STEM

IMPLEMENTATION

NUR FARHANA BINTI RAMLI

FPP 2019 34

© COPYRIG

HT UPM

DEVELOPMENT AND VALIDATION OF AN INSTRUMENT TO MEASURE SCIENCE TEACHERS’ INSTRUCTIONAL PREPAREDNESS IN STEM

IMPLEMENTATION

By

NUR FARHANA BINTI RAMLI

Thesis Submitted to the School of Graduate Studies, Universiti Putra Malaysia, in Fulfilment of the Requirements for the Degree of

Doctor of Philosophy

July 2019

© COPYRIG

HT UPM

i

COPYRIGHT

All material contained within the thesis, including without limitation text, logos, icons, photographs, and all other artwork, is copyright material of Universiti Putra Malaysia unless otherwise stated. Use may be made of any material contained within the thesis for non-commercial purposes from the copyright holder. Commercial use of material may only be made with the express, prior, written permission of Universiti Putra Malaysia. Copyright © Universiti Putra Malaysia

© COPYRIG

HT UPM

ii

DEDICATION

To mum, Salwa Md Yasan and my siblings, Fariza, Faiz, Asya, Farahin, Faizal and Farzana with love and gratitude for your patience and encouragement. To my late father, Ramli Khamis, who always believe in me, this is for you.

Al-Fatihah

© COPYRIG

HT UPM

i

Abstract of thesis presented to the Senate of Universiti Putra Malaysia in fulfilment of the requirement for the degree of Doctor of Philosophy

DEVELOPMENT AND VALIDATION OF AN INSTRUMENT TO MEASURE SCIENCE TEACHERS’ INSTRUCTIONAL PREPAREDNESS IN STEM

IMPLEMENTATION

By

NUR FARHANA RAMLI

July 2019

Chairman : Othman Talib, PhD Faculty : Educational Studies

STEM has gained increased attention in recent years as the government is trying to prepare students for the demand of future workforce. It is also an effort to produce literate citizens who can solve problems in the context of daily life, society and environment using the application of knowledge, skills and STEM values. The failure of the previous policy had found that instructional teacher is one of the aspects that causes students to be less interested in STEM. As the new STEM curriculum has been introduced in Malaysia's education system recently, it is need to build an instrument that measure the level of teachers’ instructional preparedness in order to assist the success of National Education aspirations, especially in STEM education. Thus, the STEM Instructional Preparedness Instrument (STEMTIP) was developed to test the teacher's instructional preparedness in STEM implementation.

This study was divided into two phases which are development phase and validation phase. In the development phase, the conceptualisation and items were generate using both inductive and deductive approaches. With the help of four teachers that were involved in focus group discussion and also information from literature review, 51 items we generated.

The items were then were tested during the second phase of the study by conducting reliability tests, face validation, content validation, construct validation and criterion validation. In face validation, five science teachers were selected to verify the language and the clarity of the items. As for content validation, 15 experts have been involved to determine the items content validation. Through content validation testing using the Lawshe Model, 47 items

© COPYRIG

HT UPM

ii

have gained expert consensus, however, the remaining four items were maintained during the next test with some refinement. Field studies were then conducted on 265 science teachers in Malaysia using randomised multistage cluster sampling. STEMTIP was identified to have excellent reliability and replicability through the findings of Cronbach alpha (.98), person reliability (.96) and item reliability (.98). Construct validation was determined using Rasch Model. The analysis found that the instrument has unidimensionality, local independence, polarity, excellent separation indices and effective scale. However, 10 misfit items were removed from the instrument as they showed disturbance feature in measurement. Misfit items indicate a lack of consistency in interpreting the underlying measure that requires the need for further study in the future research. Also, two items were identified as having potential biased based on the school location but were kept remained as it will be useful for stakeholder future undertaking. Criterion tests found that STEMTIP was able to contribute 40.6% to teachers' self-efficacy in STEM implementation. Overall, based on all tests, this instrument has been proven to be a good instrument. The researchers are confident that the STEMTIP instrument has passed psychometric standards and can be used by stakeholders to measure instructional preparedness of STEM teachers.

© COPYRIG

HT UPM

iii

Abstrak tesis yang dikemukakan kepada Senat Universiti Putra Malaysia sebagai memenuhi keperluan untuk ijazah Doktor Falsafah

PEMBANGUNAN DAN PENGESAHAN INSTRUMEN MENGUKUR KESEDIAAN INSTRUKSIONAL GURU SAINS DALAM IMPLEMENTASI

STEM

Oleh

NUR FARHANA RAMLI

Julai 2019

Pengerusi : Othman Talib. PhD Fakulti : Pengajian Pendidikan STEM telah mendapat perhatian dalam tahun kebelakangan ini di atas usaha kerajaan dalam menyediakan pelajar untuk permintaan tenaga kerja masa hadapan. Ia juga merupakan satu usaha untuk menghasilkan rakyat yang mempunyai literasi terhadap penyelesaian masalah yang melibatkan konteks kehidupan harian, masyarakat dan alam sekitar dengan menggunakan aplikasi pengetahuan, kemahiran dan nilai STEM. Kegagalan polisi terdahulu mendapati bahawa instruksional guru merupakan salah satu aspek yang menyebabkan murid kurang berminat terhadap STEM. Memandangkan kurikulum STEM baru diperkenalkan di dalam sistem pendidikan Malaysia baru-baru ini, terdapat keperluan untuk membina satu instrumen yang mengukur persediaan instruksional guru dalam membantu kejayaan aspirasi pendidikan kebangsaan, khususnya dalam pendidikan STEM. Maka, Instrument Persediaan Instruksional STEM (STEMTIP) telah di bina untuk menguji persediaan instruksional guru dalam implementasi STEM. Kajian ini telah dibahagi kepada dua fasa, iaitu fasa pembangunan dan fasa validasi. Dalam fasa pembangunan, proses konseptualisasi dan penjanaan item dilakukan menggunakan pendekatan induktif dan deduktif. Melalui perbincangan kumpulan berfokus yang melibatkan empat guru STEM dan juga maklumat dari tinjauan literatur, 51 item telah dihasilkan. Item tersebut kemudiannya telah diuji melalui ujian kobolehpercayaan, kesahan muka, kesahan kandungan, kesahan konstruk dan kesahan kriteria. Dalam ujian kesahan muka, lima orang guru sains telah dipilih untuk mengesahkan penggunaan bahasa dan kejelasan item. Bagi kesahan kandungan pula, 15

© COPYRIG

HT UPM

iv

pakar telah terlibat untuk mengesahkan kandungan item tersebut. Melalui ujian kesahan kandungan menggunakan Model Lawshe, 47 item telah mendapat persetujuan bersama pakar, walaubagaimanapun, baki empat item dikekalkan di dalam ujian seterusnya dengan sedikit pemurnian. Kajian lapangan dijalankan ke atas 265 orang guru sains di Malaysia menggunakan persampelan kluster pelbagai tahap. STEMTIP telah dikenalpasti mempunyai tahap kebolehpercayaan dan kebolehreplikasi yang cemerlang melalui dapatan nilai Cronbach alpha (.98), kebolehpercayaan individu (.96) and kebolehpercayaan item (.98). Kesahan konstruk ditentukan menggunakan Model Rasch. Analisis mendapati bahawa instrumen mempunyai bukti unidimension, kebebasan setempat, polariti, indeks pemisah yang cemerlang dan skala yang efektif. Walaubagaimanapun, 10 misfit item telah digugurkan daripada instrumen kerana menunjukkan ciri yang menganggu pengukuran. Misfit items menunjukkan kekurangan konsistensi dalam mentafsir yang memerlukan kajian pada masa akan datang. Juga, dua item dikenalpasti mempunyai potensi bias berdasarkan kedudukan sekolah tetapi dikekalkan kerana ia memberi maklumat yang berguna kepada pemegang taruh. Kesahan kriteria mendapati bahawa STEMTIP dapat menyumbang 40.6% kepada kecekapan guru dalam implementasi STEM. Secara keseluruhannya, berdasarkan kesemua ujian yang dijalankan, instrument terbukti sebagai satu instrumen yang baik. Pengkaji yakin bahawa STEMTIP telah melepasi keperluan piawai psikometrik dan boleh digunakan oleh pemegang taruh untuk mengukur persediaan instruksional guru STEM.

© COPYRIG

HT UPM

v

ACKNOWLEDGEMENTS

I would like to acknowledge these following individuals who play an integral role in this journey. First to my committee members, Dr Othman Talib, PM Dr Siti Aishah Hassan and Dr Umi Khalthom Abdul Manaf who provided support and guidance as I developed my ideas into a defined project and towards the completion if this study. I would also like to thank my beloved support group, OT and Gangs who kindly support and encourage me through this process.

Similarly, my expert panellist, Prof Dr T Subahan Mohd Meera, MyRasch Team, Dr Nurul Fadhly Habidin, Dr Mohd Effendi Ewan Mohd Matore, Dr Hayrol Azril Mohamed Shafril, Prof Norlide Abu Kasim, Puan Hartini Hashim and Puan Nur ‘Atikah Ahmad, who provided thoughtful and important guidance on revising the STEMTIP instrument.

Last but not least, I must thank all of my friends and colleagues who supported me with encouraging words and helpful ideas as I struggled to balance the demands of teaching and being a doctoral student. Finally, to my colleagues at Sekolah Dalam Hospital PPUKM, my Anjung Siswazah friends, Raschians, thank you for the never ending support.

© COPYRIG

HT UPM

© COPYRIG

HT UPM

© COPYRIG

HT UPM

viii

Declaration by graduate student

I hereby confirm that: this thesis is my original work; quotations, illustrations and citations have been duly referenced; this thesis has not been submitted previously or concurrently for any other

degree at any institutions; intellectual property from the thesis and copyright of thesis are fully-owned

by Universiti Putra Malaysia, as according to the Universiti Putra Malaysia(Research) Rules 2012;

written permission must be obtained from supervisor and the office of DeputyVice-Chancellor (Research and innovation) before thesis is published (in theform of written, printed or in electronic form) including books, journals,modules, proceedings, popular writings, seminar papers, manuscripts,posters, reports, lecture notes, learning modules or any other materials asstated in the Universiti Putra Malaysia (Research) Rules 2012;

there is no plagiarism or data falsification/fabrication in the thesis, andscholarly integrity is upheld as according to the Universiti Putra Malaysia(Graduate Studies) Rules 2003 (Revision 2012-2013) and the UniversitiPutra Malaysia (Research) Rules 2012. The thesis has undergoneplagiarism detection software

Signature: Date:

Name and Matric No: Nur Farhana binti Ramli, GS47623

© COPYRIG

HT UPM

ix

Declaration by Members of Supervisory Committee

This is to confirm that: the research conducted and the writing of this thesis was under our

supervision; supervision responsibilities as stated in the Universiti Putra Malaysia

(Graduate Studies) Rules 2003 (Revision 2012-2013) were adhered to.

Signature: Name of Chairman of Supervisory Committee: Dr. Othman Talib

Signature: Name of Member of Supervisory Committee:

Associate Professor AP Dr. Siti Aishah Hasan

Signature: Name of Member of Supervisory Committee:

Associate Professor AP Dr. Umi Khalthom Abdul Manaf

© COPYRIG

HT UPM

x

TABLE OF CONTENTS

Page

ABSTRACT i ABSTRAK iii ACKNOWLEDGEMENTS v APPROVAL vi DECLARATION viii LIST OF TABLES xiii LIST OF FIGURES xv LIST OF APPENDICES xvi LIST OF ABBREVIATIONS xviii CHAPTER 1 INTRODUCTION 1

1.1 Background of the Study 1 1.1.1 STEM Education in Malaysia 2 1.1.2 STEM Teacher Instructional Preparedness 3

1.2 Problem Statement 4 1.3 Objective of the Study 5 1.4 Significance of the Study 6 1.5 Scope of the study 7 1.6 Delimitation of the Study 8 1.7 Definition of terms 9

1.7.1 Instructional preparedness 9 1.7.2 STEM Teachers 9 1.7.3 Validation Process 10 1.7.4 STEM Implementation 10

1.8 Summary 10 2 LITERATURE REVIEW 11

2.1 Introduction 11 2.2 STEM Teachers’ Instructional Approach 11 2.3 STEM Teachers’ Instructional Issues 13 2.4 STEM Instructional Preparedness 15 2.5 Previous STEM Instrument Development 16 2.6 Instrument Development Model 19

2.6.1 DeVille’s Scale Development Guidelines 19 2.6.2 Miller Test Development Process 19

2.7 Measurement Model 20 2.7.1 Classical Test Theory (CTT) 21 2.7.2 Item Respond Theory 22

2.8 Theoretical Framework of the Study 24 2.8.1 Social Constructivism Theory 24 2.8.2 5E Instructional Model 25 2.8.3 STEM Teaching and Learning Approach Model

26

© COPYRIG

HT UPM

xi

2.8.4 Self-Efficacy Theory 28 2.9 Conceptual Framework of the Study 30 2.10 Summary 31

3 METHODOLOGY 32

3.1 Introduction 32 3.2 Research Design 32

3.2.1 Step 1: Instrument Conceptualisation 34 3.2.2 Step 2 Instrument Development. 34 3.2.3 Step 3 Face and Content Validity 36 3.2.4 Step 4 Pilot Study. 38 3.2.5 Step 5 Reliability Test. 38 3.2.6 Step 6 Item Analysis. 39 3.2.7 Step 7 Revising the Instrument. 43 3.2.8 Step 8 Field study. 44 3.2.9 Step 9 Criterion Validation. 44

3.3 Sampling 45 3.3.1 Focus Group Discussion 46 3.3.2 Face Validity Experts 46 3.3.3 Content Validity Experts 47 3.3.4 Pilot study 49 3.3.5 Field Study 50

3.4 Research Instrument 51 3.4.1 STEMTIP 52 3.4.2 SETIS 54

3.5 Data Collection Procedures 54 3.6 Data Analysis Procedures 56 3.7 Pilot Study Findings 58

3.7.1 Item Fit 58 3.7.2 Unidimensionality 58 3.7.3 Local Independence 59 3.7.4 Item Polarity 60 3.7.5 Scale Revision 60 3.7.6 Reliability and Separation Indices 61 3.7.7 Items in Field Study 62

3.8 Summary 62 4 RESULTS AND DISCUSSION 63

4.1 Introduction 63 4.2 Demographic Profile 63 4.3 Objective Discussion of the Studies 65

4.3.1 Objective 1 65 4.3.2 Objective 2 68 4.3.3 Objective 3 75 4.3.4 Objective 4 85 4.3.5 Objective 5 86

4.3.5.1 Item Fit 87 4.3.5.2 Unidimensionality 90 4.3.5.3 Local Independence 92

© COPYRIG

HT UPM

xii

4.3.5.4 Item Polarity 92 4.3.5.5 Separation Index 93 4.3.5.6 Scale Revision 94 4.3.5.7 STEMTIP Psychometric Summary 95

4.3.6 Objective 6 96 4.3.7 Objective 7 98 4.3.8 Objective 8 99

4.4 Summary 102 5 SUMMARY, IMPLICATION AND RECOMMENDATIONS 103

5.1 Introduction 103 5.2 Summary of the Study 103 5.3 Study Implication 107

5.3.1 Theoretical Implication 107 5.3.2 Practical Implication 107

5.4 Future Research Recommendation 108 5.5 Conclusion of the Study 109

REFERENCES 110 APPENDICES 132 BIODATA OF STUDENT 194 LIST OF PUBLICATIONS 195

© COPYRIG

HT UPM

xiii

LIST OF TABLES

Table Page 2.1 Previous Study in Instrument Development in STEM or STEM

discipline 18

3.1 Revised Critical Values for Lawshe’s (1975) Content Validity Ratio 37

3.2 Sampling Technique Based on Section of Study 45

3.3 Sample Size Range 46

3.4 List of Focus Group Discussion Panel 46

3.5 List of face validity experts 47

3.6 List of professional experts 48

3.7 List of lay experts 48

3.8 Sample Selection Based on School and Location 51

3.9 Research Instrument 51

3.10 STEMTIP Item Distribution 52

3.11 Research Procedures 55

3.12 Software used for Data Analysis 57

3.13 Pilot Study Demographic Distribution 58

3.14 Table of Standardized Residual Variance 59

3.15 Standardize Residual Correlation 59

3.16 Summary of Category Structure 61

3.17 Reliability and separation indices 62

4.1 Return Rate of the Instrument 63

4.2 Field Study Demographic Distribution 64

4.3 Definition of the construct in STEMTIP 67

© COPYRIG

HT UPM

xiv

4.4 Generation of Items based on Operational Definition 68

4.5 List of Initial Generated items 71

4.6 STEMTIP items distribution 75

4.7 CVR Comparison among Experts (N=15) 77

4.8 List of Refined Item 78

4.9 List of Pilot Items 79

4.10 STEMTIP Reliability Indices 86

4.11 Item Fit and Item Polarity Value (51 items) 88

4.12 List of misfit items 89

4.13 Standardized Residual Variance 91

4.14 Standardized Residual Correlations 92

4.15 Item-Person Separation and Strata Value 93

4.16 Summary of Category Structure 95

4.17 Summarize on Rasch Model Assumption 96

4.18 Differential Item Functioning of the Item 99

4.19 Skewness and Kurtosis Value 100

4.20 Model summary 101

4.21 Coefficients 102

© COPYRIG

HT UPM

xv

LIST OF FIGURES

Figure Page 2.1 Guidelines in Instrument Development 19

2.2 Instrument Development Process 20

2.3 STEM Teaching and Learning Approach Model 27

2.4 Theoretical Framework for STEM Teachers’ Instructional Preparedness Instrument (STEMTIP) 30

2.5 Conceptual Framework for STEM Teachers’ Instructional Preparedness Instrument (STEMTIP) 31

3.1 STEMTIP Instrument Development Process 33

3.2 Focus Group Discussion Process 35

3.3 Example of item person map 43

3.4 Multistage Random Cluster Sampling Procedure for the Pilot Study 49

3.5 Multistage Random Cluster Sampling Procedure for the Field Study 50

3.6 Scale Probability Curve 60

4.1 Scale Probability Curve 94

4.2 Item-Person Map 97

4.3 Histogram and normal Q-Q plot diagram 101

© COPYRIG

HT UPM

xvi

LIST OF APPENDICES

Appendix Page A Letters of Authorities 132 A1 Expert Appointed Letter 132 A2 UPM Student Confirmation Letter 133 A3 Authorization letter from Ministry of Education 134 A4 Authorization letter from Malacca State Education

Department 135

A5 Authorization letter from Selangor State Education

Department 136

A6 Authorization letter from Kelantan State Education

Department 137

A7 Authorization letter from Kedah State Education

Department 138

A8 Permission to use Teachers' Self-Efficacy to Teach

Science in an Integrated STEM Framework (SETIS) 139

B Research Instruments 140 B1 STEMTIP for Content Validity 140 B2 STEMTIP for Pilot study 151 B3 STEMTIP and SETIS for Field study 155 B4 STEMTIP final revision 161 C Expert Credentials 163 C1 Credentials for panel of experts 163 D Evident of Focus Group Discussion 165 D1 Focus group discussion protocol 165 D2 Report On Focus Group Discussion 166 E Evident of Face and Content Validity 170

© COPYRIG

HT UPM

xvii

E1 Face Validity Summary 170 E2 Content Validity Summary (Experts Comments) 171 F Complimentary results 178 F1 Pilot study result – Reliability and separation index 178 F2 Pilot study result – Item Fit 179 F3 Pilot study result – Unidimensionality 181 F4 Pilot study result – Local independence 181 F5 Pilot study result – Scale Revision 182 F6 Field study result – Reliability and separation index 183 F7 Field study result – Item Fit 184 F8 Field study result –Unidimensionality 185 F9 Field study result - Local Independence 186 F10 Field study result – Scale Revision 187 F11 Field study result – Item person map 188 F12 Field study result – Differential item function 189 F13 Field study Results – Normality Test 190 F14 Field study Results – Regression Analysis 193

© COPYRIG

HT UPM

xviii

LIST OF ABBREVIATIONS

UPM Universiti Putra Malaysia

FPP Fakulti Pengajian Pendidikan/ Educational Studies Faculty

KPM Kementerian Pendidikan Malaysia / Ministry of Education

STEM Science, Technology, Engineering, Mathematics

DIF Differential Item Functioning

CTT Classical Test Theory

IRT Item Response Theory

p&p Pengajaran dan Pembelajaran (Teaching and Learning)

MnSq Mean Square

ZPD Zone of peroximal development

CVR Content validity ratio

ICT Information Communication and Technology

© COPYRIG

HT UPM

1

CHAPTER 1

1 INTRODUCTION

1.1 Background of the Study

STEM is an acronym for Science, Technology, Engineering and Mathematics, an attempt to integrate these disciplines into one multidisciplinary approach. STEM began with the launch of the world's first artificial satellite, Sputnik by Russia in 1957 (U.S. Information Agency, 1959). It sparked competition between countries such as the US, United Kingdom and Russia itself to lead in the field of science and technology.

In the competition, these countries are making investments and revamping gradually against the existing curriculum. Among them are School Mathematics Project (1962), Nuffield Science Teaching Project (1966), Assessment of Performance Unit (1980) and Children’s Learning in Science Project (1980). Singapore is the first Asian country to form a specific curriculum focusing on problem solving where the outcome can be seen through the great outstanding achievement in Trends in International Mathematics and Science Study (Banks & Barlex, 2014).

STEM plays a vital role worldwide, as the strength of STEM can determine the prosperity of the country. Demands for experts soar in STEM field and opportunities of STEM based jobs such as engineers, technicians, application software developers, scientist, biostatisticians, cartographers and architect have increased exponentially. Malaysia is also feeling the pressure, as the Vision 2020 is approaching and it sets to have one million individuals with science-related training to compete in the globalised age by 2020 (Academy of Sciences Malaysia, 2015).

Unfortunately, Malaysia’s dream to achieve the number is still a distant possibility. Despite the urgent need for qualified researchers, scientists and engineers for the country’s growth, it is reported that Malaysia’s ratio of researchers to labour force is at the low number of 58.2 researchers out of every 10,000 labour force. This ratio is relatively small as compared to other Asian nation like South Korea and Singapore with 142.5 and 127.4 researchers per 10 000 labour force respectively (Malaysian Science and Technology Information Centre, 2014). One way to produce more researchers in the country is by designing a quality education system.

© COPYRIG

HT UPM

2

A quality education system equipped with well organised curriculum is a vital component to produce quality human resources and to realise the nation’s educational objectives. Earlier effort called the 60:40 Science Art Policy, which was enacted in 1967, had failed to attract students to continue their career in the STEM field (Suhanna Zainudin, Lilia Halim, & Zanaton Ikhsan, 2015). However, the Ministry of Education has taken rectification steps with embedding STEM in the secondary school standard curriculum from 2017 (Ministry of Education Malaysia, 2013).

1.1.1 STEM Education in Malaysia

Formally in Malaysia, there was no specific STEM based education programme before, but awareness of science-based education was seen since 1967 with the establishment of a Higher Education Planning Committee that subsequently created a 60:40 Science Art Policy to increase the number of human resources in science field. The aim of the policy is to have 60% of all upper secondary students focusing on the sciences and 40% enrolled in the arts (Ministry of Education, 2013). However, despite all the efforts to promote the policy, the number of science students has gradually declined in science stream enrollment in upper secondary school. It was reported that enrollment dropped from 37% in 1998 to 29% in 2012. This decline has hampered the efforts made by the Higher Education Planning Committee to get more students venturing into science streams (Academy of Sciences Malaysia, 2015). The declining number of science students was due to several reasons.

Changes in curriculum, quality teacher and ineffective teaching instruction are among the reasons on the declining interest in Science and Mathematics (Academy of Sciences Malaysia, 2015). Ineffective teaching instruction will dampen the development of higher order thinking skill which is one of the crucial skills needed in STEM. Also, a study conducted by the Malaysian Education Ministry also found additional factors contributing to the above enrolment issue, such as limited awareness about STEM, perceived difficulty of STEM subjects, content-heavy curriculum, inconsistent quality of teaching and learning and limited and outdated infrastructure (Ministry of Education Malaysia, 2013). Responding to the findings, the Ministry of Education has implemented several steps to respond to the issue.

To address the stated issues, the Ministry of Education has introduced the STEM approach in the curriculum beginning from 2017 in the Secondary School Standard Curriculum (Curriculum Development Division, 2016a, 2016c). The curriculum is based on the Malaysia STEM roadmap that has been carefully planned and it is divided into three waves.

© COPYRIG

HT UPM

3

Wave 1 (2013-2015): Strengthening the quality of STEM education was initiated through the strengthening of curriculum, testing and teacher training

Wave 2 (2016-2020): campaign and collaboration with related stakeholders are implemented to attract and publicize STEM

Wave 3 (2021-2025): STEM will be shifted towards excellence through improved flexible operation (Curriculum Development Division, 2016c; Ministry of Education Malaysia, 2013)

Although STEM education was just implemented in 2017, the study has begun earlier than that. From 1999 to 2013, there were 57 articles on STEM conducted and unfortunately the study of teachers was very limited (Kamaleswaran Jayarajah, Rohaida Mohd Saat, & Rose Amnah Abdul Rauf, 2014). However, recent studies showed that STEM education research keep increasing from time to time involving curriculum, assessment, student issues and STEM modules (Fazilah Razali et al, 2018; Mazlini Adnan et al., 2016; Ng & Adnan, 2018; Siew et. al, 2015) This trend has shown that there is an effort to develop STEM education in Malaysia.

1.1.2 STEM Teacher Instructional Preparedness

Teachers are required to have good preparation and standard before, during and after the teaching and learning process. A good teacher is a teacher who is prepared (Darling-Hammond, Chung, & Frelow, 2002). Teaching preparation involves several steps such as instructional planning, developing and selecting teaching aids and natural environment preparedness (Bruder, Dunst, Wilson, & Stayton, 2013; Slavit, Nelson, & Lesseig, 2016). All this is done prudently to ensure quality student outcomes is in line with educational objectives.

Teachers’ instructional preparedness has an impact on the students’ outcome and studies have shown that students’ interest and achievement in Science and Mathematics are related with the teachers' preparation (Henry et al., 2011; Slavit et al., 2016). However, previous studies have reported that STEM teachers feel that their practice is inadequate to meet the needs of students (Besterman, Williams, & Ernst, 2018; Siew et al., 2015).

In facing the challenge of STEM teachers’ instructional preparedness in Malaysia, the Ministry of Education has set up the School Improvement Specialist Coaches (SISC+) teams for Mathematics and Science subject. The aim of this effort is to equip STEM teachers with content knowledge and pedagogy that emphasise experimentation and application. Secondly, another initiative named Blended Learning Open Source Science and Mathematics Studies project (BLOSSOMS) produces interactive STEM videos to help

© COPYRIG

HT UPM

4

teachers reinforce critical thinking skills and content knowledge through the use of ICT in teaching (Ministry of Education, 2017).

Since STEM is a new implementation in the Malaysian curriculum, there is currently no study that focuses on STEM teachers' preparedness. Therefore, this calls the need of conducting more research regarding STEM development, specifically on STEM teachers’ instructional preparedness.

1.2 Problem Statement

The growing demand of STEM workforce worldwide is firmly established in the report by U.S Bureau of Labour Statistics in 2017. In this report, demand for STEM related workforce has increased to 10.5% from the period of May 2009 to May 2015 as compared to only 5.2% net growth in the non-STEM related workforce for the same period (Fayer, Lacey, & Watson, 2017). The same trend is projected in Malaysia as Malaysia has set to have one million qualified individual to fill the STEM workforce demand by 2020, compared to the current 120,000 STEM workforce (Academy of Sciences Malaysia, 2015).

In order to realise the vision, awareness to increase students’ interest in STEM subject has led the Ministry of Education to include it as one of the pillars in Secondary School Curriculum Standard (Curriculum Development Division, 2016a). As teachers are playing the pivotal role for the implementation STEM in school, teachers have a major influence on students’ achievements and interest (Alexander, Knezek, Bull, Christensen, & Tyler-Wood, 2014; Houseal, Abd-El-Khalick, & Destefano, 2014; Kazempour, 2014). Furthermore, teachers’ instruction can inspire and give impact on students’ choice to pursue in STEM major in college (Lichtenstein, Tombari, Sheppard, & Storm, 2014).

The previous 60:40 policy has failed to attract students in science. The studies have pointed out that outdated teacher's instructional approaches have caused the demise of 60:40 Science Art Policy (Academy of Sciences Malaysia, 2015; Suhanna Zainudin et al., 2015). This should not happened as teacher's instructional approach need evolve over time and able to fascinate students to the lesson.

Previously, the development of the STEM instrument has been carried out by some researchers such as (Benjamin et al., 2015; Lin & Williams, 2015; Mobley, 2015). However, none of the instruments were built to measure STEM teachers' preparedness from the instructional perspective. The lack of initiative may be due to the complicated and complex proses of developing the instrument itself.

© COPYRIG

HT UPM

5

In instrument development process, the main issues are validation and reliability. Incomplete validation process as suggested by DeVellis (2017) and Bond & Fox (2015), makes this instrument arguable. There are three types of essential validation in instrument development; construct, content and criterion validation, but some of the instruments have evident lack of them (Benjamin et al., 2015; Nadelson et al., 2013).

Additionally, a befitting issue arising in the instrument development is the selection of measurement theory. Most researchers use Classical Test Theory (CTT) to show the relationship between measured traits. However, Item Respond Theory (IRT) is able to give more information and able demonstrate a clearer map in scale development especially in measuring person abilities and item difficulties (Sharkness & Deangelo, 2011; Zile-tamsen, 2017). Also, another advantage of IRT is the ability to determine differential item functioning (DIF) item. DIF items can provide information on the biased items needed in assisting the Ministry of Education to reduce urban and rural school gaps (Ministry of Education Malaysia, 2013).

The development of an instrument to measure teachers’ instructional preparedness using thorough validation steps is crucial as a standard tool to measure current teachers’ level of preparedness as it relate to students interest and achievement in STEM. This is to prevent a new STEM curriculum being implemented, not to fail like the previous policy. With the information obtained, through the analysis using Rasch Model, information such as biased items, misfit item can be identified to help stakeholders in ensuring to achieve the aspirations and goals of the nation's education philosophy.

1.3 Objective of the Study

The main objective of this research is to develop a valid and reliable instrument to measure teachers’ instructional preparedness in STEM implementation. Specifically, the objectives of this study include:

1. to define the concept in STEM teachers’ instructional preparedness 2. to generate items of STEM Teachers’ Instructional Preparedness

Instrument (STEMTIP) 3. to test the content validity of STEM Teachers’ Instructional

Preparedness Instrument (STEMTIP) 4. to assess the reliability index of STEM Teachers’ Instructional

Preparedness Instrument (STEMTIP) 5. to test the construct validity of STEM Teachers’ Instructional

Preparedness Instrument (STEMTIP) 6. to explore the level of science teachers’ preparedness in STEM

implementation.

© COPYRIG

HT UPM

6

7. to examine the existance of differential item functioning (DIF) based on school location factor

8. To estimate the relationship of STEM teachers’ instructional preparedness and teachers’ self-efficacy in implementing STEM

1.4 Research Questions

1. What are the concepts in STEM teachers’ instructional preparedness? 2. What are the items in STEM Teachers’ Instructional Preparedness

Instrument (STEMTIP)? 3. What is the agreement ratio of each item in STEM Teachers’

Instructional Preparedness Instrument (STEMTIP)? 4. What are the reliability indices of STEM Teachers’ Instructional

Preparedness Instrument (STEMTIP)? 5. Does STEM Teachers’ Instructional Preparedness Instrument

(STEMTIP) meet the fit indices of Rasch Model? 6. What is the level of science teachers’ preparedness in STEM

implementation? 7. What are the differences of STEM teachers’ instructional preparedness

based on school location? 8. Does STEM teachers’ instructional preparedness predict teachers’ self-

efficacy in implementing STEM? 1.5 Significance of the Study

STEM education is crucial in creating a workforce that contributes to the nation's development and science literate citizens that able to use STEM knowledge in solving daily problems. The Ministry of Education’s efforts of including STEM in the curriculum need to be welcomed by all parties. Since teachers carry a paramount influence in raising students’ interest and outcome in STEM, a study related to teachers’ instructional preparedness should be carried out. This study is significant as in the best knowledge of the researcher, to date, there is no evidence of instrument that measures teachers’ preparedness in STEM instructional.

This newly developed instrument used Item Response Theory as this theory will allow more information to be obtained than other measurement theories. This instrument will go through nine basic assumptions in Rasch Model. The assumptions are item fit, unidimensionality, local independence, item polarity, scale revision, reliability index and separation index. The development process also went through four validation processes; face validity, content validity, construct validity and criterion validity which rarely complied by other instrument developer. All of these refined processes were performed to ensure that the instrument produced is valid and reliable to use.

© COPYRIG

HT UPM

7

In addition, this study significantly contributes to the existing knowledge for analysing and confirming the indicator factors involved in STEM teachers’ instructional preparedness. The combination of theories and models used in developing the conceptual framework was based on the Malaysian Education System. Furthermore, this study included value as one of the elements embedded on standard curriculum that is rarely emphasised in the previous STEM study (Curriculum Development Division, 2016c).

Subsequently, the results obtained from this study presented valid evidence to measure teachers’ preparedness for STEM instructional. The instrument able to help teachers to perform self-assessment to measure their own level of preparedness. Undoubtedly, this information will also benefit stakeholders especially those involved in curriculum planning to develop a more effective preparatory programs for pre-service STEM teachers.

Currently, the two departments in the Education Ministries, Teacher Education Department and Higher Education Department are two department that are responsible to prepare special programs or series of workshop to assist the group of in-service and preservice teachers in increasing the move of continuing professional development for effective STEM instruction (Siew et al., 2015). Involvement of these devision can help teachers keep up to date with the current situation, in this case, the STEM implementation

Concisely, a valid and systematically developed instrument in measuring teachers’ preparedness in STEM instructional is critically needed. Due to lack of research about teachers and lecturers across the studies in STEM education (Kamaleswaran Jayarajah et al., 2014), the development of this instrument added more knowledge in STEM education, particularly in the context of the Malaysian scenario.

1.6 Scope of the study

This study focuses on developing an instrument to measure teachers' instructional preparedness in STEM implementation. It goes through two phase, developement and validation phase. Four types of validation, namely face, content, construct and concurrent validity were conduct to verify this newly develop instrument. In addition to proving validity, this study was also conducted to indentify the level of teachers’ preparedness in STEM implementation as well as to detect any differences in urban and rural teachers’ instructional preparedness.

© COPYRIG

HT UPM

8

This study is limited to government secondary school science teachers. The science teacher is defined as teacher who teaches subjects either science, chemistry, biology or physics. In addition, the respondents involved in the study are science teachers in peninsular only due to geography, time and costs constraints.

1.7 Delimitation of the Study

There are eight delimitation that has been set up for this study. Firstly, the set limitations measured in STEMTIP are limited to instructional preparedness only and they are based on STEM Implementation Guide in teaching and learning (Curriculum Development Division, 2016c). Since the implementation was just recently introduced, the test based on the stipulation set by the Ministry of Education needs to be carried out.

Secondly, this study will be based on the 10 step instrument development adapted from DeVellis (2017) and Miller et al. (2013). The development of instrument norms involving the score comparison between respondents, profiling and manual development was not carried out to focus on the rigorous step of instrument development and validation.

Thirdly, this study only involved science teachers only. This is because science teachers are the dominant group of teachers who need to integrate all four disciplines in STEM in teaching and learning session compared to other subject teachers based on the national syllabus (Curriculum Development Division, 2015).

Next, Malaysia has a diversity of geographical area which causes some schools that cannot be contacted directly through the telephone or e-mail method especially in the interior of Sabah and Sarawak. It will also indirectly impose financial implications on time and cost to carry out research. Thus, this study will use teachers in peninsular Malaysia as a population of studies.

The use of multistage random cluster sampling method for this study is the best method to apply due to the extensive population and also limited information from the stakeholders. The researcher will randomly select the school involved based on several stages and then all the teachers from the selected schools will be respondents for this study. As teachers in Malaysia have the same basic qualification and have been exposed to STEM through Professional Learning Community programme, the findings of this study can be generalized to all Science teachers in Malaysia (Ministry of Education, 2013, 2016).

© COPYRIG

HT UPM

9

Afterward is the validation test. Other than the required reliability test, only four validity test types were conducted namely face validity, content validity, construct validity and predictive validity. This type of validity is sufficient in proving the instrument validity based on scholar suggestion (Bond & Fox, 2015; DeVellis, 2017).

Subsequently, the construct validation test used in this study is the Rasch Model. This model is one of the Item Respond Theory model. This model able to provide more information compared to Classical Test Theory, such as item discrimination, person and item reliability and scale revision. This model also complies with the requirements of parametric analysis in the generation of intervals data (Azrilah Abdul Aziz, Mohd Saidfudin Masodi, & Azami Zaharim, 2017; Bond & Fox, 2015).

Lastly, due to uneven proportion of teachers’ gender, the differential item functioning (DIF) tests only involve differences based on only school locations name urban and rural schools. The information obtained is useful to stakeholders in helping to reduce the gap between these types of schools (Ministry of Education Malaysia, 2013).

1.8 Definition of terms

1.8.1 Instructional preparedness

Seligman & Hager (1972) interpret preparedness as state of readiness formed by stimuli and responses. In the context of this research, preparedness is defined as measurement of preparedness of a science teacher to facilitate STEM instructional in teaching and facilitating session based on their practice on STEM Implementation Guide in Teaching and Learning (Curriculum Development Division, 2016c).

1.8.2 STEM Teachers

STEM is an acronym for the disciplines of science, technology, engineering and mathematics. In this study, STEM teachers is focused on teachers who teach science and elective science subjects such as Science, Physics, Chemistry And Biology (Curriculum Development Division, 2016c).

© COPYRIG

HT UPM

10

1.8.3 Validation Process

Validation is a cumulative process in determining the characteristic of how the instrument is used (DeVellis, 2017). This study involved four types of validation, namely face validation, content validation, construct validation and predictive validation in determining teachers’ instructional preparedness in STEM implementation.

1.8.4 STEM Implementation

In this study, STEM implementation is based on STEM Implementation Guide in Teaching and Learning (Curriculum Development Division, 2016c). STEM in teaching and learning includes field of study, package of study and teaching and learning approach. In this study, the teachers instructional will be focused on teaching and learning approach during the lesson.

1.9 Summary

In this chapter, the researcher has presented an overview about the proposed instrument. It involved the discussion about the background, problem statement, objective, significance, delimitation of the study and the definition of terms. The next chapter will describe the literature review which provides further explanation about this study.

© COPYRIG

HT UPM

110

6 REFERENCES

Academy of Sciences Malaysia. (2015). Science Outlook 2015: Action Towards Vision. (Academy of Sciences Malaysia, Ed.). Ministry of Sciene, Technology And Innovation.

Açişli, S., Yalçin, S. A., & Turgut, Ü. (2011). Effects of the 5E learning model on students’ academic achievements in movement and force issues. Procedia - Social and Behavioral Sciences, 15, 2459–2462. https://doi.org/10.1016/j.sbspro.2011.04.128

Adibah Abdul Latif. (2013). Pembangunan instrumen penilaian akhlak pelajar di institusi pengajian tinggi awam. Universiti Teknologi Malaysia. Universiti Teknologi Malaysia.

Ahmad Zamri Khairani, & Mohd Effendi Ewan Mohd Matore. (2016). Application of Rasch Model Analysis in Calibrating Undergraduates ’ Challenges at Malaysian Universities. World Applied Sciences Journal, 34(8), 1124–1128. https://doi.org/10.5829/idosi.wasj.2016.34.8.15654

Aimi Najlaa Muhammad Noor, & Faridah Yunus. (2017). Kualiti aktiviti di prasekolah KPM dari perspektif guru. Jurnal Penyelidikan Pendidikan, (18), 71–82.

Alabi, A. O. (2010). E-Education in Nigeria : Students ’ state of readiness for e-learning at the University of Lagos. In 2010 International Conference on Information Society (pp. 465–470). IEEE.

Aldridge, J. M., & Galos, S. (2017). Development and validation of an instrument to assess primary school students’ perceptions of the learning environment. Learning Environments Research, 1–20. https://doi.org/10.1007/s10984-017-9248-7

Alexander, C., Knezek, G., Bull, G., Christensen, R., & Tyler-Wood, T. (2014). The impact of project-based learning on pre-service teachers’ technology attitudes and skills. Journal of Computers in Mathematics and Science Teaching, 33(3), 257–282.

Alexandrowicz, R. W., Jahn, R., & Wancata, J. (2018). Assessing the dimensionality of the CES-D using multi-dimensional multi-level Rasch models. PLoS ONE, 13(5), 1–19. https://doi.org/10.1371/journal.pone.0197908

Alkan, F. (2013). the Effect of Alternative Assessment Techniques on Chemistry Competency Perceptions and Chemistry Success of Prospective Science Teachers. Journal of Baltic Science Education, 12, 774–783. Retrieved from

© COPYRIG

HT UPM

111

http://search.ebscohost.com/login.aspx?direct=true&db=aph&AN=93342843&site=ehost-live

Allinder, R. M. (1994). The Relationship Between Efficacy and the Instructional Practices of Special Education Teachers and Consultants. Teacher Education and Special Education, 17(2), 86–95.

Apple, M. T. (2013). Using Rasch analysis to create and evaluate a measurement instrument for foreign language classroom speaking nnxiety. Japan Association for Language Teaching (JALT), 35(1), 5–28.

Armstrong, A. E., Parsons, S., & Barker, P. . (2000). An inquiry into moral virtues, especially compassion, in psychiatric nurses: findings from a Delphi study. Journal of Psychiatric & Mental Health Nursing, 7(4), 297. https://doi.org/10.1046/j.1365-2850.2000.00306.x

Asghar, A., Ellington, R., & Rice, E. (2012). Supporting STEM education in secondary science contexts. Interdisciplinary Journal of Problem- Based Learning, 6(2), 85–125. https://doi.org/10.7771/1541-5015.1349

Athanasou, J., & Lamprianou, I. (2002). A teacher’s guide to assessment. Sydney: Social Science Press.

Azrilah Abdul Aziz, Mohd Saidfudin Masodi, & Azami Zaharim. (2017). Asas Model Pengukuran Rasch : Pembentukan Skala dan Struktur Pengukuran (1st ed). Bangi: Penerbit Universiti Kebangsaan Malaysia.

Baghaei, P. (2008a). Local Dependency and Rasch Measures. Rasch Measurement Transactions, 21(3), 1105–1106.

Baghaei, P. (2008b). The Rasch model as a construct validation tool. Rasch Measurement Transactions, 22(1955), 1145–1162. https://doi.org/10.1017/CBO9781107415324.004

Balsamo, M., Giampaglia, G., & Saggino, A. (2014). Building a new Rasch-based self-report inventory of depression. Neuropsychiatric Disease and Treatment, 10, 153–165. https://doi.org/10.2147/NDT.S53425

Banilower, E. R., Heck, Daniel, J., & Weiss, Iris, R. (2007). Can Professional Development Make the Vision of the Standards a Reality? The Impact of the National Science Foundation’s Local Systemic Change Through Teacher Enhancement Initiative Eric. Journal of Research in Science Teaching, 44(2), 375–395. https://doi.org/10.1002/tea

Banks, F., & Barlex, D. (2014). Teaching STEM in the Secondary School. Teaching STEM in the Secondary School: Helping Teachers Meet the Challenge. https://doi.org/10.4324/9780203809921

© COPYRIG

HT UPM

112

Baran, E., Bilici, S. C., Mesutoglu, C., & Ocak, C. (2016). Moving STEM beyond schools : students ’ perceptions about an out-of- school STEM education program. International Journal of Education in Mathematics, Science and Technology, 4(1), 9–19. https://doi.org/10.18404/ijemst.71338

Barker, B. S., Nugent, G., & Grandgenett, N. F. (2014). Examining fidelity of program implementation in a STEM-oriented out-of-school setting. International Journal of Technology and Design Education, 24(1), 39–52. https://doi.org/10.1007/s10798-013-9245-9

Barry, N. (2014). 6E Learning byDesign Model, (March), 14–19.

Basham, J. D., & Marino, M. T. (2011). Understanding STEM Education and Supporting Students Through Universal Design for Learning. TEACHING Exceptional Children, 45(4), 8–15. https://doi.org/10.1177/004005991304500401

Belland, B. R. (2014). Scaffolding : Definition , current debates and future directions. In J. M. Spector, M. D. Merrill, J. Elen, & M. J. Bisho (Eds.), Handbook of Research on Educational Communications and Technology (4th ed., pp. 505–518). New York: Springer.

Belland, B. R. (2017). Instructional scaffolding in STEM education : strategies and efficacy evidence. Utah: Springer. https://doi.org/10.1007/978-3-319-02565-0

Benjamin, T. E., Marks, B., Demetrikopoulos, M. K., Rose, J., Pollard, E., Thomas, A., & Muldrow, L. L. (2015). Development and Validation of Scientific Literacy Scale for College Preparedness in STEM with Freshman from Diverse Institutions. International Journal of Science and Mathematics Education, 1–17. https://doi.org/10.1007/s10763-015-9710-x

Besterman, K., Williams, T. O., & Ernst, J. (2018). STEM Teachers’ Preparedness for English Language Learners. Journal of STEM Education: Innovations & Research, 19(3), 33–39.

Billiar, K., Hubelbank, J., Oliva, T., & Camesano, T. (2014). Teaching STEM by design. Advances in Engineering Education, 4(1), 1–21.

Bity Salwana Alias, Rosli Ismail, Shamsudin Mohammad, Maimunah Muda, Zaini Ahmad, Ura Pin, … Akbariah Mohd Mahdzir. (2017). Kajian pengukuran tahap perpaduan 2016. Jurnal Penyelidikan Pendidikan, 18, 13–19.

Blazar, D., & Kraft, M. A. (2016). Teacher and teaching effects on students attitudes and behaviors. Educational Evaluation and Policy Analysis, 39(1), 1–25. https://doi.org/10.3102/0162373716670260

© COPYRIG

HT UPM

113

Bodzin, A. M., & Beerer, K. M. (2003). Promoting inquiry-based Science istruction : the validation of the Science Teacher Inquiry Rubric (STIR). Journal of Elementary Science Education, 15(2), 39–49.

Bond, T. G., & Fox, C. M. (2015). Applying the Rasch model : fundamental measurement in the human sciences (3rd ed., Vol. 1). New York: Routledge. https://doi.org/10.1207/S15327574IJT013&4_10

Boone, W. J., Staver, J. R., & Yale, M. S. (2014). Rasch Analysis in the Human Sciences. Springer. https://doi.org/10.1007/978-94-007-6857-4

Brinkman, S. A., Kinnell, A., Maika, A., Hasan, A., Jung, H., & Pradhan, M. (2017). Validity and Reliability of the Early Development Instrument in Indonesia. Child Indicators Research, 10(2), 331–352. https://doi.org/10.1007/s12187-016-9372-4

Brown, A. (2015). 3D printing in instructional settings : identifying a curricular hierarchy of activities. TechTrends, 59(5), 16–24.

Bruder, M. B., Dunst, C. J., Wilson, C., & Stayton, V. (2013). Predictors of confidence and competence among early childhood interventionists. Journal of Early Childhood Teacher Education, 34(3), 249–267. https://doi.org/10.1080/10901027.2013.816806

Bybee, R. W. (2009). The BSCS 5E intructional model and 21st century skills. Workshop on exploring the intersection of science education and the development of 21st century skills.

Bybee, R. W., Taylor, J. A., Van Scotter, P., Powell, J. C., & Landes, N. (2006). The BSCS 5E Instructional Model : Origin and Effectiveness. Colorado Springs.

Cappelleri, J. C., Jason Lundy, J., & Hays, R. D. (2014). Overview of classical test theory and item response theory for the quantitative assessment of items in developing patient-reported outcomes measures. Clinical Therapeutics, 36(5), 648–662. https://doi.org/10.1016/j.clinthera.2014.04.006

Capraro, R. M., Capraro, M. M., & Morgan, J. R. (2013). STEM project-based learning. Rotterdam: Sense Publishers.

Carley-Baxter, L. R., Hill, C. A., Roe, D. J., Twiddy, S. E., Baxter, R. K., & Ruppenkamp, J. (2009). Does response rate matter? journal editors use of survey quality measures in manuscript publication decisions. Survey Practice, 2(7), 1–7. https://doi.org/10.1021/jp012662w

© COPYRIG

HT UPM

114

Ceylan, S., & Ozdilek, Z. (2015). Improving a Sample Lesson Plan for Secondary Science Courses within the STEM Education. Procedia - Social and Behavioral Sciences, 177(July 2014), 223–228. https://doi.org/10.1016/j.sbspro.2015.02.395

Chandler, J., Fontenot, a. D., & Tate, D. (2011). Problems Associated with a Lack of Cohesive Policy in K-12 Pre-college Engineering. Journal of Pre-College Engineering Education Research (J-PEER), 1(1), 5. https://doi.org/10.7771/2157-9288.1029

Chen, C., Schneps, M. H., & Sonnert, G. (2016). Order Matters: Sequencing Scale-Realistic Versus Simplified Models to Improve Science Learning. Journal of Science Education and Technology, 25(5), 806–823. https://doi.org/10.1007/s10956-016-9642-4

Cho, Y., & Serena, S. (2013). Predicting teachers ’ achievement goals for teaching : The role of perceived school goal structure and teachers ’ sense of efficacy. Teaching and Teacher Education, 32, 12–21. https://doi.org/10.1016/j.tate.2012.12.003

Christensen, L. B., Johnson, R. B., & Turner, L. A. (2011). Research methods, design and analysis (11th ed.). Boston: Pearson Education.

Chua Yan Piaw. (2011). Kaedah statistik dan kaedah penyelidikan (buku 1) (2nd ed.). Kuala Lumpur: McGraw Hill.

Chyung, S. Y. (Yonnie), Barkin, J. R., & Shamsy, J. A. (2018). Evidence-based survey design: the use of negatively worded items in surveys. Performance Improvement, 57(3), 16–25. https://doi.org/10.1002/pfi.21749

Clark, V. L. P., & Creswell, J. W. (2015). Understanding Research : A Consumer ’s Guide (2nd ed.). New Jersey: Pearson Education.

Cohen, R. J., & Swerdlik, M. (2010). Psychological Testing and Assessment. Psychological Testing and Assessment - An Introduction to Tests and Measurements (7th Editio).

Cohen, Ronald Jay, Swerdlik, M. E., & Sturman, E. D. (2009). Psychological testing and assessment: an introduction to tests and measurement (7th ed). New York: McGraw Hill.

Cohen, Ronald Jay, Swerdlik, M. E., & Sturman, E. D. (2013). Psychological testing and assessment: an introduction to tests and measurement (8th ed.). New York: McGraw Hill.

Conrad, K. M., Conrad, K. J., Dennis, M. L., & Funk, R. (2012). Validation of the Self Help Improvement Scale to the Rasch measurement model GAIN methods report 1.0. Chicago. Retrieved from

© COPYRIG

HT UPM

115

http://gaincc.org/_data/files/Psychometrics_and_Publications/Working_Papers/Conrad_et_al_2011_SPS_Report.pdf

Creswell, J. W. (2014). Research Design : Qualitative, Quantitative and Mixed Methods Approaches. (V. Knight, Ed.) (4th ed, Vol. 53). SAGE Publications Inc. https://doi.org/10.1017/CBO9781107415324.004

Crippen, K. J., & Archambault, L. (2012). Scaffolded inquiry-based instruction with technology: a signature pedagogy for STEM education. Computers in the Schools: Interdisciplinary Journal of Practice, Theory, and Applied Research, 29(1–2), 157–173. https://doi.org/10.1080/07380569.2012.658733

Curriculum Development Division. (2015). Dokumen standard kurikulum dan pentaksiran Sains Tingkatan 1. Putrajaya: Kementerian Pendidikan Malaysia.

Curriculum Development Division. (2016a). Buku Penerangan Kurikulum Standard Sekolah Menengah (KSSM). Putrajaya: Ministry of Education.

Curriculum Development Division. (2016b). Panduan pelaksanaan pengajaran dan pembelajaran berasaskan inkuiri. Putrajaya: Ministry of Education.

Curriculum Development Division. (2016c). Panduan pelaksanaan Sains, Teknologi, Kejuruteraan dan Matematik (STEM) dalam pengajaran dan pembelajaran. Putrajaya: Ministry of Education.

Curriculum Development Division. (2017). Bahan sumber STEM. Retrieved May 30, 2019, from http://bpk.moe.gov.my/index.php/inisiatif-pppm/stem/buku-sumber-stem

Dani, D. E., Hartman, S. L., & Helfrich, S. R. (2017). Learning to Teach Science: Elementary Teacher Candidates Facilitate Informal STEM Events. The New Educator, 00(00), 1–18. https://doi.org/10.1080/1547688X.2017.1356413

Dare, E., Ellis, J., & Roehrig, G. (2014). Driven by Beliefs: Understanding Challenges Physical Science Teachers Face When Integrating Engineering and Physics. Journal of Pre-College Engineering Education, 4(2), 47–61. https://doi.org/10.7771/2157-9288.1098

Darling-Hammond, L., Chung, R., & Frelow, F. (2002). Variation in Teacher Preparation: How Well Do Different Pathways Prepare Teachers to Teach? Journal of Teacher Education, 53(4), 286–302. https://doi.org/10.1177/0022487102053004002

Dawson, E. (2014). Equity in informal science education : developing an access and equity framework for science museums and science centres. Studies

© COPYRIG

HT UPM

116

in Science Education, 50(2), 209–247. https://doi.org/10.1080/03057267.2014.957558

De Klerk, M., Nel, J. A., Hill, C., & Koekemoer, E. (2013). The development of the MACE work-family enrichment instrument. SA Journal of Industrial Psychology, 39(2), 1–16. https://doi.org/10.4102/sajip.v39i2.1147

Dellinger, A. B., Bobbett, J. J., Olivier, D. F., & Ellett, C. D. (2008). Measuring teachers’ self-efficacy beliefs: Development and use of the TEBS-Self. Teaching and Teacher Education, 24(3), 751–766. https://doi.org/10.1016/j.tate.2007.02.010

DeVellis, R. . (2006). Classical test theory. Medical Care, 44(11), 50–59. https://doi.org/10.1007/978-94-6209-404-8

DeVellis, R. . (2017). Scale development : theory and applications (4th ed.). SAGE Publications Inc.

Dodeen, H., Abdelfattah, F., Shumrani, S., & Hilal, M. A. (2012). The Effects of Teachers ’ Qualifications , Practices , and Perceptions on Student Achievement in TIMSS Mathematics : A Comparison of Two Countries. International Journal of Testing, 12(1), 61–77. https://doi.org/10.1080/15305058.2011.621568

Dolnicar, S., Grun, B., Leisch, F., & Rossiter, J. (2011). Three good reasons NOT to use five and seven point Likert items. In CAUTHE 2011 : 21st CAUTHE National Conference (pp. 8–11). Adelaide.

Doustmohammadian, A., Omidvar, N., Keshavarz-Mohammadi, N., Abdollahi, M., Amini, M., & Eini-Zinab, H. (2017). Developing and validating a scale to measure Food and Nutrition Literacy (FNLIT) in elementary school children in Iran. PLoS ONE, 12(6), 1–18. https://doi.org/10.1371/journal.pone.0179196

Eakman, A. M. (2012). Measurement characteristics of the engagement in meaningful cctivities survey in an age-diverse sample. American Journal of Occupational Therapy, 66(2), 20–29.

Eidietis, L., & Jewkes, a. M. (2011). Making Curriculum Decisions in K-8 Science: The Relationship between Teacher Dispositions and Curriculum Content. Journal of Geoscience Education, 59(4), 242. https://doi.org/10.5408/1.3651406

Ejiwale, J. A. (2013). Barriers to Successful Implementation of STEM Education. Journal of Education and Learning, 7(2), 63–74.

Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists (1st ed.). New Jersey: Lawrence Erlbaum Associates, Inc.

© COPYRIG

HT UPM

117

Eng Tek Ong, Puspa Ramiah, Kenneth Ruthven, Sabri Mohd Salleh, Nik Azmah Nik Yusuff, & Siti Eshah Mokhsein. (2015). Acquisition of basic science process skills among malaysian upper primary students. Research in Education, 94(1), 88–101. https://doi.org/10.7227/RIE.0021

Epstein, J., Santo, R. M., & Guillemin, F. (2015). A review of guidelines for cross-cultural adaptation of questionnaires could not bring out a consensus. Journal of Clinical Epidemiology, 68(4), 435–441. https://doi.org/10.1016/j.jclinepi.2014.11.021

Evans, M. A., Won, S. G. L., & Drape, T. (2014). Interest-driven STEM learning among youth through a social networking site. Int. J. Social Media and Interactive Learning Environments, 2(1), 3–20.

Evers, A. T., Verboon, P., & Klaeijsen, A. (2017). The development and validation of a scale measuring teacher autonomous behaviour. British Educational Research Journal, 43(4), 805–821. https://doi.org/10.1002/berj.3291

Fayer, S., Lacey, A., & Watson, A. (2017). STEM Occupations : Past , Present , And Future.

Fazilah Razali, Othman Talib, Umi Kalthom Abdul Manaf, & Siti Aishah Hassan. (2018). Students attitude towards Science , Technology , Engineering and Mathematics in developing career aspiration. International Journal of Academic Research in Business and Social Sciences, 8(5), 962–976. https://doi.org/10.6007/IJARBSS/v8-i5/4242

Fiebelkorn, F., & Menzel, S. (2013). Student Teachers’ Understanding of the Terminology, Distribution, and Loss of Biodiversity: Perspectives from a Biodiversity Hotspot and an Industrialized Country. Research in Science Education, 43(4), 1593–1615. https://doi.org/10.1007/s11165-012-9323-0

Fisher, W. P. (2006). Survey design recommendations. Rasch Measurement Transactions, 20(3), 1072–1074.

Fisher, W. P. (2007). Rating Scale Instrument Quality Criteria. Rasch Measurement Transactions, 21(1), 1095. https://doi.org/10.1017/CBO9781107415324.004

Ford, C. (2018). Effective practice instructional strategies: Design of an instrument to assess teachers’ perception of implementation. Studies in Educational Evaluation, 56, 154–163. https://doi.org/10.1016/j.stueduc.2017.12.004

Fox, C. M., & Jones, J. A. (1998). Uses of Rasch modeling in counseling psychology research. Journal of Counseling Psychology, 45(1), 30–45. https://doi.org/10.1037/0022-0167.45.1.30

© COPYRIG

HT UPM

118

Friedman, I. A., & Kass, E. (2002). Teacher self-efficacy : a classroom-organization conceptualization. Teaching and Teacher Education, 18, 675–686.

Fulton, K., Doerr, H., & Britton, T. (2011). STEM Teachers in Professional Learning Communities. The National Commission on Teaching and America’s Future (NCTAF).

Greenbaum, T. L. (1993). The handbook for focus group research (2nd ed.). New York: Lexington Books.

Gregory, R. J. (2004). Psychological testing : history, principles and application (4th rd.). Illinois: Pearson.

Groth-Marnat, G. (2009). Handbook of psychological assesment (5th ed.). New Jersey: John Wiley & Sons, Inc.

Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2013). Multivariate Data Analysis. Vectors (7th ed.). Pearson Education Ltd. https://doi.org/10.1016/j.ijpharm.2011.02.019

Han, S., Capraro, R., & Capraro, M. M. (2015). How science, techonology, engineering, and mathematics (STEM) project-based learning (PBL) affects high, middle and low achievers differently : the impact of student factors on schievement. International Journal of Science and Mathematics Education, 13(5), 1089–1113. https://doi.org/10.1007/s10763-014-9526-0

Hansen, M., & Gonzalez, T. (2014). Investigating the Relationship between STEM Learning Principles and Student Achievement in Math and Science. American Journal of Education, 120(2), 139–171. https://doi.org/10.1086/674376

Hassan Abu Bakar, Walters, T., & Haslina Halim. (2014). Measuring communication styles in the Malaysian workplace: instrument development and validation. Journal of Intercultural Communication Research, 43(2), 87–112. https://doi.org/10.1080/17475759.2014.899508

Henry, G. T., Thompson, C. L., Bastian, K. C., Kershaw, D. C., Purtell, K. M., & Zulli, R. A. (2011). Does Teacher Preparation Affect Student Achievement? Education Finance and Policy. Retrieved from http://0-search.ebscohost.com.library.uark.edu/login.aspx?direct=true&db=a9h&AN=27639277&site=ehost-live&scope=site

Hinze, S. R., Rapp, D. N., Williamson, V. M., Shultz, M. J., Deslongchamps, G., & Williamson, K. C. (2013). Beyond ball-and-stick: Students’ processing of novel STEM visualizations. Learning and Instruction, 26, 12–21. https://doi.org/10.1016/j.learninstruc.2012.12.002

© COPYRIG

HT UPM

119

Hofstein, A., & Lunetta, V. N. (2004). The Laboratory in Science Education: Foundations for the Twenty-First Century. Science Education, 88(1), 28–54. https://doi.org/10.1002/sce.10106

Holzberger, D., Philipp, A., & Kunter, M. (2013). How teachers’ self-efficacy is related to instructional quality: A longitudinal analysis. Journal of Educational Psychology, 105(3), 774–786. https://doi.org/10.1037/a0032198

Honey, M., Pearson, G., & Schweingruber, H. (2014). STEM integration in K-12 education: status, prospects and an agenda for research. (C. on I. S. E. N. A. of E. N. R. Council & Visit, Eds.). Washington: National Academies Press. https://doi.org/10.17226/18612

Houseal, A. K., Abd-El-Khalick, F., & Destefano, L. (2014). Impact of a student-teacher-scientist partnership on students’ and teachers’ content knowledge, attitudes toward science, and pedagogical practices. Journal of Research in Science Teaching, 51(1), 84–115. https://doi.org/10.1002/tea.21126

Jabrayilov, R., Emons, W. H. M., & Sijtsma, K. (2016). Comparison of Classical Test Theory and Item Response Theory in Individual Change Assessment. Applied Psychological Measurement, 40(8), 559–572. https://doi.org/10.1177/0146621616664046

Johnson, M. L., & Sinatra, G. M. (2013). Use of task-value instructional inductions for facilitating engagement and conceptual change. Contemporary Educational Psychology, 38(1), 51–63. https://doi.org/10.1016/j.cedpsych.2012.09.003

Kamaleswaran Jayarajah, Rohaida Mohd Saat, & Rose Amnah Abdul Rauf. (2014). A Review of Science, Technology, Engineering & Mathematics (STEM) Education Research from 1999–2013: A Malaysian Perspective. EURASIA Journal of Mathematics, Science & Technology Education, 10(3), 155–163. https://doi.org/10.12973/eurasia.2014.1072a

Karsli, F., & Ayas, A. (2014). Developing a laboratory activity by using 5E learning model on student learning of factors affecting the reaction rate and improving scientific process skills. Procedia - Social and Behavioral Sciences, 143, 663–668. https://doi.org/10.1016/j.sbspro.2014.07.460

Katz, I., & Cohen, R. (2014). Assessing autonomous motivation in students with cognitive impairment. Journal of Intellectual and Developmental Disability, 39(4), 323–332. https://doi.org/10.3109/13668250.2014.934791

Kazempour, M. S. (2014). I can’t teach science! A case study of an elementary pre-service teacher’s intersection of science experiences, beliefs, attitude, and self-efficacy. International Journal of Environmental and Science Education, 9(1), 77–96. https://doi.org/10.12973/ijese.2014.204a

© COPYRIG

HT UPM

120

Kelley, T. R., Knowles, J. G., Abell, S., Lederman, N., Bandura, A., Bandura, A., … Williams, D. (2016). A conceptual framework for integrated STEM education. International Journal of STEM Education, 3(1), 11. https://doi.org/10.1186/s40594-016-0046-z

Kennedy, T. J., & Odell, M. R. L. (2014). Engaging Students In STEM Education. Science Education International, 25(3), 246–258.

Khaeroningtyas, N., Permanasari, A., & Hamidah, I. (2016). STEM learning in material of temperature and its change to improve scientific literacy of junior high school. Jurnal Pendidikan IPA Indonesia, 5(1), 94–100. https://doi.org/10.15294/jpii.v5i1.5797

Khairani, A. Z. (2017). Assessing Urban and Rural Teachers’ Competencies in STEM Integrated Education in Malaysia. MATEC Web of Conferences, 87, 04004. https://doi.org/10.1051/matecconf/20178704004

Khazaee-Pool, M., Pashaei, T., Koen, P., Jafari, F., & Alizadeh, R. (2017). Decisional Balance Inventory (DBI) Adolescent Form for Smoking: Psychometric Properties of the Persian Version. BMC Public Health, 17(1), 507. https://doi.org/10.1186/s12889-017-4425-2

Kim, J. (2011). Developing an instrument to measure social presence in distance higher education. British Journal of Educational Technology, 42(5), 763–777. https://doi.org/10.1111/j.1467-8535.2010.01107.x

Klassen, A., Creswell, J., Clark, P. V., Smith, K., & Meissner, H. I. (2012). Best practices in mixed methods for quality of life research. Quality of Life Research, 21, 377–380. https://doi.org/10.1007/s11136-012-0122-x

Klassen, A. F., Grant, C., Barr, R., Brill, H., Kraus de Camargo, O., Ronen, G. M., … Gorter, J. W. (2015). Development and validation of a generic scale for use in transition programmes to measure self-management skills in adolescents with chronic health conditions: The TRANSITION-Q. Child: Care, Health and Development, 41(4), 547–558. https://doi.org/10.1111/cch.12207

Koehler, A., Feldhaus, C. R., Fernandez, E., & Hundley, S. (2013). Alternative Certification Programs & Pre-Service Teacher Preparedness. Journal of STEM Education: Innovations and Research, 14(4), 45–55. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=EJ1017059&site=ehostlive%5Cnhttp://ojs.jstem.org/index.php?journal=JSTEM&page=article&op=view&path[]=1680

König, J., Ligtvoet, R., Klemenz, S., & Rothland, M. (2017). Effects of opportunities to learn in teacher preparation on future teachers’ general pedagogical knowledge: Analyzing program characteristics and outcomes. Studies in Educational Evaluation, 53(April), 122–133. https://doi.org/10.1016/j.stueduc.2017.03.001

© COPYRIG

HT UPM

121

Lawshe, C. H. (1975). A quantitative approach to content validity. Personnel Psychology, 28(4), 563–575. https://doi.org/10.1111/J.1744-6570.1975.TB01393.X

Lerdal, A., Opheim, R., Gay, C. L., Moum, B., Fagermoen, M. S., & Kottorp, A. (2017). Psychometric limitations of the 13-item Sense of Coherence Scale assessed by Rasch analysis. BMC Psychology, 5(1), 18. https://doi.org/10.1186/s40359-017-0187-y

Lichtenstein, G., Tombari, M. L., Sheppard, S. D., & Storm, K. (2014). Does teaching matter? factors that influence high school students ’ decisions whether to pursue college STEM majors. In American Society for Engineering Education (ASEE) Annual Conference. American Society for Engineering Education.

Liew Yon Foi. (2012). Development and Validation of Teacher Curriculum Paradigm and Actual Curriculum Development Practice Instruments for Malaysian Teacher Curriculum Paradigm Model. Universiti Putra Malaysia.

Lin, K.-Y. Y., & Williams, P. J. (2015). Taiwanese Preservice Teachers’ Science, Technology, Engineering, and Mathematics Teaching Intention. International Journal of Science and Mathematics Education, 14(6), 1021–1036. https://doi.org/10.1007/s10763-015-9645-2

Linacre, J. M. (1994). Sample Size and Item Calibration Stability. Rasch Measurement Transactions, 7(4), 328.

Linacre, J. M. (2002). Optimizing rating scale category effectiveness. Journal of Applied Measurement, 3(1), 85–106.

Linacre, J. M. (2004). Test validity and Rasch measurement : construct, content etc. Rasch Measurement Transactions, 18(1), 970–971.

Linacre, J. M. (2005a). A user’s guide to WINSTEPS: Rasch Model Computer Programs. Chicago: MESA Press.

Linacre, J. M. (2005b). Standard Errors : Means , Measures , Origins and Anchor Values. Rasch Measurement Transactions, 19(3), 1030.

Linacre, J. M. (2018). A user’s guide to WINSTEPS: Rasch Model Computer Programs.

Lindell, M. K., & Brandt, C. J. (1999). Assessing interrater agreement on the job relevance of a test: A comparison of CVI, T, r-sub(WG(J)), and r*-sub(WG(J)) indexes. Journal of Applied Psychology, 84(4), 640–647.

Llamas-Ramos, I., Llamas-Ramos, R., Buz, J., Cortés-Rodríguez, M., & Martín-Nogueras, A. M. (2018). Construct Validity of the Spanish Versions of the

© COPYRIG

HT UPM

122

Memorial Symptom Assessment Scale Short Form and Condensed Form: Rasch Analysis of Responses in Oncology Outpatients. Journal of Pain and Symptom Management, 55(6), 1480–1491. https://doi.org/10.1016/j.jpainsymman.2018.02.017

Lourenço, O. (2012). Piaget and Vygotsky: Many resemblances, and a crucial difference. New Ideas in Psychology, 30(3), 281–295. https://doi.org/10.1016/j.newideapsych.2011.12.006

Mahmud, R., Ismail, M. A. H., Rahman, F. A., Kamarudin, N., & Ruslan, A. R. (2012). Teachers’ Readiness in Utilizing Educational Portal Resources in Teaching and Learning. Procedia - Social and Behavioral Sciences, 64, 484–491. https://doi.org/10.1016/j.sbspro.2012.11.057

Malaysian Science and Technology Information Centre. (2014). Malaysian Science, Technology and Innovation (STI) Indicators Report 2013. Ministry of Science, Technology and Innovation. Putrajaya. Retrieved from http://mastic.mosti.gov.my/en/web/guest/27

Maulana, R., Helms-Lorenz, M., & van de Grift, W. (2015). Development and evaluation of a questionnaire measuring pre-service teachers’ teaching behaviour: a Rasch modelling approach. School Effectiveness and School Improvement, 26(2), 169–194. https://doi.org/10.1080/09243453.2014.939198

Maxim, G. W. (2006). Dynamic social studies for constructivist classrooms : inspiring tomorrow’s social scienctists (8th ed.). New Jersey: Pearson Education Inc.

Mazlini Adnan, Aminah Ayob, Tek, O. E., Mohd Nasir Ibrahim, Noriah Ishak, & Jameyah Sheriff. (2016). Memperkasa pembangunan modal insan Malaysia di peringkat kanak- kanak : Kajian kebolehlaksanaan dan kebolehintegrasian pendidikan STEM dalam kurikulum PERMATA Negara. GEOGRAFIA Online Malaysian Journal of Society and Space, 12(1), 29–36.

McRackan, T. R., Velozo, C. A., Holcomb, M. A., Camposeo, E. L., Hatch, J. L., Meyer, T. A., … Dubno, J. R. (2017). Use of adult patient focus groups to develop the initial item bank for a cochlear implant quality-of-life instrument. JAMA Otolaryngology - Head and Neck Surgery, 143(10), 975–982. https://doi.org/10.1001/jamaoto.2017.1182

Mellahi, K., & Harris, L. C. (2016). Response Rates in business and management research: an overview of current practice and suggestions for future direction. British Journal of Management, 27(2), 426–437. https://doi.org/10.1111/1467-8551.12154

Miles, J. R., Mallinckrodt, B., & Recabarren, D. A. (2016). Reconsidering Focus Groups and Rasch Model Item Response Theory in Instrument

© COPYRIG

HT UPM

123

Development. The Counseling Psychologist, 44(2), 226–236. https://doi.org/10.1177/0011000016631121

Miller, L. A., Lovler, R. L., & McIntire, S. A. (2013). Foundations of Psychological Testing : A Practice Approach (4th ed.). California: SAGE Publications Inc.

Miller, M. B. (1995). Coefficient alpha: a basic introduction from the perspectives of Classical Test Theory and Structural Equation Modeling. Structural Equation Modeling, 2(3), 255–273. https://doi.org/10.1080/10705519509540013

Ministry of Education. (2013). Preliminary Report : Malaysia Education Blueprint 2013 - 2025. Kementerian Pendidikan Malaysia (MOE). https://doi.org/10.1016/j.tate.2010.08.007

Ministry of Education. (2016). 2015 Annual Report Malaysian Education Blueprint Report (2013-2015). Kementerian Pendidikan Malaysia (MOE). https://doi.org/10.1017/CBO9781107415324.004

Ministry of Education. (2017). Malaysia Education Blueprint 2013-2025 : 2016 Annual Report.

Ministry of Education Malaysia. (2013). Malaysia Education Blueprint 2013 - 2025 : Preschool to post-secondary education. Ministry of Education Malaysia. Putrajaya. https://doi.org/10.1016/j.tate.2010.08.007

Mobley, M. C. (2015). Development of the SETIS Instrument to Measure Teachers’ Self-Efficacy to Teach Science in an Integrated STEM Framework. University of Tennessee.

Moeini, S., V. Rasmussen, J., Wirenfelt Klausen, T., & Brorson, S. (2016). Rasch analysis of the Western Ontario Osteoarthritis of the Shoulder index – the Danish version. Patient Related Outcome Measures, Volume 7, 173–181. https://doi.org/10.2147/prom.s87048

Mohammad Rahim Kamaluddin, & Rohany Nasir. (2017). Kesahan kandungan dan muka. In Mohammad Rahim Kamaluddin & Wan Shahrazad Wan Sulaiman (Eds.), Teknik kesahan dan kebolehpercayaan alat uji psikologi (pp. 39–50). Bangi: Penerbit Universiti Kebangsaan Malaysia.

Mohd. Hamzah, M. I., & Attan, N. (2007). Tahap Kesediaan Guru Sains Dalam Penggunaan Teknologi Maklumat Tahap Kesediaan Guru Sains Dalam Penggunaan Teknologi Maklumat Berasaskan Komputer Dalam Proses Pengajaran Dan Pembelajaran. Jurnal Teknologi, 46, 45–60.

Mohd Affandi, H., Khairil Hizar, M. ., Amiruddin, M. ., & Fadzil, H. (2017). Establishing the Malaysian Contractors ’ Soft Skills Requirements for Construction Management Graduates : Applying the Rasch. Journal of

© COPYRIG

HT UPM

124

Technical Education and Training, 9(1), 61–72.

Mohd Effendi Ewan Mohd Matore. (2015). Pembinaan Instrumen Kecerdasan Menghadapi Cabaran (IKBAR) bagi Pelajar Politeknik Menggunakan Model RASCH. Universiti Sains Malaysia.

Mohd Effendi Mohd Matore, & Ahmad Zamri Khairani. (2015). Assessing the Content Validity of IKBAR using Content Validity Ratio. Australian Journal of Basic and Applied Sciences, 9(7), 255–257.

Mokkink, L. B., Terwee, C. B., Patrick, D. L., Alonso, J., Stratford, P. W., Knol, D. L., … de Vet, H. C. W. (2010). The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. Journal of Clinical Epidemiology, 63(7), 737–745. https://doi.org/10.1016/j.jclinepi.2010.02.006

Morgado, F. F. R., Meireles, J. F. F., Neves, C. M., Amaral, A. C. S., & Ferreira, M. E. C. (2017). Scale development: Ten main limitations and recommendations to improve future research practices. Psicologia: Reflexao e Critica, 30(3), 1–20. https://doi.org/10.1186/s41155-016-0057-1

Mousazadeh, S., Rakhshan, M., & Mohammadi, F. (2017). Investigation of content and face validity and reliability of sociocultural attitude towards appearance questionnaire-3 (SATAQ-3) among female adolescents. Iranian Journal of Psychiatry, 12(1), 15–20.

Murphy, K. R., & Davidshofer, C. O. (2005). Psychological testing : principles and applications (6th ed.). New Jersey: Prentice Hall.

Nadelson, L. S., Callahan, J., Pyke, P., Hay, A., Dance, M., & Pfiester, J. (2013). Teacher STEM perception and preparation: inquiry-based STEM professional development for elementary teachers. The Journal of Educational Research, 106(2), 157–168. https://doi.org/10.1080/00220671.2012.667014

Nadelson, L. S., Seifert, A., Moll, A. J., & Coats, B. (2012). i-STEM summer institute : an integrated approach to teacher professional development in STEM. Journal of STEM Education, 13(2), 69–84.

Nadelson, L. S., Seifert, A., Moll, A. J., Coats, B., & Nadelson, Louis S; Seifert, Anne; Moll, Amy J; Coats, B. (2012). i-STEM summer institute : an integrated approach to teacher professional development in STEM. Journal of STEM Education, 13(2), 69–84.

Nam, S. K., Yang, E., Lee, S. M., Lee, S. H., & Seol, H. (2011). A psychometric evaluation of the career decision self-efficacy scale with korean students: a Rasch model approach. Journal of Career Development, 38(2), 147–166.

© COPYRIG

HT UPM

125

https://doi.org/10.1177/0894845310371374

Nasarudin Abdullah, Lilia Halim, & Effandi Zakaria. (2014). VStops: A thinking strategy and visual representation approach in mathematical word problem solving toward enhancing STEM literacy. Eurasia Journal of Mathematics, Science and Technology Education, 10(3), 165–174. https://doi.org/10.12973/eurasia.2014.1073a

Nassar-McMillan, S. C., Wyer, M., Oliver-Hoyo, M., & Ryder-Burge, A. (2010). Using focus groups in preliminary instrument development expected and unexpected lessons learned. The Qualitative Report, 15(6), 1621–1634.

Nazlinda Abdullah, Shereen Noranee, & Mohd Rahim Khamis. (2017). The Use of Rasch Wright Map in Assessing Conceptual Understanding of Electricity. Pertanika Journal of Social Sciences & Humanities, 25, 81–92.

Ng, C. H., & Adnan, M. (2018). Integrating STEM education through Project-Based Inquiry Learning (PIL) in topic space among year one pupils. In IOP Conference Series: Materials Science and Engineering (Vol. 296). https://doi.org/10.1088/1757-899X/296/1/012020

Nicholls, G. M., Wolfe, H., Besterfield-Sacre, M., Shuman, L. J., & Larpkiattaworn, S. (2007). A Method for Identifying Variables for Predicting STEM Enrollment. Journal of Engineering Education, 96(1), 33–44. https://doi.org/10.1002/j.2168-9830.2007.tb00913.x

Norashady Mohd Noor, Muhammad Aziz Shah Mohamed Arip, Eftah Moh, & Haniza Yon. (2016). Assessing the content validity of PERJURA using content validity ratio. Journal of Advanced Research Design, 3(1), 13–19.

Nornazira Suhairom, Aede Hatib Musta’amal, Nor Fadila Mohd Amin, & Noor Khairul AnuarJohari. (2014). The development of competency model and instrument for competency measurement : The research methods. Procedia - Social and Behavioral Sciences, 152, 1300–1308. https://doi.org/10.1016/j.sbspro.2014.09.367

Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York: McGraw Hill.

Nur Farhana Ramli, & Othman Talib. (2017). Can education institution implement STEM ? From Malaysian teachers ’ view. International Journal of Academic Research in Business and Social Sciences, 7(3), 721–732. https://doi.org/10.6007/IJARBSS/v7-i3/2772

O’Neill, S., & Stephenson, J. (2013). One year on : first-year primary teachers ’ perceptions of preparedness to manage misbehaviour and their confidence in the strategies they use. Australasian Journal OfSpecial Education, 37(2), 125–146. https://doi.org/10.1017/jse.2013.15

© COPYRIG

HT UPM

126

Peng, C., Cao, L., & Timalsena, S. (2016). Gamification of Apollo Lunar exploration missions for learning engagement. Entertainment Computing. https://doi.org/10.1016/j.entcom.2016.12.001

Petrillo, J., Cano, S. J., McLeod, L. D., & Coon, C. D. (2015). Using classical test theory, item response theory, and rasch measurement theory to evaluate patient-reported outcome measures: A comparison of worked examples. Value in Health, 18(1), 25–34. https://doi.org/10.1016/j.jval.2014.10.005

Plonczak, I., Brooks, J. G., Lodato, G., Elijah, R., & Caliendo, J. (2014). STEM Studio : where innovation generates innovation. Kappan, 95(February).

Powell, C. (2003). The Delphi Technique: myths and realities. Methodological Issues in Nursing Research, 41(4), 376–382. https://doi.org/10.1046/j.1365-2648.2003.02537.x

Price, J. (2010). The effect of instructor race and gender on student persistence in STEM fields. Economics of Education Review, 29(6), 901–910. https://doi.org/10.1016/j.econedurev.2010.07.009

Public Servises Commision of Malaysia. (2019). Pengesahan dalam perkhidmatan. Retrieved February 24, 2019, from http://www.spa.gov.my/web/guest/pengesahan-dalam-perkhidmatan

Raes, A., Schellens, T., De Wever, B., & Vanderhoven, E. (2012). Scaffolding information problem solving in web-based collaborative inquiry learning. Computers and Education, 59(1), 82–94. https://doi.org/10.1016/j.compedu.2011.11.010

Raykov, T., & Marcoulides, G. (2011). Introduction to psychometric theory (1st ed.). Routledge.

Rehmat, A. P., & Bailey, J. M. (2014). Technology Integration in a Science Classroom: Preservice Teachers’ Perceptions. Journal of Science Education and Technology, 23(6), 744–755. https://doi.org/10.1007/s10956-014-9507-7

Reigeluth, C. M. (1999). What is Instructional-Design Theory and How Is It Changing? In C. M. Reigeluth (Ed.), Instructional-Design Theories and Models Volume II : A New Paradigm of Instructional Theory (1st ed., pp. 5–30). Routledge.

Rohani Arbaa, Hazri Jamil, & Nordin Abd Razak. (2010). Hubungan Guru-Pelajar dan Kaitannya dengan Komitmen Belajar Pelajar: Adakah Guru Berkualiti Menghasilkan Perbezaan Pembelajaran antara Jantina Pelajar? Jurnal Pendidikan Malaysia, 35(2), 61–69.

© COPYRIG

HT UPM

127

Roofe, C. G., & Mille, P. (2013). “Miss,I am not being fully prepared”: Student -teachers’ concerns about their preparation at a teacher training institution in Jamaica. Australian Journal of Teacher Education, 38(5), 1–14. https://doi.org/10.14221/ajte.2013v38n5.5

Rubio, D. M., Berg-Weger, M., Tebb, S. S., Lee, E. S., & Rauch, S. (2003). Objectifying content validity: Conducting a content validity study in social work research. Social Work Research, 27(2), 94–104. https://doi.org/10.1093/swr/27.2.94

Schelly, C., Anzalone, G., Wijnen, B., & Pearce, J. M. (2015). Open-source 3-D printing technologies for education: Bringing additive manufacturing to the classroom. Journal of Visual Languages and Computing, 28, 226–237. https://doi.org/10.1016/j.jvlc.2015.01.004

Schnittka, C., & Bell, R. (2011). Engineering design and conceptual change in science: Addressing thermal energy and heat transfer in eighth grade. International Journal of Science Education, 33(13), 1861–1887. https://doi.org/10.1080/09500693.2010.529177

Seligman, M. E. ., & Hager, J. L. (1972). Biological boundaries of Learning. New Jersey: Prentice-Hill.

Shabani, K., Khatib, M., & Ebadi, S. (2010). Vygotsky “s zone of proximal development : instructional implications and teachers” professional development. English Language Teaching, 3(4), 237–248. https://doi.org/10.5539/elt.v3n4p237

Shamsudin, N. M., Abdullah, N., & Yaamat, N. (2013). Strategies of Teaching Science Using an Inquiry based Science Education (IBSE) by Novice Chemistry Teachers. Procedia - Social and Behavioral Sciences, 90(InCULT 2012), 583–592. https://doi.org/10.1016/j.sbspro.2013.07.129

Sharkness, J., & Deangelo, L. (2011). Measuring student involvement : a comparison of classical test theory and item response theory in the construction of scales from student surveys. Research in Higher Education, 52(5), 480–507. https://doi.org/10.1007/s11162-010-9202-3

Shih, J., Shih, B., Shih, C., Su, H., & Chuang, C. (2010). The influence of collaboration styles to children’s cognitive performance in digital problem-solving game “ William Adventure ” : A comparative case study. Computers & Education, 55, 982–993. https://doi.org/10.1016/j.compedu.2010.04.009

Sias, C. M., Nadelson, L. S., Juth, S. M., & Seifert, A. L. (2017). The best laid plans: Educational innovation in elementary teacher generated integrated STEM lesson plans. Journal of Educational Research, 110(3), 227–238. https://doi.org/10.1080/00220671.2016.1253539

© COPYRIG

HT UPM

128

Siew, N. M., Nazri Amir, & Chong, C. L. (2015). The perceptions of pre-service and in-service teachers regarding a project-based STEM approach to teaching science. SpringerPlus, 4(1), 1–20. https://doi.org/10.1186/2193-1801-4-8

Singer, J. E., Ross, J. M., Jackson-lee, Y., Singer, J. E., Ross, J. M., & Jackson-lee, Y. (2016). Professional Development for the Integration of Engineering in High School STEM Classrooms, 6(1).

Siti Rahayah Ariffin. (2003). Teori, konsep & amalan dalam pengukuran dan penilaian. Bangi: Bahagian Latihan Akademik ; Pusat Pembangunan Akademik.

Siti Rahayah Ariffin. (2008). Inovasi dalam pengukuran dan penilaian pendidikan. Bangi: Universiti Kebangsaan Malaysia.

Siti Rahayah Ariffin. (2013). Ke arah kesaksamaan penilaian pendidikan : amalan dan cabaran (1st ed.). Bangi: Penerbit Universiti Kebangsaan Malaysia.

Slavit, D., Nelson, T. H., & Lesseig, K. (2016). The Teachers’ Role in Developing, Opening, and Nurturing an Inclusive STEM-focused School. International Journal of STEM Education, 3(1), 7. https://doi.org/10.1186/s40594-016-0040-5

Stangor, C. (2015). Research methods for the behavioral sciences (5th ed.). Stamford: Cencage Learning.

Steele, A., Brew, C., Rees, C., & Ibrahim-Khan, S. (2013). Our Practice, Their Readiness: Teacher Educators Collaborate to Explore and Improve Preservice Teacher Readiness for Science and Math Instruction. Journal of Science Teacher Education, 24(1), 111–131. https://doi.org/10.1007/s10972-012-9311-2

Stewart, D. W., Shamdasani, P. N., & Rook, D. W. (2007). Focus groups : theory and practice (2nd ed.). California: SAGE Publications Inc.

Suhanna Zainudin, Lilia Halim, & Zanaton Ikhsan. (2015). How 60 : 40 Policy Affect The Development Of Science Curriculum in Malaysia. In Proceeding: 7th International Seminar on Regional Education, November 5-7, 2015 (Vol. 3, pp. 1396–1405).

Sumintono, B., & Widhiarso, W. (2014). Aplikasi model Rasch untuk penelitian ilmu-ilmu sosial. (B. Trim, Ed.) (1st ed.). Cimahi: Trim Komunikata Publishing House.

Sumintono, B., & Widhiarso, W. (2015). Aplikasi pemodelan Rasch pada assessment pendidikan. (B. Trim, Ed.) (1st ed.). Cimahi: Penerbit Trim

© COPYRIG

HT UPM

129

Komunikata.

Sun, B., Shao, Y. T., Richardson, M. J., Weng, Y., & Shen, J. (2017). The caring behaviour of primary and middle school teachers in China: features and structure. Educational Psychology, 37(3), 362–379. https://doi.org/10.1080/01443410.2016.1214688

Sun, X., Mainhard, T., & Wubbels, T. (2017). Development and evaluation of a Chinese version of the Questionnaire on Teacher Interaction (QTI). Learning Environments Research, 1–17. https://doi.org/10.1007/s10984-017-9243-z

Tajularipin Sulaiman, Aminudin Hassan, & Roselan Baki. (2009). Readiness of year 1 students to learn science process skills in english: a malaysian experience. International Journal of Instruction, 2(1). Retrieved from https://files.eric.ed.gov/fulltext/ED524161.pdf

Tavakol, M., & Dennick, R. (2011). Making sense of Cronbach’s alpha. International Journal of Medical Education, 2, 53–55. https://doi.org/10.5116/ijme.4dfb.8dfd

Tengku Faekah Tengku Arifin, Bush, T., & Hasniza Nordin. (2018). Framing the roles and responsibilities of Excellent Teachers: Evidence from Malaysia. Teaching and Teacher Education, 73, 14–23. https://doi.org/10.1016/j.tate.2018.03.005

Thomas, A. S., Bonner, S. M., Everson, H. T., & Somers, J. A. (2015). Leveraging the power of peer-led learning: investigating effects on STEM performance in urban high schools. Educational Research and Evaluation, 21(7–8), 537–557. https://doi.org/10.1080/13803611.2016.1158657

Thomas, B., & Watters, J. J. (2015). Perspectives on Australian, Indian and Malaysian approaches to STEM education. International Journal of Educational Development, 45, 42–53. https://doi.org/10.1016/j.ijedudev.2015.08.002

Tinsley, H. E. A., & Weiss, D. J. (1975). Interrater Reliability and Agreement of Subjective Judgments. Journal of Counseling Psychology, 22(4), 358–376.

Tschannen-Moran, M., Hoy, A. W., & Hoy, W. K. (1998). Teacher ffficacy: its meaning and measure. Review of Educational Research, 68(2), 202–248. https://doi.org/10.3102/00346543068002202

U.S. Information Agency. (1959). Impact of US and Soviet space programs on world opinion. Kansas.

Vaingankar, J. A., Subramaniam, M., Abdin, E., Picco, L., Chua, B. Y., Eng, G. K., … Chong, S. A. (2014). Development, validity and reliability of the short

© COPYRIG

HT UPM

130

multidimensional positive mental health instrument. Quality of Life Research, 23(5), 1459–1477. https://doi.org/10.1007/s11136-013-0589-0

Van Ornum, W., Dunlap, L. L., & Shore, M. F. (2008). Psychological testing across the life span (1st ed.). New Jersey: Pearson Education Inc.

van Tuijl, C., & van der Molen, J. H. W. (2016). Study choice and career development in STEM fields: an overview and integration of the research. International Journal of Technology and Design Education, 26(2), 159–183. https://doi.org/10.1007/s10798-015-9308-1

Walsh, L., & Ross, A. B. (2015). The Visual Invention Practices of STEM Researchers: An Exploratory Topology. Science Communication, 37(1), 118–139. https://doi.org/10.1177/1075547014566990

Wang, J., Werner-Avidon, M., Newton, L., Randol, S., Smith, B., & Walker, G. (2013). Ingenuity in Action: Connecting Tinkering to Engineering Design Processes. Journal of Pre-College Engineering Education Research, 3(1), 1–21. https://doi.org/http://10.7771/2157-9288.1077

Wang, R., Hempton, B., Dugan, J. P., & Komives, S. R. (2008). Cultural Differences: Why Do Asians Avoid Extreme Responses? Survey Practice, 1(3), 1–7. https://doi.org/10.29115/SP-2008-0011

Wesolowski, B. C., Amend, R. M., Barnstead, T. S., Edwards, A. S., Everhart, M., Goins, Q. R., … Williams, J. D. (2017). The development of a secondary-level solo wind instrument performance rubric using the multifaceted Rasch partial credit measurement model. Journal of Research in Music Education, 65(1), 95–119. https://doi.org/10.1177/0022429417694873

Wilson, F. R., Wei Pan, & Schumsky, D. A. (2012). Recalculation of the Critical Values for Lawshe’s Content Validity Ratio. Measurement and Evaluation in Counseling and Development, 45(3), 197–210. https://doi.org/10.1177/0748175612440286

Winne, P. H. (1982). Minimizing the black box problem to enhance the validity of theories about instructional effects. Instructional Science, 11, 13–28.

Wood, D., Bruner, J. S., & Ross, G. (1976). the Role of Tutoring in Problem Solving. Journal of Child Psychology and Psychiatry, 17(2), 89–100. https://doi.org/10.1111/j.1469-7610.1976.tb00381.x

Wright, B. L. (2011). Valuing the “everyday” practices of African American students K-12 and their engagement in STEM learning : a position. Journal of Negro Education, 80, 5–11. https://doi.org/10.2307/2293006

© COPYRIG

HT UPM

131

Wright, Benjamin D., & Master, G. N. (1982). Rating scale analysis. Chicago: MESA Press.

Wright, Benjamin Drake, & Masters, G. N. (2002). Number of person or item strata. Rasch Measurement Transactions, 16(3), 888.

Yoon Yoon, S., Evans, M. G., & Strobel, J. (2014). Validation of the teaching engineering self-efficacy scale for K-12 teachers: A structural equation modeling approach. Journal of Engineering Education, 103(3), 463–485. https://doi.org/10.1002/jee.20049

Zainudin Awang. (2012). Research Methodology and Data Analysis (2nd ed). Shah Alam: Universiti Teknologi Mara Publication Centre (UiTM Press).

Zainudin Awang. (2015a). Postgraduate research proposal: a step by step guide in writing proposal for postgraduate students (1st ed). Bangi: MPWS Rich Publication Sdn Bhd.

Zainudin Awang. (2015b). SEM made simple: a gentle approach to learning Structural Equation Modeling. Bangi: MPWS Rich Publication.

Zainudin Awang, Asyraf Afthanorhan, & Mustafa Mamat. (2016). The Likert scale analysis using parametric based Structural Equation Modeling ( SEM ). Computational Methods in Social Sciences, 4(1), 13–21.

Zakiah Binti Nordin. (2016). Pembinaan dan Pengesahan Instrumen Literasi Pentaksiran Pelajar Ijazah Sarjana Muda Perguruan.

Zamanzadeh, V., Ghahramanian, A., Rassouli, M., Abbaszadeh, A., Alavi-Majd, H., & Nikanfar, A.-R. (2015). Design and Implementation Content Validity Study: Development of an instrument for measuring Patient-Centered Communication. Journal of Caring Sciences, 4(2), 165–178. https://doi.org/10.15171/jcs.2015.017

Zhu, J., Hu, Y., Liu, Q., & Cox, M. F. (2015). Validation of an instrument for chinese Engineering students’ epistemological development: application of Perry’s Theory. International Journal of Chinese Education, 4(2), 135–161. https://doi.org/10.1163/22125868-12340050

Zile-tamsen, C. Van. (2017). Using Rasch Analysis to Inform Rating Scale. Research in Higher Education, 58(8), 922–933. https://doi.org/10.1007/s11162-017-9448-0

© COPYRIG

HT UPM

194

8 BIODATA OF STUDENT

Nur Farhana binti Ramli was born on the 3rd October 1984 in Muar, Johor. She received her primary education at SK Gelang Patah, Johor and continue her secondary education to three different schools, SMK (P) Sultan Ibrahim Johor Bahru, SMK Kompleks Sultan Abu Bakar Tanjung Kupang and Sekolah Menengah Sains Muar.

In 2003, she furthered her undergraduate studies at Universit Kebangsaan Malaysia for the Honour Degree in Bachelor of Science (Chemistry). Because of her love in the education, she earned her Education Diploma at Insitut Perguruan Guru Perempuan Melayu Melaka in 2009. She started his profession as a teacher in Lawas, Sarawak. During her time in Sarawak, she had taught at two different schools, namely SK Kampung Lintang and SK Agama MIS Lawas.

She then continued her service at Sekolah Dalam Hospital (School in Hospital) to educate student with health problem. Previously she had served at Institut Pediatrik Hospital Kuala Lumpur dan Hospital Serdang. During her service span, she was actively involved with the curriculum development especially in STEM education. To date, she has built two STEM modules and has helped in the development of the STEM School concept under the Ministry of Education.

She furthered her master of education in Educational Administration at Universiti Putra Malaysia in 2011 with part time mode. She then continued her doctoral studies in Curriculum and Instruction in 2016 also in part time mode along with the essential task as a teacher in Sekolah Dalam Hospital Pusat Perubatan Universiti Kebangsaan Malaysia.

© COPYRIG

HT UPM

195

9 LIST OF PUBLICATIONS

Journals Nur Farhana Ramli, & Othman Talib. (2017). Can Education Institution

Implement STEM ? From Malaysian Teachers ’ View. International Journal of Academic Research in Business and Social Sciences, 7(3), 721–732. https://doi.org/10.6007/IJARBSS/v7-i3/2772

Nur Farhana Ramli, Othman Talib, Siti Aishah Hassan, & Umi Kalthom Abdul Manaf. (2018a). Content Validity of STEMTIP Using CVR Method. International Journal of Academic Research in Business and Social Sciences, 8(7), 1118–1125. https://doi.org/10.6007/IJARBSS/v8-i7/4559

Nur Farhana Ramli, Othman Talib, Siti Aishah Hassan, & Umi Kalthom Abdul Manaf. (2018b). Rasch Analysis and Differential Item Functioning of STEM Teachers’ Instructional Preparedness Instrument for Urban and Rural Teachers. International Journal of Academic Research in Progressive Education and Development, 7(4), 211–222. https://doi.org/10.6007/IJARPED/v7-i4/4848

Conference Nur Farhana Ramli, Othman Talib, Umi Kalthom Abdul Manaf, & Siti Aishah

Hassan. (2017). Instructional Approaches and Challenges of STEM Instructional Implementation: A Systematic Review. In Graduate Research in Education Seminar (GREduc 17) (pp. 171–179).

List of Awards Gold Awards – International Research and Innovation Symposium and Exposition 2018 (RISE 2018), Universiti Tun Hussein Onn Malaysia

Gold Awards – IDE4 Teaching and Learning Exhibition (IDE4 2018), Universiti Petronas Malaysia

Silver Awards – International Innovative Practices in Education Expo 2018 (I-PEX 2018), Universiti Teknologi Malaysia

© COPYRIG

HT UPM

196

Intellectual Property – Copyright

LY2018003749 Instrument Development Process of STEM Teachers’ Instructional Preparedness Instrument

LY2018003750 Framework of STEM Teachers’ Instructional Preparedness Instrument

LY2018005292 STEM Teachers’ Instructional Preparedness Instrument

© COPYRIG

HT UPM