CONDUCTING AND EXAMINING EFFECTIVENESS OF ONLINE ...worldconferences.net/ejournals/glit/wp-content/... · targeted that by the year 2000, sixty percent of examination papers should

34 http://worldconferences.net/ejournals/glit/

GLIT E-Journal on Information Technology and Language Practice (e-ISSN: 2289-7445) Volume 2, Issue 2

CONDUCTING AND EXAMINING EFFECTIVENESS OF ONLINE

ASSESSMENTS USING THE MODULAR OBJECT-ORIENTED

DYNAMIC LEARNING ENVIRONMENT (MOODLE)

Rosyati Abdul Rashid & Wan Nurhafeza Wan Salam English Language Learning Centre (ELC)

Center for Foundation and Liberal Education

Universiti Malaysia Terengganu (UMT)

[email protected]

[email protected]

ABSTRACT

This paper aims to demonstrate the robustness of Moodle, a type of Learning Management System

(LMS) in assisting language instructors to administer and analyze the quality of test items, and the test

as a whole. Using Moodle Quiz Module, a public speaking test comprising 40 multiple-choice

questions (MCQ) was created, and administered online to 131 students. Descriptive statistics obtained

through the statistical analysis procedures performed by Moodle indicated that the test was reliable,

and was a difficult one for many students since many of the test items were of high levels of Bloom‟s

taxonomy of educational objectives. The test scores obtained by 58 students who also took the Cornell

Critical Thinking Test (CCTT), Level X, were found to be significantly correlated, at a moderate level,

with the scores that the students had in the public speaking test. Data generated by Moodle such as the

data from the analysis of students‟ responses to each test item, and the data provided by items statistics

like item facility (IF), item discrimination (ID), and distractor efficiency (DE) indices, were found to

be very useful in helping the test developers to examine the effectiveness of test items and to later

improve them so that only good items can be kept in the test bank for future use in assessing a

different group of students.

Keywords: Moodle, critical thinking, MCQ, item analysis, IF, DI, and DE indices

1. Introduction

To sustain the ability to compete in the present twenty-first century, Malaysia has to produce

knowledgeable and skilled workforce who are critical in thinking and be able to function well in a

variety of work conditions. Realizing that there is an urgent need to generate the desired Malaysian

citizens, the Malaysian government has recently launched the Malaysia Education Blueprint 2013-

2015, which lists eleven shifts required to transform the Malaysian education system. One of the shifts

is to "provide equal access to quality education of an international standard". Among the listed actions

to be taken to realize this change is the act of revamping examination questions, so as to allow a

greater percentage of questions at higher- order thinking levels to be included in school and all

national examinations by the year 2016. This newly revised education policy initiative is hoped to

assist the nation successfully in generating a high-quality workforce that will help to steer the country

towards greater prosperity. However, to the skeptics, it seems questionable that such reform in school

and national assessments can be implemented successfully within the target time considering the

rather short time frame given to actualize the change and the current somewhat less satisfactory

teachers' literacy assessment level.

mailto:[email protected]

mailto:[email protected]



2. Literature review

2.1 Educational reforms in Malaysia and teachers’ assessment literacy

Many reform efforts have been taken by the Malaysian government to improve the national education

system (Rajendran, 2001). The first change made to the educational policy was based on the Razak

Report prepared before the country gaining its independence in 1957 and the most recent one is

written in the Malaysia Education Blueprint 2013-2025. One of the outcomes of the reform efforts was

the inclusion of critical thinking or higher-order thinking skills in the objective of the secondary

school curriculum (Curriculum Development Center, 1989, p.2), in the Integrated Curriculum for

Secondary Schools (ICSS), in the National Higher Education Action Plan which stressed that the focus

of the education system was “on generating world-class and holistic human capital who are

intellectually active, creative and innovative, ethically and morally upright, adaptable and capable of

critical thinking" (MOHE, 2007, p.8), and in the latest educational blueprint which states that “the

national curriculum aims to create Malaysian students that are balanced, resilient, inquisitive,

principled, informed, caring, patriotic, as well as effective thinker, communicator, and team player”

(Malaysia Education Blueprint 2013-2025, exhibit.4-2).

Although producing students with critical thinking ability has been considered as one of the

educational objectives for quite sometimes already, the question as to whether the goal is achievable

has frequently been raised. For example, Indramalar in 1997 (as cited in Rajendran, 2001, p.3)

reported the Minister of Education saying that "the education system will be revamped to encourage

rational and analytical thinking." The statement implies that the education system at that particular

time failed to realize the educational objective of producing critical thinkers. One of the measures

taken then to address the issue was to revamp public examination papers. A policy announced in 1994

targeted that by the year 2000, sixty percent of examination papers should comprise questions of the

higher-order thinking type.

The issue seems to have recurred recently when the analysis done by Pearson Education Group of the

UPSR and SPM English papers for the year of 2010 and 2011 revealed that about seventy percent of

the questions in the aforementioned national examination papers assessed "basic skills of knowledge

and comprehension"– the two low levels of cognitive domains in the Bloom's taxonomy (Malaysia

Education Blueprint, 2013-2025). To a certain extent, this finding had influenced the recent decision

of the Ministry of Education to include revamping examination questions as one of the main steps to

be taken in realizing the first type of transformation or shift to be made to the education system in this

21st century.

What went wrong? Perhaps Malaysian teachers are not yet fully prepared to assess students or they

have not adequately been given training on assessment. According to Lim, Wun, and Chew (2014),

Malaysian teachers do not have sufficient knowledge and skills to come up with assessments that can

measure the intended learning outcomes. Lim and his co-researchers made this conclusion based on

the findings of three local studies cited in their article. For example, they cited a recent survey

conducted by Suah (2012) involving 3,866 primary and secondary school teachers which presented

evidence showing that majority of the teachers had less satisfactory literacy assessment level. The

conclusion was further supported by two more studies that they cited. These studies, conducted by

Mohamad (2006) and Salbiah (1995), revealed that teachers' emphasis was more on the content of the

syllabus than the target learning outcomes whenever they were developing and carrying out

assessments.

It seems that much is said about teachers‟ low assessment literacy level and the poor quality of

examination papers, but little has been written about how to improve the standard so that the quality of

the exam papers can be enhanced. Experience and research have shown us that giving training alone

will not solve the problem that we are encountering now. In fact, to date, the Malaysian Ministry of

Education has supported teachers with various kinds of training and workshops including those on

assessments and the teaching of thinking skills (Rajendran, 2001; Lim, Wun, & Chew, 2014).



Nevertheless, majority of teachers still perceived themselves not prepared for teaching the higher-

order thinking skills and claimed themselves not capable of coming up with examination questions to

test such skills (Rajendran, 2001). Perhaps what these teachers need is ample of opportunities to

practice creating and carrying out their assessment plus the chance to improve them.

2.2 Online assessment

A plausible solution to the issue being addressed so far is to train and motivate teachers to conduct the

online assessment. This type of assessment requires teachers to assess their students' performance

through an e-learning platform. This proposed solution seems to be a practical one because most

schools and universities are equipped with computer laboratories. Online assessment allows teachers

and any educators systematically to attest their teaching practice and test development. It has been

shown to improve teachers' teaching and learning method; teachers are not only able to continuously

evaluate their teachings but also concurrently, gauge their students' academic performance through

online assessment.

2.2.1 Learning Management System (LMS): Moodle

Learning Management System (LMS) is defined as “software that has been used in a learning content

presentation which has a significant role and complexity in e-learning environment” (Aydin and

Tirkes, 2010, p.2). LMS is an e-learning platform utilized to support the move to improve educational

quality and prepare a wider range of options. Concerning assessment purpose, LMS provides a

different range of testing and evaluation format, thus, offers flexibility in grading information.

Modular Object-Oriented Dynamic Learning Environment (Moodle) is one type of LMS popularly

used today. Moodle, with its modular design, can support a variety different kinds of assessment

which allows test developers to manipulate some test constraints such time, date and test duration

(Aydin & Tirkes, 2010). It provides an advanced online module in which questions can be created in

different formats, such as multiple-choice questions, matching questions, and short answer questions.

Also, educators can prepare their online assessments based on their specified learning objectives.

Moodle is one of the top choices of learning institutions regarding robustness. Among the best features

of Moodle are its notable outstanding ability to assist educators to improve pedagogical quality (Aydin

& Tirkes, 2010), ease of maintenance or usability (Wright & Wright, 2011), and „reliability and

functionality‟ of teaching and learning content especially in regard to testing and assessment (Whelan

& Bhartu, 2007, p.1055; Costa, Alvelos & Teixeira, 2012).

3. Main Objective of the Study

The main objective of the present study was to demonstrate that online assessment can be carried out

using Moodle and that this LMS could assist test developers in examining the effectiveness of their

test items, and also furnish them with data which would enable them to improve the quality of their

tests further.

3.1. Specific Objective

Specifically, this study aimed at assessing students‟ critical thinking ability through a standardized test

uploaded at the university e-learning website, and their performance in a public speaking test

developed using the quiz module, one of the tools available in Moodle.

4. Research Questions

Follows are the research questions addressed in the study:

a. What is the reliability of the two tests used in the study?

b. Is students‟ critical thinking related to their performance in the public speaking test?

c. How can item analysis statistics provided by Moodle help instructors improve the quality of the

public speaking test?



i. How many items are considered easy and how many are categorized as difficult (i.e., what

is the IF index for each item)?

ii. How good are the test items at discriminating good students from weak ones (i.e., what is

the ID index for each item)?

iii. How good are the test item distractors (i.e., what is the DE index for test items)?

5. Methodology

5.1 Sample

The original sample of the present study consisted of 131 students from four public speaking classes

taught by the present teacher researchers. The final sample used to establish the relationship between

the two most important variables (i.e., performance in a public speaking test and critical thinking) in

this study was only fifty- eight (58) since not all of the students attempted the critical thinking test

posted on the university e-learning site.

5.2 Instruments

Two tests were used in the study: Cornell Critical Thinking Skills Test (CCTST), Level X, and a

public speaking test. The items for both tests were created using Moodle Quiz Module and were later

uploaded to the university e-learning site for students to access.

5.2.1 Cornell Critical Thinking Test Level X

Cornell Critical Thinking Skills Test (CCTST), Level X, was used to measure the critical thinking

ability of the students involved in the present study. This standardized test was developed by Ennis,

Millman, and Tomko (1985 & 2005) and later translated into Malay by Shaharom (2004). The test

contains seventy-one multiple-choice items, and a score of one (1) mark is given to a correctly

answered test item leading to a total mark of seventy-one (71) for performance on the test. A total high

score on the test is taken to reflect a high critical ability of the test taker.

5.2.2 Public Speaking Test

The public speaking test employed in this study had forty multiple-choice items. Each test item had

five options from which a test taker had to choose the correct answer. The test was used to gauge what

students had learned about the theories and guidelines provided in six chapters of The Art of Public

Speaking, a textbook (2010) that was adopted in the related course. The primary objective of the

public speaking course is to enable students to effectively develop and deliver their speeches. The

speech preparation process requires students to exercise their critical thinking skills so that their ability

to discern relationships among ideas are clearly indicated in their outlines and texts.

In the past, the public speaking test at University Malaysia Terengganu was administered by gathering

eight hundred to one thousand students in a few examination halls where test papers were distributed

to them. However, the establishment of an upgraded university e-learning site has motivated language

instructors to conduct the test online. The site adopted an LMS named Modular Object-Oriented

Dynamic Learning Environment (Moodle) as a teaching and learning platform. Moodle has made it

possible for the researchers and colleagues to construct and administer online public speaking tests

through its Quiz Module for the past six semesters. The earlier version of the online multiple-choice

question (MCQ) tests given to the students relied heavily on the test items provided in the test bank of

the adopted textbook. However, experience gained from administering online tests using such test

items had informed the present teacher researchers the need to discard some of the test bank items and

construct new items which suit local contexts to ascertain that the course educational goals could be

met and that cheating among students could be prevented. Also, the researchers felt the need to find a

mechanism to allow them to administer the test without having to call for all language instructors to

invigilate their respective groups of students taking the test in the computer laboratories. Observation



of students' behaviour while taking the test and analyses of the test results enabled the present

researchers to gain insights into the problem. They discovered that students took a longer time to

answer test bank items of high levels of Bloom's taxonomy of educational objectives. The inclusion of

such items in tests given later to a different group of students, without the presence of instructors,

showed that cheating was very much reduced. Since then, it has been considered practical to include in

the high-level test items, which require students to apply, analyze, and evaluate information.

The frequent use of Moodle quiz module to create and administer online tests caused the researchers to

explore the potential of the LMS further. The exploration led to the discovery of a tool in the quiz

module which helped the researchers to determine the reliability of a test conducted and the quality of

the test items used. This tool is a statistical technique called item analysis procedure. Relevant

descriptive statistics computed from this procedure are made available by Moodle at the statistics

section of the quiz module. Following the discovery of such a useful tool, the researchers felt a strong

urge to conduct the present research as an attempt to improve further the quality of the test given to the

students. The researchers also wanted to determine the appropriateness of the decisions made by them

throughout the previous test construction process so that they would be better informed when

constructing items for future tests.

5.2.3 Data analysis procedures

This quantitative study employed item analysis procedures offered by Moodle Quiz Module to

determine the reliability of two tests and examine the effectiveness of items in the public speaking test.

The module allowed the researchers to export the collected data and convert them into different forms

such as Microsoft Excel and SPSS data for further statistical analysis. This mechanism enabled the

researchers to compute a correlation coefficient to determine the relationship between students'

performance on the two tests used in the study.

6. Results & Discussion

6.1. Cornell Critical Thinking Test (CCTT)

The analysis done on the data revealed that the CCTT was a reliable one. The alpha value computed

was .73, a value which is within the range of reliability estimates in the test manual (Ennis et al. 1985

& 2005). The mean score for the test indicated that on average the students obtained 33 out 71 test

items correct. This value is much lower than the mean score obtained by the sample norm (consisting

of American undergraduates) in the CCTT test manual. Although the computed mean did not reach the

expected level, it still provides evidence that Malaysian undergraduates can think critically. The lowest

score obtained in the test was 18, and the highest was 50. The mean score might probably be of a

higher value if the number of samples was larger.

6.2. Public Speaking test

The reliability coefficient for the public speaking test was obtained from the Moodle quiz module by

clicking the results and statistics blocks, respectively. As indicated in Figure 1, the computed

reliability value (i.e., the coefficient of internal consistency) was at a satisfactory level: α = .77. The

Figure also illustrates that the test was difficult for the students since the average test score (i.e., the

mean score) was only 54 percent. The skewness and kurtosis values were also below the value of 1.00

which indicates that the distribution of the data approaches the normal pattern.



Figure 1: Descriptive statistics for Public Speaking Test

6.3. Relationship between Critical Thinking Public Speaking

Pearson Product-Moment Correlation analysis conducted on the collected data showed that critical

thinking was positively and significantly correlated to critical thinking. The computed value was .34

indicating that the relationship between the two variables was moderate. The correlation value also

suggests that critical thinking exerts an influence on students‟ performance in the public speaking test:

that is, at a moderate level, students who got a high mark for the public speaking test also received a

high score for critical thinking. This result implies that the effort taken by the researchers to include

higher-order test items is a worthwhile effort since the test items do not only manage to test students‟

comprehension of the content of their textbooks but also assess their ability to exercise their critical

thinking skills.

6.4. Quality of Public Speaking Test Items

One of the objectives of conducting this research was to observe and learn how statistical analysis

performed by Moodle enabled the teacher researchers to determine the efficacy of the test developed

through the Moodle Quiz Module. It was hoped that the information gathered from the module would

assist researchers in improving their test items, which would later be saved in the test bank for future

use.

6.4.1. Item analysis

Item analysis involves the application of statistical procedures on test scores to generate descriptive

and item statistics which can assist test developers in re-examining the effectiveness of their test items

and their test as a whole. These statistics can be computed manually using formulae provided in many

or most books on statistics. However, the procedures to be performed are rather tedious which may

cause many people not to consider using or applying them. With this issue, the Moodle Quiz Module

offers a practical solution. The module employs item analysis procedures which can generate the

statistical data that enables any test developers to examine the psychometric performance of each test

item and the effectiveness of a test as a whole.

The two most useful item statistics provided in the report produced by the Moodle Quiz Module are

item Facility Index (FI) and item Discrimination Index (DI). These indices are useful in assisting test

developers in making informed decisions regarding which items to retain, revise, and discard (Abdul

Rashid, 1996; ScorePak®, 2005; Kaplan & Saccuzzo, 2009; Cohen, Swerdlik & Sturman, 2013). The

two indices range from a value of 0 to 100. The FI is used to assess the difficulty level of a test item,

while the DI is used to examine the ability of an item to discriminate or distinguish between high and



low scorers on the test given. A test item will be considered an easy item if its FI value approaches the

value of 100. An item that has a DI value approaching the zero value is deemed to have a poor

discriminating power.

There is no consensus among practitioners on the definite FI and DI values that can be referred to in

determining the difficulty of an item and its discrimination power. In designing a good MCQ test,

Kaplan and Saccuzzo (2009) recommend the use of test items with FI value ranging from .30 to .70

because such items enable test developers to maximize the information about the differences among

the test takers. The recommended difficulty value range is close to the value range observed in the

normal bell curve; that is, the value range of .25 to .75.

In this study, items with FI index lower than .30 were classified as difficult items while those with FI

above .75 were categorized as easy items. The ideal difficulty value for the five-option multiple-choice

test items used in this study was .60, a value suggested by Cohen et al. (2013). For the discrimination

index (DI), values proposed by ScorePak® (2005) were referred to. Items with DI above .30 are

considered as items with good discriminating power, those with DI between.10-.29 as items with fair

power, and those with DI below .10 as items with poor power.

6.4.2. Item Difficulty Index (FI)

The results of an item analysis performed on the test data were obtained from the Moodle Quiz

Module by clicking on the statistics block, which led the researchers to the page entitled Quiz

structure analysis. The sorting of relevant data can be done on this particular page, but the present

researchers preferred to download the data and converted it into an excel data form in case there was a

need to transform it later into the SPSS data. The data was then sorted to rank test items according to

their FI and DI values. Analysis of the data based on these two values revealed that out of forty test

items, twenty-five were in the desirable FI range of .30-.70, nine items in the easy range, and six in the

difficult range. Two of the items were found to be extremely easy since more than ninety percent of

the students answered them correctly. These items may need to be replaced with more challenging

items or with those having better FI values. One item (with FI value of .8) was found to be tough since

only three out of 131 students managed to answer it right.

6.4.3. Items Discrimination index (DI)

For the individual item power of distinguishing good students from poor ones, seventeen items were

listed as having good discriminating power,( DI ranging from .31- .54), fifteen with fair ability (DI

.12- .29) and eight with poor power (DI -.19-.9). Among the items with low DI, three were observed to

have a negative value which indicates that students who did not score on the test got these items right

while those who did well on the test did not get them right (Cohen et al., 2013; ScorePak®, 2005).

These items need to be attended to immediately and decisions must be made as to revise or discard

them.

6.4.4. Using FI, DI and DE values in evaluating the effectiveness of test items.

Most advocates of item analysis recommend test developers refer to both the FI and DI values of a test

item to determine its quality. In this study, the values of these two indices generated by the Moodle

Quiz Module were used to examine the strength and weakness of each of the 40 test items. For

example, based on its FI and Di values, test item number 1 is considered as an easy item but has a poor

discriminating power. Figure 2 shows the relevant statistics gathered from the item analysis procedure

performed on test item number 1. As shown in the Figure, item 1 has a high FI value but a low DI

value. These values indicate that the item was an easy one to answer for the majority of students who

took it; however, it had a poor ability to discriminate good students from poor ones. It is a good idea to

retain this item after improving its DI value because having a few easy items in a test will provide

motivation to, especially weak students to continue attempting the test.



Figure 2: Item analysis of Question No. 1

To improve item 1, we need to analyse its distractor efficiency (DE) value and the students‟ responses

to the item. This data can be obtained from Moodle quiz structure analysis. Like its DI value, item 1

also had a low DE of 11.36%. This implies that many of its distractors were not functioning well.

According to Hingorjo and Jaleel (2012), a good test item, which has only one or without non-

functioning distractor at all, usually has DE within the range of 75-100%, while a poor item has DE of

less than 5%. Figure 3 presents the Moodle analysis of students' responses to test item 1. As indicated

in the analysis, four of the options were poor distractors. The first option only managed to attract nine

students while both the second and fourth options captured the eye of only one student. These options,

therefore, need to be further improved so that they can attract more students which in turn, can help to

increase the DI value of the item.

Figure 3: Percentage of students choosing the options given in Question No. 1

Another good example is the test item number 15 which is considered as a good item to keep in the

test bank for future use. Figure 4 shows the relevant statistics for the item. The FI value of this item is

within the desirable range (of .30-.70), and its DI value exceeds .31 indicating that the item has a good

discriminating power. However, the item only has DE of 53.61%, which is above 5% but still lower

than the desired DE of 75%.


Detailed analysis of responses for test item 15, as presented in Figure 5, clearly shows that all options

for this item managed to attract students to them which accounts for its high DI value. This item is

challenging and is a high level of Bloom's taxonomy because it requires students to read each of the

options carefully and figure out their relationships. However, the item still has its weakness; the

second option only attracted one student and should, therefore, be improved to attract more students.




Test developers should be wary of items that have negative DI values because those items indicate that

many weak students managed to select the correct option compared to good students (Kaplan &

Saccuzzo, 2009; ScorePak®, 2005). The negative DI value could either be due to a poorly constructed

item or wrongly keyed item. If the item is found to be poorly constructed on closer examination, it

should be eliminated from the test. However, if the value is due to test developer's carelessness in

keying the correct response option, it should then be revised, especially if its DI value is excellent.

Figure 6 highlights the quiz structure analysis for item 23, which is one of the items that has a negative

DI (item 23). The small FI value of about .34 reflects that the item is quite a difficult one.


Nevertheless, as indicated by its negative DI, the item failed to discriminate students who took the test

according to their ability. Upon examining the analysis of responses for the item (given in Figure 7),

the researchers realized that one of the test distractors or options attracted only one student. Besides,

the item was not that clearly stated; preposition „of' should have been inserted after the word

„objective.' Thus, the item needs revision.


7. Conclusion and Future Recommendation

This paper has demonstrated that Moodle is indeed a robust LMS. It enables educators, particularly

language instructors, to develop and administer a paperless test to a big number of students without

having to score the test manually. It also generates statistical data that can assist instructors in

determining the effectiveness of their test items and guide them as to how to improve the items. It is



clearly a useful tool for individuals who are keen to improve their test construction skills so that they

can enjoy their teaching profession better. However, educators should not rely solely on the item

analysis results generated by Moodle quiz module because the data are tentative and can be influenced

by many variables such as the type and size of test takers, and the instructional procedures used

(ScorePak®, 2005). Having had to develop expertise in test development procedures is literally a must

for all educators today. Thus, developing literacy in assessment should certainly be a constant and

deliberate effort of all educators to ensure that they have sound knowledge and skills in testing and

validation.

References

Abdul-Rashid, Rosyati. (1996). The development and validation of an institutional reading placement

test. Retrospective Theses and Dissertations. Paper 207. http://lib.dr.iastate.edu/rtd/207

Avison, D., Lau, F., Myers, M., & Nielsen, P.A.,(1999),Action Research, In Communications of the

ACM, 42:1, 94-97, Retrieved from:

http://www.researchgate.net/publication/220422055_Action_Research/file/60b7d51426bc9eb0d

.pdf

Aydin, C.C., & Cagiltay, N.E., (2007), How Assessment System of an Open Source Learning

Management System can be Integrated to a remote Laboratory Application? Problems and

Solutions, In The 18th Annual IEEE International Symposium on Personal, Indoor and Mobile

Radio Communication (PIMRC’07), 1-3, Retrieved from:

http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=4394850

Aydin, C.C., & Tirkes, G., (2010), Open source learning management systems in e-learning and

Moodle, Education Engineering (EDUCON), 2010 IEEE , pp.593,600, 14-16 April 2010.

Retrieved

from: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5492522&isnumber=5492336

Cohen, R. J., Swerdlik, M. E., & Sturman, E. D. (n.d.). Psychological Testing and Assessment: An

Introduction to Tests and Measurement (8th ed.). New York, United States of America: Mc

Graw-Hill.

Costa, C., Alvelos, H., & Teixeira, L. (2012). The Use of Moodle e-learning Platform: A Study in a

Portuguese University. In Procedia Technology, 5, 334–343. Retrieved from:

http://www.sciencedirect.com/science/article/pii/S2212017312004689

Ennis, R.H.,Millman,J. & Tomko, T.N.,(1985), Cornell Critical Thinking tests Level X and Level Z

manual, (1st ed.). CA:Midwest Publication.

Hingorjo, M.R., & Jaleel, F. (2012). Analysis of one-best MCQs: the difficulty index, discrimination

index and distractor efficiency. J Pak Med Association, 62(2):142-147.

Kaplan, R. M., & Saccuzzo, D. P. (2009). Psychological Testing: Principles, Applications and Issues

(7th ed.). CA, United States of America: Wadsworth, Cengage Learning.

Lim Hooi Lian, Wun Thiam Yew, & Chew Cheng Meng. (2014). Enhancing Malaysian teachers'

assessment literacy. International Education Studies, 7 (10): 74-81.

Malaysia Ministry of Higher Education (MOHE). (2007). Higher Education Action Plan 2007-2010.

Ministry of Education Malaysia (2013). Malaysia Education Blueprint 2013-2025: Preschool to

postsecondary education.

Rajendran Nagappan. (2001). The teaching of higher-order thinking skills in Malaysia. Journal of

Southeast Asia Education, 2(1).

ScorePak® (2005). Understanding Item Analysis Reports. Retrieved from:

http://www.washington.edu/oea/services/scanning_scoring/scoring/item_analysis.html

Whelan, R. & Bhartu, D., (2007). Factors in the deployment of a learning management system at the

University of the South Pacific. In ICT : providing choices for learners and learning. Retrieved

from: http://www.ascilite.org.au/conferences/singapore07/procs/whelan.pdf

Wright, P. & Wright, G., (2011), Using Moodle to Enhance Thai Language Learning: Instructor and

Learner Perspectives, In The Journal of Kanda University of International Studies, 23, 375-398,

Retrieved from: https://www.kandagaigo.ac.jp/kuis/about/bulletin/jp/023/pdf/019.pdf

http://lib.dr.iastate.edu/rtd/207

http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=4394850

http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5492522&isnumber=5492336

http://www.sciencedirect.com/science/article/pii/S2212017312004689

http://www.washington.edu/oea/services/scanning_scoring/scoring/item_analysis.html

http://www.ascilite.org.au/conferences/singapore07/procs/whelan.pdf

https://www.kandagaigo.ac.jp/kuis/about/bulletin/jp/023/pdf/019.pdf



Documents

CONDUCTING AND EXAMINING EFFECTIVENESS OF ONLINE ...worldconferences.net/ejournals/glit/wp-content/... · targeted that by the year 2000, sixty percent of examination papers should