Sri Krishna Institute of Public AdministrationInnovation, Planning & Management Unit

Presents a Case Study On

ONLINE EXAMINATIONWider Choice, Easier Selection

Introduction

Remember the Glorious days of Board Examination. Year around hard labor, used to

be managed by a huge Institutional set up. There were Great Recruitment Boards

conducting examination all across the country. From the perspective of recruiter this

was a costly and tedious affair. Added were the responsibility of maintaining

confidentiality and holding volumetric records for future reference. Thanks to computer

and internet we have online Examination System as an option.

For years, Administrative Training Institutes (ATI) like other Government Agencies

have followed the conventional system of conducting examination. The entire selection

procedure took around six months. At the outset, the vacant positions were advertised in

at least four different newspapers. One month time was given to the aspiring candidate

to apply. Then in one week time scrutiny of the applications were done and a list of

shortly listed candidates was to be prepared. Thereafter the shortlisted candidates were

informed the date of exam by post.

Before the examination a number of preparations like framing of question papers, selection

of examination center etc. were done. On the examination day a huge number of staff was

deputed to conduct the examination. Papers were checked in a time span of one to two

months. Results were declared followed by the interview call letters. The next fortnight was

spent in conducting the interviews. The interviews were held over a period of time as per the

number of eligible candidates. Post interviews, Declaration of final results would also take

time. If there is any trivial mistake, depending upon situation, 3-6 months additional time

may be required.

However, advent of online examination have opened up possibilities for those who otherwise

did not have a choice. A testing put online is efficient, adaptable and prudent. For

mentors/educators, they don't need to save the time to check the test in light of the fact that

the test outcomes can be reviewed consequently. For test-takers, they can take the online test

at their own favored time. For associations, testing online spares a ton of expense to utilize

teacher architects and print a great many test papers.

As the times progresses, online test will most of use by applicants many online portals are

available on web which provide the online test and mock test and thousands of other benefits

for students such like: they provide the online test series at just affordable cost as compare to

market rates, Who created huge amounts of helpful items for us to make online tests, now test

creators assume an imperative part in enhancing the efficiency of making an online test.

While in principle such developments should be adorned by various agencies, Government

system often fails to see a value in the proposition. It is in this context, Online Examination at

SKIPA demands special mention for this effort.

The Case: Process & Methodology

The online Examination was an initiative to select suitable human resource for Innovation,

Planning and Monitoring Unit (IPMU) at SKIPA. The IPMU has been sanctioned by the

Government of Jharkhand on 25.03.2015 to function under the control of Center for

Innovation in Public Service (CIPS) at Hyderabad. IPMU works to study, document, pilot

and Scale Up innovations in Public Service Delivery System.

NotificationBesides the traditional way of newspapers, online advertisements were made on the websites

of SKIPA and CIPS. The advertisement along with link to Course wise Term of Reference

was also promoted through Facebook Pages, Google Groups, LinkedIn, and Twitter. 351

candidates across the world applied for three (3) positions of Research Analyst (RA),

Program Manager (PM) and Program Officer (PO).

ApplicationOnline application formats were provided to the candidates applying along with facility for

Resume uploade in Pdf format. Online and telephonic support was made available through

dedicated Help desk. On successful completion of application process an auto genarated E-

mail was sent as Acknowledgement receipt to the concerned candidate. Log in credentials to

participate in the Examination were sent through E- Mail and SMS.

ScreeningA dedicated help desk was created at SKIPA to screen applications online and guide

candidates suitably. Individual instructions were made available to the candidates to address

their querries and technical assistance during Form Fill up. To nullify chances of Favoritism,

automated Screening of applications based on specific parameters was conducted without

manual interference were adopted

Development of QuestionnaireThe questions were developed in a highly secured manner, with three different sets, from

which one was randomly chosen for uploading in the online question format on the morning

of the day of examination. The entire process was designed in a manner which was analysis

friendly through data mining, since the entire database of candidates was tabulated. CIPS ,

Hyderabad were responsible to conduct the examination.

ExaminationThe Examination was conducted on 15th September, 2016. 351 candidates from 14 Indian

States and 5 countries participated in the Examination (Vide List 1). Computer Laboratory at

SKIPA was used as a Test center at for willing candidates at Ranchi. Candidates have option

to participate in the Examination from the comfort of their home. The entire selection

procedure was outsourced to an independent organisation.

Figure: Participant Location were automatically recorded during Online Examinationinitiative by IPMU, SKIPA. Ensuring internet connectivity and availability of desktops forcandidates attending examination at the center and developing a mechanism to checkplagiarism and legitimacy of the candidate were key challenges in front of SKIPA Team.

Result : Checks and Balances

Evaluation of papers were made on the very date of Examination conducted by Faculty

members of SKIPA. The team was able to conduct the online examination and recruit the

team members of IPMU within a period of two months. The selected candidates have been

inducted and are working at IPMU. The system has capability of conducting Instructor

assisted Evaluation and uses data analytics module to correlate result. Some examples of

correlation study is given below:

A. Overall Relationship between three Evaluators for the post of State Program Officer

(SPO): According to this result, Marks similarity between Evaluator 1 & Evaluator 2 is

70.5%, Evaluator 1 & Evaluator 3 is 76.3%, and Evaluator 2 & Evaluator 3 is 96.1%.

B. Overall Relationship between three Evaluators for the post of Project Officer (PO):

According to this result, Marks similarity between Evaluator 1 & Evaluator 2 is 60.9%,

Evaluator 1 & Evaluator 3 is 71.2%, and Evaluator 2 & Evaluator 3 is 91.8%.

C. Overall Relationship between three Evaluators for the post of RA: According to this

result, Marks similarity between Evaluator 1 & Evaluator 2 is 64.9%, Evaluator 1 &

Evaluator 3 is 70.8%, and Evaluator 2 & Evaluator 3 is 95.6%.

Check points:

Pre Examination Provision for more than one question paper- three sets of paper for three

positions was kept in fray for the exercise. Care were taken to Frame questions that cannot be

searched through Google and other websites for answers (analysis based rather than subject

based). Timer Enabled with username and password bounded question set for each candidate

was ensured. Real Time Technical Assistance through support desk was arranged.

Post Examination Circulation of Answer Sheets for evaluation on the same day (by email),

Saved a lot of time and effort as compared to sending answer sheets through post. Since data

is collected, collated and dispatched for evaluation digitally without human intervention,

there was no loss of data. Answer sheets were encoded with watermark to avoid tampering of

data using Non-editable data format and converted into Password protected PDF documents.

Plagiarism check for answer sheets with details of web documents was conducted. IP and

location based information on geographical spread of candidates appearing for online test

were tracked on real time basis during the examination. The data was stored for future

reference and comparisons in subsequent years.

Impact

The online exam conducted by SKIPA for the selection of IPMU team members was one of

the first of its kind in Government Sector and Mission bodies in Jharkhand. The entire

process of recruitment was complicated with major challenges like Voluminous Application,

chances of Favoritism, Screening of applications based on specific parameters.

1. Automated Screening of applications without manual interference leading to no scope for

favoritism were adopted.

2. There was minimal Displacement of applicants owing to the online nature of the

examination ensuring higher participation and lesser Cost / Resource involvement.

3. It was possible to answer the examination from any corner of the world. Besides a large

number of applicants across the country, candidates from countries outside India could

attend the examination.

4. This also proved to be an environmental friendly means that reduced the need for travel at

various ends and also helped in paper saving.

5. The system appears to be more secure for management user records and more reliable to

work at any conditions.

6. There is no cost involved in printing or distributing of question papers. Transport and

logistic expenses for conducting examination in several geographical locations is saved.

7. It saves the travelling time involved for both the examinee and the examiner. It alsolessens the time and cost required for filling application and posting of application. Underthis system panel of question setters and examiner are independent of geographicallocation.

8. Assured Cyber security provisions enhances Confidentiality and minimize chances of

paper leaks. It expands the scope of conducting examination. At the same time

examination could be conducted across different geographical location. It gives

opportunity to more and more physically challenged candidates to participate in the exam.

The IPMU track record shown that, candidates from 28 Different State and countries have

applied for the positions. For candidates an estimated 4108 Hours of travel time were

saved. Government could save an estimated expenditure of approximately 15 lakhs rupees

to be provided otherwise in terms of Travel Allowance (AC III tier) and manpower

deployment. The entire selection process for 351 candidates was completed within a period of

Two (2) months. A similar recruitment process at UMC (Urban Management Cell),

Jharkhand conducted during the same period following traditional methods including offline

examination and Manual Communication cum Evaluation took Six (6) months for completion

to recruit from 35 candidates.

Way forward

Online technologies can provide useful learning and assessment opportunities for practical

skills when they are developed in a manner that promotes the obvious and coherent

relationship between subject matter, performance expectations, student demonstrations of

practical skills and feedback.

This is particularly important in the relationship between the formative and summative

assessment expectations associated with practical skills development and assessment.

Resources and requirements of the protocol promoted activities and behaviors may result in

notable gains in candidate’s examination quality.

Number of candidates appearing in government examination outnumbers participants in any

other form of exercise in India. Online Examination Connected with Common Service Centre

and Mobile based applications has potential to minimize cost and maximize quality.

Government agencies, including Jharkhand Public Service Commission and Human

Resource Department, Government of Jharkhand may try this on priority basis.

At a Glance: Key Advantages for Online Examination

Cost Effective A. Substantial cost reduction in the expenses of conduction an examB. No Printing or Distribution Expenses for question papers.

C. No transport and logistics expenses while conducting examsacross several geographically scattered locations.

Secured D. Maintain confidentiality and avoid paper leaks.

E. Change question papers at the last minute by pulling questionsfrom the Question Bank.

F. Randomize sequence of questions for each examinee: No twoexaminees will see the same question at the same time.

G. No data loss even in situation of power and internet failure.

Expands Scope H. Create multiple exams on multiple subjects for multiple courses.

I. Create exams in multiple languages and overcome the languageconstraints.

J. Conduct exams in several geographically scattered locations atthe same time.

K. Panel of Questions Setters and Examiners can be independent ofgeographical location.

User

Convenience

A. Users can take exams as per their convenience. Exam can beconfigured for 24X7 availability.

B. User can view the exam results immediately after the exam.

C. Option to display the feedback for correct answers.

Analysis D. Store data for comparison in subsequent years.

E. Customizable Reports to help you retrieve and analyze thedesired statistic data.

Go Green F. You will be saving a substantial amount of paper by using onlinemode of exams. Prevent the use of paper and save the planet.

Glimpses

Figure above shows Online Examination at Ranchi Test Center, SKIPA.Based on success of this initiative, SKIPA conducted an Online Exam ofComputer Basics on 28.09.2015 for the office staffs and the result cameout on the same date of examination. Figure below shows the same testin process.

Participant List for Online Examination Conducted by SKIPA

Sl No. Name of Country/ State Name of Cities Number of Candidates

1 Karnataka Bangalore 34

2 Delhi Delhi 30

3 Maharashtra Mumbai, Pune , Sholapur 20

4 Andhra Pradesh Hyderabad 11

5 West Bengal Kolkata 11

6 Jharkhand Ranchi, Dhanbad, Bokaro 9

7 Haryana Gurgaon, Faridabad, Kaul 6

8 Bihar Patna, BeguSarai 5

9 Rajasthan Jaipur 4

10 United State (US) San Bruno, Mountain Viw 3

12 Thailand Chanthaburi 1

13 Kerala Thana 1

14 Gujarat Himatnagar 1

16 Orissa Bhubaneswar, Cuttack, Kharial 3

17 Uttar Pradesh Allahabad, Ghaziabad 2

21 Maharashtra Kolhapur 1

23 Punjab Patiala 1

24 Uttarakhand Kundan 1

25 Madhya Pradesh Jabalpur, Bhopal 2

26 Karnataka Belgaum 1

27 Telengana Telengana 1

28 Andhra Pradesh Machilipatnam 1

Total 351

Concept:Shri Sudhir Prasad, Director General, SKIPA

Editorial Team:

Hiralal Mandal, Additional DirectorAnil Kumar, Joint DirectorSanjay Ambasth, Joint DirectorDr. Sanjay Singh, Joint DirectorUsha Dwivedi, Deputy Director

Assisted By:Suman Kumari, Ratnakar Tiwari, Anuj Kumar Sinha, Pabitra Sahu

Consulting Editor: Kallol Saha

Background Research by:

BADLAO INSTITUTE OF TRAINING AND MANAGEMENTV.I.P. Road, Arsande, Kanke, Tel: 0651- 2450372, E-maill: bitm.ranchi@rediffmail.com,Website:www.bitmedu.org, Ranchi-834004, Jharkhand, India

Published by

Shri Krishna Institute of Public AdministrationAdministrative Training Institute, Govt. of JharkhandMears Road, Ranchi-834008, Jharkhand,FAX: 0651-2283806 | Phone: 2283804Email:skipa.ranchi-jhr@nic.in, skiparanchi@yahoo.co.in,Web site: http://skipa.nic.in