STANDING COMMITTEE MEETING NMEICT28/05/2014

DEVELOPING A JOINT ENTRANCE EXAMINATION (Main) (JEE) ONLINE ITEM BANK FOR REMEDIAL TEACHING AND

LEARNING OF CANDIDATES

Shri Vineet Joshi – Chairman CBSE and Executive Council CAER

Mrs. Sarita Manuja - Director CAER

• Shri Vineet Joshi, IAS (Chairman, CBSE & Chairman CAER – Executive Council)

• Ms. Sarita Manuja (Director, Centre for Assessment, Evaluation & Research)

• Prof. James Stephen Tognolini (Senior Vice President, Research and Assessment Pearson and Visiting Professor, University of Oxford)

• Dr. Sadhana Parashar (Professor and Director - Training and Academics, CBSE)

• Mr. Arjun Bahadur (Deputy Director, Centre for Assessment, Evaluation & Research)

• Dr. Priyanka (Research Officer, Centre for Assessment, Evaluation & Research)

• Dr. Akanksha Bapna (CEO, Evaldesign)

CBSE – CAER CORE PROJECT TEAM

Background

• JEE (Main)

High-stakes examination for which students from across the country spend significant amounts of time practising to maximise their score

While preparation for examinations is a very useful learning tool, it starts to have a negative impact on broader learning when the focus is purely on “practising for the test”

The fact that students are paying significant amounts of money to have this tuition is also impacting on the “social fabric” of society

DEVELOPING A JOINT ENTRANCE EXAMINATION (Main) (JEE) ONLINE ITEM BANK FOR REMEDIAL TEACHING AND

LEARNING OF CANDIDATES

Scope and Coverage of Solution

• The CBSE - CAER is proposing to develop an item bank of JEE (Main) items which students preparing for the JEE can access online. These items will be linked to learning materials which are designed to help students improve their learning• All candidates (nationally and internationally) would have access to this powerful “learning” tool which will addresses some of the concerns outlined in the previous slide in that the focus is on learning• It is designed to identify areas of potential weakness; promulgate corrective strategies that have been identified through an analysis of the assessment items; and, give further practice examples from similar JEE items which assess the same skill to make sure that the student has understood the concept• The intention is to utilise ICT tools to make the learning more interactive

DEVELOPING A JOINT ENTRANCE EXAMINATION (Main) (JEE) ONLINE ITEM BANK FOR REMEDIAL TEACHING AND

LEARNING OF CANDIDATES

Steps Involved

DEVELOPING A JOINT ENTRANCE EXAMINATION (Main) (JEE) ONLINE ITEM BANK FOR REMEDIAL TEACHING AND

LEARNING OF CANDIDATES

Develop Table of Specifications covering the

construct of JEE

Identifying and Tagging a pool of 501

Items by Subject Experts and Test

Developers of JEE.

Identifying , describing and

storing the rationales behind the distracters

Incorporating corrective learning strategies for each

incorrect option chosen

Build additional items for practice for each incorrect action to improve learning

Develop an online system for item bank including feedback

and learning components

(with technology partner)

Locate the item bank system on a website to make it accessible

world wide 24/7.

Conduct Load trials and carry out pilot

study

Process Flow

DEVELOPING A JOINT ENTRANCE EXAMINATION (Main) (JEE) ONLINE ITEM BANK FOR REMEDIAL TEACHING AND

LEARNING OF CANDIDATES

Features of Platform

DEVELOPING A JOINT ENTRANCE EXAMINATION (Main) (JEE) ONLINE ITEM BANK FOR REMEDIAL TEACHING AND

LEARNING OF CANDIDATES

Features of Platform (Contd)

DEVELOPING A JOINT ENTRANCE EXAMINATION (Main) (JEE) ONLINE ITEM BANK FOR REMEDIAL TEACHING AND

LEARNING OF CANDIDATES

Technology Used and Hosting Architecture

DEVELOPING A JOINT ENTRANCE EXAMINATION (Main) (JEE) ONLINE ITEM BANK FOR REMEDIAL TEACHING AND

LEARNING OF CANDIDATES

Programming language - Ruby v2.1.0.Web Framework - Ruby on Rails v4.1.0ORM - Active RecordDatabase - MySQLFront end - ERB templates, jQuery, SCSS,

BootstrapCaching - RedisWeb Server - Nginx Application Server - UnicornServer Operating System - Ubuntu 13.10

Hosting Architecture - Amazon Web Service stack

Deliverables and Timelines

DEVELOPING A JOINT ENTRANCE EXAMINATION (Main) (JEE) ONLINE ITEM BANK FOR REMEDIAL TEACHING AND

LEARNING OF CANDIDATES

Key deliverables Expected time of delivery

Inception of the Project 1 July 2014

Development of item bank with 501 items 31 Oct 2014

Completion of technology Platform along with Hosting 31 Dec 2014

Pilot run of the Platform 1 Jan 2015

Submission of report of Pilot 30 June 2015

Update of Platform basis Pilot 1 Aug 2015

Cost of the Project

DEVELOPING A JOINT ENTRANCE EXAMINATION (Main) (JEE) ONLINE ITEM BANK FOR REMEDIAL TEACHING AND

LEARNING OF CANDIDATES

Recurring Heads Annual Cost (INR)

Hosting Cost 3,00,000

Human Resource

Project Manager 12,00,000

Data Analyst 6,00,000

System Administration Cost 12,00,000

Academic Advisors 5,00,000

Total 38,00,000

Cost of the Project

DEVELOPING A JOINT ENTRANCE EXAMINATION (Main) (JEE) ONLINE ITEM BANK FOR REMEDIAL TEACHING AND

LEARNING OF CANDIDATES

Development Heads Cost (INR)

•Development of Table of Specifications (TOS)•Selection of 501 items coded with metadata described within TOS•Identify 10 items with characteristics similar to the 501 items and tagged as per TOS

14,00,000

Review and Quality Assurance 6,00,000

Building of rationales behind each incorrect option for all 501 itemsTo explain and build Corrective learning strategies for each

24,00,000

Review and Quality Assurance 6,00,000

CBSE CAER Cost of Managing the Development 24,00,000

Total 74,00,000

Cost of the Project (Contd)

DEVELOPING A JOINT ENTRANCE EXAMINATION (Main) (JEE) ONLINE ITEM BANK FOR REMEDIAL TEACHING AND

LEARNING OF CANDIDATES

Development Heads Cost (INR)

Platform Development

Inception Stage (Requirements ; Analysis ; Documentation) 2,80,000

U I Design 2,00,000

Coding and Development 23,00,000

Testing and Integration 11,50,000

Infrastructure and Release 2,50,000

Total (including mandatory costs) 47,00,000

Cost of the Project (Summary)

DEVELOPING A JOINT ENTRANCE EXAMINATION (Main) (JEE) ONLINE ITEM BANK FOR REMEDIAL TEACHING AND

LEARNING OF CANDIDATES

Development Heads Cost (INR)

Recurring Cost (Annual) 38,00,000

Development Cost (Content) 74,00,000

Development Cost (Platform) 47,00,000

Total 1,59,00,000

Unique Features of Solution

• Self learning model which allows learning at own pace and remediation through pedagogic interventions• Leveraging ICT to maximize reach and effectiveness of learning•24/7 access by students world wide • Can be used for student and teacher development• Model is based on Social Equity, Access to everyone and improved Quality of learning• Large pool of CBSE – CAER domain experts and assessment experts

DEVELOPING A JOINT ENTRANCE EXAMINATION (Main) (JEE) ONLINE ITEM BANK FOR REMEDIAL TEACHING AND

LEARNING OF CANDIDATES

Realize Objectives of MissionNMEICT Mission –

A proper balance between content generation, research in critical areas relating to imparting of education and connectivity for integrating our knowledge with the advancements in other countries is to be attempted. No Talent of the country should be allowed to go waste

Knowledge modules based on the personalized needs of the learner would need to be delivered to him /her at the right time with the right content interactively to take care of his/her aspirations

Project Goals and Intention

This project generates content through expert intervention, ensures that all students across the country have access to high-quality content through this online platform at minimal cost and increases accessibility of the learning material

It allows for self-paced learning by enabling students to access the question bank and concept details on their own time, making it highly personalized

DEVELOPING A JOINT ENTRANCE EXAMINATION (Main) (JEE) ONLINE ITEM BANK FOR REMEDIAL TEACHING AND

LEARNING OF CANDIDATES

CAER – RESEARCH PROJECTS COMPLETED WITH CBSE

• Some Possible Options for Aggregating Subject Examinations at the Level of Scores for Entry into Indian Tertiary Institutions – JEE Normalisation Model

• Evaluation of the Validity and Functioning of the Test (The Central Teacher Eligibility Test, November 2012) - CTET Project

• Study of the Structure of Grade XII Question Papers for Different School Examination Boards – 21 State Boards

• Evaluation of the Item Functioning of the Test (JEE Main- 2013) – Item Analysis

CAER – RESEARCH PROJECTS ONGOING WITH CBSE

• Developing On-Line Problem Solving Assessment System for the students of Grade IX and XI targeting to nurture and assess cross curricular skills

• Evaluation of the Item Functioning of the Test (JEE Advanced – 2013)• Examining Grade XII Examination Practices of CBSE and making

recommendations for Improvement• Comparative Analysis of Difficulty Level of Grade X question

papers of Board based and School Based Summative Assessment (II)• Evaluation of the Item Functioning of the Test (CTET, 2014)

Thank You!

Centre for Assessment, Evaluation and Research (CAER)4th Floor, 17 Rouse Avenue, New Delhi 110 002

Email: contactus@indiacaer.orgWebsite: www.indiacaer.org