2009 NCTA Conference San Antonio The Online Assessment Advantage at Tyler Junior College Ken Craver Director of Distance Education

2009 NCTA Conference San Antonio

The Online Assessment Advantage at Tyler Junior

College

Ken CraverDirector of Distance

Education


Slide 2

What We Will Cover Who we are Why Questionmark What we do

Our Server Configuration Assessment Creation and Scheduling Assessment Delivery

Download-ables (http://TJC-QM.4shared.com/) These slides Questionmark Bridge Questionmark Bridge User’s Guide More documents available upon request


Slide 3

Who We Are

Multi-campus junior college

10000+ students 3700 / semester

in distance education

60+ faculty use Questionmark Perception in 90 different courses

4 support persons Proctored testing

in TJC Testing Center


Slide 4

Why Questionmark

Our Students… are computer learners desire flexibility in schedules can benefit from more and varied

assessment opportunities like immediate feedback on their grades

and test questions missed can be anywhere in the world


Slide 5

Why Questionmark

Our Faculty (why they like it)… do not lose valuable class time to giving

assessments (4-6 class periods / semester) can provide more effective assessments using a

variety of question types can categorize and tag questions to allow more

intelligent question selection can give proctored exams in our testing center can give practice exams students can take from

anywhere can use assessments as learning tools can obtain a wide variety of statistical reports on

assessment results.


Why Questionmark

Some History Originally championed by our Director of

Testing I was paid a stipend to administer the

server along with training and supporting faculty

Also paid faculty member a stipend to train and support faculty

10-15 computer testing stations Tables for paper tests

Slide 6


Slide 7

Why Questionmark We Use Perception for…

Pre-course Assessments Practice Quizzes Exams Surveys


Why Questionmark

Question Types

Slide 8

Adobe Captivate Adobe Flash Drag and Drop Essay Explanation File Upload Fill in Blanks Hotspot Likert Matching Matrix

Multiple Choice Multiple Response Numeric Pull-down List Ranking Select a Blank Spoken Response Text Match True/False Yes/No


Slide 9

Server Configuration

Questionmark Version 4.4

W eb Server

InternetRouter

Databases

DatabaseServer

IP Sw itch

StudentBrow ser

StudentBrow ser

Cable m odem or DSL

Intranet

Firew all

Dial up

StudentBrow sers


Slide 10

Assessment Process Our Process 1

Item bank creation Create topic structure Import multiple-choice questions using

Questionmark Bridge (in-house program)

Assessment Creation Assessment Publishing (just a copy from

development repository to production repository)

Group Creation


Slide 11

Assessment Process Our Process 2

Participant Creation Import into group using Questionmark Bridge All class rolls are sent as text files to a secure FTP site Rolls are downloaded and participants are imported and

linked to groups

Exam scheduling form completed Online form completed by each instructor that includes

course, Exams to be given, deadlines

Assessments scheduled according to exam scheduling form

Results checked by instructor


Slide 12

Assessment Process Repository Structure

TJC.qmr Shared

Production repository

TJC-Dev.qmrShared

Development repository

Local Development

repositories(on faculty computers)


Slide 13

Assessment Process

Supporting Faculty Support Team

2 instructors, 1 staff, 1 part-time employee Semester stipend equal to 3 hour overload contract Responsibilities

Create question topic structure and import/create questions sent by faculty Create assessments Create groups and import participants Schedule assessments Provide support

Faculty Designers (Do-it-Yourself-ers) Do some or all of their own design, implementation and scheduling work

Faculty Users Deliver questions to the support team and specify the content of their

exams Complete online scheduling form for the exams they want scheduled Check student scores using Enterprise Manager web-based tool


Slide 14

Assessment Process Item Bank Creation

We import multiple choice questions1. Format questions in a text file as follows:

1. Question stem containing the text of the question.

a. Answer 1b. Answer 2=c. Answer 3 (correct answer)d. Answer 4

2. Use the Questionmark Bridge program to create an import file.

3. Import the “import file” to an Authoring Manager topic.


Slide 15

Assessment Process Questionmark Bridge

Text file

answers marked

Prepare text file Questionmark

Bridge

(“import.txt”)

Import

“import.txt” imported into

Authoring Manager


Assessment Process

Authoring Manager

Slide 16


Slide 17

Assessment Process Assessment Creation

Created in TJC-Dev development shared repository

Copied to TJC production repository when ready to be scheduled

All assessments use same custom template

No changes allowed


Slide 18

Assessment Process Assessment Publishing


Slide 19

Assessment Process Group Creation

Group typically = class section

Use naming standard to make groups easy to find

Delete old groups at start of each semester


Slide 20

Assessment Process Participant Creation 1

Download all class rolls as text files from internal FTP site that is populated automatically from our Banner system Done second week of semester

Support team or faculty developer downloads roll from FTP site

Roll is prepared for import using Questionmark Bridge


Slide 21

Assessment Process Participant Creation 2

“Import.txt” file is imported into Questionmark and participants linked to a group.


Slide 22

Assessment Process Exam Scheduling Form and Database

Zoho Creator form - www.zoho.com (Free!)


Slide 23

Assessment Process Exam Scheduling Database

Available to all of our support team and our Testing Center personnel

If deadline for an exam changes faculty will submit the change via the scheduling form


Slide 24

Assessment Process Assessment Scheduling

Scheduled according to faculty input on scheduling form

Scheduled to groups Deadlines may or may not be set on the

schedule.


Slide 25

Assessment Process Results Checking

Instructors check their own results using Enterprise Reporter.

Common Reports Score List with Coaching Report Question Statistics Report Gradebook Report Transcript Report Test Analysis Report used by Nursing

Instructors


Slide 26

Delivering Assessments

Our Testing Center 50 testing stations Test over 33 subjects Average 3000 tests /

month (long semester)

Peak of 600 tests / day

4 Full time staff 12 Part time staff


Slide 27


Our Testing CenterEntry Point



Early Testing Center

Slide 28


Slide 29


Current Testing CenterTwo types of Nova desk testing stations


Slide 30


Our Testing CenterNova desks with hoods contain computer with a monitor that is viewable through the glass top


Slide 31


Our Testing CenterSecurity Cameras


Slide 32


Our Testing CenterSecurity Monitor at front desk


Slide 33


Testing Center Procedures 1

Check scheduling database (Zoho) report for instructors using QuestionMark, special instructions, courses and their deadlines

Notify front desk at testing center of courses, instructors, and special instructions for QuestionMark users

Students must be on the classroll, on the QuestionMark roster with their personal identification number to be able to test

Students must show a valid photo ID and sign in before testing

All electronic devices must be turned off before entering the testing center

A proctor will take the student’s ID, select a computer, and enter the student into their test.

During the testing, the proctors monitor the testing area looking for any unusual activities.


Slide 34


Testing Center Procedures 2

Student will print a copy of their results if feature enabled by instructor

Constant visual monitoring by inside proctors and video cameras

Able to view any activities after the fact and download them if needed.

Give computer based test for any type of class Paper based tests are allowed (but not encouraged) for

online classes only Also, use Blackboard, SAMS, Word, and Excel for testing

purposes Give Virtual College of Texas (VCT) exams and

correspondence exams. Fee based Off-site students - send Questionmark Secure (browser add-

on) to be installed on test station.


Slide 35

Delivering Assessments QM Skills for Testing Center Manager

Can add participants and link them to a group

Can re-activate an expired exam. Can check exam scores


Slide 36


Future Satellite Testing Centers Software to enable remote access to

testing computers.


Slide 37

Delivering Assessments Security Options

Open access for low stakes Secure access for high stakes

Cameras in operation User name and password validated for monitored

assessments: Against Perception database Only proctor knows password

QuestionMark Secure browser add-in to prevent printing and browsing

Cannot access any other program once browser started Cannot close browser Can only take assessment


Slide 38

Thank You! Ken Craver

[email protected]

Downloads Web Site http://TJC-QM.4shared.com

Documents

2009 NCTA Conference San Antonio The Online Assessment Advantage at Tyler Junior College Ken Craver Director of Distance Education