Solution Report - Document Management Made Easy · Solution Report Buyersa.com ... faxes and multi-function (MFP) ... TEST GRADING SUPPORTS ESSAY QUESTIONS REPORTING CUSTOMIZABLE

OVERVIEW

BUYERS LAB

BUYERS LABSolution Report

BuyersLab.com

Product Snapshot

Product: Teaching Assistant

Version: 3

Software Developer: Kyocera Document Solutions America, Inc.

Web: www.usa.kyoceradocumentsolutions.com

Phone: 973-808-8444

For more information on Teaching Assistant 3 and other Kyocera products, see BLI’s comprehensive coverage at Buyerslab.com/Solutions.

About BLI: Since 1961, BLI has been a leading test laboratory in the world of digital imaging equipment. BLI is completely independent in all of its testing processes and subsequent reporting. All of BLI’s product evaluations are conducted by highly experienced employees in its on-site testing facilities in the United Kingdom and United States where hundreds of new copiers, printers, wide-format devices, scanners, faxes and multi-function (MFP) products are evaluated and reported on each year.

Kyocera Teaching Assistant 3

O C T O B E R 2 0 1 6

Features & Productivity

Usability

IT Admin & Security

Support & Training

Value

Designed for the education market, Kyocera’s Teaching Assistant is a business application that allows teachers to create and print multiple-choice “fill in the bubble” test forms at MFPs equipped with the solution, and then have the solution grade and analyze completed tests automatically by scanning the answer key and answer sheets at the MFP. The solution effectively automates the tasks teachers conduct on a regular basis. And thanks to its integration with Kyocera’s HyPAS-enabled devices, the application does not require a server- or cloud-based component like most competing systems: users operate the application at the control panel of supported HyPAS-enabled MFPs. Optional integration with Microsoft Excel allows teachers to use a PC to create tests if they prefer, and offers additional analysis and charting of test results.

© 2016 Buyers Laboratory LLC. Duplicated by permission. For reprints, contact [email protected].

BUYERS LAB

BUYERS LABKyocera Teaching Assistant 3

2

Chief BenefitsEasy deployment — Because the solution resides completely on the MFP, there is no added burden on an organization’s IT

infrastructure, which gives administrators more leeway in approving the placement of Teaching Assistant.

Flexible test creation — Users have a wide range of options when creating test forms at the MFP control panel: Multiple-choice questions can have between 2 and 8 answer “bubbles,” each page can hold up to 50 questions, tests can have up to 500 questions and each test can support an unlimited number of students. The solution can even provide space for essay questions.

Improved teacher productivity — Once an answer key is scanned in at the MFP, completed test forms can be scanned and graded with the press of a button, which saves educators hours of tedious manual labor.

Analysis adds value — Teaching Assistant includes a range of analytical tools to aid teachers in evaluating student performance on a given test, including a grade range histogram, top 10 easiest/hardest questions and a combined student performance chart, so teachers can readily see areas that may need further emphasis in the classroom. Optional integration with Microsoft Excel enhances reporting and analysis capabilities, helping educators develop tailored lesson plans based on test results.

Security and privacy — Test results are never stored on a server and are erased from the MFP upon exiting the program at the control panel, ensuring that unauthorized users cannot see or access test results.

OUR TAKEKyocera’s Teaching Assistant offers important benefits to any school system that uses “bubble sheet” multiple-choice test formats. It is a time- and resource-saving tool for teachers who want to concentrate more on face-to-face tutoring and classroom activities, and less on laborious administrative tasks. Moreover, unlike some competing systems that require configuration assistance and ongoing support from a school’s IT personnel, Teaching Assistant can be deployed as a completely MFP-resident solution, installed by the Kyocera autho-rized reseller, which essentially eliminates IT overhead.

From the teacher’s perspective, all steps can be completed from the control panel interface at the MFP. In addition, the solution offers schools the ability to print test forms flexibly on-demand using standard printer paper, thereby eliminating the need for the more costly preprinted OMR forms which need to be ordered in advance and stored.

Importantly, students will benefit from the solution: given that the feedback is provided essentially in real time after a test is administered, the teacher can act on the momentum and quickly put the information to use in providing feedback to individual students or the class as a whole. In addition, with the multiple analysis reports Teaching Assistant produces, teachers can quickly gauge which topics students (as a class or individually) have comprehended and which need further time spent on them and plan for upcoming classes. With the optional Teaching Assistant Workbook Excel plug-in—which enables teachers to create answer forms at a PC—the solution also supports inclusion of open-response essay-style test questions. Notably, up to 500 essay questions can be accommodated which is much more than the limited number supported by a few competi-tors. Moreover, the grades assigned by the teacher for essay questions can be conveniently included in the automated grading of the overall test and the subsequent analysis reports.

With its affordable price, terrific ease of use, ease of deployment, and handy analysis and reporting tools, Teaching Assistant will deliver great value to any K-12 school. Its ability to efficiently assess student perfor-mance, helping inform teachers on where to focus additional instruction, makes it well worth championing in these environments.

BuyersLab.com © 2016 Buyers Laboratory LLC. Duplicated by permission.

[email protected] • 973.797.2100

BUYERS LAB


Strengths• Streamlines creation and grading of multiple-choice test forms

• Robust options for creating test forms and specifying grading parameters

• Exceedingly easy for teachers to use at the MFP control panel—no server, cloud or classroom PC software required

• Efficient automated grading process eliminates manual grading’s potential for errors

• Test result reports provide analytics so teachers can identify which topics have been comprehended and which require more classroom attention

• Enables fast and timely turnaround of test results, keeping students more engaged

• Reduces teachers’ time spent on manual tasks and grading, allowing them to focus more on classroom activities

• Prints answer sheets on plain office paper; no need to order and stock expensive proprietary bubble-sheet forms

• Generally very good optical mark recognition (OMR) performance

• MFP-based solution eliminates IT overhead

• Very affordable compared to competing solutions

WEAKNESSES

• No integration with student information systems or electronic gradebooks

• Creation of test sheets with open-response answers and student IDs pre-filled requires Excel plug-in

• Kyocera’s Teaching Assistant offers fewer report types than other competing solutions

BuyersLab.com

Gerry Stoia, CEODeanna Flanick, CROBrian O’Connor, CFO Randy Dazo, Group Director, Office Document [email protected] Jamie Bsales, Director,Office Workflow Software [email protected] Marlene Orr, Director, Office Equipment Product [email protected]

U.S. ANALYSTS

George Mikolay, Senior Product Editor, A3 [email protected]

Lisa Reider,Senior Product Editor, Scanners and [email protected] Kaitlin Shaw, Editor, Printers/A4 [email protected] Lee Davis,Research Editor, Software [email protected]

Robert Watts,Research Editor, Software [email protected]

EUROPEAN ANALYSTS

Dr. Simon Plumtree, Senior [email protected]

Priya Gohil, Senior [email protected] Andrew Unsworth, Associate [email protected]

LABORATORY

Pete Emory, Director of U.S. Research and Lab Services

David Sweetnam, Director of EMEA Research and Lab ServicesCOMMERCIAL

Gerry O’Rourke,Director, BLI International Mike Fergus,Vice President of Marketing T.R. Patrick Art Director

Buyers Laboratory LLC • North America • Europe • Asia

BuyersLab.com ©2016 Buyers Laboratory LLC. Reproduced by permission.

BUYERS LAB


4

PRODUCT PROFILE

Versions Not applicable

Pricing $899 per device license; the Excel plug-in is included at no extra charge and can be installed on an unlimited number of PCs

Users Maximum number of users supported: Unlimited

Server Not applicable

Client Excel plug-in is compatible with Microsoft Office 2010 or later

Mobile Device Compatibility

Not applicable

Compatible Output

Hardware

Kyocera

Software Integration

The solution integrates with Active Directory for authentication and Microsoft Excel for advanced reporting features

Availability The solution is available for purchase through authorized Kyocera dealers in the U.S., Europe, Middle East, and Africa.

Languages English, Portuguese, Spanish, French



BUYERS LAB


5

Features & Productivity

Kyocera’s Teaching Assistant provides a solid feature set that should meet the needs of most K-12 school systems whose student performance evaluation framework employs multiple-choice tests. The solution delivers three core functions: answer-sheet creation, test grading and results analysis. Importantly, all three functions are available directly from the control panel of the MFP.

TEST CREATION EXCEL INTEGRATION

TEST GRADING SUPPORTS ESSAY QUESTIONS

REPORTING CUSTOMIZABLE TEST CREATION AND GRADING DEFAULTS

• At the MFP, teachers create customized test forms on demand, with the number of test questions permissible ranging from 2 to 500 and the number of answer choices available (AB through ABCDEFGH).

• Unlike some other systems, test forms created at the MFP cannot include an area to assign points to open-response essay questions. But to accommodate that, Kyocera includes a Teaching Assistant Workbook Excel plug-in component as part of the solution; this is a highly convenient feature which offers teachers the ability to create test answer forms at a PC directly within Microsoft Excel, rather than at the MFP.

• When teachers create the test form using the Excel Teaching Assistant Workbook plug-in, they are able to set up to 500 blank boxes for long-form essay answers. Once test results are imported from the MFP into Teaching Assistant Workbook, teachers assign grades to the essay-style answers and input the grades directly into the Excel spreadsheet before initiating the grading process; this allows the essay-style grades to be incorporated into the overall grading and analysis process. Essay questions can be weighted, which means one question can be worth more points than another.

• Test sheets created at the MFP cannot be personalized for each student with the student’s name, ID number, class name and other information. Instead, students will need to hand-print their names on the answer sheet and use the OMR number bubbles provided to enter their student ID number. However, teachers can personalize test forms with this and other information if they use the Excel-based Teaching Assistant Workbook utility to generate the answer sheets.

• Although each answer sheet has instructions at the top urging students to completely fill the oval and not to mark outside the ovals, educators can attest that such instructions aren’t always followed. So BLI used various marking implements and bubble-filling styles (including erasures) to test the mark-recognition accuracy of the OMR engine. Overall, Teaching Assistant did well. Most importantly, it correctly read the intended answers of all bubbles marked completely with pencil, red pen, black pen and blue pen. And it correctly recognized the intent of most (but not all) hastily made marks, such as incomplete erasures, checkmark fills, “doughnut” fills, partial fills, “x” fills and outside-the-line fills.

CONTENTS

Features/Productivity

Usability

IT Admin/Security

Support/Training

Value

Features and Productivity



BUYERS LAB


6

• When it comes to test-grading features, Teaching Assistant offers a great deal of flexibility. At the MFP, teachers can opt to have the system use the default A, B, C, D, F grade scale, or they can specify A through F with + and – increments. Unlike some other systems BLI has evaluated, Teaching Assistant also allows educators to specify on-the-fly the percentage thresholds that map to a grade increment. For example, by default 98 to 100 percent would map to an A+, but the teacher can change that at the MFP to be 95 or above would be an A+.

• Teaching Assistant also supports custom grade scales; for example, some schools may only use four levels (A,B,C,D), while others may want to replace “F” (which stands for Fail) with the less-judgmental “E” letter grade; numerical grading is also available as an option.

Teachers can customize grade scales and grade percentage thresholds at the MFP

• Also unlike other testing and grading systems that BLI has evaluated, analysis of the test results is handled by the MFP-embedded software, rather than in the cloud or by a server-resident component. This means tests can still be graded even if the MFP device is not connected to the network.

• The solution allows teachers to either print out the results, email them, export to Excel, or save them to a USB memory device. The availability of CSV reports is a bonus, as it enables teachers to import the data into a spreadsheet program for further processing—a feature which is not always offered by competing OMR systems.

• Teaching Assistant offers a variety of reports that help teachers understand how well the class and individual students are progressing, and identify subject areas that might need more classroom or individual attention. The reports the MFP-based component of Teaching Assistant produce are:

Student grade report: A full-size image of each scanned test sheet, with a grade (and percentage correct) printed in red at the top (assuming the results are printed

CONTENTS


Usability

IT Admin/Security

Support/Training

Value



BUYERS LAB


7

on a color-capable MFP). Incorrect answers are surrounded by a red rectangle, with the correct response printed at the end of the row. Educators can opt to not display the correct answer for incorrect responses.

Test Scores summary table: Shows student ID number, an image of each student’s handwritten name from the top of the answer sheet, the number of correct answers, the percent correct, and the student’s percentile rank.

Test Scores bar chart: A graphical representation of each student’s score plotted on a bar chart. The x-axis is the percentage of correct answers, and each bar is labeled with the total number correct for that student, the student ID number, and an image of his or her handwritten name. However, those images were cut off in some of BLI’s test samples, showing only the first 6 letters of what was written at the top of the answer sheet.

Histogram – Grades bar chart: A graphical representation of the percentage of students that received each grade (for example, 30 percent received an A, 10 percent a B, and so on).

Statistics report table: Basic statistics about the test questions and results, such as the lowest score on the test, the highest score, the average score and the median score. It also has tables showing the 10 hardest questions (a column with the question number and another with the number of correct responses) and the 10 easiest questions.

Response frequencies table: A table showing the percentage of students that marked a particular response for each question (for example, on question one, 80 percent answered “A”, 10 percent answered “B” and 10 percent left it blank). This helps an educator see which questions were missed most often, and hence which subject matter might need extra attention in class.

• When test forms are generated using the Excel plug-in feature, teachers have the ability to import the results directly into Teaching Assistant Workbook and generate an Excel report with multiple worksheets providing more granular and graphical analysis detail.

• While the solution offers a very good feature set, Teaching Assistant does not offer some features found in other testing and grading solutions BLI has evaluated. For example, while the selection of reports is good and delivers the core information educators might need, some competing systems offer even more reporting choices. Also, a leading hosted service offers integration with student information systems that Teaching Assistant lacks.

CONTENTS


Usability

IT Admin/Security

Support/Training

Value



BUYERS LAB


8

‘Test Scores’, ‘Test Scores Bar chart’, ‘Histogram’, ‘Test Results score sheet’, ‘Statistics’

Teaching Assistant offers a range of reports that can be printed, emailed or saved to a con-nected USB memory device.

Software Integration

• Kyocera Teaching Assistant integrates with Active Directory for authentication purposes.

• The solution dovetails with Microsoft Excel for exporting test results for enhanced reporting and analysis.

Usability

Intuitive to use, Teaching Assistant should not pose any issues whatsoever for teachers. They will appreciate that all core functions are accessed at the control panel of the MFP, whose menus provide clear instructions. Educators will be able to start using the solution immediately with minimal training.

• As noted, teachers have the choice of creating new tests at the MFP or from their workstation using the Teaching Assistant Workbook Excel component.

о At the MFP, the teacher presses the Application button on the MFP’s control panel, followed by Print Test. This brings up a basic screen offering various

CONTENTS


Usability

IT Admin/Security

Support/Training

Value

Usability



BUYERS LAB


9

settings for teachers to create their answer bubble sheets; it is a very straightforward process.

о Creating test answer forms in the embedded Teaching Assistant Workbook program in Excel is equally straightforward, and has the added benefit of enabling the teacher to input the student names, ID numbers and barcode information, and save the master file in case they need to print out additional sets for future class tests. In addition, when it comes to distributing tests, it allows for much smoother test conditions in the class, as students can focus on the test at hand rather than spend time filling in their credentials and worrying if they happen to fill in the wrong ID bubble numbers, and so on.

The Print Test applet allows teachers to create and print test answer sheets at the MFP.

• The ‘Print an answer key sheet’ checkbox will print out a blank master sheet on which the teacher can manually fill in the correct answer bubbles—this master is printed by default even if the box is left unchecked, so this particular checkbox option makes users take an unnecessary step.

• The ‘Print student ID bubbles’ checkbox feature will print a series of lines offering bubbles numbered 0 to 9 at the top of the first answer sheet for the student to fill in with their ID number. The Options button offers users the ability to adjust the number of ID bubble lines from five to eight lines.

• Teachers can customize the text which appears at the top of all sheets to be more meaningful and test-specific, via the Systems Settings button. By default, these are set to print the application name, “Kyocera Teaching Assistant” and software copyright details. (Note that any changes made are temporary and will revert to the default text every time someone uses the application anew. Changing the default printing and grading options requires administrative rights.)

CONTENTS


Usability

IT Admin/Security

Support/Training

Value



BUYERS LAB


10

• Unlike other testing and grading systems, communicating with server- or cloud-based software, teachers cannot select the particular class from a drop-down list presented at the MFP, so class and student information is pre-filled on the printed test forms. When ready, the teacher presses print on the touchscreen (or the hard key on the control panel pad), and the device will produce the desired blank answer forms and the answer key master, which is clearly labelled ‘For Teachers Only’. BLI would like to see the option for teachers to print pre-filled answer sheets so they don’t have to spend time filling it in themselves. This would also deliver a more random distribution of correct answers.

• Once the test has been administered, the teacher can use the same MFP to access the Grade Test program on the control panel. A key time-saving feature, it elminates the laborious task of manually grading tests. The student test forms, along with the answer key sheet, are inserted in the MFP’s ADF. The answer key and test sheets can be slotted anywhere in the stack; in BLI’s evaluation, Teaching Assistant recognized the answer key and multi-page tests no matter where they were placed. Then it’s a matter of selecting the Grade Test application from the MFP Applications area and hitting the “Scan” button to begin the scanning process. This is a very fast and efficient operation; in BLI’s testing, 25 answer forms took just under 40 seconds to scan; the grading process took another 41 seconds; so in less than a minute and half, the teacher can have all test forms marked and graded, and then be able to run reports providing in-depth analysis.

After grading test sheets, teachers can print the test results, email them or save them to a USB flash drive. Once they log out of the program, test data is auto-matically erased on the MFP to safeguard security.

• If test sheets were created using the Teaching Assistant Workbook program, once the MFP has scanned and graded the forms, the teacher can reopen their saved test file holding the class information and opt to import the results directly from the MFP (or import them from the USB device if they had used this option at the MFP).

CONTENTS


Usability

IT Admin/Security

Support/Training

Value



BUYERS LAB


11

• Once the results are imported into Teaching Assistant Workbook, the teacher can manually insert the essay question grades (if applicable) and then simply select the ‘Generate Reports’ option; this leads to the creation of a multi-tabbed Results worksheet, offering many different reports, including pie and bar charts. Usefully, teachers are able to sort the data to appear how they want, while the program prints out all data in a clean way formatted to fit the output paper, which makes it very manageable and easy for teachers.

Offering a range of test results reports, Teaching Assistant Workbook provides user-friendly graphical charts to aid with data analysis.

• BLI also noted some potential challenge for districts with multi-vendor fleets and shared printing environments.

о When printing from the Teacher Assistant Workbook, users must select an output device from a dropdown list, which inconveniently displays incompatible devices. If a user accidently selects an incompatible device, the solution does not alert the user, who won’t find out their job hasn’t been printed until they arrive at the device.

о Users cannot export graded tests to the Teacher Assistant Workbook directly from the MFP. Instead, they must select the “Import from the MFP” button. This is particularly inconvenient for schools operating in a shared print environment, as someone else can close the app before the teacher returns to her desk. The teacher will have to start the process over and hope that no one disrupts their data migration once again.

CONTENTS


Usability

IT Admin/Security

Support/Training

Value



BUYERS LAB


12

IT Admin & Security

Installation / Configuration

Kyocera’s Teaching Assistant can be up and running in no time at all. The reseller placing the system will typically handle installation and initial setup, which consists of loading the programs on the MFP and installing the Teaching Assistant Workbook Excel plug-in on a PC workstation. The technician will also set the SMTP server address and port number to enable the distribution of reports via email. (There is a Test SMTP button which validates SMTP settings by sending a test email to the Sender Email address.)

• Administrators can monitor the print usage by enabling the Account Code feature, in which case users will need to enter a valid code that corresponds with a code set up on the MFP. If all printing from the solution needs to be attributed to a single account code, then the administrator can enter this code in the System Settings page.

• Teaching Assistant offers the ability to restrict color use and set the grading of tests to be in black and white, otherwise color devices will grade papers in red by default.

• Administrators can configure the test printing default settings for Test Title, Test Information, the number of bubbles (for responses and ID bubbles), and whether to print an answer key sheet and student ID bubbles. Likewise, grading defaults can be set for which grading scale to use (for example, A,B,C,D,F) and the range for each grade (for example, 90 to 93 = A-, 94 to 96 = A, and 97 to 100 = A+).

Ongoing Administration

• Once the components are set and activated, a customer’s IT personnel will not need to administer the software on a day-to-day basis.

Security

Kyocera Teaching Assistant can be secured by requiring that users log in at the MFP before being able to access menu choices, while administrator credentials need to be entered in order to access the application’s settings. To prevent data from being compromised, test results are erased from the MFP when the user exits the program at the control panel

CONTENTS


Usability

IT Admin/Security

Support/Training

Value

IT Admin



BUYERS LAB


13

Support & Training

Customers typically call the reseller first for tech support issues; problems that the reseller cannot resolve can be escalated to Kyocera. Support is included in the purchase price, and customers are not required to purchase a separate support and maintenance contract to receive support.

• Intuitive and straightforward to use, Teaching Assistant does not require extensive documentation for end users. Kyocera does provide a comprehensive Set-up and Operation Guide in searchable PDF format that clearly explains and illustrates with screenshots how to configure and use the solution. Aimed at both administrators and end-users, the document is clearly written and features a hyperlinked table of contents and an index.

• A Teaching Assistant introduction video can be found on Kyocera Document Solutions America YouTube channel: https://www.youtube.com/user/Kyoceraamericas

• Support hours vary between countries but support is usually available during typical office hours.

• If higher-level support is required, the Kyocera Diagnostic Center is available by phone weekdays 9:00 a.m. to 6:00 p.m. Eastern Time (Tuesdays beginning at 9:30 a.m.) Email support is also available.

• Kyocera Document Solutions America, Inc. provides sales and service training, to Authorized Kyocera and Copystar dealers. In addition, KDA deploys pre- and post-sales business consultants and field support throughout North America and Latin America to assist Authorized Dealers when interacting with end-user customers.\.

Value

• Kyocera’s Teaching Assistant is sold on a device-license basis and carries a list price of $899, which is lower than that of competing MFP-based testing and grading systems.

• The Teaching Assistant Workbook Excel plug-in is included in the purchase price, and can be installed on an unlimited number of PCs for educators to access. This is in contrast to a leading third-party grading solution, which requires a license for each PC where the software is loaded.

• For educational institutions already using compatible Kyocera devices, or looking to invest in one, this is a cost-effective solution when compared to traditional standalone OMR solutions: It is less expensive overall to deploy, and school systems

CONTENTS


Usability

IT Admin/Security

Support/Training

Value

Value

Support and Training



BUYERS LAB


14

do not need to purchase and store costly blank bubble-sheet forms, since answer sheets are created on-demand using ordinary printer paper.

• Adding to the solution’s overall value proposition, the Kyocera MFPs purchased for use with Teaching Assistant can be used for general office purposes, unlike the single-purpose “bubble sheet” scanners that are used by traditional systems.

• The solution’s serverless architecture allows companies to forgo the frontloaded investment of purchasing, deploying, and maintaining another server.

• Maintenance and support is included in the purchase price, and there are no additional costs for the customer during the lifetime of the device.

• Teaching Assistant can increase productivity by saving teachers from the tedious, time-consuming tasks of creating and grading bubble-sheets.

• The solution delivers intangible ROI in terms of academic improvement.

CONTENTS


Usability

IT Admin/Security

Support/Training

Value



Documents

Solution Report - Document Management Made Easy · Solution Report Buyersa.com ... faxes and multi-function (MFP) ... TEST GRADING SUPPORTS ESSAY QUESTIONS REPORTING CUSTOMIZABLE