NYSED TEST SECURITY UNIT

STRIVING TO ENSURE THE INTEGRITY OF THE NEW YORK STATE TESTING PROGRAM

Tina Sciocchetti, Executive Director Test Security and Educator Integrity New York State Education Department May 31, 2013

Test Security Unit Genesis

“We are relying more than ever on stateexams. ... If we’re going to use the testsin these ways, we need to be absolutely certain that our system is beyond reproach ... [and] ensure thatour tests are not compromised in anyway.”

Regents Chancellor Merryl H. Tisch

USES OF TEST SCORES

Measure student achievement

Trigger remedial services

Assess and improve a school’s curriculum

“Data-driven instruction” designed to tailor instruction for student

Teacher and principal effectiveness (APPR)

Bonuses, job retention, tenure decisions

Competitive grants, honors and waiver programs

School accreditation, take-over, closure

How Can Test Scores Be Compromised by Educators?

1. Retaining 3-8 tests and field tests, using for next exam 2. Opening test materials before designated time 3. Distributing cheat sheets, formulas, definitions before test 4. Giving a student more time to take test 5. Defining words, concepts, or questions, on test 6. Using voice inflection or physical gestures to cue answers 7. Coaching a student during testing 8. Allowing a student to alter exam answers after test handed in 9. Altering a student’s answers after test handed in 10. Filling in blank bubbles left by students 11. Adding points to a student’s test score to help student pass 12. Altering a student’s test score during recording or reporting 13. Coordinated exclusion / suspension of students with poor

academic performance 14. Encouraging low performers to stay home during testing

The Stakes Are High, But Cheating Cannot Be The Easy Way Out

• Good Moral Character supporting certification requires honesty and integrity

• Educator test fraud poses serious consequences for affected students, schools, colleagues, and the educators themselves

Educator Cheating Hurts Students

Undetected Cheating Parents have invalid

proficiency measure Loss of vital support

services for low performers Improper grade-to-grade

promotion Improper diploma issuance

Cheating Detected Invalidations of scores May be forced to retake test Delay in diploma issuance

Educator Cheating Hurts School

Undetected Loss of funds to support low

performing students Invalid assessment of

progress and weaknesses in curriculum and instruction

Detected “Focus” or “Priority”

designation due to invalidation of scores (SINI)

School barred from testing or storing privileges, if widespread

Loss of reputation for school, students, and graduates

Diploma value deflated

Cheating Hurts Colleagues

Inherit low performing students unassisted by tutoring or services that had been warranted

Marked drop in scores following year unfairly reflects poorly on them

Cheaters may receive higher ratings that unfairly penalize honest, ethical colleagues

DETERRENCE IS TSU GOAL

“The penalty for cheatingmust always be more severe, morepredictable, and moreimmediate, than any consequence for not meeting a goal orobjective.”

Erroll B. Davis, Jr. Superintendent, Atlanta Public Schools

TEST SECURITY UNIT STAFF

Two seasoned prosecutors

Four investigators with combined 85 years of state and federal law enforcement experience

Dedicated IT support to create on-line, web-based processes and electronic “tip line”

Test Security Unit“ETHICAL” Approach

Evaluate and Improve Current SED Policies Train Educators How to Do it Right Help Districts Comply Investigate Allegations Crack Down on Misconduct Announce/Publicize Activities Level the Playing Field

Evaluate and Improve SED’sCurrent Testing Policies/Practices

Continuously evaluate SED testing policies/practices

Recommend and implement security improvements

Make effective use of SED scoring audits andforensic methods (erasure analysis, spike clusteranalysis, audit rescoring, marked drops/gainsanalysis, department review of scoring)

Comprehensive case tracking system to study test security trends over time

llHE STA"IE IEDUCATIDM DEl"ABTMElllf I THE UNNE.RlllY DF THE .STiii.TE OF flEW YORK ' Al.IMll¥. N't 12ZJ!ll

TEST .SECURITlf Ul!llT

To: [)istrict SUpel'intendents Superintellldents of SChoolls Principals of PUbCic, Nortpublrc, and Charter Scilools

.... / -----~ ~ ..£./'_ Rrom: Tina Sciocchetti, Execu:tirve D irector _A~- -----:;:r;.-.~~~-

Re:. Pro'hi'bition of Cell Phones ar11d ElectFonic. De1~,ices i n New Yolil<: Slate Assessments

[)ate: July 12, 2012

Currently, sludems are pro'hibited fium using oelm phones and other- communications deVices while taking New Ymk state assessn11ents_ HoweveJr, state Educatiorll Department policy does not !Prohibit studems from possessing lihe devices in exam loc:a'tlions_ To further ensure the integrity of Sta.'tle assessments, tile rummt IX)licy is being enhanced to prohi'bi:t aH stl!Jden.ts from bringing ceCI phones and certain olher electronic devices into a dassroom a o1llef' lbcalion where a New Yor1t State assessme1111: is being admini ster-ea_

Pro'hi'bi'tedl devices indude. lbctt are not !limited to: • OeH phones • BladtBerry devices and other- PDAs • iP.ods am::I MP3 players • iPads, tablets, ar11d other 1eReaders • Laptops, notebooks or any other personal co~ng devices • cameras or other- photog:~hic eql.lli;pment • Headphones, headsets .• or 1in-ear he~horles sucti as earbuds • Any deVice capable of recoming audio , pootograpruc 1CJI' villeo content, 1CJI' capable of

viewing orr pllay,ing back suoh content

Test proctors, test monitors, am::I sdlool officials, shall nave the rigll.t to 1coUect cell phones am::I other prohibited e'ledmntc deVices priCll' to th e start of the test and to hold them while 1he· test is being admin istered], irmJuding1 break petiiods_ Adrnissm to the test. shaCI lbe denied to any student wtio refuses to relinquilSh a prollmited de-v:ioe ..

Some students with disabilities 1may use certain reoominglpfiayback devices provided ~hat such an accommodation is specified i:n the studenfs IBP or 504 Plan_ Prohibited devices fi.utller may be al towed iif tllere is documentation fium a medical practitioner on file at t he school that a studernt requires such a deVice dmtng testing_ In a ll other cases, tile prohibition as provided above remains in effect andl tile stu::lent. may not enter the exam room with any· IJl"Ohfbitedl device_

llhis policy is effective immediately fnr a ll New Ymk State assessments, i:noluding Regents Exams, Regents Competency Tests, Grades 3-8 Tests in EngJPsh Language Arts and Matllematics, Grades 4 and 8 so·ienoe Tests, NYSESLA T, and tile NYS Alternate Assessment llhe Directions tor Adrrrnistering Regents :Examinalions for August 2012 and tile School Administrators Manual, 2012 Edifftm, will be amen::ledl by July 31, 2012. to incorporate this new ceo phone anid etecllonic devices policy_

Train Educators About ProperTesting Policies/Practices

Distribute clear instructions about testing protocols with specific content-based examples of prohibited conduct

Publish clear and unequivocal warnings as to consequences of engaging in prohibited conduct

School visits during testing, increased monitoring

Webinars, e-based training

Testing code of ethics, proctor security oath, mandated reporting of test fraud

IMPORTANT BEIFORE YOU AD1'MNISTER. THE JUNE 2012

REGENTS EXAMS O:R REGENTS COMPETENCY TESTS (R.Grs) PLEASE REVIEW THIS IMPORT ANT TEST SECURilY INFORMATION

The State Educ.a1ion Department is de,·ofing greater attention fo the ft'CiUli'ty ,and integrity of the New York State Testing Progt-am. "Tii e appreciate your commitment and pt-ofessionalism. as we stri'\"e to ensutre the most fair administration and! S<"oling ofRegents Exams and RCTs. Ple-ase accept this ttminde-t· of pro,hi.bited testing condud.

Some exam.ple.s: of prohibited te"51ing eonduct inclu.de:

• Git.ling a student more time to "take the test than is allowed for that student under State regulations-• Defining or explaining fo:r :a stl!ldent words, 1conoep1s, or 1qnestions, contallied in the test. • Suggesting answe:rs to a student during testing or 101b.envise ooaclllng a student dm:ing tes:ting. • Commenting on an am;,wec a student bas pro.vided on 1lhe aos\11trer sheet while the test is i.n p:rogress. • Allowing :a :smdent- to aJtec exam answers after the student bas handed in hisJ'be.t" rest materials. • Altering a studeot7

S a.Jlfi\'el:S after die studleot bas !banded in :hi.slbeF test materi.ak • Jimpl!'operly mflating a student~s valid. eamedl test score in order to help !the smdent pass. lthe exam.. • !i.Wll:Jg :my attempt to improve a student~s rest soore dming scoring, l!'ecording. or:reporting..

Some potential eon.sequences of engaging in pt-ohi.bited ~esting condo.ct include:

• The test saue of11he affected smdentwill likelybe i.nvalidatedL • The ,affected! student may have to retake the ,exam_

• The )>fi"Son who :intentionally engaged i.n. testing misconduct could :face sanctions and discipline, :imclwling tennination and/or the loss ofmslherNev.' Yo~ State teacher ceiti.ficati.OliL

Ple..ase 1·epo1"t any "\iolation e f the t.e;.5fii.ng rules to your admini'Stratior and the Department.

* Full i.:osrtruc1ioo.s concerning· the admiDiRh".ation and scoring of eiam.'5 are contained in Dfrection s: for Adminiff:eti11g Rege,1.1ts Evaminalio1M, .Tm1e 101?. Adminisluiliou, and on the Dep.artment"s website at llittp:l/www.p l?.m "'Sed.go\•/.apda/hsgen. Any pet-son .admini stering a Regents ham or RCT must .abide folly by these instructions . The examp:les proTI.ded abm·e are si:mply illnstrari,-e of prohibited testing conduct.

Test: Sec:ur-it:y and Educ:at:o..- Int:egr-it:y ~

About: U a Report Educato r T eat Freud F inge rprint:ing T e acher D i sci pli ne Re .sources for Educato r:s

TSU Polic:i e:s

TSU Guidanc·e

NYSED tfews

I ntegrity 0-fficers

NVSEO / 1-fiigh•r Ed / TSEt /TSU Guidanc- / T i ps for E thi cal Tes t i ng

Tips f or El:hi cal Tesl:i ng

All educat:or-s should st:.-ive t:o adn~inist:e.- St:at:e assessment:s in t:he most: et:hical and honest: manner . E thical testinQ ensur es that test scores are not: compromised, promotes a fa i r

tes ting process, and avoi ds serious conseQuenc es t:hat: may st:em from cheating. Unet:hical

t:estino conduct: .-esult:s in the invalidation of t:est: s c o r es, imp.-oper assessment: of student:

proficiency, loss of vita l services for low performers, unfair advantage o v er colleagues, and

pr-ofessional disc ipline such as t:ermination and loss o f ce.-t:ific at:ion. Avoidino t:he followino pr-ohibit:ed conduct: in the admini strat:ion of New vo.-k St:at:e assessments will demonstrate

y ou.- commitment: t:o ethical testing pr-actices and will enhance t::he int:egrit:y of you.- school's

t:est: r esults.

Before Test:lno DO NOT:

1. Access secur-e t:est: booklets and answer- sheets prior- t:o the time allowed by st:at:e

rules;

2 . Copy, .-ep.-oduce, o.- keep, any part of sec u.-e e x am materials ; o.-

3. Review t:est: booklet:s in order- t:o:

a. Deter·mine and record correct responses for use during testing.

b_ C reat:e pre-t:est: lessons or d isOJssions \Nit:.h sb...Jdent:s about: concept:s being

t:est:ed. c . c.-eat:e a - cheat: sheet:- fo.- students t:o use in taking the t:est:, including sharing

of formulas, concept:s, o.- definitions, necessary for t:he test:.

During Test:lno DO NOT:

1 . Give sc:-udent:s any clues o r answers, induding:

a. Coaching student s about: proper- answer-s . b. Defining terms and conc epts included in t:he t:est:.

c . Pointino out: wrono answers t:o a st:udent:, and suooestino that: t::he student: recon sider- o.- change t:he .-ecorded response .

d . Reminding students during testing of concepts t:hey learned in dass.

e. Making facia l or ot:he.- non- vernal suggestions .-ega.-ding answe.-s.

2 . Allow any student: more time t:o t:ake t:he t:est: than is a llowed for t:hat: student; or

3 . Leave any materials displayed in t:he room containing topics being t:est:ed o.- write on

t:he boa.-d formulas, con cepts, or definitions, necessary for t:he t:est:.

After Te.sting DO NOT:

1. Review an answer- sheet: for- wron o answers and .-et:urn it: t:o a student wit:h

instructions to change or- r-econside.- the wrong responses;

2. Alter , er-ase, or i n any other way change a st:udent:' s recor-ded r-esponses a fter- t:he

student: has handed in h i s/her test: materia ls ; or

3. Re-scor-e portions of t:he test in or-der to add points so t:he student: will pass .

These ar-e e x ampl es of pr-ohibit:ed t esting conduct. For the full instructions for- the

administration and scoring of st:at:e e x ams go t:o:

http://www.p1 2.nysed .gov/assessmentjma nuals/home.html.

L~st Updwtcdi ~~nu.::ary 22,,. 2013

Help Districts Comply with Rules

Integrity Officer Model – 37 BOCES and Big 5 designees – Dedicated TSU Investigator for each IO – Provide specialized training and investigative support – Pipeline to districts for training materials

IO model allows districts to monitor conduct, provide remediation (“corrective action plans”)

Move from paper-based to web-based system to provide on-line incident reporting process, electronicforms and guidance

3

-(NYSED J

Test Security and Educator Integrity --

About Us Report Educator Test fraud fingerprinting Teacher Discipffne Resources for Educators

About TSU

About OSPRA

omce of State Assessment

Press and Public Reports

NYSED I Higher Ed I TSE! I

Welcome

The New York State Education Department's Office of Test Security and Educator Integrity

consists of t wo component units:

• the Test Security Unit (TSU) and

• the Office of School Personnel Review and Accountability (OSPRA).

Both units, which are comprised of professionals possessing legal and investigativ e

expertise, as well as support personnel, report directly to the Executive Director for Test

Security and Educator Integrity.

Contact I Office of Higher Education

University of the State ofrlew York - New York State Education ~partment

Contact NY SEO I Index A - Z I Terms of Use

Last Updated : January 17, 2013

I est: Securicy and m=ducat:or Int:egricy ~-

About Us

St-udent Ch~~ting

Mis.administ:.r3.tion and T•c.ti ng Errors

T•~~r Discipl ine- R•s.our~s for Ed~tors

Report Educator Test F r aud

T h i s on linie: i ncident: reporti ng process h;:11i s b e an d8Si gned t.o p rov ide i ndi v i d u a l s. a means oF r@porti ng

possi b l e t:est:i ng m i sconduct: b y educ.at:ors ( teachi ng personnel and admi n ist:rat:ors) re lated t:o New Y ork

State assessments . ALL such testi ng I rregul ari t i es must: be reported t h rough t he submi ssi on of an

Incident Report Form . Reporti ng p•rti es s h o u l d compl ete a ll porti o n s of t h e I n c i dent Report For m v .rith

c l ear, ac:,cur-ate i nfor mati on, and :shoul d pr·o v ide contac1: i nformati on i n t h e event that f o llov1 up i s

T hoe [ncidPnt RPport Fqrm s hou l d b e used onl y to r e port a ll@gations concer n i ng edu cator m i s.condu ct

and/ or cheati ng rel ated t:o N ew York St:at:e assessments . I n f ormati on about: student: cheati ng o n s t:ate

asse-ssments o r about i nstance s of m i s takes and d i ffi cu l t i es i n t he p .-octor i ng, admi n i s trati o n and

scori ng of st.at:·e assessments shou l d b-e: referred t o t:he: Offi ce of S tate Assessm ent v i a fax : ( 518) 474-

1.989.

Exampl es of possi b l @ t@st: fra u d b y edu cators i ncl u d@ , but: are not: l i m i tie:d t.o ... t:he follov.ing :

G i v ing a s tudent: more t i me t:o take t h e t:est: t:han I s a llowed for t h at: sl:udent: under State

r egu l ati ons .

Def i n i ng o r e x p l a i n i ng for a student v "'°r ds, concepts, o r questi ons, con ta i n ed i n t h e test.

Su99esti ng ansv .rers to a student or ot:h erv.-ise coach i ng a student: dur-i ng te-sti ng .

C.Ommenti n g on an ansv "'9r a student has p r ovided on t he ansv.Jllllilr s:heet v A-. il e t he tvs ·t: i s i n

p rogr-ess .

• A llovJi ng a s t u dent: t:o a lter e.x.am a n s-v.11e.rs. after t he s t:u de:nt: h as h a nde:d i n h i~/he:r t e.st: m a te r i a ls..

A l t:erlng a st:udent:'s answers aft:er the student has handed I n h i s/ h er t:est mater i a l s .

I m p r operly i n f l ati ng a s tuden t "s va l i d , ear n ed tes t scor e i n orde.- to hel p t h e student pass t he

e >eam.

Maki ng any attempt to i mprove a student' s test score duri ng scori ng, re-cor-d i ng, or reporti ng.

* NOTICE *

• The Nev,, York State Education Department v <ii ll hand l e a ll of the i nformati on subm itted i n the

Inci dent" Report Form i n accordance v.ri t h a ll appl icabl e state and federa l l aY.s: and regu lati ons

; nc:l ud;ng the Federa l Family Educ.ati onal R ;ghts a n d Pri v acy Act ( FERPA) .

• You may be cont.actgd b y a mamb-Qr of the TSU 3Fter you subm i t tha I n c i d e nt R Ga p o rt For-m .

• Your I n c ide nt R e port For m may be forv.rarded to another- office or agency fo r handl i ng i F the

i nci dent you reported pE!rta i ns to matt@rs: outsi de of TSU ' s: arE!a or r.esponsi b i l i t i es.

F or additional i nformation contact:

Test: Securi ty Un i t:

89 \/Vashingt:on Avenue

A l bany. N e v1York 12234

t;su@mall . nysed . gov

fax: ( ~1.S) 473-~261.

Investigate Allegations of Misconduct

Rigorous, professional investigations with full regard given to educator rights and privacy

Support IO investigations in districts – Develop investigative plan – Participate in interviews – Conduct forensic analysis – Help obtain documents and testing expertise

Take lead in investigation of serious or wide-spread test fraud

Crack Down on Educator Cheating

3020-a – support districts in investigations involving educator test fraud, testify

Part 83 – more aggressively pursue moral character actions (revocation or suspension of certification, fines up to $5,000, mandatory continuing education/training)

Develop additional penalties for cheating

Announce/Publicize Activities

http://www.highered.nysed.gov/tsei/

Public reporting of TSU’s activities, including confirmed test fraud cases

Public disclosure of SED testing audits, as appropriate

Districts will be required to report on test integrity issues, including disclosure of confirmed allegations

Level the Playing Field

“The rewards for cheating and the rewards for working hard must never be the same.”

Kathi King, Board Member National Assessment Governing Board, Math Teacher, Messalonskee High, Oakland, ME

TEST SECURITY UNIT

STRIVING TO ENSURE THE INTEGRITY OF THE NEW YORK STATE TESTING PROGRAM

Tina Sciocchetti Executive Director Test Security and Educator Integrity New York State Education Department May 31, 2013