Upload
others
View
6
Download
0
Embed Size (px)
Citation preview
PROFICIENCY TESTING MANUAL
1 General Customer Information .................. 3–10• OrderConfirmation ...............................................3• RegulatoryReporting ............................................3• Binders/Glossaries ...............................................4• KitDeliveryandStorage .......................................4• CAPIdentificationNumber .....................................4• ReplacementSpecimens ....................................... 4
• TestingInstructionsandCompletionTime ..................5• SecondInstrumentReporting ..................................5• CorrectionstoResultsandCMSReporting
Instructions .........................................................6• Fax-BackResponsePrograms ................................7• Evaluations .........................................................7• CorrectionstoResults ...........................................7• CustomerSupport ................................................7• ProgramCertificates .............................................8• LimitationsofPT ..................................................9• LaboratoryAccidents ............................................9• LimitationsofPTLetter ....................................... 10
2 Result Form Data Entry ..............................11–13• Overview .......................................................... 11
• PreprintedMethodSummaryPage ........................ 11
• ExceptionCodes ................................................ 12
• MasterLists ...................................................... 12
• ReportingBoxesandDecimalPrecision .................. 13
• “LessThan”or“GreaterThan”Values ..................... 13
3 e-LAB Solutions Suite (ELSS) .................. 14–16• GettingStarted .................................................. 14
• OnlineDataSubmission ...................................... 15
• OnlineReports .................................................. 16
• PerformanceAnalyticsDashboard ......................... 16
Table of ContentsProficiency Testing (PT) Manual
4 How to Interpret Your Evaluation ............. 17–20• GeneralGuidelinesforEvaluation .......................... 17
• SelectionofaTargetValue ........................................... 17
• CalculationofSummaryStatistics .......................... 18
• ComparativeStatistics ......................................... 20
5 Continuing Education (CE) ....................... 21–22• ContinuingMedicalEducation(CME)Category1 ...... 21
• CEforNonphysicianLaboratoryPersonnel .............. 21
• CEforCytotechnologists...................................... 22
6 Discipline-Specific Reporting Information ................................................. 23–25
• HowtoCompletetheResultForm .......................... 23
7 Evaluations and Participant Summaries (PS) .............................................................. 26–29
• Overview ..................................................... 26
• ReviewingYourEvaluation .................................. 26
• PS .................................................................. 28
• HowtoPerformaSelf-Evaluation .......................... 29
8 Laboratory Legislation and Centers for Medicare & Medicaid Services (CMS) Reporting .................................................. 30–37• ProvisionofResultstoCMSandStateAgencies ....... 33
• UseofReasonCodesforNonevaluatedSpecimens .. 35
• CMSPerformanceSummary ............................... 36
• CMSPerformanceSummaryData ......................... 36
InterlaboratoryComparisonProgramCycle
1 Catalog Delivered
September:Thecatalog issenttoyourlaboratory.
2 Orders Completed
September–December: Yourlaboratoryplacesitsorder.
3 Orders Processed
September–December: Orderquantitiesarereserved.
4 Order Confirmations Sent
September–December:Confirmationreportsaresenttoyourlaboratoryafteryourorderisprocessed.
5 Kits Mailed
Kitsarepreparedandsentfromthemanufacturertoyourlaboratory.
6 Results Submitted
CompletedresultformsarereturnedtotheCAPwheredataaresummarized.
7 Evaluation Criteria Reviewed
Thescientificresourcecommitteereviewsresultsandtheimpactofevaluationcriteria.
8 Reports Available
Reportsaresenttoandmadeavailableonlineforyourlaboratory,regulatoryagency,and/orconsultants.
9 Certificate of Participation Provided
Atthebeginningoftheprogramyear,yourlaboratoryreceivesacertificateofparticipation.
Proficiency Testing Manual College of American Pathologists 3
General Customer Information
General Customer InformationOrder ConfirmationAfteryourlaboratory’sorderisreceived,anorderconfirmationissentthatcontainsthefollowinginformation:
• Shippingaddress
• Billingaddress
• Telephoneandfaxnumber
• Listofprogramsordered
• Listofagenciesand/orconsultantstowhomyouhaverequestedcopiesofyourevaluationbesent
Reviewtheconfirmationdocumentcarefully.Ifyouhavechanges,returnthedocumentwithintwoweeksofreceiptto:
Mail: CustomerDataManagement CollegeofAmericanPathologists 325WaukeganRoad Northfield,Illinois60093-2750
Fax: 847-832-8168(countrycode:001)
Regulatory ReportingTheCAPwillautomaticallyforwardresultsforanalytesregulatedforPTtotheCMSforlaboratoriesthathaveprovidedaClinicalLaboratoryImprovementAmendments(CLIA)identificationnumber.
Tomakechangestoyourlaboratory’sanalytereportingselections(ARS),contacttheCAPat800-323-4040option1oraccessyourreportonlineatcap.org.
Documentationwillberequestedandmaybefaxedto847-832-8168.Anexplanationofregulatoryreportingandcurrentlaboratorylegislationisincludedinsection8.
Binders/GlossariesToprovideyouwithaneasywaybywhichtostoreyourSurveysprogrammaterials,foreverynineeligibleprograms,onethree-ringbinderwithtabsandcustomlabelswillbesenttoyouatnocharge.Thereisnoneedtoorderthebindersastheywillbeshippedautomaticallyuponplacementofyourorder.Additionalbindersareavailableforpurchasethroughtheonlinestore.
Proficiency Testing Manual College of American Pathologists 4
General Customer Information
Ifyouareenrolledinhematologyand/orclinicalmicroscopySurveyscontainingphotographsformorphologicidentification,youwillbeabletoaccessanonlineglossaryoftermsforyourgeneraluse.Ifyouneedahardboundcopy,calltheCAP.
Kit Delivery and StorageRefertotheshippingscheduleandadviseyourreceivingdepartmentaccordingly.Ashippingscheduleisprovidedtoyourlaboratorywiththeorderforms,oryoucanobtainacustomizedshippingcalendarbyloggingintoe-LABSolutionsSuite(ELSS)atcap.organdclickingonMy PT Shipping Calendar.
ThereceivingdepartmentofyourhospitalorlaboratoryshouldbeadvisedhowtohandletheCAPkits.Insistonprompttransfertothelaboratory.Kitsreceivedbythehospitalbutnotdeliveredtothelaboratoryarenoteligibleforfreereplacement.Unlessotherwisespecified,storethespecimensintherefrigerator.
CAP Identification NumberEachSurveyparticipantreceivesaCAPidentificationnumberthatisprintedonallresultforms.Thisnumberwillalsoappearoneachevaluationreceivedbyyourlaboratory.ItishelpfultohavethisnumberavailablewhencontactingtheCAP.
Replacement Specimens Kitsmaycontainspecimensforanalysis.Checkthecontentsagainstthekitinstructionsuponreceipt.Ifthekitisincompleteorcontainsbrokenorunlabeledspecimens,contacttheCAPassoonaspossibleforafreereplacement.
Intheeventthatareplacementspecimenisrequired,retainyouroriginalresultformwhileawaitingthearrivalofthereplacementspecimens.Thereplacementspecimenswillbesentinthesamemannerasyouroriginalspecimens.Whenyoureceivethereplacementspecimens,itisimperativethatyoucompletetestingassoonaspossibletocomplywiththeoriginalduedate.Duetoregulatoryrestrictions,aduedateextensionmaynotalwaysbepossible.Occasionally,itmaynotbepossibleforthemanufacturertoreplaceyourspecimen(s)orkit.Inthiscase,filltheexceptioncode33bubbleontheresultform.Anotewillappearonyourevaluationreportandyouwillnotbepenalized.
BecausePTmaterialsmustbeprocuredinadvanceofshipment,onoccasion,additionalinventoryisavailableforanominalfee.AdditionalmaterialmaybepurchasedafterthecloseoftheformalPTevent.
TopurchasethesematerialscontacttheCAPat800-323-4040option1.Allliteratureassociatedwiththisproductincludingthesummarydatawillbeprovided.This option does not replace routine PT.
Thesematerialsmaybeusedforbutnotlimitedto:
• Competencyassessment
• Instrumenttroubleshooting
• Training
• Education
• Research
Proficiency Testing Manual College of American Pathologists 5
General Customer Information
PT Referral InformationPerCLIA,aspublishedbytheUnitedStatesFederal Register,PTspecimensmustbetestedwiththelaboratory’sregularworkload,usingroutinemethodsandtestingthePTspecimensthesamenumberoftimesitroutinelytestspatientspecimens.
WhenhandlingPTspecimens,laboratoriesmustnotcommunicateresultsnorshareorreferspecimensfortestsnotonthelaboratory’smenu.Ifreferralfortestingisroutinelyperformedforpatientspecimens,thepracticecannotbefollowedforPTspecimens.ReferralisconsideredtobemovementofthespecimenfromalaboratorywithaCLIAidentificationnumbertoanotherlaboratorythathasadifferentCLIAidentificationnumber.LaboratoriesmustensurethatpersonneldonotshareresultsorreferPTspecimensforanyreflexortestingoutsidetheirCLIAidentificationnumber.
Results Due Date TheSurveysprogramisusedforcertificationofcertainlaboratories.Sincepromptnessisconsideredindeterminingcertification,wecannotacceptlateentries.Results must be received at the CAP no later than midnight, Central Time by the due date on the result form.Resultformsreceivedbeyondthedatenotedwillnotbeevaluated.Participantswillreceiveanevaluationindicatingthattheresultswerereceivedpasttheevaluationcut-offdatealongwithaPSthatcanbeusedforself-evaluation.
Second Instrument ReportingIn2014,theCMSdirectedallPTprovidersthatlaboratoriessubjecttoCLIAregulationsarenotpermittedtotestPTspecimensonmultipleinstrumentsunlessthatishowthelaboratoryroutinelytestspatientspecimens.
Recently,theCMScommunicatedtoapprovedPTprovidersthatthis directive applies to all analytes,includingthosenotlistedinSubpartIoftheCLIAregulations,aswellasanalytes/methodscategorizedaswaivedtests(suchaswholebloodglucosemeters).
Becausethesanctionsaresevereiftheregulationsarenotfollowed,theCAPPTprogramnolongerincludestheoptionforsecondinstrument.Tomeetyourlaboratory’sneeds,theCAPnowoffersitsQualityCrossCheckprogram.SeethecurrentSurveysandAnatomicPathologyEducationProgramscatalogortheRegulatoryInformationpageviacap.orgforfurtherdetails.
Corrections to Results• Correctionstoresultscanbemadeanytimepriortotheduedatelistedontheresult
form;reviewyourlaboratorydatacarefullypriortosubmission.Forresultsapprovedonline,correctionsmustalsobemadeonline.Faxedormailedcorrectionswillnotbeaccepted.
• Toviewthestatusofyourresults,gotocap.organdclickonSubmit/EnterPTResults.
Proficiency Testing Manual College of American Pathologists 6
General Customer Information
• Foranytestingthatyoudonotroutinelyperforminyourlaboratory,leaveallreportingareasforthattestblank,includingmethodinformation.ExceptionsarenotedintheCriticalReportingInformationsectionofthekitinstructionsorontheresultform. Note:Apenaltywillnotbeappliedforblankresponsesinthecaseofeducationalchallenges,challengesnotformallygraded,ortheproperuseofexceptioncodes.
CMS Reporting InstructionsForanyregulatedanalytesthatyourlaboratorydoesnotreportormayhavediscontinued,youmustupdateyourreportingsettingsviatheonlineCMSAnalyteReportingSelectionstoolviaELSS.Additionalregulatoryinformationcanbefoundviacap.orgontheProficiencyTestingpageunderLaboratoryImprovement.Ifyouhaveanyquestions,callthecustomercontactcenterat800-323-4040option1.
Fax-Back Response ProgramsTheCAPoffersimmediate(fax-back)responsesforthesubmissionofthelaboratoryformsforthefollowingeducationalanatomicpathologyprograms:
• InterlaboratoryProgramsinGynecologicandNongynecologicCytopathology(PAP/NGC)
• Fine-NeedleAspirationGlassSlideProgram(FNAG)
Individualresultformscanbesubmittedonline(preferredmethod)orfaxedtotheCAP.CME/CEcertificatesmaybeself-claimedviacap.orgbyclickingontheLearningtab.
EvaluationsEvaluationsarepostedonlineandmailedapproximately2–5weeksaftertheshipdateofthekit.Thistimeisneededforprocessingdata,establishingevaluationcriteria,andpreparingtheparticipantsummary(PS).
Customer SupportRegularHours:Monday-Friday,7:00am –5:30pm CT
ExtendedHours:Monday-Friday9:00pm –5:00amCT(emailonly)
Contact us to:
• Updatecontactinformation(telephonenumber,emailaddress,faxnumber,oraddress)
• Receiveadditionalsupportinginformationordocumentsfororderingorimportingpurposes
• AskaCAPexpertquestionsaboutaspecificCAPprogram
Phone: 001-847-832-7000option1
Email: Gotocap.organdselectCONTACT & SUPPORTatthebottomofthehomepage.
OR
Proficiency Testing Manual College of American Pathologists 7
General Customer Information
Laboratory Accident Hotline:
AccidentalexposureswhileprocessingspecimensfromtheCAP,includingcutsandcontaminationofmucousmembranesornon-intactskin,shouldbereportedtotheCAPhotlineforevaluationandquestionsregardingprophylaxis.Pleaseprovidetheidentificationnumberoftherelevantspecimen.
Phone:
001-847-832-7000option1 DuringContactCenterhours
001-847-470-2812 Allotherhours
IncountrieswheretheCAPhasadesignateddistributor,directlycontactyourCAPdistributorifyouhavequestionsrelatedtopricingoranyotherinquirynotlistedabove.
YourlaboratorywillbenotifiedofkitshipmentviaCAPTRAKerSMandemail.Ifyoudonotreceiveyourkitwithin7–10calendardaysfollowingtheshipdate,orifyoureceiveanincompleteordamagedshipment,callthecustomercontactcenter.Requestsforreplacementsafterthe10thcalendardaycannotbeaccepted.
Fordocumentacknowledgment,includetheappropriateemailaddressonthefirstpageofyourorderforminthesectiontitled“PTShippingContact.”
Tochoosenottoparticipateinthisprogram,participantscancontacttheCAPat800-323-4040option1.
Program CertificatesAfterthecompletionoftheprogramyear,participatinglaboratorieswillreceiveacertificaterecognizingeachinstitution’sparticipationintheCAPPTprogramanditscommitmenttopatientcare.CertificatesaresignedbytheCAPpresidentandaresuitableforframing.
Deficiency ResponseTheCAPdoesnotrequirethatyousubmitdocumentationforallPTdeficiencies.However,itisrecommendedthatsuchdocumentationberetainedinyourlaboratory.TheCAPLaboratoryAccreditationProgramissuesaseparatereport,the“ProficiencyTestingExceptionSummary,”thataddressesdeficienciesforCAP-accreditedlaboratories.Instructionsforresponsewillbeincludedwiththereport.
Handle With CautionPTspecimensmustbehandledwithcaution.Eachshipmentincludesabiohazardwarningstatementexplainingproperhandling.
Laboratory AccidentsIncidentsofpersonnelexposuretoinfectiousspecimens,throughneedlesticks,contaminationofthemucousmembranesthroughsplashesoraerosolization,orcutsfromcontainers,shouldbereportedimmediatelytotheCAP.
Proficiency Testing Manual College of American Pathologists 8
General Customer Information
24-hourhotline:800-443-3244
Havethefollowinginformationavailable:
• CAPnumber
• Phonenumber
• Nameofinstitution/city/state
• Nameofpersonaffected,ifotherthancaller
• Dateandtimeofincident
• Whereandhowaffected
• Surveyandspecimennumber
• Nameandtelephonenumberoflaboratorydirector
Thisinformationwillberelayedtoapathologistmemberoftheappropriateresourcecommitteewhowillcontacttheparticipant’slaboratorydirectororhospitalemployeehealthservicesphysicianwithinstructionsconcerningprophylaxis.
Limitations of PTDuetothemanufacturednatureofthespecimensandthelogisticsofshipping,PTdoesnotalwayscorrelatewiththemannerinwhichfresh,clinicalspecimensarehandled.Aletteraddressingthesedifferencesisincludedforgeneralusebyyourlaboratory.
Proficiency Testing Manual College of American Pathologists 9
General Customer Information
Limitations of PT Letter
The College of American Pathologists (CAP) Surveys program is the largest externalquality assessment program in theworld.As such, it provides an unparalleled selectionofchallengesandoffersthelargestdatabaseinexistenceforinterlaboratorycomparison.TheCAPhasaccumulatedsignificantexperienceinmanagingthistypeofprogramandisknowledgeableinitsusesandlimitations.
Performance onCAPSurveys is not to be taken as the sole indicator of a laboratory’sabilities.AproficiencytestingSurveyisbutoneofanumberofprogramsthatlaboratoriesshould employ to assess, manage, and improve quality. In addition to Surveys, propermethodvalidation,qualitycontroltesting,periodiccalibrationandinstrumentmaintenance,employeecompetencytesting,andlaboratoryinspectionandaccreditationprovideimportanttoolsformeasuringlaboratoryperformanceandensuringquality.
TheSurveysprogram,althoughoutstanding,isnotaperfectmeasuringdevice.Anumberof factors limit this tool’s ability to measure laboratory accuracy. Specific limitationsinclude requisite use of matrix materials that may impact test systems differently thanpatientspecimens;theappropriatenessofgroupingresponsesaccordingtomethodology,instrumentation, and test platforms; varying size of comparison groups with attendantvariabilityofstatisticalparameters;regulatedlimitationsinsamplingoflaboratories’testingsystems;difficultiesinquantitationattheextremesofanalyteconcentration;andunsuitabilityofcertainfederally-mandatedevaluationlimits.
Thus,acertainnumberof responses thataregradedasunacceptable inSurveyswill infactbeacceptable,andacertainnumberofresponsesgradedasacceptablewillinfactbeunacceptable.Although unsuccessful or unsatisfactorySurveys performancemay reflectproblemswithinalaboratory,itdoesnotconstituteproofofinadequateperformanceoraninabilitytomeetpatientneeds.
Sincerely,
RaoufE.Nakhleh,MD,FCAP
Chair,CouncilonScientificAffairs
Limitations of PT Letter
The College of American Pathologists (CAP) Surveys program is the largest externalquality assessment program in the world. As such, it provides an unparalleled selectionof challenges and offers the largest database in existence for interlaboratory comparison.The CAP has accumulated significant experience in managing this type of program and isknowledgeableinitsusesandlimitations.
Performance on CAP Surveys is not to be taken as the sole indicator of a laboratory’sabilities.Aproficiency testing Survey is but one of a number of programs that laboratoriesshould employ to assess, manage, and improve quality. In addition to Surveys, propermethod validation, quality control testing, periodic calibration and instrument maintenance,employee competency testing, and laboratory inspection and accreditation provide importanttoolsformeasuringlaboratoryperformanceandensuringquality.
The Surveys program, although outstanding, is not a perfect measuring device. A numberof factors limit this tool’s ability to measure laboratory accuracy. Specific limitationsinclude requisite use of matrix materials that may impact test systems differently thanpatient specimens; the appropriateness of grouping responses according to methodology,instrumentation, and test platforms; varying size of comparison groups with attendantvariability of statistical parameters; regulated limitations in sampling of laboratories’testingsystems; difficulties in quantitation at the extremes of analyte concentration; and unsuitabilityofcertainfederally-mandatedevaluationlimits.
Thus, a certain number of responses that are graded as unacceptable in Surveys will infact be acceptable, and a certain number of responses graded as acceptable will in fact beunacceptable. Although unsuccessful or unsatisfactory Surveys performance may reflectproblems within a laboratory, it does not constitute proof of inadequate performance or aninabilitytomeetpatientneeds.
Sincerely,
A
RaoufE.Nakhleh,MD,FCAP
Chair,CouncilonScientific Affairs
2018 Proficiency Testing Manual College of American Pathologists 9
Proficiency Testing Manual College of American Pathologists 10
Result Form Data Entry
Result Form Data EntryOverviewTheresultformisadocumentonwhichyourecordyourmethodsofanalysisandPTdata.Thecompletedresultformmustbereturnedonline(preferred)viaELSSorbyfaxtotheCAPbytheduedatelistedontheform.
RetainacopyofthecompletedresultformforyourrecordsbeforesubmittingittotheCAP.
PertheCentersforMedicare&MedicaidServices(CMS),changestosubmitteddatacannotbemadeaftertheduedateontheresultform.Reviewallinformationonthepreprintedmethodsummarypageandallentriesmadeontheresultformforaccuracypriortosubmission.Onceyouhaveenrolled/registered,logintoELSSviacap.org,andclickonSubmit/EnterPTResultstoviewthestatusofyoursubmitteddata.
Preprinted Method Summary PageOnceyourlaboratoryinitiallyprovidesamasterlistcodeforamethod,instrument,and/orreagent,theCAPwillretainthisinformation,nolongerrequiringyoutoreportthemrepetitivelythroughouttheyear.Carefullycheckeachmasterlistselectiontoensurethecorrectcodesarelisted.Tochangeacode,enteritintheappropriateboxesontheresultform.
Exception CodesIfyourlaboratoryneedstoreportananalyticalproblemforanentiretestorindividualspecimenswithinatest,leavetheresultareablankandfilltheappropriatetwo-digitexceptioncodebubbleontheresultform.
Exception Code Reason
11 Unabletoanalyze(documentationtobeprovidedbylaboratory).
22 Resultisoutsidemethod/instrumentreportablerange.
33 SpecimendeterminedtobeunsatisfactoryaftercontactingtheCAP.
Itisthelaboratory’sresponsibilitytodocumenttheappropriateuseofexceptioncodesifrequestedduringalaboratoryinspection.Refertothekitinstructionsformoreinformation.
Proficiency Testing Manual College of American Pathologists 11
Result Form Data Entry
Master ListsChoosetheappropriatemethodorinstrumentprovidedonthekitinstructionsorresultform.
NotifytheCAPandthemanufacturerifyourmethodorinstrumentisnotlisted.Ifthemanufacturerofyourmethodorinstrumentisnotlistedonthekitinstructionsorresultform,enteritinthe“UseofOther”sectionattheendoftheresultform.
Identificationmasterlistsareprovidedformicrobiology,bloodcellidentification,urinesediment,clinicalmicroscopy,andprovider-performedmicroscopy.Selectacodefromtheappropriatemasterlistandenteritontheresultform.
Forbloodcellidentification,urinesediment,clinicalmicroscopy,andprovider-performedmicroscopy,allpossibleidentificationsareincludedonthemasterlists.Do not usethecode010,“Other,Specify.”Theuseofthiscodewillbeevaluatedasanunacceptableresponse.
HTML SectionTheCAPhasimplementedamorecustomer-friendlyonlineinterfaceforenteringresultsviaELSS.TheenhancementprovidesonlineresultformsinanHTMLformat,replacingthePDFtechnology,butretainsthedesign.Theformhelps:
1. Reducecommonclericalerrors
2. Increasedataaccuracy
3. Approveonlinedataefficiently
4. Searchandnavigatewithease
What is different about the form?
Itprovidesvalidationforrequiredinformation.Forexample,ifavalue/resultispopulatedforanyspecimenforasingletest,thenavalue/resultmustbepopulatedformanufacturer,method,instrument,orreagent.ItalsopreventsincompletedatafrombeingsubmittedtotheCAP.Forexample,ifyou:
• Donotenteraunitofmeasureintheonlineresultform,apop-upmessagewillappearandaskyoutoprovidetherequiredinformation.
• Enteravalue/resultinanincorrectformat(eg,###.#insteadof###.)ontheonlineresultform,apop-upmessagewillappearandaskyoutorevisetheinformation.
Inaddition,theform:
• Allowsnavigationbytest(forlaboratoriesthatdonotreporteverytestinaPTevent)
• Providesanapprovalbuttononthetopofeachresultformpage
• Aimstolessenbrowsercompatibilityissues.
Proficiency Testing Manual College of American Pathologists 12
Result Form Data Entry
‘Less Than’ or ‘Greater Than’ ValuesDonotattempttoadd“lessthan”or“greaterthan”tothevalueyousubmitunlessthisoptionisprovidedontheresultform.Wheretheoptiontoreport“greaterthan”or“lessthan”isgiven,youmustselectthebubbleunderneaththeappropriatereportingboxtoindicateyourresponseisa“lessthan”or“greaterthan”value.Allotherresultswillbeconsidered“equalto”values.
Wherenooptiontoreport“greaterthan”or“lessthan”isgiven,refertotheExceptionCodessectionortheGeneralReportingInformationsectionofthekitinstructions.
Proficiency Testing Manual College of American Pathologists 13
e-LAB Solutions Suite (ELSS)
e-LAB Solutions Suite (ELSS)Getting StartedELSSistheCAP’sonlineportaltomanageyourlaboratoryimprovementprograms.Inordertotakefulladvantageofthefunctionalitiesavailableonline,userscanselectivelyenablelaboratoriesandpersonnelby:
Creating an Online AccountInordertoaccessELSS,youmustcreateapersonalWebaccountbygoingtocap.organdclickingonLogin.UnderNewUsers,clickonCREATEANACCOUNT;thisonlyneedstobedoneonceperuser.
Logging InOnceyouhaveestablishedanonlineaccount,youwillbepromptedtoenteryouruserIDandpassword.Thismustbedoneeverytimeyouvisitcap.org.Bothmembersandnonmembersutilizethesameloginfunctionality.
Enroll/RegisterWhenyourlaboratoryfirstplacesanorder,thelaboratorydirectorwillreceiveanenroll/registeremailwithaCAPNumberandPIN#.Thelaboratorydirectormustdesignatewhowillbethelaboratory’ssiteadministrator.Yourlaboratoryonlyhastoenroll/registeronce.Theenrollment/registrationprocessestablishesthelaboratory’sonlineaccesstoCAPprogramsthroughELSSanddefinesthesiteadministrator’sresponsibilities.
Requesting AccessIndividualizedaccountsforlaboratoryusersallowflexibilityindeterminingaccesslevels.Becauseusersareindependententitiesfromthelaboratory,userscanbeaffiliatedwithmultiplelaboratoriesandtheirsecuritycanbeadministeredinadifferentmannerateachsite.
Torequestaccesstoalaboratory,gotocap.org,clickonAccesse-LABSolutionsSuite,andlogin.ClickonRequestPermissionfromYourSiteAdmin,andenteryourlaboratory’sCAP#.
IndividualprivilegescanbeaccessedbyclickingonView/UpdateMyLabPermissions.
Proficiency Testing Manual College of American Pathologists 14
e-LAB Solutions Suite (ELSS)
Online Data SubmissionOnlineresultformsmakedataentryfasterandmoreaccurate.YourlaboratorywillminimizePTfailuresondataentrythroughtheuseofanenhancedHTML-basedresultform.Youcanalsoverifyreceiptofdataandviewfaxedormaileddata.
AutomatedPTdatareportingisavailablethroughe-LABSolutionsConnect.TheCAPhaspartneredwithDataInnovationsInstrumentManager™middlewaresolutiontohelpfacilitatetheconnectionandtransmission.Withe-LABSolutionsConnect,theCAP’squantitativePTresultsareautomaticallytransmittedfromyourlaboratory’sinstrumentsorlaboratoryinformationsystemtoELSS.
Ifnecessary,faxedresultsmustbesentto866-FAX-2CAP(866-329-2227).
Online ReportsWhentheresultformduedatehaspast,dataareprocessedandgradedaccordingtostringentgovernmentalandcommitteecriteria.LaboratoryresultsandgradinginterpretationsaredisplayedinanindividualizedreportaccessiblebyclickingonEvaluationReportsunderProficiencyTesting/QualityManagement.Usingthefilteroptions,selecttheprogramandreport.
Theonlineevaluationallowstheusertoeasilynavigatethedataanalytebyanalyte.Also,usersbenefitbytheinclusionofdetailedimageshyperlinkedonthereport.LaboratorymanagerscanaccesstheAllAnalyteScorecard,allowingthemtocustomizethescorecarddatafortheirlaboratoryasameansofidentifyingdeficienciesortrendsinperformance.
Otherancillaryreportsthataccompanytheevaluationareavailableonline,aswell,forreviewandtodownloadforfuturereference,includingthePS,finalcritiques,andannualsummaries,whichcontainusefuldataandeducationthatcanbereviewedandaccessedbyalluserswithappropriatesecurity.
Performance Analytics DashboardOurPerformanceAnalyticsDashboardgivesthecomprehensivedataviewyouneedtoproactivelyfocusenergyonareasthatneedimmediateattentionwhilefilteringoutdistractions.Updateddaily,thisWeb-basedreportingsolutionforyourCAPPTandaccreditationperformancereducesthestressofmanagingtoday’ssinglelaboratoryoranexpansivenetwork’sperformancewithonlyafewclicks.
Proficiency Testing Manual College of American Pathologists 15
How to Interpret Your Evaluation
How to Interpret Your EvaluationThischapterincludesgeneralinformationregardingevaluations.Asectionexplaininghowtointerpretyourindividualevaluationwillbeincludedwitheachmailing.
General Guidelines for EvaluationOnFebruary28,1992,theSecretaryofHealthandHumanServices(HHS)publishedthefinalrulesimplementingtheCLIAof1988.TheseregulationsestablishedevaluationcriterialimitsformanyoftheanalytesincludedintheCAPSurveysprograms.ThetargetvaluesaredeterminedbythescientificresourcecommitteesoftheCAP.ForthoseanalytesnotincludedinthePTportionofCLIA,thetargetvaluesandevaluationcriteriaaredeterminedsolelybythescientificresourcecommittees.
Selection of a Target ValueTominimizetheeffectofmethoddifferencesandtoallowcomparisonofallmethods,participants’resultsarecombinedintocomparablemethod/instrumentgroupscalledpeergroups.Itisimportantforparticipantstoprovidecompleteinformationregardingthemethodorinstrumentusedinordertobecombinedintheappropriatepeergroup.
Formostanalytes,thepeergroupmeanisdesignatedasthetargetvalueforevaluation.Thepeergroupmeanisthepreferredtargetifnosingletargetvalueexiststhatcanprovideanaccuracy-basedtargetthatistraceabletothe“truevalue”asdeterminedbyadefinitiveorreferencemethodanalysis.Thepeergroupmustconsistofgreaterthannineresultsafteroutlierexclusion,andthevariabilityofthepeergroupdatamustnotbetoogreat.
Ifpeergroupdataarenotavailableoraretoovariable,methodgroupstatisticsmaybeused.Themethodgroupmustalsoconsistofgreaterthannineresultsanddemonstrateacceptablevariabilitybeforeitisusedasthetargetgroup.
Ifpeerandmethodgroupstatisticsarenotavailable,acomparativemethodgroupmaybedesignatedasthetargetmean.ThecomparativemethodisnotthemethodrecommendedbytheCAP;however,itisestablishedasahistoricallyreliablemethodandisusedforevaluatingresultsfrommethodsthathaveaninsufficientnumberofparticipantstogenerateaseparatepeergroupand/ormethodgroupstatistics.Ifnocomparativemethodexistsfortheanalyte,resultswillnotbeevaluated.
Forsomeanalytes,asingletargetvalueisusedinwhichconsistentresultsare
Proficiency Testing Manual College of American Pathologists 16
How to Interpret Your Evaluation
demonstratedacrossallpeergroups.
Calculation of Summary Statistics
Peer Group Results Resultsaregroupedaccordingtothemethodusedforanalysisandscreenedforoutliers.Variousstatisticsarecalculatedfromtheremainingdatathatsummarizethepeergroup’sresponses.Thesesummarystatisticsmayincludethefollowing:
• themean(theaverageofthereportedresults)
• thestandarddeviation(ameasureofthevariabilityoftheparticipantresults,oftenabbreviatedasSD)
• thecoefficientofvariation(CV)
• themedian(themiddlevalueinanorderedlistofthenon-outlierresults)
• thelowvalue(thelowestvaluereported)
• thehighvalue(thehighestvaluereported)
• thefinalcountofreportedresultsthatwerenotexcludedasoutliers.
Outlier Detection TechniqueOutlierexclusionisnecessarybecausealargeseriesofresultsfrequentlywillincludesomeaberrantvalues.Thesemayarisefrominstrumentmalfunction,technicalerrors,specimenmix-ups,misplaceddecimals,incorrectunitsofmeasure,ordataentryerrors.Ifanyresultsareexcluded,theoutlierprocessisrepeatedusingtheremainingvalues.Thesummarystatisticsthatappearonyourreportsdonotreflectresultsthatwereconsideredtobeoutliersduringeitheroutlierpass.
Quantitative Procedures/RoundingAllquantitativeresponsesareevaluatedbasedonarangeofacceptability.Thisrangeisdeterminedusingatargetvalueandalimit.Thelimitwillbeeitherafixedinterval(eg,±5mg/dL),apercentageofthemean(eg,±25%),anSD(eg,±3SD),oravariablerange(eg,±6mg/dLor10%,whicheverisgreater).ThePSincludedwithyourevaluationsthecriteriausedtoevaluateyourperformance.Thefollowingsectionsprovidesspecificexamplesofhowtocalculatetherangeofacceptabilitydependinguponthecriteriaused.
Benefit-of-the-doubtroundingisusedtodeterminetherangeofacceptableresults.Theupperlimitofacceptabilityisobtainedbyroundinguptothenextreportableresult,whilethelowerlimitisdeterminedbytruncating.
Fixed Range ExampleYourlaboratoryreportsasodiumresultof138mmol/L.Thepeergroupmeanis139.5mmol/L.
Theevaluationlimitforsodiumis±4mmol/L.Theacceptablerangeisdeterminedbytheformula139.5mmol/L±4mmol/L,whichis135to144mmol/L;therefore,yourreportedresultof138mmol/Liswithinthecalculatedacceptablerangeof135to144mmol/Lwhenusingbenefit-of-the-doubtrounding.
Proficiency Testing Manual College of American Pathologists 17
How to Interpret Your Evaluation
Percentage of the Mean ExampleYourlaboratoryreportsanalbuminresultof3.1mg/dL.Thepeergroupmeanis3.39mg/dL.Theevaluationlimitforalbuminis±10%.Tenpercentof3.39mg/dLis0.34mg/dL.Theacceptablerangeisdeterminedbytheformula3.39mg/dL±0.34mg/dL,whichis3.0to3.8mg/dLwhenusingbenefit-of-the-doubtrounding;therefore,yourreportedresultof3.1mg/dLiswithinthecalculatedacceptablerangeof3.0to3.8mg/dL.
Standard Deviation ExampleYourlaboratoryreportsathyroidstimulatinghormone(TSH)resultof16.4µU/mL.Thepeergroupstatisticsareasfollows:mean=15.7µU/mL,SD=1.5,andCV=9.6.TheevaluationlimitforTSHis±3SD.3x1.5=4.5.Theacceptablerangeisdeterminedusingtheformula15.7µU/mL±4.5µU/mL,whichis11.2to20.2µU/mLwhenusingbenefit-of-the-doubtrounding;therefore,yourresultof16.4µU/mLiswithintheacceptablerangeof11.2to20.2µU/mL.
Variable Range ExampleYourlaboratoryreportsatotalbilirubinresultof4.5mg/dL.Thepeergroupmeanis4.68mg/dL.Theevaluationlimitfortotalbilirubinis±0.4mg/dLor20%,whicheverisgreater.Twentypercentof4.68is0.936.Sincethepercentagelimitof0.94isgreaterthantheintervallimitof0.4,thepercentagelimitisappliedtothetargetvalue.Theacceptablerangeisdeterminedusingtheformula:4.68±0.936,whichis3.7to5.7mg/dLwhenusingbenefit-of-the-doubtrounding;therefore,yourresultof4.5mg/dLiswithintheacceptablerangeof3.7to5.7mg/dL.
ThePSincludedwithyourevaluationwilllistthecriteriausedtoevaluateyourperformance.Todeterminetheacceptablerange,abenefit-of-the-doubtroundingprocedureisutilizedwhengrading.
Calculation of the Standard Deviation Index (SDI)Theevaluationlistsyourresults,thestatisticsforyourpeergroup,andyournormalizedresultsasanSDI.ThisvalueisobtainedbysubtractingthegroupmeanfromyourresultandthendividingbytheSD.
TheSDIisexpressedintermsofthenumberofSDsfromthemean,withanarithmeticsignindicatingthedirectionofthedifference.ThecalculationoftheSDInormalizesyourresultand,therefore,allowsforacomparisonofresultsfromspecimensofdifferentconcentrationsofananalyte.
WhenacomparativemethodhasbeendesignatedforSurveysanalysis,asecondsetofstatisticsislistedcomparingyourresultswiththoseobtainedusingthecomparativemethod.ThefiguresshownarethemeanandSDforthecomparativemethodandyourresultasanSDIusingthesestatistics.Itispossibleforyourresulttobedefinedas“good”performanceinyourmethodgroupandyetproduceacomparativeSDIgreaterthantwo.ThiswilloccurifyourmethodhasalargeanalyticbiasoralargeSD.ItispossibletoreceiveacomparativeSDIlowerthanthemethodgroupSDI,althoughthisrarelyoccurs.Inpractice,mostparticipantsreceivesimilarmethodgroupandcomparativeSDIs.
Proficiency Testing Manual College of American Pathologists 18
How to Interpret Your Evaluation
Comparative Statistics
Quantitative ProceduresYourevaluationcontainsplotsoftherelativedistanceofyourreportedresultsasapercentageofallowabledeviationfromthetargetvalue.Thenumericdigitindicatesthenumberofresultsataplotlocation.Thealloweddeviationmaybecalculatedasfollows:
Ifyourresultisgreaterthanthetargetmean:
Percentageof AcceptableDeviation =100x
yourresult-targetupperlimit-targetmean
Ifyourresultislessthanthetargetmean:
Percentageof AcceptableDeviation =100x
yourresult-targettargetmean-lowerlimit-
Qualitative ProceduresForqualitativeresponses,consensusagreementofrefereeorparticipatinglaboratoriesisusedforevaluation.Generally,80%agreementisrequired.
Proficiency Testing Manual College of American Pathologists 19
Continuing Education (CE)
Continuing Education (CE) Continuing Medical Education (CME) Category 1TheCAPisaccreditedbytheAccreditationCouncilforContinuingMedicalEducation(ACCME)toprovidecontinuingmedicaleducationforphysicians.
TheCAPdesignatestheseeducationalactivitiesforamaximumofthestatednumberofAMAPRACategory1Credits™.Physiciansshouldonlyclaimcreditscommensuratewiththeextentoftheirparticipationintheactivity.
TheAmericanMedicalAssociationhasdeterminedthatphysiciansnotlicensedintheUSwhoparticipateintheseCMEactivitiesareeligibleforAMAPRACategory1Credit™.
SeethecurrentSurveysandAnatomicPathologyEducationProgramscatalogforavailableCMEprograms.
CE for Nonphysician Laboratory PersonnelThisactivityisacceptabletomeettheCErequirementsfortheAmericanSocietyforClinicalPathology(ASCP)BoardofRegistryCertificationMaintenanceProgram.ThisactivityisapprovedforcontinuingeducationcreditinthestatesofCaliforniaandFlorida.
AllnonphysicianlaboratoryprofessionalsinyourlaboratorycanearnindividualCEcreditsbycompletingtherelatededucationreadingandonlinelearningassessmentquestionsviacap.org.
Surveys CE ProgramsDiscipline MaximumCECredits
Chemistry
The number of credits are specific to the program mailing.
Go to cap.org for up-to-date activity listings.
Coagulation
Hematology
Histology(HistoQIP)
Immunology
Microbiology
TherapeuticDrugMonitoring/Endocrinology
Toxicology
TransfusionMedicine
ReproductiveMedicine
Proficiency Testing Manual College of American Pathologists 20
Continuing Education (CE)
Learning Cycle InformationEacheducationactivityprovidesinformationoncommontechnicalandnontechnicalissuesencounteredinalllaboratorysettings.ToreceiveCEcredit,youmustcompletetheeducationreadingprovidedinyourPSandanswertheonlinelearningassessmentquestions.Mosteducationactivitiesareavailablefor12monthsandmustbecompletedwithinthattimeframe.CEcreditwillbeappliedtowardtheyearinwhichtheactivityiscompleted.DetailedinformationonhowtoaccesstheonlinecomponentswillbeincludedineachPS.TheCAPisanapprovedproviderofaccreditedCME.InordertoremainaCMEprovidertheCAPmustabidebyallACCMEandAMArulesforaccreditedCME.ThismeansthattheCAPcannotopenacoursetoallowcompletionofthecourseaftertheexpirationdate.
CE for CytotechnologistsCytotechnologistsmayapplythecreditsfromthegynecologicandnongynecologic(FNA,FNAG,NGC,andTICP)programstowardtherequirededucationalactivitiesfortheAmericanSocietyforCytopathology(ASC)ContinuingEducationCreditProgram.
Online Virtual Microscopy Education ProgramsTheCAPoffersonlineeducationprogramsthatuseDigitalScopetechnologytopresentwholeslideimagesfromactualglassslidesforavarietyofsitesandspecimentypes.Thistechnologysimulatesamicroscope,allowingyoutoscantheimageandusemultiplemagnificationstoviewthematerial.Fromtheimagesandclinicalinformationprovided,youselectadiagnosis,answerlearningassessmentquestions,andreceiveimmediatefeedbackonline.
SeethecurrentSurveysandAnatomicPathologyEducationProgramscatalogforavailableonlinevirtualmicroscopyeducationprograms.
Proficiency Testing Manual College of American Pathologists 21
Discipline-Specific Reporting Information
Discipline-Specific Reporting InformationHematologyToreportyourbloodcellidentification,selectthebestidentificationcodefromtheHematologyBloodCellIdentificationMasterListprovidedinthekitinstructions.ForfurtherdescriptionoftheHematologyBloodCellIdentificationMasterListchoices,refertotheBloodCellIdentificationsectionofthecurrentCAP Hematology and Clinical Microscopy Glossary,whichcanbeaccessedatcap.org.
Ifresultsarereportedforbothbloodcellidentificationandautodifferentials,thebloodcellidentificationwillbereportedtoCMS.
TheHematologyBloodCellIdentificationMasterListchoice,“Immaturecellorabnormalcell,wouldreferforidentification,”mustbereservedforcellsyourarelyencounterandareunabletospecificallyidentify.GradingofthisresponsewillfollowtheguidelinessetforthintheJuly26,1993,Federal RegisterNotice.
CoagulationForplasma-basedcoagulationtesting[prothrombintime(PT),activatedpartialthromboplastin(APTT),fibrinogen],aninstrumentandreagentcodearerequiredforproperevaluation.ParticipantsenrolledinwholebloodtestingforPTneedonlyindicateaninstrument(ifrequested)andtheirresults.ForallPTmodules,reportingofinternationalnormalizedratio(INR)resultsisoptional.Plasma-basedandwholebloodINRareevaluated.
UrinalysisThereareseparateurinalysisandspecificgravitymethodandinstrumentmasterlistsinthekitinstructions.Toensureanaccuratepeergroupevaluationofyourresults,itiscriticaltoprovideaccuratemethodandinstrumentinformation.
Aspecificlistofreportingoptionsisprovidedforeachurinalysisprocedure.Itisnotfeasibletoprovidealistofreportingchoicesspecificforeverypossibledipstickbeingmarketedtolaboratories.Subsequently,theresultrangeslistedmaynotexactlycorrelatewiththerangesusedwithyourinstrument/dipstick.Inthesefewcases,choosetherangethatmostcloselymatchesyourintendedresult.
Proficiency Testing Manual College of American Pathologists 22
Discipline-Specific Reporting Information
Toreporturinesediment,clinicalmicroscopy,orprovider-performedmicroscopy,selectthebestidentificationcodefromtheUrineSedimentMasterListprovided.ForfurtherdescriptionoftheHematologyBloodCellIdentificationMasterListchoices,refertotheBloodCellIdentificationsectionofthecurrentCAP Hematology and Clinical Microscopy Glossary,whichcanbeaccessedatcap.org.
Urinalysis Dipstick TestsForqualitativeproceduresinurinalysis,evaluationisbasedonparticipantconsensusbymethodandinstrument.Foreachanalyte,aminimumoftwo,butnotmorethanfour,responseswillbegivenapassingscore.Analyteresultsgraded“good”performancemusthave80%participantconsensus.Eightypercentparticipantconsensuscanbedeterminedbygroupingthemodewiththenextoneortwomostfrequentresponses.Thisgroupwillbegiven“good”performance.“Acceptable”performancewillbegiventoadditionalresponsesuntilaminimumof90%ofparticipantresultsaregivenapassingscore.Inthecaseofanegativespecimen,negativeresponsesmustconstitute90%participantconsensus.Specimenswithresultsforoneormoremethodsdistributedoverbothnegativeandpositiveresponse(nonconsensus)willnotbeevaluated.Specimensforwhichthereisgreaterthan90%ofparticipantresponsesdistributedovermorethanfourresponseswillbegradedasnonconsensus.
MicrobiologyWhereappropriate,aclinicaldiagnosis,age,andsourcearelistedtosimulateatrueclinicalsituationandtoallowlaboratorypersonneltoselectappropriatemediaormethodsforprocessingthesespecimens.However,asthepathogenicbacteriapresentinanyofthesespecimensmaybeisolatedfrommultiplesourcesofthebody,allparticipantsshouldattemptidentificationoftheorganismspresentinallthesespecimens.
PertheFederal Register,aSurveymustgradealaboratory’sabilitytodistinguishbetweenapathogenandacontaminant.Culturechallengeswillbedesignatedintheinstructionstobehandledas“identifyprincipalpathogen”or“identifyallorganisms”challenges.Participantsmustreportinthismannerevenwhenthisdiffersfromtheirlaboratory’sroutinepractice.Forexample,aurinespecimencontainsKlebsiella pneumoniae andStaphylococcus epidermidis.Iftheinstructionsindicateto“Identifyallorganisms,”bothorganismsshouldbereported.Iftheinstructionsindicateto“identifyprincipalpathogen,”onlytheKlebsiella pneumoniaeshouldbereported.IftheStaphylococcus epidermidisisreported,itwouldbepenalized.
Specimenresultswillbeevaluatedif80%ormoreoftheparticipantlaboratoriesagreeontheidentificationofthetestorganism(s)togenusortogenusandspecies.Intheabsenceofparticipantconsensus,refereelaboratorieswillbeused.
TheCLIAregulationsstatethatalaboratorymustperformaminimumoffivespecimensineachtestingeventforthesubspecialtyofbacteriology.Thefivechallengescanincludeacombinationofthefollowingspecimens:
• Bacterialantigendetection
• Bacterialidentification(culture)
• Gramstain
• Antimicrobialsusceptibility
Proficiency Testing Manual College of American Pathologists 23
Discipline-Specific Reporting Information
Proceduresassayedwithwaivedmethodologieswillnotcounttowardthefive-challengeminimum.ThelaboratoryisresponsibleformaintainingthefivespecimenspertestingeventforitsremainingnonwaivedtestsinthesubspecialtywhenatestiswaivedbytheFoodandDrugAdministrationmidyear.
Antimicrobial Susceptibility TestingParticipantswillbeaskedtoperformsusceptibilitytestsusingtheantimicrobialagentsandtechniquesinroutineuseintheirindividuallaboratories.Thelaboratoriesshouldreportonlyantibioticsappropriatefortherapyofinfectionsatthesiteindicatedinthepatienthistory.SeecurrentClinicalandLaboratoryStandardsInstitute(CLSI)documentsM2,M7,M100,orotherappropriatedocumentsforguidance.
Interpretationresults(susceptible,intermediate,andresistant)willbepenalizedfor:
• Agentsthatarenotclinicallyappropriateforthesiteofinfection(meningitis,pneumonia,urinarytract,etc)
• UseofmethodsCLSIadvisesagainst
• Useofmethodsthatthemanufacturerrecommendsagainstusing,duetopoorperformance
Selectivereportingforthepresumedsiteofinfectionhelpsimproveclinicalrelevance,encouragesappropriatetherapy,andhelpstominimizeselectionofresistance.
Proficiency Testing Manual College of American Pathologists 24
Evaluations and Participant Summaries (PS)
Evaluations and Participant Summaries (PS) OverviewShortlyafteryousubmityourPTresultstotheCAP,anevaluationofyoursubmittedresultswillbeavailableonlineormailedbacktoyou.Yourevaluationcanbeusedasaqualityassurancetooltoassesshowyouperformedcomparedtootherparticipants.Itisimportantthatyoureviewandunderstandtheinformationpresentedinyourevaluation:
• Demographicinformation:Providesinformationaboutyourlaboratory,includingthenameofyourinstitution,yourCAPidentificationnumber,andanyagenciesorconsultantsyouhavedesignatedtoreceivecopiesofyourevaluation.
• Resultarea:Containsallresultsreportedforaparticularmailingandstatisticaldatausedforevaluationpurposes.AdetaileddescriptionofevaluationdataspecificforeachdisciplineispresentedineachPS.
• CMSperformancesummaryreport:IncludesinformationoncurrentandcumulativeperformanceforregulatedanalytestobesenttotheCMS.
Reviewing Your Evaluation TotrulyrealizethebenefitofPT,itisimportantthatyoutake
thetimetocarefullyreviewyourevaluation.Youcangainvaluableinsightintoyourlaboratory’soverallprocessesbyfollowingtheseeasystepsinreviewingthisreport.
1. Reviewthedemographicinformationontheevaluation.Ifanyinformationisincorrectorhaschanged,contacttheCAPat800-323-4040option1.
2. Compareinformationonyourevaluationwithresultsonyourphotocopyorprintedcopyoftheresultform.IfanyofyourdatawasenteredbytheCAPincorrectly,contactusimmediately.CorrectionsduetodataentryerrorsmadebytheCAPmustberequestedwithinfourweeksafterthefirstevaluationwasmailed.
3. Lookforanyunacceptableresults.Common,easilycorrectedreasonsforunacceptableresultsinclude:
• Incorrectorincompletemethod/instrumentdata
• Clericalerror
• Decimalpointplacement
• Specimenhandlingerror
Remember,whateverthecause,CLIAstatesthatallPTdeficienciesmustbedocumentedandcorrectiveactiontakentoresolvethedeficiency.
Proficiency Testing Manual College of American Pathologists 25
Evaluations and Participant Summaries (PS)
4. Occasionally,incorrectentryofsubmitteddataoccurs.Ifthisisduetoyourtranscriptionerrororfailuretocompletetheresultformappropriately,yourentrycannotbereevaluated.IftheerrorismadebytheCAP,contacttheCAPat800-323-4040option1forfurtherassistance.
5. Althoughtheresultsmaynotbeformallyevaluated,youcancompareyourresultswiththedataprovidedinthePSreport.Youcanusethe“allmethodmean”ormedian,low,andhighvaluestocompareyourresultsforaself-assessmentofyourperformance.
6. Forquantitativedata,justknowingthatyouare“withinlimits”doesnottellyouifyouareexperiencingaslowlydevelopingbiasthatmayresultinfuturefailures.Thekeytooptimaluseofyourevaluationdataistolookatthecolumnwherestandarddeviationindexes(SDI)arereported.Ifyounoteanyofthefollowingtendencies,itmaybeadvantageoustoexamineyourlaboratoryprocessesfurther:
• TheaverageSDIismorethan±1.5:thismayindicateasignificantsystematicerror.Reviewcalibrationdataandtechnique.Reviewexpirationdatesofcalibratorsandreagents.
• OneofyourSDIsisgreaterthan±3ortotalSDIisgreaterthan4(oneSDIis-2andoneis+2.5foratotalof4.5):thismayindicateasignificantrandomerror.Reviewyourproceduretodeterminewhereanyunwantedimprecisionmaybeoccurring.
7. Whentheevaluationhasanonevaluationcodelisted,refertoyourPSforvaluableinformation.
8. VerifythatallregulatedanalytesforwhichyoureportedresultsareincludedontheCMSPerformanceSummaryReport.
9. MakesurethelaboratorydirectorreviewsandsignsallPTevaluations.
PS Inadditiontoyourevaluation,eachlaboratoryreceivesaPSforthatmailingthatlistsresultsfromallparticipantsforeachanalytegroupedbythemethodology.Thisreportprovidesvaluableinformationtotheparticipantintheformofcomparativedataandeducationactivities.
Program UpdateThissectionoftheparticipantsummarycontainsinformationaboutevaluationcriteriainuseforthatmailing.Italsohighlightsimportantmethod,manufacturer,andspecimeninformationthatpertainstothatmailing.
Quantitative DataTheparticipantsummaryprovidesthestatisticaldataneededtoreviewyourPTresults.Thereportliststhemean,SD,andCVforpeergroupsconsistingof10ormorelaboratories.
Qualitative DataQualitativedataevaluationisbasedonconsensusofparticipantand/orrefereeresponses.ThePSliststheparticipantresponsesalongwiththepercentagereportingthatresponse.Whereavailable,refereedataisalsoincluded.Thispracticeprovideshigher-quality,evaluatedchallengestoourparticipants.
Proficiency Testing Manual College of American Pathologists 26
Evaluations and Participant Summaries (PS)
How to Perform a Self-EvaluationAsmentionedpreviously,occasionallyaPTchallengecannotbeevaluatedforavarietyofreasons:
• Lackofparticipantconsensus
• Insufficientdata(<10responsesforagivenmethod)
• Perceivedcompatibilityissues
InordertocomplywiththequalityassuranceaspectofPTasoutlinedinCLIA,youmusthavesomemechanismtoevaluateyourPTresults.HereareafewexamplesofhowthedatapresentedinthePScanassistwiththistask.
Quantitative ResultsIfyouperformatestandtherearefewerthannineotherlaboratoriesreportingresultsforthattest,yourresultwillnotbeevaluated.Youcandeterminehowwellyouperformedcomparedtoallparticipantswhoreportedresultsbyusingthe“allinstrumentmethod”datapresentedinthePS(ifprovided).Forexample,youperformhemoglobinanalysisusingtheCoulterLH500.Thereareaninsufficientnumberofresultstoformapeergroup(<10);therefore,yourresultsarenotgraded.NotethatinthePSthereisanall-instrumentmean,standarddeviation,andcoefficientofvariation,whichcanbeusedasareferencevalue.ByapplyingthepublishedCMSevaluationlimits(±7%)tothismean,youcandeterminehowwellyouperformedcomparedtothisreferencevalue.Forexample:
Your result: 13.8 g/dL
All Instrument Mean: 13.77 g/dL
Range of Acceptability: 12.8–14.8 g/dL
Inthisexample,yourresultwouldbeconsideredwithinrangewhencomparedtotheall-instrumentmean.Documentthisself-assessmentonyourevaluation.Whenyouperformthisself-assessment,anyunacceptableresultshouldbedocumentedandinvestigatedandcorrectiveactionshouldbetakenaswouldbedoneforformallyevaluatedresults.Thissametechniquecanbeusedwhenonlyamedian,low,andhighvaluearereportedforananalyte.
Qualitative Results Ifaqualitativeresultisnotevaluatedduetolackofrefereeorparticipantconsensus,youcanstillevaluatehowwellyourlaboratory’sresultagreedwiththecorrectresponsebyusingthedatainthePS.Forexample,oneoftheGramstainchallengescouldnotbegradedduetolackofparticipantconsensus(77%reportedgram-negative,23%reportedgram-positive).ThePSindicatesthattheorganismwasPseudomonas aeruginosa,agram-negativerod/bacilli.Compareyourresultwiththecorrectresult.Investigateanddocumentanycorrectiveactiontaken.Reviewtheeducationalcritiqueaccompanyingtheresultforhelpfulsuggestionsonlaboratorytechnique.
Proficiency Testing Manual College of American Pathologists 27
Laboratory Legislation and CMS Reporting
Laboratory Legislation and CMS ReportingInFebruary28,1992,theSecretaryofHHSpublishedthefinalrulesimplementingCLIA.TheCLIAregulationsreplacedtheMedicare,Medicaid,andCLIA’67standardswithasinglesetofrequirementsthatapplytoalmostalllaboratoriestestinghumanspecimens.Standardsforlaboratorypersonnel,qualitycontrol,andqualityassurancearebasedontestcomplexityandriskfactors.TheregulationsalsoestablishapplicationproceduresandfeesforCLIAcertification,aswellasenforcementproceduresandsanctionsapplicablewhenlaboratoriesfailtomeetstandards.
CLIAappliestoalllaboratories,physicianoffices,orotherentitiesperformingtestingonhumanspecimensforthepurposeofprovidinginformationforthediagnosis,prevention,ortreatmentofdiseaseorimpairmentofhumanbeings.Laboratoriesthatconducttestingforthepurposeofassessingthehealthofindividuals(eg,testingforinsurancepurposes)arealsosubjecttoCLIA.ThefollowinglistscertainlaboratoriesthatarenotsubjecttoCLIA:
• Laboratoriesconductingonlyforensictesting
• Researchlaboratoriesthatdonotreportpatientresults
• ComponentsorfunctionsoflaboratoriescertifiedbytheSubstanceAbuseandMentalHealthServicesAdministration
• LaboratorieslocatedinastateinwhichthelicensureprogramisapprovedbytheCMSasmeetingCLIAstandards
• InternationallaboratoriesnotperformingtestsonUnitedStatescitizens
UnderCLIA,allclinicallaboratories,regardlessoflocation,size,ortype,mustmeetstandardsbasedonthecomplexityoftheteststheyperform.Threelevelsoftestingcomplexityaredefinedintheregulations:waived,provider-performedmicroscopy,andnonwaived.Laboratoriesperformingprovider-performedmicroscopyornonwaivedtestingmustmeetrequirementsforPT,patienttestmanagement,qualitycontrol,qualityassurance,andpersonnel.Thesespecificrequirementsdonotapplytotestsinthewaivedcategory.
TheuniformPTprogramregulationsmandatethatyourlaboratorymustenrollinaPTprogramapprovedbytheDepartmentofHealthandHumanServices,theCMSparentregulatoryagency,foreachofthespecialtiesandsubspecialtiesforwhichitseekscertification.AllanalytesthatareregulatedforPTappearinboldtypeintheSurveysandAnatomicPathologyEducationProgramscatalog.
YourlaboratorymustnotifyCMSoftheapprovedprogram(s)inwhichyouparticipateandauthorizethePTprogramtoreleasealldatatotheCMS.YourlaboratorymustparticipateinanapprovedPTprogramforoneyearbeforedesignatingadifferentprogram.CMSmustbenotifiedbeforeyourlaboratorychangesPTprograms.ForteststhatarenotsubjecttoPTintheseregulations,yourlaboratorymuststillestablishtheaccuracyandreliabilityofitstestproceduresatleasttwiceayear.
Proficiency Testing Manual College of American Pathologists 28
Laboratory Legislation and CMS Reporting
PTspecimensmustbetestedwiththeregularpatientworkloadbypersonnelwhoroutinelyperformtesting.Yourlaboratory’sroutinetestingmethodsmustbeused.TheindividualtestingthespecimensandthelaboratorydirectormustattesttotheroutineintegrationofspecimensusingaformprovidedbythePTprogram.LaboratoriesthatperformtestsonPTspecimensmustnotengageininterlaboratorycommunicationspertainingtotheresultsofPTspecimen(s)untilafterthedatebywhichthelaboratorymustreportPTresultstotheprogramforthetestingeventinwhichthespecimensweresent.LaboratorieswithmultipletestingsitesorseparatelocationsmustnotparticipateincommunicationsordiscussionsconcerningPTspecimenresultsuntilafterthedatebywhichthelaboratorymustreportPTresultstotheprogram.Yourlaboratorymustalsomaintainacopyofallrecords,includingtheformusedtorecordthePTresults(includingtheattestationsignatures),foraminimumoftwoyears.
YourlaboratorymustsuccessfullyparticipateinaPTprogramapprovedbyCMS.“Unsuccessfulproficiencytestingperformance”isa“conditionlevel”deficiencyandmayresultinlaboratorysanctionssuchassuspensionoftheCLIAcertificateandMedicarepaymentsforthespecialty,subspecialty,andanalyteinvolved.Failuretoachieveasatisfactoryoveralltestingeventperformancefortwoconsecutivetestingeventsortwooutofthreeconsecutivetestingeventsisconsideredunsuccessfulperformance.
Proceduresassayedwithwaivedmethodologieswillnotcounttowardthefive-challengeminimum.
Failuretoattainanoveralltestingeventscoreofatleast80%isunsatisfactoryperformanceforanalytesinallspecialtiesandsubspecialtiesexceptABOgroup,D(Rh)typing,andcompatibilitytestingforwhich100%isrequired.
FailuretoreturnPTresultsforatestingeventisunsatisfactoryperformanceandwillresultinascoreof“0.”Foranyunsatisfactorytestingeventforreasonsotherthanfailuretoparticipate,yourlaboratorymustundertakeappropriatetrainingandemploythetechnicalassistancenecessarytocorrecttheproblem.Allremedialactionmustbedocumentedandsuchdocumentationkeptfortwoyearsatyourlaboratoryforpossiblereferencebyinspectionandaccreditationteams.Aspartoftheseregulations,criteriahavebeenestablishedbywhichaPTprovider’sprogrammaybeevaluatedforapprovalbyHHS.TheCAPhasmadeeveryefforttoensurethattheSurveysprogramhasmettherequirementssetforthbytheFebruary28,1992,FinalRule.
Special Considerations for the Regulatory Requirements for the Specialty of ImmunohematologyThespecialtyofImmunohematologycomprisesfoursubspecialtiesasfollows:
• ABO/Rh
• Unexpectedantibodydetection
• Compatibilitytesting
• Antibodyidentification
A100%scoreisrequiredtoachievesatisfactoryperformanceforABO/Rhandcompatibilitytesting.
Forunexpectedantibodydetectionandantibodyidentification,an80%scoreisrequired.Theconsensuspercentagerequiredtoestablishagradedchallengeissetat95%participantor100%refereeconsensusforABO/Rhandcompatibilitytestingand95%refereeorparticipantconsensusforunexpectedantibodyorantibodyidentificationchallenges.
Proficiency Testing Manual College of American Pathologists 29
Laboratory Legislation and CMS Reporting
Special Considerations for the Regulatory Requirements for the Specialty of MicrobiologyTheCLIAregulationsstatethatalaboratorymusttestaminimumoffivespecimensineachtestingeventforthesubspecialtiesofbacteriology,mycobacteriology,mycology,parasitology,andvirology.Withineachofthesesubspecialties,varioustypesoftestingarerequired.
Forbacteriology,thefivechallengesmayincludeacombinationofthefollowingspecimens:
• Bacterialantigendetection
• Bacterialidentification
• Gramstain
• Antimicrobialsusceptibility
Chlamydia trachomatisresultsareaggregatedunderthesubspecialtyofbacteriology.
Formycobacteriology,thefivechallengesmayincludeacombinationofthefollowingspecimens:
• Acid-faststain
• Mycobacterialidentification
• Antimycobacterialsusceptibility
Formycology,fiveidentificationsarerequired.
Forparasitology,fiveidentificationsarerequiredandmayconsistoftestingbyfecalsuspension,Giemsa-stainedbloodsmear,antigendetection,and/orPVAslide.
Forvirology,thefivechallengesmayincludeacombinationofthefollowingspecimens:
• Viralantigendetection
• Viralidentification(culture)
Note: In order to meet the regulatory requirements for microbiology subspecialties, carefully follow the kit instructions included with each Surveys mailing.
Regulatory Requirements for the Specialty of Cytology: Gynecologic ExaminationsForcytology,theexaminationconsistsoftenslidesfromfourdiagnosticcategories,includingUnsatisfactory,NegativeorBenign,LSILandHSILorcarcinoma.Tobesuccessfulincytology,ascoreof90%mustbeachieved.Detailedinstructionswillbeprovidedwiththetestmaterials.
Proficiency Testing Manual College of American Pathologists 30
Laboratory Legislation and CMS Reporting
Provision of Results to CMS and State AgenciesToassistincomplyingwiththerequirementthatyourlaboratoryresultsbereleasedtoHHS,theCAPwilltransferdatatotheCMSifyouhaveprovidedaCLIAidentificationnumber.Wewillforwardpaperorelectroniccopiesofyourresultstoyourstatedepartmentofhealth,actingonbehalfoftheCMS,ifyouauthorizeustodoso.Toauthorizereleaseofresultstostateagencies,indicateyourrequestontheorderconfirmationreportsentafteryourSurveysorderisprocessed,orsendalettertotheCAP.YourSurveysevaluationwillthenreflectthenameoftheagencytowhichtheinformationwasprovided.AnyquestionsregardingrequirementsforforwardingyourPTresultsmaybeansweredviayourstatedepartmentofhealthoroneoftheCMSregionaloffices.
Consultwithyourstatedepartmentofhealthforadditionallawsorregulationsconcerninginspection,accreditation,andPTrequirementsthatmayaffectthelicensingofyourlaboratoryanditspersonnel.
CopiesoftheFebruary28,1992,CLIAregulationscanbeobtainedbycontactingtheGovernmentPrintingOfficebytelephoneat202-512-1800orbyfaxat202-512-2250andrequestingCFRTitle42Parts400-429and430-End.Youmayalsoaccessdocumentsonlineathttps://www.gpo.gov/fdsys/pkg/FR-1992-02-28/pdf/FR-1992-02-28.pdf
IfyouhaveanyquestionsregardingtheautomatictransferofresultstoCMSoryourperformancesummaries,contacttheCAPat800-323-4040option1.
Proficiency Testing Manual College of American Pathologists 31
Laboratory Legislation and CMS Reporting
Use of Reason Codes for Nonevaluated SpecimensSomeindividualresultsarenotevaluatedforcertainlaboratoriesforavarietyofreasons.Areasoncodeexplainingthespecificcircumstancewillappearonindividualevaluation,alongwithabriefexplanationofwhatthatcodemeans.Belowisaledgerofpossiblereasoncodesthatcanbeassigned.
Reason Code Description or Explanation
11 Unabletoanalyze(documentationtobeprovidedbylaboratory).
20 Noappropriatetarget/responsecouldbegraded.
21 Specimenproblem.
22 Resultisoutsidethemethod/instrumentreportablerange.
24 Incorrectresponseduetofailuretoprovideavalidresponsecode.
25 Inappropriateuseofantimicrobial.
26 Educationalchallenge.
27,31 Lackofparticipantorrefereeconsensus.
28Responsequalifiedwitha“greaterthan”or“lessthan”sign;orunabletoquantitate.
29 Inappropriateresponse.
30 Scientificcommitteedecision.
33 SpecimendeterminedtobeunsatisfactoryaftercontactingtheCAP.
35 Testingnotperformedonthisspecimentype.
40 Resultsforthiskitwerenotreceived.
41 Resultsforthiskitwerereceivedpasttheduedate.
42 Nocreditassignedduetoabsenceofresponse.
43 Theorderforthiskitwascanceled;resultsnotevaluated.
44Thisdrugisnotincludedinourtestmenu.Useofthiscodecountsasacorrectresponse.
46 Quantitationnotappropriate.
55 Exemptiongrantedduetoanaturaldisaster.
77 Improperuseofexceptioncodeforthismailing.
88Labdoesnotperformtestsfromthissourceordoesnotperformthistestonpatients.
91Therewereaninsufficientnumberofcontributingchallengestoestablishacompositegrade.
92Compositegradecouldnotbeestablishedduetotheuseofmultiplenon-gradedreasoncodesfortheindividualchallenges.
It is the laboratory’s responsibility to document the appropriate use of these exception codes should this be requested during a laboratory inspection.
Proficiency Testing Manual College of American Pathologists 32
Laboratory Legislation and CMS Reporting
CMS Performance Summary
CMS Performance Summary Data1. a. Kit ID:definesthecustomer’suniquekitnumberforeachmailing.
b. Kit Mailed:liststhedatetheSurveysmailingwasshipped. c. Original Evaluation:liststhedatetheevaluationwasoriginallygenerated.
2. RegulatedAnalyte:listsallregulatedtestsincludedinthespecialty/subspecialtyasdefinedbytheCLIAregulationsforthemodulesinwhichyouareenrolled.
3. SummaryofYourPreviousResponses:liststhetotalnumberofspecimensyouhavetestedandthenumberofacceptableresponsesforprevioustestingevents.
4. CAPNumberandKit:definestheCAPnumberandkitsequence.
5. TestEvent:identifiestheproductfulfillmentgroupandshipment.
6. SummaryofYourCurrentResponses:liststhetotalnumberofspecimensyouhavetestedandthenumberofacceptableresponsesforthattestingevent.
7. Current Event Performance Interpretation:indicateseithersatisfactory(≥80%)orunsatisfactory(<80%)performanceforeachanalyteandtheoverallperformanceforthespecialty/subspecialty.ForABOgroup,D(Rho)type,andcompatibilitytesting,ascoreof100%isrequired.Whenresultshavenotbeenreceivedfromyourlaboratoryforashipment,thisareawillbeblank.
• Ascoremaynotappear(fieldisblank)duetothefollowingreasons:
• LabhasindicatedtotheCAPthattheregulatedanalyteshouldnotbereportedtoagencies
Proficiency Testing Manual College of American Pathologists 33
Laboratory Legislation and CMS Reporting
• Challengeswerenotgraded,usingreasoncodesthatarenotreportedonthescorecard
• ThemethodreportedfortheanalyteiswaivedbytheCMS
• Noresultswerereported
8. Cumulative Performance Interpretation: indicatessuccessful(≥80%)orunsuccessful(<80%)performanceforeachanalyteandforthespecialty/subspecialty.ForABOgroup,D(Rho)type,andcompatibilitytesting,ascoreof100%isrequired.A<1>symboldenotesthatyourperformanceissuccessful;however,becauseyouhadlessthan80%onthepreviousmailing,youarestillatrisktobeunsuccessfulforthenextmailing.A<2>denotesyouarecurrentlysuccessfulbutatriskforthenexttwomailingsasyouwereunsatisfactoryforthismailing.Thesecodesareapplicabletoboththeanalyteandtheoverallspecialty/subspecialtyscores.A<3>denotescurrentlyunsuccessfulperformance.A<4>denotesthatscorecardperformanceispendingafutureevaluationormaynotbeapplicableduetodiscontinuedtestingortheuseofawaivedmethod.