John C. Liu COMPTROLLER · are responsible for providing special education-mandated services that are designed to meet the unique learning needs of eligible children with disabilities

City of New York

OFFICE OF THE COMPTROLLER

John C. Liu COMPTROLLER

MANAGEMENT AUDIT

Tina Kim

Deputy Comptroller for Audit

Audit Report on the Department of Education's Controls over the Monitoring of Individual Consultants for Mandated Services

MH11-060A

December 6, 2012

http://comptroller.nyc.gov

TABLE OF CONTENTS

AUDIT REPORT IN BRIEF .......................................................................... 1

Audit Findings and Conclusions .................................................................................. 1

Audit Recommendations .............................................................................................. 2

Agency Response .................................................................................................... 2

INTRODUCTION .......................................................................................... 3

Background ................................................................................................................. 3

Objective ...................................................................................................................... 4

Scope and Methodology Statement ............................................................................. 4

Discussion of Audit Results ......................................................................................... 5

FINDINGS AND RECOMMENDATIONS ..................................................... 6

Weaknesses in Monitoring Controls over Its Individual Consultants ............................ 7

Excessive Billing by Consultants for Questionable Service Sessions ...................... 7

Recommendations ................................................................................................. 10

Lack of Written Invoice-Review Policy and Procedures ......................................... 11


Weaknesses Found in DOE’s Service Verification Notice Process ........................... 13

Inadequate Tracking and Reviewing of Responses to Parent Verification Letters . 13

Parent Verification Letters Not Appropriately Sent to All Parents ........................... 14


Formal Contracts Not Executed for Three Sampled Individual Consultants Who Reached Contract Threshold ..................................................................................... 17


DETAILED SCOPE AND METHODOLOGY ............................................. 20

APPENDIX I

ADDENDUM

Office of New York City Comptroller John C. Liu MH11-060A 1

THE CITY OF NEW YORK OFFICE OF THE COMPTROLLER

MANAGEMENT AUDIT

Audit Report on the Department of Education’s Controls over the

Monitoring of Individual Consultants for Mandated Services

MH11-060A

AUDIT REPORT IN BRIEF

The Department of Education (DOE) provides primary and secondary education to over one million children in nearly 1,600 schools, employing approximately 74,000 teachers. To achieve its goals, DOE employs pedagogic staff and procures consultant services from companies and individuals. When existing staff of an office, learning center, district, or school cannot offer the expertise to accomplish educational goals, DOE may hire consultants.

Consultants may be corporate entities, institutions, or individuals who provide services on a temporary and occasional basis, possessing special skills and knowledge in various areas, but are not employees of DOE. Consultant services may include, but not be limited to, administrative, arbitration, legal, medical, and educational. The services of consultants are also used to provide educational assistance to children with disabilities under the Federal Education for All Handicapped Children Act (1975) which, together with subsequent amendments as currently reflected in the Individuals with Disabilities Education Act (IDEA) of 2004, guarantees access to a free, appropriate public education in the least restrictive environment to every child (from pre-school to age 21) with a disability. As set forth by IDEA, States and public agencies are responsible for providing special education-mandated services that are designed to meet the unique learning needs of eligible children with disabilities so as to ensure that they are prepared for further education, employment, and independent living.

Audit Findings and Conclusions

The audit found significant control weaknesses, which prevent DOE from effectively monitoring its individual consultants for mandated services. One such weakness is an inadequate billing review process, leading to billing irregularities that resulted in DOE making payments for services that may not have been rendered. As a result of the irregularities we found, we do not have reasonable assurance that the possibility of waste or fraud is being adequately controlled.


Some of the control weaknesses and billing irregularities that we found include the following: paper invoices not designed to capture service times; inadequate edit controls at the Vendor Portal1 to prevent inappropriate billing for sessions at unreasonable hours or on federal holidays; lack of a uniform and comprehensive system that maintains an ongoing tally of each consultant’s billings; and failure to track, log, and review parent verification letters.

Although the specific instances of control weaknesses and billing irregularities we identified in this report did not appear to be numerous, they were nonetheless noteworthy and significant when considered in their entirety. In addition, some of the problems we found––such as the inadequate edit controls at the Vendor Portal––can have ramifications on the entire population of consultants.

Audit Recommendations

We make 16 recommendations, including that DOE:

Implement more effective controls to protect against inappropriate billing. Such controls should include, but not be limited to, maintaining monthly aggregates of the billings in a month for each consultant before payment is made.

Improve the validation edit controls in Vendor Portal and review procedures for the manually submitted paper invoices to ensure accurate billing and compliance with DOE’s requirements, including the disallowance of services provided on federal holidays or at odd hours.

Establish written invoice-review policies and procedures that guide its reviewers and supervisors on how paper invoices should be prepared and processed. This should include procedures to verify the accuracy of the invoices, such as ensuring that signatures are authentic and reaching out to parents whose children received services from consultants who appeared to have billed DOE inappropriately.

Ensure that parents of students who receive tutoring services are also mailed verification letters.

Agency Response

In their response, DOE officials agreed to implement or stated that they have already implemented 14 of our 16 recommendations. However, DOE disagreed with two recommendations pertaining to sending verification letters to parents in non-English-speaking households and issuing formal contracts to consultants.

1 Vendor Portal is DOE’s online management payment tool used to record billings and payments for services provided by DOE vendors.


INTRODUCTION

Background

The New York City Department of Education (DOE) provides primary and secondary education to over one million children in nearly 1,600 schools, employing approximately 74,000 teachers. To achieve its goals, DOE employs pedagogic staff and procures consultant services from companies and individuals. When existing staff of an office, learning center, district, or school cannot offer the expertise to accomplish educational goals, DOE may hire consultants.

Consultants may be corporate entities, institutions, or individuals who provide services on a temporary and occasional basis, possessing special skills and knowledge in various areas, but are not employees of DOE. Consultant services may include, but not be limited to, administrative, arbitration, legal, medical, and educational. The services of consultants are also used to provide educational assistance to children with disabilities under the Federal Education for All Handicapped Children Act (1975) which, together with subsequent amendments as currently reflected in the Individuals with Disabilities Education Act (IDEA) of 2004, guarantees access to a free, appropriate public education in the least restrictive environment to every child (from pre-school to age 21) with a disability. As set forth by IDEA, States and public agencies are responsible for providing special education-mandated services that are designed to meet the unique learning needs of eligible children with disabilities so as to ensure that they are prepared for further education, employment, and independent living.2 (See Appendix for a complete breakdown of the types of consultants procured by DOE and the services they provide.)

In addition to IDEA, DOE follows Chancellor Regulation C-190, Pedagogical Personnel, Utilization of Consultants, and its Standard Operating Procedures (SOP), Utilization of Consultants, for the monitoring of individual consultants.3 Our Office has recently conducted two audits focusing on different aspects of consultant services: Audit Report on Champion Learning Center LLC’s Compliance with the Supplemental Education Services Vendor Agreement with the Department of Education (MD11-106A), issued May 16, 2012, and Audit Report on the Procurement of Direct Student Services by the Department of Education (FK10-147A), issued January 24, 2012. The first audit determined whether one large consultant agency adhered to key contract provisions with DOE while the second audit looked at whether DOE complied with procurement requirements for a subset of consultant services. The current audit was to focus on DOE’s monitoring over all the individual consultants.

2 Mandated services are special education services mandated under IDEA and include Special Education Teacher Support Services (SETSS) and Related Supportive Services (related services). SETSS are mandated tutoring services that are specially designed to supplement instruction provided by a special education teacher. Related services are mandated developmental, corrective, and other supportive services and are defined as “developmental, corrective, and other support services required to assist a student with a disability to benefit from instruction.” Related services may include speech-language pathology and audiology services, physical and occupational therapy, counseling services, school health and social work services. Non-mandated services are not mandated under IDEA and include non-pedagogic services, such as administrative services. & 3 Chancellor Regulation C-190 is limited to individual consultants who provide administrative services while the SOP covers individual consultants who provide other-than-administrative services––which include mandated services.


Objective

The objective of the audit was to determine the adequacy of DOE’s controls over the monitoring of individual consultant services.

Scope and Methodology Statement

We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objective. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objective except for the lack of sufficient appropriate evidence to determine the population of DOE individual consultants who provided mandated services and the population which provided non-mandated services. This severely restricted our objective to tests of controls for mandated services consultants as noted below. This audit was conducted in accordance with the audit responsibilities of the City Comptroller as set forth in Chapter 5, §93, of the New York City Charter.

We made numerous attempts over an extended period of time to obtain from DOE a comprehensive list of all the individual consultants who provided services for the audit scope period. Eventually, at our suggestion, DOE provided us with a database of individual consultants for whom 1099-MISC forms had been issued. During audit fieldwork, we obtained from DOE officials the parameters that they said would allow us to segregate those consultants who provided non-mandated services from those who provided mandated services. However, those parameters did not adequately segregate mandated and non-mandated service providers; a majority of the consultants listed on the “non-mandated” list were actually mandated service providers. As a result, we were prevented from identifying reliable populations of consultants who provided mandated and non-mandated services, which hindered our ability to report on the adequacy of DOE’s efforts to monitor all of its independent consultants. In addition, we attempted to assess the completeness of the 1099-MISC list to other databases, but because of the limitations and incompatibility of the lists we obtained, we were unable to determine the completeness of the 1099-MISC list.

Subsequent to the exit conference, DOE provided a different set of parameters intended to segregate mandated and non-mandated service providers. Our review of the “non-mandated” list generated from this second set of parameters also identified a number of consultants who provided mandated services. Consequently, this additional information did not change the effect of the limitation to our intended audit objective.

As a result, our objective was restricted to tests of various DOE controls for the purpose of identifying control weaknesses that could affect DOE’s monitoring of consultants for mandated services. Our review and the conclusions disclosed in this report are focused on consultants who provided SETSS and related services.

The scope of this audit was Fiscal Years 2010 and 2011. We extended our review of controls to January 2012 for additional tests that were required to adequately evaluate the controls in place. Please refer to the Detailed Scope and Methodology at the end of this report for the specific procedures and tests that were conducted.


Discussion of Audit Results

The matters covered in this report were discussed with DOE officials during and at the conclusion of this audit. A preliminary draft report was sent to DOE officials on July 27, 2012, and was discussed at an exit conference held on August 13, 2012. On October 4, 2012, we submitted a draft report to DOE officials with a request for comments. We received a written response on October 19, 2012. In their response, DOE officials agreed to implement or stated that they have already implemented 14 of our 16 recommendations. However, DOE disagreed with two recommendations relating to sending service verification letters to parents in non-English-speaking households and issuing formal contracts to consultants.

The full text of the written comments of DOE’s response is included as an addendum to this report.


FINDINGS AND RECOMMENDATIONS

Our audit disclosed significant control weaknesses, which prevent DOE from effectively monitoring its individual consultants for mandated services. We found inadequate controls and an inadequate billing review process in place, leading to billing irregularities that resulted in DOE making payments for services that may not have been rendered. As a result, we do not have reasonable assurance that the possibility of waste or fraud is being adequately controlled.

Table I summarizes some control weaknesses and billing irregularities that we found and their effect. These weaknesses and irregularities are discussed in greater detail below.

Table I

Summary of Control Weakness and Billing Irregularities

Description of Control Weakness or Irregularity

Control Weakness

Billing Irregularity

Potential Effect

Paper invoices not designed to capture service times.

√ Potential for the consultant to submit bills for overlapping sessions.

Billing review process that does not adequately capture billing for sessions at unreasonable hours or on federal holidays.

√ √ Consultants being paid for sessions that may not have occurred or were on days prohibited by DOE.

Inadequate edit controls at the Vendor Portal to prevent inappropriate billing for sessions at unreasonable hours or on federal holidays.

√ √ Consultants being paid for sessions that may not have occurred or were on days prohibited by DOE.

Lack of uniform and comprehensive system that maintains an ongoing tally of each consultant’s billings.

√ √ Inappropriate payments for erroneous or improper billing submissions.

Review procedures that provide inadequate guidance or supervision to invoice reviewers on the appropriate way to review paper invoices.

√ √ Inconsistencies and inaccuracies could occur in the review process, leading to improper payments.

Failure to track, log, and review parent verification letters.

√ √ History of parents’ concerns of non-delivery of student services is not noted or how DOE addressed and rectified the parents’ concerns or complaints, leading to inappropriate payments.

Parent verification letters are not sent to all parents, only to those who receive related services.

√ Parental complaints of non-delivery of other-than-related services (e.g., mandated tutoring services) may go unaddressed, leading to inappropriate payments.


Although the specific instances of control weaknesses and billing irregularities we identified in this report did not appear to be numerous, they were nonetheless noteworthy and significant when considered in their entirety. In addition, some of the problems we found––such as the inadequate edit controls at the Vendor Portal––can have ramifications on the entire population of consultants.

During the course of the audit fieldwork and testing, the auditors had identified several significant control weaknesses regarding DOE’s consultant monitoring process. On January 26, 2012, we sent DOE’s Auditor General a letter detailing our concerns so that DOE could take immediate corrective action. DOE’s Auditor General responded in a March 21, 2012, letter, agreeing that there is a need for greater controls and describing the steps that DOE will implement. We commend DOE for taking our concerns seriously and for its efforts to implement new protocols to improve the agency’s monitoring efforts. We did not conduct any audit tests to assess the effectiveness of the remedies DOE has since implemented to rectify the control weaknesses identified in our letter.

The following sections of this report discuss our findings in greater detail and make reference to the remedies that DOE agreed to implement as stated in its response.

Weaknesses in Monitoring Controls over Its Individual Consultants

DOE has significant control weaknesses that, if left unaddressed, compromise its ability to effectively monitor individual consultants to ensure that payments are made only for services that are required and actually rendered.

Excessive Billing by Consultants for Questionable Service Sessions

DOE has inadequate controls to identify irregular billing practices by its consultants, including charging for service sessions4 that were provided at unreasonable hours or on federal holidays. We also found that the Vendor Portal edit controls are inadequate.

Under IDEA, students with disabilities are offered special education with a full range of supplementary services and may include Special Education Teacher Support Services (SETSS) and Related Supportive Services (related services). Related service providers are required to submit their billing electronically via DOE’s Vendor Portal. Providers of SETSS tutoring services, on the other hand, follow a manual invoicing system, whereby they submit to DOE a signed original paper invoice5 of their billings for services provided to each child.

Hard-copy invoices submitted by SETSS consultants to the Bureau of Non-Public School Payables (NPSP) are manually reviewed by DOE for billing accuracy. In addition to identifying the child, the invoice requires the entry of service-related information, such as the date of service, the location where the service was provided, the group size, and the session rate.

4 The duration of a service session varies, depending on the service being provided, and can last anywhere from 15 minutes to a number of hours. 5 The SETSS invoice is a standardized document developed by DOE that SETSS consultants must use when billing for their services.


According to DOE, there is no specific written policy as to what are considered reasonable hours during which after-school SETSS tutoring services may be provided. The general consensus among DOE personnel is that services provided during the six-hour period between 3:00 p.m. and 9:00 p.m. are reasonable. However, the paper invoices are not designed to allow the providers to identify the time of day in which SETSS services were provided. This, we found, hinders DOE reviewers from identifying whether the time of day in which the services were provided was reasonable (such as at 4:00 p.m. versus 11:00 p.m.). It also hinders DOE from determining whether consultants are inappropriately charging for overlapping services (i.e., submitting bills indicating that multiple students were serviced at different locations at the same time).

Of the 12 consultants (three SETSS and nine related services) who were identified to have received payments in excess of $50,000, a sample of five consultants with the highest dollar value was selected and reviewed—three SETSS and two related services consultants. Of the five, we identified three who appeared to have padded their bills for SETSS services. These three consultants manually submitted 45 invoices among them for services rendered in a one-month period. For the three consultants, we found numerous instances where the number of hours billed by consultants for after-school tutoring exceeded six hours on a given day, meaning that some of the tutoring would most likely have had to occur after 9:00 p.m. In addition, for one of the nine related services consultants who submitted electronic invoices through the Vendor Portal, we found there were billings that had been submitted for odd hours.

Table II shows the summary of our review for the three SETSS consultants who appeared to have excessively billed DOE for after-school tutoring services for one selected month.

Table II

Billing for Service Sessions Beyond Reasonable Hours (in excess of six hours on a given day) by

Three SETSS Consultants* (Manually Processed)

Consultants

(Month Reviewed)

No. of Service Session

Billed during the Selected

Month

Payments Made for

Billed Service Sessions during the Selected

Month

No. of Service

Sessions Billed beyond Reasonable

Hours

Percentage of Service Sessions

Billed beyond

Reasonable Hours

Payments Made for

Unreasonable Service

Sessions Billed by Consultants

Percentage of Payments Made

for Unreasonable

Service Sessions Billed by Consultants

Consultant A (March)

339 $14,609.04 179 53% $7,514.42 51%

Consultant B (November)

195 $8,983.72 63 32% $2,644.74 29%

Consultant C (June)

282 $10,422.21 156 55% $5132.73 49%

Total 816 $34,014.97 398 49% $15,291.89 45%

*Session rates for SETSS services vary. For example, group sessions of two students are calculated at $31.49 per session per student while individual session rates are calculated at $41.98 per session.

Although these three consultants billed for services provided at unreasonable hours, one of these consultant’s billing was particularly egregious. We found 15 instances where the consultant billed for after-school tutoring services for excessive hours during the month


reviewed. In seven instances, the consultant claimed to have worked 16 hours per day, and in eight instances, the consultant claimed to have worked 15 hours per day. While we could not determine with certainty that the instances of billing for excessive service hours were fraudulent, they raise serious questions about their legitimacy.

These instances of inappropriate billing were not identified in large part because DOE did not perform an adequate review of paper invoices. The paper invoices are not designed to allow the providers to identify the time of day in which SETSS services were provided. In addition, the lack of a comprehensive system that keeps a running tally of billing sessions submitted by each consultant hinders the invoice reviewers from adequately identifying all instances of inappropriate or excessive billing by SETSS consultants. If DOE had properly reviewed the paper invoices by, for example, entering the data in a spreadsheet and determining the total billings in a month for each consultant before payment was made, it would have identified the same instances we found.

We also found that DOE paid for SETSS and related services provided on federal holidays, contrary to DOE’s policy. . A review of the paper invoices submitted by 16 sampled consultants revealed that nine consultants (56 percent) billed DOE for 52 hours ($2,291.12) for services provided on federal holidays.

We found similar issues with the electronically processed related services consultant billings submitted through the Vendor Portal. The Vendor Portal contains edit controls that are supposed to reject billing errors entered by the related service consultant when billing for their services. Nevertheless, we found it did not capture billings for odd hours submitted to the system for payments. Specifically, we found that one of nine sampled related service consultants had at least five instances where service was provided at odd hours. The five sessions of occupational therapy were provided to three different students between 3:00 a.m. and 4:30 a.m. on different days. Although these entries could have been the result of data-entry errors, the Vendor Portal should have rejected them if the system had been adequately designed to reject such entries.

We also found that the Vendor Portal did not reject billings for services provided on federal holidays. We also reviewed the 5,571 electronic billings entered through the Vendor Portal by nine consultants and found that five (55 percent) of these nine consultants billed DOE 39 times for 52 hours ($3,019) for services provided on federal holidays during Fiscal Years 2010 and 2011. These issues are of concern because the lack of adequate Vendor Portal edit controls would affect any consultant billings being processed through the system.

Corrective Action Planned by DOE

In our letter to DOE’s Auditor General, we noted that DOE does not comprehensively review invoices submitted by consultants and identified a number of billing irregularities, including potentially fraudulent billing. Specifically, we noted that photocopies of duplicate invoices were submitted for payment; billing for consultant services provided to students were given at odd hours of the day or night; and consultants with exceptionally high billings during a single month staggered the submission of multiple invoices. In light of the severity of the control weaknesses (in addition to others that are discussed further below), we asked that DOE take immediate action to address those conditions and ensure that vendors are only paid for services that are legitimately provided. We also asked that DOE let us know the steps it intends to take to address the conditions.


In his March 21, 2012, letter, DOE’s Auditor General recognized the need to limit service hours and restrict service delivery to certain hours of the day, and agreed to implement changes to adhere to the new policy and investigate past conditions. In addition, he noted that, if upon follow-up inquiry it appears that there is a need for further review, the matter will be referred to the Special Commissioner of Investigation (SCI).

Recommendations

DOE should:

1. Record the start and end time for all consultant services, including SETSS.

DOE Response: “The recommendation has been implemented.”

2. Implement more effective controls to protect against inappropriate billing. Such controls should include, but not be limited to, maintaining monthly aggregates of the billings in a month for each consultant before payment is made.

DOE Response: “The hours between which a student can receive services from independent related service and SETSS providers has [sic] been built into the electronic payment system. The service hours, which will be aggregated monthly, will define the outside limit of the number of hours providers may bill daily until refinements further limiting the hours are developed and incorporated into the system.”

3. Communicate to its consultants what reasonable hours are during which they may provide tutoring and related services.


4. Improve the validation edit controls in Vendor Portal and review procedures for the manually submitted paper invoices to ensure accurate billing and compliance with DOE’s requirements, including the disallowance of services provided on federal holidays or at odd hours.


5. Implement the controls identified in the Auditor General’s March 21, 2012, response to our letter, including ensuring that: information from the Related Service Authorization (RSA) forms6 and the SETSS billing invoices is entered into DOE’s electronic payment system; restrictions are placed on the hours of day when consultant services can be delivered; past conditions are assessed by running reports of providers who billed electronically and who provided services beyond reasonable hours; and reports are run on a quarterly basis for the manually submitted invoices to flag billings for excessive hours. Instances requiring further review should be referred to SCI.

6 RSA forms are issued to the parents of children who have been evaluated and have been determined by DOE to need special services under IDEA. The RSA forms authorize the parents to use the services of appropriately licensed independent consultants if those services cannot be provided by DOE personnel or a contracted agency. ;


DOE Response: “The recommended controls have been implemented. Referrals have already been made to SCI; the DOE will make further referrals as appropriate.”

Lack of Written Invoice-Review Policy and Procedures

DOE has not developed formally written comprehensive policies and procedures that are maintained in a centralized depository for the review process for manually submitted paper invoices (SETSS). Review procedures that are not clearly promulgated in a centralized depository can result in confusion among reviewers as to what is acceptable and can lead to the acceptance of inappropriate billing. However, even where clear policies exist and are unambiguous, DOE’s invoice-review process does not consistently capture instances of non-compliance.

DOE officials stated that they do not have formal written policies or procedures to guide their personnel on how invoices should be reviewed and processed. Although DOE is mandated by both Federal and State laws to provide special education and related services to eligible children with disabilities, these laws do not outline the procedures that DOE should follow to review those invoices. Furthermore, both Federal and State laws give DOE wide latitude to implement its own policies and procedures to efficiently and accurately administer the special education mandates. These laws do not preclude DOE from adopting and incorporating in its own policies any provisions it may deem necessary to further instruct workers on how to do their jobs effectively.

DOE’s written procedures as promulgated in its SOP stipulates that the information provided by the consultant and the accuracy of amounts billed be verified. However, no mention is made as to how that is to be done or the steps that must be followed to ensure, for example, that the signatures on the invoices are not photocopied and the signature dates do not predate the indicated service dates. Because DOE does not have clear and specific procedures for its reviewers and supervisors to ensure that SETSS invoices are properly reviewed for potential billing irregularities or fraud, there is no assurance that discrepancies and instances of consultants’ non-compliance will be detected. Billing irregularities will continue to go undetected without clear and specific written policies and procedures that are promulgated in a centralized depository that can be referred to by DOE personnel.

Even where clear policies exist, DOE’s review process does not consistently identify invoices that do not comply with stated policy. For example, DOE’s policy of disallowing payments for services rendered on federal holidays is clearly printed on the paper invoices. However, as previously reported, we found questionable billings. Additionally, DOE requires the consultant to get the parent’s signature on the invoice as an attestation that the sessions were provided as reported in the invoice. However, these policies are not consistently considered by the reviewers when examining the invoices submitted. Our review of the paper invoices for the three SETSS vendors who received payments in excess of $50,000 (out of the 16 that we sampled) disclosed that DOE employees did not consistently ensure that the proper signatures were in place; or that there were no mathematical errors in the invoices. For the three vendors whose invoices we reviewed, we found that all three vendors had a total of 27 irregularities that DOE did not identify. These irregularities included instances of invoices that had been photocopied, with service dates whited out and changed or not listed at all, or where the original parent’s signature was missing or predated the date the service was provided. Insufficient oversight and supervisory review allowed these instances to go undetected.


Although the instances we identified did not appear to be numerous when compared to the total number of 480 invoices reviewed for the three vendors, they were nonetheless noteworthy and significant when considered in combination with the other billing irregularities that went undetected.


In his March 21, 2012, letter, DOE’s Auditor General stated that DOE’s payable staff responsible for reviewing hard-copy invoices has been provided with additional review protocols for the manually submitted invoices. DOE will now require that any photocopy of an invoice be supported with a written statement by the provider affirming that the original had been sent to the Department, that the photocopy is a true and accurate copy of the original, and that the services had been provided as indicated and were not included within a previously submitted invoice. In addition, invoices that appear suspicious will be set aside and, if necessary, referred to SCI. Furthermore, DOE data-entry personnel will begin entering information using hard-copy RSA and SETSS invoices into its “electronic payment system” so that the controls and edits inherent in that system can be applied.

We commend DOE for the steps it intends to take. However, additional steps can be taken, including reaching out directly to parents whose children received services from consultants who appeared to have billed DOE inappropriately in order to verify the accuracy of the invoices submitted. DOE can also initiate random or selective audits of consultants’ billing histories to ensure that their original records (e.g., appointment schedules, logs) match their bills.

Recommendations

DOE should:

6. Establish written invoice-review policies and procedures that guide its reviewers and supervisors on how paper invoices should be prepared and processed. This should include procedures to verify the accuracy of the invoices, such as ensuring that signatures are authentic and reaching out to parents whose children received services from consultants who appeared to have billed DOE inappropriately.

DOE Response: “The recommendation has been implemented in large part. Written guidelines have been disseminated by NPSP; a more comprehensive manual is in production. As for ensuring the authenticity of signatures, the DOE seeks clarification from the Comptroller respecting exactly what steps the DOE can take to derive such assurance other than making a visual comparison among provider and parent signatures that were already submitted by the same provider. As for ‘reaching out to parents whose children received services from consultants who appeared to have billed DOE inappropriately,’ the DOE will follow protocols requiring that such matters be referred to SCI.”

Auditor Comment: The visual comparison of signatures presented by DOE appears to be a step in the right direction. Service verification letters could also be used as a mechanism to verify the authenticity of signatures. However, the additional steps it decides to follow to obtain reasonable assurance that the signatures are authentic are entirely up to DOE.


7. Enhance its review process, which may include performing random or selective audits of consultant billings and contacting the parents to verify the accuracy of the invoices submitted.

DOE Response: “The DOE agrees with the recommendation . . . The DOE Office of Auditor General already has initiated a review of judgmentally-selected invoices, including those submitted by individual consultants, for services provided in School Year 2011/2012. Findings will be referred to SCI as warranted.”

8. Implement the controls identified in the Auditor General’s March 21, 2012, response to our letter to ensure that DOE staff is provided with protocols that should be followed to make sure photocopied invoices are accurate representations of services provided. Invoices that appear suspicious (such as whited-out dates or signatures that appear to be photocopied) should be referred to SCI.


Weaknesses Found in DOE’s Service Verification Notice Process

For bills electronically processed through the Vendor Portal, DOE uses service verification letters to confirm that the services being billed by consultants were indeed provided. However, we found that DOE does not have an effective system in place for the review and follow-up of the responses it receives to those letters. Our review of the parental responses received by DOE from March 2010 to December 2011 found that in 41 percent of them, the parents stated that their children were not receiving the services for which the consultants billed. However, we found DOE did not track or follow up on the parents’ responses. In addition, we found that DOE does not send service verification letters for the manual invoices it receives, which includes all tutoring-related consultant services. Finally, service verification letters are sent out in English only, even to non-English-speaking households.

Inadequate Tracking and Reviewing of Responses to Parent Verification Letters

DOE does not maintain a log of the responses it receives in reply to the service verification letters it mails to the parents of children who receive related services. Our review further determined that there were no written policies and procedures in place that direct DOE personnel on how parents’ responses to the verification letters are to be processed.

Consultants who provide related services submit their billing electronically via the Vendor Portal. At the end of each month, the Vendor Portal generates service verification letters that are mailed to parents asking them to review the service dates and times that the consultants stated on their bills. If the parents disagree with any billing dates or times, they are asked to contact DOE either through email or on the comment section on the verification letter and then fax or mail the responses to the assigned DOE personnel referred to in the letter. Throughout the audit, DOE officials have stated that the mailing of verification letters to parents is a major control mechanism in place to ensure that vendors are providing the services for which they are billing.


However, we consider the use of service verification letters by DOE to be a weak control feature. It relies heavily on proactive parents in stable households. Some of these parents may be dealing with severely physically, mentally, or emotionally challenged children and may not be able to respond to the letters as DOE would like. In addition, DOE has limited assurance that the verification letters are even reaching the parents because DOE does not follow up on change of address notices.

When we asked to review the logs or other recording mechanisms used by DOE to track the parents’ responses, we were informed that none existed. When we asked DOE officials on January 11, 2012, to be given all the responses DOE had received from the parents for Fiscal Year 2011, we were provided a box containing 194 letters received between March 2010 and December 2011. DOE officials stated that those were all the responses they received and that none had been reviewed. When asked why the parents’ responses had not been reviewed, DOE personnel informed us that they had limited staff and were currently in the process of training staff to review them.

Upon review of the responses, we found that 82 (42 percent) of the 194 were returned either by parents notifying DOE that the family no longer resides at the address or by the post office as undeliverable. For the remaining 112 responses, parents disputed claims for 46 (41 percent) of them. Of the 46 parental responses, 13 were clear denials that any services had been provided. For these 13, the parents stated the child was not in New York, was away on vacation, or had not received any services in months.

Despite parents’ efforts to have their addresses updated in DOE’s databases, we found two particular instances where the parent clearly attempted to notify DOE of a change in address, yet DOE continued to send verification letters to them at their former addresses. In one instance, the parent wrote her new address on a verification letter mailed to her in April 2011 yet DOE sent another verification letter to the old address in August 2011. In a second instance, another parent provided her corrected address and phone numbers in her response to a letter sent in March 2011. But again, DOE sent another verification letter to the former address in August 2011.

The fact that DOE has taken no steps to ensure addresses are updated is of concern, especially considering DOE’s position that the verification letters are a major control mechanism. DOE’s lack of follow-up to address the parents’ responses indicating that services were not provided increases DOE’s risk of potential fraud and waste. In addition, DOE’s lack of an effective mechanism to log and track the responses it receives from parents and how it handles their concerns raises questions as to whether we were provided with all verification letter responses sent by the parents. Without an effective tracking system, there is an increased risk that some responses may be misplaced or lost and that vendors may be inappropriately paid for services not provided.

Parent Verification Letters Not Appropriately Sent to All Parents

DOE does not mail verification letters to parents whose children receive SETSS tutoring services (manually processed bills). In addition, the verification letters that were mailed to the parents were provided in English only, even to households that may speak a language other than English.

As noted above, SETSS tutoring service providers do not use the Vendor Portal; they instead manually submit paper invoices. This excludes a significant number of parents who, upon


receiving the verification letters, could help DOE identify billing irregularities by SETSS providers and address parental concerns about whether their children are receiving needed services.

Also, parents in non-English-speaking households are faced with the task of possibly not knowing or understanding what is being communicated in those letters. If parents in non-English-speaking households do not have relatives or acquaintances who can translate, they may miss the opportunity to respond to DOE’s request that they verify that services were indeed provided to their children.

According to DOE’s Standard Operating Procedures Manual (February 2009), schools and the Committee on Special Education Offices are responsible for providing notices to parents in appropriate DOE-identified languages7 when it has been determined that their primary language is not English. Although we concede that this citation does not directly refer to parental verification letters automatically generated by the Vendor Portal, enabling the Vendor Portal to generate these letters in the primary language of the household would enhance DOE’s monitoring of consultant services to ensure that children are actually receiving the services to which they are entitled.


In the letter we sent to DOE’s Auditor General, we noted that service verification letters to parents were only sent for billings submitted electronically through the Vendor Portal. As a result, notification letters were never sent to parents whose children received tutoring services because the consultants manually submitted their invoices when billing for those services. Verification letters are sent out in English only, even to non-English-speaking households. We also pointed out that DOE does not have an effective system in place for the review and follow-up of those service verification letters it does receive. In his March 21, 2012, letter, DOE’s Auditor General agreed that “it is essential to create a tracking log and protocols for reviewing and documenting cases where a parent has advised that services were not delivered at the time and/or place reported by the provider.”

Recommendations

DOE should:

9. Develop written procedures and disseminate those procedures to the appropriate personnel describing how parental responses to service verification letters are to be handled.

DOE Response: “The recommendation has been implemented. However, we do not want to let pass without comment that despite the auditors’ insistence that the DOE views the mailings to parents as a ‘major control mechanism,’ the mailings are merely additional to other, more critical controls which have been built into the payment system.”

7 DOE-identified languages include Arabic, Bengali, Chinese, Haitian Creole, Korean, Russian, Spanish, and Urdu.


Auditor Comment: The other controls to which DOE is referring, such as the Vendor Portal edit controls mentioned in this report, are primarily aimed at determining whether bills submitted by vendors appear reasonable. However, they are not designed to ascertain whether the billed services, no matter how reasonable they appear, were actually provided. The service verification letter is the only control identified by DOE that addresses that concern. Accordingly, in the absence of any information to the contrary, we see no reason to alter our finding.

10. Investigate the 13 parental responses to the notification letters where the parents stated the child had not been in New York, was away on vacation, or had not received any services in months.


11. Implement a process by which students’ addresses are updated to ensure that the most recent contact information is on file.

DOE Response: “While NPSP has the ability to obtain data from ATS, NPSP does not have authority to make entries or amend address information in that system . . . However, since implementation of the recommendation is worth perusing, NPSP has been working with the ATS technology team to develop a means by which NPSP can bring to a school’s attention a wrong address or notification of a change of address. The school then will be responsible for contacting the parent to determine whether the school should change the information in ATS.”

Auditor Comment: Any process that DOE implements that ensures that parents receive verification letters should enhance the effectiveness of this control measure.

12. Send verification letters to parents in non-English-speaking households in the appropriate DOE-identified language.

DOE Response: DOE disagreed. “DOE will not implement this recommendation because, first, the monthly service verification letters sent to parents are not within the category of communication that must be drafted in the primary language of the household. Indeed, the auditors concede that much. And, second, the letter urges parents to review the service information and respond to NPSP if a discrepancy is discovered. Even if NPSP had the resources to have non-standardized letters translated into one of nine recognized languages every month, there certainly is no efficient or cost effective way for NPSP to have responses written in a language other than English translated or to communicate to the non-English-speaking parent if further information is required.”

Auditor Comment: As we stated above, the service verification letter is the only control identified by DOE that attempts to verify that billed services were actually provided. By sending this letter in English only, the effectiveness of this control for non-English-speaking households is questionable and leaves DOE with no effective way to verify that services billed for students in these households were actually provided. Additionally, DOE overstates the difficulty in translating this letter. Contrary to DOE’s assertion, the letter is a standard letter; the only difference between letters is the addressees to whom the letters are sent and the provided


service-related information (i.e., the consultant’s name, location, and the date and time during which the service was provided). Accordingly, although it is not required to do so, we believe that DOE should make efforts to translate this letter for non-English-speaking households until such time that it develops an alternate control that addresses the above-stated concern. As such, we reiterate this recommendation.

13. Ensure that parents of students who receive tutoring services are also mailed verification letters.

DOE Response: “Hardcopy invoices . . . are being processed by NPSP through data entry into the automated payment system. As such, the services will be picked up for mailing of service verification letters.”

14. Implement the controls identified in the Auditor General’s March 21, 2012, response to our letter to ensure that the new protocols for tracking and reviewing parent verification letters and parental responses have been implemented; take appropriate action regarding the 61 responses it had identified; and report cases requiring further review to SCI.


Formal Contracts Not Executed for Three Sampled Individual Consultants Who Reached Contract Threshold

For the three consultants (two SETSS and one related service) in our sample who billed in excess of $100,000 in Fiscal Year 2010 or 2011, DOE did not enter into formal contracts as required by DOE regulations. According to DOE’s SOP on the Utilization of Consultants and Chancellor Regulation C-190, if total expenditures are anticipated to be in excess of $100,000, a formal Request for Proposal (RFP), including public advertising, is required, and a contract must be prepared and executed. Based on the stated procedures, the Division of Contracts and Purchasing identifies SETSS and related service providers who were paid $80,000 or more in any one year to determine whether a formal contract may be required to be executed the following school year. Once the consultants reach the $100,000 threshold, the contract procurement process begins, where an RFP is required and a formal contract is prepared. This helps ensure that the lowest qualifying bids are awarded the contract and that funds are adequately encumbered at the start of the budget process. The consultants are then subjected to background checks, are required to complete VENDEX8 forms, and are required to meet more comprehensive insurance requirements. The underlying purpose of DOE’s procurement policies and procedures is to ensure that public money is used wisely and consistent with law by, among other things, guarding against favoritism and fraud.

For the three consultants who were found to have received payments in excess of $100,000 in either Fiscal Year 2010 or 2011, we went back to the previous fiscal year to see if they also

8 VENDEX is a comprehensive contractor database maintained by the City. It includes vendor questionnaires which contain information about vendors’ principals, ownership, affiliations, and involvement in government investigations; contract performance evaluations; tax filing status; outstanding liens and warrants; and cautionary information disclosed by any City agencies or law enforcement organizations.


received payments in excess of $80,000. In all three instances, we found that to be the case. However, there were no indications in the files for these three consultants that suggest a contract was prepared or executed. Table III shows a comparison of the amounts paid by DOE for services provided by the three consultants for the year randomly selected in our sample versus the previous year’s payments made to them.

Table III

Contract Should Have Been Executed for Three Contractors

Consultant Fiscal Year Reviewed Prior Fiscal Year

Year Payments over $100,000

Year Payments over $80,000

Consultant A (SETSS) 2011 $110,662.64 2010 $80,782.29 Consultant B (SETSS) 2011 $137,054.23 2010 $87,161.37 Consultant C (Related 2010 $155,685.00 2009 $103,395.00

We commend DOE for having clear policies and procedures in this area and a mechanism to identify consultants who may be reaching the threshold where an RFP may be required to be prepared and a formal contract may need to be executed. However, no mechanism is effective unless it is fully implemented (e.g., its policies and procedures are consistently followed).

Recommendations

DOE should:

15. Ensure that individual consultants’ expenditures are periodically monitored so that consultants approaching the $100,000 threshold can be red-flagged and notified that a contract process needs to take place in order for them to continue receiving payments for services.

DOE Response: “DOE will periodically review expenditures for RSA and SETSS services and direct vendors to submit Vendex and other appropriate materials to complete background checks preparatory to entering a contract. Upon its failure to do so, the vendor will be subject to escalating warnings by the DOE; ultimately, a vendor that still has not complied will be barred in the DOE’s accounting system from obtaining payments until all required documents are provided and a contract is executed.”

16. Issue formal contracts to consultants where appropriate as per DOE regulations.

DOE Response: DOE disagreed. “The services that are the subject of this report . . . are not subject to the procurement rules. A competitive RFP is not an appropriate process for ‘independent consultant’ services since in those cases, although the DOE sets the rates, the student’s parent – not DOE – selects the vendor.”


Auditor Comment: We are puzzled by DOE’s response to this recommendation. This is the first time that DOE has presented this argument, although it had numerous opportunities to do so when this issue was raised during the course of the audit. Furthermore, nowhere in DOE’s Standard Operating Procedures on the Utilization of Consultants and Chancellor Regulation C-190 does it state that an RFP is not an appropriate process for independent consultants nor does DOE cite any other rules that indicate that consultants for mandated services are exempted from these regulations. Although we do not dispute that parents may select the vendor, DOE is ultimately responsible for the payments made. As we note in our report, the underlying purpose of procurement policies and procedures is to ensure that public money is used wisely and consistent with law by, among other things, guarding against favoritism and fraud. Without such assurance, the risk that favoritism or fraud may occur is greater. Accordingly, in the absence of sufficient evidence to the contrary, we reiterate our recommendation.


DETAILED SCOPE AND METHODOLOGY

We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objective. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objective except for the lack of sufficient appropriate evidence to determine the population of DOE individual consultants who provided mandated services and the population who provided non-mandated services. This severely restricted our objective as noted below. This audit was conducted in accordance with the audit responsibilities of the City Comptroller as set forth in Chapter 5, §93, of the New York City Charter.

We made numerous attempts over an extended period of time to obtain from DOE a comprehensive list of all the individual consultants who provided services for the audit scope period. Eventually, at our suggestion, DOE provided us with a database of individual consultants for whom 1099-MISC forms had been issued. During audit fieldwork, we obtained from DOE officials the parameters that they said would allow us to segregate those consultants who provided non-mandated services from those who provided mandated services. However, those parameters did not adequately segregate mandated and non-mandated service providers; a majority of the consultants listed on the “non-mandated” list were actually mandated service providers. As a result, we were prevented from identifying reliable populations of consultants who provided mandated and non-mandated services, which hindered our ability to report on the adequacy of DOE’s efforts to monitor all of its independent consultants. In addition, we attempted to assess the completeness of the 1099-MISC list to other databases, but because of the limitations and incompatibility of the lists we obtained, we were unable to determine the completeness of the 1099-MISC list.

Subsequent to the exit conference, DOE provided a different set of parameters intended to segregate mandated and non-mandated service providers. Our review of the “non-mandated” list generated from this second set of parameters also identified a number of consultants who provided mandated services. Consequently, this additional information did not change the effect of the limitation to our intended audit objective.

As a result, our objective was restricted to tests of various DOE controls for the purpose of identifying control weaknesses that could affect DOE’s monitoring of consultants for mandated services. Our review and the conclusions disclosed in this report are focused on consultants who provided SETSS and related services.

The scope of this audit was Fiscal Years 2010 and 2011. We extended our review of controls to January 2012 for additional tests that were required to adequately evaluate the controls in place.

To obtain an understanding of the objectives, responsibilities, and laws governing individual consultants, we reviewed:

Article 52-A §2590 (h) of the New York State Education Law of 1997;

Individuals with Disabilities Education Improvement Act of 2004;

Chancellor Regulation C-190, Utilization of Consultants;


DOE’s Standard Operating Procedures for Utilization of Consultants;

DOE’s Standard Operating Procedure Manual: The Referral, Evaluation, and Placement of School-Age Students with Disabilities;

DOE’s Special Education Services As Part of a Unified Service Delivery System (The Continuum of Services for Students with Disabilities);

A Parent Guide to Special Education Services for School-Age Children;

Sample of Related Service Authorization forms; and

Sample of Special Education Teacher Support Services Approval Letters.

We interviewed DOE officials from various departments, including the Committees on Special Education, Bureau of Non-Public School Payables, Human Resources, Budgeting, and Related Services. We also conducted a walk-through of DOE’s Electronic Invoicing System to understand how related service providers use it to submit billing requests to DOE and a walk-through of DOE’s Special Education Component System to understand how it interfaces with the Vendor Portal.

To assess the reliability and completeness of the data provided, we obtained a database of individual consultants who provided independent consultant services and were issued 1099-MISC forms for Fiscal Years 2008 to April 18, 2011, totaling $76,539,280 that was paid to approximately 6,878 consultants during that time. Our audit focused on two periods—Fiscal Years 2010 and 2011. As a result, our data reliability review was limited to the population of consultants who received payments from DOE for Fiscal Year 2010 to April 18, 2011. To understand the requirements for the submission of 1099-MISC forms and determine whether all consultants who received payments should have been included in the 1099-MISC database, we reviewed IRS Instructions for Form 1099-MISC. We used the parameters provided to us by DOE in an effort to segregate the consultants who received payments for mandated services from those who received payments for non-mandated services. To verify the reliability and completeness of the 1099-MISC database, we also attempted to reconcile it to other databases. However, we were unable to do so because of the limitations and incompatibility of the lists.

Controls over the Invoice‐Review Process

Using the data of individual consultants who were issued 1099s from Fiscal Years 2008 to April 18, 2011, a random sample of 30 individual consultants was selected for review from two periods—Fiscal Years 2010 and 2011 (15 from each fiscal year). In addition, we selected all 34 individual consultants in our population who were paid $50,000 or more in each of the two fiscal years.

We excluded 32 of the 64 consultants’ files from our review because the payments made to these consultants were based on impartial hearing decisions, which result from a mandated adjudication process that is afforded to parents of students whose request for services have been reduced or eliminated.

For the remaining 32 files, we requested and obtained all 16 files containing invoices that were manually processed and the electronic data records for nine of the 16 files that were electronically processed. (DOE provided the electronic data for the remaining seven consultants on March 5, 2012, six months after our initial request on September 9, 2011.) We performed the following tests on the 25 files and data records we received initially (16 SETSS that were manually processed and nine related services that were electronically processed):


To determine whether the session rates paid to the consultants were accurate, we reviewed 966 manual invoices associated with the 16 consultants and compared the rates used in the calculations by DOE to the rates stated in the RSA/SETSS approval letters.

To determine whether payments made to consultants were mathematically accurate, we reviewed the 966 invoices associated with the 16 consultants’ files that were manually processed. We multiplied the number of sessions billed on each invoice by the session rate corresponding to the services provided.

To determine whether consultants were paid for services provided on federal holidays, we obtained a list of federal holidays for Fiscal Years 2010 and 2011 and then compared it against the dates billed by consultants. This test was done for the 966 invoices associated with the 16 consultants in our sample whose invoices were manually processed and the 5,571 electronic entries associated with the nine related service consultants in our sample whose invoices were processed via DOE’s Vendor Portal.

For 549 manual invoices associated with the five consultants who were paid the highest dollar amount during the reviewed period, we reviewed each original invoice to see whether the submitted invoices had original signatures and a copy of the approval letters. In addition, we also reviewed the invoices to ensure they were dated properly in accordance with DOE procedures.

To determine whether consultants billed DOE for related services provided at reasonable hours, we conducted a computer sort of the 5,571 electronic entries for the nine consultants who submitted electronic billings to ascertain the start and end times that were entered in the Vendor Portal system attesting that services had been provided. Because there were no written policies and procedures as to what constitutes reasonable hours, we interviewed DOE employees to determine what they considered ‘reasonable hours’ to be. The informed us that reasonable hours are between the hours of 3 p.m. and 9 p.m. Therefore, service sessions delivered after 9 p.m. were considered to be questionable. To determine the aggregate number of service hours and the associated billing amounts for services provided to each student, we selected the five consultants in our sample who were paid $50,000 or higher during Fiscal Years 2010 and 2011—three SETSS and two related service providers. We then judgmentally selected the month with the highest payments for each of these consultants. For the three consultants who we determined were paid for services that appeared to be excessive during that month, we reviewed the consultants’ bills for the selected months. We determined the number and the length of the tutoring sessions per day for each child for that month and calculated the aggregate number of service sessions billed beyond reasonable hours. This allowed us to determine the total amounts billed beyond reasonable hours by the three consultants.

Controls over the Parent Verification Letters

To ascertain whether DOE took appropriate action regarding service verification letters received from parents, we reviewed all 194 responses DOE claimed to have received between March 2010 and December 2011.

To determine approximately how much was paid to consultants for services that 13 parents clearly said their children had not received, we used the lowest service rates and multiplied it by the number of hours that the parents disputed.

Office of New York City Comptroller John C. Liu MH11-060A

APPENDIX I

ADDENDUM Page 1 of 7







Documents

John C. Liu COMPTROLLER · are responsible for providing special education-mandated services that are designed to meet the unique learning needs of eligible children with disabilities