GAO-21-287, Procurement Technical Assistance Program

PROCUREMENT TECHNICAL ASSISTANCE PROGRAM

Opportunities Exist for DOD to Enhance Training and Collaboration

Report to Congressional Committees

March 2021

GAO-21-287

United States Government Accountability Office

United States Government Accountability Office

Highlights of GAO-21-287, a report to congressional committees

March 2021

PROCUREMENT TECHNICAL ASSISTANCE PROGRAM Opportunities Exist for DOD to Enhance Training and Collaboration

What GAO Found The Defense Logistics Agency (DLA) of the Department of Defense administers and oversees the Procurement Technical Assistance Program (PTAP), which funds Procurement Technical Assistance Centers (PTAC) that provide counseling and training to businesses interested in government contracting. DLA plans to improve its oversight of the program.

• DLA has an online system for PTACs to report their quarterly performance, but the system has not allowed DLA to accurately aggregate data. As a result, DLA has been unable to easily assess the extent to which PTACs have met their performance goals. As of March 2021, DLA was updating its system so that it correctly aggregates these data. The system fixes should allow DLA to readily assess program-wide performance and goal attainment.

• Before fiscal year 2021, DLA delegated responsibility for award administration and performance reviews of PTACs to the Defense Contract Management Agency and Office of Naval Research. But those two agencies conducted performance reviews for 2017–2019 infrequently and the scope of those reviews varied. DLA assumed most administrative functions delegated to the agencies in fiscal year 2021 and plans to conduct quarterly performance reviews that should allow grants officers to more consistently identify and address performance issues.

DLA implemented new PTAC training requirements in 2020 and added a template (a standardized format) to track completion of training requirements on its website in May 2020, but made its use optional. By requiring use of the template, DLA could more consistently receive the information it needs to assess compliance with the requirements. DLA required the Association of PTACs to develop a training curriculum for PTAC staff and an associated proficiency test. However, DLA does not have access to the test results, even in the aggregate, which would help it measure the effectiveness of the training curriculum in improving PTAC counseling.

PTACs and Small Business Development Centers (SBDC) both provide government contracting assistance to small businesses, and this overlap can have positive and negative effects. PTACs serve businesses of all sizes but are limited to clients pursuing government contracts. SBDCs assist only small businesses, but cover a range of topics, including government procurement. The overlap could create more ways for businesses to access help on government procurement, but also could lead to clients getting advice that is not as good as from PTAC counselors (because SBDC counselors are not required to take procurement training). DLA also identified instances of PTAC counselors providing counseling on SBDC-related topics, potential double-counting of services, and comingling of funds at PTACs that are co-located with SBDCs. A collaborative agreement between DLA and the Small Business Administration (SBA), which administers the SBDC program, could help address the overlap between PTACs and SBDCs, such as by better clarifying their respective responsibilities when providing assistance on government contracting. View GAO-21-287. For more information,

contact William B. Shear at (202) 512-8678 or [email protected].

Why GAO Did This Study PTAP helps businesses pursue and perform under contracts with the Department of Defense (DOD), other federal agencies, state and local governments, and prime contractors. In 2020, the program operated 90 centers in 49 states, the District of Columbia, Guam, and Puerto Rico.

The House Report on the National Defense Authorization Act for Fiscal Year 2020 includes a provision for GAO to review PTAP. This report examines, among other things, oversight of PTAP, how DLA plans to track and evaluate new training requirements for PTACs, and whether PTAC and SBDC services overlap.

GAO reviewed DOD policies and procedures for overseeing PTAP and new training requirements and analyzed performance data for 2018–2020. GAO also interviewed agencies responsible for overseeing PTAP, as well as SBA, the Association of PTACs, and a selection of six PTACs and three SBDCs (selected based on geography, funding amount, and co-location of the centers).

What GAO Recommends GAO is making three recommendations to DOD: to require PTACs to use DOD’s template to track training, reach an agreement with the Association of PTACs to provide DLA with aggregate test results to measure training effectiveness, and develop a collaborative agreement with SBA. DOD concurred with all three recommendations. The Association of PTACs said its test could help in assessing training, but should not be the only metric. SBA had no comments.

https://www.gao.gov/products/GAO-21-287

https://www.gao.gov/products/GAO-21-287

mailto:[email protected]

Page i GAO-21-287 Procurement Technical Assistance Program

Letter 1

Background 3 DLA Plans to Improve Oversight by Aggregating Performance

Data and Performing More Frequent Performance Reviews 8 DLA Has New PTAC Training Requirements but Does Not

Require Uniform Tracking of Training or Obtain Trainee Test Results 16

DOD Has Been Considering Three Options for Transitioning PTAP 21

Some PTAC and SBDC Services Overlap, but DLA and SBA Do Not Have a Collaboration Agreement 23

Conclusions 29 Recommendations for Executive Action 30 Agency and Third Party Comments and Our Evaluation 30

Appendix I Objectives, Scope, and Methodology 34

Appendix II Comments from the Department of Defense 38

Appendix III Comments from the Association of Procurement Technical Assistance Centers 40

Appendix IV GAO Contact and Staff Acknowledgments 43

Table

Table 1: Procurement Technical Assistance Program Performance Statistics for Fiscal Years 2018–2020 11

Figures

Figure 1: Number of Procurement Technical Assistance Centers by State, Fiscal Year 2020 5

Figure 2: Simplified Organizational Chart of Department of Defense Agencies with Responsibility for the

Contents

Page ii GAO-21-287 Procurement Technical Assistance Program

Procurement Technical Assistance Program, as of January 2021 7

Figure 3: Selected Services Provided by Procurement Technical Assistance Centers (PTAC) and Small Business Development Centers (SBDC) Based on Fiscal Year 2020 Requirements 24

Page iii GAO-21-287 Procurement Technical Assistance Program

Abbreviations APTAC Association of Procurement Technical Assistance

Centers COVID-19 Coronavirus Disease 2019 DCMA Defense Contract Management Agency DLA Defense Logistics Agency DOD Department of Defense ONR Office of Naval Research OSBP Office of Small Business Programs PTAC Procurement Technical Assistance Center PTAP Procurement Technical Assistance Program SBA Small Business Administration SBDC Small Business Development Center

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.

Page 1 GAO-21-287 Procurement Technical Assistance Program

441 G St. N.W. Washington, DC 20548

March 31, 2021

Congressional Committees

The Procurement Technical Assistance Program (PTAP) was established in 1984 to expand the number of businesses capable of participating in government contracts. Under the program, a network of Procurement Technical Assistance Centers (PTAC) help businesses pursue and perform contracts with the Department of Defense (DOD), other federal agencies, state and local governments, and government prime contractors. In 2020, PTACs assisted over 57,000 clients and PTAC clients won over 327,000 government contracts and subcontracts valued at about $24.2 billion.

The House Report on the National Defense Authorization Act for Fiscal Year 2020 raised questions about PTACs not being fully utilized and PTACs duplicating services offered elsewhere, such as at Small Business Development Centers (SBDC).1 The report also includes a provision for us to assess several aspects of PTAP, including training requirements for staff and statistics used to measure PTAC performance. The authorization act itself includes a provision that moves oversight of PTAP from the Director of the Defense Logistics Agency (DLA) to the Under Secretary of Defense for Acquisition and Sustainment, effective October 1, 2021.

This report examines (1) how DOD oversees PTAP; (2) how DLA plans to track and evaluate its new training requirements; (3) what steps DOD has taken to transition PTAP to the Under Secretary of Defense for Acquisition and Sustainment; and (4) whether PTAC and SBDC services overlap.

To assess PTAP oversight, we reviewed DLA’s policies and procedures for overseeing PTAP, metrics used to evaluate PTAC performance, and planned changes to DLA’s oversight responsibilities. We also reviewed data on PTAC performance in fiscal years 2018–2020; we determined the data were sufficiently reliable for the purpose of reporting on the number

1H.R. Rep. No. 116-120, pt. 1, at 176 (2019). Under the SBDC Program, the Small Business Administration provides funding to SBDCs to offer counseling and training to current and prospective small business owners.

Letter


of counseling hours and events PTACs provided to businesses and on the number of active PTAC clients.2

To assess how DLA plans to track and evaluate new training requirements for PTAC staff, we reviewed DLA’s new training requirements and interviewed officials to understand how they were created. We compared documentation on DLA’s training against internal control standards and our leading practices for strategic training and development efforts.3 To understand how PTACs will implement the requirements, we spoke with the Association of PTACs (APTAC) and with six PTACs selected to represent different geographic regions, funding award sizes, and relationships with SBDCs (co-located or not).4

To obtain information on the PTAP transition, we interviewed officials at DLA and DOD’s Office of Small Business Programs (OSBP), which reports to the Office of Industrial Policy in the Office of Acquisition and Sustainment. We also spoke with representatives of APTAC to understand how DOD has communicated its plans.

To evaluate any PTAC and SBDC overlap, we reviewed documentation— such as authorizing statutes, regulations, terms and conditions, and funding guidance—for both programs. We also reviewed documentation on the entities that were awarded funds to host PTACs and SBDCs in fiscal year 2020 to determine how many entities hosted both programs. We interviewed officials at the Small Business Administration (SBA) and three SBDCs that were co-located with the PTACs we interviewed. See appendix I for additional details on our scope and methodology.

We conducted this performance audit from March 2020 to March 2021 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain

2DLA compiled these data manually based on data provided by PTACs. As discussed later in this report, we determined that the performance data captured in an online system created by DLA were not sufficiently reliable for our use.

3GAO, Standards for Internal Control in the Federal Government, GAO-14-704G (Washington, D.C.: Sept. 10, 2014); and Human Capital: A Guide for Assessing Strategic Training and Development Efforts in the Federal Government, GAO-04-546G (Washington, D.C.: Mar. 1, 2004). The guide summarizes attributes of effective training and development programs and is intended to help managers assess an agency’s training and development efforts and implement improvements.

4We use co-located to refer to being located at the same address or sponsored by the same organization or university.

https://www.gao.gov/products/GAO-14-704G



sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Procurement Technical Assistance Centers serve as a resource for businesses to obtain information and training on contracting opportunities with DOD, other federal agencies, state and local governments, and government prime contractors. Procurement professionals at PTACs provide assistance to businesses seeking to participate in government contracts. Most of the assistance they provide is free. PTACs assist businesses of all sizes but must make a concerted effort to seek out and assist small businesses, including those owned by women and service-disabled veterans.

PTACs are funded through awards that are structured as cost-sharing cooperative agreements between DLA and eligible organizations that apply and are approved to establish a PTAC. Those organizations that can enter into a cooperative agreement with DLA to establish a PTAC include state and local governments, Indian-owned economic enterprises, tribal organizations, and nonprofit organizations. Many PTACs are established by institutions of higher education. PTAC employees are employees of these organizations, not DOD.

PTAP funding opportunity announcements include information on eligibility, cost-sharing requirements, application requirements, and DLA’s process for reviewing applications. DLA officials stated that the agency determines funding for individual PTACs based on statutory limitations, applications received in response to the funding opportunity, funding appropriated to the program, and grants officers’ determinations of reasonable, allowable, and allocable costs in the budgets PTACs propose in their applications.5

5Statutory limitations for PTAC funding are based on the extent of the territory the PTAC covers. PTACs that provide statewide coverage or operate in more than one service area of the Bureau of Indian Affairs may receive up to $1 million. PTACs that provide less than statewide coverage may receive up to $750,000. PTACs that operate in one service area of the Bureau of Indian Affairs may receive up to $450,000.

Background

Funding, Geographic Distribution, and Numbers


According to DLA, each year, the agency first sets aside funds for PTACs serving tribal organizations.6 DLA then funds existing PTACs that have been complying with the program’s general terms and conditions, after which it reviews new applicants.7 The share of funding that DLA provides each PTAC depends on whether the PTAC is in a distressed area.8

In fiscal year 2020, PTAC awards totaled $43.8 million to 94 PTACs. Individual PTAC awards ranged from about $100,000 to $1 million. For instance, 57 PTACs received awards up to $500,000, 35 received awards of more than $500,000 and less than $1 million, and two received awards of $1 million.

The 90 PTACs operating in fiscal year 2020 operated in 49 states, the District of Columbia, Guam, and Puerto Rico (see fig. 1).9 South Carolina did not have a PTAC.10 PTAC service areas may be statewide or limited to a particular region of a state, or PTACs may serve federally recognized Indian tribes and Alaska Native entities. Fifty-two PTACs are considered statewide and 38 regional.11 According to DLA officials, six PTACs exclusively serve Native American-owned businesses. Depending on the size of the area they cover, some PTACs establish offices in more than one location.

6In fiscal year 2018, the five PTACs that served tribal organizations received about $2.7 million of the $3.6 million set aside for them. In fiscal years 2019–2020, the six PTACs that served tribal organizations received about $3.1 million and $3.4 million, respectively, of the $4.5 million set aside for them.

7DLA occasionally receives applications to cover areas already served by existing PTACs. In these instances, DLA evaluates the merit of having two PTACs providing service. Duplicate coverage is authorized by exception and is not standard.

8Distressed areas have per capita incomes that are 80 percent or less of the state average or unemployment rates that are at least 1 percentage point greater than the national average or are reservations. For PTACs in distressed areas, DLA provides up to 75 percent of funds. PTACs in nondistressed areas can request up to 60 percent of their funding from DLA. In fiscal year 2020, 41 PTACs served distressed areas.

9By “operating in 2020,” we mean the 90 PTACs that operated for the full fiscal year. Four additional PTACs received DLA funding in 2020, but they did not begin performance until or after October 1, 2020 (that is, in fiscal year 2021).

10According to DLA officials, the South Carolina PTAC closed in 2019 because of poor performance that could not be remediated. DLA has been working to identify a replacement.

11The statewide count includes the PTACs that serve the District of Columbia and entire territories, such as Puerto Rico.


Figure 1: Number of Procurement Technical Assistance Centers by State, Fiscal Year 2020

Note: The map shows the 90 Procurement Technical Assistance Centers that operated for the full fiscal year.


PTAP’s general terms and conditions state that PTAC services include, but are not limited to, identifying marketing opportunities; assisting clients with preparation and submission of applications and other documents; and offering advice about federal, state, and local government contracting laws, policies, and procedures.12 PTACs must not provide general business training or financial assessments, assistance to new businesses, or employee development training. They also must not act as the client’s representative. PTACs primarily provide their services through one-on-one counseling or by holding events for clients, such as hosting seminars or conferences.

According to DLA, businesses learn about PTAC services through several means, including PTAC outreach, events, and referrals. PTAP’s general terms and conditions require PTACs to maintain websites and a social media account and perform outreach (through print, television, and internet media, including advertisements). In addition, DLA officials indicated that businesses may learn about PTACs from federal websites, such as SAM.gov, GSA.gov, and SBA.gov; classes, seminars, and events hosted by PTACs; or from referrals by economic development entities that offer similar services or by government employees.

DLA oversees PTAP (see fig. 2). DLA also has delegated responsibilities for post-award administration, including performance reviews and audit resolutions, to the Defense Contract Management Agency (DCMA) and the Office of Naval Research (ONR). As discussed later in this report, in October 2021 program responsibility will transition from DLA to another office within DOD’s Office of Acquisition and Sustainment or to another agency.

12The cooperative agreements that PTACs have with DLA refer to these general terms and conditions.

Services and Outreach

Oversight


Figure 2: Simplified Organizational Chart of Department of Defense Agencies with Responsibility for the Procurement Technical Assistance Program, as of January 2021

DOD oversight of PTACs includes pre-award activities, collection and review of quarterly performance data, and post-award activities.

• DLA’s pre-award oversight includes reviews of funding applications. After a PTAC applicant submits its funding application, a DLA grants officer reviews the submitted documentation for compliance with the program’s general terms and conditions and funding opportunity requirements.13

• DLA collects and reviews performance data from PTACs quarterly, including the number of new clients, counseling hours, and events.

• DLA delegated some post-award administration of PTACs, including performance review responsibilities, to DCMA and ONR. DCMA administers the cooperative agreements (funding awards) of PTACs run by tribal entities, nonprofits, and governments, and ONR

13The pre-award review process begins after DLA posts the PTAP funding opportunity for the upcoming year and PTACs submit their applications with the required documentation. As of fiscal year 2020, funding awards were for a 1-year base period with four 1-year option periods. Previously, awards were for a 1-year base period with two 1-year option periods.


administers those of PTACs run by colleges and universities. As discussed in more detail later in this report, beginning in fiscal year 2021, DLA plans to take on more of the post-award administrative duties historically performed by DCMA and ONR.

Small Business Development Centers provide counseling and training to current and prospective small businesses on topics such as access to capital, business planning and operations, financial management, marketing and sales, and government contracting. Through the SBDC Program, SBA generally provides funding to one recipient organization in each state, the District of Columbia, Puerto Rico, the U.S. Virgin Islands, Guam, and American Samoa. 14 PTACs and SBDCs can be co-located at the same address or sponsored by the same organization or university.

DLA’s process for reviewing funding applications includes compliance reviews, evaluations of past performance and proposed budgets, and negotiations with PTACs. More specifically, a DLA grants officer reviews the submitted documentation for compliance with relevant laws and regulations and the requirements of the funding announcement. The grants officer documents the review in a memorandum that includes a checklist; an evaluation that rates areas such as past performance, technical capability, and cost; and the results of the evaluation, including whether the grants officer will grant the award. According to representatives of DLA, if the initial review does not reveal any major problems, the grants officer and the PTAC negotiate the cooperative agreement for the year, which includes the budget and goals for new clients, counseling hours, and events. The negotiation includes discussions about whether a PTAC’s goals are reasonable, based on its 14There are two exceptions—Texas has four regional SBDC networks and California has five regional networks.

Small Business Development Centers

DLA Plans to Improve Oversight by Aggregating Performance Data and Performing More Frequent Performance Reviews

DLA Reviews of Funding Applications Include Assessing Proposed Costs against a Specific Metric


proposed budget. The negotiated goals are then documented in the PTAC’s cooperative agreement with DLA, which both parties sign.

A focus of DLA’s reviews of funding applications is assessing proposed PTAC budgets against a target for cost per counseling hour (the total program cost, which is the amount provided by the federal government and the PTAC, divided by a PTAC’s total counseling hours). Grants officers compare the target cost of $416.80 to the PTAC’s proposed cost per counseling hour. According to DLA, it derived the target figure from its data analysis. DLA described the target as an upper limit of reasonableness to use during negotiations with PTACs. DLA included the target cost per counseling hour in the program’s funding announcement for the first time in fiscal year 2021.

In fiscal year 2020, PTACs’ costs per counseling hour ranged from $162 to $665. DLA officials said costs per counseling hour that are too low or too high indicate a need for grants officers to look more closely at a PTAC’s proposal. For example, a rate that is too low may indicate a PTAC has inflated its counseling hours, while a rate that is too high may indicate excessive spending on personnel, facilities, or travel. However, grants officers may deem a higher cost per counseling hour to be reasonable—for instance, for newer PTACs with start-up expenses or for PTACs that plan to host many events. After considering all relevant factors, the grants officer may insert a special condition into an award for a PTAC with a high cost. For example, DLA officials said the PTAC with the highest cost per counseling hour in fiscal year 2019 received a fiscal year 2020 award that limited the government’s funding to a cost per counseling hour of about $430.

Representatives of the Association of PTACs told us the target cost per counseling hour has caused concern for many PTAC managers because it does not take into account differences between PTACs that can affect calculation of the rate. For example, PTACs in more expensive cities or states could have a higher cost per counseling hour because the rent for their facilities and overall cost of living increases the total cost of their award. APTAC also said the rate is just one of many factors that identify a successful PTAC and, because there is great variation among PTACs, excessive focus on a single metric would provide an inaccurate view of a PTAC’s success.

DLA said it considered whether to switch from a cost-per-counseling hour to a “cost of services provided” measure that would capture more of the services PTACs render to clients. However, DLA officials said only one or


two program managers voiced support for the change and most expressed concerns that the effort required to capture their time spent planning and working on events could outweigh the benefits of the measure. According to DLA officials, while adding events to the measure could help provide more data on PTACs, it would increase administrative burden and leave less time for counseling. The officials said they are satisfied that DLA’s current calculation is an incentive to PTACs to provide the most efficient service that is best for their clients.

DLA has been working to change its online reporting system for PTACs to allow it to accurately and readily aggregate data on PTAC performance. PTAP’s general terms and conditions contain reporting requirements for PTACs to regularly provide performance data to DLA. Quarterly, PTACs submit data that capture their progress on cooperative agreement goals (new clients, counseling hours, and events).15 Formerly, these data were submitted in a form emailed to DLA. Since 2019, DLA has required PTACs to enter the data into an online system. According to DLA officials, DLA created this online system to improve the integrity of the program’s data because the prior manual record keeping was not satisfactory. They also said that DLA uses these data to track the performance of individual PTACs and determine if any issues need to be addressed.

Although the online system allows grants officers to review the data of individual PTACs, as of early March 2021, DLA was unable to use the system to correctly aggregate the data to show the performance of all PTACs on a quarterly or annual basis.16 DLA officials said that while the online system primarily is used to provide quarterly reports to grants officers, it does not satisfactorily compile program-wide statistics. Officials also said the agency was continuing to work on correcting the aggregation function in its online system. As of March 11, 2021, the system developers were addressing issues that arose after an initial attempt to fix the system failed.

In the absence of a reliable aggregation function in its online system, DLA said it used Past Performance Data Sheets for reviewing and reporting on aggregate program data. The data sheets were created in 2018, and 15PTACs’ quarterly reports also include other data and information such as number of active clients; noteworthy client success stories; number and dollar value of federal, state, and local contract awards (including those received by small businesses); and number and dollar value of subcontract awards (including those received by small businesses).

16When data were aggregated, data from some PTACs were not included in totals.

DLA’s Planned Changes to Online System Should Allow It to Readily Assess Program-wide Performance and Goal Attainment


existing PTACs were required to submit them with their application each award year. The sheets captured PTAC performance on two of the three program goals.17 A DLA staff member then manually aggregated the information from each data sheet into one spreadsheet, which was used for reporting to the program’s stakeholders. See table 1 for a summary of PTAC performance during 2018–2020.18

Table 1: Procurement Technical Assistance Program Performance Statistics for Fiscal Years 2018–2020

Year 2018 2019 2020 Active clients 50,843 54,824 57,160 Events 5,316 5,307 4,780 Counseling hours 163,387 169,716 190,903

Source: GAO presentation of data from the Defense Logistics Agency. | GAO-21-287

Because the data sheets did not include information on PTACs’ goals and the online system did not produce a report comparing PTAC performance to goals, there was no one place to review aggregate goal attainment, as of March 2021. For individual PTACs, grants officers had to review the PTAC’s award to find the goals and then compare them with the statistics in the PTAC’s quarterly performance reporting forms.

We compared each PTAC’s fiscal year 2020 performance statistics to the goals listed in each PTAC’s award for that year. We found that the majority of PTACs (63 percent) met or exceeded their goals for number of events and just under half (48 percent) met or exceeded their goals for hours of counseling time. We could not review PTAC performance on one of the 2020 goals—number of new clients—because DLA did not collect this information in its performance data sheets.

As mentioned previously, the update to the online system was still under development as of early March 2021. Based on our review of

17The Past Performance Data Sheets capture the number of active clients rather than the number of new clients. According to DLA, it collects active client data so that it can compare active clients to counseling hours to assess the effect of PTACs on their localities and identify reporting irregularities.

18According to DLA officials, the number of events decreased in 2020 because of concerns about Coronavirus Disease 2019 (COVID-19). However, the number of counseling hours increased in 2020 as businesses sought help in bidding on government contracts related to COVID-19.


documentation and interviews with officials, this system fix should allow DLA to assess program-wide performance and goal attainment.

The Defense Contract Management Agency and the Office of Naval Research infrequently conducted performance reviews, which also varied in scope. This contributed to DLA’s decisions to institute site visits and assume many of the administrative responsibilities DCMA and ONR had over PTAC awards.

DLA said it asked DCMA and ONR to conduct performance reviews each year, but based on a review of DLA documentation, most PTACs did not receive a performance review annually for their 2017–2019 cooperative agreements. At least 95 PTACs operated each year over this 3-year period, but DCMA and ONR completed a total of 48 reviews.19 Only six of the 93 PTACs that operated all 3 years had more than one review, and 40 PTACs that operated all 3 years did not receive a review during this time. DCMA conducted more performance reviews than ONR (38 of the 48 reviews).

We also found differences in the scope of the reviews conducted in this period. DLA created a template for DCMA and ONR to use for performance reviews, which included a checklist of areas to review.20 DCMA generally used the template while ONR usually provided a one or two-page memorandum, including four memorandums that reviewed only the PTAC’s finances or goals. According to ONR, it was up to its individual reviewers whether to use DLA’s template, and the San Diego office was usually the only ONR office that chose to complete the full review template. Although most performance reviews (41 reviews) recommended that the PTAC should continue with the cooperative agreement or receive a follow-on award, six reviews did not include any

19The number of PTACs operating each year varied, ranging from 95 to 98.

20Before 2019, the template included an executive summary with a results section (that had ratings in four categories and an overall rating) and a recommendation on continuing the cooperative agreement or follow-on award. The template also included a required checklist of areas to review: performance reporting, client services, program implementation, administrative requirements, program income, facilities, personnel, and financial management. In 2019, DLA made the checklist optional and removed the rating requirement for the executive summary.

DLA Plans to Assume Many DCMA and ONR Oversight Responsibilities

Performance Reviews


recommendation, and one review included a recommendation that did not state whether the PTAC should continue.21

Differences in how frequently reviews were performed and their scope may be the result of varying interpretations of guidance on review responsibilities.

• DOD regulations (Grant and Agreement Regulations) have no explicit requirement for DCMA or ONR to conduct programmatic reviews such as the performance reviews DLA requested, which included looking at PTACs’ goals, client services, and program implementation.22 According to DLA, it requested that DCMA and ONR conduct performance reviews of PTACs annually based on these regulations. But DLA also told us that the regulations do not explicitly require either agency to perform such reviews annually or to the level of detail DLA requested.

• ONR did not believe it was required to conduct programmatic reviews of PTAP. According to ONR, under the DOD regulations, it was only responsible for reviewing PTACs’ financial management.

• A performance-based agreement between DLA and DCMA that expired in 2018 stipulated that DCMA would conduct annual performance reviews of PTACs.23 ONR never had such an agreement with DLA for performance reviews.

• DCMA’s internal policies state that it is to conduct annual performance reviews no later than 60 days before the end of a PTAC’s performance period.

21Although most of the performance reviews recommended that PTAC awards be renewed, they also identified some shortcomings. According to DLA officials, DLA can require a PTAC to take corrective action to address a finding from a performance review or it can terminate the PTAC’s award. Officials also said it is DLA’s goal to remediate and retain a PTAC rather than to close it and find another one, as long as the PTAC had successfully provided services in the past. Remediation allows DLA to mitigate the risk to the government, while giving PTACs a reasonable period of time to take corrective actions.

2232 C.F.R. § 22.715. The DOD Grant and Agreement Regulations provide uniform policies and procedures for the award and administration of awards by DOD components.

23DLA and DCMA have a current performance-based agreement that no longer includes the PTAP program. According to DLA, the agencies’ leaders streamlined the new agreement to include only the most critical topics, but DCMA continued to perform PTAC performance reviews although no longer required to do so by the agreement.


DLA said that while DCMA agreed to perform in-depth reviews of PTACs, it understands that ONR does not have the workforce to focus on low-risk (small) awards such as those provided by PTAP.

According to DLA officials, DLA grants officers conduct site visits to supplement performance reviews, focusing on PTACs that are administered by ONR because many of those PTACs have never hosted a DOD representative onsite. Based on our review of DLA’s site visit reports, the scope of the reviews are similar to performance reviews and cover the same topics. According to DLA, the visits allow grants officers to assess compliance, offer strategies to correct or improve PTAC performance, and gather best practices to share with other PTACs. They are also intended to build relationships between grants officers and the PTACs they oversee, and to educate grants officers on PTACs’ clients, employees, environments, and challenges.

Site visits by DLA grants officers began in 2015, and DLA intended to complete 12 site visits a year (three visits by each of the four grants officers who oversee PTACs). However, DLA completed a total of 47 site visits from 2015 through 2019 rather than 60 site visits. According to DLA officials, it did not achieve its goal for reasons including availability of staff and competing priorities, such as developmental initiatives that reduce the time staff can take to conduct site visits.

A September 2020 review by an independent audit firm found that DLA did not provide sufficient financial oversight over the awards delegated to DCMA and ONR. Specifically, DLA did not have sufficient policies and procedures for the cooperative agreement review process and could not demonstrate that expenses recorded for PTACs were complete and accurate.

In response to these audit findings, DLA officials said DLA would assume post-award administration of all awards in fiscal year 2021. Specifically, they said that DLA would perform quarterly performance monitoring on all PTACs, in lieu of the post-award performance reviews previously done by DCMA and ONR. They noted that this would help them identify corrective actions earlier in the performance period. According to the officials,

• DLA would create a new post-award process and update its standard operating procedures to reflect the new process. As of March 2021, DLA had updated its standard operating procedures and drafted an updated quarterly reporting template. DLA also said it had created

Site Visits

Planned Changes to Performance Review Processes


new documentation for reviewing PTAC budgets and tracking grant expenditures.

• As part of these changes, DLA planned to revise its quarterly reporting form to capture data desired by PTAP’s external government stakeholders (such as DOD’s OSBP and Congress). However, DLA officials told us that DLA subsequently received feedback that the data that external stakeholders desired were more extensive than could be documented in the quarterly reporting form. As of March 2021, the effort to update the form was on hold as DLA reassessed its metrics.

• DLA would hire two additional grants officers. As of March 2021, both grants officers had been hired, according to a DLA official.

According to DLA, these changes were expected to decrease DOD costs by making the oversight process more efficient, particularly for performance monitoring, because grants officers would no longer have to coordinate program oversight with administrative grants officers at DCMA and ONR. The changes were also expected to reduce the burden on grant recipients because they would need to respond to only one oversight entity, DLA, rather than two, which would decrease the amount of data they needed to gather and submit. Ultimately, DCMA and ONR only would retain their responsibilities as the cognizant agencies for issues such as audit resolutions and some cost determinations. DLA’s plans to monitor PTACs’ performance more frequently should allow grants officers to more consistently identify and address performance issues.

In addition, DLA said if it continues to conduct site visits, the two additional grants officers would help the agency achieve its goal of 12 visits per year. However, DLA said the objectives it desired through in-person visits, such as problem solving and idea sharing across PTACs, might be achieved virtually in the future. DLA officials noted that virtual communications combined with DLA’s increased oversight, including reviews of additional records and documentation, might make in-person visits unnecessary.


DLA recently implemented new training standards for PTAC staff. According to DLA officials, the House Committee on Armed Services expressed concern about the quality of counseling provided by PTAC counselors (specifically, how consistent it was) during a February 2019 briefing and discussed ways in which DLA could be more involved in training for PTACs and working with the Association of PTACs on training. In response to the briefing, DLA incorporated new training standards for PTAC staff in the program’s general terms and conditions. The House Report accompanying the National Defense Authorization Act for Fiscal Year 2020 required DLA to brief the same committee on the new training standards for PTACs by December 1, 2019.24 DLA provided briefing slides to the committee in January 2020.

The following requirements went into effect in fiscal year 2020: program managers must receive a minimum of 40 hours of management, leadership, or contracting training each year. Other full-time personnel must obtain 40 hours of training per year on federal contracting laws, regulations, systems, and procedures, of which a minimum of 20 hours are to be provided by the federal government.25 Separate from the 40-hour requirements, staff are also encouraged to obtain at least 20 hours of training in interpersonal skills or counseling. According to DLA officials, these hourly requirements were modeled after requirements for DOD contracting professionals. The officials also stated that the training subjects were based on the scope of PTACs’ work and the desire to

24H.R. Rep. No. 116-120, pt. 1, at 181-182 (2019).

25Part-time staff who work 20 hours or less must receive 10 hours a year of federal contracting-related training.

DLA Has New PTAC Training Requirements but Does Not Require Uniform Tracking of Training or Obtain Trainee Test Results

DLA Set Training Requirements and Has Been Working with APTAC on Curriculums


provide program managers with maximum flexibility to determine specific training topics based on employee needs.

To help PTACs get the type of training DLA expects, DLA officials said the agency implemented a program that provides at least one 2-hour training session per month for program managers. Topics include the award application process, collaborating with other PTACs, and hiring counselors. Officials said the training program would be expanded starting in February 2021 to include monthly training for PTAC counselors, which would focus on the contracting-related information counselors need to share with businesses.

In addition, DLA required APTAC to make available two optional curriculums as a resource to help PTACs meet the requirements—one for counselors and the other focused on program managers.

• Counselor curriculum. PTAP’s general terms and conditions state that the counselor training curriculum must include a certification program, a proficiency test, and online training offered by the Defense Acquisition University. The certification requirements must include or allow some non-APTAC training to count toward qualification. DLA told us it created these requirements because it had concerns about the training provided at APTAC conferences and sought to diversify the training counselors received.

APTAC officials said APTAC has had a counselor curriculum and related certifications in place for many years and the training that supports the curriculum includes Defense Acquisition University resources.26 According to these officials, the general emphasis of the curriculum is on client intake, federal outreach, statutes and regulations on government contracting, and marketing within the confines of government contracting rules. They also said that because DLA has not provided APTAC with any indication that the curriculum’s content should change, they have no plans to redesign the curriculum.

According to DLA, the proficiency test to accompany the curriculum is still under development. The results of the tests will be available only to the tested employees and their program managers. DLA said the test will allow program managers to determine whether their

26APTAC has two certification programs for counselors. One of the programs requires 1 year of experience as a counselor and the other requires 2 years. Certification under both programs requires counselors to take a test that demonstrates their understanding of PTAP and government contracting.


counselors meet minimum standards for counseling clients. According to APTAC officials, DLA required APTAC to create a proficiency test that must be online and provide instantaneous scoring, but did not provide further guidance on the purpose or content of the test. APTAC drew from the test it formerly used for its certification programs for the new test, which was created by the September 2020 deadline in PTAP’s general terms and conditions and was to have been made available to PTACs in January 2021.

• Program manager curriculum. APTAC drafted a curriculum for program manager training and planned for it to cover topics such as understanding reporting requirements, cost principles, and staff recruitment. APTAC officials said that in the process of creating the curriculum, they determined there were no available training resources for many of the topic areas. They also said that creating training for program managers was not as simple as creating training for counselors and would require more work. Because DLA did not provide guidance to APTAC about the content of the program manager curriculum, other than mandating that it must include training by the Defense Acquisition University, APTAC said the creation of this curriculum likely would be an iterative process. In addition, APTAC said that because virtually no training by the Defense Acquisition University is applicable to program managers, training content for program managers would have to be developed by APTAC and DLA.

DLA and PTAC representatives told us the new requirements are not expected to create major challenges for PTACs. DLA said the cost of the training should not be a burden because most PTACs already spend more on training than the new requirements will cost. Representatives of three PTACs said they did not need to make changes to their training or did not need to seek out new sources of training to meet the requirements.27 Two PTACs questioned whether the requirements needed to apply to more experienced staff. One PTAC said that while DLA limiting staff attendance to one of the two APTAC conferences held annually was a challenge, there were enough other training options for staff that it would not have to find additional training sources for staff for another 3 years.

27The PTACs we interviewed said sources they would use to meet the new requirements included internal training, APTAC, SBA, and the Defense Acquisition University.


According to DLA officials, in 2021 DLA grants officers will start evaluating PTAC compliance with the new training requirements during application reviews. Grants officers will initially spot-check the training records of one or two counselors. If that check raises questions, the grants officer will request the full training log for all employees. According to the officials, DLA will take this approach to mitigate the burden on grants officers performing pre-award reviews. They noted that a grants officer can request a PTAC’s training record at any time during the award period if the officer has reason to believe employees are not being properly trained.

When DLA first instituted the new training requirements in 2019, it did not provide guidance to PTACs on how to track their compliance with the requirements. In the absence of such guidance from DLA, representatives from APTAC and individual PTACs told us that PTACs have been tracking their training in a variety of ways, such as on a spreadsheet, in a database used for tracking client information, or by keeping staff’s training certifications. Some PTACs noted they were tracking the name of the training and hours spent, while others said they were tracking additional information such as the training source (provider) and the type of training. Two PTACs mentioned that keeping track of their training would be a challenge, although one PTAC noted this as a minor challenge.

According to federal internal control standards, management should externally communicate the necessary quality information to achieve the entity’s objectives. 28 This includes communicating quality information externally through reporting lines so that external parties can help the entity achieve its objectives and address related risks.

According to DLA, in May 2020 it created a template for PTACs to use in tracking their training that is available on DLA’s website. The template includes columns for information such as course name, topic, hours, date completed, and source (federal versus nonfederal). However, PTACs were not aware of this template when we interviewed APTAC and PTACs in July and August 2020. In those interviews, APTAC and PTACs said DLA had not told them what information PTACs had to record when documenting training. Revisions to PTAP’s general terms and conditions (issued in September 2020) reference the template and state that it is optional.

28GAO-14-704G.

DLA’s Only Guidance on Tracking Training Is Optional, and DLA Lacks Access to Test Results to Evaluate Training Effectiveness

Tracking Training



DLA officials noted three reasons they made the template optional. First, the entities that run PTACs vary in the way they track training. Second, PTACs may choose to rely on APTAC tracking of all members’ training. Third, DLA does not want to create extra work for recipients as long as they are able to show how they are complying with the general terms and conditions. However, DLA’s template does not appear to be burdensome to fill out and lays out in one place everything that DLA expects PTACs to track.

Allowing PTACs to use multiple processes to track training could lead to inconsistencies in the training information that is collected. Establishing a requirement for PTACs to use a standardized template would help ensure that PTACs consistently record all the information DLA needs to assess PTACs’ compliance with the new training requirements.

DLA does not have access to the results of proficiency tests that participants take after APTAC training that would help it measure the effectiveness of the training. GAO guidance for assessing strategic training and development efforts states that agencies should establish ways to measure the contributions that employees’ training and development make to achieving results. 29 This process should involve obtaining up-front agreement with key stakeholders on what success is and how it will be measured.

According to DLA officials, to evaluate the effectiveness of the training sessions DLA will be providing, it will request feedback from program managers on the utility and benefit of the sessions. In addition, according to DLA officials, the effectiveness of PTAC counselor training can be assessed indirectly by measuring PTACs’ effectiveness in assisting businesses. DLA plans to use a feedback form on its website where businesses will provide feedback on the assistance they received from their PTAC on contracting-related topics. Although PTACs will be required to post the URL for the feedback form in their communications with clients, the form will be voluntary.

However, DLA could more directly measure the effectiveness of PTAC training by reviewing the results of the proficiency tests that counselors take at the completion of the training. DLA required APTAC to develop a proficiency test for its counselor curriculum, but DLA does not have access to the results of these tests, which are available only to tested

29GAO-04-546G.

Evaluating Effectiveness of Training



staff and their program managers. According to DLA officials, they have not asked for test results because the test is intended to help new or inexperienced program managers understand whether their counselors have the federal contracting knowledge necessary to pass the test. In addition, DLA officials said that because the test does not cover the state and local contracting knowledge counselors need, it would not be an appropriate measure of the minimum qualifications of a PTAC employee. However, DLA could benefit from receiving the test results even if the test does not cover all of the knowledge PTAC employees must have. Obtaining and reviewing these test results in the aggregate would help DLA better measure the effectiveness of the training curriculum in improving PTAC counseling on federal contracting.

Under the National Defense Authorization Act for Fiscal Year 2020, DOD must transition oversight of PTAP from the Director of DLA to the Under Secretary of Defense for Acquisition and Sustainment by October 2021. DOD was also required to write a report on and brief Congress by November 2020 on plans to implement the transition. In December 2020, DOD sent a letter to Congress stating that it needed additional time to respond to the requirement and would provide the required report and briefing by the end of March 2021.

As of December 2020, DOD was considering the following three options for the transition:

• Moving PTAP to the Office of Small Business Programs in the Office of Industrial Policy (within the Office of Acquisition and Sustainment). DLA staff who currently oversee PTACs would move to OSBP. No additional employees would be assigned to the program.

• Keeping PTAP in DLA, with OSBP providing additional oversight over PTAP’s general terms and conditions and the data collected on PTAC performance. Changes would focus on clarifying oversight roles and responsibilities and one employee within OSBP would be responsible for overseeing DLA’s administration of PTAP.

• Working with Congress to transition the program to SBA. DOD officials said this option would require legislative changes. According to a DLA official, the Office of Management and Budget has said that DOD should work with SBA to create a legislative change proposal and until that time, should increase collaboration with SBDCs. However, SBA officials told us that while the agency sees value in having greater collaboration between PTAP and the SBDC program, it is not likely to partner with DOD on a legislative proposal to transfer

DOD Has Been Considering Three Options for Transitioning PTAP


PTAP to SBA. The SBA officials also said that SBA does not currently have the capacity to take over PTACs and making such a change would require significant resources (funding) and coordination between SBA, DOD, and Congress. In lieu of such a proposal, DLA and SBA formed a working group in December 2020 with members such as SBDCs, the association representing SBDCs, and APTAC to increase collaboration.

Because of initial plans for OSBP to assume PTAP oversight, OSBP, DLA, and APTAC set up a working group that met periodically through Spring 2020 to plan for the transition, according to DLA and APTAC officials. OSBP officials said the group’s meetings addressed topics such as the program’s general terms and conditions, measuring PTAC goals, and whether the program would move to OSBP or stay in DLA.

APTAC officials expressed concern about the option of keeping PTAP in DLA with additional oversight from OSBP. APTAC officials said they support placing PTAP with the Under Secretary for Acquisition and Sustainment and the Deputy Assistant Secretary of Defense for Industrial Policy.30 They also noted that although PTACs work extensively with small businesses, they also serve large businesses and exclusive association with the small business community could limit PTAP’s acceptance and utility within DOD.

DOD officials said that OSBP and DLA have been meeting periodically to continue to discuss the transition options and funding for the administration and oversight of the program. According to DLA, it regularly updates PTACs on the status of the transition to ensure transparency, even if DLA does not have new information to provide.

30The National Defense Authorization Act for Fiscal Year 2020 changed the official responsible for PTAP from the DLA Director to the Under Secretary of Defense for Acquisition and Sustainment. The House report accompanying the Act stated that the PTAC program should be placed in the Department’s Office of Industrial Policy, which is part of the Office of the Under Secretary of Defense for Acquisition and Sustainment, where it would align with their activities.


We previously reported that overlap occurs when multiple agencies or programs have similar goals, engage in similar activities or strategies to achieve them, or target similar beneficiaries. 31 PTACs and SBDCs overlap to some extent in that they both can cover procurement-related topics. However, they tend to serve different clients; PTACs narrowly focus on businesses pursuing government contracting and SBDCs focus on small businesses.

PTACs and SBDCs occasionally offer similar services to their clients (see fig. 3). PTACs provide mostly free assistance to businesses seeking to participate in government contracts. They offer one-on-one counseling; identify marketing opportunities for clients; and advise clients in preparing and submitting applications, certifications, and other documentation. SBDCs provide individualized business advice and technical assistance and sometimes host workshops and training. They counsel clients and assist them in preparing financial statements and other documentation.

31GAO, Fragmentation, Overlap, and Duplication: An Evaluation and Management Guide, GAO-15-49SP (Washington, D.C.: Apr. 14, 2015).

Some PTAC and SBDC Services Overlap, but DLA and SBA Do Not Have a Collaboration Agreement PTACs and SBDCs Both Provide Training and Counseling, but the Businesses They Assist and Topics They Cover Can Differ

Types of Services Provided

https://www.gao.gov/products/GAO-15-49SP


Figure 3: Selected Services Provided by Procurement Technical Assistance Centers (PTAC) and Small Business Development Centers (SBDC) Based on Fiscal Year 2020 Requirements

Although PTACs and SBDCs both assist small businesses, the programs focus on different clients who usually do not overlap, according to PTACs and SBDCs with which we spoke. PTACs serve as a resource for large and small businesses pursuing or performing contracts for federal, state, and local governments. Although PTACs are required to make a concerted effort to seek out small businesses (including small disadvantaged businesses, women-owned small businesses, Historically Underutilized Business Zone small businesses, and service-disabled veteran-owned small businesses), they also assist clients other than

Businesses Served


small businesses.32 But PTACs are not allowed to assist pre-startup or new businesses unless their potential for success in government contracting in the near term is limited only by a lack of understanding of government contracting processes. In contrast, SBDCs provide assistance to prospective and existing small businesses regardless of whether they are interested in pursuing government contracts.

Representatives of three of the six PTACs told us they generally work with more mature businesses than SBDCs do because PTAC clients must be in a position to pursue government contracts. In contrast, these PTAC representatives told us that SBDC clients have been in business for less than 2 years.

Furthermore, according to the three PTACs co-located with SBDCs we interviewed, most clients of a PTAC or SBDC do not use the services of the other, or might do so only in limited circumstances. For example, an SBDC client could mention an interest in government procurement, which could lead the SBDC counselor to refer the client to the local PTAC. However, PTAC and SBDC representatives told us this does not happen regularly. For example, representatives from one co-located PTAC and SBDC said this happens about three or four times a year, while a representative from one SBDC co-located with a PTAC said about 5 percent of clients use both PTAC and SBDC services. A representative from the third co-located PTAC and SBDC estimated that 30–40 percent of PTAC clients also sought SBDC assistance.

PTACs provide counseling and training on a narrower set of issue areas than SBDCs, but both offer services related to government contracting. PTAP’s general terms and conditions require PTACs to limit counseling and training to procurement-related issues and activities. PTACs are not permitted to offer clients training, advice, or assistance on topics such as access to loans, human resources, hiring, retention, health care, and quality certifications, except where the assistance specifically relates to compliance with government-specific contract requirements.

32Federal agencies conduct a variety of procurements that are reserved (set aside) for small business participation. The set-asides can be for small businesses in general or be specific to small businesses meeting additional eligibility requirements in the Service-Disabled Veteran-Owned Small Business Concern, Historically Underutilized Business Zone, 8(a) Business Development, and Women-Owned Small Business programs.

Issue Areas and Focus


SBDCs provide assistance to small businesses on a range of broader topics, including accessing capital; improving business planning, administration, operations, sales, and marketing; and government procurement. In addition, the annual funding opportunity for the SBDC program identifies providing government contracting assistance to small businesses as a “required service” SBDCs are expected to provide.33 As part of this service, SBDCs are to work cooperatively with PTACs.

As we previously reported, program overlap can have positive and negative effects.34 Understanding such effects can help determine whether actions to improve efficiency or reduce overlap are warranted.

Potential positive effects of overlap. The overlap we identified in PTAC and SBDC services could lead to positive effects for businesses, primarily by creating more ways for them to access advice on government procurement. For example, the programs refer businesses to each other, as appropriate, which helps each program focus on its areas of expertise. In particular, one PTAC co-located with an SBDC told us that being co-located makes it easy for both programs to introduce clients to the most appropriate counselor for the client’s needs, while another said that being co-located provides both programs the opportunity to participate in each other’s events. Another potential positive effect of the overlap is that SBDCs may be in a position to provide government contracting advice to small businesses in areas without a PTAC. As previously mentioned, one state did not have a PTAC in fiscal year 2020 and a second state’s PTAC was only funded in mid-2020, meaning it did not have a PTAC for part of the year. Finally, SBDCs are limited to providing assistance to small businesses. PTACs, which can assist businesses of any size, help ensure that all businesses interested in government procurement have training and resources to assist them.

Potential negative effects of overlap. The overlap we identified in PTAC and SBDC services also could have negative effects. For example, SBA does not require SBDC counselors to take training related to procurement, which could lead to clients getting advice that is not as 33According to the SBA, SBDCs operate under the provisions of the Uniform Administrative Requirements, Cost Principles and Audit Requirements for Federal Awards, 2 C.F.R. Part 200, a Notice of Award (the Cooperative Agreement) issued by SBA, and the provisions of the annual Funding Opportunity. See, e.g., SBA, Office of Small Business Development Centers, OSDBC Funding Opportunity No. OSBDC-2021-01 & OSBDC-2021-02, sec. 5.1.5.

34GAO-15-49SP.

PTAC and SBDC Overlap Has Positive and Negative Effects



reliable as advice they would receive from a PTAC counselor. As previously mentioned, PTAC counselors currently must take at least 40 hours a year of training related to federal contracting laws, regulations, systems, and procedures. According to DLA officials, PTAC counselors often received this much training annually even before the current 40-hour requirement was implemented. While SBA officials said SBA encourages SBDC counselors to provide technical assistance related to government contracting, SBA does not require the counselors to take training related to government contracting.

Another potential negative effect of overlap is double counting of services, an issue specific to PTACs and SBDCs that are co-located. Although DLA does not track it, we found that at least 27 of 90 PTACs were co-located with an SBDC in fiscal year 2020.35 Stakeholders generally spoke favorably of the benefits of co-location, but some negative effects have been reported. For example, based on its oversight and complaints received from program managers, DLA believes some SBDCs claim PTAC accomplishments as accomplishments for their co-located SBDC. APTAC expressed similar concerns to DLA in 2017, including reports from some PTACs that they have been told to enter data into their co-located SBDC’s database for activities the PTAC conducts that also support SBDC goals, including counseling hours.

In addition, DLA officials told us that PTAC staff sometimes have been asked to do SBDC work. For example, they said they have found instances from at least three co-located PTACs and SBDCs of PTAC counselors providing counseling on SBDC-related topics and have received complaints about PTAC staff being required to take training more appropriate for SBDC counselors. According to APTAC, there should be well-defined distinctions between the services that PTACs and SBDCs provide so that both focus on their respective areas of expertise while fostering a collaborative environment between PTACs and SBDCs and other business resources.

The differing missions and requirements help limit PTAC and SBDC overlap and the negative effects of that overlap, but SBDCs’ broader

35We defined PTACs and SBDCs as co-located if the lead PTAC and lead SBDC were located at the same address or were sponsored by the same organization or university. In addition to the 27 co-located lead PTACs and SBDCs, we identified two PTACs that had satellite offices that were co-located with an SBDC. According to SBA, nearly 20 percent of SBDC networks (12 networks) managed a PTAC and one-third of SBDC networks (21 networks) had a service center co-located with a PTAC in fiscal year 2020.


areas of focus mean some overlap will continue. According to DLA, it updated PTAP’s general terms and conditions to address concerns about PTAC resources being used for SBDC activities at some co-located centers. In addition, DLA officials noted that the agency’s normal oversight processes for PTACs include grants officers reviewing PTAC counselors’ notes and other documentation on working with clients to ensure that PTACs only provide assistance on approved topic areas. According to SBA officials, the amount of government contracting assistance SBDCs provide depends on local needs. However, SBDCs are expected to work cooperatively with PTACs as they provide these services.

As previously stated, PTACs and SBDCs often work out of the same facilities (about 30 percent of PTACs are co-located with an SBDC), provide government procurement assistance, and may share some clients. In addition, SBDCs are expected to work with PTACs when providing government contracting assistance to small businesses. However, DLA and SBA have not documented how SBDCs and PTACs should collaborate.

Collaboration is one way to reduce or better manage overlap.36 Collaborative efforts can include agreeing on roles and responsibilities and establishing compatible policies, procedures, and other means to operate across agency boundaries. When appropriate, agencies can document their agreement regarding collaboration.

According to DLA in October 2020, collaboration with SBA had been minimal and limited to issues related to potential comingling of funds because SBA did not perceive an issue with overlap beyond the potential for comingled funding. According to SBA, it does not have a formal agreement, such as a memorandum of understanding, on the relationships between SBDCs and PTACs because SBDCs establish relationships at the local level with PTACs that reflect the requirements and needs of their service areas. However, as discussed above, there are potential negative effects of the overlap between PTAC and SBDC services, and DLA has identified problems, such as comingling of funds, at PTACs co-located with SBDCs.

36GAO-15-49SP.

DLA Does Not Have a Collaborative Agreement with SBA on Overlap between PTACs and SBDCs



The DLA-SBA working group offers an opportunity for the two agencies to collaborate more on providing contracting assistance.37 According to DLA officials, the first meeting of the working group was held in January 2021. According to SBA officials, because the working group was just getting underway, it was not clear what the group’s focus would be. SBA officials also told us that DLA and SBA had not yet discussed plans for a memorandum of understanding on providing contracting assistance but that they were open to the idea.

A formal collaborative agreement between DOD and SBA could have several benefits. First, it could help ensure that clients of PTACs and SBDCs receive advice from counselors with more government contracting training when possible. Second, a collaborative agreement could limit the potential double counting of services at co-located PTACs and SBDCs. Finally, such an agreement could help clarify the responsibilities of employees that work at co-located centers.

PTAP is undergoing many changes, which offer opportunities to enhance oversight and training and leverage resources for PTACs. DLA’s changes to its online system, allowing for aggregation of performance data and goal information, should serve to help the agency assess whether PTACs meet their goals. DLA’s new performance review and site visit policies should further enhance its oversight of PTACs. As the PTAC program transitions to a new office, such improvements can help DOD better assess the benefits PTACs provide to businesses interested in government contracting and the expertise of PTAC staff.

However, certain opportunities for improvement remain:

• Use of DLA’s template for reporting training information is optional. Requiring its use—thus standardizing the format of training information collected from PTACs—would help ensure that PTACs maintain the information DLA needs to track fulfillment of training requirements.

• Concerns remain about the inconsistent quality of training. DLA has required APTAC to develop a training curriculum for PTAC staff and an associated proficiency test, but DLA does not receive aggregate

37As noted previously, the DLA-SBA working group was formed in December 2020 to increase collaboration between the two agencies after DOD suggested transferring PTAP to SBA.

Conclusions


test results. Working with APTAC to obtain aggregate test results would help DLA assess the effectiveness of training.

• PTAC and SBDC services overlap in that both assist small businesses with government procurement, and this overlap has both positive and negative effects for businesses. By formalizing programmatic collaboration, DLA and SBA could help ensure clients receive advice from counselors with training on government contracting, limit potential double counting of services at co-located PTACs and SBDCs, and clarify the responsibilities of employees that work at co-located centers.

We are making the following three recommendations to DOD:

The Under Secretary of Defense for Acquisition and Sustainment should require PTACs to use the template created to help track fulfillment of training requirements. (Recommendation 1)

The Under Secretary of Defense for Acquisition and Sustainment should reach an agreement with the Association of PTACs to provide DLA with the aggregate results of proficiency tests administered to measure the effectiveness of PTAC counselor training. (Recommendation 2)

The Under Secretary of Defense for Acquisition and Sustainment should work with SBA to formalize a collaborative agreement for PTACs and SBDCs in relation to providing client services on government contracting. (Recommendation 3)

We provided a draft of this report to DOD, SBA, and APTAC for their review and comment. We received comments from DOD and APTAC, which we have reproduced in appendixes II and III, respectively. DOD concurred with our recommendations, but did not provide any details on plans to address them. SBA did not have any comments on the report.

APTAC commented on all three of our recommendations. Regarding DOD requiring a standardized template for tracking training, APTAC said it supports a standardized training reporting template and looks forward to working with DLA to further develop and refine such a template. Regarding DOD working with SBA to formalize collaboration between PTACs and SBDCs, APTAC stated that it supports formalizing such collaboration, noting that SBDCs and PTACs should collaborate with one another while adhering to their respective statutory mandates.

Recommendations for Executive Action

Agency and Third Party Comments and Our Evaluation


Regarding our recommendation that DOD reach an agreement with APTAC to provide DLA with the aggregate results of proficiency tests, APTAC said that with further development, its new test could be a helpful metric in assessing the overall effectiveness of training, but the best use of the test is as a diagnostic tool for program managers to assess their counselors. It noted that because the test focuses on federal procurement and does not account for the full range of state and local procurement rules needed to counsel businesses, the test should not be the only metric used to assess training effectiveness. In the report, we note that DLA could benefit from receiving the test results even if the test does not cover all of the knowledge PTAC employees must have. We continue to believe that obtaining and reviewing these test results in the aggregate would help DLA assess the effectiveness of its training curriculum.

APTAC also provided the following comments on findings in the report:

• Regarding the statement in the report that DLA had concerns about training provided at APTAC conferences, APTAC said it believes it has addressed DLA’s concerns by collaborating extensively with DLA on training conference approaches, agendas, and core topics.

• APTAC stated that it supports removing PTAP from DLA and placing it with the Under Secretary for Acquisition and Sustainment and (Deputy) Assistant Secretary of Defense for Industrial Policy. Rather than placing the program in DOD’s OSBP, it said that the most effective transition for PTAP would be for it to remain a separate program managed by the Assistant Secretary for Industrial Policy.

We are sending copies of this report to the appropriate congressional committees, the Secretary of Defense, the Administrator of SBA, and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

http://www.gao.gov/


If you or your staff have any questions about this report, please contact me at (202) 512-8678 or [email protected]. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix IV.

William B. Shear Director, Financial Markets and Community Investment



List of Committees

The Honorable Jack Reed Chairman The Honorable James M. Inhofe Ranking Member Committee on Armed Services United States Senate

The Honorable Jon Tester Chairman The Honorable Richard Shelby Ranking Member Subcommittee on Defense Committee on Appropriations United States Senate

The Honorable Adam Smith Chairman The Honorable Mike Rogers Ranking Member Committee on Armed Services House of Representatives

The Honorable Betty McCollum Chair The Honorable Ken Calvert Ranking Member Subcommittee on Defense Committee on Appropriations House of Representatives

Appendix I: Objectives, Scope, and Methodology


Our objectives were to examine (1) how the Department of Defense (DOD) oversees the Procurement Technical Assistance Program (PTAP); (2) how the Defense Logistics Agency (DLA) plans to track and evaluate its new training requirements; (3) what steps DOD has taken to transition PTAP to the Under Secretary of Defense for Acquisition and Sustainment; and (4) whether Procurement Technical Assistance Center (PTAC) and Small Business Development Center (SBDC) services overlap.

For all of the objectives, we interviewed officials from the Association of PTACs (APTAC)—which serves as the professional organization representing PTACs—and a nongeneralizable sample of six PTACs. We chose the PTACs to represent different U.S. regions, service areas (statewide or regional), size of award in fiscal year 2019, and co-location status (three were co-located with an SBDC and three were not). 1 The views of the PTACs we interviewed are not generalizable to other PTACs but offered important perspectives.

To examine how DOD oversees PTAP, we reviewed DOD’s Grant and Agreement Regulations and DLA’s policies and procedures for overseeing PTAP, including the program’s general terms and conditions and annual funding announcements. We focused on the roles that DLA, the Defense Contract Management Agency (DCMA), and the Office of Naval Research (ONR) play in PTAP oversight, metrics used to evaluate PTAC performance, and DLA’s plans to assume many of the oversight responsibilities DCMA and ONR previously had over PTAC awards. We interviewed officials from DLA, DCMA, and ONR to understand their respective views of their roles in overseeing PTACs.

To understand the metrics used to assess PTAC performance, we reviewed PTAP’s general terms and conditions to determine the goals set for the program. We also reviewed performance data DLA collected from PTACs for fiscal years 2018–2020. Based on our review of program documentation and discussions with DLA officials, we determined that the performance data captured in DLA’s online system were not sufficiently reliable for the purpose of aggregating data for all PTACs, because only data from a subset of PTACs were included in the totals.

In lieu of the data from the online system, we reviewed data that DLA manually compiled on PTACs’ counseling hours, events, and number of

1We selected PTACs based on fiscal year 2019 funding because not all fiscal year 2020 awards had been issued when we began our review.


Analysis of PTAP Oversight



active clients in Past Performance Data Sheets. To assess the reliability of these data, we interviewed DLA officials knowledgeable about the data. We determined that the data were sufficiently reliable for the purpose of reporting on aggregate PTAC performance. To determine whether PTACs met their goals in fiscal year 2020, we compared the goals in each PTAC’s 2020 award to the performance data in DLA’s 2020 Past Performance Data Sheet. Because DLA does not aggregate performance data on PTACs’ new clients, we could not make a comparison of PTACs’ performance against their goals for this metric.

To determine how frequently DCMA and ONR conducted performance reviews of PTACs, we requested from DLA all performance reviews conducted of PTACs’ 2017–2019 cooperative agreements. 2 To assess how consistently DOD conducted performance reviews of PTACs, we reviewed all 48 performance reviews conducted of 2017–2019 cooperative agreements. Using a standard form, one analyst reviewed each performance review to determine whether the reviews were conducted by DCMA or ONR, whether the reviewer used DLA’s suggested template, and what recommendation the reviewer made. A second analyst reviewed the identification and categorization. Where there were differences in the reviews of the first and second analyst, the two conferred and entered a final decision.

To understand DLA’s purpose and process for conducting site visits to PTACs, we interviewed DLA officials and requested the reports for all site visits conducted since they began in 2015. We reviewed these reports to assess the format of site visits and whether the number of site visits conducted each year met DLA’s annual goal.

To understand how oversight of PTACs may change in the future, we reviewed an independent audit by EY related to DLA’s oversight of PTACs and interviewed DLA officials about the changes they planned to make to the oversight process, including changes to the roles of DCMA and ONR.

To assess how DLA plans to track and evaluate its training requirements for PTAC staff, we reviewed PTAP’s general terms and conditions for the new training requirements. We also reviewed the template DLA created

2We could not review the performance reviews of 2020 cooperative agreements because we conducted our work before those performance reviews began.

Analysis of New Training Requirements



for tracking training and interviewed DLA officials to understand how they created the requirements and the template. We compared documentation on DLA’s training template against internal control standards.3 We determined that the information and communication component of internal control was significant to this objective, along with the principle that management should externally communicate the necessary quality information to achieve the entity’s objectives. We assessed whether the design of DLA’s plan to track training would allow it to communicate to PTACs what information they would have to track and report to help DLA achieve its training objectives.

We compared DLA’s approach to creating the requirements to our leading practices for strategic training and development efforts, specifically our leading practices related to planning and front-end analysis for developing training.4 To understand how PTACs will implement and track the new requirements, we spoke with representatives of APTAC and the six selected PTACs. To understand how APTAC planned to create new training curriculums for PTAC staff and program managers, we reviewed DLA’s requirements for APTAC’s curriculums and interviewed DLA officials about the requirements. We also interviewed APTAC representatives about their plans and time frame for creating the curriculums and about communication between APTAC and DLA during their development.

To understand DOD’s requirement to transition PTAP, we reviewed the National Defense Authorization Act for Fiscal Year 2020. To obtain information about DOD’s plans for the transition, we interviewed officials in DOD’s Office of Small Business Programs—which reports to the Office of Industrial Policy within the Office of Acquisition and Sustainment—and DLA. We also spoke with representatives of APTAC to understand how DOD has communicated its plans.

3GAO, Standards for Internal Control in the Federal Government, GAO-14-704G (Washington, D.C.: Sept. 10, 2014).

4GAO, Human Capital: A Guide for Assessing Strategic Training and Development Efforts in the Federal Government, GAO-04-546G (Washington, D.C.: Mar. 1, 2004). The guide summarizes attributes of effective training and development programs, and is intended to help managers assess an agency’s training and development efforts and implement improvements. The guide discusses key questions to consider when assessing the components of an agency’s training and development process: (1) planning/front-end analysis, (2) design/development, (3) implementation, and (4) evaluation.

Review of DOD’s Plans to Transition PTAP





To evaluate any overlap in the services provided by PTACs and SBDCs, we reviewed documentation—such as authorizing statutes, regulations, terms and conditions, and funding guidance—for both programs. We compared key aspects of each program, such as the types of services provided, purpose, goals, and intended beneficiaries, to identify areas of overlap. We used our guidance on fragmentation, overlap, and duplication to guide the comparison.5 We also reviewed documentation on the entities that were awarded funds to host PTACs and SBDCs in fiscal year 2020 to determine how many entities ran both programs. Specifically, we reviewed the names and addresses of SBDC recipients and compared it to PTAC recipients. If an SBDC or PTAC had the same address or were run by the same entity (the same college or university), we defined the centers as co-located. Because we compared names and addresses of the recipients, our number of co-located PTACs may not include satellite SBDC or PTAC offices that are co-located with a PTAC or SBDC.

To understand stakeholders’ views on overlap between PTACs and SBDCs, we interviewed representatives of DLA, APTAC, six PTACs, the three SBDCs that were co-located with the PTACs we interviewed, and SBA. We also interviewed DLA and SBA to understand the extent to which they track co-location of PTACs and SBDCs and what guidance they provide to centers that are co-located.

We conducted this performance audit from March 2020 to March 2021 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

5GAO, Fragmentation, Overlap, and Duplication: An Evaluation and Management Guide, GAO-15-49SP (Washington, D.C.: Apr. 14, 2015). The evaluation guide includes a comparative tool for identifying potential overlap and leading practices for collaboration, which can reduce or better manage overlap.

Overlap Between PTACs and SBDCs


Appendix II: Comments from the Department of Defense





Appendix III: Comments from the Association of Procurement Technical Assistance Centers







Appendix IV: GAO Contact and Staff Acknowledgments


William B. Shear, (202) 512-8678 or [email protected]

In addition to the contact named above, Paige Smith (Assistant Director), Shannon Smith (Analyst in Charge), Meghana Acharya, Rudy Chatlos, Jonathan Harmatz, Charlene Lindsay, John McGrail, Barbara Roesmann, and Jena Sinkfield made key contributions to this report.

Appendix IV: GAO Contact and Staff Acknowledgments

GAO Contact Staff Acknowledgments

(104204)


The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no cost is through our website. Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. You can also subscribe to GAO’s email updates to receive notification of newly posted products.

The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s website, https://www.gao.gov/ordering.htm.

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537.

Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube. Subscribe to our RSS Feeds or Email Updates. Listen to our Podcasts. Visit GAO on the web at https://www.gao.gov.

Contact FraudNet:

Website: https://www.gao.gov/about/what-gao-does/fraudnet

Automated answering system: (800) 424-5454 or (202) 512-7700

Orice Williams Brown, Managing Director, [email protected], (202) 512-4400, U.S. Government Accountability Office, 441 G Street NW, Room 7125, Washington, DC 20548

Chuck Young, Managing Director, [email protected], (202) 512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, DC 20548

Stephen J. Sanford, Acting Managing Director, [email protected], (202) 512-4707 U.S. Government Accountability Office, 441 G Street NW, Room 7814, Washington, DC 20548

GAO’s Mission

Obtaining Copies of GAO Reports and Testimony Order by Phone

Connect with GAO

To Report Fraud, Waste, and Abuse in Federal Programs

Congressional Relations

Public Affairs

Strategic Planning and External Liaison

Please Print on Recycled Paper.

https://www.gao.gov/

https://www.gao.gov/subscribe/index.php

https://www.gao.gov/ordering.htm

https://facebook.com/usgao

https://flickr.com/usgao

https://twitter.com/usgao

https://youtube.com/usgao

https://www.gao.gov/about/contact-us/stay-connected

https://www.gao.gov/subscribe/index.php

https://www.gao.gov/podcast/watchdog.html

https://www.gao.gov/

https://www.gao.gov/about/what-gao-does/fraudnet




Documents

GAO-21-287, Procurement Technical Assistance Program