Social Dynamics, LLC

Evaluation of the Employment and Training Administration/Office of Disability

Employment Policy Disability Employment Initiative (DEI)

AUTHORS

Social Dynamics, LLC: Anne Chamberlain, Douglas Klayman,

Heather Isbell Parker, and Kimether Shari Barlow

In Partnership with Altarum Institute: Andy Crouter and Laura Skaff

&

Berkeley Policy Associates: Kelley Akiya, Sherry Almandsmith,

Kristin Bard, Kay Magill, Paola Rochabrun Oliveira, and Linda Toms Barker

CONTACT INFORMATION Anne Chamberlain, Program Evaluation Director

achamberlain@socialdynamicsllc.com 301-990-1105

First Draft Submitted September 2, 2011

Final Draft Submitted October 3, 2011

PREPARED FOR: U.S. Department of Labor

Office of Disability Employment Policy

200 Constitution Avenue, NW

Washington, DC 20210

PREPARED BY: Social Dynamics, LLC

481 North Frederick Ave., Suite 410

Gaithersburg, MD 20877

Office: 301-990-1105

Fax: 301-990-1270

  

 

Table of Contents

Executive Summary............................................................................................................................. 4

1.0 Introduction .................................................................................................................................. 6

1.1 Purpose of the Report................................................................................................. 6

1.2 Report Overview............................................................................................................ 6

2.0 Background ................................................................................................................................... 6

2.1 Purpose of the DEI Grant........................................................................................ 7

2.2 Key Components of the DEI Grant ................................................................. 7

2.3 DEI Grantee Proposals: Basic Features ....................................................... 8

2.4 The DEI Evaluation Plan.........................................................................................10

2.4.1 Evaluating Implementation......................................................................10

2.4.2 Evaluating Outcome and Impact .........................................................11

2.4.3 Office of Management and Budget (OMB)

Submission............................................................................................................11

3.0 Progress on DEI Work Plan Tasks ..........................................................................11

4.0 Disability Employment Initiative Grantee Descriptions: What

We Learned on the Getting to Know You Visits.........................................12

4.1 A Typical Getting to Know You Visit..........................................................12

4.2 State Workforce Structure System and Policies....................................14

4.2.1 Workforce Investment Act (WIA) Services ...............................15

4.2.2 Wagner-Peyser Employment Service ..............................................15

4.2.3 Vocational Rehabilitation (VR)............................................................16

4.3 Ticket to Work .....................................................................................................................17

4.4 One-Stop Center Service Processes ....................................................................17

4.5 Disability Resource Coordinator (DRC) Roles and

Organizational Structure .............................................................................................19

4.6 Partnerships, Collaboration, and Leveraging Resources.....................22

4.7 Issues Specific to Working with Customers with

Disabilities ..........................................................................................................................23

4.7.1 Disclosing Disabilities .................................................................................23 4.7.2 WIA Performance Measures...................................................................24 4.7.3 Cultural Differences in Definitions and

Understanding of Disability ....................................................................25

4.8 Initial Steps Toward Systems Change ...............................................................26

2

  

 

 

Table of Contents (Continued)

5.0 The DEI Data System ...........................................................................................................27

5.1 Purpose of the DEI Data System.......................................................................27

5.2 Design of the DEI Data System .........................................................................27

5.2.1 Participatory Work with Grantees ......................................................28

5.2.2 Technical Meetings ........................................................................................28

5.2.3 Accommodating Different Intake Procedures and Systems ....................................................................................................................29

5.3 Issues and Solutions in the DEI Data System .........................................31

5.4 Anticipated Year Two Outcomes and Impact.........................................32

6.0 Summary .......................................................................................................................................... 33

7.0 Moving Forward: Year Two Plans ..........................................................................34

References ..................................................................................................................................................... 35

Exhibits

Exhibit 1: DEI Grantee Structures............................................................................................. 8

Exhibit 2: Progress on DEI Evaluation Work Plan Tasks.......................................12

Exhibit 3: Town Hall Meetings ...................................................................................................14

Exhibit 4: State DRC Structure....................................................................................................19

Exhibit 5: DEI Data System User Type and Associated System Capabilities.........................................................................................................................27

Exhibit 6: Grantee Operating System(s) Used to Collect/Manage WIA/WP Customer Data.........................................................................................30

Exhibit 7: Data Collection System Activity Completion Status ........................30

3

  

Executive Summary

As of July 2011, there is a 40 percent gap between the employment rate of people without disabilities and that of people with disabilities. The Disability Employment Initiative, or DEI, is a joint effort by the U.S. Department of Labor’s (DOL) Office of Disability Employment Policy and its Employment Training Administration to facilitate systemic change in the workforce system in order to “increase the effective and meaningful participation of people with disabilities in the workforce” (DOL, 2010). One year ago, nine states were awarded DEI grants and set forth to change policy, practices, and, ultimately, culture, as they relate to the employment of people with disabilities. In tandem with this effort, a comprehensive evaluation was launched to maximize this opportunity to learn from each state’s implementation of the DEI. As DEI progresses, the evaluation will also be able to answer key questions related to the initiative’s outcomes and impacts. This first synthesis report on the Evaluation of the Disability Employment Initiative reviews DEI evaluation activities through July 31, 2011.

In this start-up year, we have laid the groundwork for evaluation activities for the next two years and beyond. Local Workforce Investment Areas (LWIAs) participating in DEI within each state were randomly assigned to either the DEI pilot or comparison (business-as-usual) condition. The evaluation team planned comprehensive site visits to DEI grantees for spring 2012 that will document DEI implementation, and then planned and conducted a preface “Getting to Know You Visit” for Spring 2011. Each state was visited, and evaluators held informal meetings with DEI and comparison staff at state, LWIA, and One-Stop Career Center levels. We documented these visits and checked and discussed implementation and program start-up issues with each state. While site visit planning and activities were progressing, we were also designing the DEI Data System, a major program-wide web-based system that will accommodate Workforce Investment Act (WIA), Wagner-Peyser (WP), and the DEI Data Elements for tens of thousands of customers. This system is being customized state-by-state and has been planned collaboratively with DEI administrators and data personnel from each grantee. Social Dynamics worked with each state to map its existing data collection system with the DEI Data System to identify gaps that would need to be resolved. We also worked with state agencies to develop customized data sharing and confidentiality agreements, designed data security procedures, and held numerous “technical” conference call meetings with state leaders.

Our activities this year have yielded insights into DEI that we think are noteworthy for their potential impact on the evaluation, as well as DEI itself. These insights, which are illustrated in more detail in the report, include the following:

For the most part, State Leads and other state-level grantee staff understand the purpose and scope of the DEI grant. They are deeply committed to its success as a system-wide initiative that will impact the environment of the One-Stop Career Centers, and the experience and outcomes of many customers. However, at the local (One-Stop Center) level, staff in many sites described DEI as a specific program or service(s) that particular staff would only provide to particular customers. Some staff called DEI “the DRC [Disability Resource Coordinator] grant.” We hope to see the understanding and commitment of the DEI leaders (including State Leads and DRCs) spread among their colleagues in the coming year.

4

  

We know that contextual issues (such as state policy) are bound to affect the implementation of any initiative, yet we can never be certain what those contextual issues will be. In the case of this initiative, DEI states reported delays in hiring key staff due to state hiring freezes or personnel policy requirements. Additionally, cuts in WIA and other governmental funding forced budget realignments at some One-Stop Career Centers, affecting DEI implementation.

States have developed unique ways of registering and tracking One-Stop Center customers with disabilities. While this is often in response to local needs, the variation between and within states presents a challenge for the DEI evaluation. For example, who gets “counted” as a customer with a disability? Is a customer asked if s/he has a disability? How? By whom? Is a customer given the option to describe his or her disability? At what level of detail? The answers to these questions vary from state to state, and often within states (when comparing, for example, a large, urban One-Stop Center that struggles to serve vast numbers of customers, to a small, rural one, where customers can receive more one-on-one attention). Another issue is the process of following-up customers six months after intake. There is considerable variation across and within states in terms of how this accomplished. In some areas it is simply not done and One-Stop Centers rely on “soft exits,” a computer-generated closing-out of the administrative file. While some areas have in place a systematic follow-up process for collecting information on the status of customers six months after intake, this is not the norm across the nine DEI grantees. Few states and/or local areas have consistent and systematic procedures in place to follow-up One-Stop Career Center customers. Variation is also widespread in states’ record-keeping of customers being served within the One-Stop Center by different partners (WIA, Wagner-Peyser, Veterans, Vocational Rehabilitation, etc.).

The purpose of this report is to inform the Department of Labor’s Office of Disability Employment Policy (ODEP) and Employment and Training Administration (ETA) of the progress made to date on DEI evaluation activities. This not only ensures that we are “on track” with our proposed evaluation, but it also provides information to facilitate DEI start-up (on both the program and the evaluation side) for the 2011 cohort of DEI states. The evaluation team is looking forward to beginning work with the next cohort, as well as to continuing our partnership with the first nine states.

5

  

1.0 Introduction

This is the first annual report (synthesis report) for the Evaluation of the Disability Employment Initiative (DEI) and includes a detailed review of DEI evaluation activities through July 31, 2011. This report is submitted by Social Dynamics, LLC, in partnership with Berkeley Policy Associates and Altarum Institute.

1.1 Purpose of the Report

The purpose of this report is to inform the Department of Labor’s (DOL) Office of Disability Employment Policy (ODEP) and Employment and Training Administration (ETA) of the progress made to date on DEI evaluation activities. This not only ensures that we are “on track” with our proposed evaluation, but it also provides information that may be useful in smoothing DEI start-up (on both the program and the evaluation side) for the next cohort of DEI states.

Additionally, this report sets the stage for the DEI evaluation findings that will emerge in Year Two and be reported in the second synthesis report (August 2012).We have structured this report to provide the background and contextual information necessary to understand future findings.

1.2 Report Overview

In Section 2, we provide background information on DEI, how it was developed, and its goals and objectives. In Section 3, we outline the progress made on the DEI work plan tasks and in Section 4 we detail what the evaluation team learned during what we called “Getting to Know You” visits with each state in Spring 2011. Section 5 reviews the progress made in preparing our DEI Data System to collect customer outcomes beginning in Year Two. Section 6 summarizes our findings, and Section 7 concludes the synthesis report by considering the upcoming Year Two activities in light of what we have learned so far.

2.0 Background

The current employment rate of people with disabilities is 18.0 percent, compared to a rate of 63.8 percent among individuals without a disability (DOL, 2011).These disproportionate employment rates reveal a need for programs and practices that improve employment outcomes of people with disabilities. Previous DOL initiatives directed at improving the employment outcomes of people with disabilities include the ETA’s Work Incentive Grants (WIG) and the Disability Program Navigator (DPN) program, as well as employment service models, including ODEP’s Customized Employment (CE), Workforce Action Grants (WAG), Self-Employment Technical Assistance and Research (START-UP), disability grants, and the State Intermediary Youth Grants administered through ODEP. Through these initiatives and programs, DOL has identified practices and policies that are being implemented in the public workforce development system and designed to improve employment outcomes of people with disabilities (PWD).

6

  

2.1 Purpose of the DEI Grant

In August 2010, ETA and ODEP collaboratively released a Solicitation for Grant Applications (SGA) that made provisions for approximately $22 million dollars in funds for cooperative agreements to go to state Workforce Investment Act (WIA) entities. This funding was a product of the Combined Appropriation Act of 2010, of Public Law 111-117, which allocated $12 million to ETA and $12 million to ODEP in order for the two agencies to develop and implement an approach that would increase the effective and meaningful participation of people with disabilities in the workforce (DOL, 2010). In order to accomplish this goal, ODEP and ETA conceptualized DEI to: (a) refine and verify those strategies for employing people with disabilities that have proven effective or promising; and (b) foster the replication of those strategies across the workforce system. DEI is designed to help people with disabilities find a way to the middle class through model service delivery via the public workforce system (DOL, 2010).

2.2 Key Components of the DEI Grant

DEI supports state efforts to improve opportunities for and outcomes of youth and adults with disabilities who are unemployed, underemployed, and/or receiving public assistance (DOL, 2010). A key component to DEI is expanding the public workforce development system through the Social Security Administration’s Ticket to Work (TTW) Program (ETA, 2010). Eligibility for DEI funds required a commitment to participate (at state and/or local levels) as Employment Networks (ENs) in the TTW program. The SGA also required each state to identify a state lead who would be responsible for coordinating DEI though the local workforce investment areas (LWIAs) and to create a Disability Resource Coordinator (DRC) position. States could propose the number, location, and responsibilities of their DRCs. Proposals were required to identify how the DRC position and DEI strategies would be sustained beyond the timeframe of the grant. All participating LWIAs were required to ensure the physical and programmatic accessibility of their One-Stop Centers.

After identifying the target population as either youth or adult, each state applying to participate in DEI had to select a minimum of two of seven strategies that would be the core foci of their grant. The seven strategies, identified as effective or promising by ETA and ODEP, are:

1. Integrated Resource Teams (IRTs);

2. Customized Employment (CE);

3. Integrating Resources and Services, Blending and Braiding Funds, Leveraging

Resources;

4. Self-Employment;

5. Guideposts for Success (youth specific);

6. Asset Development Strategies; and

7

  

 

 

 

 

7. Partnerships and Collaboration (DOL, 2010).

2.3 DEI Grantee Proposals: Basic Features

Provided that they met the eligibility requirements listed above, applicants could present a plan to implement DEI flexibly in the manner most appropriate to their unique locales and circumstances. Exhibit 1, below, shows the structural components for all nine states awarded DEI grant funds.

Exhibit 1: DEI Grantee Structures State/

Award/ Population/

LWIAs

Strategic Service Components Key Partners Outcome Goals

Alaska $2,727,000 Adults 1 LWIA

1. Integrated Resource Teams 2. Integrating Resources/

Blending & Braiding funds 3. Customized Employment 4. Self-Employment 5. Asset Development Strategies 6. Partnerships & Collaboration

1. Medicaid Infrastructure Grant (MIG) 2. Vocational Rehabilitation 3. Independent Living 4. Governor’s Council on Disabilities 5. Veterans Employment and Training Service

(VETS) 6. TANF

1. Increase # of PWD who use One-Stop Career Centers by 20%

2. Bring all One-Stops to a level III of DPN maturity model

3. Increase # of CWICs & # of CE providers by 15%

4. Increase availability of local resources

1. Youth Development Collaborative 2. Department of Career Education

Arkansas $1,500,000 Youth 4 LWIAs

1. Integrated Resource Teams 2. Customized Employment 3. Guideposts for Success

3. Economic Development Commission 4. Department of Higher Education 5. Association of Two-Year Colleges 6. AR Science and Technology Authority 7. Disability Rights Center 8. Increasing Capabilities Access Network

1. Increase the workforce 2. Expand the services of One-

Stops 3. Facilitate systems change to

improve overall efficacy 9. Community Work Incentive Coordinators 10. Independent Living Centers

1. Establish goals related to assessment, mentoring, plan development, leadership

1. Department of Education development, completion of Delaware $1,490,409 Youth 1 LWIA

1. Integrated Resource Teams 2. Self-Employment 3. Guideposts for Success

2. Division of Employment and Training 3. Department of Health and Human Services 4. Divisions of Developmental Disabilities Services 5. Substance Abuse and Mental Health

education or training, employment achieved, & employment retention

2. Success will be determined 6. Technical and Community Colleges through education & work

experiences for youth & increased self-sufficiency post-high school for youth

8

  

 

Exhibit 1: DEI Grantee Structures (continued) State/

Award/ Population/

LWIAs

Strategic Service Components Key Partners Outcome Goals

Illinois $1,839,588 Adults 4 LWIAs

1. Integrated Resource Teams

2. Integrating Resources/ Blending & Braiding Funds

3. Customized Employment 4. Asset Development

Strategies 5. Partnership & Collaboration

1. MIG 2. Work Incentive Planning and Assistance Program 3. Department of Employment Security and their

Veterans Representatives 4. Department of Health and Family Services 5. Southern IL University and Community Colleges 6. Disability and Business Technical Assistance 7. Easter Seals 8. Local and Regional Chamber of Commerce(s)

1. Provide core services to 1,068 adults

2. Provide 912 intensive services 3. Provide 549 training services 4. Achieve employment of 977

adults 5. Achieve employment retention

at or above $12.17 hourly wage for 781 adults

Kansas $1,879,459 Adults 4LWIAs

1. Integrated Resource Teams

2. Integrating Resources/ Blending & Braiding Funds

3. Self-Employment 4. Asset Development

Strategies 5. Partnership & Collaboration

1. Independent Living Centers 2. Community Mental Health 3. Community Developmental Disability Organization 4. KS Assistive Technology Cooperative 5. Cerebral Palsy Research Foundation 6. Social Rehabilitation Services 7. Commission on Disability Concerns 8. Business Leadership Network 9. Department of Corrections 10. VETS

1. Provide pathways to middle class through improved education & training for program participants

2. Increase the # of PWD who register in workforce system, enter employment, & retain employment & wages

Maine $1,500,00 Adults 2 LWIAs

1. Integrated Resource Teams

2. Integrating Resources / Blending & Braiding Funds

3. Asset Development Strategies

4. Partnership & Collaboration

1. Bureau of Rehabilitation Services 2. MIG 3. Choices CEO 4. Department of Health and Human Services 5. Work Incentive Planning and Assistance Project 6. Protection and Advocacy Program for Beneficiaries of

Social Security 7. Commission on Disability and Employment 8. Educational orgs and Community providers

1. Host at least 2 IRT meetings per month

2. Fully implement TTW program in Coastal Counties Region

3. Increase TTW enrollments by 10–15 participants per year

4. Increase by 50% participants in career exploration workshops

New Jersey $2,479,280 Youth 6 LWIAs

New York $4,945,060 Adults 25 LWIAs

1. Integrated Resource Teams

2. Integrating Resources / Blending & Braiding Funds

3. Self-Employment 4. Guideposts for Success 5. Partnership & Collaboration

1. Integrated Resource Teams

2. Integrating Resources/ Blending & Braiding Funds

3. Asset Development Strategies

4. Partnership & Collaboration

1. Division of Vocational Rehabilitation 2. MIG 3. Division of Disability Services 4. TANF 5. Elizabeth Boggs Center 6. Henry H. Kessler Institute 7. Business Leadership Network 8. NJ Chamber of Commerce 9. Mental Health Association 10. VETS 11. Juvenile Justice Commission 12. NJ Schools and Community Colleges System 1. Adult Continuing Career Educational Services 2. Commission for the Blind and Visually Handicapped 3. Offices of Mental Health 4. People with Developmental Disabilities 5. Alcoholism and Substance Abuse Services 6. Veteran Services 7. MIG 8. Work Incentives Information Network 9. NYS Federal Bonding Program

1. Create systems change to establish ongoing network of services that maximizes funding

2. Increase access to & use of the One-Stop Career Centers

3. Improve education, training, &/or employment outcomes

1. Increase in services provided to PWD

2. Increase # of customers entering employment, retaining employment, & average six-month earners

3. Increase # of tickets served 4. Increase # of IRTs conducted

9

  

                                                            

Exhibit 1: DEI Grantee Structures (continued) State/

Award/ Population/

LWIAs Strategic Service Components Key Partners Outcome Goals

1. Increase employment &economic 1. Department of Rehabilitative Services self-sufficiency of PWD 2. MIG1. Integrated Resource Teams 2. Increase access to One-Stop Virginia 3. Veterans Workforce Outreach Project 2. Integrating Resources/ system$2,915,779 4. Mid-Atlantic ADA CenterBlending & Braiding Funds 3. Enhance servicesAdults 5. Assistive Technology @Work3. Asset Development Strategies 4. Engage employers9 LWIAs 6. TANF4. Partnership & Collaboration 5. Increase the use of work7. Work Incentive Planning and Assistance Program incentives & asset development 8. VA Individual Development Accounts Program strategies

2.4 The DEI Evaluation Plan

The DEI evaluation was designed to facilitate the understanding of DEI implementation, outcomes, and impact, as well as the use of this data for possible program and policy change. Evaluation of DEI processes and outcomes will be conducted through a qualitative analysis of data generated by document review (such as LWIAs’ annual reports and outreach materials), observations, interviews, and focus groups to supplement the quantitative analysis of outcome (customer employment) data using a randomized experimental design.

The commencement of DEI for grantees occurred concurrently with the start-up of the independent evaluation conducted by Social Dynamics and its partners, Berkeley Policy Associates, Altarum Institute, and Mathematica Policy Research. The evaluation team began by collectively reviewing the proposals from the DEI awardees. Participating LWIAs (determined by each state) were stratified within states based on locale type (rural, urban, or combination rural/urban). Following stratification, LWIAs were randomly assigned within category to either treatment (DEI) or comparison (business-as-usual) condition.1 The DEI evaluation team assigned a state evaluation liaison to each state to better support grantee’s unique needs and to effectively manage evaluation communications.

2.4.1 Evaluating Implementation

Implementation of DEI will be assessed from multiple perspectives using a number of data collection methods. Interview data will be collected via formal interview protocols for state-level personnel, state Workforce Investment Board (WIB) administrators, local WIB (LWIB) administrators, and One-Stop Center managers and staff, including DRCs. To gather customer-level data, the evaluation team will conduct customer focus groups of DEI participants. Within this first DEI year, it was expected that each state would begin implementation of DEI. In order to gauge progress to date, the evaluation liaisons arranged for Getting to Know You site visits with their respective states. These visits were informal (i.e., structured interviews were not

1 Delaware does not have LWIBs and Alaska operates as a single LWIA even though technically it is comprised of two LWIAs. For this reason, DE and AK were not part of the random assignment process. All of their One-Stop Career Centers will be considered treatment (DEI) sites.

10

  

                                                            

conducted) and were intended to familiarize the sites with the upcoming evaluation and increase the evaluation team’s knowledge about the structure, policies, and goals in each DEI state.

As the DEI evaluation progresses, quarterly WIA and Wagner-Peyser (WP) reports from DEI and comparison sites will also provide implementation data. These reports will detail the customers served and services offered by One-Stop Centers, whether and how these change over time, and challenges and successes.

2.4.2 Evaluating Outcome and Impact

The outcome and impact of the DEI evaluation will also be assessed using a mixed methods design. Through a DEI data collection process that has been vetted for integration into existing workforce case management systems (see Appendix 2), quantitative measures of customer outcomes and progress will be collected quarterly from each state through the state’s WIA and WP reports, and state-level case management data, where available. Any remaining customer data will be captured by added “DEI Data Elements” that are currently being integrated or attached to each state’s existing data systems. Data from these sources will allow for comparisons of DEI and comparison sites (across states)2 in terms of, for example, the number and types of customers served, types of services, average length of time receiving services, and average length of time receiving benefits.

The combined implementation, outcome, and impact data, in both quantitative and qualitative forms, will yield a comprehensive and holistic picture of DEI in practice. This information will culminate in a final evaluative report for the Department of Labor.

2.4.3 Office of Management and Budget (OMB) Submission

In compliance with the Government Paperwork Reduction Act of 1980 (44 U.S.C. 3501 et seq.), the evaluation team submitted an OMB clearance package to ODEP on March 31, 2011. The OMB package provides the Federal Government with the scope and requirements of a study, the rationale for data collection, and planned data use. This information allows the government to ensure that the burden placed on participants on behalf of the government is not unnecessary or inappropriate. The OMB submission was advertised in the Federal Register on June 20, with a 60-day period during which the public may comment on its contents. ODEP will include any relevant comments in the request for approval, in which case they become a matter of public record attached to the OMB’s final decision regarding the study’s request for permission to collect data.

3.0 Progress on DEI Work Plan Tasks

The evaluation team has, to date, met the timelines and deliverables promised in the Evaluation Work Plan of December 31, 2010. Exhibit 2 reviews progress on each Task.

2 Delaware and Alaska will not be part of the cross-state impact evaluation; however, customer outcomes from these states will still factor into the larger DEI evaluation.

11

  

                                                            

Exhibit 2: Progress on DEI Evaluation Work Plan Tasks Task Number/Description Progress to Date

1. Develop Work Plan Completed. 2. Assist in Determining Control Sites or Comparison Groups Completed.

3. Documenting Start-Up Issues: In-Depth Site Visits

On schedule: • Questions for in-depth site visits developed & approved by ODEP & ETA. • OMB package submitted to ODEP.3 • Protocol developed for informal Getting to Know You visits. • Getting to Know You visits conducted in each DEI state (see Section 4). • Getting to Know You visits documented & evaluator notes on state structure verified

by state DEI personnel.

4. Design and Implement Data Collection System and Database Development

On schedule: • Technical meetings conducted with DEI & data staff from each DEI state. • Cross-walking of existing case management &registration systems cross-walked with

required DEI Data Elements. • DEI Data Elements determined per state. • Preferred delivery method/interface determined per state. • Database and web-portal development on time for September start-up (see Section

5.2.3). 5. Write Synthesis Report Year One draft one completed. 6. Prepare Progress Reports DEI weekly updates & monthly progress reports are shared among key DEI evaluation stakeholders. 7. Present Findings at Meetings and Presentations to ETA and ODEP

We look forward to presenting findings to ETA and ODEP following Year Two. Additionally, we would be pleased to present preliminary findings from Year One.

8. Develop OMB Package for Accessibility Study of One-Stop Centers

The DEI evaluation team prepared the OMB package for the study, including a comprehensive data collection instrument that covers communications, programmatic and physical accessibility indicators.

4.0 Disability Employment Initiative Grantee Descriptions: What we learned on the Getting to Know You Visits

This section provides a summary of some of the key highlights of the Getting to Know You Visits to the nine DEI grantees. After first describing the visits, we summarize some preliminary information and observations about the DEI pilot and comparison sites that provide a glimpse of the early stages of implementation of the initiative. The information provided in this section represents the opinions and perspectives of the staff we spoke with on the visits.

4.1 A Typical Getting to Know You Visit

The first step we took in preparing for a Getting to Know You Visit began before the actual visit started, when the evaluation team analyzed and reviewed relevant materials about the site. Relevant documentation included grantee applications, project descriptions, current state reporting instruments and reports, and observation instruments and protocols from related DOL site visit studies. The general background knowledge of the evaluation team also included a foundational understanding of the workforce investment system and the history of DOL efforts to serve PWD through the One-Stop Career Centers.

3 Project timing did not allow for OMB submission in time to receive approval for in-depth site visits. Instead, researchers conducted informal Getting to Know You Visits. See Section 4.

12

  

                                                            

Based on this review, the evaluation team developed a list of very general discussion topics and an overall plan for the Getting to Know You Visits. The plan provided an overview of the individuals to meet on-site, potential meetings or group activities to be observed during each visit, an estimate of the amount of time that each meeting would require, and the topics to be discussed. Unlike the more structured site visits planned for future years,4 the Year One Getting to Know You Visits centered on informal conversations with staff about their initial implementation of DEI and their plans for gathering and providing the data needed for the evaluation. The plan for these visits did not include formal data collection, interview protocols, or lists of specific questions to ask One-Stop Career Center staff.

It is important to emphasize that we deliberately designed these visits as a way to introduce the evaluation team to the grantees and gather preliminary information about their sites in an informal, unstructured way. The evaluation team also planned to use the visits as an opportunity to explain the DEI evaluation to staff at the state, LWIA, and One-Stop Career Center levels, and to address any questions and concerns. Actual site visit data collection will begin next year after OMB approval of the site visit data collection protocol.

The evaluation team used a grantee-driven approach in making the arrangements for the Getting to Know You Visit in each state. To begin this collaborative process, the evaluation liaison for each state called and e-mailed the State Lead to explain the purpose of the visit and to request assistance with scheduling meetings and activities. This conversation also addressed logistics, such as which sites to visit, the order of site visits, and travel time between sites. Typically, agendas were jointly developed by executive DEI personnel and the evaluation liaison.

In each state, a team of between two and five evaluators conducted the Getting to Know You Visit over the course of three to five days. At least two members of each site visit team were senior staff, so that they could potentially divide the team, if necessary, to visit different LWIBs or One-Stop Centers after joint meetings with state-level staff.5 In addition to spending at least a half-day with the state-level staff working on DEI, the evaluation team visited as many different pilot and comparison sites as possible, visiting at least two One-Stop Centers in each state. We also offered to conduct multiple town hall meetings during each visit in order to meet with local evaluation participants (both pilot and comparison sites) and address issues related to the DEI evaluation. Exhibit 3 lists the town hall meetings by state.

4 During this year, the evaluation team developed formal site visit protocols for visits in Years Two and Three of the

grant that were reviewed and approved by ODEP and ETA.

5 Because of scheduling conflicts, only one senior and one junior staff member visited New Jersey.

13

  

                                                            

Exhibit 3: Town Hall Meetings State Town Hall NotesMeetings

Alaska 0

Arkansas 3 Town hall meetings were held to introduce participants to DEI in general; the discussion of the DEI evaluation was cursory. Delaware 0 None necessary, as evaluators were able to visit every One-Stop.

Illinois 1 Two additional town hall meetings were scheduled but cancelled by LWIA/center personnel during the visit. Kansas 3

Maine 3 New

Jersey 0

New York 2

Virginia 0 LWIB directors were at a meeting during visit so evaluators were invited to present DEI at an executive meeting, which included all LWIB directors. The evaluation team attended 3 LWIB meetings to discuss the implementation of the DEI in Virginia..

During each visit, the evaluation team met with the DEI State Lead, state data personnel, DRCs, LWIB members, One-Stop Center managers and staff, and key personnel from partner agencies. When possible, the team also observed state meetings or other DEI-related activities. Team members gathered information on the structure of WIA services and the Wagner-Peyser program, lists of partners and collaborators, service brochures and project descriptions, and other relevant documents.

4.2 State Workforce Structure System and Policies

As noted, the goal of DEI is to help PWD improve their employment outcomes through services delivered via the public workforce development system funded under WIA. The entities that received DEI grant funds provide employment and training services for disadvantaged adults, dislocated workers, and youth through One-Stop Career Centers. ETA also provides services to both job- seekers and employers under the Wagner-Peyser Act of 1933, which established a nationwide system of public employment offices known as the Employment Service (ES).6 The ES participates in the One-Stop Center system, although considerable variation exists among and within states in the type and degree of ES involvement. In addition, as a major employment services provider for PWD, the state/federal Vocational Rehabilitation (VR) system is a key player in the workforce system’s efforts to serve PWD, and therefore a critical partner in the DEI effort.

Appendix 1 provides information on the systems, structures, and policies of the public workforce development system in the nine grantee states, including WIA, ES, and VR. It includes a chart of state workforce agency structures and policies relevant to the DEI grant.

6http://www.doleta.gov/programs/wagner_peyser.cfm

14

  

4.2.1 Workforce Investment Act (WIA) Services

Federal WIA policy affects DEI implementation in many ways. For example, both state and local workforce agencies are subject to performance measurement requirements to ensure their accountability. These measurements typically include outcome measures of job placements of registered WIA customers. If more than one of the agencies providing employment services for PWD wishes to claim “credit” for an employment outcome and no mechanism exists for shared credit, there is potential for conflict or even a disincentive to collaboration. Several grantees, such as Arkansas and Virginia, addressed this concern by making a state-level decision that WIA staff and staff from VR or ES could share the credit for assisting people with disabilities in obtaining or maintaining employment. Credit-sharing will be explored further in our spring 2012 site-visits.

Another factor in the grantees’ implementation of DEI is the WIA service delivery model, which comprises three different levels of service: core, training, and intensive (see Appendix 1). DOL has no requirement that One-Stop Centers register individuals who utilize core services only (although some One-Stop Centers, such as many in Virginia, record anyone who comes to the Center, no matter the level of service utilized). Therefore, there is no way to uniformly track those One-Stop Center customers with disabilities who use core services exclusively. For this reason, the present evaluation focuses on customers with disabilities who receive WIA intensive or training services.

4.2.2 Wagner-Peyser Employment Service

ES provides a variety of employment-related labor exchange services to both job seekers and employers with job openings. Services for job seekers include job search assistance, job referral, and placement assistance. Depending on the needs of the local labor market, in some circumstances ES staff may provide other services, such as assessments of job seekers’ skill levels, abilities, and aptitudes; career guidance; job search workshops; and referrals to training. The job seekers served by the ES system include veterans, individuals with disabilities, migrant and seasonal farmworkers, ex-offenders, youth, minorities, and older workers. ES staff provides assistance via one of three modes:

1. Self-service;

2. Facilitated self-help services; and

3. Staff-assisted services.

ES is responsible for providing reemployment services to unemployment insurance (UI) claimants whom profiling methods identify as likely to exhaust benefits. These claimants must participate in reemployment services to continue receiving unemployment benefits. ES staff provides the individual with an orientation to the program, conducts an assessment, and in consultation with the claimant, develops an Individual Service Plan that describes appropriate services in which the claimant must participate. ES staff offers employers assistance in: developing job order specifications and fulfilling special recruitment needs; arranging for job

15

  

                                                            

fairs; analyzing hard-to-fill job orders; job restructuring; and dealing with layoffs. In some states, WP staff members are also responsible for matching job seeker experience with employer job requirements and referring job seekers to employers with job openings.

States and local WIBs have negotiated various roles for ES within their One-Stop Centers. For example, New York State requires that all UI recipients visit a One-Stop Center for reemployment services (as compared to other states that require only a percentage of UI claimants to attend reemployment services), resulting in disproportionately large numbers of customers entering the One-Stop Centers to attend orientations. In fact, at the time of the Getting to Know You Visit, the majority of visitors to the One-Stop Centers in New York were UI claimants. In Alaska, ES staff operates the One-Stop Centers with little involvement from WIA staff. In other states, such as Maine, ES operates quite separately from the rest of the One-Stop Center.

4.2.3 Vocational Rehabilitation (VR)

While a later section (4.6) discusses partnerships and collaboration between the One-Stop Centers and other entities, the relationship with vocational rehabilitation is important to highlight here because of VR’s key role in providing employment services for individuals with disabilities. The VR system has been providing employment services for PWD for over 75 years and has expertise that can be valuable to the mainstream public workforce development system. On the other hand, the VR system serves only a small proportion of job seekers with disabilities (due to the eligibility requirements of the VR enabling legislation).

The WIA and VR systems can complement each other and leverage each other’s resources if they work closely together. Staff at the sites we visited consistently emphasized the importance and value of colocation of VR counselors within the One-Stop Centers.7 They explained that, in their experience, not only does this foster better communication between employment counselors on both sides, but being able to walk a customer over and introduce him or her to the other agency’s counselor can help overcome any reluctance on the part of the customer to seek additional services. Some sites reported that a major challenge to colocation is the cost of sharing space. This seems to be a particular challenge in some areas where VR either lacks the funds or where state or district administrators interpret the enabling legislation as restricting the agency from spending VR funds on space where non-disabled customers are also served.

A key factor in determining the roles of VR and WIA in collaborating to provide services is the availability of VR case service funds that can be leveraged to pay for needed services. States vary in the amount of resources available, and those with insufficient funds must implement an Order of Selection giving priority to individuals with the most significant disabilities. The state must put applicants who do not qualify for services under the Order of Selection on a waiting list. One-Stop Center staff we spoke with who knew that VR in their state was currently on Order

7 It is important to note that colocation was not always synonymous with integration. At some of the sites we visited, simply being under the same roof did not guarantee that the two agencies worked together to serve customers. Also, the scope of the relationship between the two agencies, even with colocation, ran the gamut from cold referrals to integrated service delivery.

16

  

                                                            

of Selection (Delaware, Maine, Virginia8) reported that they expected to see increasing numbers of customers coming to them from VR and that they were worried that VR staff would have less time for providing consultation and support to One-Stop Center staff about disability issues.

4.3 Ticket to Work

Another critical factor in DEI implementation is the Social Security Administration’s TTW program. In the SGA, DOL required that the state workforce agency or the LWIBs involved in DEI must apply for Employment Network (EN) status within the TTW program if they did not already hold that status. Alaska,9 Arkansas, Kansas, and Maine are seeking EN status at the state level, while the other states have been actively working toward gaining EN status at the LWIB level. Some states/sites have completed the application process and have begun to participate in TTW. At the time of this report, these include Juneau, Alaska; all pilot sites and one comparison site in New York; Peoria, Illinois; the Delaware WIB; the State Bureau of Employment Services in Maine; and the New Jersey pilot sites.

Despite this progress, the grantees vary a great deal in the type and degree of implementation of the requirement to become an EN. The TTW program is very complex, from learning how the tickets work and how the billing and reimbursement system operates, to understanding what the Partnership Plus program for sharing tickets with VR is and how best to assist customers in determining the feasibility of earning sufficient income to give up their Social Security benefits. Most grantees are still early in their learning curve. Luckily, individuals who we spoke to during site visits reported that the actual process of applying to become an EN has not been too difficult because the technical assistance available to assist in the application process (provided by SSA through the National Disability Institute) has been excellent. Only Juneau reported difficulty, and not with the application itself, but with the need to secure its own EIN and DUNS numbers.

4.4 One-Stop Center Service Processes

We found considerable variation in the processes by which the One-Stop Centers serve their customers from “intake” to exit (and, sometimes, follow-up). Variations include which staff is involved in which phases of the service process, how and when internal and external referrals are made, and how decisions are made about going beyond core services. These variations in service delivery process provide a structural context for how DEI is being implemented in each state and LWIA, and can have significant implications both for the progress of the demonstration and for evaluating its effectiveness.

For example, an important consideration in evaluating systems change outcomes is the process by which PWD enter the public workforce development system and make their way into the WIA intensive and training services that have the potential to substantially impact their employment outcomes. How customers become involved with a One-Stop Center, which staff are involved in helping them identify employment goals and needed services, and how their disability might be identified and taken into consideration during the service process are all

8 New Jersey is also on Order of Selection, but is currently able to serve all eligible applicants.

9 Although Alaska is an EN at the state-level, it is implementing this status in one One-Stop Center at a time,

beginning in Juneau.

17

  

critical to understanding the impact of DEI on serving customers with disabilities. Procedures for becoming involved with the workforce system vary widely across states. In Delaware, for example, the state VR agency is the grantee and oversees implementation of DEI, focusing on serving youth through the high schools throughout the state.

“Registration” in the workforce system also varies considerably across states. In Alaska, customers need to register in ALEXsys, a unique online job search database that can be used by individuals living in extremely remote communities. Customers can work on ALEXsys from home or the local library, or can come into the One-Stop Center and use a computer in the resource room after they sign in. The Illinois workNet online system, which provides One-Stop Center visitors with information on employment and related services and tools, is also universally accessible to visitors no matter where they are located. At KansasWorks, customers identified as seeking core services self-register and open a job search account; customers seeking intensive and training services must have a KansasWorks plus account.

The level of training and expertise of staff, and the extent and timing of contact with staff, can also have a significant impact on customers’ experiences with the workforce system. For example, in Illinois, sites usually have one to three people to help out in the resource room. These individuals may include staff from other agencies, such as VR, that rotate into the One-Stop Centers and can provide agency-specific information, as well as general services. However, core/universal customers who use only the resource room typically have limited staff contact. In other states, however, most customers who visit the One-Stop Centers have the option to meet one-on-one with a career consultant (Maine), caseworker (New York), or a case manager (Virginia). At most of the sites visited in New York, nearly all customers enter as WIA intensive customers and go through a sit-down, one-on-one assessment with a caseworker and complete an employment plan. In the initial one-on-one assessment, staff determines which service track the customer needs: (a) Job Search Ready, if the customer just needs help with job leads; or (b) Career Development Services, if the customer needs more help with his/her résumé, interviewing, etc. The state defined these two service tracks, which all of the New York One-Stop Centers use.

When asked in the informal Getting to Know You Visit to describe their typical service process and how they serve individuals with disabilities, managers and staff at most One-Stop Centers we visited indicated that customers with disabilities are served in the same way as customers without disabilities until they disclose their disability, at which point they are usually referred to a “specialist,” such as a VR counselor or a DRC/DPN. In general, each One-Stop Center has one or more work stations reserved for use with assistive technology and at least one One-Stop Center staff member is familiar with the assistive technology. There are some differences, however, in how One-Stop Centers are adapting to increasing their focus on serving customers with disabilities. For example, in Illinois, select staff members in Peoria and Chicago have been trained to have special expertise in the technology, via assistive technology teams created by the former Chicago DRC. In Kansas, the Topeka Workforce Center put in place a notification system that customers can use to discretely inform staff that they need additional assistance or have a request.

18

  

Another important consideration for the evaluation is the definition of “exit” from One-Stop Center services, as well as the process by which an individual exits. Variations are important to take into account in the evaluation for two reasons: (a) because the definition of those who leave services is integral to determine the denominator for outcome measures; and (b) because of the need to maximize the consistency with which grantees collect follow-up measures. In some states (for example, New York, Kansas, Maine, and Alaska) customers have “soft” exits if they go 90 days without service. The exit date is the last date of service received. In Illinois, exits fall into three categories: positive (90 days of employment, training completion, employment); neutral (hospitalization, death, incarceration); and negative (no contact for 90 days, dropout). In some states, such as Kansas and New York, staff identify the customers who are approaching 90 days post-service and—depending on customer flow in the Center and staff availability—staff follows up by phone and/or letter to determine if the customer wishes to engage in additional services.

Delaware state-level personnel raised an important issue related to the youth population being served by the DEI grant. Their concern is the loss of students following graduation from high school despite the VR involvement in the high schools. Since many youth do not conceptualize themselves as having a disability (self-disclosure is an issue), there is often a break in services due to the student “dropping off the grid” post-graduation. Finding and tracking students, especially in a state as small as Delaware, can be difficult due to migration across state lines. We will work with Delaware to capture this phenomenon as accurately as possible.

4.5 Disability Resource Coordinator (DRC) Roles and Organizational Structure

In addition to appreciating the differences in One-Stop Center service processes between and within states, it is necessary to recognize the variety of DRC structures in order to better understand the context of the DEI grant. Beyond basic requirements that DRCs facilitate TTW and partnerships, states’ definition of the DRC responsibilities, assignment of this position, and determination of stakeholders served varies considerably, as illustrated in Exhibit 4.

Exhibit 4: State DRC Structure # of Key Elements of NotesDRCs Region(s) Served DRC Employer DRC Job Description

Alaska (15) Statewide

Alaska Department of Labor & Workforce Development

In-house (Center) focus: work with PWD & staff

Position will rotate among many Center staff approx. every 6 months

West Central

Arkansas Department of Workforce Services

Develop IRTs; leverage funds; use Guideposts for Success; provide access to customized employment; conduct outreach to disability community & service providers; assist PWD to access services; promote interagency collaboration; present info to the public; serve as point of expertise on programs & services that impact PWD employment employability; serve as resource to

Arkansas (4) Northwestern

Eastern

19

  

# of Key Elements of NotesDRCs Region(s) Served DRC Employer DRC Job Description

Southeastern Center staff & businesses; build capacity of Centers to serve PWD effectively

Delaware (4)

Areas of Operation (AORs) connected to One-Stop Centers

Vocational Rehabilitation

Work specified hours in the Center; conduct community outreach at schools, correctional facilities, & other youth-concentrated institutions.

DEI project also includes 2 part-time employment specialists

LA (Local Area) 4 South Central: Kingman, Sedgwick, Butler, Harper, Sumner, & Cowley Counties

Kansas Department of Commerce (Employment Service)

Direct customer IRTs; conduct outreach to disability community, disability service organizations, & employers; assist PWD in navigating the public workforce system; train center staff on disability issues & resources; work with LWIB on DEI sustainability plan; expand the use of the TTW program in assigned One-Stops.

The LA 5 DPN will transition into the DRC position.

The LA 4 DRC had not been hired at time of the Getting to Know You Visit (April 2011) due to a hiring freeze.

Kansas (2)

LA (Local Area) 5 Southeast: Lyon, Bourbon, Greenwood, Coffey, Miami, Anderson, Linn, Woodson, Allen, Elk, Wilson, Neosho, Crawford, Chautauqua, Montgomery, Labette, & Cherokee Counties

Illinois (2)

LWIA 8 Northern Cook County

Illinois Department of Commerce and Economic Opportunity (DCEO) (Employment Service)

Establish, coordinate, & maintain one or more IRTs; create customized employment opportunities; leverage resources & partnerships; facilitate access to resources, supports, & services that provide transition to employment. Disabilityworks (DCEO) centrally coordinates the DRCs. DRCs do not manage a caseload now, but this is subject to change.

The DRC position has been in existence since disabilityworks absorbed the DPN program in 2007. DRCs will continue to serve all 26 local areas in IL, but the 2 DEI pilot DRCs will focus their efforts in their assigned LAs.

LWIA 15 Woodford, Peoria, Stark, & Marshall Counties

Maine (2)

Region 1 (Northern Maine) Aroostook & Washington Counties

Northern Maine Development Corporation

Conduct business/global education & outreach; speak, train, & educate. Position designed to complement new VR Resident Counselor 1 position.

Region 1 DRC will be located at the Bangor One-Stop.

Region 2 (Tri-County) Piscataquis, Penobscot, & Hancock Counties

Eastern Maine Development Corporation

Region 2 DRC will travel between One-Stops

New Jersey (3)

Bergen, Burlington, Passaic, Cumberland, & Salem Counties

Educate staff in disability-related service delivery, disability sensitivity, & assistive technology; serve as the conduit to business for hiring PWD. Each local area will determine a county-specific protocol for how to use the DRC.

Burlington County’s DPN became the county’s DRC. The Cumberland One-Stop Center Operator will be the Cumberland-Salem Counties’ DRC. The Passaic & Bergen County DRC had not been hired at time of the Getting to Know You Visit (July 2011).

20

  

                                                            

# of Key Elements of NotesDRCs Region(s) Served DRC Employer DRC Job Description Broome Tioga Counties Broome County (?)

Responsible for staff capacity building internal to the One-Stop Center & LWIA but also in the community; case management as needed; community outreach, increasing employer & business knowledge; providing training & information systems about disability employment to staff in various state agencies & other areas associated with their focused population; focus on TTW; participate in IRTs

DRC role involves more capacity-building than DPN.

Capital Region Albany, Rensselaer, & Schenectady Counties

Rehabilitation Initiatives

Chautauqua County Career Systems Development

Chenango, Delaware, & Otsego Counties

Otsego County, Chenango-Delaware-Otsego WIB

New York (22)10

GLOW Counties: Genesee, Livingston, Orleans, & Wyoming Counties

Livingston County, Wyoming County Community Action

Niagara County Niagara County

North Country Region: Clinton, Essex, & Franklin Counties

Adirondack Community Action Program, Clinton County

Onondaga County CNY Works

Orange County Orange County Oswego County Oswego County Tompkins County Tompkins County Ulster County Ulster County Westchester County City of Yonkers Piedmont Workforce Network (LWIA 6): Albemarle, Culpeper, Fauquier, Fluvana, Greene, Louisa, Madison, Nelson, Orange & Rappahannock Counties; Charlottesville (city)

Thomas Jefferson Partnership for Economic Development

Virginia (5)

South Central LWIB (LWIA 8): Brunswick, Halifax, Mecklenburg, Amelia, Buckingham, Charlotte, Cumberland, Lunenburg, Nottoway & Prince Edwards Counties

LWIB DRC started June 1 (after site visit)

Arlington/Alexandria LWIB (LWIA 12): Arlington County; Alexandria (city)

Arlington County/City of Alexandria Department of Community & Human Services

Greater Peninsula Council for Workforce Development Peninsula Worklink

10 Evaluation team visited 2 of 13 demonstration sites.

21

  

# of Key Elements of NotesDRCs Region(s) Served DRC Employer DRC Job Description (LWIA 14): Gloucester, James City, & York Counties; Hampton, Newport News, Poquoson, & Williamsburg (cities)

4.6 Partnerships, Collaboration, and Leveraging Resources

During the Getting to Know You Visits, all state-level staff, as well as staff working at One-Stop Career Centers, spoke to the evaluation team about partnerships between the Career Centers and other state and local agencies and non-governmental organizations. Several DEI grantees reported that they collaborated with other agencies within the workforce system. For example, in New York, the grantee received financial support from VR. In Delaware, the grantee reported strong and ongoing relationships with a comprehensive set of entities including ETA, the WIB, the Secretary of Labor, and Youth Rehabilitative Services. Across states, we often saw instances of workforce system partners colocated (on-site) at the One-Stop Center, or housed in nearby locations. Moreover, staff at the majority of One-Stop Centers in Virginia told the evaluation team that customers receive VR services on-site. Conversely, in a number of One-Stop Centers in both Maine and New York, VR staff is only at the One-Stop Career Centers once per week or per month.

Staff at all of the One-Stop Career Centers visited reported working with partners that provide education, health care, veterans’ services, family and child welfare assistance, and youth services. Alaska was just one of the grantees where staff from the One-Stop Centers mentioned working with a wide range of partners that included school districts, native organizations, mental health agencies, youth development services, programs for immigrant, vocational, and technical education providers, veterans’ services, TANF programs, women’s shelters, and other public assistance providers.

In addition to government and non-governmental service providers, staff at both DEI and comparison sites across states described collaboration and outreach with employers. Staff in Maine noted that they had made outreach to employers a key feature of their implementation of DEI. In Alaska, the local Home Builders Association initiated a construction academy at a One-Stop Center. Staff at One-Stop Career Centers in several states also mentioned inviting employers to recruit applicants and conduct interviews on-site, partnering with staffing agencies and employing staff members specifically devoted to business and employer outreach.

During the Getting to Know You Visits, DEI grantees and One-Stop Career Center staff discussed leveraging financial resources as part of their working relationship with various partners. State-level staff in multiple states mentioned using funds from the Medicaid Infrastructure Grant (MIG) to support DEI. Delaware is working with a supported education program. At the One-Stop Career Center level, staff in one Illinois One-Stop Center said that they use a “network system,” whereby they can identify which One-Stop Centers have available funds in order to refer customers if their own One-Stop Center does not or cannot provide needed WIA services (beyond universal services provided to all core customers). A One-Stop Center in

22

  

Virginia also mentioned receiving financial support from the local community college for a classroom on-site that is available for core and intensive services.

There are numerous examples of partnerships with organizations that offer services and supports specifically to individuals with disabilities. Some of these are common across sites within a state while others are unique to local One-Stop Centers. New York offered several examples of both. New York Makes Work Pay is a partnership between New York State Office of Mental Health and Syracuse and Cornell Universities. Funded through a MIG, the partnership is working on asset development, collaboration among agencies, and employer assistance in hiring and retaining workers with disabilities. Real JOBS is a key partner of the Syracuse, New York DEI site. Funded by a Projects with Industry grant from the Rehabilitation Services Administration, Real JOBS is a collaboration of the Workforce Development Institute, Burton Blatt Institute at Syracuse University, Schenectady County Community College, Adirondack Community College, and the New York Association of Psychiatric Rehabilitation Services. The program assists and supports individuals with psychiatric disabilities in securing and maintaining employment. Independent Living Centers (ILCs) offer both employment and non-employment services and supports to individuals with disabilities. In New York, the Utica, Niagara, Monroe, and Buffalo sites all named ILCs as important partners.

Alaska also offered an example of a partnership, with the Alaska Mental Health Trust Authority (a unique funding source originating from a settlement against the state for not providing mental health services). The Mental Health Trust Authority funds initiatives such as the “You Know Me” disability awareness campaign, as well as supported employment services.

According to staff in several states, such as Arkansas and Maine, a disability-related partner that may be particularly important to DEI is the Social Security Administration, not only because of the criticality of Ticket to Work to the initiative, but because of its Work Incentive Planning and Assistance (WIPA) projects, which assist SSA beneficiaries with disabilities in seeking and maintaining employment and taking advantage of SSA work incentives. Each of the more than 100 WIPA projects across the U.S. employs a Community Work Incentives Coordinator (CWIC), who, working with government agencies, service providers, and employers, provides benefits planning and assistance services that include: conducting outreach to individuals and groups—including employers and providers of employment services—who need information about TTW and SSA work incentives; providing information and resource referrals regarding employer and other health benefits coverage that may be available to a beneficiary, such as Medicare or Medicaid; providing information on accessing protection and advocacy services for beneficiaries with disabilities; and referring beneficiaries with disabilities to appropriate ENs.

4.7 Issues Specific to Working with Customers with Disabilities

4.7.1 Disclosing Disabilities

Across all states, the individuals we spoke to reported that many PWD do not disclose their disabilities. One-Stop Center staff and management offered estimates of disclosure rates that ranged from “less than half” to “one in ten.” Staff across states agreed that individuals with non-visible disabilities, especially those with mental health/substance abuse disabilities, are the least

23

  

                                                            

likely to disclose. Staff at the sites reported that customers are more likely to disclose their disabilities after they have had time to develop a rapport with their customers. Physical space for private conversations was also an important factor, as well as ensuring that customers understand that they may be eligible for additional services if they have a disability. Given this information, ensuring that either customers or staff members have a mechanism to change an individual’s disability status in the WIB’s data system later in the service process (for customers who disclose only after a rapport has been established with staff) is critical to achieving accurate data.

For youth with disabilities, fear of stigma or the desire not to be “labeled” are particularly strong challenges to disclosure, given the desire among teens not to be perceived as “different.” For this and other reasons, many disabilities among youth, especially mental health and learning disabilities, may go undiagnosed. Youth are not always able to obtain documentation (such as an Individual Education Plan) for their disabilities, particularly if they have relocated since leaving school or have been out of school for several years. Assessments can be costly and schools may not immediately have the resources to do update assessments for youth who lack documentation. In light of the difficulty of obtaining disability documentation, staff may not record disabilities for youth, particularly if the customer already qualifies for WIA youth services based on income alone.

How One-Stop Centers approach asking customers about disabilities during the registration process varies considerably from state to state, and sometimes within the state.11 Privacy (and perceived privacy) is also a key factor in the disclosure issue. In Alaska, for example, most customers register for job search online on the state’s Labor Exchange website. Because the Labor Exchange site also acts as a job-matching system that employers may access, staff at both the state and One-Stop Center levels reported that customers were concerned that potential employers could see all of their information, including disability information. The evaluation team noted that the system provided little or no clear indication of which types of information were accessible to employers and which portions were not. The state’s workforce development staff was very receptive of this information and is exploring ways to add clarification so that customers understand that their disability information is private. The differences in state questions about disability will lead to variability in the accuracy of our reports on the involvement of PWD in the DEI.

4.7.2 WIA Performance Measures

One key concern about serving PWD is whether they are likely to achieve the same level of success as other customers, or whether serving individuals with disabilities might reflect poorly in the WIA performance measures (measures to which One-Stop Centers are accountable, such as amount of programming received by customers and subsequent employment outcomes). A director at one One-Stop Center told the evaluation team that she tends to keep customers out of the WIA data system unless they seem likely to succeed in order to maintain the One-Stop Center’s performance measures. In another One-Stop Center, PWD served in the TTW program are simply not registered as WIA customers. By contrast, one LWIB described itself as being “sanctioned for serving too many customers with disabilities,” explaining that its low

11 In Chicago, staff reported that it was their understanding that asking an individual whether s/he has a disability in the context of delivering employment services is a violation of the Americans with Disabilities Act (ADA).

24

  

                                                            

employment outcomes/performance measures were a consequence of working with a large population of PWD, who are less likely to find employment than other customers. While these phenomena are not unique to serving PWD, some staff members expressed concern about the impact that the DEI focus on PWD would have on their WIA performance measures.

The perception that PWD will negatively affect a WIB’s performance measure is perhaps even more critical in WIA youth programs, since these services incorporate a significant academic component, including testing. The common measures for youth programs reflect this emphasis on academic performance and include a measure of “the percentage of basic skills-deficient12 out-of-school youth who have increased their basic skills proficiency by one or more levels of educational functioning13 within one year of the date of program participation.” Youth programs assess participants’ math and reading (“basic skills”) at program entry and periodically throughout customers’ participation so that the program can report on this measure. Earlier research reported that programs do not enroll youth who score below the fifth or sixth grade level on assessments because such youth do not have a realistic chance of achieving the program’s objectives within 18 months.14

During the site visits of spring 2012 and 2013, the DEI evaluation team will learn more about the ways in which PWD are served in One Stop Career Centers.

4.7.3 Cultural Differences in Definitions and Understanding of Disability

States serving indigenous peoples and large numbers of immigrants may run into various cultural differences in how PWD and their families understand the concept of disability. Many languages have no word for “disability.” Many cultures consider disability to be shameful or a punishment for past sins. In other cultures, the extended family takes responsibility for relatives with disabilities and understands neither the concept of disability services nor the civil rights extended to PWD by the ADA. Depending on the disability expertise of DEI staff and the extent to which One-Stop Centers increase the number of PWD served, these cultural differences may become more apparent over time. In Alaska, for example, even in One-Stop Centers serving a significant number of Alaskan natives, staff was unaware of any cultural differences. In some parts of Kansas, Native Americans are not yet well represented on the LWIBs, although the American Indian Council is on the Workforce Alliance of South Central Kansas (Wichita) Board and the CP Foundation is partnering with, and providing services on, a reservation that is part of a comparison site.

12“Basic skills deficient” is defined as an “ individual [who] computes or solves problems, reads, writes, or speaks English at or below the eighth grade level or is unable to compute or solve problems, read, write, or speak English at

a level necessary to function on the job, in the individual’s family, or in society.”U.S. Department of Labor, Employment and Training Administration. (2005). Training and Employment Guidance Letter (TEGL) 28-04,

Common Measures Policy, Attachment C, Definitions of Key Terms. Washington, DC: USDOL/ETA. 13Each educational level is roughly equivalent to two grades in school. Ibid.

14Lorentzen & Almandsmith. (2010, March). Accommodations and Alternate Assessments in Basic Skills

Assessments for Youth with Disabilities in the Workforce Investment System: Briefing Paper. Berkeley Policy

Associates.

25

  

4.8 Initial Steps Toward Systems Change

DEI is a systems change initiative, the premise of which is to use proven systems-change strategies (the seven strategies referenced in the Introduction) to make lasting, impactful change in the employment outcomes of PWD. The DOL has defined systems change as it applies to the public workforce development system as encompassing nine elements: capacity, coordination/integration, partnerships, co-enrollment, customer choice, employer partnerships, development or adaptations of existing practices, dissemination of effective practices, and sustainability (Riley, Kruger, Frey, & Elinson, 2001; Klayman, McGill, Skaff, Stapleton, & Wozny, 2010). Systems-level change typically takes a long time to evolve. For this reason, we expect to see the emergence of DEI systems change towards the end of the DEI grant period.

Systems change will look different in each grantee state. Each has a unique culture, unique strengths and challenges, and has chosen unique approaches to improve its workforce system. Even though our Spring 2011 Getting to Know You Visits were informal, the information culled during those visits will be invaluable in helping us identify and recognize emerging systems change in the future. Already we have seen indications of systems change, or the readiness for it. Indications of systems change include Alaska’s DRC position, which has been developed into a system-wide professional development thrust. The DRC position will be rotated, approximately every six months, augmenting the knowledge and skills of multiple existing staff throughout the One-Stop Career Center system. This builds capacity and knowledge at the institutional (i.e., One-Stop Center) level. In Maine, VR was written into the DEI proposal, ensuring coordination and integration. This One-Stop Center partner will not only be informed about change efforts, but will participate in those efforts. Virginia offers another glimpse of systems change. DEI and its associated data collection for the evaluation are being leveraged to encourage One-Stop Center staff to adopt the common registration form, developed for use across multiple agencies in the Virginia workforce systems. Adapting multiple practices will allow all LWIAs participating in the grant to use a consistent enrollment form. This will ease crossover of information, staff, and customers between LWIAs.

The readiness for systems change is clear, too. In several locations DEI was referred to by LWIB or One-Stop Center staff as “the DRC grant” or “the DEI program.” It was clear in these instances that staff misperceived DEI as a stand-alone “program” that was implemented by a particular person. There was also sometimes the misperception that DEI was a funding source to maintain a DPN who would now be called a DRC. While we could not recall hearing these misperceptions corrected by anyone other than our own evaluation team members, we were able to be part of the audience when State Leads introduced the initiative for the first time to staff and/or partners in Alaska, Arkansas, and New Jersey. In these arenas, DEI State Leads were consistent in explaining DEI as a broad (system-wide), more lasting initiative. It is important that all workforce system staff are “on the same page” in understanding what they are doing, and why. However, it will take more time and effort before there is a common understanding of the concept of systems change.

26

  

-

5.0 The DEI Data System

5.1 Purpose of the DEI Data System

The DEI Data System is the customer data collection and management system for DEI. It will warehouse all WIASRD, Wagner-Peyser, wage, and DEI data. Additionally, it will provide grantees a web interface for entering DEI customer-level information not already captured by grantees’ existing case management or registration systems.

5.2 Design of the DEI Data System

In order to accommodate the multi-site design of DEI, the complexity of the One-Stop Career Center system, and the need for a systematic approach for collecting and warehousing data, the DEI Data System will be integrated into the existing data collection processes (i.e., customer/staff interaction) at all participating One-Stop Centers. These processes are unique to each state, and often from One-Stop Center to One-Stop Center. Altarum Institute is leading the design and implementation of the DEI Data System on the Altarum Secure Network (ASN). Data security is of utmost importance, particularly when system data contains personally identifiable (PI) information. The ASN has been assessed to be within 98 percent compliant with National Institute for Standards and Technology Special Publication 800-53 Revision 2 guidelines, and has been qualified for storage of such data. The ASN-supported DEI Data System provides the administrative processes to (a) customize the web interface of the DEI Data System for each grantee; (b) define users (customer, case worker/DRC, evaluation team member/administrator) and specify their system capabilities; (c) log all user activity; (d) store, protect, and access PI information in a controlled manner that complies with all privacy act and HIPPA regulations; and (e) make modifications to the system as needed.

User account and access credentials for the ASN will be managed by Altarum Institute. States that elect to use the web interface of the DEI Data System will receive user accounts and access credentials for caseworkers and DRCs in pilot and comparison sites. Customer access to the system will not require user accounts and access credentials.

User Type Username

and Password Required

Can View Customer

Level Data (PI data removed)

X X X X X

X X

Exhibit 5: DEI Data System User Type and Associated System Capabilities Can Add Data to Can View

Can Existing Record Aggregate State-Create (unable to view Level Frequency Record previously entered Reports

data) (PI data removed) Customer Case Worker/DRC

Select DEI Evaluation XTeam/Administrator

Grantees are required to use secure file transfer protocols for quarterly uploads of customer data to the DEI Data System. Rules and regulations concerning the confidentiality of the quarterly upload data are outlined in Attachment 5. A quarterly data reporting schedule will support a rigorous analysis of program impact by ensuring that enough data points are available to detect

27

  

change in customer-level outcomes. It will also allow the evaluation team to monitor the submission of data to ensure that the incoming data is of appropriate quality and includes all required baseline and follow-up/exit data elements. This also enables the evaluation team to provide timely feedback to states to maximize data quality. A quarterly schedule is also intended to streamline reporting requirements for states, as it is synchronized with the existing federal reporting schedule for WIA. Customer data entered via the web interface of the DEI Data System will be linked to the existing WIASRD, WP, and state wage data in the system.

5.2.1 Participatory Work with Grantees

Due to the complexity of the One-Stop Career Center system, it was essential for the design of the DEI Data System to be a collaborative effort between the evaluation team, DOL, and DEI grantees. Evaluators depended on the grantees to describe the characteristics of their customer data collection/case management systems. Grantees provided aids to facilitate the evaluators’ understanding of their existing system(s) and data collection procedures, including electronic copies of their customer registration forms and operating system data dictionaries. The collective exchanges of information through DEI Year One facilitated the completion of a state-by-state data element crosswalk between the required elements of the DEI Data System and the existing data elements in the grantee’s customer data collection and management systems. This process allowed the evaluators to customize the web interface of the DEI Data System for each grantee, asking only for those data elements not present in the existing systems or included in the quarterly WIASRD and WP data.

The DEI Data System design has been reviewed extensively by DOL and the DEI grantees. A pilot test was conducted with nine respondents in January 2011 to determine respondent burden. Beta testing of the DEI Data System prototype is complete and a final pilot test will be conducted in September 2011 to assess system usability. Tests will focus on time to complete data entry, comprehension of system instructions, ease of use, and human-computer interaction. The evaluation team will provide technical assistance to all grantees throughout the DEI evaluation period to ensure that users of the system, regardless of the mode of data collection, complete data collection in a timely and accurate manner. These technical assistance activities also will include a DEI Evaluation Manual, a toll-free helpline, and quarterly monitoring of incoming data for data quality and completion. Additionally, evaluators will continue to encourage an open line of communication with state-level technical/data and analysis personnel to identify and problem solve global issues that may arise with the ftp protocol, grantee’s data system, and the DEI Data System.