DOCUMENT RESUME

ED 424 432 CE 077 414

AUTHOR Sperazi, Laura; DePascale, Charles A.TITLE Evaluation Report for the Massachusetts Workplace Literacy

Consortium. National Workplace Literacy Program Wave 6, Year3

INSTITUTION Massachusetts Workplace Literacy Consortium, Boston.SPONS AGENCY Office of Vocational and Adult Education (ED), Washington,

DC. National Workplace Literacy Program.PUB DATE 1997-10-00NOTE 66p.; For a related report, see CE 077 413.CONTRACT V198A40054-97PUB TYPE Reports Evaluative (142)EDRS PRICE MF01/PC03 Plus Postage.DESCRIPTORS Adult Education; *Consortia; *Partnerships in Education;

Program Effectiveness; Program Evaluation; State Programs;*Teamwork; *Workplace Literacy

IDENTIFIERS *Massachusetts; Quality Indicators

ABSTRACTThe Massachusetts Workplace Literacy Consortium sought to

upgrade work-related literacy skills at 22 partner sites in the state.Members included manufacturers, health care organizations, educationalinstitutions, and labor unions. In its third year, the consortium served1,179 workers with classes in English for speakers of other languages, adultsecondary education/General Educational Development, and adult basiceducation. Findings of an evaluation of the third year included thefollowing: (1) consortium members felt the Planning and Evaluation Team (PET)model provided many benefits, such as cost effectiveness, resource sharing,and communication; (2) confusion over roles of the state department ofeducation and program coordinators hindered the first 2 years; (3) PET wasconsidered critical to program success because it kept stakeholders informed,enabled open discussion, and promoted stakeholder buy-in; (4) obstacles tostudent participation in PET were difficulties with English, programcompletion, and a rotation plan that contributed to high turnover; (5)

underrepresentation of important stakeholders was a serious threat to PET;and (6) PET was considered helpful but not sufficient to ensure that theprogram was institutionalized at all sites. (Appendices contain thefollowing: program standards and quality indicators; evaluation interviewquestions; list of PET site visits; PET participation chart;_and PET surveyand meeting observation forms.) (SK)

********************************************************************************

Reproductions supplied by EDRS are the best that can be madefrom the original document.

********************************************************************************

Evaluation Report

for the

Massachusetts Workplace Literacy Consortium

National Workplace Literacy ProgramWave 6, Year 3

Award #V198A40054 - 1997

Submitted to theMassachusetts Department of Education

by

Laura Sperazi, Evaluation Research_ Charles A. DePascale, Data Analysis & Testing Associates. Inc.

October, 1997

BEST COPY AVAIL ABLE

T`\

)4L.

U.S. DEPARTMENT OF EDUCATIONOffice of Educational Research and Improvement

EDUCATIONAL RESOURCES INFORMATIONCENTER (ERIC)

is document has been reproduced asreceived from the person or organizationoriginating it.

1:1 Minor changes have been made toimprove reproduction quality.

° Points of view or opinions stated in thisdocument do not necessarily representofficial OERI position or policy.

Acknowledgements

Many people helped to bring this evaluation report into being over the course of the lasttwelve months. Thanks to all participants in the Massachusetts Workplace LiteracyConsortium who enhanced this effort by participating in interviews, completing surveys,or contributing to the materials reviewed as part of this evaluation. Particular thanks goto

Program Coordinators from the seven education providers who participated ininterviews, provided background and historical information about their programs,and/or coordinated our visits to planning and evaluation team meetings.

Planning and Evaluation Team members including student workers, teachers, andrepresentatives from business and labor unions who took the time to meet with us,participate in the group interview, and completed and returned the individual surveys.

Massachusetts Workplace Literacy Consortium Year 3 Evaluation, October, 1997 Page ii

3

Table of Contents

List of Tables and Figures iv

Executive Summary

I. Introduction 1

Indicators of Quality for Workplace Education Programs 4

III. Consortium Planning and Evaluation Team 8

IV. Planning and Evaluation Teams 1 3

Appendix A: Indicators of Quality

Appendix B: CPET Interview Protocol

Appendix C: Site for PET Meeting Visits

Appendix D: PET Visit FormsParticipation ChartGroup Interview ProtocolIndividual SurveyPET Observation Form

4

Massachusetts Workplace Literacy Consortium Year 3 Evaluation, October, 1997 Page iii

List of Tables and Figua es

Table 1. Year 3 Massachusetts Workplace Literacy Consortium Pat ticipants 1

Table 2. Number of students enrolled by program in Period 5 and Period 6 2

Table 3. Summary of enrollment and course offerings for Period 5 and Period 6 2

Table 4. Advantages of the Consortium Model 11

Table 5. Characteristics of Sample Sites 14

Figure 1. Respondents to individual survey by position 16

Figure 2. Efforts taken to promote student participation in the PET 19

Figure 3. Benefits of the PET Model 70

5

Massachusetts Workplace Literacy Consortium Year 3 Evaluation, October, 1997 Page iv

Executive Summary

The Massachusetts Workplace Literacy Consortium (MWLC), Wave 6 was funded forthree years from November 1, 1994 to October 31, 1997 through the: National WorkplaceLiteracy Program (NWLP). The MWLC was administered through the Adult andCommunity Learning Services Cluster of the Massachusetts Department of Education andseven educational providers. Members of the Consortium included manufacturingcompanies, health care organizations, educational institutions, and Labor unions. In itsthird year, the MWLC provided workplace education services to up,grade work-relatedliteracy skills at twenty-two partner sites throughout the Commonw.:.alth ofMassachusetts.

During the third year of the project, the MWLC administered 69 classes serving 572student workers in Period 5 and 63 classes serving 602 students in Period 6. The primarytype of content taught was ESOL, followed by ASE/GED and ABE

The focus of the Year 3 evaluation was on a key aspect of the origir.al MWLC concept:the planning and evaluation team model. In particular, the focus of the current evaluationwas on the organization and operation of planning and evaluation te ams at the consortiumand partnership site levels. An additional piece of the Year 3 evaluation was a summaryof work on a key component of the project: the Indicators of Quality for WorkplaceLiteracy Programs. Information about students served in the MWL 2 is presented in theMWLC: Annual Performance Report and MWLC: Semi-annual Pet:Formance Report.

Highlights of Evaluation Questions and Findings

Consortium Planning and Evaluation Team

MWLC members felt that they were part of a team working tow ard a single goal: theimprovement of workplace literacy programs in Massachusetts! They also indicatedthat the MWLC model provided many benefits over working alone.

The major advantages of the MWLC model according to project coordinators were a)the cost effectiveness of the model, b) the ability to share resouices and draw on theresources of the state and other programs, and c) the opportunit es for communicationwith colleagues.

Coordinators did not identify any disadvantages of the MWLC model.

At the beginning of the project there was not a clear understanding of the roles of theDepartment of Education, program coordinators, or the CPET i :self in governing theprogram. This resulted in the CPET consuming a significant al nount of human andtime resources over the first two years of the program in defining and accepting theirroles.

6Massachusetts Workplace Literacy Consortium Year 3 Evaluation, Octobe 1997 Page v

Planning and Evaluation Teams

There was no evidence of explicit or implicit barriers to the participation of any

members of the PET with the possible exception of student workers. Respondentsindicated that all other PET members were able to fully participate in discussions and

decision-making.

Members felt that the PET was critical to the success of a workplace literacy programbecause the PET a) provides the opportunity to keep all stakeholders informed on a

regular basis, b) presents a forum for open discussion, c) strengthens the partnershipand team approach within the organization, and d) promotes buy-in from all

stakeholders.

Attendance records indicated that most current PET members attend nine or ten

meetings per year.

The greatest obstacles to student participation in the PET were difficulties inexpressing themselves in English and a high rate of turnover on the PET due to a)students' completing the program, or b) a planned system of rotating students throug,h

the PET.

Under-representation of important stakeholders such as company management,supervisory staff, or students was seen as the most serious threat to the PET.

A properly functioning PET was identified as necessary but not sufficient to ensurethe success and institutionalization of a workplace literacy program. The decision tocontinue the workplace literacy program is often based on factors other than the

success of the program.

Quality Indicators

Institutionalization is often not a quality issue. Initial decisions whether toinstitutionalize a program are often based on whether there is money available in the

budget to continue the program:

There are many factors which affect the quality of a program over which the

education provider has little, if any, control. It is necessary to define standards for allstakeholders that address the expectations of the entire program.

7Massachusetts Workplace Literacy Consortium Year 3 Evaluation, October, 1997 Page vi

Section I: Introduction

Section I: Introduction

Background

The Massachusetts Workplace Literacy Consortium (MWLC) was funded for three yearsfrom November 1, 1994 to October 31, 1997 through the National Workplace LiteracyProgram (NWI.,P). The MWLC was administered through the Adult and CommunityLearning Services Cluster of the Massachusetts Department of Education (DOE) andseven education providers representing business, labor, and education. Its membersincluded manufacturing companies, health care organizations, educational institutions,and labor unions. In its third year, the MWLC provided workplace education services toupgrade work-related literacy skills at 22 partner sites throughout the Commonwealth ofMassachusetts.

Table 1: Year 3 Massachusetts Workplace Literacy Consortium ParticipantsEducation Partners Business/Organization Partners (Location)

Bristol Community College Helix Technology (Mansfield)Jostens, Inc. (Attleboro)Mason Box (North Attleboro)Robbins Company (Attleboro)

Jewish Vocational Services Beth Israel Deaconess/Children's Hospital (Boston)Carter Fuller Mental Health Center and AFSCME (Boston)C&K Components (Watertown)Massachusetts General Hospital (Boston)Servo lift/Eastern Corporation (Dorchester)

Labor Education Center at theUniversity of Massachusetts,Dartmouth

Lightolier Corporation (Fall River)

Labor Management WorkplaceEducation Program at theUniversity of Massachusetts,Amherst

Smith and Wesson Corporation (Springfield)University of Massachusetts (Amherst)

Literacy Volunteers Network Holyoke Card and Paper (Springfield)Sealed Air Corporation (Holyoke)

Quinsieamond CommunityCollege

Beaumont at the Willows Nursing and Rehabilitation Center(Westboro)Jewish Healthcare Services (Worcester)

Service Employees InternationalUnion/Worker Education Program

Boston University Medical Center (Boston)Harvard Street Neighborhood Health Clinic (Dorchester)Jewish Memorial Hospital (Boston)Metrowest Medical Center (Framingham)St. John of God Hospital (Bri2hton) .

Women's Educational Industrial Union (Boston)

The governance structure of the MWLC is based on the Planning and Evaluation TeamModel. The Consortium Planning and Evaluation Team (CPET) was comprised of theprogram coordinators at each of the seven education provider partners and representativesfrom the DOE. The CPET met monthly to oversee all MWLC activities, identify needsamong its membership, ensure delivery of appropriate training support and technicalassistance, and review and address general issues related to providing education services

8Massachusetts Workplace Literacy Consortium Year 3 Evaluation, October, 1997 Page 1

Section I: Introduction

in the workplace. At the site level, the workplace education program at each site was

organized around a local Planning and Evaluation Team (PET). The local PET wascomprised of representatives from company management, supervisory staff,student/workers, labor unions, and the education provider. The local PETs met regularly

to oversee the workplace education program at the local level.

The structure of the MWLC, its educational scope, and its capacity to include large andsmall businesses from diverse sectors of the economy developed out of nine years ofexperience and learning within the DOE about how to best provide workplace literacyservices to employed workers. It is expected that the lessons learned from the MWLC

will provide a strong foundation as workplace initiatives within the Commonwealth moveforward into their next decade.

Year 3 Services

During the third year of the project, the MWLC administered workplace literacy classes

at twenty-two sites throughout Massachusetts. During Period 5, 572 workers wereenrolled in 69 classes. During Period 6, 602 students were enrolled in 63 classes. In each

period, more than half of the workers were enrolled in ESOL programs. A breakdown ofstudents by program is presented in Table 2.

Table 2: Number of Students Enrolled by Program in Period 5 and Period6Period 5 Period 6

ESOL 389 (68%) 326 (54.2%)

ASE/GED 100 (17.5%) 188 (31.2%)

ABE 83 (14.5%) 80 (13.2%)

Other/Missing Data 0 (0%) 8 (1.3%)

TOTAL 572 602

Given the wide variety of partner sites in the MWLC, there was a wide range in thenumber of classes offered across sites as well as the number of students enrolled in each

class. Table 3 provides summary descriptive statistics on the enrollment figures and

course offerins for Periods.5 and 6.

Table 3: Summary of Enrollment and Course Offerings for Period 5 and Period 6average (minimum - maximum) Period 5 Period 6

Number of classes per siteAverageMedianRange

3.59

1 12

9.99

1 13

Number of students per classAverageMedianRange

8.39

1 19

9.58

1 33

Massachusetts Workplace Literacy Consortium Year 3 Evaluation, October, 1997 Page 2

Section I: Introduction

Year 3 Evaluation

The National Workplace Literacy Program requires that an independent externalevaluation be conducted of each of its projects. In Year 2, the external evaluation wasstructured by four main objectives related to the goals and objectives of the MWLC:

I. Determine if the goals of the MWLC are being met.2. Determine the level of implementation of the Massachusetts Indicators of

Quality for Workplace Literacy Programs and the relationship of the Indicatorsto learner outcomes, workplace outcomes and the program partnership.

3. Determine the relationship between instructional methodologies andworker and workplace outcomes.

4. Develop recommendations for project improvement.

Based on the results of the Year 2 evaluation and previous evaluations it was determinedthat the Year 3 evaluation would focus on a key aspect of the original MWLC concept:the planning and evaluation team model. In particular, the focus of the current evaluationis on the organization and operation of planning and evaluation teams at the MWLC andpartnership site levels. In addition, a final piece of the Year 3 evaluation will be asummary of the history of the Indicators of Quality for Workplace Literacy Programs.

Massachusetts Workplace Literacy Consortium Year 3 Evaluation, October, 1997 Page 3

Section II: Indicators of Quality for Workplace Education Programs

Section II: Indicators of Quality for Workplace Education Programs

Overview

A fundamental element of the Massachusetts Workplace Literacy Consortium has beenthe Indicators of Quality for Workplace Education Programs or Quality Indicators. TheQuality Indicators were established for six basic components of workplace educationprograms:

Partnership and PlanningCurriculumAssessment, Evaluation, and OutcomesSupport ServicesStaffAdministration.

The underlying assumption of the Quality Indicators is that they describe the conditionswhich are necessary for a program to deliver effective services. Work on the QualityIndicators predates the MWLC and will continue after the conclusion of the currentproject. In this section we present a summary of the Quality Indicator project to date andplans for the future.

Background

Initial Development

Drafts of Quality Indicator definitions were initially developed during previous fundingcycles of the Massachusetts Workplace Literacy Program. They were developed byeducators brought together through the Massachusetts Workplace Education Initiative.

In March, 1994, the MWLC's funding application for Wave VI of the NationalWorkplace Literacy Program outlined a central role for the Quality Indicators in theprogram evaluation. The MWLC evaluation provided the opportunity to test theassumption of the relationship-between the Quality Indicators and the -effectiveness-of aprogram. A key component of the evaluation was to examine whether the presence orabsence of the Indicators correlated with desired outcomes at three levels:

I. workers' educational gains:2. productivity gains or improvements in quality of services, and3. the quality of the business-union/education provider partnership.

11Massachusetts Workplace Literacy Consortium Year 3 Evaluation, October, 1997 Page 4

Section II: Indicators of Quality for Workplace Education Programs

Creating an Evaluation Tool

Before the Quality Indicators could be used in an evaluation, it was necessary to movethem from initial definitions to a valid and reliable measurement tool. The creation of aset of evaluation instruments was a major focus of the initial year of the MWLC.

During the first year of the program, the external evaluators designed and pilot-testedinstruments which fully operationalized the Quality Indicators. The evaluators assigned ascore to each indicator on a 6-point Likert-type scale on which the indicator was rated. Inmaking the ratings, evaluators and coordinators considered the dimensions of extent ofimplementation and quality of implementation. Evaluators developed instruments forboth external and self evaluations. The external ratings were made on the basis of acentral interview protocol called Indicators PLUS. In addition, shorter versions of theIndicators PLUS were designed to be completed by business partners, unionrepresentatives, teachers, and workers. Finally, a self-evaluation tool was developed forcoordinators to provide ratings of their own programs.

Evaluation Results

The evaluation conducted during the second year of the program showed that the sites andpartners have implemented the Quality Indicators at a very high level. The averageratings on 18 of the 25 indicators were above 5 on the 6-point scale where 6 was thehighest level of implementation. The strongest areas of implementation were Staff,Curriculum, and Administration; the weakest was the area of Assessment, Evaluation.and Outcomes. Although the overall level of implementation was quite high, analyses didreveal some differences that may be due to type of industry or education provider. Moreresearch on these factors should be conducted in the future.

The relationship between the implementation of the Quality Indicators and outcomes wasnot clear from the evaluation. In large part, this was due to the lack of good dataavailable on outcomes. Learner outcome data were available in the form of self-reportedlearning gains scores. Other outcome data was scant at best, a by-product of theunexpected dearth of NWLIS data. Also, the lack of data is consistent with the relativelylow level of success at implementing the Quality Indicators in the area of Assessment.Evaluation, and Outcomes.

The Quality Indicators have many benefits when used as a guide for programdevelopment. However, future research on relationships between implementation of theQuality Indicators and outcomes will be necessary to validate the Quality Indicators as anaccountability measure for workplace education programs. The resources and timeneeded to continue the proper study of these relationships, however, are quite extensive.

1 2

Massachusetts Workplace Literacy Consortium Year 3 Evaluation, October, 1997 Page 5

Section II: Indicators of Quality for Workplace Education Programs

Performance Standards

One method of improving the usefulness of the Quality Indicators is to link them toclearly defined performance standards. Performance standards can guide programs intheir development and also serve as benchmarks for developing a scoring system for theQuality Indicators. In the third year of the project, efforts were initiated to develop asystem of program performance standards based on the Quality Indicators.

Current Status

The context in which the Quality Indicators were developed was a spirit of collaborationbetween public and private sources to create model workplace literacy programs. Theemphasis was on education and the focus of the Quality Indicators was on excellence indesigning and delivering the ideal program.

The Quality Indicators continue to evolve as

a) the context expands from public/private collaboration and funding to privatefunding; and

b) participants have gained three years of experience in striving for theinstitutionalization of their programs

New issues are emerging and the applications for the indicators are expanding.

One outcome of the effort to institutionalize programs over the past year is the findingthat institutionalization is not necessarily a quality issue. In many cases, at least at thefirst cut, the decision whether to institutionalize is a straightforward economic issue: "Isthere money available in the budget to continue the progyam?"

A second finding based on experience is that there are many factors which affect thequality of a program over which the education provider has little, if any control. Creatingstandards to hold only education providers accountable to deliver an ideal program haslimited effectiveness. It is necessary to define standards for all stakeholders that addressthe expectations of the entire program. Such standards would address questions about_what is required of the company and what is required of the funding agency as well aswhat is required of the education provider.

The central questions that program staff had to answer were "What purpose can theQuality Indicators serve?" and "Who is their audience?-

The answer to the first question is that the Quality Indicators should be a resource for allstakeholders regardless of funding source, company size, type of business, or partnershipconfiguration. The challenge is to create a set of indicators that reflect universal qualityfactors that is flexible but still useful. As the final year of the current project was being

Massachusetts Workplace Literacy Consortium Year 3 Evaluation, October, 1997 Page 6

Section II: Indicators of Quality for Workplace Education Programs

completed, the quality indicators and supplemental documents were being revised tomeet this challenge.

The answer arrived at to the second question is that there are multiple audiences for theQuality Indicators. Employers and unions who will be making decisions about whether tobegin or continue a workplace education program are one type of audience. Agenciesmaking funding decisions are another audience. Finally, programs who wish to conduct aself-evaluation are another important audience.

As funding for the MWLC ends, the Quality Indicators and related documents have beenturned over to the Massachusetts Workplace Education Committee (MWEC) formerly theadvisory committee to the MWLC. They are reviewing the documents again and willcontinue with further research, development, and implementation. The latest draftversion of the workplace indicators/standards are included in Appendix A.

14

Massachusetts Workplace Literacy Consortium Year 3 Evaluation, October, 1997 Page 7

Section III: Consortium Planning and Evaluation Team

Section III: Consortium Planning and Evaluation Team

Overview

This portion of the evaluation focuses on the functioning of the Consortium Planning andEvaluation Team (CPET). Previous evaluations raised questions about the CPET as agoverning body, the role/authority of the program coordinators, and the role/authority ofthe DOE. Additional questions about the advantages and disadvantages of theconsortium model for administering workplace education programs are also addressed.

The information provided here is intended to supplement information obtained in priorevaluations. It is based primarily on interviews conducted with the seven programcoordinators.

Methodology

Interview0

The primary method for collecting additional information regarding the functioning of theCPET was an individual interview with each of the program coordinators from the seveneducation providers. Two formats were used to conduct the interviews. Fivecoordinators were interviewed as part of site visits to PET meetings. Two coordinatorswhose programs were not part of the PET site visit process were interviewed viatelephone. The interview was administered in approximately 30 minutes.

The loosely structured interview focused on areas related to the functioning of the CPET:

The role of the CPET and DOE in administering the MWLC:The roles of the DOE and program coordinators in the CPET: andThe sense of belonging to a consortium.

Two initial questions in each area were scripted. Follow-up questions were asked basedon responses to the initial questions. Additional questions were also included tosupplement the information gathered about the PETs. A copy of the interview protocol isprovided in Appendix B.

Observation

Observational information about the functioning of the CPET was collected by attendingthe June 3, 1997 CPET meeting in Springfield, Massachusetts. The evaluator paidparticular attention to

how the meeting was conductedinteractions between program coordinators and DOE personnelinteractions among program coofdinators.

Massachusetts Workplace Literacy Consortium Year 3 Evaluatin5October, 1997 Page 8

Section III: Consortium Planning and Evaluation Team

The meeting lasted approximately 4 hours. It was attended by coordinators from three ofthe seven programs, a representative from a fourth program, and three representatives ofthe department of education.

Archival Records

To provide background and supplementary information the evaluator reviewed archivalmaterials related to the functioning of the CPET. These materials included minutes ofCPET meetings, surveys administered to CPET members, materials presented to theCPET, and information collected in previous evaluations.

Background

The CPET was established to mirror the team approach of the PET model at thepartnership level. The CPET includes the project staff from the DOE and each of theseven program coordinators. The original application for funding for the MassachusettsWorkplace Literacy° Consortium (March, 1994) provides the following description of theCPET:

All Consortium activities are Overseen by the Consortium Planning andEvaluation Team (Consortium P.E.T.). This united body of partnershipcoordinators and a state-hired Consortium staff (full-time Coordinator andhalf time Assistant Coordinator) works to identify needs among itsmembership, ensures delivery of appropriate training support and technicalassistance, and, like the partnership P.E.T.s, evaluates its activities in twowritten reports during the course of the three years' funding. (p.48)

Questions/Discussion

Did the CPET act as a governing body?Was there a sense of belonging to a consortium?What were the advantages/disadvantages of the consortium model?

Question 1: Did the Consortium PET act as a governing body?

Working Toward Quality, the Year 2 evaluation of the MWLC addressed the issue of"The Consortium PET as Governing Body." It is clear from the discussion in that reportthat during the implementation of the project there was some level of disagreement aboutthe responsibilities of CPET members and the role of the CPET in governing the MWLC.A review of minutes of CPET meetings from the period March, 1995 through November,1996 revealed that the function of the CPET and its range and limits of authority togovern were two issues that dominated the agenda during that time period.

.116

Massachusetts Workplace Literacy Consortium Year 3 Evaluation, October, 1997 Page 9

Section III: Consortium Planning and Evaluation Team

Based on the materials reviewed and interviews conducted for the current evaluation, itcan be concluded that the CPET did not govern the MWLC. As described by the programcoordinators, the CPET served two roles in the governance of the MWLC. First, the

CPET served in an advisory capacity to the DOE. Second, the CPET members (programcoordinators) served as managers carrying out the directives of the DOE and fulfilling

reporting requirements for the program.

It was clear from the interviews, however, that several of the coordinators did notunderstand or accept this role at the beginning of the project. These coordinators believedthat the governance of the consortium would be shared by the DOE and programcoordinators through the CPET. Over time, all coordinators undersi.00d their role in theproject although they did not all fully accept it. An example from the final CPET meetingin June, 1997 illustrates this point. At that meeting, the Consortiurr Coordinator, a DOErepresentative, presented the coordinators with a list of options for using remaining fundsavailable to the project. The list had been compiled by the DOE prior to the CPETmeeting and some potential options had been eliminated from consideration.Interactions at the meeting and conversations between the evaluator and coordinatorsafter the meeting revealed that some coordinators were not pleased with their limited role

in the decision-making process.

Question 2: Was there a sense of belonging to a consortium?

The program coordinators had mixed responses to the question of whether there was a

sense of belonging to a consortium in the project. The overall imprssion that emergedfrom responses to this question was that coordinators felt that the operating model thatemerged during the project did not live up to their expectations of a consortium, but didhave many benefits over working alone. They did feel that they were part of a teamworking toward a single goal. The coordinators provided strong pOsitive feedback about

the role of the CPET in accomplishing tasks such as

establishing mechanisms for support and sharing of resources, cxpertise, and effective

operational models for all partnerships, andestablishing a curriculum focus group to develop curriculum guidelines andframeworks to link curricula of different projects.

Overall, the lack of a sense of a consortium was attributed to two rc lated factorsconcerning the management of the project by the DOE:

1 . lack of control over spending decisions; and2. more DOE control than anticipated.

Throughout the project, there was a sense that many decisions were being made outsideof the CPET and delivered to the CPET by the Consortium Coordinators. Adding to the

feeling of a lack of control were the many reporting requirements of the Massachusetts

17Massachusetts Workplace Literacy Consortium Year 3 Evaluation, Octobe,-, 1997 Page 10

Section Ilk Consortium Planning and Evaluation Team

DOE and the US DOE. The failure of the computerized system for collecting datarequired for the US DOE added to the coordinators' sense of frustrat ion.

Throughout the interviews, coordinators separated their opinions on the control exercisedby the DOE and the support provided by the Consortium Coordinators (representatives ofthe DOE). As will be discussed in more detail in response to the ne:a question,coordinators were generally positive about the quantity and quality of efforts of theConsortium Coordinators.

In joining the Massachusetts Workplace Literacy Consortium, many programcoordinators felt that their programs were entering into a full panne' ship or alliance withthe MA DOE and the other education providers. In the end, the con ;ensus was that therelationship was something less than a full partnership. However, this did not lessenmany of the benefits derived from the consortium model.

Question 3: What were the advantages/disadvantages of the consortium model?

The consortium model was designed to enhance the literacy programs in each of itspartner sites. Through features such as cooperative planning efforts comprehensiveprograms of staff training and resource Sharing, technical assistance and instructionalsupport the MWLC would offer partners a more cost effective method for deliveringservices. Further, the pooling of resources in the MWLC would allow many smallbusinesses to offer services that otherwise would be beyond their nwans. The applicationfor funding for the Massachusetts Workplace Literacy Consortium hicluded a list ofadvantages to education providers and business partners of the consortium model (Table4).

Table 4: Advantages of the Consortium MOdelThe consortium model:

builds upon nine years of Massachusetts Workplace Education successes andmistakesis most cost effectivecoordinates and distributes available resourcesavoids duplication of effort and services, andgreatly facilitates communication and dissemination of in ..ormation amongpartnerships sharing similar concerns:

The model provides clear strateeies to:address the needs of small business within a larger frameworkbuild the capacity of individual partnershipsdraw upon the wealth of expertise and resources within tl-e state. andproduce and disseminate materials in the field that are specific to industriescritical to the state's economic recovery and renewed contietitiveness.

There was general agreement among the program coordinators that :he MWLP didprovide the benefits listed in Table 4. Asked during the interview to discuss theadvantages of belonging to the consortium at least one coordinator mentioned each of the

.18Massachusetts Workplace Literacy Consortium Year 3 Evaluation, October, 1997 Page 11

Section III: Consortium Planning and Evaluation Team

nine advantages listed and most of the advantages were mentioned by several of thecoordinators.

There were three themes that dominated the coordinators' discussion of the advantages ofthe consortium model. In one form or another, the coordinators' responses focused on

1. the cost effectiveness of the model,2. the ability to share resources and draw on the resources of the state, and3. the opportunities for communication with colleagues.

Coordinators also mentioned several specific products or services that weredeveloped/disseminated during the course of the MWLC. In particular, severalcoordinators praised the consortium's efforts in the area of distance education. Oneproject developed was the use of televised mini-courses which were offered in the fall of1995 and 1996. Another project was the development and implementation of a computer-assisted instruction package with a focus on basic literacy skills in the health careindustry. Finally, the MWLC also explored options for the use of video conferencing toeffectively deliver material to more students. Apparently, some of these efforts weremore successful than others. However, even when discussing an offering that was notvery successful or not as relevant to their particular program, coordinators consistentlypresented the effort in a very positive manner.

A final observation about coordinators' discussion of the advantages of the consortiummodel concerns the role of the MA DOE. In contrast to their discussion of the role of theDOE in the governance of the MWLC, coordinators were quite positive about the role ofthe DOE in facilitating a) the development and dissemination of materials and services,and b) the opportunities for communication among the MWLC partners. Separating outthe role of the DOE as manager of the MWLC, the program coordinators were positiveabout the efforts of the DOE, and in particular the Consortium coordinators, to providethe support needed to accomplish the goals of the MWLC.

Coordinators did not provide any specific examples of disadvantages of the consortiummodel. That is, even those who were especially dissatisfied with the governance of theMWLC, did not indicate that their program suffered at all from membership in theMWLC. The only negative-concern expressed by coordinators Was the tinie required totravel to and from the monthly CPET meetings.

19Massachusetts Workplace Literacy Consortium Year 3 Evaluation, October, 1997 Page 12

Section IV: Planning and Evaluation Teams

Section IV: Planning and Evaluation Team

Overview

The central focus of the Year 3 evaluation was the functioning and organization of thelocal Planning and Evaluation Teams (PETs). The PETs were designed to mirror thepartnership model that characterized the MWLC. The PETs were established as the localgoverning body for the workplace education program'within each organization. Theyincluded representatives from all relevant stakeholders including the education provider,teachers, company management, unions, and student/workers. This evaluation addressedissues such as

participation and roles of members of the organization:problems/barriers to full participation in the PET;obstacles faced by the PET;types of decision-making process/style:meeting environment; androle of the PET in the institutionalization of the workplace literacy program.

Methodology

Data Collecti6n

A variety of methods were used to collect data concerning the PETs. Most of the datawere collected during site'visits to a sample of 10 PETs. Each site visit provided a)observational data, b) a participation chart, c) a group interview, and d) individual surveyscompleted by PET members. In addition to the site visits, additional informationconcerning the PETs was collected as part of the individual interviews with the programcoordinators conducted for the CPET evaluation. The program coordinator interviewprocess is described fully in the CPET section of this report. This section will focus onthe methods used to collect data during the site visit.

Site Visit Sample

The sample used for this portion of the evaluation was PETs from 10 of the 22partnership sites. The sites selected represented a cross section of the MWLCpartnerships based on factors such as type of industry. location. organization (union/non-union), and company size. The breakdown of the sample selected on each of these factorsis presented in Table 5. A complete list of the sites visited is presented in Appendix C.

20

Massachusetts Workplace Literacy Consortium Year 3 Evaluation, October, 1997 Page 13

Section IV: Planning and Evaluation Teams

Table 5: Characteristics of Sample SitesIndustry:

Healthcare - 5Manufacturing 4Education I

Location:Greater Boston 5

Central Massachusetts 2Southeastern Massachusetts - lWestern Massachusetts - 2

Organization:Union 3

Non-union 7

Size:Mid-size 5

Large 5

Site Visit Process

The evaluator conducted the site visits between April, 1997 and July, 1997. The originalplan called for the following process to be conducted at each site visit:

PET participation chart completedobservation of the PET meetingadministration of the group interview with PET membersdistribution of the individual surveys.collection of ancillary materials such as meeting agenda and reportsdistributed at the meeting.

This format was followed at six of the ten sites. At the remaining four sites, theevaluation was the only item on the PET meeting agenda which eliminated theopportunity to observe a full PET meetin2. Full PET meetings lasted approximately onehour. Evaluation only visits took approximately 30 minutes.

PET Participation Chart

A PET Participation Chart (see Appendix D) was completed at the beginning of eachmeeting to collect the following information from each person attending the meeting:

NamePosition (e.g., education provider, management, worker/student)Current PET membership statusTime served on PETNumber of PET meetings attended during the past 12 months.

21Massachusetts Workplace Literacy Consortium Year 3 Evaluation, October, 1997 Page 14

Section IV: Planning and Evaluation Teams

The program coordinator was also asked to complete the chart for current PET memberswho were unable to attend the meeting. The chart provided information on representationon the PET and turnover of PET members.

Group Interview

The group interview was the primary method for collecting data during the site visit. Theinterview was conducted by the evaluator using a survey/interview protocol which wasdistributed to all PET members. PET members were given the option of providingadditional information in written form and returning the interview protocol with theirindividual survey. This option was included to accommodate PET members who may nothave been comfortable responding in the group setting.

The interview protocol was developed with input from the evaluator, the ConsortiumCoordinator, and the evaluation unit of the DOE. A draft of the survey was distributed toprogram coordinators for their review and comment. The protocol contained 11questions concerning

participation in the PET;appropriateness of the PET model;success/obstacles to the workplace education program;support structure for the PET:advantages/disadvantages of the PET model; andinstitutionalization.

All but one of the questions were either open-response or a combination of forced-choiceand open-response. The first question was a forced choice question concerningparticipation in PET governance tasks. A copy of the interview protocol is provided inAppendix D.

Individual Survey

An individual survey was used to supplement information collected during the groupinterview. The survey addressed many of the same issues as the group interview, but wasdesigned to elicit opinions that PET members may not have been comfortable expressingin front of their colleagues. PET members were provided with a postage-paid, addressedenvelope to return the survey.

The survey contained eight questions. Two of the questions were forced-choicequestions. The remaining questions were open-response or a combination of forced-choice and open response. Like the group interview, the individual survey was developedwith input from the evaluator, the Consortium Coordinator, and the evaluation unit of theDOE. A copy of the individual survey is provided in Appendix D.

22

Massachusetts Workplace Literacy Consortium Year 3 Evaluation, October, 1997 Page 15

Section IV: Planning and Evaluation Teams

Completed surveys were received from 28 PET members. At least one survey wasreturned from each PET visited. Four surveys each were returned from three of the PETsites. One of the returned surveys was completed by a current student and one wascompleted by a former student (currently a union representative). A distribution of thepersons returning surveys by position is provided in Figure 1.

Figure 1: Respondents to Individual Survey by PositionNumber of surveys returned

0 2 3 4 5 6 7 8

Education Provider

Teacher

Human Resources

Management Staff

Student

Supervisory Staff

Union Representative

7

Observation Form

3

6

8

The evaluator completed an observation form at each PET site visit. The form was usedto collect information on the layout of the meeting room, seating arrangements.participation, and organization of the meeting. A copy of the form is provided inAppendix D.

Questions/Discussion

Question 1: Is there participation on the PET by all relevant stakeholders?

To properly answer this question, it is necessary to separate it into two parts:

1. Is there participation on the PET by all other relevant stakeholders'?2. Is there participation on the PET by students/workers?

23Massachusetts Workplace Literacy Consortium Year 3 Evaluation, October, 1997 Page 16

Section IV: Planning and Evaluation Teams

It was clear from both observations of the PET meetings and responses from PETmembers that student participation on the PET was a distinct issue from participation byall other members. Issues such as language barriers, position within the company, andlength of time on the PET present unique challenges to student participation. Therefore,student participation on the PETs will be addressed in response to Question 2. In thissection, the focus will be on the participation of other stakeholders.

There is no evidence of explicit or implicit barriers on the participation of any membersof the PET. Observations of PET meetings revealed a high level of interaction andparticipation by all PET members. In general, informal interactions among all PETmembers began prior to the start of the meeting and continued throughout the meeting.Members appeared to be comfortable voicing their opinions. Survey responses alsoindicated that all members were able to fully participate in PET discussions and decision-making.

A more serious threat to full participation, however, may be certain groups opting out ofthe PET process. At two sites, lack of consistent participation by company management(other than the human resources director) was identified as a problem for the PET. Atanother site, under-representation of supervisory staff was a concern.

Although participation by all members was considered important, PET members placedparticular importance on the participation of supervisory level staff. Gaining input andbuy-in from workers' direct supervisors was identified as a critical factor in determiningthe success of the PET and the workplace education program. One problem identified atthree of the large company sites was the difficulty of gaining buy-in from representativesfrom all departments affected by the workplace education program. Their PETs had beendesigned to include one or two staff members from the supervisory level, but there werefive or more departments with students participating in the program. They did notrecommend increasing the size of the PET to include all supervisors, but emphasized thatmeasures had to be taken to ensure that input was received from all supervisors.

Question 2: Is there participation on the PET by students/workers?

According to coordinators, concern about student participation on the PETs has been amajor issue since the beginning of the project. There was a fear that although studentinput was critical, obtaining student participation could be the weak link in the PETmodel. Coordinators anticipated that the greatest obstacles to student participation wouldbe students'

difficulties in expressing themselves in Englishreluctance to speak in front of their supervisors and other companymanagement.

Observations of the PET meetings and survey responses indicate that lack of studentparticipation was a problem for some PETs. One-quarter of the survey respondents

24Massachusetts Workplace Literacy Consortium Year 3 Evaluation, October, 1997 Page 17

Section IV: Planning and Evaluation Teams

responded "No" to the question "Do students have a full voice in the activities of yourPET?". The most common explanation for their response was students' inability tospeak English. This was especially true for beginning ESL students who were PETmembers. The reluctance of students to speak in front of management was alsomentioned on the surveys by a small number of respondents. Observations at the PETmeetings support the argument that language barriers were a greater problem thanreluctance to speak. At four PET meetings where the student member was fluent in

English, he/she actively participated in the discussions. (It should also be noted,

however, that in only one case was a controversial or confrontational issue discussed atthe meetings attended by the evaluator. That issue was raised by the student

representative.)

Another issue that emerged as a potential barrier to students' effectiveness on the PETwas their rate of turnover on the PET. On some PETs there was a high rate of turnover ofthe student representative due to a) students' completing the program, or b) a plannedsystem of rotating students through the PET. In other cases, the same student had servedon the PET since its inception. As expected, students with greater longevity on the PETappeared to be more comfortable and more willing to participate.

As was the case with supervisors, under-representation of students was mentioned as aproblem at the larger companies. In many cases, there was a single student representativein a program with more than 100 students spread across 10-15 classes. One siteattempted to solve this problem by forming a "student PET" with members from each ofthe classes. This group would meet shortly before the regular PET meeting and prepare areport to be delivered by their representative on the PET.

PETs have tried a number of approaches to support and encourage student participation.The most commonly used approaches involved attempts to make student involvement aregular part of the PET meeting. In some cases, the program coordinator made it a pointto regularly direct questions to the student representative. In other cases, a student reportwas made a regular agenda item. Also, many teachers reported meeting with the studentrepresentative individually prior to the PET meeting. The distribution of responses to the

survey question, "What does your PET do to encourage student participation and inputinto PET discussions and decisions?" is presented in Figure 2.

25

Massachusetts Workplace Literacy Consortium Year 3 Evaluation, October, 1997 Page 18

Section IV: Planning and Evaluation Teams

Figure 2: Efforts Taken to Promote Student Participation in the PET

Incentives

Increasing Numbers of Students

Student Reports at Meetings

Meeting Style/Environment

One-on-One Discussions

2

2

8

-^

0 2 4 6

Number of Responses

10

10

13

12 14

Question 3: Is a PET important to the success of a workplace education program?

There was unanimous agreement that the PET model was critical to the functioning of aworkplace education program. In the group interview, individual interviews with theprogram coordinators, and responses to individual surveys PET members consistentlyexpressed the opinion that it would be very difficult to develop, implement, andinstitutionalize a workplace education program without a PET. PET membersunanimously responded 'yes' to the following questions:

If you were starting a new workplace education program, would you have a Planningand Evaluation Team?Would you want to be part of a future Planning and Evaluation Team?

PET members provided a variety of reasons for the importance of the PET. The mostcommonly cited reasons were that the PET

provides the opportunity to keep all stakeholders informed on a regular basis;presents a forum for open discussion;strengthens the partnership and team approach within the organization; andpromotes buy-in from all stakeholders.

26Massachusetts Workplace Literacy Consortium Year 3 Evaluation, October, 1997 Page 19

Section IV: Planning and Evaluation Teams

A complete distribution of stated benefits of the PET model in enhancing the workplaceeducation program is provided in Figure 3.

Figure 3: Benefits of the PET ModelNumber of Responses

0 2 10 12 14

Encourage Problem-Solving

Enhance Institutionalization

Inform All Stakeholders

Promote Buy-In

Provide a Forum for Discussion

Provide a Forum for Students

Provide Feedback to Teachers

Set an Example for theOrganization

Set Direction for Program

Strengthen Partnership andTeamwork

2

12

6

10

12

A great deal of emphasis was placed on the importance of the PET in obtaining buy-infrom all stakeholders. In particular, participation and buy-in from management andsupervisory staff was considered a key to success.

In addition to informing all stakeholders, a significant group of respondents also made ita point to mention the importance of the PET in providing feedback to the teacher.Teachers were able to better shape/adjust their curriculum, make scheduling changes, andunderstand the organization because of their participation on the PET.

= The only disadvantage of the PET model mentioned by PET members was the timecommitment necessary to have an effective PET. One suggestion consistently offered asa change to the PET model was to have less frequent (but regularly scheduled) meetingsafter the program was established. It should be noted that attendance records indicatedthat most current PET members attend nine or ten meetings per year.

Another issue mentioned as detrimental to the PET model was the lack of participation bykey company personnel. In three cases, the PET had a great deal of trouble obtainingconsistent participation from company management due to a high rates of turnover andcorporate restructuring.

2 7Massachusetts Workplace Literacy Consortium Year 3 Evaluation, October, 1997 Page 20

Section IV: Planning and Evaluation Teams

Question 4: How are PET meetings conducted?

Across industries and types of organization there was a uniformity to the PET meetingsobserved for this evaluation. In all cases the PET meetings were conducted in aninformal and relaxed manner. Although there was a chairperson conducting eachmeeting, interactions between all members were quite unrestrained. In general, the roleof the chairperson was to ensure that all items on the agenda were discussed and that adate was set for the next meeting.

In all but one case, the program coordinator chaired the meeting. Teachers, however.were often the focus of most of the discussion: providing class updates, attendancereports, and providing/seeking information of other PET members. With the exception ofsome student representatives, all PET members participated at every meeting attended.The interactions were all friendly and professional.

In general, the meetings were held in a comfortable, quiet environment. Most meetingswere held in a conference room or board room. At the manufacturing companies, themeetings were held close to the work floor, but in a quiet room.

Question 5: Did the PET receive sufficient support?

PET members were asked to indicate what additional support and assistance their PETcould have received during the course of the project. Individuals were also asked to ratethe quality of support provided by the following groups:

Department of EducationConsortium PETProgram CoordinatorOther PET membersBUsiness/OrganizationUnion

Overall, in both the group interview and on the individual surveys, members gave highratings to the support provided by the program coordinator, other PET members, theorganization, and the union. There were specific cases, noted previously, where themanagement has not been very involved in the PET process and this was reflected in theratings given at those sites.

Support from the DOE was discussed primarily in terms of materials provided orproduced during the project and conference opportunities. PET members were positiveabout these efforts. The only specific additional support requested of the DOE was moreof a presence on site. Several PET members indicated that they would have liked to havehad a DOE representative visit their meetings, classes, or activities two to three times peryear.

2 8

Massachusetts Workplace Literacy Consortium Year 3 Evaluation, October, 1997 Page 21

Section IV: Planning and Evaluation Teams

The discussion of support from the Consortium PET was quite inter:sting. PET membershad virtually no knowledge of the Consortium PET or the consortium. The fewexceptions were members who had attended state or national confer:nces. Many PETmembers were aware that their provider was in partnership with othr businesses, butwere unaware of a statewide network.

Question 6: What role does the PET play in the institutionalizalion of the program?

The simple answer to this question is that the PET is helpful but not sufficient to ensurethat the program is institutionalized. Program coordinators and PET members at all sitesreported that the decision to continue or institutionalize the workpla:e education programwould be based on factors other than the success of the program. In many cases,however, they did feel that without the buy-in and teamwork promored by the PET thatthe organization would not even be considering continuing the program. At one site witha strong PET, the continuation of support for the workplace education program was oneof the first items that labor and management agreed upon during a tcnse contractnegotiation.

At all sites, PET members expressed the opinion that all of their act: vities throughout thecourse of the project were aimed at enhancing the chances for the success of theirprograms and therefore helping to ensure its continuation. Additionally, at four sites thePET had been actively involved in preparing reports, budEzets. or proposals for thecontinuation of the program beyond the current funding cycle. HONN ever, as PETmembers at two sites mentioned, it must also be remembered that many PET members(e.g., program coordinators, teachers, students, unions) may have hi;.,h personal stakesassociated with the continuation of the program. This could confou al an interpretationof the actions of the PET toward ensuring that the program continues.

29

Massachusetts Workplace Literacy Consortium Year 3 Evaluation, October, 1997 Page 22

Appendix A

30

Massachusetts Workplace Literacy Consortium Year 3 Evaluation, October. 1997

Program Standards and Quality Indicators

for Workplace Education

Massachusetts Department of Education

October, 1997

31

Introduction

The Development of Standards and Indicators

The Draft Standards and Quality Indicators for Workplace Education Programs define theconditions under which effective education programs can best develop, and identify outcomesand impacts expected of those programs. The current iteration is the result of many rounds ofinput from representative groups of experienced workplace educators, employers, unionrepresentatives, policy-makers, academics and evaluators. Their on-going review has enabled theStandards/Indicators to become more specific, while retaining their relevance to programs thatmay be diverse by size, funding source, industry, union involvement, and type of partnership(workplace and education provider, consortium, in-house labor-management program, etc.).The current Draft Standards and Indicators are intended to make the collective knowled2e andexperience of the field available to guide program start-up and development, on-going self-evaluation, and funding decisions.

Underlying Assumptions

Embedded in the Standards/ Indicators are some assumptions that should be made explicit. Thefirst is that learning to read, write, do math, or speak a new language requires a significantcommitment of time. Unlike skill enhancement workshops that may be effective after a fewhours, basic education for those who are learning for the first time takes hundreds of hours (seeAdult and Community Learning Services (ACLS)/DOE service delivery chart below). There areno long-lasting shortcuts. The workplace context demands thoughtful consideration of how tosatisfy work coverage and production needs while addressing educational goals realistically.

Second, Standards/Indicators presume the availability of technical assistance and support fordeveloping programs. Since many of the most promising practices in workplace education haveshort histories and limited documentation, there remains a need for program support and staffdevelopment. Specifically, programs may need assistance in areas such as workplace needsanalyses, authentic assessment of program outcomes and impacts, and curriculum development.

Finally, the standards acknowledge that workplace education is only effective when it respondsto both the self-defined needs of worker-students and employers, which may differ. Workers mayprefer to practice their skills using community and family topics, which may not immediatelyappear to be work-related. However, high-interest lessons will expedite the acquisition of basicskills, which will then transfer to the workplace. This transfer is the responsibility of theeducators, who relate the coursework to workplace applications, and of the other partners, whocreate workplace opportunities for students to practice and demonstrate their learning.

The Document

The Standards/Indicators document is organized according to three aspects of a workplaceprogram governance, staffing, and student services. Each of the three opens with a cover sheetoutlining the standards (expectations of performance) and related quality indicators (evidence of

3 2

achievement) of excellence. Attached to this is a chart which repeats the standards and lists notonly the quality indicators of excellent programs, but also quality indicators of basic anddeveloping programs. These descriptions incorporate what has been learned by seven years ofpublic/private collaborations and can be helpful in self-evaluating a program's strengths andweaknesses.

Definitions of Terms

The follow terms are used throughout the Standards/Indicators document and are intended tomean:

stakeholders: all parties that have an interest in workplace education including, but not limited to:management, supervisors, employees, union, education coordinator/provider, and teachers. Thegoverning team (GT) is made up of all stakeholders.

workplace needs analysis (WNA): a process of assessing the needs of an organization in order toidentify which needs can be appropriately addressed by a workplace education program. In thisprocess, a representative sample of all employees are interviewed.

adult basic education (ABE) curriculum frameworks: field-generated guidelines for applyingcurrent adult learning theory to the classroom.

ACLS GUIDELINES FOR CLASS SIZE AND HOURS

S ervice Type Class Sizo Estimated HoursNeeded toCompleteCategory

MinimalContactHours

Needed

_

Hours of InstructionPer Week

Range Preferred Range Optimal

ABE/Literacy(0-5.9)

5-20 5-10 650-900 200 5-20 8-15

Pre-ASE(6-8.9)

5-20 7-15 150-300 65 5-20 8-15

ASE(9-12)

5-20 9-15 120-300 20 5-20 8-15

ESOL(SPL 0-4)

5-20 11-20 450-700 125-150 5-20 8-15

ESOL(SPL 5-6)

5-20 11-20 300-450 100 5-20 8-15

ESOL(SPL 7-10)

5-20 11-20 200-300 50 5-20 8-15

BEST COPY AVAILABLE33

Governance

Standard #1: There is a formal, well-attended partnership/governing team (GT) established among allstakeholders (including management, supervisors, employees, education provider, teachers, and unionwhere present) which oversees the design, development, implementation, and evaluation of the progam.

Indicators: All stakeholders can access meetings and participate consistently on the GT; policies arein place which remove barriers to employee participation. All stakeholder representativesparticipate in more than 75% of governance activities

GT has established legitimacy and the support of all stakeholders

There are regular, facilitated meetings of the GT and on-going communication aboutprogram and governance activities

GT has full authority to develop and implement program goals and policies, and to directpractices

There are regular (class) meetings for students to articulate needs and prepare their repre-sentatives to participate in GT meetings. There is a student representative from every class

Standard #2: There are policies and procedures in place to ensure that the GT functions effectively,makes informed decisions, and maintains a supportive learning environment.

Indicators: There are standardized procedures for governing and a clear decision-making process.Roles and resp-onsibilities are clear; policies and procedures written

GT is informed by the stakeholders' workplace needs and strengths, resources,demographics, principles of adult education, and trends in the workplace/industry

- GT defines clearly-articulated goals and expectations for the program which are alignedwith the long- and short-term goals of all stakeholders, and which are used to develop anaction plan. There is a process in place to integrate GT goals and class-defined goals

GT uses evaluation data that is relevant to program goals, reliable, accessible, andregularly updated to evaluate and revise program goals and practices. GT respondsflexibly to changing circumstances

There is a full workplace needs assessment (WNA) of contextual, organizational, andlearner needs, strengths, and goals

Standard #3: Resources and policies support daily program operations as well as the long-terminstitutionalization of workplace education into the workplace culture and union contract.

Indicators: Resources are adequate to support a full range of student services, competitive salariesfor staff, staff training, workplace needs analyses. program evaluation, and governanceactivities

1

Gov

erna

nce

Stan

dard

#1:

The

re is

a f

orm

al, w

ell-

atte

nded

par

tner

ship

/gov

erni

ngte

am (

GT

) es

tabl

ishe

d am

ong

all s

take

hold

ers

(inc

ludi

ng m

anag

emen

t, su

perv

isor

s, e

mpl

oyee

s,ed

ucat

ion

prov

ider

, tea

cher

s, a

nd u

nion

whe

re p

rese

nt)

whi

chov

erse

es th

c de

sign

, dev

elop

men

t, im

plem

enta

tion,

and

eva

luat

ion

of th

e pr

ogra

m.

Bas

icD

evel

opin

gE

xcel

lent

- E

ffor

ts a

rc m

ade

to e

nsur

e ac

cess

to G

T b

y al

lst

akeh

olde

r re

pres

enta

tives

thro

ugh

the

deve

lop-

men

t of

polic

ies

that

enc

oura

ge a

nd s

uppo

rtpa

rtic

ipat

ion.

All

repr

esen

tativ

es w

ho e

xpre

ss a

nin

tere

st p

artic

ipat

e in

mos

t of

the

gove

rnan

ceac

tiviti

es

- T

here

are

irre

gula

r m

eetin

gs o

f th

e G

T im

dco

mm

unic

atio

n w

hen

a pr

oble

m a

rise

s

GT

has

pro

visi

onal

aut

hori

ty to

dev

elop

and

impl

emen

t.pro

gram

goa

ls a

nd p

olic

ies

- T

here

is a

stu

dent

rep

rese

ntat

ive

on th

e G

T.

Stud

ents

con

vey

thei

r ne

eds

and

idea

s to

thei

r re

pin

form

ally

35

All

stak

ehol

ders

or

thei

r re

pres

enta

tives

can

acce

ss m

eetin

gs a

nd c

arry

out

gov

erna

nce

activ

ities

.A

ll st

akeh

olde

r re

pres

enta

tives

par

ticip

ate

in m

ost

of th

e go

vern

ance

act

iviti

es

GT

is d

evel

opin

g le

gitim

acy

and

the

supp

ort o

f al

lst

akeh

olde

rs

- T

here

are

reg

ular

mee

tings

of

the

GT

and

on-

goin

g co

mm

unic

atio

n ab

out p

rogr

am a

ctiv

ities

- G

T is

wor

king

to e

stab

lish

full

auth

ority

toim

plem

ent p

rogr

am g

oals

and

pol

icie

s

The

re is

mor

e th

an o

ne s

tude

nt r

epre

sent

ativ

e on

the

GT

. Rep

s ar

e gi

ven

oppo

rtun

ities

to s

tay

into

uch

with

all

clas

ses/

stud

ents

- A

ll st

akeh

olde

rs c

an a

cces

s m

eetin

gs a

nd p

artic

ipat

e co

nsis

tent

ly o

nth

e G

T; p

olic

ies

are

in p

lace

whi

ch r

emov

e ba

rrie

rs to

em

ploy

eepa

rtic

ipat

ion.

All

stak

ehol

der

repr

esen

tativ

es p

artic

ipat

e in

mor

e th

an75

% o

f go

vern

ance

act

iviti

es

GT

has

est

ablis

hed

legi

timac

y an

d th

e su

ppor

t of

all s

take

hold

ers

The

re a

re r

egul

ar, f

acili

tate

d m

eetin

gs o

f th

e G

T a

nd o

n-go

ing

com

mun

icat

ion

abou

t pro

gram

and

gov

erna

nce

activ

ities

- G

T h

as f

ull a

utho

rity

to d

evel

op a

nd im

plem

ent p

rogr

am g

oals

and

polic

ies,

and

to d

irec

t pra

ctic

es

- T

here

are

reg

ular

(cl

ass)

mee

tings

for

stu

dent

s to

art

icul

ate

need

s an

dpr

epar

e th

eir

repr

esen

tativ

es to

par

ticip

ate

in G

T m

eetin

gs. T

here

is a

stud

ent r

epre

sent

ativ

e fr

om e

very

cla

ss

36

Sta

ndar

d #2

: The

rear

e po

licie

s an

d pr

oced

ures

in p

lace

to e

nsur

e th

at th

e G

T f

unct

ions

eff

ectiv

ely,

mak

es in

form

ed d

ecis

ions

,an

d m

aint

ains

a s

uppo

rtiv

e le

arni

ngen

viro

nmen

t.

Bas

icD

evel

opin

gE

xcel

lent

Rol

es a

nd r

espo

nsib

ilitie

s ar

e ou

tline

d fo

rpi

lotin

g

GT

is in

form

ed b

y th

e st

akeh

olde

rs' w

orkp

lace

need

s, s

tren

gths

, wal

s an

d re

sour

ces,

and

prin

cipl

es o

f adu

lt ed

ucat

ion

GT

def

ines

goa

ls a

nd e

xpec

tatio

ns fo

r th

epr

ogra

m

Lear

ner

need

s, s

tren

gths

, and

goa

ls a

re a

sses

sed;

ther

e is

lim

ited

asse

ssm

ent o

f wor

kpla

ce n

eeds

Rol

es a

nd r

espo

nsib

ilitie

s ar

e fu

rthe

r de

fined

- G

T is

info

rmed

by

the

stak

ehol

ders

' wor

kpla

cene

eds,

str

engt

hs, g

oals

and

res

ourc

es, a

nd p

rinci

ples

of a

dult

educ

atio

n

- G

T d

efin

es g

oals

and

exp

ecta

tions

for

the

prog

ram

and

is b

egin

ning

the

proc

ess

of id

entif

ying

how

prog

ram

can

alig

n w

ith lo

ng-t

erm

goa

ls o

fst

akeh

olde

rs. T

here

is r

ecog

nitio

n th

at th

c G

T a

ndcl

ass

have

pot

entia

lly c

ompe

ting

but l

egiti

mat

ego

als

,

- Le

arne

r ne

eds,

str

engt

hs, a

nd g

oals

are

ass

esse

d;pr

ogra

m b

egin

s to

res

earc

h th

e w

orkp

lace

to b

ette

run

ders

tand

the

wor

k co

ntex

t

- T

here

are

sta

ndar

dize

d pr

oced

ures

for.

gove

rnin

g an

d a

clea

r de

cisi

on-

mak

ing

proc

ess.

Rol

es a

nd r

espo

nsib

ilitie

s ar

e cl

ear;

pol

icie

s an

dpr

oced

ures

writ

ten

- G

T is

info

rmed

by

the

stak

ehol

ders

' wor

kpla

ce n

eeds

and

str

cngt

hs,

reso

urce

s, d

emog

raph

ics,

prin

cipl

es o

f adu

lt ed

ucat

ion,

and

trcn

ds in

the

wor

kpla

ce/in

dust

ry

- G

Tde

fines

cle

arly

-art

icul

ated

goa

ls a

nd e

xpec

tatio

ns fo

r th

epr

ogra

mw

hich

are

alig

ned

with

the

long

- an

d sh

ort-

term

goa

ls o

f all

stak

ehol

ders

, and

whi

ch a

re u

sed

to d

evel

op a

n ac

tion

plan

. The

re is

apr

oces

s in

pla

ce to

inte

grat

e G

T g

oals

and

cla

ss-d

efin

ed g

oals

- G

T u

ses

eval

uatio

n da

ta th

at is

rel

evan

t to

prog

ram

goa

ls, r

elia

ble,

acce

ssib

le, a

nd r

egul

arly

upd

ated

to e

valu

ate

and

revi

se p

rogr

am g

oals

and

prac

tices

. GT

res

pond

s fle

xibl

y to

cha

ngin

g ci

rcum

stan

ces

- T

here

is a

full

wor

kpla

ce n

eeds

ass

essm

ent (

WN

A)

of c

onte

xtua

l,or

gani

zatio

nal,

and

lear

ner

need

s, s

tren

gths

, and

goa

ls

.

Sta

ndar

d #3

: Res

ourc

es a

nd p

olic

ies

supp

ort d

aily

pro

gram

'ope

ratio

ns a

s w

ell

as th

e lo

ng-t

erm

inst

itutio

naliz

atio

n of

wor

kpla

ce e

duca

tion

into

the

wor

kpla

ce c

ultu

rean

d un

ion

cont

ract

.

Bas

icD

evel

opin

gE

xcel

lent

Res

ourc

es a

re a

dequ

ate

to s

uppo

rt a

full

rang

e of

stu

dent

ser

vice

s,co

mpe

titiv

e sa

larie

s fo

r st

aff,

staf

f tra

inin

g, w

orkp

lace

nee

ds a

naly

sssn

prog

ram

eva

luat

ion,

and

gov

erna

nce

activ

ities

0 :d

3 '7

Student Services

Standard #1: There are learning opportunities and support services designed to meet the needs and goalsof adults at the workplace, and an educational ladder linked to a full sequence of educational services inthe community.

Indicators: Classes and other learning activities are offered at the appropriate level, time, and place(with transportation if off-site) to meet the needs of all students. Classes and otherlearning activities are scheduled consistently and in long enough duration for learning tooccur. Delivery may be in the form of classes, 1-1 tutoring, on-the-job support, and otherapplicable models

There is a non-discriminatory process for selecting and placing interested students. Allthose who want services can access instructional opportunities/support services

consistently because the program has policies and incentives that support participationwithout jeopardizing coverage, production or service delivery, and family responsibilities.Students have access to program offerings until program goals are met

Students receive 100% paid release time and other incentives for participation,completion, and/or achievement (or compensation as stipulated in union contract)

There are no employment-related repercussions for participation,.non-participation orlack of progress. Policies protect the confidentiality of student records

Support services (educational counseling, transportation to off-site services, peeradvocates, daycare, etc) are in place on-site, as needed, or readily available and easilyaccessible. All students reccive information and counseling support regardingopportunities for further education and training

Facilities and services are handicapped-accessible and provide for a safe and comfortablelearning and working environment for students and staff. Program ability to provideaccommodations exists. All staff are trained and integrate support services into dailyprogram operations

Standard #2: The program has a curriculum that flexibly responds to the changing needs, strengths, andaoals of students and the GT.

Indicators: Program uses a systematic process for regularly assessing learning needs, strengths, andgoals and revising the curriculum accordingly

Student learning needs are determined by a variety of tools. including individual self-assessment and group goal-settimg

Lessons consistently incorporate the needs, strengths, learning styles, experiences, andperspectives of the students

Teacher adapts materials so that they are relevant to the comext (the students' lives,union, and workplace). Teacher uses a variety of materials and activities in an adult-appropriate manner and applies principles of adult learning as articulated in the ABEcurriculum frameworks

Classes focus on the development of skills, knowledge and abilities that are transferable

39 BEST COPY AVAILABLE

to various contexts (within and outside of work) and help students make thesetransferences. Lessons include but are not limited to workplace-related content

- Curriculum encourages critical thinking skills so that students can improve their livesand workplace, and can apply new skills and knowledge to problematic situations.

Standard #3: There is demonstrable progress, according to all stakeholders, toward program goals.

Indicators: There is demonstrable evidence of substantial progress (toward anticipated andunanticipated goals) for all stakeholders.

There is recognition that changing conditions and/or goals affect performance andimpact.

Program uses a variety of evaluation tools to document a comprehensive, "authentic"view of student progress and satisfaction.

- Policies and practices are in place to ensure that workers are given opportunities topractice and demonstrate new abilities on the job.

At least 75% of the students enrolled in the program participate consistently in plannedinstructional activities.

At least 75% of the students achieve the learning standards or objectives of instruction.

Those who do not achieve the standards or objects are evaluated and receive guidancefor further education or services

All students can apply what they have learned at work, in the union, at home, and in thecommunity.

Students who pursue further education and training are tracked at least once after leavingthe program.

4 0*

1

Stud

elvi

t Ser

vice

s

Sta

ndar

d #1

: The

re a

re le

arni

ng o

ppor

tuni

ties

and

supp

ort s

ervi

ces

desi

gned

to m

eet t

he n

eeds

and

goa

ls o

f adu

lts a

t the

wor

kpla

ce, a

ndan

edu

catio

nal l

adde

rlin

ked

to a

full

sequ

ence

of e

duca

tiona

l ser

vice

s in

the

com

mun

ity.

Bas

icD

evel

opin

gE

xcel

lent

- C

lass

es a

re o

ffere

d at

leve

ls, t

imes

, and

pla

ces

that

acc

omm

odat

e th

e m

ost s

tude

nts

The

re is

a n

on-d

iscr

imin

ator

y pr

oces

s fo

rse

lect

ing

and

plac

ing

inte

rest

ed s

tude

nts.

The

orga

niza

tion

is d

evel

opin

g po

licie

s so

that

stu

dent

sca

n ac

cess

inst

ruct

iona

l opp

ortu

nitie

s/su

ppor

tse

rvic

es c

onsi

sten

tly w

ithou

t jeo

pard

izin

gco

vera

ge, p

rodu

ctio

n or

ser

vice

del

iver

y, a

ndfa

mily

res

pons

ibili

ties.

Stu

dent

s ha

ve a

cces

s to

clas

ses

and

othe

r le

arni

ng a

ctiv

ities

on

a sp

ace-

avai

labl

e ba

sis

Stu

dent

s re

ceiv

e 50

% p

aid

rele

ase

time

(or

com

pens

atio

n as

stip

ulat

ed in

uni

on c

ontr

act)

The

re a

re n

o em

ploy

men

t-re

late

d re

perc

ussi

ons

for

part

icip

atio

n, n

on-p

artic

ipat

ion

or la

ck o

fpr

ogre

ss. P

olic

ies

prot

ect t

he c

onfid

entia

lity

ofst

uden

t rec

ords

Sup

port

ser

vice

s ar

e av

aila

ble

thro

ugh

refe

rral

..A

t lea

st 5

0% o

f the

stu

dent

s re

ceiv

e in

form

atio

nan

d co

unse

ling

supp

ort r

egar

ding

opp

iwto

nitie

s fo

rfu

rthe

r ed

ucat

ion

and

trai

ning

.

41

The

re is

a n

on-d

iscr

imin

ator

y pr

oces

s fo

r se

lect

ing

and

plac

ing

inte

rest

ed s

tude

nts.

At l

east

50%

of

thos

e w

ho w

ant s

ervi

ces

can

acce

ss in

stru

ctio

nal

oppo

rtun

ities

/sup

port

ser

vice

s el

:insi

sten

tly b

ecau

seth

e pr

ogra

m h

as p

olic

ies

and

ince

ntiv

es th

atsu

ppor

t par

ticip

atio

n w

ithou

t jeo

pard

izin

gco

vera

ge, p

rodu

ctio

n or

ser

vice

del

iver

y, a

ndfa

mily

res

pons

ibili

ties.

Stu

dent

s ha

ve a

cces

s to

120

hour

s of

inst

ruct

ion

- S

tude

nts

rece

ive

50%

pai

d re

leas

e tim

e an

d ot

her

finan

cial

(e.

g., b

onus

, inc

reas

e) a

nd n

on-f

inan

cial

ince

ntiv

es (

e.g.

, rec

ogni

tion)

for

part

icip

atio

n,co

mpl

etio

n, a

nd/o

r ac

hiev

emen

t (or

com

pens

atio

nas

stip

ulat

ed in

uni

on c

ontr

act)

- T

here

are

no

empl

oym

ent-

rela

ted

repe

rcus

sion

s

for

part

icip

atio

n, n

on-p

artic

ipat

ion

or la

ck o

fpr

ogre

ss. P

olic

ies

prot

ect t

hc c

onfid

entia

lity

ofst

uden

t rec

ords

Edu

catio

nal c

ouns

elin

g se

rvic

es a

re a

vaila

ble

info

rmal

ly o

n-si

te; o

ther

ser

vice

s ar

e av

aila

ble

thro

ugh

refe

rral

. At l

east

75%

of t

he s

tude

nts

rece

ive

info

rmat

ion

and

coun

selin

g su

ppor

tre

gard

ing

oppo

rtun

ities

for

furt

her

educ

atio

n an

dtr

aini

ng

-- C

lass

es a

nd o

ther

lear

ning

act

iviti

es a

re o

ffere

d at

the

appr

opria

tele

vel,

time,

and

pla

ce (

with

tran

spor

tatio

n if

off-

site

) to

mee

t the

need

s of

all

stud

ents

. Cla

sses

and

oth

er le

arni

ng a

ctiv

ities

arc

sche

dule

d co

nsis

tent

ly a

nd in

long

eno

ugh

dura

tion

for

lear

ning

tooc

cur.

Del

iver

y m

ay b

e in

the

form

of c

lass

es, 1

-1 tu

torin

g, o

n-th

e-jo

bsu

ppor

t, an

d ot

her

appl

icab

le m

odel

s

The

re is

a n

on-d

iscr

imin

ator

y pr

oces

s fo

r se

lect

ing

and

plac

ing

inte

rest

ed s

tude

nts.

All

thos

e w

ho w

ant s

ervi

ces

can

accc

ssin

stru

ctio

nal o

ppor

tuni

ties/

sup

port

ser

vice

s co

nsis

tent

ly b

ecau

se th

epr

ogra

m h

as p

olic

ies

and

ince

ntiv

es th

at s

uppo

rt p

artic

ipat

ion

with

out

jeop

ardi

zing

cov

erag

e, p

rodu

ctio

n or

ser

vice

del

iver

y, a

nd fa

mily

resp

onsi

bilit

ies.

Stu

dent

s ha

ve a

cces

s to

pro

gram

offe

rings

unt

ilpr

ogra

m g

oals

are

met

- S

tude

nts

rece

ive

100%

pai

d re

leas

e tim

e an

d ot

her

ince

ntiv

es fo

rpa

rtic

ipat

ion,

com

plet

ion,

and

/or

achi

evem

ent (

or c

ompe

nsat

ion

as

stip

ulat

ed in

uni

on c

ontr

act)

- T

here

are

no

empl

oym

ent-

rela

ted

repe

rcus

sion

s fo

r pa

rtic

ipat

ion,

non-

part

icip

atio

n or

lack

of p

rogr

ess.

Pol

icie

s pr

otec

t the

conf

iden

tialit

y of

stu

dent

rec

ords

Sup

port

ser

vice

s (e

duca

tiona

l cou

nsel

ing,

tran

spor

tatio

n to

off-

site

serv

ices

, pee

r ad

voca

tes,

day

care

, etc

) ar

e in

pla

ce o

n-si

te,

as n

eede

d,or

rea

dily

ava

ilabl

e an

d ea

sily

acc

essi

ble.

All

stud

ents

rec

eive

info

rmat

ion

and

coun

selin

g su

ppor

t reg

ardi

ng o

ppor

tuni

ties

for

furt

her

educ

atio

n an

d tr

aini

ng

42

2

- F

acili

ties

and

serv

ices

arc

han

dica

pped

-ac

cess

ible

and

pro

vide

for

a sa

fe a

nd c

omfo

rtab

lele

arni

ng a

nd w

orki

ng e

nviro

nmen

t for

stu

dent

s an

dst

aff.

Pro

gram

abi

lity

to p

rovi

de a

ccom

mod

atio

nsex

ists

thro

ugh

refe

rral

sys

tem

- F

acili

ties

and

serv

ices

are

han

dica

pped

-acc

essi

ble

and

prov

ide

for

a sa

fe a

nd c

omfo

rtab

le le

arni

ng a

ndw

orki

ng e

nviro

nmen

t for

stu

dent

s an

d st

aff.

Pro

gram

abi

lity

to p

rovi

de a

ccom

mod

atio

ns e

xist

sth

roug

h a

com

bina

tion

of r

efer

rals

and

som

e tr

aine

dst

aff

- F

acili

ties

and

serv

ices

are

han

dica

pped

-acc

essi

ble

and

prov

ide

for

asa

fe a

nd c

omfo

rtab

le le

arni

ng a

nd w

orki

ng e

nviro

nmen

t for

stu

dent

san

d st

aff.

Pro

gram

abi

lity

to p

rovi

de a

ccom

mod

atio

ns e

xist

s. A

ll st

aff

are

trai

ned

and

inte

grat

e su

ppor

t ser

vice

s in

to d

aily

pro

gram

oper

atio

ns

Sta

ndar

d #2

: The

pro

gram

has

a c

urric

ulum

thz

t fle

xibl

y re

spon

ds to

the

chan

ging

nee

ds, s

tren

gths

, and

goa

ls o

f stu

dent

s an

d th

e G

T.

Bas

icD

evel

opin

gE

xcel

lent

- T

here

is a

pro

cess

for

asse

ssin

g le

arni

ng n

eeds

,st

reng

ths,

and

goa

ls a

nd r

evis

ing

the

curr

icul

umac

cord

ingl

y, h

ut it

is in

o ye

t art

icul

ated

or

cons

iste

nt

- P

rogr

am u

ses

at le

ast o

ne e

valu

atio

n to

ol to

docu

men

t stu

dent

pro

gres

s an

d sa

tisfa

ctio

n

- Le

sson

s re

late

to th

e w

orkp

lace

, but

inco

nsis

tent

ly in

corp

orat

e th

e ne

eds,

str

engt

hs,

expe

rienc

es, a

nd p

ersp

ectiv

es o

f the

stu

dent

s

- M

ater

ials

are

rel

evan

t to

a w

orkp

lace

set

ting

but

not c

usto

miz

ed to

this

wor

kpla

ce. T

each

er is

awar

e of

AB

E c

urric

ulum

fram

ewor

ks a

nd a

pplie

sso

me

prin

cipl

es o

f adu

lt le

arni

ng

Tea

cher

has

bas

ic tr

aini

ng a

nd is

aw

are

of th

ene

ed fo

r fu

rthe

r tr

aini

ng/te

chni

cal a

ssis

tanc

e

- T

here

is a

reg

ular

pro

cess

for

asse

ssin

g le

arni

ngne

eds,

str

engt

hs, a

nd g

oals

but

it is

une

venl

yap

plie

d to

cur

ricul

1111

1de

velo

pmen

t

Pro

gram

use

s m

ore

than

onc

app

ropr

iate

eval

uatio

n to

ol to

doc

umen

t stu

dent

pro

gik.

ss a

ndsa

tisfa

ctio

n

- Le

sson

s pa

rtia

lly in

corp

orat

e th

e ne

eds

and

stre

ngth

s, e

xper

ienc

es, a

nd p

ersp

ectiv

es o

f stu

dent

s

- T

here

is a

. doc

umen

ted

atte

mpt

to a

dapt

mat

eria

lsto

the

cont

ext'.

Tea

cher

mak

es e

ffort

s to

impl

emen

tth

c A

BE

cur

ricul

um fr

amew

orks

and

app

lypr

inci

ples

of a

dult

lear

ning

- P

rogr

am u

ses

a sy

stem

atic

pro

cess

for

regu

larly

ass

essi

ng le

arni

ngne

eds,

str

engt

hs, a

nd g

oals

and

rev

isin

g th

e cu

rric

ulum

acc

ordi

ngly

Stu

dent

lear

ning

nee

ds a

re d

eter

Min

ed b

y a

varie

ty o

f too

ls,

incl

udin

g in

divi

dual

sel

f-as

sess

men

t and

gro

up g

oal-s

ettin

g

- Le

sson

s co

nsis

tent

ly in

corp

orat

e th

e ne

eds,

str

engt

hs, l

earn

ing

styl

es, e

xper

ienc

es, a

nd p

ersp

ectiv

es o

f the

stu

dent

s

- T

each

er a

dapt

s m

ater

ials

so

that

they

are

rel

evan

t to

the

cont

ext (

the

stud

ents

' liv

es, u

nion

, and

wor

kpla

ce).

Tea

cher

usc

s a

varie

ty o

fm

ater

ials

and

act

iviti

es in

an

adul

t-ap

prop

riate

man

ner

and

appl

ies

prin

cipl

es o

f adu

lt le

arni

ng a

s ar

ticul

ated

in th

e A

BE

cur

ricul

umfr

amew

orks

- C

lass

es fo

cus

on th

e de

velo

pmen

t of s

kills

, kno

wle

dge

and

abili

ties

that

are

tran

sfer

able

to v

ario

us c

onte

xts

(with

in a

nd o

utsi

de o

f wor

k)an

d he

lp s

tude

nts

mak

e th

ese

tran

sfer

ence

s. L

esso

ns in

clud

e bu

t are

4no

t lim

ited

to w

orkp

lace

-rel

ated

con

tent

- C

urric

ulum

enc

oura

ges

criti

cal t

hink

ing

skill

s so

that

stu

dent

s ca

nim

prov

e th

eir

lives

and

wor

kpla

ce, a

nd c

an a

pply

new

ski

lls a

ndkn

owle

dge

to p

robl

emat

ic s

ituat

ions

3

Sta

ndar

d #3

: The

re is

dem

onst

rabl

epr

ogre

ss, a

ccor

ding

to a

ll st

akeh

olde

rs, t

owar

d pr

ogra

m g

oals

.

Bas

ic,

Dev

elop

ing

Exc

elle

nt-

The

re is

dem

onst

rabl

e ev

iden

ce o

f sub

stan

tial

- T

here

is d

emon

stra

ble

evid

ence

of s

ubst

antia

lT

here

is d

emon

stra

ble

evid

ence

of s

ubst

antia

l pro

gres

s (t

owar

dpr

ogre

ss (

tow

ard

antic

ipat

ed a

nd u

nant

icip

ated

prog

ress

(to

war

d an

ticip

ated

and

una

ntic

ipat

edan

ticip

ated

and

una

ntic

ipat

ed g

oals

) fo

r al

l sta

keho

lder

sgo

als)

for

at le

ast o

ne s

take

hold

ergo

als)

for

the

part

icip

ant a

nd a

t lea

st o

ne o

ther

stak

ehol

der

-The

re is

rec

ogni

tion

that

cha

ngin

g co

nditi

ons

-The

re is

rec

ogni

tion

that

cha

ngin

g co

nditi

ons

The

re is

rec

ogni

tion

that

cha

ngin

g co

nditi

ons

and/

or g

oals

affe

ctan

d/or

goa

ls a

ffect

per

form

ance

and

impa

ctan

d/or

goa

ls a

ffect

per

form

ance

4nd

impa

ctpe

rfor

man

ce a

nd im

pact

- P

rogr

am u

ses

a va

riety

of e

valu

atio

n to

ols

to d

ocum

ent a

com

preh

ensi

ve, "

auth

entic

" vi

ew o

f stu

dent

prog

ress

and

sat

isfa

ctio

n

- P

rogr

am is

dev

elop

ing

polic

ies

and

prac

tices

are

- P

olic

ies

and

prac

tices

are

in p

lace

to e

nsur

e th

at w

orke

rs a

re g

iven

in p

lace

to e

nsur

e th

at w

orke

rs a

re g

iven

oppo

rtun

ities

to p

ract

ice

and

dem

onst

rate

new

abili

ties

on th

e jo

b

oppo

rtun

ities

to p

ract

ice

and

dem

onst

rate

new

abi

litie

son

the

job

-.H

alf t

he s

tude

nts

enro

lled

in th

e pr

ogra

m,

- 75

% o

f the

stu

dent

s en

rolle

d in

Ate

pro

gram

At l

east

75%

of t

he s

tude

nts

enro

lled

in th

e pr

ogra

m p

artic

ipat

epa

rtic

ipat

e co

nsis

tent

ly in

pla

nned

inst

ruct

iona

lpa

rtic

ipat

e co

nsis

tent

ly in

pla

nned

inst

ruct

iona

lco

nsis

tent

ly in

pla

nned

inst

ruct

iona

l act

iviti

esac

tiviti

esac

tiviti

es,

.- A

t lea

st 5

0% o

f the

stu

dent

s de

mon

stra

te-

At l

east

75%

of t

he s

tude

nts

dem

onst

rate

pro

gres

sA

t lea

st 7

5% o

f the

stu

dent

s ac

hiev

e th

e le

arni

ng s

tand

ards

orpr

ogre

ss to

war

ds th

e le

arni

ng s

tand

ards

or

tow

ard

the

lear

ning

sta

ndar

ds o

r ob

ject

ives

of

obje

ctiv

es o

f ins

truc

tion.

Tho

se w

ho d

o no

t ach

ieve

thc

stan

dard

sor

obje

ctiv

es o

f ins

truc

tion

inst

ruct

ion

obje

cts

arc

eval

uate

d an

d re

ceiv

e gu

idan

ce fo

r fu

rthe

r ed

ucat

ion

orse

rvic

es

At l

east

hal

f the

stu

dent

s ca

n ap

ply

wha

t the

yha

ve le

arne

d at

wor

k, in

the

unio

n, a

t hom

e, a

nd-

Mos

t of t

he s

tude

nts

can

appl

y w

hat t

hey

have

lear

ned

at w

ork,

in th

e un

ion,

at h

ome,

and

in th

e-

All

stud

ents

can

app

ly w

hat t

hey

have

lear

ned

at w

ork,

in th

e un

ion,

at h

ome,

and

in th

e co

mm

unity

in th

e co

mm

unity

com

mun

ity

At l

east

hal

f of t

he s

tude

nts

who

pur

sue

furt

her

- S

tude

nts

who

pur

sue

furt

her

educ

atio

n an

dS

tude

nts

who

pur

sue

furt

her

educ

atio

n an

d tr

aini

ngar

e tr

acke

d at

educ

atio

n an

d tr

aini

ng a

re tr

acke

d at

leas

t onc

eaf

ter

leav

ing

the

prog

ram

,

trai

ning

are

trac

ked

once

afte

r le

avin

g th

e pr

ogra

mle

ast o

nce

afte

r le

avin

g th

e pr

ogra

m

4 5

BE

ST

CO

PY

AV

AIL

AB

LE

4 6

Staffing

Standard #1: Staff is competent to teach the relevant subject matter to a diverse population of adults in theworkplace.

Indicators: Staff has experience teaching adults in workplace settin,;s, and has experience workingwith multi-ethnic populations

- Staff has experience and skill in teaching the relevant st!bject matter: reading andwriting, oral communication, math, etc.

- Staff is skilled in creating original curricula tailored to each worksite and responsive tothe goals of each group of students

Staff is skilled in assessing and evaluating learners' skills and progress and in teachingstudents how to evaluate their own learning

Staff is skilled in working within a business, union, or labor/management context. Staffhas some expertise in gathering qualitative and quantitative data for diverse stakeholders

Standard #2: Staff is compensated at competitive salary levels for teaching, curriculum development.professional development, student assessment, and program development.

Indicators: Staff is a mix of part-time and full-time, with full benefit s

Staff is paid for contact hours and at least 2 hours of prep time, 2 hours of curriculumdevelopment time, 1 hour of assessment and doCumen-tation time, and I. tiour of planning/adminstrative time for each four contact hours worked (6:4). At least 2.5% of their time isallocated to training and development. At least 4.5% of th.eir time is allocated to programplanning, recruitment. evaluation, and improvement. This includes attending company-

sponsored training, communicating with company personnel, time on the floor

All stakeholders participate in offering a comprehensiv4taff orientation to theworkplace, its culture and dynamics

Staff development opportunities include specialized workplace education trainingsinformed by trends in the industry, the workplace, the labor movement, and the field ofadult education, and in-house company training in which 3tudents/employees attend. Thiswill increase staff's understanding of business issues, nee.is, and processes

Standard #3: Staffing is adequate to cover the tasks and functions needed to support students through theprogram.

Indicators: Staff is skilled in educational counseling and uses referral network

Program involves all workplace personnel and orients thcm to principles of adultlearning

There are trained support service workers on staff (counelors, day care workers, etc.)Staff turnover is low

47

Staf

fing

Stan

dard

#1:

Sta

ff is

com

pete

nt to

teac

h th

e re

leva

nt s

ubje

ctm

atte

r to

a d

iver

se p

opul

atio

n of

adu

lts in

the

wor

kpla

ce.

Bas

icD

evel

opin

gE

xcel

lent

- S

taff

has

teac

hing

exp

erie

nce

but n

o w

orkp

lace

expe

rienc

e, O

R is

hig

hly

expe

rienc

ed in

the

wor

kpla

ce s

ettin

g w

ith s

ome

trai

ning

in a

dult

educ

atio

n m

etho

ds/m

ater

ials

Sta

ff ar

e kn

owle

dgea

ble

abou

t the

rel

evan

tsu

bjec

t mat

ter

Sta

ff is

aw

are

of th

e ne

ed fo

r fu

rthe

rpr

ofes

sion

al d

evel

opm

ent i

n th

e ar

eas

ofcu

stom

ized

cur

ricul

um d

evel

opm

ent,

stud

ent-

cent

ered

edu

catio

n, a

nd/o

r au

then

tic a

sses

smen

tan

d ha

s id

entif

ied

next

ste

ps fo

r pr

ofes

sion

alde

velo

pmen

t

- S

taff

has

basi

c kn

owle

dge

abou

t ass

essi

ng a

ndev

alua

ting

stud

ents

ski

lls a

nd p

rogr

ess

Sta

ff ha

s ba

sic

awar

enes

s an

d ab

ility

to r

each

and

coor

dina

te w

ith o

ther

sta

keho

lder

s an

dre

sour

ce e

ntiti

es

48

Sta

ff ha

s at

leas

t 1 y

ear

expe

rienc

e in

wor

kpla

ce s

ettin

gs, O

R is

hig

hly

expe

rienc

ed in

the

wor

kpla

ce s

ettin

g an

d is

wor

king

tow

ard

expe

rtis

e in

adu

lt ed

ucat

ion

met

hods

/ mat

eria

ls

Sta

ff pa

rtic

ipat

es in

pro

fess

iona

l dev

elop

men

top

port

uniti

es in

the

area

s of

cus

tom

ized

curr

icul

um d

evel

opm

ent,

stud

enke

nter

eded

ucat

ion,

and

/or

auth

entic

ass

essm

ent,

lear

ning

mor

e ab

out t

he c

ompa

ny, a

nd a

n aw

aren

ess

of th

cde

man

ds p

lace

d on

the

adul

t wor

ker/

stud

ent /

and

is b

egin

ning

to a

pply

new

idea

s in

thes

e ar

eas

Sta

ff ha

s so

me

expe

rienc

e in

wor

king

am

oniz

stva

ried

stak

ehol

ders

- S

6ff h

as e

xper

ienc

e te

achi

ng a

dUlts

in w

orkp

lace

set

tings

, and

hasl

expe

rienc

e w

orki

ng w

ith m

Ulti

-eth

nic

popu

latio

ns

- S

taff

has

expe

rienc

e an

d sk

ill in

teac

hing

the

rele

vant

sub

ject

mat

ter:

rea

ding

and

writ

ing,

ora

l com

mun

icat

ion,

mat

h, e

tc.

- S

taff

is s

kille

d in

cre

atin

g or

igin

al c

urric

ula

tailo

red

to e

ach

wor

ksite

and

res

pons

ive

to th

c go

als

of e

ach

grou

p of

stu

dent

s

- S

talf

is s

kille

d in

ass

essi

ng a

nd e

valu

atin

g le

arne

rs' s

kills

and

prog

ress

and

in te

achi

ng s

tude

nts

how

to e

valu

ate

thei

r ow

nle

ardi

ng

-S

taff

is s

kille

d in

wor

king

with

in a

bus

ines

s, u

nion

,or

labo

Om

anag

emen

t con

text

. Sta

ff ha

s so

me

expe

rtis

e in

gat

herin

gqu

alita

tive

and

quan

titat

ive

data

for

dive

rse

stak

ehol

ders

49

Sta

ndar

d #2

: Sta

ff is

com

pens

ated

at c

ompe

titiv

esa

lary

leve

ls fo

r te

achi

ng, c

urric

ulum

dev

elop

men

t, pr

ofes

sion

alde

velo

pmen

t, st

uden

t ass

essm

ent,

and

prog

ram

dev

elop

men

t.

Bas

icD

evel

opin

gE

xcel

lent

Sta

ff is

par

t-tim

e, w

ith p

ro-r

ated

ben

efits

Sta

ff is

pai

d fo

r co

ntac

t hou

rs a

nd a

t lea

stI

hour

of p

rep

time,

I ho

ur o

f cur

ricul

umde

velo

pmen

t tim

e, a

nd I

hour

of a

sses

smen

t and

docu

men

tatio

n lim

e fo

r ea

ch fo

ur c

onta

ct h

ours

wor

ked

(3:4

). A

t lea

st 2

.5%

of t

heir

time

isal

loca

ted

to tr

aini

ng a

nd d

evel

opm

ent.

At l

east

4.5%

of t

heir

time

is a

lloca

ted

topr

ogra

mpl

anni

ng, r

ecru

itmen

t, ev

alua

tion,

and

deve

lopi

nent

Sta

ff is

orie

nted

, inf

orm

ally

, hy

inte

rest

edst

akeh

olde

rs

- T

here

are

sta

ff de

velo

pmen

t opp

ortu

nitie

s

- S

taff

is a

mix

of f

ull-t

ime

and

part

-tim

e, w

ithpr

o-ra

ted

bene

fits

- S

taff

is p

aid

for

cont

act h

ours

and

at l

east

I

hour

of p

rep

time,

1 h

our

of c

urric

ulum

deve

lopm

ent t

ime,

and

I ho

ur o

f ass

essm

ent a

nddo

cum

enta

tion

time

for

each

four

con

tact

hou

rsw

orke

d (3

:4).

At l

east

2.5

% o

f the

ir tim

e is

allo

cate

d to

trai

ning

and

dev

elop

men

t. A

t lea

st4.

5% o

f the

ir tim

e is

allo

cate

d to

pro

gram

plan

ning

, rec

ruitm

ent,

eval

uatio

n, a

ndde

velo

pmen

t

A c

ompr

ehen

sive

sta

ff or

ient

atio

n to

the

wor

kpla

ce is

des

igne

d an

d pi

lote

d

- S

taff

deve

lopm

ent o

ppor

tuni

ties

incl

ude

aba

lanc

e of

wor

kpla

ce-s

peci

fic a

nd a

dult

liter

acy

topi

cs

Sta

ff is

a m

ix o

f par

t-tim

e an

d fu

ll-tim

e, w

ith fu

ll be

nefit

s.

- S

taff

is p

aid

for

cont

act h

ours

and

at l

east

2 h

ours

of p

rep

timc,

2 ho

urs

of c

urric

ulum

dev

elop

men

t tim

e, 1

hou

r of

asse

ssm

ent

and

docu

men

tatio

n tim

e, a

nd I

hour

of p

lann

ing/

adm

inis

trat

ive

time

for

each

four

con

tact

hou

rs w

orke

d (6

:4).

At l

east

2.5

%of

thei

r tim

e is

allo

cate

d to

trai

ning

and

dev

elop

men

t. A

t lea

st4.

5%of

thei

r tim

e is

allo

cate

d to

pro

gram

pla

nnin

g, r

ecru

itmen

t,ev

alua

tion,

and

impr

ovem

ent.

Thi

s in

clud

es a

ttend

ing

com

pany

-sp

onso

red

trai

ning

, com

mun

icat

ing

with

com

pany

per

Son

nel,

time

on th

e flo

or

- A

ll st

akeh

olde

rs p

artic

ipat

e in

offe

ring

a co

mpr

ehen

sive

sta

ffor

ient

atio

n to

the

wor

kpla

ce, i

ts c

ultu

re a

nd d

ynam

ics

- S

taff

deve

lopm

ent o

ppor

tuni

ties

incl

ude

spec

ializ

ed w

orkp

lace

educ

atio

n tr

aini

ngs

info

rmed

by

tren

ds in

the

indu

stry

, the

wor

kpla

ce, t

he la

bor

mov

emen

t, an

d th

e fie

ld o

f adu

lt ed

ucat

ion,

and

part

icip

atio

n in

in-h

ouse

em

ploy

ee tr

aini

ng. T

his

will

incr

ease

sta

ff's

unde

rsta

ndin

g of

bus

ines

s is

sues

, nee

ds, a

ndpr

oces

ses

Sta

ndar

d #3

: Sta

ffing

is a

dequ

ate

to c

over

the

task

s an

d fu

nctio

ns n

eede

d to

sup

port

stu

dent

sro

ugh

the

prog

ram

.

Bas

icD

evel

opin

gE

xcel

lent

Pro

gram

invo

lves

sel

ecte

d w

orkp

lace

per

sonn

elan

d or

ient

s th

em to

prin

cipl

es o

f adu

lt le

arni

ng

Sta

ff is

aw

are

of a

nd u

ses

refe

rral

net

wor

k fo

rsu

ppor

t ser

vice

s

ncei

r pr

am/ M

IMI

-Sta

ff ar

e kn

owle

dgea

ble

abou

t ref

erra

l net

wor

k

- P

rogr

am in

volv

es a

ran

ge o

f wor

kpla

ce p

erso

n-ne

l and

orie

nts

them

to p

rinci

ples

of a

dult

lear

ning

- S

taff

prov

ides

edu

catio

nal c

ouns

elin

gin

form

ally

and

use

s re

ferr

al n

etw

ork

1114

I F

- S

taff

is s

kille

d in

edu

catio

nal c

ouns

elin

g an

d us

es r

efer

ral

netw

ork

- P

rogr

am in

volv

es a

ll w

orkp

lace

per

sonn

el a

nd o

rient

s th

em to

prin

cipl

es o

f adu

lt le

arni

ng

- T

here

are

trai

ned

supp

ort s

ervi

ce w

orke

rs o

n st

aff (

coun

selo

rs,

day

care

wor

kers

, etc

.)

Appendix B

52

Massachusetts Workplace Literacy Consortium Year 3 Evaluation, October, 1997

NWLP, Wave 6, Year III Evaluationof the

MASSACHUSETTS WORKPLACE LITERACY CONSORTIUM

CPET Interview Probes

I. Organization/Governance

With regard to the organization and functioning of the CPET, how would youdescribe the role of a) the DOE, and b) the program coordinators?What were the major accomplishments of the CPET?

II. Consortium

Do you have a sense of belonging to a consortium?How much interaction or sharing of resources was there among the programcoordinators?

III. Institutionalizing Programs

At each of your sites, what have been the major forces for and againstinstitutionalization of workplace literacy programs?What role has the CPET or Consortium played in promoting institutionalization?

IV. PET

How much variation is there among the PETs across your sites?Based on your experiences, what are the characteristics of a good PET?

5 3

Massachusetts Workplace Literacy Consortium Year 3 Evaluation, October, 1997

Appendix C

5 4

Massachusetts Workplace Literacy Consortium Year 3 Evaluation, October, 1997

PET

Site

Vis

its

Site

Dat

eE

duca

tion

Prov

ider

Loc

atio

nIn

dust

ryI.

C &

K C

ompo

nent

sA

pril,

199

7Je

wis

h V

ocat

iona

l Ser

vice

sW

ater

tow

nM

anuf

actu

ring

2.M

assa

chus

etts

Gen

eral

Hos

pita

lA

pril,

199

7Je

wis

h V

ocat

iona

l Ser

vice

sB

osto

nH

ealth

Car

e

3.B

eth

Isra

el D

eaco

ness

/Chi

lren

's H

ospi

tal

Apr

il, 1

997

Jew

ish

Voc

atio

nal S

ervi

ces

Bos

ton

Hea

lth C

are

4.Se

rvo

lift /

Eas

tern

Cor

pora

tion

Apr

il, 1

997

Jew

ish

Voc

atio

nal S

ervi

ces

Dor

ches

ter

Man

ufac

turi

ng

5.Je

wis

h H

ealth

care

Cen

ter

May

, 199

7Q

uins

igam

ond

Com

mun

ity C

olle

geW

orce

ster

Hea

lth C

are

6.B

eaum

ont a

t the

Will

ows

May

, 199

7Q

uins

igam

ond

Com

mun

ity C

olle

geW

estb

oro

Hea

lth C

are

7.H

olyo

ke C

ard

& P

aper

Com

pany

May

, 199

7L

itera

cy V

olun

teer

Net

wor

kSp

ring

fiel

dM

anuf

actu

ring

Hea

lth C

are

8.C

arte

r Fu

ller

Men

tal H

ealth

and

AFS

CM

EM

ay, 1

997

Jew

ish

Voc

atio

nal S

ervi

ces

Bos

ton

9,L

ight

olie

r C

orpo

ratio

nJu

ne, 1

997

Lab

or E

duca

tion

Cen

ter

at U

nive

rsity

of

Mas

sach

uset

ts D

artm

outh

Fall

Riv

erM

anuf

actu

ring

10. U

nive

rsity

of

Mas

sach

uset

ts a

t Am

hers

tJu

ly, 1

997

Lab

or M

anag

emen

t Wor

kpla

ce E

duca

tion

Prog

ram

at U

nive

rsity

of

Mas

sach

uset

tsA

mhe

rst

Am

hers

tSe

rvic

e In

dust

ry

5655 M

assa

chus

etts

Wor

kpla

ce L

itera

cy C

onso

rtiu

m Y

ear

3 E

valu

atio

n, O

ctob

er,1

997

Appendix D

5 7

Planning and Evaluation Team Participation Chart.

After the interview meeting, Coordinator should complete the chart below for eachperson not present who participated on the PET during the past 12 months. Pleaseidentify a title category for each participant using the following code:

E = Education Provider W = Worker/StudentM = Management T = TeacherS = Supervisor 0 = Other (please specify)U = Union

NAME TITLE CODE(See List Above)

Currently onPET

# Years /Months

On PET

# Of PETMeetings Attendedin past 12 Months

Y N

Y N

Y N

Y N

Y N

Y N

Y N

Y N

Y N

Y N

Y N

Y N

Y N

N

Y N

Y N

Y N

56

NWLP, Wave 6, Year III Evaluation: PET SURVEY

NWLP, Wave 6, Year III Evaluationof the

MASSACHUSETTS WORKPLACE LITERACY CONSORTIUM

P.E.T. Survey

The Massachusetts Workplace Literacy Consortium is now in its third and final year of funding through theNational Workplace Literacy Program. The legislation that funds your program requires an externalevaluation each year. The purpose of the evaluation is to determine how the Consortium is meeting its goals.We hope that your participation in the evaluation will help you improve your program.

One focus of the evaluation is the effectiveness of the Planning and Evaluation Team (PET) model foradministering your workplace education programs. This survey contains two sets of questions related to theorganization and operation of your PET. The first set of questions contains group questions. PET membersanswered these questions during our visit to your PET meeting. Please respond to the group questions if youwere unable to attend that PET meeting. You may also respond to the group questions to add to theinformation provided at the PET meeting. The second set of questions contains individual questions. W eask all PET participants to complete the individual questions.

We ask you to answer all questions as completely as you can. We estimate that it will take you no more than20 minutes to complete the survey.

Only the external evaluators will view your completed survey. We will not identify individuals in thereporting of evaluation results. We will use current position and amount of PET experience only todetermine groupings for analyses. If you have any questions concerning the survey or the evaluation, pleasecontact Charles DePascale, one of the external evaluators at (508) 287-5170.

Thank you for taking the time to provide information about your PET. Please return your completed surveywithin the next week in the postage paid envelope provided to

Charles DePascaleDATA, Inc.PO Box 395

Concord, MA 01742

NWLP, Wave 6, Year III Evaluation: PET SURVEY

GROUP QUESTIONS

1. For each of the following PET Governance Tasks, indicate (../ ) who participated in each task.

PET Governance Tasks

PET includingCoordinator

Coordinatoralone

Other(Specify)

a. Develop project goals

b. Develop project plan

c. Review plan regularly

d. Develop project activities

e. Input into creating curriculum

f. Evaluate the PET

g. Develop an institutionalization plan

h. Overcome barriers to participation in PET

i. Overcome barriers to participation in classes

j. Provide incentives to program participation

k. Evaluate progress of student/worker

I. Evaluate impact of program on the worksite

m. Evaluate stability of the partnership

n. Other

2. The PET model was designed to be a democratic participatory approach to governance of a workplace education program.This model may not be appropriate for all individuals in a PET or for all companies/unions or volunteer organizations involvedin the MA Workplace Literacy Consortium. Please comment on the following

A. Does a team approach fit in well with your organization?

B. How has this democratic and open team approach to decision-making worked for your PET? (For example: Doworkers and supervisors each have equal voices in decision making? Has the model been myth or reality for yourPET?)

50

NWLP, Wave 6, Year III Evaluation: PET SURVE1/

3. What are the key factors that have contributed to the success of your workplace educatio program?

4. To what extent is the success of your workplace education program attributable to your PET? (Please comment)

1 2 3 5

no connection 100% result ofPET

5. What have been the barriers to the workplace education program?

6. To what extent has this PET successfully overcome the harriers? Please comment.

no success

7. What additional support and assistance could your PET have received from the folloqing groups:

5

100% successful

GROUP Please indicated what additional support was needed

Department of Education

Consortium PET

PET Coordinator/Education Provider

Other PET members

Business

Union

Other, please specify

8. If you were starting a new workplace education program, would you have a planning and evaluation team?

yes

NWLP, Wave 6, Year III Evaluation: PET SURVEI!

no Why?

9. What would you keep or change about your "future program" PET?

10. What steps, if any, is your PET taking to ensure the continuation of your workplace education program at the conclusion of the

current funding'?

11. What additional comments would you like to make regardintz the organization. operatioe, or effectiveness of your PET'?

62

NWLP, Wave 6, Year III Evaluation: PET SURVEY

Individual Questions

Current Position

Site Months on Planning and Evaluation Team

1. How many PET meetings have you attended during the past twelve months?

2. Indicate your level of involvement in the following PET activities on the following 4-point scale

1 = no involvement3 = moderate involvement

2 = slight/limited involvement= total involvement

PET Govetnance Tasks Level of Involvement

I. Develop project goals 1 2 3 4

2. Develop project plan 1 2 3 4

3. Review plan regularly 1 2 3 4

4. Develop project activities I / 3 4

5. Input into creating curriculum 1, 3 4

6. Evaluate the PET I / 3 4

7. Develop an institutionalization plan 1 ? 3 4

8. Overcome barriers to participation in PET 1 2 3 4

9. Overcome barriers to participation in classes 1 / 3 4

10. Provide incentives to program participation 1 ? 3 4

11. Evaluate progress of student/worker 1 2 3 4

12. Evaluate impact of program on the worksite 1, 3 4

13. Evaluate stability of the partnership I ? 3 4

14. Other 1 / 3 4

3. Do you feel that all relevant stakeholders (i.e., educators, workers/students, unions, management.supervisors. volunteers, etc.) are able to fully participate in PET discussions and decision-making?yes noIf no, which stakeholders did not fully participate?

4. Do students have a full voice in the activities of your PET?yes no

If no, why?

63

NWLP, Wave 6, Year III Evaluation: PET SURVEY

5. What does your PET do to encourage student participation and input into PET discussions anddecisions?

6. How would you describe the role of the representative of the education provider (PET ProjectCoordinator) on your PET?

7. Based On your PET experiences, what are the major pros and cons of the PET model in enhancing thesuccess of a workplace education program?

Pros Cons

8. Would you want to be part of a future Planning and Evaluation Team?yes no

Why or why not?

9. Has your PET received sufficient support and assistance from the following esoups:1= insufficient support in all areas 2=suficient support in some areas3=sufficient support in most areas 4=sufficient support in all areas

GROUP QUALITY OF SUPPORT

Department of Education 1 2 3 4

Consortium PET 1 2 3 4

PET Coordinator/Education Provider 1 2 3 4

Other PET members 1 2 3 4

Business 1 2 3 4

Union 1 2 3 4

Other 1 2 3 4

6 4

NWLP, Wave 6, Year III Evaluationof the

MASSACHUSETTS WORKPLACE LITERACY CONSORTIUM

PET Meeting Observation Form

Note:Table/Chair LocationObstaclesLightingDoors/Windows

I. Describe the meeting room in terms ofA. Style (board room, conference room, classroom, etc.)B. LocationC. Ambience (noise level, carpeting, comfort, etc.)

II. Speakers

Indicate (I ) which people speak at meeting

Program Coordinator Supervisory Staff Labor Union Representative

HR Manager Student/Worker Other (specify)

Corporate Management Teacher

III. Materials

Indicate (1) materials available at meeting

Agenda Meetinu4:.Minutes Teacher Reports Other Reports

IV. General Notes on Meeting IncludingA. Chair of meetingB. Organization of meetingC. Interactions among members

66

U.S. Department of EducationOffice of Educational Research and Improvement (OERI)

National Library of Education (NLE)Educational Resources Information Center (ERIC)

NOTICE

REPRODUCTION BASIS

ERIC

This document is covered by a signed "Reproduction Release(Blanket) form (on file within the ERIC system), encompassing allor classes of documents from its source organization and, therefore,does not require a "Specific Document" Release form.

This document is Federally-funded, or carries its own permission toreproduce, or is otherwise in the public domain and, therefore, maybe reproduced by ERIC without a signed Reproduction Release form(either "Specific Document" or "Blanket").

EFF-089 (9/97)