Articulating A Practice Model Of Service IntegrationAlberta College of Social Workers Annual Conference

2008

Scott Smillie, BSW; RSWChrista Gilroy, BSc., Human EcologyValerie Streit, BA.,RLSSusan Watson, BSc., Home Economics

Introductions: Round Robin

Workshop participants to introduce themselves, interest in the workshop topic, and their learning interests/objectives for this workshop.

Presenters Workshop Objectives:

Share our experiences and learning. Receive input into “The Model”.

Objectives

You Will: Learn about a community based research project,

focused on discovering the benefits of service integration for low-income families.

Explore the development process and how program development and community engagement fit within the process.

Hear about the complexities and key processes of community based research within the context of a paraprofessional, multi-sectoral, Randomized Control Trial

Objectives: cont.

Learn about the Families Matter Partnership Initiative (FMPI) program, practices conceived, (practice theories & principles adopted) and the need/importance of developing and providing a conceptual program model for management purposes.

Explore the concepts imbedded within the “Ecological Model of Service Integration” and provide input regarding its validity.

Historical Context

Dec. 2000 – Release of “Listen to the Children” by Quality of Life Commission.

June 2001 – Decision made to focus on improving existing services.

March 2002 – Workshop on “When The Bough Breaks” & “Benefitting all the Beneficiaries”.

August 2003 – Logic Model InitiatedJuly 2005 – RFP went out.Aug. 2005 – YMCA granted service delivery contract.January 2006 – FMPI started receiving referrals from research

recruitment and data collector team.

Alberta Context:Support to Low-Income Families

In Edmonton, about 27,000* families (15% of all Edmonton families) are considered low income, and struggle to make ends meet.

These families have limited resources, often barely enough to pay for food and shelter.

As well, they often have little opportunity to learn skills, access preventive health care, or enjoy family activities.

*Urban Poverty Project, CCSD, 2001

Support to Low-Income Families

Although there are agencies and supports available to help, these families also have difficulty accessing supports.

Transportation, awareness, language, and culture are among the barriers to accessing supports.

Families with low-incomes are also more likely to have other social and health needs.

Support to Low-Income Families

Interpersonal/family relationships, life skills, parenting…or just the ability to play. Difficulty with these issues can put stress on the whole family.

Supporting families can help build on their strengths, set goals, and make dreams come true.

But, how do we know what works best?

Who is Families First Edmonton?

Families First Edmonton is a partnership with two co-leads, Alberta Employment, Immigration & Industry, and the City of Edmonton Community Services.

They are joined by 10 other organizations, a service provider, and researchers from the Community-University Partnership at the University of Alberta.

Partners In Families First Edmonton

Project Co-Chairs: Alberta Employment, Immigration & Industry City of Edmonton Community Services

Service Delivery Providers: YMCA of Edmonton Multi-cultural Health Brokers Cooperative Bent Arrow Traditional Healing Society KARA Family Resource Centre

Members: Alberta Children’s Services Alberta Health and Wellness Alberta Mental Health Board Capital Health Edmonton Aboriginal Urban Affairs Committee Edmonton Community Foundation Edmonton & Area Child and Family Services Authority – Region 6 Quality of Life Commission United Way of the Alberta Capital Region University of Alberta – Community – University Partnership for the Study of Children, Youth, and Families

(4 Universities Involved).

What is Families First Edmonton?

Families First Edmonton is a $10 million research project to determine whether delivering health, family support and recreation services in a coordinated way can lead to better outcomes for families with low incomes. Families First Edmonton seeks to determine the most cost-effective, efficient use of resources to proactively assist families.

The research project is based on a similar study conducted in Ontario by Dr. Gina Browne. Her award winning study concluded that when families with low incomes are provided health, social, childcare and recreation in a comprehensive and combined way, families benefit more and governments realize cost savings. There was a desire to build on the Gina Brown study, but within an Alberta context.

Families First Edmonton will determine if similar positive results can be seen in Alberta with an adapted and enhanced approach, that takes into account the differences between Alberta’s and Ontario’s economy, social service environment, legislation, programs and processes.

What is Families First Edmonton? Cont.

FFE was seeded out of a broad realization that low-income families receiving income support are not connecting with other community services.

A research hypothesis arose that perhaps if we become more deliberate in reducing barriers to services for families by improving the integration of existing services, that families and their children would experience improved health and development outcomes.

A research framework and service delivery framework were developed to test key research questions.

Research Framework:

Involved key research questions:1. Does FFE service delivery model enhance families’ access to established

services?2. What are the effects of the service delivery model on the quality of life of

family, type of service accessed, and the level of satisfaction families feel toward the existing services?

3. What are the costs to service systems of each of the service delivery models over time?

4. What are the achievement outcomes of the children, and physical and psychosocial health outcomes of family members, over time, associated with each of these delivery models?

5. What are the role of problem solving skills, communication skills, and family connections to community on the intervention effects of four service delivery models, and how does this role change over time?

6. Do socio-demographic characteristics influence the family linkages to services and the outcomes in cost and health?

Research Approaches:

FFE is a Randomized Control Trial (evidence based)

FFE sought a sample size of 1200 families A Recreation Pilot was implemented and

utilized to base some research decisions upon.

Recruitment Targets and Strategies.

Participant Eligibility

• Eligible families contacted by mail at random• Completely voluntary

To be eligible, families must: have at least one child aged 0-12 years be receiving Alberta Works Income Support and/or Alberta

Child Health Benefit for the past six months live in designated area: north-central and north-east

Edmonton (This changed to city wide).

What Does Success Look Like?

Researchers will determine which approach most successfully achieves the following outcomes among participants in the project:

Increases appropriate access to existing social, health, and community services

Enhances physical and psycho-social health of the families

Increases attendance and achievement at school

Decreases the number of behavioural/emotional problems among children in the study

Increases self-reliance and enhances life management (e.g. grocery shopping, appropriate child care, etc.)

What Does Success Look Like? - Continued

Decreases use of emergency services (emergency medical services, hospital emergency, child welfare, police, etc.) and increases proactive use of health promotion services

Is most cost-effective by avoiding future costs or being cost-neutral, while achieving better results

Results in more efficient use of existing resources

Increases the ability of agencies and organizations to work together across sectors

Test 4 Service “Models”

1. The Family Healthy Lifestyle Program: The family will continue to receive the same services they do now plus a family support coordinator helps the family solve problems and link to health, education, child care, and family support services in the community.

2. The Recreation Coordination Program: The family will continue to receive the same services they do now plus a recreation coordinator helps place the children (0-17 years) into recreational programs with some funding provided.

3. The Comprehensive Program: The family receives the same services they do now plus a combination of the Recreation and Family Healthy Lifestyle Program

Test 4 Service “Models” - Continued

4. Self-Directed Group

The self-directed group will continue to access high quality services in the usual manner. Data will continue to be collected at intervals consistent with the other three groups.

The self-directed group will allow researchers to compare the delivery of the three service models to how services are delivered without them.

Service Provider

The YMCA is contracted to provide services to participating families.

The YMCA joined with the Multicultural Health Brokers Cooperative, KARA Family Resource Centre, and Bent Arrow Traditional Healing Society; to coordinate a comprehensive array of services to families involved with FFE. Together, they form “Families Matter Partnership Initiative.”

Service Delivery Framework:

Named FMPI (Families Matter Partnership Initiative)

Separated from influences of research side. Each program stream to be distinct. All programs shared common practices. 4 Practice Pillars in Common: family centred;

culturally sensitive; capacity building, reflective practice.

FFE models: assisted services

Family Healthy Lifestyle

• 300 families • Holistic Assessment• Supportive brokerage (service-links & follow-up)• Recreation access not funded• Family Support Coordinators

Recreation • 300 families • Rec. Assessment• Active outreach (placements, follow-up & funding) • Other needs self-directed• Recreation Coordinators

Models will enhance service integration and family capacity building

• 300 families • Holistic Assessment • Services based on needs• Rec Assessment• Placements, follow-up & funding• Family Support Coordinators

Comprehensive

Overview of Service Delivery Framework & Deliverables

A collection of theories

and principles from

various sources:

Type SourceEcological Perspective

Research Proposals

Best Practices:Reflective PracticeCapacity Bldg.Family Centred PracticeCultural Sensitivity

RFP

Project Charter

3 Distinct Service Streams

Recreation Pilot DocumentsResearch ProposalsRFPProject Charter

Service Delivery Deliverables – Common To Each Service Stream

Quick engagement of families Family Assessment Goal Setting/planning Progress Reports/monitoring Intervention Fidelity

Learning Process To Service Integration

Active Outreach ApproachMapping a Change Process and Goal Attainment with

NO timelines attached

Facilitating a Learning Process(AwarenessKnowledgePracticalAttitude)

** Notes:-Prior to Initial Contact Data has been collected and referral has been made to Coordinators

-Progress within Tiers is determined by level of need of the Families-Progress may vary – ie. May stay in Tier 1 continuously, may jump from Tier 1 to 3 or Tier 2 back to 1 etc.

-Exit/Entry may vary – may enter at Tier 3 or leave at Tier 2, etc.-Awareness, Knowledge, Skills and Attitude may be developed/pre-existing at any level

Tier 2(Emerging, Maintenance

& Self-Sufficient)

“I CAN”

Implementation- ie. Feedback from referring source

Ie. Involvement in programs

AwarenessKnowledge

SKILLSAttitude

Tier 1(Intense Engagement)

“I HAVE”

Initial InitialContact Assessment IPP Unique

Action Plan)

Secondary ReferralAssessment

Based on tthe PLM

AWARENESSKnowledge

SkillsAttitude

AwarenessKNOWLEDGE

SkillsAttitude

Tier 3(Transition & Closure)

AwarenessKnowledge

SkillsATTITUDE Wrap Up

- 3 months to endFamily interacting interdependently or

End of Research(reasons for leaving program may

vary)

“I AM”

Do these array of Practices, Theories and Deliverables Constitute a Service Delivery Model?

How does this framework help us to ensure intervention fidelity?

The information void!

Service Integration Spectrum

Helping Change Relationship Weighting

Personal Change Process

Hierarchy Of Needs

Service Integration Spectrum

Helping Change Relationship Weighting

Personal Change Process

Hierarchy Of Needs

The Ecological Perspective

“The ecological perspective uses ecological concepts from biology as a metaphor with which to describe the reciprocity between persons and their environments… attention is on the goodness of fit between an individual or group and the places in which they live out their lives” (Sands, 2001).

The Ecological Perspective: Brief History

In the mid 20th. Century, the Family Systems model was adopted, that incorporated a perspective that family members are influenced equally by environmental systems with equal power. This perspective has been expanded to “explain that an individual is ‘constantly creating, restructuring, and adapting to the environment as the environment is affecting them’ (Ungar, 2002). The systems approach now added the social elements to the interactive process. In the 1960’s and 1970’s, the systems theory was expanded based on an ecological approach, breaking down the term ‘environment’ into social determinants with varied levels of power and influence, as deemed by individual stress and need and level of connectedness. Unlike most behavioural and psychological theories, ecological theories focus on interrelational transactions between systems, and stress that all existing elements within an ecosystem play an equal role in maintaining balance of the whole.”

Systems in Environment:

Bronfenbrenner (1979), suggests four levels of ecological components as a useful framework in understanding how individuals or family processes are influenced by hierarchical environmental systems in which they function: the Microsystem, Mesosystem, Exosystem and Macrosystem.

The Relationship of the Ecological Model of Service Integration to the FFE Logic Model:

Families first is a large and complex project. The logic model itself, which is usually a visual schematic of a single page, is 17 pages for this project. The 4 overarching goals of the project encompass micro issues related to service interventions with families and macro issues such as informing policy changes and research. For any individual in the project, understanding ones role and relationship in relative to these larger goals can be mind boggling. The Ecological Model helps the front-line worker to conceptualize and link how their work at the microsystems level is linked to, and supports the broader goals of the project.

To be an effective link therefore, the principles encompassed in the model need to be congruent with the Logic Model. Such congruence can be found in many parts of the logic model document. For example:The first page of the logic model has a section called “Statement Of Need”. This is consistent with the ecological model premise that needs are an important area of focus for assessing conditions of well-being that the interventions seek to improve upon.

Relationship Continued:

In the Rationale: Notions of diversity, interconnectedness between families, staff and community; inclusive, mutually respectful and equitable communities, are all concepts that fit within an ecological model. p.2The notion of a relationship between “Outputs” and “Activities” that result in “Outcomes” speaks to the need for the strong capacity of community systems to support the interventions. This again demonstrates the relationship between private troubles and public issues and the fit for individuals in their environment. Families can not thrive in a community that doesn’t has the capacity to support them.“The basic premise of a community-based approach is the recognition that services should be accessible, continuous, comprehensive, and coordinated…..”p.2 A service delivery model outlines in a conceptual framework principles that respond to the need to ensure these community based approaches are honored. The coordinated piece of this, supports intervention fidelity.“Adequacy of social support is a very important foundation of family adaptation and hence child development”p.3. This notion of adequacy is linked to the degree of fit between individuals and their environment, which is an ecological concept.

In The Four Overarching Goals: The logic model outlines four overarching goals that illustrate a hierarchical conceptualization of systems related to improving upon service integration. The ecological model places the FMPI intervention at the micro, meso and exosystem levels without mitigating its interrelationship to the 4th. Goal which is clearly at the macro systems level.Goals of increased family capacity, sustained collaborations amongst systems stakeholders, improving community responsiveness to family needs, documenting practices, and informing research, policy and systems, all speak to an ecological approach to working with the range of issues families face.

Goal 1 and Practices:

The logic model outlines 1-2, 2-3, and 3-5 year outcomes for families, related to the goal of families gaining increased capacity to access services. This suggests a linear progression of change experienced by families.

The logic model describes these outcomes in a manner that reflects increased awareness, knowledge, skills and changed attitudes of families. The ecological model provides a visual schematic of this linear progression. This outcomes are targeted through a experiential learning approach with families and is reflected in the ‘Personal Change Process’ quadrant of the model schematic.

The logic model also acknowledges the importance of families moving from a state of dependence, to independence to interdependence with their environment. Short Term: “family accepts responsibility for some aspect of the linkage process”; Intermediate: “Family follows up on linkages on their own. ….Family is more connected in their community.” Long-term: “Family demonstrates sustained ability to apply learned problem solving skills. ….demonstrates increase self-confidence and efficacy.”p.8 These goal statements are reflected in the ecological model in the ‘Service Integration Spectrum’ quadrant as well as the ‘Helping Change Relationship Weighting’ quadrant of the model schematic.

What Benefit Does This Ecological Model Serve To The FFE Project

There are benefits to FFE at 3 levels:At the front-line worker level: Front-line worker are motivated to do this

work with families, in part because of the value of contributing towards research and potential outcomes that go beyond working with the family. These same workers however, may struggle with how their role contributes to the larger picture. Without this understanding, workers can lose perspective of the program parameters and lose their motivation to work within these parameters. At the service delivery level, a model serves the following purposes:

Provides a contextual framework with which to train workers and support their work with families.

Clarifies outcomes that the workers practices aim to achieve. Provides a reference point to start family work and a process and check points

along the way to assess and reflect on this work. The model can articulate the broader areas of practice domains from where

service delivery strategies are framed.

Front-line Workers Continued:

The model helps to differentiate an ecological approach from other philosophies that are perhaps more focused on families as independent units in isolation from their communities.

Models are value and philosophically based and the ecological model encompasses many of the practice principles stakeholders felt were important. A conceptual model helps workers to understand these sets of values and practice principles. (see appendix for description of theories and principles).

The model hypothesizes that family needs can change and evolve within a supportive learning environment, where the degree of engagement and the roles in the reciprocal relationship between worker and client changes as families move through a spectrum of empowerment and healthier inter-dependence with their communities. Using an assessment strategy, this model attempts to measure this change process.

The model provides the opportunity to measure and inform the change process, providing the necessary tools for case management and caseload management. Imbedded within these processes is the notion of Intervention Fidelity, that is, ensuring a consistent quality and measurable degree of service for families.

Benefits continued:

At the Community Stakeholder Level: (including funders, service delivery partners, etc.)

It provides awareness of the values, practice principles and theories with which the service providers are operating from and delivery interventions.

It builds confidence that ‘good practices’ are being incorporated and the service delivery providers have the necessary knowledge and skill base to provide services.

The model provides an opportunity for examination and engagement into the area of developing innovative practices.

Benefits continued:

At the Research Level: the conceptual model provides the following: Provides the research partners with the broader practice framework that service

delivery is working from. A broader service delivery framework, described in a conceptual model acts as a

reference point to evaluate the service delivery teams understanding of intervention goals in relationship to the Logic Model, conceived by a broader set of stakeholders.

This degree of understanding is important to determine how service delivery activities will support the outcomes listed in the Logic Model.

The ecological model also helps to clarify disparities in other documents. For example, the dose of intervention (stipulated in the RFP) appears to be linked to the notion of linking families to services while the logic model supports practices associated with a learning process with families that develops their capacities to be less dependent upon services. This latter service objective is not reflected in the smaller service doses prescribed and associated simply with linking families to services. The ecological model to service integration bridges both of these objectives, dispelling any confusion that service dosage parameters may imply that capacity building and service integration are some how unrelated concepts.

What is Intervention Fidelity?

“the adherence and competent delivery of an intervention by the interventionist as set forth in the research plan”

Source: Santacroce, Maccarelli & Grey, 2004, p. 63

Components of Intervention Fidelity

Adherence (or integrity)The degree to which a given intervention is implemented in accordance with essential theoretical and procedural aspects of the model

CompetenceThe level of interventionist skill in utilizing core intervention techniques and responding to the unique needs of each participant

DifferentiationThe theoretical distinctiveness of the model’s main intervention principles in comparison to those of other models of interest or within efficacy trials, in comparison to those of competing study conditions

Researcher questions directed at establishing intervention fidelity for the FFE service integration practices:

Adherence:

What are the practices associated with the awareness, knowledge, skills and attitudes of FFE service integration?

Are practices implemented consistently? How do the key elements of FFE appear in the service

integration practices: being family centred, culturally responsive, building capacity, and reflective practice?

Researcher questions directed at establishing intervention fidelity for the FFE service integration practices:

Competence:

How do the interventionists implement the practices associated with the awareness, knowledge, skills and attitudes of FFE service integration?

How do the interventionists focus on the key elements of FFE service integration in their work?

How do the interventionists use their uniqueness in their implementation of the FFE service integration practices?

How does Families Matter support the interventionists in their service integration practices?

Researcher questions directed at establishing intervention fidelity for the FFE service integration practices:

Differentiation:

Are each of the FFE service integration vehicles (Family Healthy Lifestyle and Recreation Coordination Programs) distinct?

How does Families Matter keep the FFE service integration vehicles distinct?

What Is Intervention Fidelity in Summary

Intervention fidelity is about doing what you said you would do.

Intervention fidelity is about protecting your interventions from influences that can “water down” or “change” the impact of the intervention.

A lack of intervention fidelity therefore, makes it hard to determine if the intended interventions can be attributed to causing the measured outcomes.

The strength of a randomized control trial is “controlling” the amount and type of intervention, so that outcomes aren’t attributed to other variables (in-deliberate or unplanned for activities/influences), which creates “Rival Hypothesis”.

What we want to accomplish is to create as much certainty as possible about which practices/interventions resulted in which outcomes. Without this certainty, it is difficult to replicate the interventions. The short coming of some similar studies has been that even though positive outcomes were noted, the research couldn’t clearly demonstrate the practices that led to those outcomes, making it difficult to replicate. It also makes it difficult to confidently be able to generalize the findings to those outside of the study, who we wish to serve, weakening the impact from a public policy perspective.

Why Is Intervention Fidelity Important?

Increases the likelihood of an effect for FFE through:

Internal Validity and the ability to attribute the effect of FFE to the FFE Interventions.

External Validity and the ability to generalize effects beyond the project participants

Increases the validity of findings that helps to answer the original Research Questions

Supporting Methodology Increases retention Increases Confidence of Stakeholders Replicability/Transferability of FFE

Why Is Intervention Fidelity Important?

Improves FFE intervention through the discipline of consistent monitoring and feedback

Is the foundation of evidence-based practice

Promotes the avoidance of harm to partners/participants by:– Improving outcomes– Increasing the ethical component– Reducing unintended consequences– Enhancing reputations/confidence

Project 5, FFE Models, Approaches, and Practices, is a program evaluation that includes intensive analyses of the implementation of the different service-delivery models. These analyses are critical for determining the fidelity and effectiveness of the service-delivery models, and also for documenting practices (including engagement and retention) so that the lessons learned can be maintained and transferred (Appendix 1E).Drummond, J.E., Family First Edmonton, Year 1 Research Proposal, p.12b

Replicability Is A Goal Of FFE

Importance of Intervention Fidelity

In short, Intervention Fidelity can best be achieved by ensuring all stakeholders are on the same page. The logic model as a tool, is the visual schematic that describes the nuts and bolts of the intervention. As a process, logic modeling reinforces continuity and collaboration that supports clarity, reinforces adherence in the implementation of program interventions, and the differentiation of different interventions being tested. (The FFE Logic Model is a 17 page document that does not offer a clear visual schematic).

Variables Needing To Be Managed To Ensure Intervention Fidelity

Human Resource Management Distinct Programs with Common Practices Equitable caseload management, ensuring

‘dose’ of intervention, mitigating worker burden/bias, inconsistent practices, etc.

Consistent practices – data monitoring. Clarity of outputs and outcomes.

Intervention FidelityAdherence & Competence:

Best Practices

3 distinct programs with Common Practices

The 4 common practice pillars, (principles) include:

Family Centered Culturally Sensitive Reflective Practice Capacity Building

Family Centered

Families First Edmonton supports families in defining their family, based on their own cultural, religious, and personal experiences.

The strength-based model focuses on family capacity and assets rather than deficiencies and problems.

Family Centered

Working with families from a family centered approach means different things across the 3 programs. Challenges are different for each program.

Rec/FHL: it is more challenging to work with families from a family centered approach within the confines of the research project for these 2 program areas

Comp: Less challenging because there is more flexibility to work with families in the areas that they choose (no need to stay with in ‘rec’ or ‘health’ boundaries

Culturally Sensitive

This area is similar across the 3 service delivery program areas. Supervisors and staff use reflective practice in one on one supervision situations and peer support to talk about different ways of being culturally sensitive.

We have received training from our different partners around cultural sensitivity as well.

Culturally Sensitive – Cont.

Two of our partners (MCHB and Bent Arrow) who have helped with Cultural Sensitivity training and practices

Challenges: Families needing cultural brokering have

resulted in different timelines Staff with different cultural backgroundsLearnings: CPEP Internship Pilot

Reflective Practice

Checking biases- individual biases

With in the different program areas there are sometimes worker perceptions that families don’t want to be in a specific program (ie, health) however it is important to focus on what we do have to offer (capacity building). Workers need to be able to have self awareness and check their biases so that they do not influence the perceptions of the family negatively

Reflective Practice Cont…

Reflective Practice is used through out the project:

by the frontline workers with families, in peer support/debriefing, by supervisors as a tool to reinforce

consistent practices, in case conferences

Capacity Building

Easier to build on the strengths of the family in comp- doesn’t matter what areas the strengths are in

All 3 programs work from a capacity building approach

Practices

These ‘Best Practices’ or Practice Principles help us to be intentional about our practices in our work with families and with each other.

Case Conferencing

Case Conferencing is used by supervisors and staff as an opportunity for resource sharing and reflective practice

Staff present individual cases and then have round table discussion with whole team about their work with the family

Opportunity for all staff to hear what workers have tried, what has worked/not worked, what resources are available, how they can be reflective in their work to ensure that they are approaching difficult situations from family centered, capacity building approaches

Training

Training is an important way of ensuring intervention fidelity across the 3 program areas.

Staggered hire has led to a need to re-create a training plan. The original training plan did not account for hiring spread over such a long period of time.

Repeating some training at frequent intervals and have staff shadow one another have proven to be the most effective training methods.

Practice Theories

Readiness for ChangeMotivational Interviewing

We have done training on both of these theories and they are consistent with the Ecological Model for Service Integration and work well with the idea of flow between Awareness, Knowledge, Skills, and Attitudes

Intervention Fidelity

…from a TRACKING and DEVELOPMENT perspective

**the tracking and development area supports all 3 areas of:

distinction, adherence and competence**

Tracking and Development Variables – Written Processes and Subjectivity

Variable = Lack of written processes leave staff with no base reference

Implications: - staff tend to guess or follow another staff’s lead (could be incorrect).

- can result in the same process entered in different ways

Variable = Subjectivity is a reality with many staff entering in 1 database

Implications: - data fields and drop down choices can be interpreted in numerous ways.

- Time is needed to determine methods to out-rule subjectivity (ie. rating exercises).

- Subjectivity can reduce validity of data

Tracking and Development Variables – Code Book

Variable = Lack of reference to defined terms or a code book attached to the Database can lead to misinterpretations for both inputs and outputs

Implications: - data fields and choices to enter in fields (ie. drop down boxes) can be interpreted in numerous ways

both for input and output of reports.

- a code book should be referenced as to how the field is defined for data entry.

- When reports are being interpreted, a code book should be referenced as to what columns each

individualized column means.

- Without these tools, validity of the data is reduced

Tracking and Development Variables – Database Development and Orientation

Variable = Learning and DEVELOPING the database to the needs of the project DURING the project can cause inconsistency in data entry and processes

Implications:

extreme amounts of time and resources were required to learn and develop (ie. customize) the layers of the database.

danger of doing this learning and development as we are already working with families is the impact on consistency.

large emphasis placed on audits to ensure processes are being used for entry into the database in a consistent manner

challenge of doing this work ahead of time is knowing what scenarios we would face with the families.

Tracking and Development Variables – Database Development and Orientation cont…

Variable = Time in the beginning and at various intervals (due to staggered

hiring process or staff turnover) is required to provide ORIENTATION to staff to

the HOMES database

Implications: - requires time from supervisors and co-workers to orientate staff to database…at times may not happen in a timely

manner

- multi- layers within database can be overwhelming

- feedback = learn various processes as needed – otherwise all can be forgotten

- HOMES has numerous navigations to do the same process which can cause confusion

Tracking and Development Variables – Data Entry – Timely and Accurate Entry

Variable = Data at times may not be entered in a timely or accurate manner

Implications: - results in reports being invalid and represents inconsistent intervention - impacts reports in such a way that does not represent all

families in study. - again added emphasis on audits is critical to check for

such variables to data entry and tracking.

Tracking and Development Variables – Data Entry – Overload

Variable = Amount of information to gather can lead to data not being entered (missed) into the database (overload)

Implications: - data missed will not be represented in reports- checklists have been created to assist staff with

complete data entry- waiting for information to be uploaded into

HOMES can result in connecting information being missed (ie. referrals and links, Rec Bank)

**timeliness of entry, accuracy and overload variables are all impacted by caseload size, amount of data , ease of inputting, checklists for inputting

Hiring and Retaining Staff to Fit The 3 Program Areas

FHL Staff

Social service background Focus on connecting to services not

providing the service Staff must recognize that FHL has a lot to

offer and be able to stay in the program boundaries to ensure intervention Fidelity

Computer and Administrative Skills

Recreation staff

• Recreation background and interest• Must be able to stay in boundaries• Computer skills• Additional Administrative burden due to

dollars attached to each child, payments and tracking

Comprehensive staff

• Generally need social services back ground but also an interest in recreation (multidiscipline)

• Computer skills• Additional administrative burden due to

dollars attached to each child, payments, tracking etc.

Changes over time

Lots of evenings as demographics of families changed when criteria changed– More working poor and students

Project procedures and tracking evolving and changing

Training plan based on guiding principles set up but had to be revised with staggered start

Impact of Evolution of the Project

High staff turn over can impact outcomes As project evolved the kind of person hired changed

which can have an effect on intervention fidelity Staggered recruitment of families and staff can

impact caseload management. (higher caseloads, more new families)

Staggered recruitment of staff which can impact consistency of processes/data entry

Why The Staff Turnover

Labor market Big job doing research and service delivery When project extended there was more staff

turnover

Wages linked to organizational grids versus market competition

Retaining qualified staff

YMCA – Recognition– Team building– Benefits

Opportunity to get good training and experience Exposure to community Opportunity to attend workshops and presentations

Retaining qualified staff cont.

Nurture interest in research– Focus groups– Researchers attend meetings– Presentations by CUP– Sense of contributing to the outcomes as well as

service delivery

Maintaining Intervention Fidelity During Project Close Out

Project has end date unlike many other jobs therefore well trained staff looking to move to more permanent jobs– Surplus from staggered hiring process at start up

used for retention and employment transition – Impact on intervention fidelity if staff leave before

close out so risk needs to be minimized – Case load size will decrease but dose must remain

the same cannot spend more than 1 hour a month or every 3 months, depending upon program

Complexities Of Articulating A Model

Different Professional Cultures The question of being Community-based (who owns

what?) Historical disjointing Pre empting Research versus the need to meet

Research needs for rigor Complex Structure Funding Issues Time restraints

Learnings:

Research needs to be engaged in the RFP process Service Delivery needs to be engaged in the planning process but RFP process

doesn’t afford this opportunity Research and Service Delivery have common work that needs to be well

planned and part of a development phase prior to implementation Data collection systems need to be well researched and not only funder driven Inferences made from pilots need to be measured A logic model and logic modeling process should occur with all “current”

stakeholders to ensure everyone is on the same page and should occur at the inception of the project

Concepts of Community Based Research need to be clear and support issues of collaborative capacity development that reflects mutual knowledge sharing and paraprofessional contributions to the process and learning.

Feedback on Model