Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
FY 2017 MDHHS Deliverable Submission Form
Name of Project: Oral Health Evaluation
Document Name: Evaluation Plan for the Year 1 Implementation of the 2020 Michigan State Oral Health Plan
Project Number: G-64928-11
Program Name: Center for Child and Family Health
Director or
Coordinator Name:
Diane Wing, Project Coordinator
Submitted By: Diane Wing
Type of Deliverable: (Please check one)
Final report
Specialized report
Publication
Articles published
Brochures
Software application
Web screen shot
No report, publication, or study
Deliverable pending
OTHER material
If OTHER, please describe:
Evaluation Plan
0
Prepared by:
Center for Child and Family Health
Michigan Public Health Institute
Evaluation Plan for
Year 1 Implementation of the
2020 Michigan State Oral Health Plan
1
Contents Introduction ................................................................................................................................................ 2
Plan Description ......................................................................................................................................... 2
Evaluation ................................................................................................................................................... 3
Purpose .................................................................................................................................................... 3
Evaluation Team .................................................................................................................................... 3
Evaluation Questions ............................................................................................................................ 3
Evaluation Design .................................................................................................................................. 4
Data Collection ........................................................................................................................................... 5
Data Analysis .............................................................................................................................................. 5
Data Interpretation .................................................................................................................................... 5
Data Dissemination .................................................................................................................................... 6
Appendix A: Logic Model for the 2020 Michigan State Oral Health Plan ........................................ 7
Appendix B: Key Partner Survey ............................................................................................................ 8
2
Introduction The Michigan Department of Health and Human Services (MDHHS) was awarded a five-year
grant (2013/2018) from the Center for Disease Control and Prevention (CDC). The purpose of
the funding was to support Michigan in strengthening and maintaining a strong oral health
program. As recommended by the CDC, one of the core aspects of a strong oral health program
is the development of an oral health plan, as it provides a framework for action to
collaboratively accomplish the goals and objectives that will work to improve oral health in
Michigan.
The MDHHS contracted with the Center for Child and Family Health (CCFH) at the Michigan
Public Health Institute (MPHI) to implement a one-year evaluation (2016/2017) of stakeholder
implementation of the State Oral Health Plan. The purpose of the evaluation is to assess
stakeholders’ engagement in implementing strategies within the plan as well as to monitor
progress toward goal achievement within the first year of plan dissemination. A participatory
approach will be used in the evaluation, as the evaluators will work closely with project
partners and other stakeholders to ensure that the evaluation objectives and methods align with
the project goals and that useful results are produced.
Plan Description The MDHHS Oral Health Program and the Michigan Oral Health Coalition (MOHC) worked
together to develop a five-year state oral health plan with the ultimate goal of achieving optimal
oral health among all Michiganders. Key oral health partners and stakeholders across the state
worked together to develop the plan; establishing goals, implementation steps, and a
monitoring plan. The plan additionally serves as a tool for identifying and enlisting
collaborators and partners needed to ensure its success. The process of developing the plan
began in April 2015 and continued through to December 2015 when the final plan was
completed.
Through an action-planning process, participating stakeholders and key oral health partners
developed a practical vision for the plan and identified three key priority areas in which to
focus. These priority areas were refined to establish clear goals, create SMART objectives, and
to identify activities and strategies to focus the work of stakeholders and key oral health
partners throughout the state. Progress towards these goals and objectives will be reviewed on
an annual basis and shared with key oral health partners. The intended outcomes of the plan
are to increase the number of stakeholders that work to enhance professional integration
between providers across the lifespan, increase knowledge and awareness of the importance of
oral health to overall health, and to increase access to oral health care among underserved
populations with a long-term outcome of achieving optimal oral health for all Michiganders.
The logic model for the State Oral Health Plan is presented in Appendix A.
3
Evaluation Purpose
This evaluation will focus on the implementation of the 2020 Michigan State Oral Health Plan
by assessing stakeholder engagement in carrying out the plan within the first year since plan
dissemination. In addition, progress toward goal achievement will be monitored for select
strategies to establish baseline measures for plan monitoring. The information gathered
through this evaluation will be used to inform the MDHHS Oral Health Program and key
partners of stakeholder engagement and of progress achieved within year one of the plan and to
identify any potential areas for improvement.
Evaluation Team
The evaluation team consists of staff from the MDHHS Oral Health Program (OHP) and
program staff from the Center for Child and Family Health (CCFH) at the Michigan Public
Health Institute (MPHI). Staff from the OHP will provide feedback on the evaluation design
and planning and provide input on the interpretation of evaluation findings. Staff from MPHI
will lead the evaluation and ensure that evaluation activities are conducted as planned. Table 1
presents the roles and responsibilities of the evaluation team.
Table 1: Roles and Responsibilities of Evaluation Team Members
Individual Role Responsibilities
MDHHS OHP Staff:
Chris Farrell, Susan Deming,
Sandy Sutton, Jill Moore, Beth
Anderson
Stakeholder/Funder
Provide feedback on evaluation design
and planning and provide input on
interpretation of findings
Coordinate data collection with key oral
health partners
MPHI CCFH staff:
Crystal Tyler, Diane Wing,
Morgan Richardson
Evaluator, Data Analysis
Oversight of all evaluation activities to
ensure the evaluation is conducted as
planned
Gather and analyze data
Compile results and write final report
Evaluation Questions
As the 2020 Michigan State Oral Health Plan is in its first year, the evaluation will consist of
process evaluation questions addressing stakeholder engagement and short-term outcome
questions to establish baseline measures for plan monitoring.
Overarching evaluation question: In the first year since plan dissemination, how engaged are
stakeholders and key oral health partners in implementing the 2020 Michigan State Oral Health
Plan?
In 2016, how many key oral health partners reported reading the 2020 Michigan State
Oral Health Plan? If they didn’t, why not?
4
In 2016, to what extent did key oral health partners feel their organizations’ goals
aligned with the goals in the 2020 Michigan State Oral Health Plan?
In 2016, did key oral health partners use the 2020 Michigan State Oral Health Plan as a
source to make decisions or develop plans for action?
In 2016, how was the 2020 Michigan State Oral Health Plan used to make decisions or
develop plans for action?
Short-term outcome questions:
Professional Integration
How many oral health partners had a formal relationship with providers from different
healthcare disciplines?
Has the number of formal relationships increased since the release of the plan?
How many key partners promoted the viewing of one or more Smiles for Life online
training modules?
Has the number of key partners who promoted viewing of the Smiles for Life modules
increased since the release of the plan?
How many key partners viewed one or more Smiles for Life online training modules?
Has the number of key partners who viewed one or more Smiles for Life modules
increased since the release of the plan?
Health Literacy
How many key partners use social media to educate providers/consumers on oral
health?
Have key partners use of social media to educate providers or consumers on oral health
increased since the release of the plan?
How many key partners shared social media messaging from the MDHHS OHP and /or
MOHC?
How many key partners worked to educate pregnant women on oral health preventive
measures?
How many key partners worked to create and support county advocacy networks across
the state of Michigan?
How many key partners collaborated with Michigan’s public school districts to increase
oral health awareness activities?
Evaluation Design
A non-experimental study design will be used for the evaluation. Data gathered will be used to
assess stakeholder engagement and monitor the implementation of activities outlined in the
2020 Michigan State Oral Health Plan.
5
Data Collection A key partner survey will be used to assess stakeholder engagement and to monitor progress
toward goal achievement for select strategies outlined in the plan. The OHP Director will email
the electronic survey to key oral health partners via Survey Monkey in February of 2017. The
purpose of the survey will be to gain a better understanding of how key partners used the State
Oral Health Plan in the past year (2016), whether or not it was used to develop plans for action,
and what parts of the plan, if any, key partners have implemented. To establish baseline
measurements for plan monitoring, survey participants will be asked whether or not they
implemented specific strategies. Members from the evaluation team will export the data from
Survey Monkey into a Statistical Package for the Social Sciences (SPSS) database. The survey
tool is attached in Appendix B.
Data Analysis Both quantitative and qualitative methods will be used in data analysis. The Statistical Package
for the Social Sciences (SPSS) will be used to conduct analyses on quantitative data. Descriptive
statistics, including frequency distribution and central tendencies, will be used to report
findings on stakeholder engagement and short-term outcomes. Responses to open-ended
questions will be assessed to identify patterns or trends.
Data Interpretation The MPHI evaluation team with the OHP Program Director and staff will meet to interpret the
evaluation findings. From this meeting, evaluation findings will be finalized and
recommendations will be made accordingly. Table 2 presents the timeline for evaluation
activities.
Table 2: Timeline for Evaluation Activities
Dec Jan Feb Mar Apr
2016
Evaluation Planning X
Survey Tool Development X
2017
Finalize Survey Tool X
Email survey to key partners X
Analysis of key partner survey data X
Develop success story draft X X
Final success story/dissemination X
6
Data Dissemination The evaluation team at MPHI will provide monthly updates on evaluation activities to the OHP
Program Director and staff through monthly conference calls. A success story formatted report
will be developed and delivered to the OHP in April, 2017. The success story will be
electronically delivered to key oral health partners and will be made available on the MDHHS
OHP website.
7
Appendix A: Logic Model for the 2020 Michigan State Oral Health Plan
Inputs Activities Outputs Long-Term
Outcomes
CDC
MDHHS Oral
Health Program
(OHP)
Broad array of
stakeholders
Data on oral
health indicators
in Michigan
Stakeholders are
surveyed to
identify cursory
focus areas
Stakeholder
meeting is
convened to
initiate strategic
planning
Oral health burden
is assessed
Oral health mission
is established
Potential goals and
objectives are
established
Plan
implementation is
evaluated and
progress towards
goals achievement
is monitored
Stakeholders
review draft of
plan and submit
feedback to
MDHHS OHP
State Oral Health
Plan, supported
by stakeholders, is
complete
State Oral Health
Plan is presented
and disseminated
(e.g. Michigan
Oral Health
Coalition Spring
meeting, MDHHS
website)
Intermediate
Outcomes
Short-Term
Outcomes
Increase in the
number of
stakeholders that
work to:
Enhance
professional
integration
between
providers across
lifespan
Increase
knowledge and
awareness of
importance of
oral health to
overall health
Increase access
to oral health
care for
underserved
and/or hard to
reach
populations
Enhanced
professional
integration
between providers
across the lifespan
Increased
knowledge and
awareness of the
importance of oral
health to overall
health
Increased access to
oral health care
among
underserved
and/or hard to
reach populations
By 2020,
Michiganders
have the
knowledge,
support and
care they need
to achieve
optimal oral
health
8
Appendix B: Key Partner Survey
The 2020 Michigan State Oral Health Plan, which was released in January 2016, is adocument designed to guide policy makers, providers, public health workers,community members and other stakeholders through the next 5 years as they workto improve oral health in Michigan. As it has been a full year since the 2020 StateOral Health Plan was released, we would like to ask some questions of key oralhealth partners on how the plan was used in 2016. These questions will inquire onthe use of the State Oral Health Plan but also on some specific strategies outlined inthe plan.
The survey will take approximately 10 to 15 minutes to complete. Your answers willbe kept confidential and combined with responses of other survey participants. Your open and honest answers are important to the yearly monitoring of planimplementation and to evaluating the overall progress toward accomplishing thegoals outlined in the State Oral Health Plan.
If you have any questions about this survey, please contact Chris Farrell, OralHealth Program Director at the Michigan Department of Health and Human Servicesat (517) 335-8388 or at [email protected].
Introduction
Michigan State Oral Health Plan Key Partner Survey
1. Would you like to participate in the survey?
Yes
No
1
Michigan State Oral Health Plan Key Partner Survey
2. How would you describe yourself or the organization you represent? (please select only one)
Coalition or Council
Community-based organization
Educator
Government or Policymaker
Insurer or Third Party Payer
Professional Organization
Provider/Health Care Professional
Public Health Agency
Other (please specify)
3. Are you or your organization a member of the Michigan Oral Health Coalition?
Yes
No
2
Next we would like to ask you about how you used the State Oral Health Plan in the past calendaryear (2016).
Michigan State Oral Health Plan Key Partner Survey
4. Have you received a copy of the State Oral Health Plan (either in print or electronically)?
Yes
No
I don't know
5. Have you had the opportunity to read the State Oral Health Plan?
Yes, all
Yes, most
Yes, some
No, not at all
3
Michigan State Oral Health Plan Key Partner Survey
6. How aligned or not aligned is your or your organization’s work with the State Oral Health Plan?
Completely aligned
Mostly aligned
Somewhat aligned
Not at all aligned
I don’t know
4
Michigan State Oral Health Plan Key Partner Survey
7. Please check the areas in which there is alignment between your or your organization’s goals and theState Oral Health Plan’s goals (please check all that apply).
Professional Integration
Health Literacy
Increasing access to oral health care
5
Michigan State Oral Health Plan Key Partner Survey
8. In the past year (2016), have you personally used the plan to make decisions or develop plans foraction?*
Yes
No
6
Michigan State Oral Health Plan Key Partner Survey
9. Can you briefly describe how you or your organization used the plan to make decisions or developplans for action? Please provide a short paragraph as to what was accomplished, the purpose of thework, the target population, and whether there was a start and end date or if this is ongoing work.
10. Which overall goal of the State Oral Health Plan did this focus on? (please select one goal it mostapplies to)*
Professional Integration
Health Literacy
Increasing access to oral health care
7
Michigan State Oral Health Plan Key Partner Survey
11. Within the professional integration goal, which specific objective did this relate to? (please select all thatapply)
Increase the number of health care providers who have formal relationships
Increase the number of educational opportunities that allow oral health and other health providers to work as a single team toaddress patient needs
Establish an equitable payment rate for oral health services among both medical and dental providers
Update oral health guidelines within the Michigan Quality Improvement Consortium (MQIC) guidelines
Increase the number of programs that educate oral health providers on the social determinants of oral health among underservedpopulations
None of these
8
Michigan State Oral Health Plan Key Partner Survey
12. Within the health literacy goal, which specific objective did this relate to? (please select all that apply)
Develop and promote consistent messages to educate providers and consumers on oral health through the internet
Increase the number of programs and/or interventions that educate parents on how to prevent early childhood caries amongchildren 0 to 3 years
Increase consumer and health care provider use of evidence-based prevention strategies
Create and support county advocacy networks across that state of Michigan
Collaborate with Michigan's public school districts, including school health and safety programs and school based health centers,to increase oral health awareness activities
None of these
9
Michigan State Oral Health Plan Key Partner Survey
13. Within the increasing access to oral health care goal, which specific objective did this relate to? (pleaseselect all that apply)
Decrease the number of children, young adults, adults, and older adults who are underinsured or without dental insurance
Reduce the number of children, young adults, adults, and older adults who experience difficulty, delays or barriers to receivingoral health care
Increase the number of infants, children, and young adults who receive comprehensive dental services
Increase the number of children and adults with disabilities who receive comprehensive dental services
Increase the number of pregnant women who receive comprehensive oral health care
None of these
10
Michigan State Oral Health Plan Key Partner Survey
14. Are you or your organization willing to use the State Oral Health Plan as a source to make decisions ordevelop plans for action?*
Yes
No
11
Michigan State Oral Health Plan Key Partner Survey
15. Please describe what you or your organization would like to work on in 2017. For example, in a fewsentences describe the work you would like to accomplish, the purpose of the work, the intended targetpopulation, etc.
16. Which overall area of the State Oral Health Plan would this focus on? (please select one goal it mostapplies to)*
Professional Integration
Health Literacy
Increasing access to oral health care
12
Michigan State Oral Health Plan Key Partner Survey
17. Within the professional integration goal, which specific objective would this relate to? (please select allthat apply)
Increase the number of health care providers who have formal relationships
Increase the number of educational opportunities that allow oral health and other health providers to work as a single team toaddress patient needs
Establish an equitable payment rate for oral health services among both medical and dental providers
Update oral health guidelines within the Michigan Quality Improvement Consortium (MQIC) guidelines
Increase the number of programs that educate oral health providers on the social determinants of oral health among underservedpopulations
None of these
13
Michigan State Oral Health Plan Key Partner Survey
18. Within the health literacy goal, which specific objective would this relate to? (please select all thatapply)
Develop and promote consistent messages to educate providers and consumers on oral health through the internet
Increase the number of programs and/or interventions that educate parents on how to prevent early childhood caries amongchildren 0 to 3 years
Increase consumer and health care provider use of evidence-based prevention strategies
Create and support county advocacy networks across the state of Michigan
Collaborate with Michigan's public school districts, including school health and safety programs and school based health centers,to increase oral health awareness activities
None of these
14
Michigan State Oral Health Plan Key Partner Survey
19. Within the increasing access to oral health care goal, which specific objective would this relate to?(please select all that apply)
Decrease the number of children, young adults, adults, and older adults who are underinsured or without dental insurance
Reduce the number of children, young adults, adults, and older adults who experience difficulty, delays or barriers to receivingoral health care
Increase the number of infants, children, and young adults who receive comprehensive dental services
Increase the number of children and adults with disabilities who receive comprehensive dental services
Increase the number of pregnant women who receive comprehensive oral health care
None of these
15
Michigan State Oral Health Plan Key Partner Survey
Name:
Organization name:
Email:
20. Please provide your name, your organization’s name, and your email address so that we can contactyou to learn more about the proposed work.
16
Michigan State Oral Health Plan Key Partner Survey
21. Can you please tell us why you haven’t read the State Oral Health Plan?
I have been too busy to read it
I was not interested in reading it
I forgot that I received it
I wasn’t sure where to find it
The plan seemed too long
The plan was not clearly written
The plan doesn't pertain to the work I do
I wasn't aware the plan existed
Other (please specify)
17
Now we would like to ask you about some specific strategies outlined in the plan and the work youmay have completed in 2016 that was related to these.
The State Oral Health Plan details objectives and activities that work to enhance professionalintegration between oral health providers, medical providers, and social services providers acrossthe lifespan.
Michigan State Oral Health Plan Key Partner Survey
22. In the past year (2016), did your organization have a formal relationship with other providers fromdifferent healthcare disciplines?
Yes
No
I don't know
23. Has the number of formal relationships you or your organization had with providers from differenthealthcare disciplines increased since the release of the 2020 Michigan State Oral Health Plan in 2016?
Yes
No
I don't know
18
Smiles for Life is a National Oral Health Curriculum that produces online educational resources toensure the integration of oral health and primary care.
Michigan State Oral Health Plan Key Partner Survey
24. In the past year (2016), have you or your organization promoted the viewing of one or more of theSmiles for Life online training modules?
Yes
No
NA - I/my organization do not/does not work in this capacity
25. Has the number of Smiles for Life modules you or your organization promoted the viewing ofincreased since the release of the 2020 Michigan State Oral Health Plan in 2016?
Yes
No
I don't know
NA - Not applicable
26. In the past year (2016), have you viewed one or more of the Smiles for Life online training modules?
Yes
No
NA - I/my organization do not/does not work in this capacity
27. Has the number of Smiles for Life modules you viewed increased since the release of the 2020Michigan State Oral Health Plan in 2016?
Yes
No
I don't know
NA - Not applicable
19
The State Oral Health Plan details objectives and activities that work to increase knowledge andawareness of the importance of oral health to overall health among health professionals, policymakers, and consumers
Michigan State Oral Health Plan Key Partner Survey
28. In the past year (2016), did you or your organization use social media to educate providers and orconsumers on oral health?
Yes
No
I don’t know
29. Has your use of social media to educate providers or consumers on oral health increased since therelease of the 2020 Michigan State Oral Health Plan in 2016?
Yes
No
I don't know
30. Do you or your organization share social media messaging (e.g. posts on Twitter or Facebook) from theMDHHS Oral Health Program or the Michigan Oral Health Coalition?
Yes, always
Yes, occasionally
No, never
I don't know
20
An objective within the State Oral Health Plan’s Health Literacy goal is to increase parentknowledge on how to prevent early childhood caries among children 0 to 3 years.
Michigan State Oral Health Plan Key Partner Survey
31. In the past year (2016), did you or your organization work to educate pregnant women on oral healthpreventive measures?
Yes
No
NA - I/my organization do not/does not work with this population
32. In the past year (2016), did you or your organization work to educate parents of children 0-3 years onhow to prevent early childhood caries?
Yes
No
NA - I/my organization do not/does not work with this population
21
An objective within the State Oral Health Plan’s Health Literacy goal is to create and support countyadvocacy networks across the state of Michigan.
Michigan State Oral Health Plan Key Partner Survey
33. In the past year (2016), have you or your organization worked to recruit an Oral Health Champion inyour legislative district?
Yes
No
I don’t know
NA - I/my organization do not/does not work in this capacity
34. In the past year (2016), have you or your organization worked to align local/legislative district levelcoalition activities and messages with Michigan Oral Health Coalition activities and messages?
Yes
No
I don’t know
NA - I/my organization do not/does not work in this capacity
35. In the past year (2016), have you or your organization worked to maintain relationships with policymakers to advocate for oral health?
Yes
No
I don’t know
NA - I/my organization do not/does not work in this capacity
22
An objective within the State Oral Health Plans’ Health Literacy goal is to collaborate withMichigan’s public school districts, including school health and safety programs and school basedhealth centers, to increase oral health awareness activities.
Michigan State Oral Health Plan Key Partner Survey
36. In the past year (2016), have you or your organization provided or facilitated oral health educationwithin Michigan’s school system? This could include promoting or working within school-based sealant programs, integrating oral healthmessages within the K-12 school environment, engaging partners from the Department of Education onintegrating oral health messages into the school curriculum, etc.
Yes
No
I don’t know
NA - I/my organization do not/does not work in this capacity
23
This concludes the survey! Thank you for your participation!
Michigan State Oral Health Plan Key Partner Survey
24