Upload
jonny
View
27
Download
0
Embed Size (px)
DESCRIPTION
An analysis of evaluation activities planned in NN/LM outreach subcontracts 2001-2006. Mellanye Lackey 2 nd year Associate Fellow National Library of Medicine UNC-Chapel Hill Health Sciences Library [email protected]. What I’ll talk about today. Gaining background knowledge - PowerPoint PPT Presentation
Citation preview
An analysis of evaluation activities planned in NN/LM outreach subcontracts 2001-2006
Mellanye Lackey2nd year Associate FellowNational Library of MedicineUNC-Chapel Hill Health Sciences [email protected]
2
What I’ll talk about today
Gaining background knowledge Literature review and conversations w. experts Barriers
Project data Collection methods Analysis Results and graphs
Opportunities to learn more
3
Barriers to doing outreach evaluationPrior experiences
usefulness motive fear negative results
Getting buy-in relationships cooperation from partners support from authority appreciation of evaluation
Lack of time money knowledge about
evaluation processes
-Taut, S. M., & Alkin, M. C. (2003). Program staff perceptions of barriers to evaluation implementation. American Journal of Evaluation. 24, 213-226.
4
5
Data gathering
6
Collecting project data
Projects includedPublic Health Information ModelAccess to Electronic Health Information
SubcontractConsumer Health Initiative SubcontractResource Library Outreach Subcontract Health Professional Information Outreach
7
Project data
Funding from $2,000 to $70,000 reporting evaluation plan
Reviewed >500 records (1/2 of all NN/LM projects)
n = 144
8
Data collection
Methods
QuestionnaireTally/LogPre/Post dataInterview/AnecdoteEvaluation formFocus groupObservation/Usability study
Approaches
Evidence for future useInternal committeeNeeds assessmentSocial capitalCase study
10
Evaluation methods/approaches planned(n=144, Number of methods>total subcontracts b/c of multiple methods planned)
61
37
29
1916
8
27
18
42 2
77
0
10
20
30
40
50
60
70
80Q
uest
ionn
aire
Tally
/Log
P
re/P
ost d
ata
Inte
rvie
w/ A
necd
ote
Eva
luat
ion
form
Focu
s gr
oup
Obs
erva
tion/
Usa
bilit
y st
udy
Evi
denc
e fo
r fut
ure
use
Nee
ds A
sses
smen
tS
ocia
l cap
ital
Cas
e st
udy
Inte
rnal
com
mitt
ee
Method / approach
Nu
mb
er
of
su
bc
on
tra
cts
11
Evaluation methods/appraoches vs. funding levels
0
10,000
20,000
30,000
40,000
50,000
60,000
70,000
0 1 2 3 4 5 6 7 8 9 10 11 12 13
Evaluation methods/approaches
Fu
nd
ing
le
ve
ls
Questionnaire (77)
Tally/Log (61)
Pre/Post data (37)
Interview/Anecdote (29)
Evaluation form (19)
Focus group (16)
Observation/Usability study(8)Case study (2)
Future Use (27)
Needs Assessment (18)
Social capital (4)
Internal committee (2)
77 8 2 27 18 4 229 16193761
12
Number of outreach subcontracts planning 1-6 methods (n=144)
5048
34
10
1 10
5
10
15
20
25
30
35
40
45
50
1 method 2 methods 3 methods 4 methods 5 methods 6 methods
Number of methods
Nu
mb
er
of
su
bc
on
tra
cts
$2500-$50,019
$2000-$56,000
$3200-$50,000
$2500-$50,000
$17,973 $40,000
13
Limits of the data
Records created at project inception Data sometimes rich, more often brief
How many participants? Focus groups? How much time? Terms vague (eg. evaluation form)
Growing emphasis on evaluation with Outreach Evaluation Resource Center (OERC)
14
Further research ideas
Compare evaluation planned with final report
Get more specifics about each plan Examine formative vs. summative Collect information about “approaches” How get more data?
15
Conclusions
At all funding levels: Intent to evaluate Variety of methods
Opportunities to learn more Bibliography of resources
http://www.hsl.unc.edu/AboutLib/Staff/LiaisonWebPages/PHliaison.cfm
Outreach Evaluation Resource Center
http://nnlm.gov/evaluation/index.html