Embed Size (px)
Citation preview
http://Learning-Layers-eu – Scaling up Technologies for Informal Learning in SME Clusters – [email protected]://Learning-Layers-eu – Scaling up Technologies for Informal Learning in SME Clusters – [email protected]
Learning LayersScaling up Technologies for Informal Learning in SME Clusters
Informal learning in healthcare – an exploratory evaluation approach
Christina Sargianni, Stefan Thalmann, Tamsin Treasure-Jones, Micky Kerr, Ronald Maier & Markus Manhart
1
http://Learning-Layers-eu – Scaling up Technologies for Informal Learning in SME Clusters – [email protected]
Pilot Groups
2
Pilot A Pilot D
Practice Managers from a network (Federation) of 18 General Practices. The tool setting used by this pilot group is the Confer-LD. The Federation is mainly working on joint projects and focuses on developing shared services.
Practice Managers from a network (Federation) of 15 General Practices. The tool setting used by this pilot group is the B&P-LD. The Federation is mainly working on joint projects and focuses on developing shared services.
Table 1: Healthcare Context 1 – Federation/Network
Pilot B Pilot C
Pilot B consists of 6 members from a national team. The tool setting used by this pilot group is the Confer-LD. The team has a specific focus on exploring how technology-enhanced learning can improve education/training in the NHS. Their work involves engagement across all the NHS organisations nationally.
Pilot C consists of 5 members from a regional team. The tool setting used by this pilot group is the Confer-LD. The team has a specific focus on quality improvement training in the NHS. Their work involves engagement across the 44 NHS organisations in their region.
Table 1: Healthcare Context 2 – Distributed teams
http://Learning-Layers-eu – Scaling up Technologies for Informal Learning in SME Clusters – [email protected]
2016 2016February March April May June July
Kick-off Pilot A2/16/2016
Kick-off Pilot B3/15/2016
Kick-off Pilot C3/16/2016
Intermediate Workshop Pilot A5/17/2016
Intermediate Workshop Pilot C5/24/2016
Intermediate Workshop Pilot B6/6/2016
Final Workshop Pilot A7/19/2016
Final Workshop Pilot C7/19/2016
Final Workshop Pilot B7/22/2016
2/16/2016 - 7/19/2016Evaluation Pilot A (Confer-LD)
3/15/2016 - 7/22/2016Evaluation Pilot B (Confer-LD)
3/16/2016 - 7/19/2016Evaluation Pilot C (Confer-LD)
Evaluation Timeline Healthcare
3
http://Learning-Layers-eu – Scaling up Technologies for Informal Learning in SME Clusters – [email protected]
2016Feb Mar Apr May Jun Jul
Federation Meeting Kick-off Pilot A2/16/2016
Accompanying Conversation 12/25/2016
Accompanying Conversation 23/3/2016
Federation Meeting Pilot A3/9/2016
Accompanying Conversation 33/24/2016
Accompanying Conversation 44/14/2016
Federation Meeting Pilot A4/19/2016
Federation Meeting Intermediate Workshop Pilot A5/17/2016
Federation Meeting Final Workshop Pilot A7/19/2016
Individual Interview 1
7/19/2016Individual Interview 27/22/2016
Timeline Pilot A (Confer-LD)
4
http://Learning-Layers-eu – Scaling up Technologies for Informal Learning in SME Clusters – [email protected]
2016Mar Apr May Jun Jul Aug
Team Meeting Kick-off Pilot B
3/15/2016
Accompanying Conversation 13/24/2016
Accompanying Conversation 24/5/2016
Team Meeting Intermediate Workshop Pilot B
6/6/2016
Team Meeting Final Workshop Pilot B7/22/2016
Individual Interview 17/22/2016
Individual Interview 27/22/2016
Individual Interview 3
8/31/2016
Timeline Pilot B (Confer-LD)
5
http://Learning-Layers-eu – Scaling up Technologies for Informal Learning in SME Clusters – [email protected]
2016Mar Apr May Jun Jul
Kick-off Pilot C Team Catch Up
3/16/2016
Accompanying Conversation 13/23/2016
Accompanying Conversation 24/14/2016
Intermediate Workshop Pilot C Team Catch Up
5/24/2016
Final Workshop Pilot C Team Catch Up
7/19/2016
Individual Interview 17/19/2016
Individual Interview 27/19/2016
Timeline Pilot C (Confer-LD)
6
http://Learning-Layers-eu – Scaling up Technologies for Informal Learning in SME Clusters – [email protected]
Activities • Kick off workshops
– to explore users’ tool expectations – focus on research areas like knowledge
boundaries, knowledge scaling and connectivity
• Accompanying conversations– for tool feedback and updates
• Weekly nudging emails – for tool support and usage alert
• Interim workshops – to allow users to move their projects’ forward
and reflect on tool usage• Final workshops
– for tool feedback and reflections on the tools
• Q-study focus on IT adoption • Individual interviews
– specific focus on learning and working practices
• Log files (see detailed slides 38-40)– for tracking the users’ activity
• 1 day workshop – specific focus on B&Ps
7
http://Learning-Layers-eu – Scaling up Technologies for Informal Learning in SME Clusters – [email protected]
Data Collection Methods• Focus groups with duration up to 2 hours
– Pilot A: 6 participants– Pilot B: 2 participants– Pilot C: 5 participants
• 8 individual semi-structured appreciative interviews (18-30 min.)• Q-study with 15 participants (to date)• Capture logs (see detailed slides 38-40)• 1-day workshop
– 6 participants (3 groups of 2 participants)– Hands on activities with B&Ps – 45 min. recorded focus group
8
http://Learning-Layers-eu – Scaling up Technologies for Informal Learning in SME Clusters – [email protected]
Data Analysis
• The evaluation team analyses the data from different lenses:– Pilots’ Context (Tamsin)– Anticipated, Actual & Potential Uses of the Tools (Micky)– Design Patterns of Confer (Patricia)– Barriers & Challenges (Christina)– Learning & Working Practices (Collective)
9
http://Learning-Layers-eu – Scaling up Technologies for Informal Learning in SME Clusters – [email protected]
Joint Interpretation Analysis Scheme
Layers Tools
Evaluation Timeline
Practice Before
Practice After (Expected or Unexpected)
Practice After (Anticipated)
Change
Barriers
Design Patterns
Anticipated Uses
Pilots‘ Context
10
http://Learning-Layers-eu – Scaling up Technologies for Informal Learning in SME Clusters – [email protected]
Data Analysis_ Challenges (Preliminary Results)
11
http://Learning-Layers-eu – Scaling up Technologies for Informal Learning in SME Clusters – [email protected]
Data Analysis_ Challenges
Tool Challenges
• Safety• Access Issues• Technical Issues• Tool complexity• Tool
functionalities
User Challenges
• Time Barriers• IT affinity• Fear of
responsibility• Tool is not a habit• Traditional ways
of working
Network Challenges
• Work Complexity• Group Influence• Project
identification• Need for a leader• Work Duplication• Face-to-Face vs.
ICT• Priorities
Sector Challenges
• Additional tool to log in
• Lack of engagement
• Culture• Lack of network
stability• Dependence on a
wider group
12
http://Learning-Layers-eu – Scaling up Technologies for Informal Learning in SME Clusters – [email protected]
Data Analysis_ Tool Challenges
• Safety: „and there are all web-based, so used from your browser, it‘s in the cloud […] That means it‘s not safe.“
• Access issues: „Logging into Confer is not as easy as I would expect. The „sign in with Learning Layers“ button isn‘t always there and I have to refresh the page to force it to load.“
• Technical Issues: „I think I initially had problems logging in, that took me about a couple of weeks […]“
• Tool Complexity: „Following all the stages can seem laborious but only in the same way that following any quality improvement process (such as fishbone analysis) is. So the idea of systematically going through something is a bit alien to a lot of people (not intuitive)”
• Tool functionalities: „When switching from brainstorm to identifying issues, it took three clicks to duplicate an item. Maybe I am impatient?“
13
http://Learning-Layers-eu – Scaling up Technologies for Informal Learning in SME Clusters – [email protected]
Data Analysis_ User Challenges• Time Barriers: „time is the one thing that we struggle with probably more than anything […] our
struggle is with time.“• IT affinity: „I‘m not brilliant with technology and I like something that‘s going to be intuitive and
easy so you just want to have a go really don‘t you?“• Fear of responsibility: „So it‘s still attributed to me so I don‘t know what it achieves, because I‘m
still putting something in.“• Tool is not a habit: „With any new tool, it needs to become a habitual to get increased use; it
hasn‘t become a habit yet unfortunately, like checking Twitter or LinkedIn is a habit.“• Traditional ways of working (pen & paper): „Yeah, but that having been said my learning style is
such that I write things down and by writing then down they‘re lodged in memory so I didn‘t actually refer to the notes because I then process things.“
14
http://Learning-Layers-eu – Scaling up Technologies for Informal Learning in SME Clusters – [email protected]
Data Analysis_ Network Challenges• Work complexity: „I think my feeling would be that we‘re, you know, adding another layer of complexity to what
we already do, you know, that‘s my fear.“• Group Influence: „So if I sent a document out on Word and said, right can you all comment on it and send it back
to me individually, I would get your honest opinion […]“• Project Identification: „In the kick-off workshop they choose to set up a Confer workgroup to look at one
particular topic „****“. However, that is just a simple decision, so Confer may not be the best tool for this.“• Need for a leader: „What they need to do is to do it with, well I think the drive needs to come from Monica, I
think what we need to do is have a word with Monica and say, let‘s look at this in the next couple of weeks, let‘s try and push this project forward using this.“
• Work Duplication: „it‘s lack of clarity about the tools we‘re using, so it‘s like we are using this tool for this or are we doing that, so there‘s a lot of duplication or things falling through the cracks.“
• Face-to-Face vs. ICT: “I think the tools that we use help us connect better, but there’s no replacement for when we manage to have our face-to-face meetings.”
• Priorities: „Unfortunately, setting up and using another project management tool is just not something they can prioritise at the moment.“
15
http://Learning-Layers-eu – Scaling up Technologies for Informal Learning in SME Clusters – [email protected]
Data Analysis_ Sector Challenges• Additional tool to log in: „A barrier will be the logging in of to yet another system (NHS, email,
clinical system, intranet and now this).“• Lack of engagement: „[…] but there‘s still a heck of people that aren‘t usually engaged with us
[…]“• Culture: „I think technology‘s not really helping us a lot because we‘re all busy people and we
could be making life simpler for ourselves.“• Lack of network stability: „[…] so you have three or four people who push it on, and then
everybody who turns up at the meeting next time goes, hello, that‘s not what we think, you know.“
• Dependence on a wider group: „[…] we have a core team who work together, but then we‘re heavily reliant on our network […]“
16
http://Learning-Layers-eu – Scaling up Technologies for Informal Learning in SME Clusters – [email protected]
Confer Log Files – Total (1/3)
17
http://Learning-Layers-eu – Scaling up Technologies for Informal Learning in SME Clusters – [email protected]
Confer Log Files – Total (2/3)
18
Number in brackets indicating number of groups user contributed to
http://Learning-Layers-eu – Scaling up Technologies for Informal Learning in SME Clusters – [email protected] 19
Confer Log Files – Total (3/3)
