View
216
Download
0
Category
Tags:
Preview:
Citation preview
Training Needs AnalysisStudent Systems
Matthew TaylorTraining & Communications Manager
Student Systems
Training Needs Analysis Process
Timelines• Phase 1 (Oct):
– Draft survey questions– Staff interviews– Student meetings– Analysis of Event booking statistics– Analysis of staff requesting new roles– Analysis of e:Vision role attribution– Final Draft Survey questionnaires
• Phase 2 (Nov):– Issue surveys– Gather responses– TNA survey analysis & conclusions
• Phase 3 (Dec)– Design training approach
Overall Web analytics (April to Oct ‘13)
86, 609 Unique views113, 753 Repeat Page views
Students; 72.8%
Staff; 27.2%
Unique Views
StudentsStaff
Student Unique Views
62, 818 Unique views79, 834 Repeat Page views
Student Index Page 29.59%
Support 25.30%
FAQ 22.60%
Self Service 12.40%
Mini-Portfolio
7.20%Personal Tutors 2.90% Other 0.01%
Overall Student Page Unique Views
Student Index PageSupportFAQSelf ServiceMini-PortfolioPersonal TutorsOther
Student Unique Views
62, 818 Unique views79, 834 Repeat Page views
Scotland; 23.8%
England; 15.1%
China; 7.0%US; 4.3%Greece; 3.4%India; 3.3%
Germany; 2.2%Italy; 1.8%
Malaysia; 1.7%France; 1.7%
Other; 35.7%
Unique Views by Country
ScotlandEnglandChinaUSGreeceIndiaGermanyItalyMalaysiaFranceOther
Student Web Analytics – Key Observations1. The Student pages are heavily viewed
– Many pages need review & enhancement e.g. EUCLID Portal & Online PG Application
2. The top 5 student pages and sub-pages make up approx. 72.5% of the total unique views.– 47.9% of these pages are made up of Student/FAQ & Student/Support based pages– A third of all FAQ unique hits related to:
• Completing the PG Online application• What happens after application submission?
– The Support pages are divided between the following subjects:• PG Online Application (26.5% of all support views)• The EUCLID portal (35.5% of all support views)
– These areas require review and enhancement
Staff Unique Views
23, 528 Unique views33, 852 Repeat Page views
Admissions 13.4%
Curricula Man 47.9%
Student Admin 24.5%
Staff Index Page 6.9%
Staff Training 6.3% Other 1.0%
Overall Staff Unique Page Views
AdmissionsCurricula ManStudent AdminStaff Index PageStaff TrainingOther
Staff Unique Views
23, 528 Unique views33, 852 Repeat Page views
CCAM, Teach-ing & Learn-
ing 15.1%
CCAM, Assess-ment
Methods 11.5%
Staff User Guide Index
6.9%
Personal Tutors 14.1%
Mark Entry 12.4%
Rest 40.0%
Top Staff pages - Unique View
CCAM, Teaching & LearningCCAM, Assessment MethodsStaff User Guide IndexPersonal TutorsMark EntryRest
Staff Web Analytics – Key Observations1. Curricula Management makes up to 47.9% of all staff unique page views, 65.5%
contains CCAM pages.– These pages need review to reduce size & enhance navigation for staff use
2. There are a large number of SACS web pages with <10 unique views– These should be reviewed for archiving
3. The Mini-Portfolio, PPMD, Reporting & Reference Material web pages are not flagged for SACS Google Analytics.– These need to be flagged as soon as possible.
Web Analytics (All) – Key Observations1. End page content with anchored links to FAQ within the content are not useful for
statistical analysis– This can be remedied by having FAQs linked to unique page content, however, is
this good web design? Will it inhibit the web experience?2. Analysing the Google statistics is hindered by the variations of URL capitalisation for
the same web page. For example, student/support, will have a set of view statistics, however, Student/support will have a unique set of statistics,. – If it is an issue that can be resolved here, this should be resolved to aid analysis.
However, it might be the end user typing error?3. Great care needs to be taken on reading too much into Web analytics, especially
when dealing with less that 1 full academic year!
Student Surveys• Student Focus Group (30 attendees, 3 sessions)
– Web Comparison (UoE vs Stanford)– Web Usability– Learning Styles Analysis
• Student Questionnaire (11/12/13 – 20/12/13)– Draft sign off (Student Survey Unit/EUSA/School/College feedback)– Pilot (handful students)– School of Education– 1737 recipients– 2.5% response rate (43 students)
Student Forum Observations• Web Comparison
– UoE vs. Stanford• UoE rated higher than Stanford for navigation• UoE site considered ‘Simple’• UoE Video tutorial considered ‘Long’• Stanford Video Tutorial narration considered ‘Clearer’• UoE Video narration considered ‘too fast’
• Web Usability– Student Self Help pages
• ‘Simple & Easy’, ‘Very Clear’– Help Request Form
• ‘hard to locate’ , ‘not able to find’• Learning Styles analysis
– Heard of SACS? 4/16 Yes– Heard of EUCLID? 16/16 Yes– Required help with EUCLID? 7/16 Yes
• Online Application 3• EUCLID Portal 4
– Blended learning?• Shorter videos, shorter text guidance• “the absolutely best thing about online help is the direct chatbox”
Student Survey Observations• Timing
– National Student Survey (NSS) & Edinburgh Student Experience Survey (ESES) due to be issued in January
– PG Survey in February– Ethics committee sign off required for survey the crosses schools (2 weeks turnaround)– Personal Tutors, Courses, Induction surveys (August-Sept)
• Section 1: Your details:– 81.4 Full-time, 14% Part time, Other 4.7%– 81.4% Year 1– 86% Mature students (over 21)
PGT; 69.8%
UG; 20.9%
PGR; 9.3%
Study
PGTUGPGR
UK; 55.8%Outside EU; 34.9%
EU; 9.3%
Nationality
UKOutside EUEU
Student Survey Observations cont.• Section 2: Previous Experience
– 23.3% had heard of Student, Admissions & Curricula (SACS)– 95.3% heard of EUCLID (SITS e:Vision)– 58.1% did not recall an issue with software– Of 23.3% who had a training issue , 69.2% searched for online assistance with issue
• 44.4% found it difficult to find online guidance• 70% found the online guidance resolved the issue• 71.4% satisfied with online help
Student Survey Observations cont.• Section 3: Preference for online help
– Most effective (i.e. rated effective to very effective):1. User guide (text/screen shots)
– Effective to Very Effective: 25 (59.5%), Not effective to Satisfactory: 11 (26.2%)
2. FAQs– Effective to Very Effective: 25 (58.1%), Not very effective to Satisfactory: 13 (30.3%)
3. Forum – Effective to Very Effective: 19 (46.3%), Not very effective to Satisfactory: 16 (39%)
4. Narrated Video – Effective to Very Effective: 18 (42.8%), Not effective to Satisfactory: 15 (35.7%)
5. Online Course – Effective to Very Effective: 16 (40%), Not effective to Satisfactory: 11 (27.5%)
6. Webinar – Effective to Very Effective: 15 (35.7%), Not effective to Satisfactory: 9 (21.5%)
7. Wiki – Effective to Very Effective: 13 (31.8%), Not very effective to Satisfactory: 15 (36.6%)
Can you think of any ways to improve online help?
Redesign your pages/portals to be more friendly
& intuitive.
Redesign your pages/portals to be more friendly
& intuitive.
Have people upload videos of their own
issues, that others can comment on
Live operator to provide ‘chat’ help
Quick, easy & clear help on the
main page is very helpful
Staff Survey• Staff Interviews (Oct-Dec 13)
– 2-3 nominated staff from each school & central service– 1 hour interview, with questionnaire
• Staff Questionnaire (07/01/14 to 17/01/14)– Draft sign off (Student Survey Unit/HR/School/College feedback)– Pilot (training contacts: 14)– All staff using Student System software– Bristol Online Survey tool– 3010 recipients– 9.3% response rate (279 submitted)
Staff Survey - Observations• Staff Interviews (Oct-Dec 13)
– Key training contact sign up– Peer review training material– Identify silent partners– Encourage engagement
• Section 1: Your Details– 37.3% Managers– 80% School/College, 20% Corporate Services (Registry, SRA, etc.)– Most staff utilising Student Administration, Reporting software (non-SITS) &
Admissions
Support; 59.1%
Academic; 35.5%
Other; 5.4%
Staff Response
SupportAcademicOther
Staff Survey - Observations• Section 2: Your student system use
– 62% using e:Vision daily, 26.5% weekly– 35.8% used e:Vision 1-3 year, 50.5% for more than 3 years
Confident/Very confident;
66.7%
Could be more confident;
25.1%
Not very confident; 4.7%
Not confident at all; 3.6%
Confidence
Confident/Very confidentCould be more confidentNot very confidentNot confident at all
Col-league; 34.7%
Online user guide; 22.7%
System Support; 17.0%
Don't recall; 15.0%
Other; 10.6%
1st contact for help
ColleagueOnline user guideSystem SupportDon't recallOther
Staff Survey - Observations• Section 3: Experience & Preferences
– 24.4% attended classroom training in last year, 73.5% did not
Effective/Very Ef-fective; 53.0%Satisfactory;
38.2%
Not effective/Not very; 8.8%
Classroom training effectiveness
Effective/Very EffectiveSatisfactoryNot effective/Not very
Effective/Very effective;
34.7%
Satis-factory; 37.6%
Not effective/Not very; 27.7%
Online training effectiveness
Effective/Very effectiveSatisfactoryNot effective/Not very
Where to start;
28%
Couldn't find; 25%
Too much detail; 19%
Too techni-
cal; 14%
Out of date; 12% Other; 2%
Online training problems
Where to startCouldn't findToo much detailToo technicalOut of dateOther
Staff Survey – Observations cont.• Section 3: Experience & Preferences
– In office training:• 34.1% Yes• 63.1% No
– Training provision preference:• 49.5% local• 9.3% central• 41.2% both
– Induction training?• 67% No, 28% Yes, 5% Don’t know
User guide; 23.0%
FAQs; 18.0%
Narrated Video; 14.4%Webinar; 13.7%
Online Course; 12.0%
Wiki; 10.2%
Forum; 8.7%
Online material preferences
User guideFAQsNarrated VideoWebinarOnline CourseWikiForum
Taught; 28%
Remote; 7%
Mixed; 42%
1:1; 23%
Learning preferences
TaughtRemoteMixed1:1
Staff feedback?
What is the SACS online user guide?
Never heard of it.
What is the SACS online user guide?
Never heard of it.
Everyone use systems in different ways - it is difficult to streamline
the training
Training too basic with no follow up
courses
If I need training, then the software is poorly designed (Academic)
Quick wins• New student ‘Self Help’ web pages• Embedding web help into student e:Vision software (existing software)• Creating standards for Developers to embed web help in e:Vision (ongoing)• Staff peer testing of online training materials• Brief, FAQ based online tutorials developed for staff & students
TNA Space Invaders
Training Needs Analysis Process
New Timelines• Phase 1 (March):
– Analysis of Event booking statistics– Analysis of staff requesting new roles– Analysis of e:Vision role attribution
• Phase 2 (March):– TNA survey analysis & conclusions
• Phase 3 (April)– Design training approach
What’s next?• Statistical Analysis• Publish findings• Consultation with business in proposed training approach• Training Design• Resources estimate• Develop materials• *Gamification of eLearning• *Learning Management System creation (Learning & Development Forum)• * Follow up training needs analysis (6 months?)
– Student/Staff Surveys– Web analytics, etc.
Questions?
Recommended