Virtual reality objects improve learning ultrasound · 2018-08-08 · Ultrasound imaging data are...

VirtualrealityobjectsimprovelearningefficiencyandsustainedabilitiesinfetalultrasoundJ.Ebert1,B.Tutschek2

1MedicalSchool,ZürichUniversity,Zürich,Switzerland,2PrenatalZürich,Zürich,Switzerland,andMedicalFaculty,HeinrichHeineUniversity,Düsseldorf,Germany

ACCEPTEDFORPUBLICATION,JULY2018,UltrasoundinObstetricsandGynecology

Abstract

ObjectiveVirtualreality(VR)objectsoffetalultrasound(US)volumeshavebeenproposedforteachingandlearningdiagnosticultrasound.WehypothesizedthatVRobjectsimprovelearningefficiencyandretentionofdiagnosticabilities.

MethodsMedicalstudentsandjuniordoctorsweretaughtnormalandabnormalsonographicfetalbrainanatomyusingconventionalmeans(videolecturesandreviewarticles;controlgroup)oradditionallywithselectedVRobjectsfromanovelfetalbrainatlas(http://pb.fetal.ch;studygroup).Knowledge,speedofrecognitionandretentionofdiagnosticabilitiesweretestedoneandfourmonthslater.

ResultsThestudygroupansweredsignificantlymorequestionscorrectlyandsolvedthetestsquickerthancontrols,bothoneandfourmonthsaftertheteaching.

ConclusionTheuseofVRobjectssignificantlyimproveslearningefficiencyandknowledgeretentioninfetalultrasoundteaching.

TeachingultrasoundusingVR

2

IntroductionUltrasoundisthemaintoolforprenataldiagnosis.Specificprotocolshavebeendevelopedforscreeninganddiagnosticaspects1.However,theperformanceofdiagnosticultrasoundisveryoperator-dependent.Increasinglysophisticatedequipmentandextendedprotocolswidenthegapbetweenthetheoreticallypossiblediagnosesandtheskillsofexaminers2.Ultrasoundeducationinvolvesdirecttuitionbyateacher,supervisedimageacquisitionandapplicationofpracticalskills,andobservationofeducationalnormalandpathologicalcases.Someaspectscanbeachievedbyself-study,usingdigitalteachingmaterial2,3.

Ultrasoundimagingdataarewellsuitedfortransformationintodigitallearningobjects4.Virtualreality(VR)objectsoffersuchanewwayofsimulation-basedultrasoundtraining5,6.„PocketBrain“isawebbasedlearningtoolthatuseshighlyinstructiveimagingdatafromnormalandstructurallyabnormalfetuses,capturedusingvolumeultrasoundandpresentedasVRobjects,whichare“‘scroll-through’movies“,withafixedorientationtoenablesimpleandconvenientuse7.Westudiedtheeffectof“PocketBrain”onlearningofjuniordoctorswithregardtofetalcerebralmalformations.Themainobjectivewastostudyiftheuseofvirtualrealityobjectsimproveslearningefficiencyandknowledgeretention,usingmultiple-choicequestionsinstandardizedtestsaboutfetalbrainanomalies.Secondaryobjectivesweretoanalyzehowthiseffectdiffersovertime,testingtheparticipantstwice,threemonthsapart;toexamineapossibleinfluenceontestsolvingspeed;andifvirtualrealityobjectscanhelpwithdiagnosticallychallenginganomalies.

MethodsThestudywasperformedbetweenMayandDecember2016.Participantswerejuniordoctorsandmedicalmasterstudentswithoutspecificpriorultrasoundteaching,recruitedfrommedicalschoolsandvariousuniversityandnon-universityteachinghospitals(seeAppendixA)throughpersonalcontactortherespectivedepartmentaleducators.

Theentirestudywasconductedonline,usingaweb-basedlearningplatform(Moodle,MoodlePtyLtd,Perth,Australia)thatcontainedallteachingandtestmaterial(seeAppendixB).

Forrandomallocationofparticipantstothecontrolandstudygroups,firstly,raffleticketswiththeassignmentofparticipantstostudyorcontrolgroupwereprepared,pleatedtohidetheassignmentandshuffled.Then,theywerehandedouttoparticipantsaccordingtothereceiptoftheenrollment.

Afterconfirmationofparticipation,registrationandlog-in,thetraineeshadtocompleteatwo-hourlearningsession,accordingtotheirgroupallocation,withoutorwithvirtualrealityobjects.Participantswererandomizedtothecontrolortheexperimentalgroup.Participantsofbothgroupseachreceivedatotaloftwohoursteaching.Testcompletionandtimetrackingwasregisteredperparticipant,usingthewebsite’stools.

Teachingconsistedeitherofanone-hourvideolectureonthefetalbrainanditsmalformationsaswellasindividualreadingoftwoteachingarticlesforoneadditionalhourinthecontrolgroup;or,intheexperimentalgroup,participantswereadditionallyexposedtoselectedVRobjectsfrom„PocketBrain“,anovelonlineatlasandteaching

TeachingultrasoundusingVR

3

systemthatusesVRobjectsofnormalandabnormalfetalbrainvolumes7(seefig.1).Thecompleteteachingandtestingenvironmentisnow,aftertheendofthestudy,freelyaccessible(seeAppendixB).Diagnosticabilitiesweretestedinbothgroupsusingadigitalmultiple-choicequestionnaire(MCQ)oneandfourmonthsaftertheteaching(fig.1).TheMCQconsistedof20questionsaboutthefetalbrainanditsanomalies.Eightquestionstestedfactualknowledgethathadbeenpresentedinthelecturesandarticlesprovided,and12questionsrequiredrecognitionoffetalbrainmalformationsonultrasoundimages.

Onespecificquestion(MCQno.17)concernedvisualrecognitionofagenesisofthecorpuscallosum),whichhasbeensuggestedasaparticularlydifficultand,therefore,discriminatingquestion8.TheresultsofthegroupsregardingMCQ17werecomparedseparatelyinadditiontotheentiretestresults.

Onlyparticipantscompletingbothtestswereevaluated.

ThestatisticalprogramR(R.Gentlemanetal.,StatisticsDepartmentoftheUniversityofAuckland,NewZealand),version3.4.1,wasusedforstatisticalanalyses.Todetectanassumedsignificantdifferenceof5%forcorrectlyansweredquestions,thepowercalculation,performedusingG*Power(version3.1,HeinrichHeineUniversityDüsseldorf),showedarequiredparticipantnumberofn=84(42pergroup).NormaldistributionwastestedaccordingtoShapiro-Wilk.Levene’stestwasusedtoassesshomogeneityofvariance.Forvariablewithhomogeneityofvarianceandnormaldistributionoftheresidualsone-wayANOVAwasused(indicating“mean”values)includingCohen’sdtestforeffectsize;notnormallydistributedvariablesweretestedusingtheMann-WhitneyUTest(indicating“median”values)includingZvaluesforeffectsizer9.TheFisher'sexacttestwasusedforevaluationofthespecificquestion17.

Fig.1Recruitment,randomization,teachingandtesting(oneandfourmonthsaftertheteaching)

TeachingultrasoundusingVR

4

ResultsOftheinitiallyrecruitedparticipantsfromtwocountries(Switzerland,Germany)and13differenthospitals,51completedbothtests:30inthecontrolgroup,and21inthestudygroup.

VRsignificantlyimproveslearningefficiency.Thestudygroupansweredmorequestionscorrectlythancontrols,bothoneandfourmonthsaftertheteaching;thisdifferencewasbiggerfourmonthsaftertheteaching(seetable1andfig.2).Cohen’sdbothtests(oneandfourmonthsaftertheteaching)showedstrongeffects:Cohen’sd(test1)=1.05;Cohen’sd(test2)=1.32.

A. B.

Fig.2:Testresultsofcontrols(group1)andstudygroup(group2)onemonth(test1;fig.2A)andfourmonthslater(test2;fig.2B).UseofVRobjectinteachingimprovesabilitiesoneandfourmothsafterteaching.Thestudygroupalsosolvedthetestsquicker,bothonemonth(medianVR-group=13.2minvs.mediannon-VR-group=15.9min,Z=6.2148,p-value<0.001,effectsizer=0.87,i.e.astrongeffect)andfourmonthsaftertheteaching(medianVR-group=10.3minvs.mediannon-VR-group13.5min,Z=6.2155,p-value<0.001,effectsizer=0.53,i.e.astrongeffect).Theabilitytorecognizeagenesisofthecorpuscallosum,MCQno.17inourtest,hasbeenproposedasadiscriminatingfactorforappliedspecificknowledge8.ThestudygroupdidbetteratMCQ17comparedtothecontrols,bothonemonthandfourmonthsaftertheteaching(71%and67%vs.55%and57%),butthesedifferenceswerenotstatisticallysignificant(p=0.25andp=0.57).

Threefourthsoftheparticipantsinbothgroupswerefemale.Bothgendersscoredthesameaveragenumbersofcorrectanswersinbothtests(test1:femalemean=13.1points,SD3.21vs.malemean=13.1points,SD3.12,p=0.96;test2:femalemean=12.3points,SD4.14vs.malemean=12.9points,SD4.01,p=0.62).Fig.S1showsthecorrelationbetweencorrectanswersandgenderoftheparticipants.

Notallregisteredparticipantscompletedtheteachingandbothtests.Aftertest2,therewere21medicalstudents(70%)and9juniordoctorsinthecontrolgroup,5studentsand16juniordoctors(76%)intheVRgroup.Toassessaneffectofthisuneven

TeachingultrasoundusingVR

5

distributionofstudentsbetweenthetwogroups,separateanalysesofonlythestudentsandonlythejuniorresidentsinbothgroupsweredone.Overall,juniordoctorsscoredbetter,butjuniordoctorsandmedicalstudentsinthestudygroupperformedbetterthanpeercontrols.ThecomparisonsbetweenstudentsandjuniordoctorsaswellasbetweencontrolsandVRgroupareshownintable1.Fig.S2showstestperformances(intest1and2)ofjuniordoctororstudentstatus.

Allparticipants

Medicalstudents

Juniordoctors

Test1

Test2Test1

Test2Test1

Test2

Controlgroup

(n=30)

§median=11.9

IQR=3.51

§median=10.6

IQR=5.0

§median=11.2

IQR=3.1

#median=9

IQR=5

#median=14

IQR=2

§median=12.6

IQR=6.21

VRgroup

(n=21)

§median=14.9

IQR=4.2

§median=15.1

IQR=4.05

§median=12.0

IQR=3.51

#median=14

IQR=5

#median=16.5

IQR=4.25

§median=15.8

IQR=2.43

p-values<0.001

<0.001

0.52

<0.001

<0.001

0.07

Tab.1Compariso

noflearningeffectb

ygroupassig

nmentandpriortra

ining(co

rrecta

nswerso

utof20questio

nsperte

st).Afterte

stingfor

normaldistrib

ution(datanotsh

own),m

eanvaluesandsta

ndarddeviations(S

D)w

erecalculatedfornormallydistrib

utedvariables

(indicatedby“§”),u

singone-wayANOVA;th

eywerethenconvertedintomedian10andIQR 11fo

runiformityofpresentation.Fornot

normallydistrib

utedvariables(in

dicatedby“#”)m

edianvaluesandinterquartile

ranges(IQ

R)w

erecalculated,usingtheMann-WhitneyU

test.

DiscussionTheuseofultrasoundishighlyoperator-dependent.Developmentofincreasinglysophisticatedequipmentmagnifiesthepotentialfordiagnosticerror.Duetoalackoftraining,thereisawideninggapbetweenthesophisticationofthemostadvancedmachinesandtechniquesandtheskillsofthoseexpectedtousethemandinterprettheimages2.Clinicalexperienceanddirectobservationandtraininginspecializedultrasoundunitspredicttrainees’confidenceinperformingdiagnosticobstetricultrasoundexamindependently,butconcernsexistabouttheadequacyofcurrentultrasoundtrainingprograms.Simulation-basedtrainingmayimprovelearningefficiencyandknowledgeretention12.

VirtualReality(VR)objectshavebeenproposedasteachingandself-studytools3,4,6,12.Inaclinicallyrelevantcontext,i.e.teachingjuniordoctorsandmedicalstudentstorecognizefetalbrainanomaliesonultrasoundimaging,westudiedifVRobjectsimproveteachingsuccess.OurresultsshowthattheuseofVRobjectsimproveslearningsignificantlyandinasustainedway.Theexperimentalgrouphadself-studyexposuretoVRobjectsforaslittleasonehour,yettheyansweredmorequestionscorrectlythanthecontrolgroup.KnowledgeretentionwasalsobetterafterVRteaching.Thesefindingsareconsistentwithotherresultsonlearningbehaviorthatshowbeneficialeffectsofvisuallearning13.

Werecruitedstudentsandjuniordoctorsforourstudy,becauseconventionalsimulation-basedultrasoundtrainingworksbetteronnovicetraineesthanonexperiencedvolunteers14.WeexplicitlylimitedtheaccesstotheVRobjectstoarelativelyshortperiodoftimeforthestudygrouptoavoiduncontrolledadditionalself-study.Now,aftertheendofthestudy,thisweb-basedtooliscontinuallyavailable,cantheoreticallybeupdatedandenhancedtorepeatandaugmentthelearningprocessandtofurtherconsolidateindividualknowledgeandabilities.

Anotheradvantageofhavingweb-basedvirtualrealityobjectsfortheparticipantswastheresource’savailabilityatanytime,dayornight.Whilethisallowedtheparticipantstoselecttheirbestindividualconditionsforthelearningsessionandthetests,itmighthavebeenaconfoundersincedaytimeandlocationoftestperformancewerenotstandardized.Theoretically,externalauxiliarytoolsmighthavebeenused,buttheconditionswereidenticalforbothgroups.

Limitationsofourstudyare(i)themoderatenumberofparticipantswhocompletedbothtestsand(ii)theunevendistributionofthejuniordoctorsandthestudentssecondarytodropoutsofparticipants.Tryingtoavoidaneventualunevendistributionofjuniordoctorsandstudentstostudyandcontrolgroups,infurtherstudies,onecouldfirstbreakupparticipantsintojuniordoctorsandstudentsandthenallocaterandomly.

Theeffectsizeeventuallyfoundwasbiggerthanexpected,yieldingsignificancesforfewerparticipantsthaninitiallycalculated.OverestimatingtheutilityofVRobjectsisunlikelytohaveoccurred:Weacceptedonlymotivatedparticipants(finalyearsmedicalstudentandjuniordoctors)whoapproachedusvoluntarily,havingbeenalertedtothepossibilityofparticipation.Then,randomizationtookplace.Focusingonaveryspecifictopic(fetalbrainabnormalities)andspecifictestquestions(onlytestinginformationpreviouslypresentedtobothgroups)andequalallottedtimeduringteachingandexecutionofthetestsreducedpotentialerrorinthejudgmentoftestefficacy.

TeachingultrasoundusingVR

8

Inconclusion,weforthefirsttimedemonstratetheutilityofVRobjectsforacquisitionofdiagnosticabilitiesandknowledgeretentioninprenataldiagnosis.Basedonourresultsandoncurrentresearchonthetheoryoflearningandthevariouspossibilitiesofthedigitalage,weproposetomakeultrasoundtrainingasvisualaspossible.VRobjectsprovideanidealtoolforthis;theycanbeeasilycombinedwithadditionallearningresourcessuchasonlinelecturesandconventionalscriptsortextbooks.TheVRobjectsusedforthisstudyaswellastheteachingandtestingenvironmentarefreelyavailable,canbeusedbyothersandcouldbeadaptedtocoverotheraspectsofdiagnosticimaging.

VRobjectsincreasetheefficiencyoftrainingandtheretentionoftheacquireddiagnosticabilities.Furtherstudieswithmoreparticipantsandforotherorganregionsthanthefetalbrainshouldbeperformed.

AcknowledgementsWearegratefultoDr.OlivierMerlo,statistician,ZHAW,Wädenswil,Switzerland,andtheInstituteforStatisticsinMedicine,HeinrichHeineUniversity,Düsseldorf,Germany,fortheirvaluablestatisticalhelpandadvice.

TeachingultrasoundusingVR

9

Appendices

AppendixAWeareverygratefultotheparticipantswhowererecruitedfromthefollowinginstitutions:Dept.ofObstetrics,CharitéBerlin;Dept.ofObstetrics,UniversityHospitalDüsseldorf;ZürichUniversityMedicalSchool;UniversityHospitals,Basel,BernandZürich;CantonalhospitalsofSt.Gallen,Lucerne,Frauenfeld,andMünsterlingenandhospitalsZollikerberg,Grabs,Herisau,Wil.

AppendixBTeachingmaterial:anone-hourlectureonthenormalandabnormalfetalbrain(B.Tutschek,Zürich,andG.Pilu,Bologna);onehourreadingoftwoaccompanyingreviewarticles/guidelines1,13;and“PocketBrain”,anovelonlineatlasandlearningtool7thatisfreelyavailableathttp://pb.fetal.ch.Foraccesstotheteachingmaterialandenvironmentpleaseemailthecorrespondingauthor.

TeachingultrasoundusingVR

10

Supplementalmaterial

• Recognizingcorpuscallosumundcavumseptipellucidi,bananasignundlemonsign,Dandy-Walkermalformation,megacisternamagna,holoprosenzephaly,myelocele,myelomeningozele,Blake’spouchcyste,vermishypoplasy,agenesisofcorpuscallosum

• Namingthetwoaxialsectionsasthediagnosticstandardsectionsofthefetalbrainscreeningexamination

• Theatriaoftheposteriorhornastheregionwheretheventricularwidthshouldbemeasured

• Definitionofmildtomoderateventriculomegalyaslateralventricularwidthof10-15mm

• Determiningthecoronalplaneinanultrasoundimageofanormalfetalbrain• Diagnosisofpartialagenesisofthecorpuscallosuminthesagittalsection• The"hourglassconfiguration"andBlake’spouchcyst.• Agenesisofcorpuscallosumnotbeingpartofthedifferentialdiagnosisincystic

spacesintheposteriorfossa.• Knowingthemaincausesofcysticenlargementsoftheposteriorfossa• Associationofthelemonsignwithspinabifida;incidenceoftheclosedform

spinabifidaTab.S1Specificteachingandtestingobjectives(containedinlectures,teachingarticlesandinPocketBrain)

A. B.

Fig.S1Correlationofcorrectsanswersintests1(onemonthafterteaching;fig.S1A)aswellas2(fourmonthsafterteaching;fig.S1B)bygenderoftheparticipants(1:XX,2:XY).

TeachingultrasoundusingVR

11

A. B.

Fig.S2Correlationbetweenaccuracyofanswersintests1(onemonthafterteaching;fig.S2A)aswellas2(fourmonthsafterteaching;fig.S2B)byprioreducation.Overall,juniordoctorsscoredbetterthanmedicalstudents,butjuniordoctorsandmedicalstudentsinthestudygroupperformedbetterthanpeercontrols.(3:medicalstudents,4:juniordoctors).

TeachingultrasoundusingVR

12

References

1. InternationalSocietyofUltrasoundinO,GynecologyEducationC.Sonographicexaminationofthefetalcentralnervoussystem:guidelinesforperformingthe'basicexamination'andthe'fetalneurosonogram'.UltrasoundObstetGynecol2007;29:109-116.2. SalvesenKA,LeesC,TutschekB.BasicEuropeanultrasoundtraininginobstetricsandgynecology:whereareweandwheredowegofromhere?UltrasoundObstetGynecol2010;36:525-529.3. TreleaseRB,NiederGL,DorupJ,HansenMS.GoingvirtualwithquicktimeVR:newmethodsandstandardizedtoolsforinteractivedynamicvisualizationofanatomicalstructures.AnatRec2000;261:64-77.4. TutschekB.Simplevirtualrealitydisplayoffetalvolumeultrasound.UltrasoundObstetGynecol2008;32:906-909.5. TutschekB,PiluG.Virtualrealityultrasoundimagingofthenormalandabnormalfetalcentralnervoussystem.UltrasoundObstetGynecol2009;34:259-267.6. TreleaseRB,NiederGL.Transformingclinicalimagingand3Ddataforvirtualrealitylearningobjects:HTML5andmobiledevicesimplementation.AnatSciEduc2013;6:263-270.7. TutschekB,PiluG.PocketBrain,aninteractive,web-basedultrasoundatlasofnormalandabnormalfetalbraindevelopment.UltrasoundObstetGynecol2017;49:431-432.8. WiechecM,NocunA,KnafelA,BeithonJ,StettnerD.FourStepsinDiagnosingCompleteAgenesisoftheCorpusCallosuminPrenatalLife.UltraschallMed2016;37:92-99.9. PallantJ.SPSSSurvivalManual.OpenUniversityPress:Buckingham,Philadelphia,2007.10. GausW,MucheR.MedizinischeStatistik:AngewandteBiometriefürÄrzteundGesundheitsberufe.Schattauer,2013,220.11. PeckR,DevoreJL.Statisticstheexplorationandanalysisofdata.CengageLearning,2011,118.12. TutschekB,TercanliS,ChantraineF.Teachingandlearningnormalgynecologicalultrasonographyusingsimplevirtualrealityobjects:aproposalforastandardizedapproach.UltrasoundObstetGynecol2012;39:595-596.13. DeCatteL,DeKeersmaekerB,ClausF.Prenatalneurologicanomalies:sonographicdiagnosisandtreatment.PaediatrDrugs2012;14:143-155.