Confidence:Analysisoftheconfidencedataindicatesarangeofbehavioursfromstudentshavinglowconfidenceincorrectanswersindica8nguncertaintyinknowledgeorability,throughtostudentswithhighconfidenceincorrectanswers,exemplifyingtheDunning-KrugerEffect.Theadvantageofadiagnos8ctestmaybetoencouragethesestudentstoconfronttheirlevelofknowledge.Inthe2ndyeardiagnos8ctest,ques8on5wasansweredcorrectlyby26studentsandwasbasedonVSEPR.Thefrequencyofpointsontheconfidencescaleissimilarandinthemoderatelyconfidentrange(3–5).Itdiffersforlowandhighconfidencedependingonwhethertheques8onwasansweredcorrectlyorincorrectly.Ques8on10wasansweredcorrectlybythemajorityofstudents,buthasaverydifferentprofiletoQues8on5.Changinghowwewriteexamques6ons:Construc8ngcorrectstructuresisviewedasanessen8alprocessthat‘expert’chemistsdotounderstandamolecule.Itisalsoviewedasonethatismorecogni8velydemandingandmayinvolveseveralsteps:iden8fybondingandnon-bondingelectrons,createa2-Drepresenta8on,translatethatintoa3-Drepresenta8on2,determinewhethersomesubs8tuentswillbe‘different’(e.g.axialandequatorialsubs8tuents),iden8fyanysymmetrythemoleculemayhave.Bysepara8ngthemolecularstructureiden8fica8onstepfromthemNMRthinkingstepinexamina8onques8ons,studentsarelesslikelytobeoverwhelmedwithextraneousinforma8onwhiles8llworkingthroughthedemandsofstructuredetermina8on.Thisalsoallowseaseofapplica8onofan‘errorcarried’approachtomarking.OldStyle:Predictthetheore8callowtemperature19FNMRspectrumofPF5.[6marks]

RevisedStyle:(a)UseVSEPRtheorytodrawthestructureofPF5.[1mark](b):Predictthetheore8callowtemperature19FNMRspectrumofofPF5.[5marks}

DiagnosingAlterna8veConcep8onsinNMREileshTolly-Brewster,SamGoodwinandKatherineHaxton*

k.j.haxton@keele.ac.uk@kjhaxtonSchoolofChemical&PhysicalSciences,KeeleUniversity

Introduc6on:diagnos8ctestscanprovideinsightintopriorknowledgeandalterna8veconcep8onsheldbystudentsbeforeteaching.Thiscanthenbeusefulindirec8ngstudentstowardsaddi8onallearningresources,andtailoringtaughtsessionstothelevelofthestudents.Mul8plechoiceques8ons(MCQs)canbecarefullydesignedtoprobecommonerrorsoralterna8veconcep8ons,andinsomecasestheabilityofthestudenttoapplytheirpriorknowledgetounfamiliarscenariosthroughrecallorextrapola8on1.Our1styeardiagnos8ctestcoversintroductorytopicsinspectroscopyincludingbasicNMRandisintendedtoevaluatethepriorlearningofthestudents,aswellasprobingsomeknownalterna8veconcep8onsthattheymayholdfrompre-universityteaching.Our2ndyeardiagnos8ctestcoversVSEPR(mainlyhypervalentp-blockcompounds)andNMR(I=1/2nuclei),aimedatstudentsajer2ndyearorganicNMRteachingbutbeforeinorganicmul8nuclearNMRteaching.The2ndyeartestincludesques8onsthatrequirethestudentstoapplytheirknowledgeofNMRtounfamiliarscenarios,andisideallyplacedtoiden8fyalterna8veconcep8onsarisingfrom1styearteaching.

Methodology:Alterna8veconcep8onsandcommonerrorsinunderstandingandapplyingNMRtheorieshavebeeniden8fiedby:-analysisofwrikenexamanswers-focusgroupsandinterviews-analysisoffreetextanswerstopreliminarydiagnos8ctestMCQshavebeencreatedtotestspecificalterna8veconcep8ons.Eachdistractorismappedontoaspecificalterna8veconcep8onorcommonerror.Ques8onsrangefromstraighlorwardtestsofknowledge(familiar,recall),throughtoques8onsthattestconceptualunderstanding1(unfamiliar,extrapola8on).MCQsarecoupledwithfree-textspacetoexplainanswers/showworking,anda7-pointconfidencescalefrom‘notconfident’throughto‘highlyconfident’.Analysisoffree-textcommentsallowsforgreaterunderstandingofanyissues,andwhethertheques8onsarethemselvesconfusing.CommonErrors:Analysisof2ndyearmul8nuclearexamanswerstocategorisecommonerrorswasundertakenand12scriptsanalysed,reasonsforlostmarksrecordedbycategoryoferror.‘Other’referstoanerrorthatwasspecifictothedesignoftheques8onsowouldunlikelytobeseenagain.‘Omission’referstonotansweringpartoftheques8on.Allanswersanalysedhadmul8pleerrors.Themajorityoferrorscamefromincorrectmolecularstructures,highligh8ngthatmolecularshaperemainsabarriertostudentlearningathigherlevelsofstudies2.Someintegra8onandsplinngerrorsweredirectlylinkedtoincorrectmolecularstructure,otherswereincorrectforthepostulatedstructure.2ndYearDiagnos6cTest(DT):Therela8onshipbetweenDTresultsandexamques8onmarkswasprobed(Graph2).Fourgroupsofstudentswereiden8fiedbasedonthemagnitudeanddirec8onofchangeinmarks.GroupsCandDindicatestudentsshowingimprovementfromDTtoexam,withD(increase>37%)showingpar8cularlygoodachievementintheexam.Brepresentsstudentswithlimitedchangeinmark(±7%),andArepresentsstudentswho’smarksdeclinefromDTtoexamwhichmaybeforavarietyofreasonsnotnecessarilyrelatedtothecontent.TherangeinmarksissuchthattheDTisoflimitedprognos8cvalue,howeverplansforfutureyearsinvolvedirec8ngstudentswithlowDTscorestoaddi8onallearningresources.

References:1.Holme,LuxfordandBrandriet,J.Chem.Educ.,2015,92,1477-14832.Tiekmeyeretal,J.Chem.Educ.2017,94,282OnlineMaterial:wanttoreadmoreortrysomediagnos8ctestques8ons?

Pleasevisit:bit.ly/Diagnos6cNMR

0102030405060708090

100

0 20 40 60 80 100

DTScore

ExamQues6onScore

A

B

C

D

HowM

uchPe

opleThink

The

yKn

ow

HowMuchPeopleKnow

Dunning-Kruger Effect

HighConfidence,WrongAnswer

LowConfidence,CorrectAnswer

IncreasingConfidence,CorrectAnswerLearning?

0

10

20

30

1 2 3 4 5 6 7

%Stude

nts

Q5(26/75correct) Q10(61/75correct)

0

10

20

30

40

1 2 3 4 5 6 7

%stud

ents

Q5Incorrect Q5correct

0 20 40 60 80

VSEPR

Splinng

Integra8on

Omission(partofques8on)

Other(ques8ondesignissue)

%oferrorsinanswersanalysed