AnEvaluationofAcceleratedLearning

intheCMUOpenLearningInitiativeCourse“Logic&Proofs”

ChristianD.Schunn&MellisaM.Patchan

LearningResearchandDevelopmentCenter

UniversityofPittsburgh

May31,2009

2

TableofContents

EXECUTIVESUMMARY ....................................................................................................................................... 3THETYPICALINTRODUCTIONTOSYMBOLICLOGICCOURSE .............................................................. 4SYLLABIANALYSIS .................................................................................................................................................................... 4SamplingofUniversities .................................................................................................................................................... 4CodingSyllabi......................................................................................................................................................................... 5

SURVEYANALYSES.................................................................................................................................................................... 5Participants............................................................................................................................................................................. 5Survey ........................................................................................................................................................................................ 5

RESULTS...................................................................................................................................................................................... 5SyllabiAnalyses ..................................................................................................................................................................... 5SurveyAnalyses ..................................................................................................................................................................... 7

SUMMARY .................................................................................................................................................................................10INCLASSVS.ONLINEINLARGECOURSESTUDIES ..................................................................................11PARTICIPANTS .........................................................................................................................................................................11Fall2007.................................................................................................................................................................................11Spring2008...........................................................................................................................................................................11

RESULTS....................................................................................................................................................................................13SUMMARY .................................................................................................................................................................................13

SAMETIME/MORECONTENTINACOMMUNITYCOLLEGE ..................................................................15CONTEXT&PARTICIPANTS...................................................................................................................................................15ATTRITIONRATES ..................................................................................................................................................................15TIMEONTASK .........................................................................................................................................................................15PERFORMANCEONMIDTERMEXAMS .................................................................................................................................16FINALEXAMPERFORMANCE ................................................................................................................................................16SUMMARY .................................................................................................................................................................................16

ACCELERATIONFORCOMMUTERSTUDENTSWITHOPTIONALRECITATIONSESSIONS ..........18CONTENTLEARNINGRESULTS .............................................................................................................................................18ATTENDANCE...........................................................................................................................................................................18SUMMARY .................................................................................................................................................................................19

SAMETIME/MORECONTENTWITHHYBRIDINSTRUCTIONINAUNIVERSITY............................20CONTEXT&PARTICIPANTS...................................................................................................................................................20TIMEONTASK .........................................................................................................................................................................21PERFORMANCEONFINALEXAM ..........................................................................................................................................21SUMMARY .................................................................................................................................................................................22

REFERENCES ........................................................................................................................................................23

3

ExecutiveSummaryCMU’sOpenLearningInitiativeseekstocombineexpertiseinthecognitivescienceoflearningwithdisciplinaryinstructionalexpertiseandhumancomputerinteractiontoproduceonlinelearningenvironmentsthatcanbroadlyimproveinstruction.ThecourseLogic&Proofs(L&P)isonesuchcourse,developedunderthestewardshipofWilfriedSiegfromthePhilosophyDepartmentofCarnegieMellonUniversity.

ThecurrentreportdescribesevaluationsofL&P,specificallytestingthehypothesisthatuseofL&Pinahybridinstructionalmode(onlineinstructioncombinedwithreducedandstrategicallytargetedface‐to‐faceinstruction)canleadtoacceleratedlearning,eitherintheformofmorecontentlearningwithinthesameamountofstudenttimeorthesamecontentlearningwithinashortertimeperiodofstudenttime.

Beforebeginningtheevaluationexperiment,weconductedacontextanalysistodeterminetherealmarketoftheLogic&Proofscourse:whatkindsofuniversitiestendedtoteachthisoverallcourse,whatcontenttendedbecovered,andwhatkindsoftoolsforproofconstructionwerealreadybeingused.Overall,L&P‐likecontentistaughtinamajorityofuniversitiesintheUS,butwithsomevariationbyuniversitytype.Further,L&Pcontainssomecontentthatmanycoursesdidnotcover.Thus,theapplicabilityofthecourseishigh,andtheopportunitiesforaccelerationviamorecontentcoveragealsoexist.

Theevaluationswereconductedintwophases.First,twoevaluationswereconductedatalargepublictier‐oneinstitution,showingthatface‐to‐faceinstructionproducedequivalentlearningoutcomestoonline‐onlyinstruction.Inaddition,onlineinstructionappearedtoresultinamuchloweredattritionrate,providingefficienciesforstudentsofanotherform.

Second,anevaluationwasconductedofacceleratedlearningwasconductedatthreeinstitutions,eachrepresentingdifferenttypesofinstitutions,usingdifferentformsofaccelerationandevaluationlogicssuitedtothecontext(e.g.,randomassignmenttoconditionvs.naturalexperiment,morecontentinequaltimevs.equalcontentinlesstime).Onestudyfoundstudentsatacommunitycollegewereabletolearnsignificantlymorecontentandsomecontenttohigherperformancelevelsusinghybridinstructioncomparedwithface‐to‐faceinstruction.Asecondstudyfoundthatcommuterstudentsataregionalcampusofapublicuniversitysigningupforonlineinstructionwereresistanttoparticipatinginhybridinstructionandtherewasnotaclearassociatedbetweenamountofhybridinstructionandstudentlearning.Thethirdstudyfoundthattraditionalstudentsfromprimarilycomputerscienceandengineeringbackgroundsatanationaluniversityactuallyshowedsmallbutsignificantdeclinesinexamperformancewithhybridinstructionrelativetoface‐to‐faceinstruction.

4

TheTypicalIntroductiontoSymbolicLogicCourseInordertoprovideacomprehensivedescriptionofthetypicalintroductiontosymboliclogiccourse,weexaminedthecoursesintwosteps:1)wesystematicallyexaminedcoursesyllabiand2)webroadlysurveyedthosewhotaughtsymboliclogiccourses.

SyllabiAnalysis

SamplingofUniversities

AlistofuniversitiesintheUnitedStateswascompiledusingthedatabasefromtheCarnegieFoundationfortheAdvancementofTeaching,US&WorldNewsReport’sAmerica’sBestColleges,andWikipedia.Additionalinformationabouteachschoolwasalsorecorded(seeTable1).Arandomsampleof40ofeachtypeofuniversity(i.e.National,LiberalArts,Comprehensive,HBU,and2‐yearuniversities)andallnineoftheTribalCollegeswascollected.Atotalof209USuniversitieswereexamined.

Table 1. Additional information collected on each school.

Category Source Category DefinitionFunding Privatenot‐for‐

profit

Privatefor‐profit

CarnegieFoundationfortheAdvancementofTeaching

Public Attendance Full‐time

CarnegieFoundationfortheAdvancementofTeaching Medium&High

Part‐time

Transfer Hightransfer

CarnegieFoundationfortheAdvancementofTeaching Lowtransfer

Size Verysmall&Small

CarnegieFoundationfortheAdvancementofTeaching Medium&Large

Type National Offersbothundergraduateandgraduatedegrees

LiberalArts Offersmainlyundergraduatedegreesintheliberalarts

US&WorldNewsReport

Comprehensive Offersmainlyundergraduatedegreeswithlessthan50%inliberalarts

Wikipedia HistoricallyBlackUniversity

TribalCollege Rank Top

US&WorldNewsReportTier3&4

5

CodingSyllabi

Foreachofthe209universities,theuniversitywebsitewasvisitedtoseeiftheuniversityhadaphilosophydepartmentandofferedasymboliclogiccourse.Ofthe114universitiesthatofferedsymboliclogic,64syllabiwerecollectedeitherbybeingpostedonawebsiteorbycommunicationwiththeinstructor.Thebook,topicscovered,andtypeofassignmentswererecordedusinginformationcontainedinthesyllabus.

SurveyAnalyses

Participants

MembersoftheAssociationforSymbolicLogicwererecruitedtocompletetheonlinesurvey.Thisassociationisaninternationalorganizationwithover1500membersfromover60differentcountries.Themajorityofthemembers(52%)resideintheUnitedStates.Responseswerecollectedfrom199participantsfrom39differentcountries.Overhalfoftheresponderswerefromaphilosophydepartment,28%werefromamathematicsdepartment,and13%werefromacomputersciencedepartment.Theremainingparticipantswerefromotherdepartments.

Survey

Thesurveyconsistedof26questionsdividedintofourmainsections.Themainfocusofthesurveywastodeterminehowsymboliclogiccourseswerebeingtaughtandwhethertechnologicaltoolswerebeingused.Thefirstsectionaskedforgeneralinformationabouttheresponder(e.g.,institutionname,department,country)andaboutthecourse(e.g.,whooffersthecourse,howoftenitisoffered,howmanystudentstakeit,whatmajorandlevelarethetypicalstudents).Thesecondsectionfocusedonthecoursespecifics,suchastheprimarytextbookandtopicscovered.Thefinalquestioninthissectionaskedwhethereducationalsoftwarewasused.Basedontheresponse,thenextsectionfocusedoneitherwhyeducationalsoftwarewasnotusedorhowtheeducationalsoftwarewasbeingused.Thelastsectionfocusedonfutureeducationalsoftwareuse.Iftheresponderindicatedthatheorshewouldbeconsiderusingafully‐computer‐basedandhighlyinteractivecourse,theyweredirectedtoafinalfollow‐upquestionabouthowtheywouldlikelyusethistypeofcourse.

Results

SyllabiAnalyses

Tobeginouranalyses,wewantedtoknowhowmanyuniversitiesofferedasymboliclogiccoursethroughthephilosophydepartment.Inordertoprovideanaccurateestimate,wefirstdeterminedhowmanyuniversitiesofferedthecoursefromourrandomsample.Outof209universities,55%(ormorespecifically114universities)offeredasymboliclogiccourse.Becausethepercentageofuniversitiesofferingthecoursediffereddependingonthetypeofuniversity,wecalculatedaweightedestimateofuniversitiesofferingsymboliclogic:1258universities(seeTable2).

6

Table 2. Estimated number of universities offering symbolic logic

2­year Comp. HBULiberalArts National Tribal Total

#ofschools 1618 277 83 199 239 9 2425Percentageofschools

offeringsymboliclogic 48% 38% 40% 75% 85% 0% 55%

Estimated#ofschoolsofferingsymboliclogic 769 104 33 149 203 0 1258

Wewerealsointerestedinwhatvariablesmightpredictwhethertheuniversitiesofferedasymboliclogiccourse(i.e.,type,level,funding,attendance,transfer,size,andrank).TheLiberalArtsandNationaluniversitiesweremorelikelytoofferasymboliclogicthanthe2‐year,Comprehensive,andHBUuniversities(seeFigure1).Thepublicuniversitiesweremorelikelytoofferasymboliclogiccoursethantheprivate,not‐for‐profituniversities,whoweremorelikelytoofferitthantheprivate,for‐profituniversities.Universitiesthathavemostlyfull‐timestudentsweremorelikelytoofferasymboliclogiccoursethanuniversitiesthathavemostlypart‐timestudents.Mediumandlargeuniversitiesweremorelikelytoofferthecoursethansmallandverysmalluniversities.Top‐tieruniversitiesweremorelikelytoofferthecoursethantier3and4universities.Therewerenodifferencesinlevel(i.e.,2‐yearversus4‐year)ortransferrates(i.e.,hightransferratesversuslowtransferrates).

Figure 1. Frequency of offering symbolic logic by different type of universities

Next,thesyllabiwerecodedtoseewhichtopicswerecovered.Nineofthesyllabididnotincludeascheduleorlistoftopicstobecovered,sotheywereexcludedfromtheanalyses.Therewere12possibletopicsofinterest.Fiveofthesetopicspertainedtosententiallogic(i.e.,syntax,symbolization,semantics,derivations,andMetamathematics),andsevenof

7

thesetopicsconcernedpredicatelogic(i.e.,syntax,symbolization,semantics,derivations,identity,functions,andMetamathematics).Overall,therewerenodifferencesbetweenthetypesofuniversitiesinthenumberoftopicstheycovered(M=7.3,SD=2.3).However,thetypesofuniversitiesdiddifferinthenumberofsententiallogictopicscovered(seeFigure2).TheComprehensiveuniversitiescoveredmoresententiallogictopics(M=4.2,SD=0.4)thananyothertypeofuniversity,andtheHBUuniversitiescoveredthesecondmostsententiallogictopics(M=4.0,SD=0).Therewerenodifferencesinthenumberofpredicatelogictopicscovered(M=3.6,SD=1.8).

Figure 2. Number of sentential topics covered by type of university

Finally,weexaminedwhichbookswereusedtoteachsymboliclogic.Twenty‐fivedifferentbookswerefound.Threeinstructorschosetonotuseabook,butinsteadtodistributetheirownnotes.Themostpopularbookwas“AConciseIntroductiontoLogic”byHurley,whichwasusedby13courses.Otherbooksusedinmultiplecoursesincluded:“TheLogicBook”byBergman,Moor,andNelson(6courses),“Language,Proof,andLogic”byBarwiseandEtchemendy(5courses),and“IntroductiontoLogic”byCopiandCohen(5courses).

SurveyAnalyses

Whiletheintentionofthesurveywastodeterminehowsymboliclogiccourseswerebeingtaughtandwhethertechnologicaltoolswerebeingused,thefocusofthecurrentanalysisistojustprovideacomprehensivedescriptionofthetypicalcourse.Therefore,wewillonlybereportingresultsfromthefirsthalfofthesurvey.Thefirstsectionofthesurveyfocusedongeneralaspectsofthecourse(e.g.,whooffersthecourse,howoftenitisoffered,whotakesit).Thesecondsectionfocusedonthecoursespecifics,suchastheprimarytextbookandtopicscovered.

Whooffersthecourse?Accordingtoourparticipants,thedepartmentmostlikelytoofferasymboliclogiccourseisthephilosophycourse(53%).Otherdepartmentsthattypicallyofferthecourseincludethemathematics(28%)andcomputerscience(13%).

8

Howoftenisthecourseoffered?Theparticipantshadtheoptiontochoosebetweeneverysemester/quarter,onceeveryyear,onceeverytwoyears,orlessoften.Typically(63%),thesymboliclogiccoursesareofferedonceayear.Lessfrequently,thecourseisofferedeverysemester(26%)andeverytwoyears(9%).

Whotakesthecourse?Inordertobetterunderstandwhotakesthecourse,weaskedparticipantshowmanystudentstypicallytakethecourse,whatmajorarethey,andwhatlevelarethey.Theaveragenumberofstudentstakingasymboliclogiccourseis68(SD=83),whichsuggeststhattensofthousandsofstudentsintheUSaretakingacourseinsymboliclogiceachsemester.Therewasalargerangeofresponses,fromaslittleas2studentstoasmuchas550students.Notsurprisingly,mostofthestudentstakingthiscourseweremajoringinphilosophy(63%),computerscience(50%),ormath(45%).Othermajorsincludehumanities(26%),socialsciences(17%),engineering(10%),naturalscience(8%),andbusiness(5%).Studentsaremostlikelytobejuniors(51%).However,studentsatalllevelstaketheclass(sophomores40%;freshman39%;seniors35%;andgraduatestudents7%).

Textbooks:Outofthe97participantswhoreportedtheirtextbook,almost60differenttextbookswerementioned.Fourteenparticipantsstatedthattheydidnotuseatextbook,andnineofthoseparticipantsusedtheirownnotesinstead.ThemostpopulartextbooksusedwasBarwise&Etchemendy’s“Language,Proof,andLogic”(11participants),Enderton’s“AMathematicalIntroductiontoLogic”(7participants),andBergmann,Moor,&Nelson’s“TheLogicBook”(5participants).

Topics Covered: Participantsindicatedmanydifferenttopicsthattheytypicallycoverintheirsymboliclogicclass(seeTable3).

Table 3. Topics typically covered in a symbolic logic course Sentential:syntax&semantics 93%Sentential:truthfunctionalcompleteness&normalforms 69%

Sentential:naturaldeductionproofs 63%Sentential:truthtrees(tableaux) 38%Sentential:completeness 56%Predicate:syntax&semantics 95%Predicate:naturaldeductionproofs 64%Predicate:truthtrees(tableaux) 33%Predicate:identity&definitedescriptions 55%Predicate:introductionoffunctionsymbols 39%Predicate:completeness 46%Aristotelianlogic 12%Modallogic 10%Intuitionisticlogic 13%Additionalsententialandpredicatelogic 24%Basicincompletenessandundecidabilityresults 19%

9

Modeltheory 25%Settheory 17%Computabilitytheory 12%Prooftheory 25%Topicsincomputerscienceormathematics 14%Topicsinmethodologyandphilosophicallogic 19%Othertopics 25%

TopicWishList:Finally,wealsoaskedparticipantswhethertheywishedtheycouldcoveradditionaltopics.Onlyhalfoftheparticipantsrespondedthattheywouldliketocoveradditionaltopics.Theseadditionaltopicswerecategorizedintooneofninedifferentgroups:1)Additionalsententialandpredicatelogic,2)Basicincompletenessandundecidabilityresults,3)Modeltheory,4)Settheory,5)Computabilitytheory,6)Prooftheory,7)Topicsincomputerscienceormathematics,8)Topicsinmethodologyandphilosophicallogic,and9)Othertopics(seeTable4).

Table 4. Topics wish list Additionalsententialandpredicatelogic 12%Basicincompletenessandundecidabilityresults 18%Modeltheory 8%Settheory 4%Computabilitytheory 11%Prooftheory 16%Topicsincomputerscienceormathematics 7%Topicsinmethodologyandphilosophicallogic 23%Othertopics 12%

DifficultiesinTeaching:Participantsdescribedmanydifficultiesthattheyhaveencounteredwhileteachingthesematerials.Themostcommondifficultiesincluded‘adiversestudentbackgroundwithawiderangeofabilities’(16%),‘studentshaveweakmathematical/logicalbackground’(15%),and‘studentsarenotmotivatedanddonotcomeprepared’(15%).Someoftheparticipantsindicatedthatcertaintopicsarespecificallydifficulty,suchasformalreasoning(e.g.,naturaldeduction,proofs)andformallanguage(e.g.,symbolization,translations).

UseofEducationalSoftware:Mostoftheparticipants(81%)saidtheydidnotuseeducationalsoftwareinthistypeofcourse.Manyreasonsweregiven,butthemostpopularexplanationsincluded‘softwareisnotuseful—don’tlikethesoftware—don’tneedthesoftware’(28%)and‘notimetolearnthesoftware—itdistractsfromthematerials’(15%).Severalparticipantsindicatedthattheywouldbewillingtousethesoftwareifcertainconditionsweremet,suchas‘ifthesoftwarecouldhandlespecificfunctionslikeSATsolvers,proofs,andmodels’(18%),‘softwarewaseasytolearn’(11%),and‘iftherewereteachingassistantsandlabs’(11%).

10

Thosewhodouseeducationalsoftwareindicatedwhichsoftwaretheyused.ThemostcommonsoftwareusedweretheonesincludedwiththeBarwise&Etchmendybook(Tarski’sWorld,Fitch,andBoole),whichwereusedby22%oftheparticipants.Thereseemedtobethreemainusesofthesoftware:assessment(62%),demonstrations/examples(29%),andoptionalpractice(9%).Whilealmost30%oftheparticipantsindicatedthattheyhadnoproblemswiththesoftware,someparticipantsindicatedacoupledifficultiesthattheyhaveusingthesoftware,suchas‘studentsdon’tlikeitbecauseitwastooeasyortheydidn’tgetenoughfeedback’(29%),‘thereweretechnicalproblemswiththesoftware’(24%)and‘iteliminatesthinking’(18%).

Theparticipantswhoalreadyuseeducationalsoftwarewereaskedwhethertheywouldconsiderusingafullycomputer‐basedandhighlyinteractivesoftware.Approximately40%oftheparticipantssaidtheywouldconsideritundercertaincircumstances:asareplacementforastandardtextbook(62%),asthebasisforenrichedinstruction(55%),asthebasisforindividualizedinstruction(43%),andasafullreplacementforalecturecourse(18%).

SummaryWefoundthatsymboliclogic,whilenotuniversallytaught,isfrequentlytaughtatalltypesofuniversitiesexcepttribalcolleges,andthuswewishedtoexplorehybridinstructioninarangeofuniversitytypes.Further,thetotalpoolofuniversitiesteachingasymboliclogiccourseiswellover1000.Themostcommonsettingforteachingsymboliclogicwasinaphilosophydepartment,andthusweselectedphilosophydepartmentsasoursettingforexperimentation.ThereseemedtobegeneralopportunitiesforacceleratedlearningviamorecontentcoverageasthesurveysuggestedthatanumberoftopicsfoundintheL&Pcoursewerenotalwaystaught.Further,asignificantproportionofinstructorsseemedopentotheuseofatoollikeL&Pasareplacementofanexistingtextbook.

11

InClassVs.OnlineinLargeCourseStudiesInFall2007andSpring2008,studieswereconductedcomparingstudentspurelyusingL&Pwithnoface‐to‐facecomponentagainststudentsinatraditionalface‐to‐faceclass.Bothstudiesinvolvedthesameclassandsameinstructor.Theuniversitywasalargetop‐tierpublicresearchinstitutionwithahighproportionofEnglish‐as‐a‐SecondLanguagelearners.Theinstructorwasafull‐timelecturer,withsignificantresearchexperienceinsymboliclogic,significantteachingexperiencewithsymboliclogic,andsignificantexperiencewithvarioustoolsforonlineinstruction.Theclasswastraditionallyaverylargeclass,andthusincludedanumberofsimpleonlinetoolsembeddedintotheface‐to‐faceclass(e.g.,onlinequizzes).BothsectionshadaccesstoTAsforhelpwithexamsandhomework.

Participants

Fall2007

Condition Online In­classStartingcourse 50 57Finishingcourse 45 43%retentionrate 90% 75%Anewsectionofthecoursewascreatedshortlybeforethebeginningofthesemester,producingfewerthattypicalstudentsinthecourse.Studentsvolunteeredforin‐classversusonlineinstructionduringthefirstweekofclass.Apretestincludingdemographicswasgiventoassesspre‐existingdifferencesbetweenstudentsineachcondition.Critically,therewasnodifferencebetweentheperformanceonpretestoftheOLIstudents(M=8.7,SD=3.2,N=45)andtheIn‐classstudents(M=9.1,SD=2.8,N=49).Onalldemographicdimensions,therewereatmostminordifferences(i.e.,10%orless)betweensections(i.e.,gender,year,GPA,major,priorexperiencewithlogiccourses,priorexperiencewithformalproofs,priorexperiencewithweb‐basedorprogrammingenvironments).Therewasahigherattritionrateinthein‐classcondition,likelyproducingaperformancebiasinthatcondition.

Spring2008

Condition Online In­classStartingcourse 84 259Finishingcourse 83 105%retentionrate 99% 41%

12

Topics/ScheduleforFall2007andSpring2008

Week In­class Online Differences1 Introductiontocourse&

reasoningvalidityIntroductiontocourse&reasoningvalidity

2 Quantifierfreelogic:symbolizationandsemantics

Sententiallogic:syntaxandsymbolization

onlinedelayscoveringiff

3 Quantifierfreelogic:formalsyntaxandlogicalconcepts

Sententiallogic:semantics(truthfunctionsandlogicalconcepts)

onlinecoverstruthtreesoneweekearlier

Midterm1

SymbolizationNOT,AND,OR,IF...THEN,IFFsententiallogic.Syntax.Truth‐tablesemantics.Conceptsvalidity,logicaltrue,logicalfalse,logicalindeterminate

4 Quantifierfreelogic:semantictableau

Sententiallogic:derivations/proofs

5 Quantifierfreelogic:derivations/proofswithsententialconnectives

Sententiallogic:derivationswithindirectrules

6 Quantifierfreelogic:proofstrategies

Sententiallogic:strategiesandderivedrules

onlineaddsderivedrules

Midterm2

truthtrees/tableauandderivationsNOT,AND,OR,IF...THEN,IFF.

7 Predicatelogic:syntaxofquantification

Sententiallogic:metatheory onlineaddsmetatheory

8 Predicatelogic:translationsofquantification

Predicatelogicwithoutquantifiers:syntaxandtranslationsofquantification

9 Predicatelogic:quantificationsemantictableau

Predicatelogic(withquantifiers):syntaxandtranslationsofquantification,truthtrees

Midterm3

derivationsNOT,AND,OR,IF...THEN,IFF,symbolization,syntax,semantics,tableauALL,EXISTSpredicatelogic

10 Predicatelogic:quantificationsemantictableaucont.

Predicatelogic:derivations

11 Predicatelogic:derivations Predicatelogic:derivationswithstrategiesandderivedrules

onlineaddsderivedrules

12 Predicatelogic:derivationswithstrategies

Predicatelogic:identity(numericquanifiers,definitedescriptions,andderivationswithtruthtrees)

13 Predicatelogic:identity(semantictableauandderivationswithidentity)

Predicatelogic:functionsymbols

onlineaddsfunctionsymbols

FinalExam

Comprehensive

13

Studentsregisteredforface‐to‐faceversusonlineinstructioncourses,withadditionalrecruitingfortheonlinecoursetakingpartjustpriortothebeginningofthesemesterfromstudentsenrolledinthein‐classcourse.Thelecturecoursewasmuchlargerthaninthepriorsemester,reflectingmoretypicalenrollmentsinthiscourseatthisuniversity.Interestingly,attritionintheonlinecoursewasalmostnon‐existent,whereasthein‐classconditionhadveryhighattrition(althoughattypicallevelsforthislarge,rigorouscourse).Thisdifferentialattritionratelikelyproducesverysignificantperformancebiasesinfavorofthein‐classcondition.Thesamepre‐testwasadministered.TherewasnosignificantdifferencebetweenOnline(M=10.8,SD=4.7,N=65)andtheIn‐classstudents(M=10.4,SD=4.5,N=83)onthepre‐test.Therewerealsonodemographicdifferencesgreaterthan10%betweenconditions.

ResultsInbothsemesters,studentshadthreeexamsduringthesemester,onecumulativefinalexam,andweeklysurveysregardingtimespentattendinglecture,readingmaterial,gettingTAhelp,doinghomeworkonthecomputer,anddoingpracticeexamquestions.Theexamswereconservativelyselectedbasedonpriorexamsfromtheinclasssection,producingalikelyminorbiastowardstheinclasssection.Somequestionsweremodifiedfortheonlinesectiontoreflectminordifferencesinlogicnotationsusedinthetwosections.

Semester Fall2007 Spring2008Condition Online Inclass Online Inclass

Exam1 68% 69% 72% 70%Exam2 44% 55% 55% 60%Exam3 55% 58% 54% 54%

FinalExam 63% 62% 62% 65%Totaltime 8hrs/wk 7hrs/wk 8hrs/wk 9hrs/wk

Inthefall,therewerenosignificantdifferencesbetweensectionsonanyoftheexams,althoughatrendtowardsbetterperformanceintheinclassconditiononexam2.Onlinestudentsreportedslightlyhighertotaltimeonthecourse,primarilyreflectinglargespikesintimeontaskinweeks5,6,8,and9.Thesemomentsreflectthebeginningofmoredifficultproofconstructions.Asaresultofthesereportedspikesineffort,thehomeworkcontentwasadjustedforthefollowingsemester.

Inthespring,therewereagainnosignificantdifferencesbetweensectionsonanyoftheexams,althoughtherewasthesametrendtowardsbetterperformanceintheinclassconditiononexam2.Thistimetheonlineconditionreportedmoretotaltimeperweekintheweeklysurveys,althoughthedifferencewasnotstatisticallysignificant.

SummaryTheresultsofthesetwostudiessuggestthatsimilarlearningoutcomescanbeobtainedinthiskindofsettingfromapureonlinecourseasfromaprimarilyface‐to‐faceinstructioncoursewithroughlyequivalenttime‐on‐task.Notehoweverthatthiscomparisonisconservativebecausethereweremuchhigherattritionratesintheface‐to‐faceinstruction

14

case.Further,althoughwedidnothaverandomassignmenttocondition,thehugedifferencesinattritionratearestronglysuggestedofanimportantretentionratebenefitoftheL&Pcourseovertheexistingtraditionalinstructionmodel.Largelecturecourseswithrigorouscontent(ascanbeseenbytherelativelylowexamperformanceacrosstheboard)arefrequentlysubjecttohighattritionrates,andthusthisretentionratebenefitcouldbeaveryimportantresultforimprovingtheefficiencyofuniversityeducation.

15

SameTime/MoreContentinaCommunityCollegeThe first accelerated learning study involved trying to cover significantly more content in a community college context. As would commonly be the case in the community college context, the instructor had a high load (4 courses/semester) of moderate enrollment courses (25 to 30), no research experience with symbolic logic, a modest teaching experience with symbolic logic, and little experience with online tools in instruction. The existing course typically covered less than half of the L&P full course: it included only sentential logic and with relatively little inclusion of full proofs even within sentential logic. The instructor’s goal for the current study was to cover as much of the full L&P course as possible in a hybrid instruction mode: one lecuture per week, with students experience primary instruction through online instruction and practice. Student performance on the shared content would be compared against prior semester performance on exams derived from previous semester exams (3 sections from the prior semester, n=85 students, against 3 sections of the current semester, n=85 students).

Context&ParticipantsParticipants included students registered for one of three sections of an Introduction to Symbolic Logic course offered at a midwestern U.S. community college (see Table 5 for background information—i.e., gender, year, major, GPA, current course load, reason for taking course, goal grade, and computer experience). The majority of the participants had not experienced formal/symbolic logic, history of logic, or rigourous proofs. Only 30% of the participants took a single course that had formal/symbolic logic—11% took more than one of these courses. Only 17% of the participants took a course with history of logic—1% took more than one of these courses. Only 15% of the participants took a course with rigorous proofs—8% took more than one of these courses. Overall, the participants did not do well on the pretest (M = .31; SD = 1.59; range 1 to 8 out of 16). They only got 4% of the statement questions correct (SD = 0.36; range 0 to 1 out of 5), 15% of the argument questions correct (SD = 1.00; range 0 to 4 out of 4), 64% of the equivalence questions correct (SD = 1.08; range 0 to 5 out of 5), and none of the proof questions correct (out of 2).

AttritionRatesThecommunitycollegeproducedananalysisoftheattritionratebetweenthetwosemestersinquestion.Theanalysiscomparedtheseatsattritionfromtenthdaytoendofterm.TenthdayseatsarebasedontotalgradesexcludingX,whileendoftermseatsarebasedontotalgrades(excludingX)minusWsandN,followingthecollege’sinternaldefinitionofattrition.Theattritionratesforbothpriorandcurrentcourseswereidenticalat‐8.2%.

TimeonTaskParticipants in the current semester were asked to rate the workload, difficulty, and amount of material covered in this course compared to other classes that they are taking. A 5-point scale (-2 to 2) was used, with negative numbers indicating less than other classes and positive numbers indicating more than other classes. A one sample t-test was used to compare the average (workload = -0.18; difficulty = 0.25; amount of material = 0.16) on each of the possible ratings against 0 (same as other courses). The averages were only not significantly different from 0,

16

indicating that the workload, difficulty, and amount of material was the same as in other classes. Further, while difficulty and amount of material might have trended towards being higher than average, workload (critical for an accelerated learning study) trended towards lower than average.

PerformanceonMidtermExamsThe prior semester had three midterm exams and an optional final exam, while the current semester only had one midterm exam (basically the content of the prior midterm 1) and one required final exam (including content from prior midterms 2 and 3 plus new content not previously covered). The current semester performed at considerably higher levels on their midterm (M = 81%; SD = 15) than those who took the comporable first midterm exam in the prior semester (M = 69%; SD = 20), t(155)=4.24, p<.0001, Cohen’s D=0.7.

FinalExamPerformanceThepriorsemesterdidnothavearequiredfinalexam,sothefinalexamforthecurrent(fall2008)semesterwascreatedfromthesecondandthirdmidtermfromthespringsemester.Additionalquestionswerealsoincludedthatcoverednewmaterialthatwasnotcoveredinthepriorsemsester.Thismaterialincludedbeingabletoreflectandexplainstrategiesusedtosolveaproof,constructingatruthtree,andtranslationsusingpredicatelogic.Thetwosectionswereanalyzedseparately.

Therewerenosignificantdifferencesbetweenthespringsemester’sperformance(M=75%,SD=26)andthefallsemester’sperformance(M=72%,SD=16)onmaterialthatwascoveredbybothsemesters,t(150)=‐0.72,p=.47.Thefallsemesterdidfairlywellontheextraquestions(M=78%;SD=18).

SummaryInacommunitycollegecontext,acceleratedlearningwasachievedusinghybridinstruction:someofthepreviouslycoveredcontentwaslearnedtogreaterlevelsandconsiderableadditional,moreadvancedcontentwasalsolearned.Thisgainincontentlearnedtookplaceoverthesamesemestertime,withoutcreatinganatypicallyhighworkloadcourseandwithoutanyriseinthecourseattritionrate.

17

Table 5. Background information for the Community College Acceleration Study Gender 34 male; 28 female Year 19 freshmen; 29 sophomores; 12 junior; 0 seniors; 1 5th year or above Major

GPA

Current Course Load

1 one course; 2 two courses; 3 three courses; 2 four courses; 39 five courses; 15 six courses

Reason for Taking Course

18 meets the university requirement; 17 thinks the subject is interesting; 14 major requirement; 6 covers skills I want to develop; 4 fits well into schedule; 1 expect the course to be challenging; 1 hear the teacher was good; 1 prerequisite

Goal Grade 50 A; 9 B; 3 no goal grade Computer Experience

15 commented on a blog/wiki 13 used educational software 27 worked with spreadsheets 3 worked with statistical packages 4 proficient in computer programming

18

AccelerationforcommuterstudentswithoptionalrecitationsessionsInmanyregionalcampusesofpublicinstitutionsintheUS,alargeproportionofthestudentspreferonlineinstructionbecauseitallowsthemtoworkand/orcareforfamilymemberswhileattendingcollege.Inthiscontext,itisdifficulttorequirehybridinstruction.Thisstudyexaminedwhethertherelativelyamountofhybridinstructionwouldallowstudentstocovermorecontentwithinthesemester.

Atthisuniversity,L&Phadbeenusedinonlineonlyformatforanumberofyears,althoughthecurrentinstructorwasnewtoteachingthiscourse.AsinpriorsemestersoftheL&Pcourseinthiscontext,studentswereallowedtocompletethecourseattheirownpace.Additionally,gradesfocusedoncontentcoverageratherthanthemoretypicaldegreeofmastery:Themorethatstudentscompleted,thehigherthegradetheywouldbeassigned(assumingminimallevelsofproficiencyweremetoneachsectioninthefinalexam).

Throughoutthesemester,optionalrecitationsessionswereofferedtohelpthestudentscompletetheworkinatimelyfashion.Itwaspredictedthatifstudentstookadvantageoftherecitationsessions,thentheywouldlikelycompletemorematerial.

ContentLearningResultsForeachstudent,wecalculatedamountofmateriallearnedbystudentsasthenumberofchaptersforwhichtheyreceivedapassinggradeinthefinalexam.Unlikewhatwashypothesized,therewasnotasignificantrelationshipbetweennumberofrecitationsattendedandamountofmaterialcompleted(r=.1,p>.4).

AttendanceOneclearproblemwiththisstudy,however,isthatthemajorityofthestudentsdidnotattendanyoftherecitationsessions,andonlyacoupleofstudentswereregularattendees(seeTable6).Basedonthedistributions,cutoffsforagroupingvariableseemedtobethosewhodidnotattendanysessions(N=23),thosewhoattendedonetothreesession(N=2),andthosewhoattendedfourormoresessions(N=11).1However,analysesofexamperformanceinvolvingthesegroupswerenotsignificanteither(p>.6),althoughtherewasaslighttrendforthezerosessiongrouptodolesswellthantheothertwogroups.

Increasesinfinalgradeswereofferedasincentivetoattendtherecitationsessions.Ifstudentsattendedsixormoresessions,thentheywouldreceiveabumpintheirfinalgrade(i.e.,B‐wouldbeincreasedtoaB).Ifstudentsattendedtwelveormoresessions,thentheywouldreceivetwobumpsintheirfinalgrade(i.e.,B‐wouldbeincreasedtoaB+).Consideringthesecutoffs,fourgroupswerecreated:thosewhodidnotattendanysessions(N=23),thosewhoattendedatlessthansixsessions(N=4),thosewhoattendedatleastsixsessionsbutlessthan12(N=7),andthosewhoattended12ormoresessions(N=2).Again,therewerenosignificantdifferencesbetweentheseconditions(p>.6)

1 Total N for this analysis is less than the amount indicated in the table. There was some loss due to students not taking the final exam.

19

Table 6. Distribution of attendance at the recitation sessions.

numberof

sessions

numberofstudentsinattendance

cumulativenumberofstudentsinattendance

(increasing)cumulativenumberofstudentsinattendance(decreasing)

0 68 68 931 10 78 252 2 80 153 1 81 134 3 84 126 2 86 97 3 89 710 1 90 411 1 91 312 1 92 215 1 93 1

SummaryEvenwithagradeincentiveforattendingrecitationsections,studentsinthisuniversitycontextwereunlikelytodoso.Thesmallamountofattendenceinhybridinstructionwasnotclearlyassociatedwithincreasesinamountofmateriallearned.Insufficentstudentscompletedthepre‐testforustoassesswhetherrecitationattendencewascorrelatedwithweakerperformance.However,thatremainsaclearpossibleexplanationforwhyrecitationattendencewasnotcorrelatedwithbettercourseperformance.

20

SameTime/MoreContentwithHybridInstructioninaUniversityThecontextwiththehighestlikelihoodofteachingsymboliclogicisthenationaluniversity.Relativetothecommunitycollegecontext,theinstructorsarelikelytobemorefamiliarwithsymboliclogiccontent,theirstudentsaremorelikelytobealreadybelearningsomeofthemoreadvancedcontentfoundinL&P,andtheinstructorshavelowerteacherloadsbutperhapslargerclasses.However,therearesomeopportunitiesforcoveringmorecontentusingL&PthanwiththeexistingcourseasthesurveyofcoursesdidfindthatnotallofL&Pcontentwastypicallycovered.Further,inthiscontextstudents,studentswerepreviouslyattendingtraditionalsymboliclogicclasses,andthushybridinstructionispossible.

Theparticularcontentthatwasnotpreviouslyfoundinthestudiedcontextinvolvesomeaspectsofmetamathematicsandstrategicelementsofproofconstruction.

Context&ParticipantsParticipantsincludedstudentsregisteredforanIntroductiontoSymbolicLogiccourseofferedata4thtieruniversity.Two‐thirdsofthesestudentsweresophmoresfromcomputerscienceorengineeringbackgrounds,andthuswererelativelywellmatchedtolearningthiskindofcontent(e.g.,halfthestudentshadpriorexperiencewithrigorousproofs).

Studentswereofferedanopportunitytoparticipateinthisstudy.Outofthetotalof81(63male)registeredstudents,31studentsvolunteered(22male).Thevolunteerswererandomlyassignedtooneoftwoconditions:face‐to‐face(N=10)orhybrid(N=21).The50non‐volunteerswerecompletedthesameworkastheface‐to‐faceparticipants.

Thefirstanalysescomparedthebackgroundandpriorknowledgeofthevolunteers(i.e.,collapsingtheface‐to‐faceandhybridconditions)tothenon‐volunteers.Therewereveryfewdifferencesbetweenthetwoconditions.Thevolunteersoutperformedthenon‐volunteersonthepriorknowledgeportionofthepretest(volunteers:M=10.1,SD=2.1;non‐volunteers:M=8.9,SD=2.5;t(79)=‐2.2,p=.03)—especiallyonthestatementsquestions(volunteers:M=3.1,SD=1.1;non‐volunteers:M=2.6,SD=1.3;t(79)=‐2.1,p=.04).Thevolunteersonlyhadslightlymorerigorousproofclassesthanthenon‐volunteers(volunteers:M=0.7,SD=1.1;non‐volunteers:M=0.3,SD=0.6;t(79)=‐1.9,p=.07).Therewerenosignificantdifferencesforknowledgequestionsinvolvingarguments,equivalence,orproofs,gender,yearinschool,major,totalcoursescurrentlytaking,overallgradepointaverage,computerexperience,andhowmanypreviousformallogicandhistoryoflogiccoursestheytook.

Becausethereweresofewdifferencesbetweenthevolunteersandnon‐volunteers,thenon‐volunteersandface‐to‐facestudentswerecollapsedforalltheremaininganalyses.Therewerenotevenmarginallysignificantdifferencesinbackgroundandpriorknowledgebetweentheresultingface‐to‐facestudentsandhybridstudentgroups.

21

TimeonTaskAttheendofthesemester,studentscompletedasurveyaboutthecourseworkload.Sixstudents’surveyswerenotincludedintheanalysis:threecouldnotbeidentifiedfromtheinitialclassroster,twodidnotprovidenames,andonedidnotcompletehalfthesurvey.

Overall,therewerenodifferencesinthetotaltimeontaskbetweentheface‐to‐facestudents(M=14.1,SD=6.4)andthehybridstudents(M=11.1,SD=3.0),t(55)=1.6,p=.11.Notsurprisingly,theface‐to‐facestudents(M=1.9,SD=0.8)spentmoretimeinlecturethanthehybridstudents(M=0.5,SD=0.7),t(55)=5.5,p<.0001).Theyalsospentmoretimeontheirhomework(face‐to‐face:M=3.0,SD=1.3;hybrid:M=1.9,SD=0.5;t(55)=2.9,p=.01).However,thehybridstudentsspentmoretimeinrecitation(face‐to‐face:M=1.5,SD=0.8;hybrid:M=2.1,SD=1.0;t(54)=‐2.2,p=.03).

Studentswerealsoaskedfourquestionsabouttheirperceptionsofthecourse(i.e.,howinterestingwasthecoursecomparedtoothercourses;howhelpfulwastherecitationcomparedtoothercourses;howdifficultwasthematerialcoveredinthiscoursecomparedtoothercourses;howlikelyareyoutotakeafollow‐upcoursethatbuildsonthismaterials).Theyweretoratethequestionsonascaleof1(notatall)to5(verymuch/likely).Therewerenodifferencesbetweenthethreeconditions.

PerformanceonFinalExamAllstudentstookthesamefinalexam,whichcoveredthefollowingtopics:SymbolizationintoRL(multiple‐choice);TranslationfromPL(multiple‐choice);OtherQuestions(multiple‐choice);Symbolization(short‐answer);Translation(short‐answer);OtherQuestions(short‐answer);DerivationJustifications;DerivationViolations;TruthTableMethod;Derivation;TreeMethod;Translation&Tree/Derivation.Eachofthesetopciswereanalyzedseparately.Threestudents’examswerenotincludedintheanalysesbecausetheycouldnotbeidentifiedoninitialtheclassroster.

Twoversionsofthetestweredistributedwithineachgroup.Theonlydifferencebetweenthetwoversionswastheorderinwhichthequestionswerepresented.Therewasnosignificantdifferencesinperformancebetweenthetwoversions(version1:M=43.0,SD=14.7;version2:M=47.5,SD=14.2;t(60)=‐1.2,p>.2)andthusthetwoversionswerecollapsedfortheremaininganalyses.

Overall,theface‐to‐facestudents(M=47.6,SD=13.2)outperformedthehybridstudents(M=37.3,SD=16.6)onthefinalexam,t(60)=2.4,p=.02).Theydidbetterontheshort‐answersymbolizationquestions(face‐to‐face:M=3.5,SD=1.9;hybrid:M=2.2,SD=2.0;t(60)=2.2,p=.03),thederivationviolationquestions(face‐to‐face:M=2.5,SD=0.9;hybrid:M=1.4,SD=1.4;t(60)=3.2,p=.001),thetruthtablemethodquestions(face‐to‐face:M=3.8,SD=1.6;hybrid:M=2.7,SD=2.3;t(60)=2.0,p=.05),andthederivationquestions(face‐to‐face:M=15.6,SD=7.5;hybrid:M=10.4,SD=8.1;t(60)=2.2,p=.03).Thehybridstudentsdidslightlybetteronthemultiple‐choicesymbolizationquestions(face‐to‐face:M=2.3,SD=1.7;hybrid:M=3.2,SD=1.8;t(60)=‐1.7,p=.10).

22

SummaryInthiscontext,therewasnoapparenttestperformancebenefitforhybridinstructionovertraditionalface‐to‐faceinstruction.OnemightarguethatbetterperformancewouldhaveresultedfromhybridinstructionhadtheinstructorandTAhadmorefamiliaritywiththeL&Psystem.Goodhybridinstructioninvolvesinstructorslearninghowtomakesenseoffeedbackthesystemprovidesonstudentlearningandthenprovidingtargettedinstruction.ItispossiblethattheinstructorandTAcontinuedtouseoldinstructionalstylesinthenewcontext.However,theresultsdosuggestthatgainsmaynotbefoundimmediatelyontheswitchtothiskindofhybridinstructioninthiskindofcontext(contentknowledgeableinstructorsworkingwithrelativelystrongstudents).

23

AcknowledgementsWewouldliketoacknowledgethetirelesscontributionsofWilfriedSieg.Thisevaluationeffortwasamajorundertakingthatcouldnothavebeendonewithoutin‐depthinputfromhimateverystepoftheway.Inaddition,DawnMcGlauglin,DavinLafon,LeslieBurkholder,RebeccaThall,DaveBeisecker,andVictoriaRogersprovidedconsiderablehelpalongtheway.

24

ReferencesCarnegie Foundation for the Advancement of Teaching (http://www.carnegiefoundation.org/) US & World News Reports America’s Best Colleges

(http://www.usnews.com/usnews/edu/college/rankings/rankindex_brief.php) Wikipedia

(http://en.wikipedia.org/wiki/List_of_historically_black_colleges_of_the_United_States)