Next-Generation Learning: What Is It? And Will It Work? (166264520)

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 1/43

BarbaraMeansCenterforTechnologyinLearning

February6,2013

Next Generation Learning:What Is It? And Will It Work?

Presentation for the annual meeting of the EDUCAUSE Learning Initiative (ELI),Denver, CO.

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 2/43

Overview

•  TheConceptofNext-GeneraonLearning

•  Next-GeneraonLearningChallengesWaveI

•  EvaluaonFindings•  ContrasngWaveIProjects&MOOCs

•  TwoViewsontheNatureofEffecvenessEvidenceinLearningTechnologyR&D

•  StrategiesforResolvingtheTensions

•  FinalThoughts

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 3/43

NextGeneraonLearning:APersonalView

•  BeUerpreparesstudentsforaworldthatvaluesandrewardsdeeperlearning,collaboraon,skilledcommunicaon,selfmanagement,theabilitytoworkacrossdisciplines,andinnovaonpracces.

• Meetseachstudentwhereheorsheisandprovidescontent,pedagogy,andaccessopportuniestomeetindividualneeds.

•  Capitalizesontheaffordancesoftechnologyforlearning•  Collectsdetaileddataabouttheprocessoflearningthatcanbeusedtodiagnosestudentneeds,providefeedbacktothe

instruconaldeveloper,andcombinewithdatafromotherlearningsystemstogeneratenewinsightsaboutlearningandinstrucon.

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 4/43

•  Addressre-shapingthecurrentlyfragmented,sub-scalemarketforlearningsoluons

•  Supportdisrupvechangeinbothsupplyanddemandforeffecve

soluons

•  Fundflexibleresourcesthatinnovatorscanusetocreatechange•  Breakdowntechnical&instuonalbarrierstoadopng&scalingeffecvesoluons

NextGenera*onLearningChallengesseektopinpointtechnologysolu*ons

thatcanmeasurablyimprovethequalityoflearningexperiencesand

improvestudents’collegereadinessandchancesofcomple*on. 

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 5/43

•  Goalofdemonstrangeffecvenessatscale

•  MajorgrantfromtheGatesandHewleUFoundaonsto

EDCASE

•  SubgrantsfromEDCASEto29WaveIprojectsunder4

challengeareas

•  Projectsranfromspring2011tofall2012orspring2013(15or

21months)

NextGenera*onLearningChallengesseektopinpointtechnologysolu*ons

thatcanmeasurablyimprovethequalityoflearningexperiencesand

improvestudents’collegereadinessandchancesofcomple*on. 

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 6/43

TheChallenges:DeeperLearning

• Definion–Richlyinteracvetechnologiesthatincreasestudentengagementandlearningofconceptualcontentand21st-centruryskills

• Example–UniversityofWisconsin-Milwaukee’sU-Pace

 – Self-pacedIntroductoryPsychologycourse – Mastery-basedwithshortermodulesandend-of-module

quizzes

− Timelyandtailoredfeedback

− Instructorsupportandencouragement

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 7/43

TheChallenges:BlendedLearning

•  Example–CaliforniaStateUniversityNorthridge

 – Redesignedgatewaymathcourseashybridalternaveto

convenonalcollegeAlgebra

o Individualuseofanintelligenttutorforremediangmathskills

o Interacvelecturesusingacommonsetofmaterialsacrosssecons

o Facilitatedgroupworkforanhouraweek

o Instantfeedbackononlinehomework

o Commonstudentassessmentusedacrossallcoursesecons

• Definion–Combinaonsofonlineandteacher-ledinstruconthatimprovelearning,increasecompleon,andlowercosts

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 8/43

TheChallenges:OpenCoreCourseware

• Definion–High-quality,modular,openlylicensedcourse-warefordevelopmental,gateway,andhigh-enrollmentcorecourses

•  Example–CerritosCollege’sProjectKaleidoscope

 – Large-scaleprojectpromongtheadoponofopeneducaonresources(OER)

− Targengstudentsfromunder-representedgroups

− 12differentOERcoursesimplementedon9campuses

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 9/43

TheChallenges:LearningAnalycs

• Example–MaristCollege’sOpenAcademicAnaly*cs

Ini*a*ve

 – SakaiApplicaonProgrammingInterfacetosupportcapturingstudents’courseeffortfromSakaieventlogs

 – Open-sourceacademicpredicveanalycsmodel,basedonPurdue’sCourseSignals

• Definion–sowareforcollecon,analysis,andreal-meuseoflearningdatabystudents,instructors,andadvisorstoimprovestudent

success

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 10/43

AssumponsBehindtheIniave

•  Thereisasetof“islandsofinnovaon”witheffecvetechnology-supportedsoluonsthatcouldbescaledtootherinstuons.

•  Technicalandinstuonalbarriersgetinthewayof

scalingtechnologysoluonsthatimprovelearningandchancesofcompleon.

• NGLCprojectscancreateanimprovedmarketplacewithdevelopersandadoptersworkingtogether.

•  Thesepartnershipswilldevelopandsustaineffecvesoluonswiththeresultthatstudentengagement,performance,persistenceandcompleonallrise.

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 11/43

By the Numbers

29 funded proposals

200 institutions

78 community

colleges

117,000 students currently

served

72,945 students by grant end

[Wave I Planned]

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 12/43

By the Numbers

29 funded proposals

121 institutions

53 community

colleges

students currently served

63,567 students by grant end

[Wave I Actual to Date]

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 13/43

NGLC Wave I Participants

•  78% of faculty who used Wave I innovations said they woulduse them again.

•  48% of studentsexperiencing the NGLCinnovations were at 2-year colleges.

• 40% of student participantswere low income (Pell granteligible).

•  Average course completionrate was 77% (93% in 4-year colleges and 64% incommunity colleges).

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 14/43

WhatWeLearned

•  MostoftheWaveIinnovaonsdidnotreallyhaveevidenceofeffecvenessbeforethegrantsbegan.

•  Manytechnologycomponentswerenotcompletelydeveloped

beforefall2011.

• Themostfrequentlyreportedimplementaondifficulesweretechnologyandstudentresistance.

•  Manyfacultywillvolunteertotryoutnewlearning

technologies.

•  Mostfacultydidnotviewtheseparcularinnovaonsas

parcularlychallengingtoimplement.

•  Butthosewhofelttheinnovaonrequiredthemtochange

thewaytheytaughtweremoreposiveabouttheinnovaon

theyhadused.

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 15/43

Technology Problems Were the Biggest

Barrier to Implementation

But most instructors did not regard the barriers as major.

Implementa)onBarrier Percent

Technologyproblems(e.g.,access,reliability,ease

ofuse)

57

Studentresistance 41

Lackofalignmentwithmyteachingapproach 11

Didn’tworkwellwithcoursescheduleoralloUed

me

9

Lackofalignmentwithcoursegoals 4

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 16/43

Instructors Were More “Favorable” Who . . .

•  Had volunteered to use the innovation

•  Attended training on the innovation

•  Said the innovation provided student data and theyused it

•  Said online assessment data had affected how theytaught

•  Described the innovation’s effect on the way theytaught as “major”

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 17/43

Instructors Were Less “Favorable” Who

Reported . . .•  Technology challenges

•  Lack of alignment with their teaching approach

•  Lack of alignment with their course goals

•  That the NGLC innovation didn’t fit into their allottedclass time

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 18/43

BroaderImplicaons

•  Thecombinaonofcampusimpedimentstoafaststartandlackofsowarereadinessmeantthat15monthswasnotenoughme.

•  Manyfacultywillvolunteertotryoutnewlearningtechnologies,andthesevolunteersrespondmoreposivelytoinnovaonsthatpushthemtochangetheirinstruconal

pracces.•  Fewonlineandblendedlearninginiavesaresetuptocollectrigorousevidenceoftheirinnovaon’simpactonstudents.

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 19/43

BarrierstoCollecngRigorousEvidence

•  CampuspoliciesorIRBsmayprohibitassigningstudentstocourseswithsignificantonlinecomponentsatrandom.

•  CampusIROsmaynotbewillingtoreleasethestudent-leveldatathatwouldbeneededtocontrolforpriordifferencesin

analysescomparingcurrenttopriorcohorts.•  It’sdifficulttogetdifferentinstructorstoadministerthesameassessment.

•  Valid,reliableassessmentswerenotreadilyavailableformanyoftheprojects’learningobjecves.

MostoftheWaveIinterven*onswerenotmatureenoughtoconsiderstudentoutcomesduringthegranttermasthefinalwordontheireffec*veness.Butstudentoutcomedataareimportantas

benchmarksforimprovementefforts.

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 20/43

Example:U-Pace

• Compared230studentsin-PacePsychologyto334studentsinconvenonalPsychologycourse

• Posiveeffectson – PercentageofstudentsearninganAorB(ES=+.96) – Coursecompleon(ES=+.35)

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 21/43

Example:CaliforniaStateUniversityNorthridge

• Compared4,479studentswhotooktheNGLCHybridcourseto1,824studentsfrompastcohortswhotookCSNcollegealgebra

• Posiveeffectson – Coursegrade(ES=+0.49) – Selectedcommonfinalexamproblems(ES=+0.68)

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 22/43

Example:ProjectKaleidoscope

•  Posiveeffectsoncoursecompleonandpersistencecomparedtopriorcoursetaughtbysameprofessorsfor

 – BeginningAlgebra(ES=+.51) –  IntermediateAlgebra(ES=+.60)

 –  IntroducontoPsychology(ES=+.48)• Negaveeffectsfor – PhysicalGeography(ES=-.24)

•  Nosignificantdifferencefor –  DevelopmentalWring(ES=-.14)

 –  DevelopmentalReading(ES=+.08)

 –  EnglishComposion(ES=-.34)

 –  BusinessFundamentals(ES=+.11)

 –  BusinessManagement(ES=+.01)

 –  FundamentalsofBiology(ES=+.38)

When students do better or the same with OER, it’s advantageous because

students are spared the costs of textbooks.

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 23/43

ChallengesforInterprengAvailable

Evidence• Categoriesareill-defined,intervenonsarediverse,andbothtechnologyandpraccearechangingrapidly.

•  It’sdifficulttogetapple-to-applescomparisons.Oen

- Instructorisconfoundedwithcoursemedium.

- Mulpleaspectsofinstruconvarybetweencoursesorsecons

withandwithouttheNGLCinnovaon.

- Studentcharacteriscsmayvarybetweencourses,secons,and

cohorts.- Differenalretenoninonlineandclassroomseconscanbias

outcomes.

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 24/43

BoUomLineonNGLCWave1

•  Impactsonstudentoutcomesweremixed.

• TheNGLCassumponthattherewerealreadyproven-effecvenext-generaonlearninginnovaonsthatjustneededmodestsupportfor

scalingprovedtobeincorrect.• TheWaveIinnovaonsdidnotrequirefundamentalchangesonthepartofhighereducaoninstuons,andthemajorityofthemdidnotrequirefacultytomakemajorchangesinhowtheyteach.

• Themajorityofprojectsneededmorethan15monthstomeettheirgoals.

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 25/43

BoUomLineonNGLCWave1

•  Impactsonstudentoutcomesweremixed.

• TheNGLCassumponthattherewerealreadyproven-effecvenext-generaonlearninginnovaonsthatjustneededmodestsupportfor

scalingprovedtobeincorrect.• TheWaveIinnovaonsdidnotrequirefundamentalchangestohighereducaoninstuonsandthemajorityofofthemdidnotrequirefacultytomakemajorchangesinhowtheyteach.

• Themajorityofprojectsneededmorethan15monthstomeettheirgoals.

 At the same time we saw . . .

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 26/43

The Meteoric Rise of MOOCs

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 27/43

In the News

• HowCalifornia’sOnlineEducaonPilotwillEndCollegeasWeKnowItTechCrunch

• OnlineCoursesNeedHumansto

EducateCNN

• APromisingTimeforOnlineLearning,HigherEducaonKansasCityStar

•  Is2013theYearoftheMOOC?

Informa*onWeek

• AreTheyLearningorCheang?Forbes

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 28/43

•  Scores of courses from 33 universities•  Over 2.3 million people have signed up for Coursera’s

free courses

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 29/43

Typical MOOC Features

• Availableforfreeorverylowcost

• Notforcredit(yet)• Designedtobeusablewithverylargenumbersofstudents

• Classpacingandassignmentsandtests

• Automatedscoringofstudentresponsesand/orpeergrading• Mechanismsforpeer-to-peerinteracons

• Bite-sizedlecturesegments 

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 30/43

WhydidMOOCstakeoffwhileNGLC

innovaonsscaledincrementally?• Aerbriefincubaoninauniversity,theywerelaunchedbyexternalorganizaons

• Theseorganizaonswereincenvizedtoscaletheir

innovaonstomanyinstuons,usuallyIHEsarenot.

• Theyreceivedmajorventurecapitalfunding(millions,notthousands)

• Theyembodiedabig,excingidea

• Theyhadtop-notchprogrammingtalent

• TheirleadershadstrongPRskills 

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 31/43

NotReallyaFairComparison

• MOOCsarefreeandopentoanyone

• Registraonisnotthesameascompleon

0

10

20

30

40

50

60

70

80

90

AchievingtheDream NGLCWaveI MOOCs

Series1

 Achieving the Dream NGLC Wave I MOOCs

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 32/43

TwoViewsonEvidenceinaDigitalWorld

•  ProducedforOfficeofEducaonalTechnologyof.S.DepartmentofEducaon

•  Describesapproachestoevidencegatheringthatcapitalizeondigitallearningdata

•  Presentsaguidanceconcerningthestrengthofevidenceneededfordifferentkindsoflearningtechnologydecisions

http://www.ed.gov/edblogs/technology/evidence-framework/

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 33/43

TwoViewsonEvidenceinaDigitalWorld:

PrevailingModelinEduca*onResearch

EarlyStage

EfficacyStudy

EffectivenessStudy

ProvenInnovation

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 34/43

TwoViewsonEvidenceinaDigitalWorld:

TechIndustryR&DModel

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 35/43

TensionBetweenIHEConstraints&Rapid

Scaling•  Tradionofindividualfacultydevelopmentoftheirowncoursesreducesinterestinexternallydevelopedcourseware.

•  IHEsmoveatanacademic(statelyifnotglacial)pace.

• Courseapprovalandposngpoliciesdelayimplementaonofnewkindsofcourses.

•  Instuonsrequireinformingstudentsinadvanceifacoursewillbeconductedlargelyonlineandsomemes

prohibitofferingacourseonlyonline.

• Anycourse-lengthintervenoncanonlyfitin2-3iteraonsayearonagivencampus—prevenngtherapiditeraonswithrefinementthatspeedupimprovement.

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 36/43

TwoStrategiesforResolvingtheTension

• Gooutsidethecoreformaleducaonsystem(cf.Christensen’sDisrup*ngClass)

 – StudyHelp – Games(self-iniated)

 – MOOCs

• Collaboraonsamongdesigners,researchers,andfacultytoconduct“micro-trials”

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 37/43

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 38/43

TheImprovementCycle

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 39/43

WhySpeedMaUers

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 40/43

•  PathwaytoCollegeMathiniavebringsresearchersandcollegestogethertotackle“apersistentproblemofpracce.”

•  Sharedgoalofdoublingthenumberofstudentswhoearn

collegemathcreditwithinayearofconnuousenrollment.•  Combinaonsofnewpolicies,coursestructures,supportsand

coursecontenttriedoutatvariousparcipanginstuons.

•  Offeringnewcoursesonadigitallearningplaormmakesitpossibletocollectdetailedlearningdataonstudents’

engagementwithindividualproblems.•  Groupisexaminingthesedatabothtopinpointaspectsofthe

coursethatcanbeimprovedandtounderstandlocalvariaons.

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 41/43

FinalThoughts

• Wemaynotbequitesurewhatitis,but“next-generaonlearning”iscoming

• Now isthemetostateandtestdesignprinciplesfornext-

generaonlearning.

• Ascollegeinstruconbecomesincreasinglydigital,there’sawealthofdatathatcanbeanalyzednotonlytoimprovetheparcularproductbutalsototestdesignprinciples.

•  Thatrequirescollaboraonsamongcolleges,researchers,

dataminers,andtechnologydevelopers.•  Ifwedon’tcollectcredibledataonnext-generaonlearningimpacts,the“winners”willbethosewiththebestPR.

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 42/43

Headquarters:SiliconValley

SRIInterna,onal

333RavenswoodAvenue

MenloPark,CA94025-3493

650.859.2000

Washington,D.C.

SRIInterna,onal

1100WilsonBlvd.,Suite2800

Arlington,VA22209-3915

703.524.2053

Princeton,NewJersey

SRIInterna,onalSarnoff

201WashingtonRoad

Princeton,NJ08540

609.734.2553

 Addi*onalU.S.andinterna*onalloca*ons

www.sri.com

ThankYou

7/29/2019 Next-Generation Learning: What Is It? And Will It Work? (166264520)

http://slidepdf.com/reader/full/next-generation-learning-what-is-it-and-will-it-work-166264520 43/43

Pleasecompleteanevaluaonforthistalk.

WhatcouldIdotoimproveit?