Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
GRADUATE ETS®
TESTING REPORT Class of 2019
DivisionofStudentAffairs
CSAO:CristinaGuerraRomero
DoralCollege
1
TABLE OF CONTENTS
Table of Contents ................................................................................................................................. 1
OVERVIEW ............................................................................................................................. 2
About the Assessment ............................................................................................................................ 2
Test Administration ................................................................................................................................ 2
Sample Size ............................................................................................................................................. 2
Use of Results .......................................................................................................................................... 2
Comparative Data ................................................................................................................................... 2
Class of 2019 Results & Comparative Data .............................................................................. 3
Total Scores ............................................................................................................................................. 3
Scaled Scores for Skills and Context-Based Subscores ........................................................................ 3
Proficiency Classifications ...................................................................................................................... 5
Essay Scores ............................................................................................................................................ 7
Summary of Results ................................................................................................................ 8
APPENDIX A—ETS® Proficiency Profile Proficiency Levels ....................................................... 10
Reading and Critical Thinking ............................................................................................................ 10
Level 1 ................................................................................................................................................ 10
Level 2 ................................................................................................................................................ 10
Level 3/Critical Thinking ..................................................................................................................... 10
Writing ............................................................................................................................................... 10
Level 1 ................................................................................................................................................ 10
Level 2 ................................................................................................................................................ 10
Level 3 ................................................................................................................................................ 11
Mathematics ...................................................................................................................................... 11
Level 1 ................................................................................................................................................ 11
Level 2 ................................................................................................................................................ 11
Level 3 ................................................................................................................................................ 11
2
OVERVIEW
ThefollowingreportsummarizestheresultsoftheETS®ProficiencyProfile(AbbreviatedForm)fortheDoralCollegegraduatingClassof2019,hereinafterreferredtoas“thecohort.”Asofthe2014-2015academic
year,allgraduatesoftheCollege’sAssociateinArtsprogramarerequiredtocompletethisassessmentprior
tograduation.However,resultsdonotaffecteligibilityforgraduation,andindividualstudentscoresarekept
confidentialandnotreleasedinthisreport.
TheETS®ProficiencyProfileassessesallfourgeneraleducationskills—criticalthinking,reading,writing
andmathematics.ItisoneofthreeinstrumentsapprovedforuseinmeetingVoluntarySystemofAccountability(VSA)requirements,andprovidestheCollegewithabasisforcomparisonusingcomparative
dataonover500institutionsand550,000students.
About the Assessment
DoralCollegeutilizestheETS®ProficiencyProfile(AbbreviatedForm),ashortenedversionofthe
standardassessment.TheETS®ProficiencyProfile(AbbreviatedForm)consistsof36multiple-choice
questions,asfollows:
• ninequestionstestingcriticalthinkingskills
• ninequestionstestingreadingskills
• ninequestionstestingwritingskills
• ninequestionstestingmathematicsskills
TheoptionalETSessayportionisalsoadministeredtoallstudentsinthecohort.Totaltestingtimeis
approximatelyonehourand30minutes.
Test Administration
PersonnelfromtheCollege’sDivisionofStudentAffairsadministeredtheETS®ProficiencyProfiletoall49
studentsontracktograduateduringthespring2019semester.Theassessmentwascompletedoncomputers
andproctoredinpersonbystafftrainedintestsecurityandadministration.
Sample Size
Thisyear’scohortexceedstherecommendedsamplesizeofatleast30studentswhenusingtheETS®
ProficiencyProfile.Itisalsoimportanttonotethatinstitutionsoftentestonlyasampleofthetargetedpopulation(e.g.,10%ofseniors),whileDC’sresultsarereflectiveoftheentiretargetedpopulation(e.g.100%
of2019graduates).
Use of Results
TheCollege’sDivisionofStudentAffairspreparesthisreportfordisseminationtotheCollegecommunity.
Academicpersonnel,ledbytheChiefAcademicOfficer,reviewtheresultsinrelationtotheacademicprogram
andmakeadjustmentsasappropriate.Forexample,lowerthanaveragescoresinacertaingeneraleducationskillwouldresultinareviewofrelatedcoursessothatimprovementsmaybemade.
Comparative Data
DoralCollegeevaluatesitsstudents’resultsusingthecomparativedatareleasedbyETS®forthispurpose.
Aspectsoftheassessmentwillbeaddressedseparatelyasfollow:
3
• totalscores
• scaledscoresforthefourskillssubscores(criticalthinking,reading,writing,andmathematics)and
threecontext-basedsubscores(humanities,socialsciences,naturalsciences)
• proficiencyclassificationsfortheskillareasofreadingandcriticalthinking,writing,and
mathematics
• essayscores
AvailablecomparativedatareflectstheresultsofproctoredexamsfromJuly1,2014andJune30,2019.All
data,withtheexceptionofessayscores,isbrokendownbyinstitutiontypeandclasslevel.DoralCollegeisa
two-yearprivatecollegethatoffersanAssociateinArtsdegree.Therefore,thebestpointofcomparisonfor
thecohortisSophomoresatAssociateDegreePrograms/Colleges.However,inordertomakebroadercomparisonsregardingthecohort’sperformance,additionaldatasetswerealsoincluded.
Thefollowingcomparativedatasetsforproctoredexaminationswereusedandarereferencedherein:
• Sophomore(30-60semesterhourscompleted)atAssociateDegreePrograms/Colleges(includes
77institutionsand32,584students)
• Sophomore(30-60semesterhourscompleted)atBachelor’sColleges(includes44institutionsand
4,903students)
• Allstudentgradesatallinstitutiontypes(includes323institutionsand369,745students)
AllcomparativedatareferencedwasexcerptedfromtheETS®websiteandcanbelocatedat
https://www.ets.org/proficiencyprofile/scores/compare_data/.
CLASS OF 2019 RESULTS & COMPARATIVE DATA
Total Scores
TotalscoresfortheETS®ProficiencyProfilerangebetween400-500.Themeanscoreforthecohortwas
461.74.Allgraduatesscoredabovea423.
Furthermore,themeanscoreofthecohortexceededthemeanscoreofallcomparativedatasetsused.Table1belowprovidesmoreinformation.
Table1:TotalScoreMeanComparison
Data Set Mean Score for set DC 2019 Mean Score Percent Below DC Mean*
Sophomores at AA 437.0
461.74
100
Sophomores at Bachelors 436.2 100
All student grades, all types 440.1 98
*ThisisthepercentageofinstitutionswhosemeanscoreisstatisticallybelowtherangeinwhichDoralCollege
cohort’smeanscorefalls.
Scaled Scores for Skills and Context-Based Subscores
Inadditiontothetotalscorereferencedabove,theETS®ProficiencyProfileprovidesascaledscorefortheareasofcriticalthinking,reading,writing,math,humanities,socialsciences,andnaturalsciences.Scoresfor
eachofthesecategoriesrangefrom100to130.However,itshouldbenotedthatthedifferentsubscoresare
notcomparabletooneanother(i.e.a125inreadingisnotcomparabletoa125inmathematics).
4
Table2andFigure1belowprovidethemeansubscoresfortheDCClassof2019,aswellasinformationon
howtheycomparetothemeansubscoresforthecomparativedatasets.
Table2:SubscoresMeanComparison
Data Set Mean Score for set DC 2019 Mean Score Percent Below DC Mean*
CRITICAL THINKING
Sophomores at AA 109.8
115.2
100
Sophomores at Bachelors 109.7 98
All student grades, all types 110.4 97
READING
Sophomores at AA 115.8
122.6
100
Sophomores at Bachelors 115.4 100
All student grades, all types 116.6 98
WRITING
Sophomores at AA 112.6
118.3
100
Sophomores at Bachelors 112.4 100
All student grades, all types 113.3 98
MATHEMATICS
Sophomores at AA 111.8
118.3
100
Sophomores at Bachelors 111.7 98
All student grades, all types 112.7 96
HUMANITIES
Sophomores at AA 112.8
118.8
100
Sophomores at Bachelors 112.9 98
All student grades, all types 113.4 98
SOCIAL SCIENCES
Sophomores at AA 111.5
116.7
100
Sophomores at Bachelors 111.2 98
All student grades, all types 112.1 97
NATURAL SCIENCES
Sophomores at AA 113.7
118.0
100
Sophomores at Bachelors 113.4 98
All student grades, all types 114.3 96
Figure1:SubscoresMeanComparisonGraph
5
Proficiency Classifications
Inadditiontoscaledscores,theETS®ProficiencyProfileprovidesproficiencyclassifications(proficient,marginal,ornotproficient)todescribehowwellstudentshavemasteredeachlevelofproficiencyinthe
followingareas:
• Reading(Levels1and2)andCriticalThinking(Level3)
• Writing(Levels1,2,and3)
• Mathematics(Levels1,2,and3)
AdditionalinformationonproficiencylevelsisattachedasAppendixA.
AsDoralCollegeutilizestheabbreviatedformoftheETS®ProficiencyProfile,studentsansweronlythree
questionsateachproficiencylevelinwritingandmathematicsandonlyfourorfiveateachlevelinreading.
Thus,theproficiencyclassificationsassignedareestimatesofwherestudentswouldfallhadtheytakentheStandardFormoftheexam.Theseresultswerenotusedtomakedecisionsaboutindividualstudents.
Table3andFigure2illustratethecohort’sproficiencyclassificationsbyskillarea.
Table3:ProficiencyClassificationEstimates—DCClassof2019
SKILL DIMENSION PROFICIENCY CLASSIFICATION
Proficient Marginal Not Proficient
Reading, Level 1 88% 2% 2%
Reading, Level 2 69% 20% 20%
Critical Thinking 14% 37% 37%
Writing, Level 1 86% 14% 0%
Writing, Level 2 43% 43% 14%
Writing, Level 3 27% 43% 31%
Mathematics, Level 1 86% 10% 4%
Mathematics, Level 2 67% 18% 14%
Mathematics, Level 3 16% 41% 43%
11
5.2
12
2.6
11
8.3
11
8.3
11
8.8
11
6.7
11
8.0
10
9.8
11
5.8
11
2.6
11
1.8
11
2.8
11
1.5 11
3.7
10
9.7
11
5.4
11
2.4
11
1.7
11
2.9
11
1.2 11
3.4
11
0.4
11
6.6
11
3.3
11
2.7
11
3.4
11
2.1 11
4.3
C R I T I C A L
T H I N K I N G
R E A D I N G W R I T I N G M A T H H U M A N I T I E S S O C I A L
S C I E N C E S
N A T U R A L
S C I E N C E S
DC Class of 2019 Sophomores at AA Sophomores at Bachelors All students, all institution types
6
Figure2:ProficiencyClassificationEstimatesGraph—DCClassof2019
Furthermore,Figures3-5providegraphicalrepresentationswhichillustratethecohort’sresultsineachskill
area,andhowtheycomparetotheresultsofeachrespectivecomparativedataset.
Figure3:ProficiencyClassificationComparison—ReadingandCriticalThinking
0%10%20%30%40%50%60%70%80%90%
100%
Readin
g, Le
vel 1
Readin
g, Le
vel 2
Critic
al T
hinki
ng
Writ
ing,
Leve
l 1
Writ
ing,
Leve
l 2
Writ
ing,
Leve
l 3
Mat
hemat
ics,
Leve
l 1
Mat
hemat
ics,
Leve
l 2
Mat
hemat
ics,
Leve
l 3
Proficiency Classification Proficient Proficiency Classification Marginal
Proficiency Classification Not Proficient
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Proficient Marginal Not
Proficient
Proficient Marginal Not
Proficient
Proficient Marginal Not
Proficient
Reading, Level 1 Reading, Level 2 Critical Thinking, Level 3
DC 2019 Sophomores at AA Sophomores at Bachelors All students, all types
7
Figure4:ProficiencyClassificationComparison—Writing
Figure5:ProficiencyClassificationComparison—Mathematics
Essay Scores
Allstudentsinthecohortwererequiredtotaketheoptionalessayportion.Scoresfortheessayrangefrom
one(1)tosix(6).AvailablecomparativedatareflectstheresultsofproctoredessaysadministeredfromJuly2013throughJune2018,butismorelimitedinscopeanddoesnotbreakdownscoresbyinstitutiontypeand
classlevel.Atotalof4,712studentsareincludedinthisset.
ThemeanscorefortheDoralCollegeclassof201was4.6.Allgraduatesscoredatleastafour(4).Table4,
below,comparesthedistributionofindividualstudentscoresinthecohorttothedistributionofindividual
studentscoresinthecomparativedataset.
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Proficient Marginal Not
Proficient
Proficient Marginal Not
Proficient
Proficient Marginal Not
Proficient
Writing, Level 1 Writing, Level 2 Writing, Level 3
DC 2019 Sophomores at AA Sophomores at Bachelors All students, all types
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Proficient Marginal Not
Proficient
Proficient Marginal Not
Proficient
Proficient Marginal Not
Proficient
Mathematics, Level 1 Mathematics, Level 2 Mathematics, Level 3
DC 2019 Sophomores at AA Sophomores at Bachelors All students, all types
8
Table4:ComparisonofIndividualEssayScoreDistribution
Score
Percent at each score
level
Percent at each score
level
Percent with a score
of at least…
Percent with a
score of at least…
(DC 2019) (Comparative Data) (DC 2019) (Comparative
Data)
6 2.07% 2.30% 1.50% 2.30%
5 63.27% 24.50% 65.30% 26.70%
4 34.69% 40.70% 100.00% 67.40%
3 0.00% 21.80% 100.00% 89.20%
2 0.00% 7.60% 100.00% 96.80%
1 0% 3.20% 100.00% 100.00%
unscored 0% Not listed N/A N/A
Table5,below,providesthemeanscoreforthecohort,aswellasthemeanscoreofallinstitutionalmeansin
thecomparativedataset.
Table5:ComparisonofInstitutionalMeanEssayScores
DATA SET MEAN SCORE
DC 2019 4.6
Comparative data set—Institutional Mean 3.8
SUMMARY OF RESULTS
Asillustratedherein,thedatashowstheDCcohortoutperformedallcomparativedatagroupsinalltested
areas.Thisincludesthe“Allstudents,allinstitutiontypes”dataset,whichincludesjuniorsandseniorsat
Bachelorsprograms/degrees.
Wherescaledscoresareconcerned,thecohort’smeanscorewassignificantlyhigherthanthemeanscoresof
allcomparativesetsused,bothinthetotalandsubscorecategories.Forexample,thecohort’stotalmean
scorewasstatisticallyhigherthanthetotalmeanscoreof100%oftheinstitutionsincludedinthe
sophomoresatAssociatesDegreePrograms/Collegesdataset.Furthermore,thecohort’smeanscorein
eachsubscorearea(criticalthinking,reading,writing,mathematics,humanities,socialsciences,andnatural
sciences)wasalsostatisticallyhigherthanthetotalmeanscoreof100%oftheinstitutionsincludedin
thesophomoresatAssociatesDegreePrograms/Collegesdataset,andbetween100-98%,
respectively,ofthesophomoresatBachelor’sCollegesdataset.
Additionally,whencomparingproficiencyclassificationsinthethreeareasofreadingandcriticalthinking,
writing,andmathematics,itbecomesclearthatagreaterpercentageofstudentsinthecohortscoredas
proficientineachlevelofeachareawhencomparedtoanyandallcomparativegroupsreferenced.Similarly,
alesserpercentageofstudentsinthecohortscoredasnotproficientineachlevelofeachareawhen
comparedtoanyandallcomparativegroupsreferenced.Forexample,88%ofthecohortscoredas
“proficient”onReading,level1,comparedto58%ofsophomoresatassociatedegreeprograms/institutions,
9
49%ofsophomoresatBachelor’scolleges,and60%ofallstudentsatallinstitutiontypes.Similarly,27%of
thecohortscoredas“proficient”onWriting,level3,comparedtoonly3%ofsophomoresatassociatedegree
programs/institutionsandallstudentsatallinstitutiontypes,4%ofsophomoresatBachelor’scolleges,and
8%ofallstudentsatallinstitutiontypes.
TheseresultsstronglyindicatethemembersoftheDoralCollegeClassof2019arebetterpreparedingeneral
educationskillsthanaretheircounterpartsatbothtwoandfour-yearinstitutions.Thisisatrendthathas
continuedconsistentlysincetheCollegebeganusingtheETStoevaluateitsgraduates’proficiency,asthe
Collegehasroutinelyoutscoredthecomparativedatasets,scoringhigherthanbetween100-93%ofall
groups(comparisonofmeantotalscoresandsubscores)overtheyears.
10
APPENDIX A—ETS® PROFICIENCY PROFILE PROFICIENCY LEVELS1
READING AND CRITICAL THINKING
LEVEL 1
TobeconsideredproficientatLevel1,studentsshouldbeableto:
§ recognizefactualmaterialexplicitlypresentedinareadingpassage
§ understandthemeaningofparticularwordsorphrasesinthecontextofareadingpassage
LEVEL 2
TobeconsideredproficientatLevel2,studentsshouldbeableto:
§ synthesizematerialfromdifferentsectionsofapassage
§ recognizevalidinferencesderivedfrommaterialinthepassage
§ identifyaccuratesummariesofapassageorofsignificantsectionsofthepassage
§ understandandinterpretfigurativelanguage
§ discernthemainidea,purposeorfocusofapassageorasignificantportionofthepassage
LEVEL 3/CRITICAL THINKING
TobeconsideredproficientatLevel3,studentsshouldbeableto:
§ evaluatecompetingcausalexplanations
§ evaluatehypothesesforconsistencywithknownfacts
§ determinetherelevanceofinformationforevaluatinganargumentorconclusion§ determinewhetheranartisticinterpretationissupportedbyevidencecontainedinawork
§ evaluatetheappropriatenessofproceduresforinvestigatingaquestionofcausation
§ evaluatedataforconsistencywithknownfacts,hypothesesormethods
§ recognizeflawsandinconsistenciesinanargument
WRITING
LEVEL 1
TobeconsideredproficientatLevel1,studentsshouldbeableto:
§ recognizeagreementamongbasicgrammaticalelements(e.g.,nouns,verbs,pronounsand
conjunctions)
§ recognizeappropriatetransitionwords§ recognizeincorrectwordchoice
§ ordersentencesinaparagraph
§ orderelementsinanoutline
LEVEL 2
TobeconsideredproficientatLevel2,studentsshouldbeableto:
§ incorporatenewmaterialintoapassage
1 The information above is released by ETS® for client use and can be located at:
https://www.ets.org/proficiencyprofile/scores/proficiency_classifications/levels
11
§ recognizeagreementamongbasicgrammaticalelements(e.g.,nouns,verbs,pronounsand
conjunctions)whentheseelementsarecomplicatedbyinterveningwordsorphrases
§ combinesimpleclausesintosingle,morecomplexcombinations§ recastexistingsentencesintonewsyntacticcombinations
LEVEL 3
TobeconsideredproficientatLevel3,studentsshouldbeableto:
§ discriminatebetweenappropriateandinappropriateuseofparallelism§ discriminatebetweenappropriateandinappropriateuseofidiomaticlanguage
§ recognizeredundancy
§ discriminatebetweencorrectandincorrectconstructions
§ recognizethemosteffectiverevisionofasentence
MATHEMATICS
LEVEL 1
TobeconsideredproficientatLevel1,studentsshouldbeableto:
§ solvewordproblemsthatwouldmostlikelybesolvedbyarithmeticanddonotinvolveconversionof
unitsorproportionality.Theseproblemscanbemultistepifthestepsarerepeatedratherthanembedded.
§ solveproblemsinvolvingtheinformalpropertiesofnumbersandoperations,ofteninvolvingthe
NumberLine,includingpositiveandnegativenumbers,wholenumbersandfractions(including
conversionsofcommonfractionstopercent,suchasconverting"1/4"to25percent).
§ solveproblemsrequiringageneralunderstandingofsquarerootsandthesquaresofnumbers.
§ solveasimpleequationorsubstitutenumbersintoanalgebraicexpression.
§ findinformationfromagraph.Thistaskmayinvolvefindingaspecifiedpieceofinformationinagraphthatalsocontainsotherinformation.
LEVEL 2
TobeconsideredproficientatLevel2,studentsshouldbeableto:
§ solvearithmeticproblemswithsomecomplications,suchascomplexwording,maximizingorminimizingandembeddedratios.Theseproblemsincludealgebraproblemsthatcanbesolvedby
arithmetic(theanswerchoicesarenumeric).
§ simplifyalgebraicexpressions,performbasictranslations,anddrawconclusionsfromalgebraic
equationsandinequalities.Thesetasksaremorecomplicatedthansolvingasimpleequation,though
theymaybeapproachedarithmeticallybysubstitutingnumbers.
§ interpretatrendrepresentedinagraph,orchooseagraphthatreflectsatrend.§ solveproblemsinvolvingsets;problemshavenumericanswerchoices.
LEVEL 3
TobeconsideredproficientatLevel3,studentsshouldbeableto:
§ solvewordproblemsthatwouldbeunlikelytobesolvedbyarithmetic;theanswerchoicesareeitheralgebraicexpressionsornumbersthatdonotlendthemselvestoback-solving
§ solveproblemsinvolvingdifficultarithmeticconcepts,suchasexponentsandrootsotherthan
squaresandsquareroots,andpercentofincreaseordecrease
§ generalizeaboutnumbers(e.g.,identifythevaluesof(x)forwhichanexpressionincreasesas(x)
increases)
§ solveproblemsrequiringanunderstandingofthepropertiesofintegers,rationalnumbers,etc.
§ interpretagraphinwhichthetrendsaretobeexpressedalgebraicallyoroneofthefollowingisinvolved:exponentsandrootsotherthansquaresandsquareroots,percentofincreaseordecrease
§ solveproblemsrequiringinsightorlogicalreasoning
12