A Holistic Review of Gender Differences in Engineering Admissions and Early Retention
Beth M. HollowayP.K. Imbrie
Teri Reed-RhoadsSchool of Engineering Education
Purdue UniversityWest Lafayette, Indiana, USA
15th International Conference for Women Engineers and ScientistsAdelaide Australia, 19-22 July, 2011
Motivation
o Purdue’s College of Engineering (COE) has been working to increase the representation of women in its first year class for many years.
o Over the last 5 years, we have seen a 46% increase in the number of applications received from women, but only a 24% increase in the number of women admitted.
o At the same time, casual analysis seems to indicate that admitted women have higher metrics, on average, than admitted men.
Holistic Reviewo Subject matter expectationso Overall high school grade point average (GPA), on a 0 –
4.0 scaleo Core high school GPA (GPA of English, math, science,
foreign language, and speech classes only.), on a 0 – 4.0 scale
o High school class rank, in percentileo Standardized test scoreso Overall grades in academic courseworko Grades related to intended majoro Strength of student’s overall high school curriculumo Trends in achievemento Ability to be successful in intended majoro Personal background and experienceso Time of year student applieso Space availability in intended program
Research Questions
1. Are the metrics of women admitted to CoE statistically higher than those of men admitted to CoE?
2. To what extent do affective and cognitive measures from the Student Access and Success Instrument (SASI) model differences of success as measured by retention and graduation based on sex?
Admission Years: 2006, 2007, and 2008o Applicants to the College of Engineering
(resulting in 26,396 total records over the 3 cohort years)
o Applicants who are considered “Beginners”. (Transfer students, for example, were filtered out) (25,587 total records remaining)
o Applicants for the Fall semester ( 25,361 total records remaining)
o Applicants with complete applications (incomplete applications were filtered out)
o 23,068 total records remaining
Demographics of Fall 06, 07, and 08 Applicants to Eng.
Number % Number % Number % Number % Number % Number %
1632 5964 1652 6430 1369 6021
Caucasian, Non-Hispanic 1116 74.5% 4269 78.9% 1037 71.4% 4262 77.0% 926 74.3% 4023 76.8%
African American, Non-Hispanic 106 7.1% 214 4.0% 113 7.8% 206 3.7% 95 7.6% 259 4.9%
Hispanic American 73 4.9% 229 4.2% 76 5.2% 244 4.4% 66 5.3% 219 4.2%
Native American 15 1.0% 44 0.8% 6 0.4% 46 0.8% 5 0.4% 44 0.8%
Aisan American / Pacific Islander 160 10.7% 531 9.8% 178 12.3% 637 11.5% 135 10.8% 562 10.7%
Other 20 1.3% 93 1.7% 32 2.2% 92 1.7% 19 1.5% 89 1.7%
Not Reported 8 0.5% 29 0.5% 10 0.7% 46 0.8% 1 0.1% 43 0.8%
All Domestic 1498 91.8% 5409 90.7% 1452 87.9% 5533 86.0% 1247 91.1% 5239 87.0%
Indiana (% of Domestic) 334 22.3% 1510 27.9% 318 21.9% 1624 29.4% 272 21.8% 1715 32.7%
International 134 8.2% 555 9.3% 200 12.1% 897 14.0% 122 8.9% 782 13.0%
Race / Ethnicity
Residency
Total Number of Records
Demographics of Applicants2006
Women MenMenWomen2008 2007
Women Men
Analysis of Metric Medians for Applicant Pool
Women Men p-value
Median 3.9 3.7N 4457 17441
Median 3.74 3.48N 4603 18113
Median 93 86N 3029 11346
Median 620 600N 4611 18148
Median 670 680N 4611 18148
Median 1300 1280N 4611 18148
Overall GPA
Core GPA
Class Rank
SAT Verbal
SAT Math
SAT Total
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
All ApplicantsTotal
Boxplot of Overall GPA -Applicants
MenWomen
4.0
3.5
3.0
2.5
2.0
1.5
Overa
ll GPA
Boxplot of Overall GPA's for Men and WomenAll Applicants to Engineering
Boxplot of SAT Total Scores - Applicants
MenWomen
1750
1500
1250
1000
750
500
SAT
Tota
l Sco
re
Boxplot of SAT Total Scores for Men and WomenAll Applicants to Engineering
Analysis of Metric Medians for Admits to Engineering
Women Men p-value
Median 4.0 3.8N 3829 12790
Median 3.80 3.60N 3935 13201
Median 94 90N 2558 7963
Median 630 620N 3911 13127
Median 680 700N 3911 13127
Median 1320 1330N 3911 13127
SAT Math
SAT Total
All Admits to Engineering
Overall GPA
Core GPA
Class Rank
SAT Verbal
Total
0.0000
0.0000
0.0000
0.0000
0.0000
0.0100
Boxplot of Overall GPA - Admits
MenWomen
4.0
3.5
3.0
2.5
2.0
Overa
ll GPA
Boxplot of Overall GPA's for Men and WomenAll Admits to Engineering
Boxplot of SAT Total Scores - Admits
MenWomen
1700
1600
1500
1400
1300
1200
1100
1000
900
800
SA
T To
tal S
core
Boxplot of SAT Total Scores for Men and WomenAll Admits to Engineering
Analysis of Metric Medians for Denied Students
Women Men p-value
Median 3.4 3.2N 241 2071
Median 3.06 2.91N 255 2202
Median 75 66N 171 1485
Median 490 510N 277 2324
Median 550 590N 277 2324
Median 1050 1110N 277 2324
SAT Verbal
SAT Math
SAT Total
All Denies
Overall GPA
Core GPA
Class Rank
0.0000
Total
0.0000
0.0000
0.0000
0.0002
0.0000
Boxplot of Overall GPA - Denied
MenWomen
4.0
3.5
3.0
2.5
2.0
1.5
Overa
ll GPA
Boxplot of Men's and Women's Overall GPADenied Students
Boxplot of SAT Total Scores - Denied
MenWomen
1500
1250
1000
750
500
SAT
Tota
l Sco
re
Boxplot of Men's and Women's SAT Total ScoresDenied Students
Discussion❍ An unbiased process would result in no
statistical differences in the metrics of the admitted populations.
❍ SAT/ACT are intended to be a predictor of first year college grades, not academic achievement.
❍ Research shows that high school metrics are a better predictor of first year college grades than SAT (correlation coefficient of 0.42 vs. 0.36) Adding the two together gives a correlation coefficient of 0.52.
❍ 37 studies have shown a consistent gender bias in standardized tests. One study showed a 35 point bias in favor of males on the SAT math section.
Possible Conclusions
o Only the highest ability women are encouraged and/or self-select to apply to the College of Engineering, and men with a much wider range of academic ability are encouraged and/or self-select to do so.
o Women are held to a higher standard than men with regard to their high school performance.
o The admissions counselors put more weight on test scores than high school performance in the admissions process.
Bias at Work?
o According to Sevo & Chubin, “In situations where we evaluate the professional competence of men and women, and where there is much room for interpretation, men will have significant advantage due to unconscious assumptions. Our schema for males is a better fit for professional success, and especially for high-intensity scientific and engineering careers.”
Bias at Work?
o If a policy or tradition of an institution is to require a certain level of achievement on a test that is know to disadvantage a certain group, institutional bias exists.
1 2 3 4 5 6 7 8 9 10 11 120
10
20
30
40
50
60
70
80
90
100
Retention in Engineering for Several Large and Mid-Size Institutions
Male FemaleMinority
Semester
% R
eten
tio
n
o Understand the factors that impact students’ persistence in engineering. Such information could provide:
• provide a bases to assess the impact of program/institution-level decisions aimed at attracting students to engineering as well as student retention and success.
• evaluate the influences of current classroom pedagogical practices; and modify those deemed less effective; and
• more thoughtfully develop targeted interventions aimed at retaining students who otherwise have a propensity to leave engineering;
o Improve current retention modeling methods that are used to predict engineering students’ retention in engineering.
Motivation: Why study student retention and success?
Model of Student SuccessStudent Attitudinal Success Instrument (SASI)
( Imbrie, Lin & Malyscheff 2008; Reid 2009 )
Modified Model of Student SuccessStudent Attitudinal Success Instrument (SASI)
Methods
o Psychometric properties• Internal consistency (reliability)
− Cronbach’s coefficient alpha (α > 0.80)• Spearman-Brown formula used for subfactors with
< 10 items 2
o Exploratory Factor Analysis (EFA) • Used to establish subfactor structure or verify
structure if pre-defined• SAS proc factor, promax rotation
o Confirmatory Factor Analysis (CFA)• LISREL fit indices
−χ2
−Goodness of Fit (GFI > 0.90)−Comparative Fit Index (CFI > 0.95)−RMS Error Approximation (RMSEA < 0.08 for
acceptable fit)
Methods, continued
o Normative taxonomy: cluster analysis• McDermott’s 3-stage cluster analysis
−Standard cluster analysis of mutually exclusive groups
−Combining clusters from individual groups−Review to determine if individual data points
actually fit within a different cluster−Cattell’s between cluster similarity
coefficient – rp > 0.95 excellent similarity, rp < 0.7 poor
similarity 1
• Determines the number of groups based on normalized z-scores of overall constructs
Resultso Psychometric properties
• Cronbach’s coefficient alpha values for all constructs and subfactors > 0.80
− Spearman-Brown formula used to extrapolate subfactors to 10 items
− Exceptions: – Self-worth construct (0.69, 2007 cohort)– Team vs. Individual / Individual orientation subfactor
(0.74, 2006 cohort)o Exploratory Factor Analysis (EFA)
• Subfactor structure verified or defined for each construct
o Confirmatory Factor Analysis (CFA)• Subfactor structure verified for each construct• Fit indices 2,3,4 in all cases showed excellent fit*
− GFI>0.90, CFI>0.95
*RMSEA < 0.05 for excellent fit, <0.08 for acceptable fit
Results
o Normative taxonomy• 3 clusters indicated for each cohort (2004 –
2007)• 2004 – 2007 cohorts
−Visual inspection and −Values of Cattell’s between cluster
similarity coefficient again show three distinctly different clusters.
Cluster analysis results
Model of Student SuccessStudent Attitudinal Success Instrument (SASI)
( Imbrie, Lin & Malyscheff 2008; Reid 2009 )
Results: Ability to Identify At-Risk Students
Performance from New Model E’
Results: Important Factors by Different Methods
So What!!!!!
o Model results provide insight that can be used institutionally, programmatically, and individually to make informed decisions that will enhance undergraduate Engineering Education as well as provide a more personal learning experience for our students.
SAT_V SAT_M
SEM_ENGL
AVG_ENGL
SEM_MATH
AVG_MATH
SEM_SCI
AVG_SCIExpectMeta
Deep
Surface
Leader
Major
Motivation
Efficacy
TeamInd
0
0.5
1
Male (N=3852)
Female (N=823)
Institutional View2004 Cohort, 1 Year Retention
Institutional View
SAT_V SAT_M
SEM_ENGL
AVG_ENGL
SEM_MATH
AVG_MATH
SEM_SCI
AVG_SCIExpectMeta
Deep
Surface
Leader
Major
Motivation
Efficacy
TeamInd
0
0.5
1
Caucasian,Asi-Am,Other (N=4217)Underrepresented Minority (N=178)
Aggregated 2004-2006 Cohorts – 1 Year Retention
SAT_V SAT_MSEM_ENGL
AVG_ENGL
SEM_MATH
AVG_MATH
SEM_SCI
AVG_SCIExpectMeta
Deep
Surface
Leader
Major
Motivation
EfficacyTeamInd
0
0.5
1
SAT_V SAT_MSEM_ENGL
AVG_ENGL
SEM_MATH
AVG_MATH
SEM_SCI
AVG_SCIExpectMeta
Deep
Surface
Leader
Major
Motivation
EfficacyTeamInd
0
0.5
1SAT_V SAT_MSEM_ENGL
AVG_ENGL
SEM_MATH
AVG_MATH
SEM_SCI
AVG_SCIExpectMeta
Deep
Surface
Leader
Major
Motivation
EfficacyTeamInd
0
0.5
1
Male (N=1219)Female (N=289)
10 Semester Graduation
8 Semester Graduation
Institutional View
1 Year Retention
2004 Cohort, 1 Year Retention and 8, 10 Semester Graduation
Programmatic View
Individual View
Update
o Used this information for a discussion with the Admissions office staff
o For 2011 Admission process, • female applicants are up an additional
11% (Now 55% over the past 6 years)• Female admits are up 19%
o Also presented to Presidential Scholarship Committee prior to selections• Female awards up from 28 to 51%• Female yield is up 33% (Headcount of
489)
Acknowledgment
The researchers wish to acknowledge the support provided by a grant from the National Science Foundation, Division of Engineering Education and Centers (Award No. 0416113).
Discussion and Questions
Affective, Multidimensional Constructs o Motivation
• Control, challenge, curiosity, career outlook− Defined in terms of one’s pursuit of an activity for its
own sake » Pintrich & Schunk, 1996
o Metacognition• Planning, self-checking, cognitive strategy,
awareness− Strategies for planning, monitoring and modifying
one’s own cognitions.» Pintrich & DeGroot, 1990
o Propensity towards Deep and/or Surface Learning• Deep: Motive, strategy: Surface: Studying,
memorization− Propensity of a student within a learning environment
to adjust their learning style (deep or surface) to achieve the learning goal.
» Biggs, Kember and Leung, 2001
❍ Academic Self Efficacy− “Individuals’ beliefs of their competence affect
everything they do, and proposes that self-efficacy should prove to be an excellent predictor of their choice and direction of behavior. “
» Bandura, 1993− Studies have related self efficacy to retention
» Besterfield-Sacre et al., 1999; Pajares, 1996; House, et al., 1995; Bandura, 1986; Lent, Brown and Larkin, 1986
❍ Leadership• Motivation, planning, self-assessment, teammates
− The student’s self appraisal of their leadership abilities was identified as a noncognitive characteristic effecting student retention
» Tracy & Sedlacek, 1984; Hayden & Holloway, 1985; Ting, 2000
❍ Team vs. Individual Orientation• Individual, team dynamic
− Industry continues to seek graduates who can function as a team member and leader
» McMaster, 1996
Affective, Multidimensional Constructs
❍ Expectancy-Value• Community involvement, employment
opportunities, persistence, social engagement− Perception of the expectancy and value of
academic, social and employment expectancies» Wigfield & Eccles, 2000; Besterfield-Sacre
et al., 1999; Hayden & Holloway, 1985; Schaefers et al., 1997
❍ Major Decision• Certainty of decision, difficulty in decision,
personal issues, urgency of decision, independence
− Related to student success» Schaefers et al., 1997; Smith & Baker,
1987; Haislett & Hafer, 1990; Osipow, 1999
Affective, Multidimensional Constructs
Admissions Process (thru Fall ‘08)
Application arrives in ADMS data processing
Send request for more info. Code as “I”.
Yes
No Make recommendation and send to committee
Clear Admit?
Admit student to E. Code as “A”.
Send to designated ADMS counselor.
Clear Admit?
Admit student to E. Code as “A”.
Committee meets and agrees
Admit student to E. Code as “A”.
Admit student to 2nd choice. Code as “A”.
Hold decision. Code student as “E” or “P”
Yes
Yes
No
No
Deny admission to student. Code as “D”
App complete?