National Survey of Student National Survey of Student Engagement, 2008Engagement, 2008

Results for UBC-VancouverResults for UBC-Vancouver

NSSE OverviewNSSE Overview Supported as an assessment tool by UBC many other Canadian

Universities including all of G13 Over 1200 colleges and universities across Canada and the U.S.

have participated in NSSE since the first pilot in 1999; ~450,000 first and senior year respondents in 2008

Supported by strong research; NSSE provides an estimate of how undergraduates spend their time and what they gain from attending college

Survey items represent empirically confirmed "good practices" in undergraduate education. That is, they reflect behaviors by students and institutions that are associated with desired outcomes of college.

125+ questions (core, contextual, experimental, consortium) and 5 composite engagement benchmarks

Results provided for participating university and selected comparator groups

Additional info at http://nsse.iub.edu

Engagement BenchmarksEngagement BenchmarksNSSE provides five benchmarks of effective educational practices:

Level of Academic Challenge (LAC): amount of time studying, reading, writing; academic effort; coursework emphasis on analysis, synthesis, applying theories

Active and Collaborative Learning (ACL): asking questions, class presentations, teamwork in class, discussions and learning activities outside of class and involvement in community based projects

Student Faculty Interaction (SFI): discussions with faculty on grades, coursework, careers, research involvement with faculty, worked on other projects or committee with faculty

Enriching the Educational Experience (EEE): participation in co-curricular activities, volunteer work, field experience, co-op, community service, serious conversations with students from different ethnic backgrounds, political/religious beliefs, etc.

Supportive Campus Environment (SCE): student, faculty, staff relationships; campus services to help students with both their academic and non-academic responsibilities

Peer GroupsPeer Groups G13 Peer Group: Ontario and Quebec universities had their own

consortiums for participation in NSSE in 2008 so the G13 peer group includes UBC, The University of Alberta, University of Calgary and Dalhousie (eventually we will have comparisons for all G13 institutions)

“Carnegie” Cdn Peers consists of McGill, Concordia, McMaster, Toronto, Alberta, Calgary, York (large undergraduate populations)

Selected Peers consists of participating large research intensive universities in the U.S.: The University of Texas at Austin, University of Illinois at Urbana-Champaign, University of Minnesota-Twin Cities, University of North Carolina at Chapel Hill, University of Virginia, University of Wisconsin-Madison

Interpreting the DataInterpreting the Data Responses are reported in the form of means and frequencies. Items that

make up the five benchmarks of effective educational practices are identified in the means report and are also aggregated in the benchmarks report.

This year we report differences among peer groups AND differences for UBC since 2006. Important differences are identified by:

a) statistical significance (is the change unlikely to be simple chance variation?) and

b) effect sizes

Effect size indicates the “practical significance” of the mean difference. In practice, an effect size of .2 is often considered small, .5 moderate, and .8 large.

Summary of FindingsSummary of Findings UBC 2006 compared to UBC 2008: Areas of improvement and decline

UBC compared to Canadian Peers

UBC compared to U.S. Peers

Overall Satisfaction

Student Priorities on improving the learning environment

Next Steps

0

10

20

30

40

50

60

Acad Challenge Activecollaborative

learning

Stud/facinteraction

Enriching/educexperiences

Supportivecampus

environment

UBC-V NSSE First Year Benchmarks

2006

2008

All changes are statistically significant. All effect sizes are small.

0

10

20

30

40

50

60

Acad Challenge Activecollaborative

learning

Stud/facinteraction

Enriching/educexperiences

Supportivecampus

environment

UBC-V NSSE Fourth Year Benchmarks

2006

2008

All changes are statistically significant. All effect sizes are small.

Areas of improvement, First Year, 2006 - 2008Areas of improvement, First Year, 2006 - 2008

Very small, statistically significant increases, especially in areas where we were weak.

ACL, Worked with classmates outside of class ACL, Tutored or taught other students ACL, Participated in a community-based project as part of course EEE, Had serious conversations with students of a different race or ethnicity EEE, Community service or volunteer work EEE, Encouraging contact among students from different economic, social, ethnic backgrounds LAC, Worked harder to meet expectations LAC, Making judgments about the value of information, arguments, or methods SCE, Providing support to succeed academically SCE, Helping you cope with your non-academic responsibilities SFI, Talked about career plans with a faculty member or advisor SFI, Discussed ideas from your readings or classes with faculty members outside of class SFI, Received prompt written or oral feedback from faculty on your academic performance SFI, Worked with faculty members on activities other than coursework (committees, orientation,

student life activities, etc.) SFI, Work on a research project with a faculty member outside of course or program

requirements

Areas of decline, First Year, 2006 - 2008Areas of decline, First Year, 2006 - 2008

None of the decreases in benchmark items were statistically significant.

(and very few items actually declined)

Areas of improvement, Fourth Year, 2006 - 2008Areas of improvement, Fourth Year, 2006 - 2008

Very small, statistically significant increases, especially in areas where we were weak.

ACL, Worked with other students on projects during class ACL, Worked with classmates outside of class to prepare class assignments ACL, Tutored or taught other students (paid or voluntary) ACL, Participated in a community-based project as part of a regular course EEE, Used an electronic medium to discuss or complete an assignment EEE, Had serious conversations with students of a different race or ethnicity than your own EEE, Practicum, internship, field experience, co-op experience, or clinical assignment EEE, Participate in a learning community where groups of students take two or more classes together EEE, Culminating senior experience (capstone course, senior project or thesis, comprehensive exam, etc.) EEE, Encouraging contact among students from different economic, social, and racial or ethnic backgrounds LAC, Worked harder to meet expectations LAC, Making judgments about the value of information, arguments, or methods LAC, Applying theories or concepts to practical problems or in new situations LAC, Hours per 7-day week spent preparing for class LAC, Spending significant amounts of time studying and on academic work SCE, Relationships with faculty members SCE, Providing the support you need to help you succeed academically SCE, Helping you cope with your non-academic responsibilities (work, family, etc.) SCE, Providing the support you need to thrive socially SFI, Talked about career plans with a faculty member or advisor SFI, Discussed ideas from your readings or classes with faculty members outside of class SFI, Worked with faculty members on activities other than coursework (committees, orientation, etc.) SFI, Work on a research project with a faculty member outside of course or program requirements

Areas of decline, Fourth Year, 2006 - 2008Areas of decline, Fourth Year, 2006 - 2008

None of the decreases in benchmark items were statistically significant.

(and very few items actually declined)

010203040506070

LAC ACL SFI EEE SCE

UBC-V, Canadian Peers, U.S. Peers, NSSE 2008, First Year

UBC

Cdn Peers

U.S. Peers

0

10

20

30

40

50

60

LAC ACL SFI EEE SCE

UBC-V, Canadian Peers, U.S. Peers, NSSE 2008, Fourth Year

UBC

Cdn Peers

U.S. Peers

UBC compared to Canadian and U.S. PeersUBC compared to Canadian and U.S. Peers

Canadian Peers:

UBC’s benchmark scores are similar to those of other Canadian Peer Universities.

UBC does relatively better in the area of Enriching Educational Experiences (first and fourth year) and Active and Collaborative Learning (first year)

UBC scores slightly lower in the area of Supportive Campus Environment (first year) and Level of Academic Challenge (fourth year).

U.S. Peers:

UBC’s scores are significantly lower on all benchmarks compared to U.S. Peer Institutions (first and fourth year, statistically significant, moderate effect sizes).

We can hypothesize several reasons for this:- Different demographics - Different Socio-Economic status of students- Lower funding, larger classes- Commuter campus

Overall SatisfactionOverall Satisfaction

In addition to the questions which comprise the five educational benchmarks, NSSE also asks questions which explore overall student satisfaction as well as other areas related to both the academic and non-academic environment.

UBC Carnegie CDN Peers

Selected Peers

Class Mean Mean Sig Effect Size Mean Sig

Effect Size

Overall, how would you evaluate the quality of academic advising you have received at your institution? FY 2.63 2.73 *** -0.12 2.98 *** -0.41

SR 2.45 2.52 ** -0.08 2.78 *** -0.35

How would you evaluate your entire educational experience at this institution? FY 2.91 2.94 -0.04 3.35 *** -0.62

SR 2.87 2.92 * -0.06 3.36 *** -0.69

If you could start over again, would you go to the same institution you are now attending? FY 3.18 3.15 0.04 3.46 *** -0.38

SR 3.02 2.99 0.03 3.43 *** -0.52

Student PrioritiesStudent Priorities

NSSE allows for the inclusion of institutional specific questions for consortium participants. UBC, together with Dalhousie, Alberta and Calgary, provided customized questions which were asked of all first and fourth year students.

Students were asked to identify two areas that UBC needs to address in order to improve student learning in the classroom and outside the classroom

Issues UBC needs to address IN the classroom (2 responses per student)

Improving the quality of classrooms or lecture halls

Improving the quality of course instruction by professors

Improving the quality of teaching assistants

Ensuring a better fit between course content, assignments andtests/exams

Increasing the number or variety of course offerings IN YOURMAJOR

Increasing the number or variety of course offerings OUTSIDEYOUR MAJOR

Reducing class sizes overall

Improving the quality of labs

Improving student access to information technology

Providing more current/relevant courses and curriculum

Changing the mix of lectures, seminars, tutorials and labs

Increasing opportunities to learn more about global issues

4th Yr

1st Yr

Issues UBC needs to address OUTSIDE the classroom (2 responses per student)

Increasing contact with professors outside of class (e.g.office hours)

Expanding and/or improving the quality of academicsupport services

Expanding and/or improving the quality of counselingservices

Providing students with more opportunities to undertakeresearch with faculty

Improving the library collection

Improving library services (e.g. circulation, staffavailability, internet/computer availability, etc.)

Improving the quality/availability of study spaces

Increasing opportunities for international experiences(e.g. exchanges, study abroad)

Providing a better social environment for students

4th Yr

1st Yr

Next StepsNext Steps

Analyze student comments and compare to select peer institutions

Provide NSSE data to deans at faculty level

Drill down to specific majors for Arts, majors/honors for science, and ILP programs (eg. CAP)

NSSE Deep learning scales and analysis by aboriginal, visible minority, international

Use NSSE as the stimulus for discipline specific learning outcomes (e.g.: what more do you need to know to guide learning in your discipline?)

Focus NSSE discussion to align with institutional strategic goals (Trek 2010)