AssessingStudentWritingAcrossProgramsandTime:(Inter)disciplinary

andProgrammaticPerspectivesPresentedby

AnnM.Blakeslee,Ph.D.,Director,OfficeofCampusandCommunityWritingW.DouglasBaker,Ph.D.,AssociateDean,CollegeofArtsandSciences

SessionParts1. Challenges,potential,andimportanceofassessingstudent

writingacrosstimeandprograms2. EMU’scontextandhistory3. Ourresponseandtheresultsandoutcomes

• Facultyinvestment• UseofcampusLMSsandotherresources

4. Plansandnextsteps

First, why are campus-wide writing programs and assessments of those programs and student writing important?

Importance of Campus-wide Writing and Assessment of Writing

• Proven importance of writing to help students improve as writers and to support their learning and critical thinking (Anderson et al., 2015; Peer Review 2017)

• First-year, intermediate, and advanced [writing in the disciplines (WID)] offerings at many universities (Morris, 2017; Rutz and Grawe, 2017)

• Deliberate/non-deliberate scaffolding of writing across the curriculum• Importance—and difficulty—of assessing these initiatives

(NCTE, 2014; Sparks et al., 2014; Wardle and Roozen, 2012)

Challenges in and Potential for Assessing Student Writing• Faculty perceptions of responsibility• Disciplinary differences• Programmatic vs. individual• Past/current experiences with “assessment”

• Example: “It turns out that the assessment program your college imposed on you was probably never going to improve anything. A new article by an assessment insider [David Eubanks] explains why this is so and suggests that assessors have known for sometime now that assessment does not work.” - The Chronicle of Higher Education, January 12, 2018

EMU Assessment Context and History• General Education Reformed (approved 2006-07)• Plans to assess Gen Ed learning outcomes (2007-12)• Centralized approach: Office of Institutional Effectiveness and

Accountability (2009-2012)• Decentralized: Programmatic assessment of student learning to

colleges and General Education Program (2012-current)• Developing local capacity and leadership• Soliciting participation (e.g., College of Arts and Sciences,

approximately 70% of 134 programs contribute)

Our response to the challenges and to our context and history…

Small Group ShareWhat is the situation on your campus with student writing and the assessment of student writing programs?Discuss in a small group of 3-4.

GenEd/WIhistory…1. GeneralEducationreform– 2002/03;Implementation– 2006/07

• EducationforParticipationinaGlobalCommunity• EffectiveCommunication– first-yearwritingandspeech• Upper-levelwriting-intensivecourserequirement

2. FYWPassessment– 2013-16(threeparts)

3. WIassessment– startingin2015/16;ongoing

Background…1. AssessmentofWIwasplannedfromtheoutset

2. WIclassesaretaughtandapproachedinavarietyofways

3. WritingAcrosstheCurriculum(WAC)hassupportedthedevelopmentandteachingofWIclassessincetheirinception

4. Colleges,departments,schools,andprogramsallhavedifferentneedsandinterestsandfacedifferentchallengesandopportunitieswithWIclasses

WI Outcomes• Literacy Strategies that Support Inquiry: Develop and employ

successful and flexible reading and writing strategies that support sustained inquiry in a discipline

• Rhetorical Awareness: Use writing strategies that achieve the purposes for writing and address expectations of audiences in a disciplinary context

• Research Strategies: Formulate research questions and employ strategies for researching and responding to those questions

• Genre Awareness: Use discipline-specific genres to communicate info• Disciplinary Conventions: Understand conventions for

communicating, disseminating, and interpreting information in a discipline

Our Initiative• Began with WAC/UWC impact survey in 2016• WI assessment pilot was carried out in 2016

• Dietetics, social work, history• Used AACU value rubric for written communication

• Most recent assessment started in winter 2017• Participation was voluntary – sent invitation to faculty on

record for teaching a WI class that semester• 37 initial faculty volunteers• Meetings to explain and orient volunteers to assessment

Components and Rationales• Faculty questionnaire:

• Writing assigned, genres taught, strategies used• Understanding of outcomes• Perceptions of effectiveness in teaching and achieving outcomes• Perception of student success in developing proficiency with outcomes• Ease in teaching, allocation/use of class time

• Collection and review of syllabi: • Presence/absence of required rationale• Presence/absence of WI/any writing outcomes• Uses of drafts, peer review, instructor feedback, revision, other strategies

• Student questionnaire: • Students’ perceptions of proficiency with the outcomes

Components and Rationales• Rubric in Canvas:

• Faculty assessment of student success with outcomes on a particular assignment• Post-assessment questionnaire:

• Any changes they would make as a result of the assessment• Use of class time after vs. before assessment• Understanding of outcomes after assessment• Perception, after assessment, of

• Success and effectiveness in teaching and achieving outcomes• Student success in becoming proficient with outcomes• Ease or difficulty of teaching outcomes

• Focus groups: • Students’ experiences with the classes• Faculty members’ experiences with the classes and perceptions…

Approach• Collected course section numbers• Developed and loaded rubric into Canvas• Held information sessions• Developed and distributed questionnaires• Provided written instructions for using the rubric• Provided individualized instruction as needed

Small Group ShareWhat have you, or others on your campus, done (or what would you like to do) to assess writing programs and/or student writing? Why? What are your reasons/rationales?Discuss in a small group of 3-4.

Faculty Questionnaires…

Demographics/Information• Most tenured (17), tenured track (8)• Disciplines: STEM, humanities, social sciences,

business/professional• Majority face-to-face (76%)

• 21% hybrid/web-enhanced

• Majority had taught class more than twice • 46% had taught it 6+ times

• Enrollments ranging from 15 to 45 • Most 20-25

Writing• Mix of formal/informal• Genres: research papers (66%), lit reviews, essays,

abstracts, summaries, multi-genre projects, other (48%)• Approaches to assignments – 86% scaffold

• Drafts (76%)• Revision (68%)• Peer review (64%)• Project proposal, annotated bib (52%)• Rubrics (79%)• Feedback on drafts (83%)• Opportunities for revision (86%); Require revision (60%)

Writing FeedbackI'm developing Canvas discussions in which students explore ideas in their formative stages, and students respond to each others' Canvas writings in this way, and I also comment on and grade these postings. For each of the two assigned "analytical essays" (no research), I comment on essays and require/recommend resubmission with revisions. For the final "research" essay, I require that students defend their research questions orally and then they bring in rough drafts for peer review or individual consultation. I also require an "annotated bibliography" for this assignment in the weeks that precede the draft, so that I can help students to find the best research for their ideas (and so that I can assess -- and intervene when necessary -- how well they understand and are able to use their sources). (high touch approach)

Writing FeedbackMy feedback is typically at the higher and middle levels of concern with an emphasis on what worked/did not work for this assignment and questions to answer or things to be aware of when preparing for the next assignment. To allow students time to review and benefit from the feedback, I work to return feedback within one week of the assignment due date and make sure that major writing assignments are reasonably spaced throughout the term.

Outcomes – Understanding ofExcellent Good Average Limited Wtd. Avg.

Literacy 38% 52% 7% 3% 1.76

Rhetorical 52% 38% 7% 3% 1.62

Research 61% 25% 14% 0 1.54

Genre 55% 34% 10% 0 1.55

Conventions 69% 28% 3% 0 1.34

Outcomes – Perception of Effectiveness in Teaching

HighlyEffective

Effective Somewhat Effective

Not Effective Wtd. Avg.

Literacy 28% 45% 21% 7% 2.07

Rhetorical 17% 66% 17% 0 2.00

Research 22% 44% 30% 4% 2.15

Genre 34% 41% 24% 0 1.90

Conventions 29% 43% 25% 4% 2.04

Outcomes – Perceptions of Student Success in Achieving Proficiency with

Highly Successful (exceeds expectations)

Successful(meets expectations)

Somewhat Successful(approaches expectations)

Unsuccessful(does not achieve expectations)

Wtd. Avg.

Literacy 7% 66% 24% 3% 2.24

Rhetorical 3% 72% 24% 0 2.21

Research 19% 33% 33% 15% 2.44

Genre 14% 69% 17% 0 2.03

Conventions 10% 62% 28% 0 2.17

Student Questionnaires…

Student Questionnaires• Disciplines - STEM, humanities, social sciences,

professional• Majority were native English speakers• Self-rated proficiency levels:

1) Literacy: highly = 42%; proficient = 57%2) Rhetorical: highly = 48%; proficient = 50%3) Research: highly = 42%; proficient = 45%; minimally proficient = 12%4) Genre: highly = 46%; proficient = 46%; minimally proficient = 8%5) Conventions: highly = 47%; proficient = 50%

• Moving forward – start and end-of-semester surveys

Rubrics…

Outcome 1 -Literacy

Total Responses

Exceeds Expectations (4)

Meets Expectations (3)

ApproachingExpectations (2)

In Progress (1)

N/A(0)

Course Averages

BIO310W 25 7 9 8 1 0 2.88

CHEM381W 12 4 2 6 0 0 2.83

CLSC401W 27 5 16 6 0 0 2.96

COSC481W 26 16 6 3 1 0 3.42

CTAC460W 24 10 6 4 1 3 2.79

GEOG314W 14 10 1 3 0 0 3.5HIST490W 15 7 6 0 0 2 3.07

NURS375W 37 26 11 0 0 0 3.7

TM314W 14 4 4 3 2 1 2.57

WRTG328W 16 9 6 1 0 0 3.5

Overall 210 98 67 34 5 6 3.17

Outcome 2 -Rhetoric

Total Responses

Exceeds Expectations (4)

Meets Expectations (3)

ApproachingExpectations (2)

In Progress (1)

N/A(0)

Course Averages

BIO310W 25 3 10 11 1 0 2.6

CHEM381W 12 4 4 4 0 0 3

CLSC401W 27 1 21 4 1 0 2.81

COSC481W 26 16 6 4 0 0 3.46

CTAC460W 24 6 10 4 1 3 2.63

GEOG314W 14 8 4 2 0 0 3.42HIST490W 15 5 6 2 0 2 2.8

NURS375W 37 22 14 1 0 0 3.57

TM314W 14 2 6 3 2 1 2.43

WRTG328W 16 10 5 1 0 0 3.56

Overall 210 77 86 36 5 6 3.06

Outcome 3 -Research

Total Responses

Exceeds Expectations (4)

Meets Expectations (3)

ApproachingExpectations (2)

In Progress (1)

N/A(0)

Course Averages

BIO310W 25 7 9 8 1 0 2.88

CHEM381W 12 3 7 2 0 0 3.08

CLSC401W 27 5 9 13 0 0 2.7

COSC481W 26 1 1 0 0 24 0.27

CTAC460W 24 7 9 4 1 3 2.67

GEOG314W 14 10 1 3 0 0 3.5HIST490W 15 7 4 2 0 2 2.93

NURS375W 37 21 16 0 0 0 3.57

TM314W 14 2 3 5 3 1 2.14

WRTG328W 16 8 5 3 0 0 3.31

Overall 210 71 64 40 5 30 2.67

Outcome 4 -Genre

Total Responses

Exceeds Expectations (4)

Meets Expectations (3)

ApproachingExpectations (2)

In Progress (1)

N/A(0)

Course Averages

BIO310W 25 4 16 5 0 0 2.96

CHEM381W 12 1 11 0 0 0 3.08

CLSC401W 27 1 18 7 1 0 2.7

COSC481W 26 18 3 5 0 0 3.5

CTAC460W 24 4 9 6 2 3 2.38

GEOG314W 14 11 1 2 0 0 3.64HIST490W 15 5 7 1 0 2 2.87

NURS375W 37 10 27 0 0 0 3.27

TM314W 14 3 4 5 1 1 2.5

WRTG328W 16 9 6 1 0 0 3.5

Overall 210 66 102 32 4 6 3.04

Outcome 5 -Conventions

Total Responses

Exceeds Expectations (4)

Meets Expectations (3)

ApproachingExpectations (2)

In Progress (1)

N/A(0)

Course Averages

BIO310W 25 3 13 9 0 0 2.76

CHEM381W 12 4 4 4 0 0 3

CLSC401W 27 1 22 4 0 0 2.89

COSC481W 26 17 4 5 0 0 3.46

CTAC460W 24 8 7 6 0 3 2.71

GEOG314W 14 11 0 3 0 0 3.57HIST490W 15 5 6 2 0 2 2.8

NURS375W 37 16 21 0 0 0 3.43

TM314W 14 2 7 3 1 1 2.57

WRTG328W 16 7 5 4 0 0 3.19

Overall 210 74 89 40 1 6 3.07

Compilation Total Responses

Outcome 1 Literacy

Outcome 2 Rhetoric

Outcome 3 Research

Outcome 4 Genre

Outcome 5 Conventions

BIO310W 25 2.88 2.6 2.88 2.96 2.76

CHEM381W 12 2.83 3 3.08 3.08 3

CLSC401W 27 2.96 2.81 2.7 2.7 2.89

COSC481W 26 3.42 3.46 0.27 3.5 3.46

CTAC460W 24 2.79 2.63 2.67 2.38 2.71

GEOG314W 14 3.5 3.42 3.5 3.64 3.57

HIST490W 15 3.07 2.8 2.93 2.87 2.8

NURS375W 37 3.7 3.57 3.57 3.27 3.43

TM314W 14 2.57 2.43 2.14 2.5 2.57

WRTG328W 16 3.5 3.56 3.31 3.5 3.19

Overall 210 3.17 3.06 2.67 3.04 3.07

Other Findings• Majority of participants include some writing

outcomes on syllabi (73%)• Most devote class time to talking about/instructing

students in writing (93%)• Talk about writing styles/conventions (86%)• Teach rhetorical concepts (64%)• Conference with students (57%)• Assign readings about writing (54%)• Offer UWC targeted writing workshops (46%)

• Importance of professional development

Plans• Carry out a more granular assessment• Revise/update WI outcomes• Enhance support for instructors, courses, and

students• Cultivate a deliberate and intentional vertical

writing curriculum across the university• Create new opportunities and initiatives for

writing support and instruction across campus

Next Steps• Create a culture of ongoing assessment• Take final steps in assessment• Update outcomes• Obtain approvals of outcome revisions• Begin assessing new outcomes• Continue and enhance professional development

for faculty• Continue and enhance writing support for

students

Group ShareWhat are your next steps – one or two things you would like or plan to do on your own campus?What are your questions, concerns, responses?

Campus and Community WritingEastern Michigan University

www.emich.edu/ccw734-487-4813

campus_communitywriting@emich.edu

v University Writing Centerv Writing Across the Curriculum

v The Eastern Michigan Writing Projectv The Family Literacy Initiative

v The Disciplinary Literacies Initiativev Writing-Intensive Classes in Majors

Engage• In what ways have you assessed writing on your campus?

• For how long?• What challenges have you encountered and how have you

addressed them?• How have you recruited faculty participants?

• What has/has not proven successful?• How have you used your assessment data?

• In what changes has it resulted?• In what ways is writing supported/promoted on your campus?

• What is the story of writing on your campus?