CTCH 826: Short Paper - Web viewCTCH 826: Short Paper. ... These initiatives may be adapted to fit a larger university structure ... offering options for resume' building and experience

Running Head: ACADEMIC ADVISING LEARNING OUTCOMES 1

Academic Advising Learning Outcomes

Megan Tucker

CTCH 826: Short Paper

George Mason University

ACADEMIC ADVISING LEARNING OUTCOMES 2

Academic Advising Learning Outcomes

Assessment is a way for universities and colleges to systematically gather, analyze and

interpret evidence to determine how student learning matches the school's expectations (Suskie,

2009). Specifically, outcomes-based assessment has become a prevalent area of research with

most colleges and universities assessing the impact of the college experience on student learning

and growth in the respective majors, general education programs, as well as in individual courses

(Bresciani, 2006; Driscoll & Wood, 2007; Smith, Szelest & Downey, 2004; Terenzini & Wright,

1987). However, research has begun to shift focus to other academic structures such as academic

advising (Haney & McClellan, 2009; Hurt, 2007; McClellan, 2011; Smith, Szelest & Downey,

2004). Though the research in this area has expanded, researchers have approached assessment

of academic advising in very different ways. The goal of this paper will be to identify the

prevalent methods of assessment for undergraduate academic advising and use this information

to create a proposed protocol for general academic advising assessment with the focus on

establishing student learning outcomes.

This research explores existing literature on learning outcomes related to academic

advising. Furthermore, the paper examines a variety of ways to assess these learning outcomes

using different theories and models. Through an analysis of literature on outcomes-based

assessment, learning outcomes in academic advising and pre-existing models of assessment, a

new option for assessing academic advising is introduced. The proposed assessment for

academic advising includes a mission and established goals, learning outcomes based in Bloom's

(1956) taxonomy, and a three-pronged assessment protocol encompassing progress and degree

completion statistics, student focus groups and an exit survey. Finally, implications for further

research and implementation are discussed.


Background and Literature

Outcomes-Based Assessment

A plethora of literature exists on outcomes-based assessment as it pertains to the program

level of academia. Bresciani's (2006) text explores ways of showing how to conduct assessment

through sharing best practices of institutions that focus on improvements in teaching/learning,

research and service. Specifically the text focuses on good practices in outcomes-based

assessment programs in an effort to transform institutional decision making regarding student

learning and development (Bresciani, 2006). Likewise, Driscoll and Wood (2007) approach the

how-to process of outcomes-based assessment through the themes of intrinsic motivation and

promoting faculty trust through constructive dialogue and collaboration.

Terenzini and Wright (1987) utilize an ex post facto survey system for students to assess

their academic development. This study models the sources of influence on students' reported

academic development over a four-year period, focusing on institutionally controllable

influences on student growth and assessing how those influences vary from one year to another.

The surveys address the following points: 1 . Gaining factual knowledge; 2. Evaluating ideas,

materials, and methods; 3. Applying abstractions or principles in problem solving; 4. Learning

principles, generalizations, and theories; 5. Recognizing general principles in specific

events/conditions; 6. Understanding a particular discipline's research methods; 7. Understanding

a particular discipline's various schools of thought; 8. Developing a sense of the interrelatedness

of different disciplines. This survey approach is utilized in the assessment protocol introduced in

this paper via exit surveys.


Finally, Smith, Szelest and Downey (2004) utilize Palomba and Banta's (1999) six

essentials of successful assessment that provide a guide to performing outcomes assessment and

apply it to academic support structures, such as academic advising. First, there should be an

agreement regarding the goals and objectives for learning, followed by a concrete plan of

assessment to measure outcomes. The third essential requires representation of students, faculty,

and staff in the process. The fourth component regards the selection of instruments used to

measure the outcomes of objectives. The last two essentials speak to the improvement loop

whereby results are used to improve practice and to assess the assessment (Smith, Szelest &

Downey, 2004). According to the authors, this framework, which has been successful in the

assessment of academic majors and the assessment of general education, can also be applied to

academic support units (Smith, Szelest & Downey, 2004). The last two essentials here of the

improvement loop and continual assessment are incorporated into the proposed model of

assessment in this paper (See Appendix D)

Models for Assessment

Research on assessment of academic advising has used a variety of models and

approaches. Haney and McClellan's (2009) research in assessment of student learning within

student affairs evolved through three transformative initiatives: (1) the institution’s decision to

shape itself into a learning college; (2) the revision of accreditation standards, placing increased

emphasis on institutional assessment and assessment of student learning; and (3) a statewide

initiative to develop a model for the assessment of student learning within student affairs units at

community colleges. These initiatives may be adapted to fit a larger university structure.

Likewise, Smith, Szelest, and Downey (2004) presents the development and

implementation of a program of outcomes assessment at a 4-year public university in the


northeast. Results indicate that students come to college with varying expectations of the role of

academic advisors and range on a continuum of academic and social preparation to meet the

demands of higher education. A majority of students are meeting the stated learning outcomes.

Feedback from the assessment model is being utilized to improve the quality of services

provided to successive cohorts of students at the institution (See Appendix B).

Erlich and Russ-Eft's (2011) review of social cognitive theory constructs of self-efficacy

and self-regulated learning is applied to academic advising for the purposes of assessing student

learning. They incorporate the development of a model for assessing student learning outcomes

in academic advising using these theoretical constructs first established by Bandura (1986). The

use of a theoretical framework compliments their incorporation of the model. In future research

related to this proposed protocol, a similar framework would be utilized.

Metzner (1989) collected data from 1,033 freshmen at a public urban university and

examined the effect of the perceived quality of academic advising on student attrition in a model

of the student attrition process (See Appendix A). Metzner found that high-quality advising

negatively influenced attrition through effects on GPA, satisfaction in the role of a student, the

value of a college education for future employment, and intent to leave the university. Low-

quality advising was related to greater attrition than was high-quality advising, but, on the other

hand, low-quality advising was associated with less attrition than no advising at all. The focus on

quality of academic advising contributes to this study because of the focus on attrition and

establishing greater satisfaction in advising. The goal of the proposed assessment model is to

foster high quality advising and discourage attrition, which is the same goal for the model

developed here.

Learning Outcomes for Academic Advising


Establishing learning outcomes for academic advising may need to broader in language

than outcomes for a specific program would be. The reason for this is the interdisciplinary focus

on academic advising and the attempt at creating a universal set of outcomes that can be applied

in most any academic program by advisors. Hurt (2007) discusses possible learning outcomes for

academic advising based on Bloom's taxonomy of educational outcomes:

1. List the classes student can take to fulfill general-education critical-thinking

requirements (knowledge)

2. Explain in their own words the importance of general education in their degree

(comprehension)

3. Calculate their grade point average (application)

4. Compare and contrast career options based on choice of major (analysis)

5. Develop a plan to graduate within 4 years (synthesis)

6. Decide which job offer to accept (evaluation)

These outcomes address a broad spectrum of concerns students many have without being too

specific to one program or college. I have utilized this sequence of outcomes as a baseline for

this paper's recommendations.

McClellan (2011) discusses student learning outcomes and the idea of using a balanced

scorecard for academic advising. The proposed the use of the balanced scorecard concept to

advising program assessment and evaluation (See Appendix C for balanced scorecard for

advising). There are five suggested areas of focus: learning outcomes, student perspective,

advisor perspective, program perspective, and budget perspective (McClellan, 2011). These areas

of focus became part of the structure of the proposed mission and goals statement for this

project.


Similarly, Campbell and Nutt (2008) note that while there are many ways to support

student immersion and engagement in learning, one strategy that is progressively more

acknowledged for its potential in this regard is academic advising. According to the authors,

outcomes-based assessment for academic advising demands the development of an assessment

plan through which "learning opportunities are developed, benchmarks for success are identified,

and multiple measure are used to gather evidence to inform improvement and demonstrate

program success" (Campbell & Nutt, 2009, p.7). The focus here is on learning outcomes in

regards to academic advising. The use of benchmarks here are recycled in the application of the

proposed assessment in this paper. The benchmarks in this case are identified through the

learning outcomes established below.

Application

Based on the literature there may be a variety of ways to implement and assess learning

outcomes for academic advising. One way to assess learning outcomes for academic advising

would be through a self-study (Bresciani, p. 24). Bresciani (2006) notes the following goals for

learning outcomes: (1) Program mission and goals, (2) Major learning outcomes, (3) Assessment

protocol for each MLO, and (4) Student learning.

Keeping these goals in mind I first developed a mission and goals for an academic

advising program. For the purpose of this paper the intended mission and goals are similar to

those in place for an advising program for which I work at George Mason University, as well as

drawing from the balanced scorecard approach that McClellan (2011) discusses. However, in

order to fit the needs of this paper they have been broadened for a general application to a variety

of programs and colleges, and the budget aspect has been removed to maintain focus on the

advisor and advisee and less on the university. First and foremost the mission of the academic


advising program is to assist students in their progress to degree completion. Focusing on student

development and learning, the advisor will engage in a professional, interpersonal relationship

with the student in order to identify academic interests and strengths, career goals, and any

barriers or conflicts the students face during their college tenure. Finally, the goal of the advisor

is to offer enough advice and information for the student to feel confident in decision-making

and their own self-efficacy as they reach the end of their degree and look towards career

development.

The next step would be developing major learning outcomes that keep these goals and the

mission in mind (Bresciani, 2006). Using Hurt's (2007) suggested learning objectives, I have

altered the six levels of outcomes using Bloom's (1956) taxonomy as well as my own

undergraduate academic advising experience and use of best practices (Tucker & Wright, 2013):

1. Knowledge: Identify general education and major course options with the student; If

undeclared, open up the discourse of interests, strengths and long-term goals to help

narrow in on major options.

The first step may be the most important for new college students. At this point (within

year one), the student may not have a major area of study yet, but they still will have a series of

general education courses to begin taking. Also, at this point the advisor should start a dialogue

with the student about their future interests to help them narrow in on a major path to graduation.

The student may take up to two years before deciding on a major course of study, so the earlier

the advisor begins that conversation, the more opportunity the student has to explore options

early-on.


2. Comprehension: Explain the process of degree completion in four years to the student,

as well as registration procedures and campus services.

The focus of this step is to ensure that the student understands their own accountability in

reaching degree completion, and following guidelines set forth by the university and in their

catalog year.

3. Application: Calculate their degree progress and GPA each semester or year based on

need; Identify possible barriers to timely degree completion.

The third step is crucial as a reoccurring process. Each semester when a student comes

for advising, the advisor should take the time to calculate their degree progress and monitor

changes in their GPA. The student could identify issues they have faced and address them with

their advisor. During this step, the first benchmark of barrier identification is incorporated.

4. Analysis: Discuss with student the possible career options based on major choices, as

well as opportunity for complimentary minors or certifications that fit into degree

progress.

By the midpoint of degree-seeking (around 60 credit hours completed) the student should

have a major chosen and most general education courses completed. It is at this point that the

advisor should steer meetings towards the focus of the major and options following graduation.

This step identifies the next benchmark of choosing a major and/or minor.

5. Synthesis: Integrate internship, study abroad, student work and community service

options/information into the advising session based on student's major and expressed

interests.


By the students' junior year, offering options for resume' building and experience may

further assist the student in reaching degree completion with confidence in establishing a career

soon after. Thus, discussing internship options, study abroad opportunities and community

service projects as a compliment to their coursework and extracurricular activities may prove

beneficial to the student. The synthesis step offers the final benchmark of career preparation and

resume'-building.

6. Evaluation: Follow up on student's reflections of courses taken, grades earned and

prior advising sessions in order to identify paths towards moving forward to successful

degree completion.

By the end of the students' senior year, the exit survey can be taken regarding the

students' thoughts on their journey to degree completion. This survey should allow for the

student to provide reflection on their course choices, grade changes and advising sessions

throughout their tenure at college to identify successes and areas of weakness. This is a crucial

outcome to meet because of the cyclical nature of assessment. Weaknesses addressed here by

graduating students may have an impact on assessment changes for future students.

The third step would be developing an assessment protocol for each major learning outcome

(Bresciani, 2006). This would include focus groups of students while they are enrolled (ideally at

the midpoint of degree completion, or around 60 credit hours). The focus groups would

incorporate questions drawn from the first three established learning outcomes. Also, at the end

of the student's academic career they would take part in an exit survey created with the learning

objectives in mind (Smith, 2003). The exit survey would be comprised of questions regarding

each learning outcome, but would focus more on outcomes 4, 5 and 6.


Finally, the last step in the process would be assessing student learning through

incorporation of the assessment protocols. The proposed protocol involves three main methods

of assessment: focus groups, exit surveys and degree progress and completion information (See

Appendix D for Proposed Model of Assessment). Degree progress and completion information

would include a student's academic transcript and transfer credit evaluation (if valid), as well as

recorded date of enrollment and date of degree conferral.

Conclusion

Though the literature on assessment of academic advising can be scattered, the focus on

quality and student satisfaction remains at the core of the research. This paper sought to find in-

depth literature and a variety of methods to assess in order to create a model and plan for

academic advising assessment at a large public university in the Mid-Atlantic region. The

struggle with any assessment is making sure it is a proper fit for the program or organization

being assessed. In this case, the assessment tool and model of outcomes were made to be broad

and very generalizable to appeal to a wide array of academic advising for different programs.

However, this is not to say that this proposed protocol would fit every advising department's

needs or address every issue or outcome it could. This is merely a starting point to a much bigger

goal of creating a current, general assessment protocol for academic advising in regards to

student development and degree completion.

Though this is only a proposed model of academic advising assessment, the framework

for further research is present. Similar to what has been addressed in our assessment course,

finding the best "fit" for assessment at the university is by taking some pieces of successful

models and applying them to the needs of the university being assessed. I focused on creating

learning outcomes based in Bloom's (1956) taxonomy due to the success in using these outcome


structures in past research. Using Hurt's (2007) outcomes as a guide, I pulled in my own

experience as an academic advisor and make changes accordingly. I also, separated the learning

outcomes by time to make this an ongoing cycle of assessment and not just every 4 years.

Student needs are constant, and assessing the structure of advising is an on-going process. Next,

I chose the three assessment strategies of progress and completion rates, focus groups and exit

surveys for a few reasons. First, there has been heavy emphasis in the literature on using exit

surveys for graduating college students as a way to use final assessment. Likewise, focus groups

were utilized, but usually early on in the student's college career. I chose to use focus groups at

the midpoint as a way to gauge what changes needed to be made as the student progress, and not

wait until after the fact to make changes. Finally, I chose to focus on statistics of degree progress

and completion for means of triangulating these three protocols. Perhaps there is a link between

them that influences the other. A full-scale research project would need to be initiated to be

certain. Regardless, the movement to assess academic advising is a much needed step in ensuring

student success.


Annotated Bibliography

Bloom, B. (1956). Taxonomy of educational objectives, handbook 1: Cognitive domain. New York: Addison Wesley Publishing Company.

Bloom's text identifies three domains of learning for creating learning outcomes: cognitive, affective and psychomotor. The author created levels for both the cognitive and affective domains.

Bresciani, M. (2006) Outcomes-based Academic and Co-curricular Program Review. Sterling: Stylus Publishing.

Focuses on best practices for outcomes-based assessment and program reviews. Offers examples of assessment models, learning outcomes and criteria for good practices.

Campbell, S., & Nutt, C. (2008). Academic advising in the new global century: Supporting student engagement and learning outcomes achievement. Peer Review, 10, 4-7.

While there are many ways to support student immersion and engagement in learning, onestrategy that is increasingly being acknowledged for its potential in this regard is academic advising.

Driscoll, A., & Wood, S. (2007) Developing Outcomes-based Assessment for Learner-centered Education. Sterling: Stylus Publishing.

A narrative-based text of insights and shared expertise on outcomes-based education and assessment models. Describes best practices and the focus of a learner-centered approach.

Erlich, R., & Russ-Eft, D. (2011). Applying social cognitive theory to academic advising to assess student learning outcomes. NACADA Journal, 31, 5-15.

Review of social cognitive theory constructs of self-efficacy and self-regulated learning is applied to academic advising for the purposes of assessing student learning. A brief overview of the history of student learning outcomes in higher education is followed by an explanation of self-efficacy and self-regulated learning constructs and how they can be applied to academic advising. The article concludes with the development of a model for assessing student learning outcomes in academic advising using these theoretical constructs.

Getty, L., Young, D., & Whitaker-Lea, L. (2008). Casting the assessment net wide: Capturing all student learning. About Campus, 10-16. DOI: 10.1002/abc.247.

North Georgia College and State University has joined a national effort to move away from the isolated approach to measuring student learning. Educators from three divisions offer their perspectives. The challenge for any institution is to use the energy from an impetus event to build momentum. The culture of assessment can become normal operating procedure as long as it is prevented from becoming so routine that it becomes stagnant again. The involvement of


many groups across campus can revitalize the entire community and make the goal of measuring outcomes across the entire student experience a reality.

Haney, R., & McClellan, D. (2009). Community college strategies: Building a model for assessment within student affairs. Assessment Update, 21, 10-12. DOI 10.1002/au

Frederick Community College’s assessment of student learning within student affairs has evolved as a result of the confluence of three transformative initiatives: (1) the institution’s decision to shape itself into a learning college; (2) the revision of accreditation standards, placing increased emphasis on institutional assessment and assessment of student learning; and (3) a statewide initiative to develop a model for the assessment of student learning within student affairs units at Maryland community colleges.

Hurt, R. (2007). Advising as teaching: Establishing outcomes, developing tools, and assessing student learning. NACADA Journal, 27, 36-40.

Using Bloom’s taxonomy of educational outcomes, Hurt explains how to develop learning objectives within advising contexts. The article also suggests commonly available educational materials, such as university catalogs, as content delivery mechanisms for students; in addition, it offers ideas such as reading guides and on-line lectures. Finally, Hurt explains how to assess student learning, that is, how to determine the extent to which students' learning outcomes mirror the objectives established by the advisor.

McClellan, J. (2011). Beyond student learning outcomes: Developing comprehensive, strategic assessment plans for advising programmes. Journal of Higher Education Policy and Management, 33, 641-652.

The article suggest an extended framework for understanding and engaging in comprehensive assessment of advising programs. Focuses on expanding learning outcomes to include a balanced scorecard approached to advising. The scorecard may be helpful in creating a list of learning outcomes.

Metzner, B. (1989). Perceived quality of academic advising: The effect on freshman attrition. American Educational Research Journal, 26, 422-442.

Using data from 1,033 freshmen at a public urban university, this study examined the effect of the perceived quality of academic advising on student attrition in a model of the student attrition process. High-quality advising negatively influenced attrition through effects on GPA, satisfaction in the role of a student, the value of a college education for future employment, and intent to leave the university.

Smith, J., Szelest, B., & Downey, J. (2004). Implementing outcomes assessment in an academic affairs support unit. Research in Higher Education, 45, 405-427.

This article presents the development and implementation of a program of outcomes assessment at a 4-year public university in the northeast. Results indicate that students come to college with


varying expectations of the role of academic advisors and range on a continuum of academic and social preparation to meet the demands of higher education. A majority of students are meeting the stated learning outcomes. Feedback from this assessment model is being used to improve the quality of services provided to successive cohorts of students at this institution.

Smith, J. (2003). Developing a survey to assess student learning outcomes in academic advisement. Assessment Update, 15, 6-15.

The article describes the process of developing an instrument to assess student learning objectives associated with academic advising at the University of Albany. The instrument is one of three components of a comprehensive model of assessment with the emphasis on an exit survey. This will be incorporated into my application section.

Suskie, L. (2009). Assessing Student Learning: A Common Sense Guide. San Francisco: Jossey-Bass.

A general reference guide for using assessment and understating how it changes as well as ways to promote assessment culture. Definitions and characteristics of good assessment are used in this paper.

Terenzini, P., & Wright, T. (1987). Influences on students' academic growth during four years of college. Research in Higher Education, 26, 161-179.

This study modeled the sources of influence on students' reported academic development over a four-year period, focusing on institutionally controllable influences on student growth and assessing how those influences vary from one year to another. Social integration was influential in students' reported academic growth only (but prominently) in the junior and senior years. The nature and strength of the influences varied over time, however, with academic and social integration.

Tucker, M. & Wright, C. (2013) Department of Communication: Academic Advising Training Protocol, Proceedings from NACADA Drive-In Conference, Fairfax, VA.

Training protocol provides detailed job description of academic advisor as well as protocol for meetings and communicating with students about degree progress.


Appendix A

Metzner's Model of the Attrition Process


Appendix B

Smith, Szelest, and Downey (2004) Model of Outcomes Assessment


Appendix C

McClellan's Components of a Balanced Advising Scorecard


Appendix D

Proposed Model of Assessment

Documents

CTCH 826: Short Paper - Web viewCTCH 826: Short Paper. ... These initiatives may be adapted to fit a larger university structure ... offering options for resume' building and experience