Academic Program Assessment Form C: Assessment Results

Department: Business Administration and Economics Prepared by Dr. Jeff Zink and Dr. Pamela Mickelson

The Department of Business Administration and Economics has collected data from two direct assessment tools beginning in 2011. The Comprehensive Business Exam (CBE) is a standardized exam designed to measure proficiency in eight business domains. The second tool is a Sophomore Exam designed in the department to measure proficiency in the first five core business courses. College Learning Objectives (LOs) Student Learning Outcomes (SLO) #1: Morningside SLO #1: Demonstrate analytic, synthetic creative, evaluative and quantitative thinking. Assessment Method: Sophomore Comprehensive Exam

a) What did you measure? Student performance on a standardized, multiple choice test that covers five courses that all Business Administration Majors are required to complete. These are: BUSN 231: Principles of Management; ACCT 203: Introduction to Financial Accounting; ACCT 204: Managerial and Cost Accounting; ECON 200: Principles of Economics I; and ONE of the following: MATH 115: Mathematical Understanding and Reasoning; MATH 150: Elementary Probability and Statistics; Math 205: Calculus and Analytic Geometry I. Students complete ten questions for each course, for a total of fifty questions.

b) Who did you get measurements from (e.g., seniors, students in a class, etc.)? The department obtained measurements from sophomores enrolled in Accounting 204 (ACCT 204) Managerial and Cost Accounting, typically in the spring of the sophomore year. That is when students should have completed the five “core” courses.

c) Observed results (See Table 1): In 2011, of the forty-eight students that took the exam, ZERO scored a 70% (35/50) or better. Averages in what the department considered to be the ‘Qualitative” areas (BUSN) 231 were low; the

average score in the BUSN 231 portion was 4.44/10. Scores in the more quantitative areas were also low. Students improved their performance in many areas and overall for 2012 and 2013 compared to 2011. The average scores for each course are provided in the table below. In 2012, twenty two students out of sixty scored a 70% or better – a rate of 36.7%. In 2013, twenty one students scored a 70% or better for a rate of 32.3%. Each course is listed across the top, and each year is listed on the side. The numbers in each column represent the average score out of ten that students achieved on the exam that year. The average is the average score out of fifty that students achieved on the exam that year. Table 1: Sophomore Test Score out of 10 for Core business courses

Year Avg. out of 50

BUSN 231

ACCT 203

ACCT 204

ECON 200

MATH 115

MATH 150

MATH 205

# over 35

Total students

Percent over 35

2011 24.4 4.44 7.21 3.85 3.96 6.67 4.64 4.89 0 48 0

2012 32.10 6.28 7.3 7.18 5.53 6.67 5.19 6.21 22 60 36.7%

2013 31.24 6.27 6.33 6.73 5.71 6.33 6.04 8.00 21 65 32.3%

d) Is this what you expected? Explain. Yes and No. No: The BUSN 231 score for 2011 was surprisingly low – a 4.44/10. The BUSN scores for 2012 and 2013 are about 1.6 points above the 2011 score. The ACCT 204 score showed an even larger jump from 2011 to 2012, and then fell off slightly. ACCT 203 showed a similar pattern, but the jump from 2011 to 2012 was small compared to other jumps. The score for ECON 200 increased about 1.5 points from 2011 to 2012. The MATH 150 and MATH 205 scores have shown steady improvement, while MATH 115 scores have been roughly level. Yes: the scores from 2012 and 2013 fall roughly in the same range.

e) Why do you believe students performed this way? Dr. J. Jeffrey Zink undertook the role of Sophomore Exam Coordinator in spring 2011. The department administered an older version of the exam in 2011. Dr. Zink, with the assistance of four faculty in the core sections, redesigned the exam for 2012. As a result, the exam more closely matched faculty goals and learning objectives in the four accounting and business courses (ACCT, ECON, and BUSN). With some help from the Math department, Dr. Clovis redesigned the MATH 150 section, Dr. Thom Swanke redesigned the MATH 115 section, and Dr. Zink redesigned the MATH 205 section. As exam coordinator, Dr. Zink compiled a database of at least fifty multiple choice questions for each of the courses listed. The department does not have an explanation for the observed trends in the MATH courses.

f) What possible changes might improve student learning? Try to be specific. There are some changes

that might improve student learning. It is always possible to revise content and teaching approaches in the four business department courses. ECON 200 in particular remains weak despite its considerable improvement from 2011 to 2013. It is the only area where the average score is not above six out of 10. It is possible that we could change the type of exams given in each course, and /or institute more project based learning. Such open ended analysis is more appropriate for what students will face in a business environment. Those skills translate well into a multiple choice assessment, even if multiple choice exams aren’t given in particular courses. Many faculty members in the department are already doing this to some extent. The department might also consider requiring a second economics course.

Any changes would be made with the goal of having average student scores rise above a 7/10 for each course covered by the exam. At the same time, we would like to the average sophomore assessment exam score to jump to 35/50, or 70%. The department set the standard for demonstrated proficiency in these five courses at 70%. That is the stated policy in the course catalog. This definition of proficiency differs from the CBE, the nationally normed senior assessment exam. The CBE defines proficiency in its eight Common Professional Components as scores falling into the 83-99% range. A score of 70% on a common professional component falls in the range of ‘Basic’ understanding, defined as 66-82%. For more on the CBE, see below.

g) If you have instituted changes prior, please describe the impact, if any, you believe the change had on

student learning: The department has begun to advise more students to take at least two MATH courses. Even without this instrument, it was clear to the department that quantitative analysis had been a weak area. Whatever improvements there might be will likely not be reflected on this instrument, since students could easily take the second or third MATH course AFTER completing the Sophomore Assessment Exam. The expected improvements in learning as a result of this action may spill over into the performance in other areas, ACCT and ECON in particular, but again, these may occur after students have taken the Sophomore Assessment Exam. The real aim in advising students to take additional MATH course was to improve their quantitative reasoning ability, prepare them for possible entry in to graduate school, and show improvement on the CBE exam for seniors – on which see below.

Student Learning Outcomes #2: Communicate effectively. Assessment Method: Beginning in 2014-2015 outside reviewers will be utilized for BUSN 484 final project. Measurement falls to faculty in individual courses, particularly the Capstone BUSN 484 Business Policy, where it is assumed that

students who pass this course pass the writing proficiency component. AS of this writing we do not have formal, systematic method to assess this objective.

f) What possible changes might improve student learning? Try to be specific. Implement a systematic method of evaluation. Among the possibilities is to have an outside panel of business leaders from the community review student writing in BUSN 484. g) If you have instituted changes prior, please describe the impact, if any, you believe the change had on student learning: None – responsibility for teaching BUSN 484 and some other courses has rotated amongst faculty in the department.

Student Learning Outcomes #3: Morningside SLO #3: Behave ethically and responsibly. Assessment Method: CBE Results – Section 4 Social environment. There are six questions that address two outcomes. Objective 1) Identify ethical issues and choose the most ethical action. Objective 2) Demonstrate ethical and social responsibility in given business scenarios.

a) What did you measure? Student responses to three questions for each objective, for a grand total of six questions covering ethics and related areas.

b) Who did you get measurements from (e.g., seniors, students in a class, etc.)? Seniors in BUSN 484: Business Policy – the capstone.

c) Observed results: The CBE benchmark for proficiency in any of its sections is 83-99% correct. For the spring

2011 exam, 50% of students taking the CBE scored 83% or above, and placed into the either the ‘Proficient’ category or the ‘Mastery’ category. This number fell to 39% in the fall of 2011, and fell again to 38% in the spring of 2012.

d) Is this what you expected? Explain. No. We would have liked to see the social environment score remain

where it was. Our score dropped to become much more in line with those scores achieved by our “peer” institutions (Carnegie Classification) and all institutions. The pattern for ‘peer’ institutions was 37.5 proficient or mastery for spring 2011, 36.1% for fall 2011, and 37.4% in spring 2012. For all institutions taking the exam, the pattern was 36.8% in spring 2011, 35.4% in fall 2011, and 35.0% in spring 2012.

e) Why do you believe students performed this way? The spring 2011 score is unexpectedly high given the following results in 2012. The department does not know why this score would be so high. The other results are much more in line with both ‘Peer’ institutions and ‘All’ institutions. It had been the case that many students in the past had been taking our ethics course, BUSN 430: Contemporary Leadership and Ethics, at the same time as BUSN 484: Business Policy. Some faculty in the department have begun advising students to take these courses in different semesters, one in fall and one in spring. BUSN 430 is NOT a pre-requisite for 484, but is a required course for the major. As a result, some students may not have taken BUSN 430 at the time they took the CBE.

f) What possible changes might improve student learning? Try to be specific. It might be possible to include more

of an ethics component in each of the courses required for the major, leading up to BUSN 430: Contemporary Leadership and Ethics. The first week of each business course might be restructured to address ethical issues pertinent to that course. This would be a course content change within the department that does not require the approval of other departments or faculty. For example, discussion in economics could revolve around the necessity of sharing data and methods that allow other researchers to replicate results or detect problems. Another example would be ethical behavior in BUSN 314: Business Law – how does a manager conduct a fair and impartial investigation of a sexual harassment complaint? Third, students in BUSN 231: Principles of Management could examine how to set up a work environment that minimizes the chances that sexual harassment could occur. While these suggestions are valid, it would be up to individual faculty in each course to implement this change, so buy in from all faculty would be critical for this to have a systematic impact.

g) If you have instituted changes prior, please describe the impact, if any, you believe the change had on student learning: None.

Student Learning Outcomes #4: Use knowledge of cultures to enhance their understanding of themselves and others. Assessment Method: None: Neither the Sophomore Comprehensive Exam nor the CBE are designed to address this outcome. Further, departmental course offerings that address this outcome are very limited.

Student Learning Outcomes #5: Morningside SLO: Apply knowledge and skills from multiple, diverse disciplines and practical experiences to understand complex issues and to solve problems Assessment Method: IACBE Curriculum Analysis, and the CBE. The CBE will be discussed as a separate item. The Sophomore Assessment Exam is not designed to address this learning objective.

a) What did you measure? The International Assembly of Collegiate Business Education measures the level of recommended core courses for a degree in business. The IACBE is an accrediting body for many smaller liberal arts based business degrees. The BA&E department investigated being accredited in the last five years, and evaluated the department curriculum with the IACBE core requirements.

b) Who did you get measurements from (e.g., seniors, students in a class, etc.)? To determine any strengths or weaknesses in the department offerings, to aid in the assessment of student learning, the Department examined the core curriculum in light of IACBE requirements for accreditation.

c) Observed results: The current curriculum of the department has been based on the CPCs outlined by

IACBE. One of the tools used by IACBE to measure the level of coverage for these areas in the curriculum is a matrix arrayed with the core courses on the vertical axis and the CPCs arrayed along the horizontal access (See Table 2). The Matrix for the current curriculum of the Business department is included below. Contact hours in each CPC are mapped into the matrix. A minimum of 20 contact hours in each CPC is considered a minimum number of contact hours. As stated above, the department has reached out to the Math and Computer Science department for ideas on how to build information systems into our offerings. Further, because of the current credit hour structure of our curriculum, finding more contact time for areas such as Organizational Behavior, Human Resource Management or Operations Management—in the core—is problematic.

d) Is this what you expected? Explain. The shortness of curriculum offerings in both quantitative and management could help explain the poor performance by students in particular learning outcomes.

e) Why do you believe students performed this way? The core B.A. sequence does not require BUSN 338:

Production and Operations Management, a key course in many other programs.

f) What possible changes might improve student learning? Try to be specific. Requiring more instruction in each of those areas will improve student learning significantly.

g) If you have instituted changes prior, please describe the impact, if any, you believe the change had on

student learning: None. Table 2: IACBE Matrix of Curriculum Categories and BA&E degree requirements

A

cct

Mrk

t

Fin

c

Mgm

t

Org

Beh

HR

Op

sMgt

Law

Eco

n

Eth

ics

Info

Sys

Qn

t/St

ats

Intnatl

Bu

sPo

l/C

ps

t

Tota

ls

MATH REQ

Math Requirement

48

48

ACCT 203

Intro to Fin Acct

30

4

2

4

2

3

3

48

ACCT 204

Mng/Cost Acct I

30

5

1

4

1

5

2

48

ECON 200

Principles of Econ I

1

1

39

1

1

6

48

BUSN 231

Principles of Mgmt

33

2

2

1

2

6

1

2

48

BUSN 314

Business Law

5

4

33

4

2

48

BUSN 321

Principles of Finance

6

33

2

1

3

2

1

48

BUSN 341

Principles of Mrktng

30

2

3

1

2

2

2

2

4

48

BUSN 430

Cont Ldrshp & Ethics

14

12

6

16

48

BUSN 484

Business Policy

2

8

4

2

2

2

2

26

48

Totals

66

32

44

53

20

20

20

39

40

40

8

54

16

31

Student Learning Outcome #5: Morningside SLO: Apply knowledge and skills from multiple, diverse disciplines and practical experiences to understand complex issues and to solve problems. Assessment Method: The CBE.

a) What did you measure? Student Performance on the CBE in three Core Professional Components: Management, International Business / Global Dimensions, and Marketing.

b) Who did you get measurements from (e.g., seniors, students in a class, etc.)? Seniors taking BUSN

484: Business Policy.

c) Observed results: Morningside outperformed ‘peer’ and ‘all’ institutions in Management and International Business/Global dimensions. Morningside underperformed compared ‘peer’ and ‘all’ institutions in Marketing.

d) Is this what you expected? Explain. No, especially for marketing. We had thought that we were holding our own here, but the CBE reveals a gap in our performance. We had been strong in management and in International Business/Global Dimensions, which continued as expected.

e) Why do you believe students performed this way? See detailed discussion under the Department SLOs.

f) What possible changes might improve student learning? Try to be specific. See detailed discussion under the Department SLOs.

g) If you have instituted changes prior, please describe the impact, if any, you believe the change had on student learning: See detailed discussion under the Department SLOs.

Student Learning Outcomes #6: Exhibit a passion for life-long learning Assessment Method: Individual faculty taught May Term evaluations. Neither the Sophomore Comprehensive exam nor the CBE is designed to address this outcome.

Discipline Specific Student Learning Outcomes (D-SLOs):

1. Demonstrate a thorough understanding of business operations, issues of ethical conduct, and successful business strategies in an ever-changing business environment

2. Be innovative and apply successful business strategies in an uncertain and diverse world 3. Conduct relevant quantitative analysis essential in business and accounting 4. Think critically, communicate effectively, and demonstrate technological competencies for today's business environment.

Overview: We, as a department, have come to depend on the Comprehensive Business Examination (CBE). We began administering the assessment instrument in the fall of 2007 and have done so several semesters since. Overall, our results indicate we have made improvements in our program, but we are not making significant progress. However, when measured against other institutions in the country, we are performing near the average levels. Overall, we are graduating students who have above average performances in social and legal environments, management, international business and marketing. We are a bit below average in accounting, finance and economics—our quantitative course areas. As was mentioned above, we are moving to build a quantitative course to be administered by our department to help mitigate this noticeable shortfall.

Department SLO #1: Demonstrate a thorough understanding of business operations, issues of ethical conduct, and successful business strategies in an ever-changing business environment. Department SLO #2: Be innovative and apply successful business strategies in an uncertain and diverse world. Department SLO #3: Conduct relevant quantitative analysis essential in business and accounting

Assessment Method: CBE

a) What did you measure? Student achievement on the CBE, the senior assessment instrument put together by Future Business Leaders of America –Phi Beta Lambda (FBLA-PBL). The exam is recommended by the IACBE. The exam measures student knowledge in eight Core Professional components: Accounting, Finance, Economic Environment, Social Environment, Legal Environment, Management, International Business/Global Dimensions,

and Marketing.

b) Who did you get measurements from (e.g., seniors, students in a class, etc.)? Seniors enrolled in BUSN 484: Business Policy, the department capstone course.

c) Observed results (See Table 3): Raw data is provided in the table below. The measurement here is the

percentage of students taking the CBE that score an 83% correct or above in the CPCs, falling into either ‘Proficient’ (83%-99%) correct, or Mastery (100% correct). The overall trend for the data we have is that Morningside has a higher percentage of students that scored at ‘proficient’ or above than its peer and all other institutions in the CPCs of Finance, Social Environment, Legal Environment, Management, and International Business/Global Dimensions. Morningside has a lower percentage of students that scored at ‘proficient’ relative to peer and all other institutions in the CPCs of Accounting, Economic Environment, and Marketing.

d) Is this what you expected? Explain. Yes. In class assessments for the quantitative disciplines in particular are

similar to the observed performance on the CBE. The differences in Finance, Economic Environment, and Legal Environment are small, 1-2 percentage points. The trend in Economic Environment is upward for the data we have, even if Morningside’s achievement still falls below ‘peer’ and ‘all’ institutions. The gap in Management is larger. Morningside outperformed ‘peer’ institutions by about 3 percentage points (22.8% proficient or mastery vs. 19.8%), and outperformed ‘all’ institutions by about 5 percentage points (22.8% proficient or mastery vs. 17.7%) There are more pronounced differences in performance (over 5 percentage point gaps) in the CPCs of Social Environment, International Business/Global Dimensions, and Marketing. The shortfall in performance in Marketing comes as a bit of a surprise, since it is so large. Morningside falls short by about 10 percentage points relative to ‘peer’ institutions, and falls short of ‘all’ institutions by about 6 percentage points. The relevant numbers are highlighted on the Excel spreadsheet, with light gray indicating the areas where Morningside outperforms others, and darker gray where Morningside falls short. Table 3: CBE Results for combined three consecutive testing periods in 2011 and 2012

Morningside Peer All

NotProf Basic Prof Mastery

MorningsideProficiency+

Mastery

NotProf Basic Prof Mastery

PeerProficiency+Mastery

NotProf Basic Prof Mastery

AllProficiency+Mastery

Accounting 0.57 0.28 0.14 0.00 0.14 0.56 0.24 0.18 0.03 0.20 0.58 0.24 0.16 0.02 0.18

Finance 0.70 0.17 0.11 0.02 0.13 0.66 0.23 0.09 0.02 0.11 0.66 0.23 0.09 0.02 0.11

EconEnv 0.52 0.40 0.06 0.02 0.08 0.57 0.34 0.07 0.02 0.10 0.57 0.34 0.07 0.02 0.09

SocialEnv 0.41 0.16 0.26 0.17 0.43 0.35 0.28 0.24 0.13 0.37 0.39 0.25 0.24 0.11 0.36

LegalEnv 0.37 0.38 0.25 0.00 0.25 0.53 0.24 0.16 0.07 0.24 0.51 0.25 0.18 0.06 0.24

Management 0.25 0.52 0.23 0.00 0.23 0.36 0.44 0.19 0.01 0.20 0.37 0.46 0.16 0.01 0.18

IntBus/Global 0.33 0.33 0.27 0.08 0.35 0.31 0.42 0.20 0.08 0.28 0.32 0.39 0.19 0.10 0.29

Marketing 0.20 0.47 0.33 0.00 0.33 0.21 0.37 0.39 0.04 0.43 0.23 0.39 0.36 0.03 0.39

Note: Data for this table is an average of student performance in the core CPCs listed in the far left column. The numbers are the percentages of students scoring at Not Proficient (Not Prof), Basic, Proficient (Prof), and Mastery, and Proficient plus Mastery (Proficiency + Mastery). Not Proficient is a score below 66% correct. Basic is a score between 66% and 82% correct. Proficient is a score between 82% and 99% correct. Mastery is a score of 100%. For example, looking across the Accounting row, Morningside had 14% of its students score at “Proficient” or “Mastery”. The data covers the spring 2011, fall 2011, and spring 2012 semesters

e) Why do you believe students performed this way? In some ways these score reflect wider trends. Many institutions are struggling with quantitative aspects in particular. The performance in other CPCs such as Social Environment may have been enhanced because some students had been taking electives that address these areas. Morningside has recognized as a college that it has a shortcoming in the overall quantitative reasoning of its students as a whole, and this affects the quantitative performance of business students.

f) What possible changes might improve student learning? Try to be specific. For accounting in particular it seems

that more instruction is needed. As a department, we could consider requiring two math courses and an additional economics course. This might help finance too. We could consider making BUSN 338: Production and Operations Management a requirement for the major. It is currently required for the B.S. with Management Emphasis, but is an elective for the B.A.

g) If you have instituted changes prior, please describe the impact, if any, you believe the change had on student learning: Currently only the most self–motivated students are taking any additional quantitative courses. These are students that typically expect to get into a regional MBA program. The department has been trying to advise students to take at least one additional math course beyond the one we currently require, but this has not been successful. The department has also begun to strictly enforce academic qualifications for elective courses beyond B.A. core. We expect this to actually improve student learning both in the core and in the electives, since students are guided into more appropriate course selections.

Assessment Results Summary: Morningside Student Learning Outcomes

Student Learning Outcome

Measurement Used Results Results Evaluation

(what does this mean?)

Anticipated Changes & Who is Responsible

Prior Change Impact (if any)

1. Demonstrate analytic, synthetic, creative, evaluative, and quantitative thinking.

Sophomore Comprehensive Exam

Zero % with 70% correct or more in

2011, 36% with 70% or more correct in 2012, and 32% with 70% or

more in 2013.

Exam is better aligned with content in core courses for

2012 and 2013

Individual faculty for reflection on results. Dr. J. Zink as exam

coordinator for tracking performance

and sharing results and trends as they

build.

Advising to steer students in to more quantitative/math courses. Likely no

impact on this instrument, since

many of these courses could be taken after

this exam.

2. Communicate effectively.

Outside reviewer in 2014-2015

NA NA NA NA

3. Behave ethically and responsibly.

CBE Results: Section 4 Social Environment

CPC

Proficient/Mastery achievement fell from

50% of students achieving this

benchmark in spring 2011, to 39% in fall

2011, to 38% in spring 2012

Reversion to mean and that Ethics course is taught late in our sequence. (BUSN

430: Contemporary Leadership and

Ethics.

Individual Faculty – could begin with

ethics component in all courses.

Department – could develop lower level

business ethics course. Department – require lower level ethics course from

philosophy department

None.

4. Exhibit a passion for life-long learning. MSLO #6

May Term Evaluations NA NA NA NA

Assessment Results Summary: Department Student Learning Outcomes

Student Learning Outcome

Measurement Used Results Results Evaluation:

(what does this mean?)

Anticipated Changes & Who is Responsible

Prior Change Impact (if any)

Department SLO #1: Demonstrate a thorough understanding of business operations, issues of ethical conduct, and successful business strategies in an ever-changing business environment

CBE

Mixed: We see some strong areas, like management, and some unexpected weak areas like

marketing.

Department needs to consider strategies to

improve student achievement in Marketing and

Accounting

Department curricular revision. Individual instructor critical reflection

As noted above, started advising students to take additional math

courses, but there is significant

resistance, and impact will be limited

for near future.

Department SLO #2: Be innovative and apply successful business strategies in an uncertain and diverse world.

CBE

Strong in management and

International Business / Global Dimensions

Means that management and Int Business areas are fairly strong, but always room for improvement.

Department could change curriculum to

include BUSN 338: Production and Ops

Management. Though outperforming

peer institution in Finance, additional

improvement is seen as necessary.

NA

Conduct relevant quantitative analysis essential in business and accounting

CBE

Accounting remains a week point, as does

Economic Environment

Room for improvement. Econ

Environment is trending up, but still falls below ‘peer’ and

‘all’ institutions.

Department – additional instruction

in accounting and Economics seen as

desirable, but virtually no room to

add into the BA core.

NA