Assessing the

Citizenship ISLO:

Spring 2014

FACILITATOR: KAREN WONG

WITH MUCH GRATITUDE TO THE PARTICIPATING INSTRUCTORS

SOURCE: SKYLINE COLLEGE OFFICE OF PLANNING, RESEARCH, AND INSTITUTIONAL EFFECTIVENESS

Citizenship ISLO 1) DEMONSTRATE SCIENTIFIC LITERACY CONCERNING A RANGE OF

GLOBAL ISSUES.

2) ARTICULATE SIMILARITIES AND CONTRASTS AMONG CULTURES,

DEMONSTRATING KNOWLEDGE OF AND SENSITIVITY TO VARIOUS

CULTURAL VALUES AND ISSUES.

3) DEVELOP ATTITUDES CENTRAL TO LIFELONG LEARNING: OPENNESS,FLEXIBILITY, INTELLECTUAL CURIOSITY, AND A BROAD PERSPECTIVE THAT

VALUES DIVERSITY OF THOUGHT.

4) DEMONSTRATE APPROPRIATE SOCIAL SKILLS IN GROUP SETTINGS,

LISTENING AND BEING RECEPTIVE TO OTHERS’ IDEAS AND FEELINGS,

EFFECTIVELY CONTRIBUTING IDEAS, AND DEMONSTRATING LEADERSHIP

BY MOTIVATING OTHERS.

5) DEMONSTRATE COMMITMENT TO ACTIVE CITIZENSHIP.

Assessment Strategy

• Eighteen faculty across the disciplines whose

courses mapped to at least two of the five

descriptors participated.

• Faculty evaluated 523 students’ work– a major

assignment, performance, or activity-- using the

relevant parts of the rubric.

• The PRIE office aggregated and disaggregated

results for discussion.

Citizenship Rubric: Novice, Basic, Proficient, Advanced

• Scientific Literacy

• Scientific Method

• Understanding of the impact of decisions made by local, national and international

organizations, societies, environments and economies

• Understanding of the impact of the global economy on political decision-making

• Understanding of the interconnectedness of the world

• Participation in a global society

• Cultural Sensitivity

• Lifelong Learning

• Social Skills

• Social Interaction

• Participation

• Teamwork

• Leadership

• Active Citizenship

Rubric Example: Demonstrate

commitment to active citizenship.

Indicator □ Novice □ Basic □ Proficient □ Advanced

Demonstrate commitment to active citizenship via civic engagement: having a sense of responsibility to the larger community

No commitment- student may be aware of issues but does not participate in active citizenship.

Minimal commitment-student is aware of issues; may make conscious choices and change own daily habits.

Adequate commitment-student demonstrates “basic” skill level behaviors; in addition, informs and attempts to persuade family and friends to make similar changes and/or collaborates with others to enact change.

Full commitment-student demonstrates “proficient” skill level behaviors; in addition, adopts a leadership role in enacting change.

EIGHTEEN PARTICIPATING

DISCIPLINE FACULTY

Communications (3)

English (3)

English for Speakers of Other Languages (1)

World Languages (1)

Kinesiology– Varsity

Teams (2)

Administration of Justice (1)

History (1)

Paralegal Studies (1)

Political Science (1)

Biology (2)

Physics (1)

Surgical Technology (1)

Telecommunications (1)

SAMPLE SIZE

REPRESENTATION

3%

53%22%

22%

Kinesiology Language Arts Social Sciences Science/ Technology

Assignments/ Activities Activity

Students worked with a partner and then reflected on what they learned about their partner’s culture, and which intercultural communication competency skills might help them/did help them through this process.

Major Writing Assignment

Students researched and wrote about ecological issues; the psychological, political, social and economic contexts of crime; the causes of entrenched poverty in a third world country, etc.

Students reflected on how their personal actions and food consumption affect the world around them.

Performance and Reflection

Students who play team sports; participate in science labs; are on site for clinical observations were evaluated and/or evaluated each other’s social skills.

Exams

Students answered multiple choice test questions on their legal rights and the courts, and indicated whether they knew the information prior to taking the course.

Guiding Questions

• WHICH OF THE DESCRIPTORS ARE MOST PREVALENT IN

THE COURSES THAT WERE ASSESSED?

• HOW ARE STUDENTS FARING WITH EACH?

• WHAT ARE THE IMPLICATIONS OF THESE FINDINGS?

Which of the rubric descriptors are most prevalent in the

courses that were assessed?

10783

63

154179

316

234

287 287303

284

192

0

50

100

150

200

250

300

350

Scientific Method (SM) SM- Impact of Decisions SM- Impact of Global Economy

SM- World Interconnectedness SM- Global Participation Cultural Competency

Lifelong Learning Social Skills (SS) - Interaction SS- Participation

SS- Teamwork SS- Leadership Active Citizenship

How are students faring?

Mastery Which criteria have the highest number of “advanced” and “proficient” scores?

Challenge(s) Which criteria have the highest number of “novice”scores?

Insights?

Concerns?

Questions?

20% 18%13%

17%11%

33%26% 28%

20% 18%24%

11%

36% 40%

35%

41%

36%

43%

43%46%

40%49%

47%

48%

35%37%

46%34%

45%

19%

20%18%

22%

27% 24%

34%

9%5% 6% 8% 7% 5%

10% 8%

19%

5% 6% 7%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Overall Results

Advanced Proficient Basic Novice

How are students faring

with social skills?

3326 28

20

43

4346

40

1920

18

22

510 8

19

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Social Interaction Participation Teamwork Leadership

Advanced Proficient Basic Novice

Equity–

How are different groups faring?

What are the various levels of mastery for

each group?

Who, if anyone, appeared to be

disproportionately impacted?

What may account for different group’s

outcomes?

Disaggregated by Gender

2013/14

Demographics

Citizenship ISLO

Demographics

Difference

Female 51 % 53 % + 2 %

Male 47 % 43 % - 4 %

Unreported 2 % 4 % + 2 %

Disaggregated by Gender

22%16%

27%

37%43%

41%

19%

31%

30%

22%

11%

2%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Female (1434) Male (1151) Other/ Unreported (103)

Advanced Proficient Basic Novice

Disaggregated by Ethnicity

2013/14

Demographics

Citizenship ISLO

Demographics

Difference

African American 3 % 3 %

Asian/ Pacific Islander 21 % 23 % +2 %

Filipino 17 % 18 % +1 %

Hispanic 18 % 18 %

White 19 % 12 % - 7 %

Multi-ethnic 18 % 16 % - 2 %

Unreported 3 % 5 % + 2 %

Disaggregated by Ethnicity

17%20%

11%

22%18%

23%

30%25%

67%

40%54%

45%

43%

42%

49%

38%

17%

27%

29% 26%

29%27%

21%

30%

0%

12%6% 7% 10% 8%

0%

7%

0%

20%

40%

60%

80%

100%

120%

American Indian

(6)

Asian/ Pacific

Islander (576)

African

American (82)

Filipino (462) Hispanic (469) Multi-ethnic (406) White (302) Other/

Unreported (124)

Advanced Proficient Basic Novice

Disaggregated by Age

2013/14

Demographics

Citizenship ISLO

Demographics

Difference

Under 18 6 % 1 % - 5 %

18-22 53 % 67 % +14 %

23-28 23 % 18 % - 5 %

29-39 11% 8 % -3 %

40-49 4 % 3 % -1%

50-59 2 % 1% -1%

60+ 1% 3 % +2 %

Disaggregated by Age

31%

21%15%

27%

35%39%

32%

69%

40%49%

60% 46%

30% 43%

0%

29%28%

13%

11%

26%

24%

0%

10% 8%

0%

8%4%

1%

0%

20%

40%

60%

80%

100%

120%

Under 18 (13) 18-22 (1634) 23-28 (429) 29-39 (206) 40-49 (63) 50+ (23) Other/ Unreported

(79)

Advanced Proficient Basic Novice

Positive Findings (from 9/22/14 Town Hall)

• ASSESSED STUDENTS ARE SOCIALLY COMPETENT.

• FEMALES ARE MORE HIGHLY REPRESENTED ON OPPOSITE ENDS OF

THE SPECTRUM, “ADVANCED” AND “NOVICE.” ON THE OTHER

HAND, ALL THREE GENDER CATEGORIES (INCLUDING

“OTHER/UNREPORTED”) ARE ABOUT EVENLY SPLIT BETWEEN

“NOVICE AND BASIC” AND “PROFICIENT AND ADVANCED.”

• INSTEAD OF ONLY FOCUSING ON ASSESSMENT AS THE END

PRODUCT, WE’RE MOVING IN THE DIRECTION OF USING THE

SLOAC AS A MEANS TO HAVE MORE DISCUSSIONS AROUND

CURRICULUM AND TEACHING PRACTICES, THE “INPUT” THAT

ENABLES STUDENTS TO PRACTICE AND DEVELOP THESE

COMPETENCIES.

Areas for Additional Dialogue(from 9/22/14 Town Hall)

• HIGHEST PERCENTAGE OF “NOVICES” WAS UNDER

“LEADERSHIP.”

• THE RESULTS MAY BE DUE TO THE TYPES OF ASSESSMENTS AND/OR CONTEXT OF

LEADERSHIP BEING ASSESSED.

• LOWEST LEVELS OF “MASTERY,” AND HIGHEST LEVELS OF

“BASIC/NOVICE” ALL INVOLVE GLOBAL CRITERIA.

• PERHAPS GLOBAL TOPICS ARE NOT REINFORCED ENOUGH IN OUR

CURRICULUM.

• POSSIBLY THE AGE OF THE STUDENTS ALSO FACTORED INTO THE RESULT. THE

VAST MAJORITY ARE 18- 22 AND MAY NOT HAVE BEEN SUFFICIENTLY

EXPOSED TO GLOBAL TOPICS.

• ALSO THERE WERE LOW LEVELS OF REPRESENTATION OF THOSE AREAS IN THE

COURSES SAMPLED.

Potential Action Plans (from 9/22/14 Town Hall)

LEADERSHIP CRITERION

Professional development-- May want to initiate more conversation about how these ISLO competencies manifest themselves within our disciplines, and techniques for both teaching and assessing each ISLO component.

GLOBAL CRITERIA

May want to offer more professional development about how we can incorporate global topics into our curriculum.

May want to launch a “One [Global] Book” initiative such that students study a common book from different disciplinary perspectives.

May want to integrate more service learning opportunities into classes.

May want to disaggregate the data according to the number of units students have taken to see if there’s a difference between the two groups (e.g., 36+ units and those with fewer).

Is there a better way to select courses so that representation of the areas is better balanced? If more courses had this as a focus, the results on the rubric might be different.

May want to prompt people to review their mapping, which could help to improve the sampling of courses used for ISLO assessments.

Action Plan: What might you do in

response to these ISLO findings?

How is this information pertinent to your discipline/ area?

What can you do to reinforce or improve learning?

Revise how you convey the information?

Revise the assignment/ task?

Provide more opportunities for practice and feedback?

Revise assessment methodology?

Revise ISLO descriptors?