OUTCOMES AND ASSESSMENT INTHE OSU BIOLOGY PROGRAM

Robert T. Mason, Chair5 October 2011

OUTLINE

• Outcomes

• How to Assess our Outcomes?

• Direct Assessment

• Indirect Assessment

• Reflections and Future Directions

Biology Program Learning Outcomes Students taking courses within the Biology Program represent diverse interests and skills. The following general outcomes match the Biology Program goals, and are reflected in the outcomes and assessments of the various courses Biology majors complete.   1. Conceptual Understandings

Students will understand:

A. fundamental concepts that characterize Biology, including concepts that characterize the various life science fields, including Botany, Environmental Science, Genetics, Microbiology, and Zoology. B. how scientific knowledge develops, including the historical development of foundational theories and laws, the nature of science (particularly tentativeness and creativity), and the relationship between science, technology, and society. C. the process of scientific inquiry, including that scientists use varied methods; scientists test ideas; scientists use creativity, critical thinking, and current knowledge; and investigations commonly lead to additional questions. D. the relationship between science and other subject areas, including interdisciplinary approaches to global issues and the relationship of core concepts from Chemistry, Mathematics, Geology and other disciplines to life science concepts.

2. SkillsStudents will be able to:

A. conduct background research on life science topics to make informed decisions, and develop understanding of basic concepts, processes and keywords necessary to investigate topics; skills for accessing and critiquing information; decision-making steps for specific Biology issues; and ways to present and discuss potentially controversial information. B. function successfully within laboratory and field settings, including use of basic equipment (microscopes, measurement devices, and computer technologies); developing and utilizing appropriate safety protocols; and putting into practice conceptual understandings of the research process. * C. utilize a variety of skills related to learning Biology, including thinking skills of reasoning, critical thinking, creative thinking, and problem solving; self-regulatory behaviors of organization, self-management, and monitoring through feedback from classmates and instructors. D. utilize a variety of skills to share knowledge effectively, including oral and written communication skills; planning skills; and implementation skills of presenting material, motivating others, clarifying concepts and confirming understandings.

3. Appreciation of the Discipline

Students will have an opportunity to develop attitudes, beliefs and values related to the life sciences, including attitudes toward course participants, Biology, and science in general; beliefs related to their own abilities, as well as the abilities of others; and values, specifically personal values related to biological issues, the relationship of science to other ways of acquiring knowledge, and scholarly ethics.

4. Student Engagement and Professional DevelopmentStudents will have the opportunity to:

A. work, volunteer or intern as a biologist in a professional context.B. perform relevant professional activities such as laboratory and/or field research, clinical work, presentations at professional meetings, and scientific writing.C. apply conceptual understandings and skills learned in coursework to professional situations in biology. D. explore professional avenues in various biological disciplines.E. develop successful professional strategies such as appropriate codes of conduct and the ability to work with diverse people, as well as skills in planning, communication and organization.

We use an indirect measure of outcomes from our senior exit survey. (Outcomes 1-4)  We use a direct measure for Outcome 1. We have used the Educational Testing Service’s (ETS) Major Field Test in Biology for the past three years (fall, winter and spring) administering it to all graduating seniors. (Outcome 1). We are currently collecting our third year of data and now have a significant sample size available to analyze our initial findings. As part of our first year experience we offer a term-long Biology Freshman Orientation course (BI 198) that meets weekly with students to help them with the skills needed to successfully complete the Biology major, but also to begin to appreciate what they will do ultimately with their Biology degree (Outcomes 2-4). For the past three years we have formed a Biology Student Advisory Committee (BSAC) that consists of 10-12 seniors in the major. They are tasked with providing general feedback to the Biology Curriculum Committee and Chair about the major and assessment, as well as other specific projects to improve the student experience in Biology. (Outcomes 1-4).

Methods of Assessment

Composite senior exit survey data Learning Outcome 1: Conceptual Understandings • 94% of students somewhat or strongly agree that lecture courses cover the fundamental

concepts in biology.• 88% of students agree or strongly agree that they applied concepts from chemistry,

mathematics and physics to understand concepts in the life sciences in completing the major.

• 77% agree or strongly agree that they explored the historical development of scientific ideas in the major.

• 62% of students agreed or strongly agreed that they used a variety of methods to critique and test scientific ideas in the major.

Indirect AssessmentThanks to Ginny Lesser and Lydia Newton of the OSU Survey Research Center!

Learning Outcome 2. Skills • 63% of students somewhat or strongly agree that labs provided knowledge and skills

important to laboratory and field settings.• 78% agree or strongly agree that they developed informed opinions regarding issues of

science in relationships to society in the major.• 55% of students agreed of strongly agreed that they developed effective verbal and

written communication skills in the major.

Learning Outcome 3: Appreciation of the Discipline 77% agree or strongly agree that they explored the historical development of scientific ideas in the major.62% of students agreed or strongly agreed that they used a variety of methods to critique and test scientific ideas in the major.78% agree or strongly agree that they developed informed opinions regarding issues of science in relationships to society in the major.

Learning Outcome 4. Student Engagement and Professional Development Biology students are very involved with opportunities outside the classroom.

o 48% were involved in internshipso 44% were involved in researcho 28% were involved in international experienceso 23% were involved in teaching experienceso 21% were involved in clinical experiences

• 68% of students agreed or strongly agreed that experiences beyond the classroom required them to apply skills and knowledge learned in major courses.

• 74% of students agreed or strongly agreed that experiences beyond the classroom required them to develop laboratory, clinical and/or field methods relevant to their professional goals.

• 78% of students agreed or strongly agreed that experiences beyond the classroom allowed them to explore disciplines and refine career goals.

• 53% of students agreed or strongly agreed that experiences beyond the classroom required them to engage in scientific inquiry and to develop and test scientific

Biology Student Advisory Committee Provided feedback on the implementation of the ETS Major Field Test including:

Dates and times of the examTest environmentFeedback on the communication planFeedback on exam registration process

Provided feedback on the websites, checklists and other advising resources (Outcome 4)Provided feedback on laboratories, writing courses and other courses they found useful both in our curriculum and around OSU (Outcomes 1 and 2)Provided feedback on the value of co- and extra-curricular experiences (Outcome 4)Provided feedback on their appreciation of the discipline (Outcome 3) and the value of courses such as History of Science and other elective course they took.Other topics including undergraduate mentoring, advising, increasing student involvement and others (Outcome 4)

Direct Assessment• We use the Educational Testing Service’s (ETS) Major Field Test

All Biology majors must register for BI 499 (Major Field Test) the last quarter before they graduate. This “course” is required to clear the graduation audit.It is a 1 credit P/N course that the students must register for, but are not charged for (Thank you Kent Kuo and the Registrar’s staff!)The cost of each exam is $22. The Dean of Science is covering the cost of all the exams. The students are not charged for this course or anything associated with the exams. The students can see their score at the end of the 2 hour exam, but the score is not reportable.We do not have access to anyone’s individual score, only the groupscores. ETS will provide lots of data—for a fee!

Total Test Scaled Score

2008-09 2009-10 2010-11*

OSU Score / % Below 156 / 54 160 / 63 160 / 68

National Mean Score 153.6 153.6 153.2

* New version of test

2008-09 2009-10 2010-11(OSU Mean % Correct / National Mean % Correct)

Biochemistry/Cell Energetics 50 / 43 55 / 43 55 / 47

Cellular Structure, Organization, Function 60 / 55 63 / 55 59 / 51

Molecular Biology/Molecular Genetics 48 / 45 54 / 45 55 / 48

Diversity of Organisms 55 / 49 59 / 49 50 / 43

Organismal-Animals 57 / 56 62 / 58 48 / 42

Organismal-Plants 48 / 44 47 / 44 39 / 33

Population Genetics/Evolution 59 / 54 65 / 57 58 / 51

Ecology 63 / 57 65 / 57 61 / 52

Analytical Skills 55 / 53 59 / 53 55 / 47

# of students who took the test 44,089 47,446 7,898

*

*

*

*

*

Reflections and Future Directions

• Students are highly involved in international, community and profession development and see great value in them for their education

• Students want more labs, writing and critical thinking in the curriculumo Increased laboratory offerings – genetics, phage genomics

oExpanded history and philosophy of science offeringsoIncreased WIC offerings

• HHMI grant funding for GTA training to improve quality of labs and train effective teachers

o GTA lab developmento Graduate and undergraduate outreacho Redesigning our labs to promote more critical thinking (Z371 and Z362)

• Biology has the most International Degree and UHC students. These theses and our Marine Biology course are capstone experiences. We need more.

• MFT for three years running are above national average and for all three years our total test scaled score has increased

• We continue to reflect on our assessment and enlist the help of our student advisory committee, our curriculum committee, and the faculty, staff and students in Biology