43
An Assessment of Two Different Teaching Formats for Natural Science (GSTR 232), A General Studies Course: Background, Assessment Results, and Recommendations for Consideration

An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

  • Upload
    others

  • View
    3

  • Download
    0

Embed Size (px)

Citation preview

Page 1: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

An Assessment of Two Different Teaching Formats for Natural Science (GSTR 232),

A General Studies Course:

Background, Assessment Results, and Recommendations for Consideration

Page 2: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

Background/History

Course began in 1984, consisted of several instructors teaching within disciplines with recitation for smaller groups of students.

Page 3: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

Tentative Assessment Plan

• Student Attitudinal Survey• Evaluation of Student Knowledge/Performance• Faculty Meetings/Discussions• Individual Instructor Classroom Assessments• Faculty Feedback/Perspectives

Page 4: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

Additional Assessments and Initiatives

•Analysis of Instructor Evaluation QuestionnaireData (old versus new format of teaching the course)

•Faculty Development Proposal (to ensure that newinstructors to the course would have preparation)

Page 5: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

Student Attitudinal SurveyDesigned using a collaborative approach of faculty firstdeciding what the basic and major goals of the course wereand also agreeing on what other survey information would be ofinterest to faculty.

Administered to the last “old format” class in Fall 1999 and then to “new format” classes through Spring 2001; the instrumentwas administered both at the beginning and the end of each course.

Designed specifically to assess students’ predisposition to studyingscience, their ratings of the importance of several core learning goals,and the effectiveness of the course in fulfilling the goals (on the “end-of-course” version.

Page 6: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

Student Attitudinal Survey Results

Page 7: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

“My experience with studying science in high school was…”

60 63

0102030405060708090100

OLD NEW

Percentage of students who marked “Very” or “Somewhat Positive”

Page 8: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

“I feel prepared to take this course.”

39

56

0102030405060708090100

Percentage of students who marked “Completely” or “Somewhat Agree”

OLDNEW

Differenceis marginallystatisticallysignificantlydifferent,(p=.055)

Page 9: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

“Generally, I would describe my science background as…”

45 44

0102030405060708090100

OLD NEW

Percentage of students who marked “Excellent” or “Good”

Page 10: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

“The amount of time that I expect to devote to this course outside of

class in an average week.”

5752

0102030405060708090100

OLD NEW

Percentage of students who marked more than 5 hours

Page 11: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

“The amount of time that I devoted to this course outside of

class in an average week.”

21

34

0102030405060708090100

OLDNEW

Percentage of students who marked more than 5 hours.

Page 12: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

“My personal feeling about studying science this term is…”

4550

0102030405060708090100

OLD NEW

Percentage of students who marked “Very” or “Somewhat Positive”

Statisticallysignificant difference, (p<.05)

Page 13: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

“My personal feeling about studying science this term

was…”

42

56

0102030405060708090100

OLDNEW

Statistically significant difference (p<.05)

Percentage of students who marked “Very” or “Somewhat Positive”

Page 14: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

“I look forward to taking this course.”

38

49

0102030405060708090100

OLD NEW

Statisticallysignificant difference,(p<.05)

Percentage of students who marked “Completely” or “Somewhat Agree”

Page 15: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

“I enjoyed taking this course.”

46

60

0102030405060708090100

Percentage of students who marked “Completely” or “Somewhat Agree”

Statistically significant difference,(p<.05)

OLDNEW

Page 16: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

“I believe science is useful in my everyday life.”

7177

68

86

0102030405060708090100

OLD NEW

PRE POSTPRE

POST

Post scores arestatistically significantly different (p<.05)

Pre vs. Post scoresare statisticallysignificantlydifferent for newformat, (p<.05)

Percentage of students who marked “Completely” or “Somewhat Agree”

Page 17: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

Students rated the importance of six major goals:

•Increasing my general understanding of scientific knowledge.

•Increasing my general understanding of the development of science over time.

•Increasing my general understanding of the methods scientists use.

•Knowing more about particular topics in science.

•Increasing my understanding and appreciation of natural phenomena.

•Broadening my world view.

Page 18: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

Rate the Importance of..

“Increasing my general understanding of scientific knowledge.”

7384

68

85

0102030405060708090100

OLD NEW

PRE POSTPRE POST

Pre and Post scoresare statistically significantlydifferent betweenthe old and newformat,(p<.05)

Percentage of students who marked “Extremely” or “Somewhat Important”

Page 19: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

Rate the Importance of..

“Increasing my general understanding of the development of science over time.”

60 6454

67

0102030405060708090100

OLD NEW

PRE POSTPRE POST

Percentage of students who marked “Extremely” or “Somewhat Important”

Page 20: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

Rate the importance of..

“Increasing my general understanding of the methods scientists use.”

5262

39

65

0102030405060708090100

OLD NEW

“Percentage of Students Who Marked “Extremely” or “Somewhat Important”

PREPOST

PRE POST

Pre scores arestatistically significantlydifferent,(p<.05)

and Post scoresare marginallystatisticallysignificantlydifferent (p<.10)

Page 21: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

Rate the importance of..

“Knowing more about particular topics in science.”

7081

64

85

0102030405060708090100

OLD NEW

PREPOST

PRE POSTPre and Post scorescomparing old vs.new format are statisticallysignificantlydifferent (p<.05)

Percentage of students who marked “Extremely” or “Somewhat Important”

Page 22: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

Rate the importance of..

“Increasing my understanding and appreciation of natural phenomena.”

63

79

59

79

0102030405060708090100

OLD NEW

Percentage of students who marked “Extremely” or “Somewhat Important”

PRE POSTPRE POST

Pre and Post scores comparing old vs.new formatare statisticallysignificantlydifferent (p<.05)

Page 23: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

Rate the importance of..

“Broadening my world view.”

8591

7989

0102030405060708090100

OLD NEW

PREPOST

PRE

Pre vs. postscores in theold formatare statisticallysignificantlydifferent,(P<.05) andpre scores arestatisticallysignificantlydifferent comparing old vs. new format,p<.05)

POST

Percentage of students who marked “Extremely” or “Somewhat Important”

Page 24: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

On the end-of-course survey,students rated how well the Natural Science course had helped them to fulfil each of the six major goals.

Page 25: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

How well has this course in natural science helped you to..?

“Increase your general understanding of scientific knowledge.”

62

80

0102030405060708090100

OLDNEW

Percentage of students who marked “Extremely” or “Somewhat Well”

Statisticallysignificant difference (p<.05)

Page 26: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

How well has this course in natural science helped you to.?.

“Increase your general understanding of the development of science over time.”

6775

0102030405060708090100

OLDNEW

Percentage of students who marked “Extremely” or “Somewhat Well”

Marginally statisticallysignificant difference (p<.10)

Page 27: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

How well has this course in natural science helped you to..?

“Increase your general understanding of the methods scientists use.”

50

70

0102030405060708090100

OLDNEW

Statisticallysignificantdifference (p<.05)

Percentage of students who marked “Extremely” or “Somewhat Well”

Page 28: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

How well has this course in natural science helped you to.?.

“Know more about particular topics in science.”

68

80

0102030405060708090100

OLDNEW Statistically

significantdifference (p<.05)

Percentage of students who marked “Extremely” or “Somewhat Well”

Page 29: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

How well has this course in natural science helped you to..?

“Increase your general understanding and appreciation of natural phenomena.”

43

74

0102030405060708090100

Percentage of students who marked “Extremely” or “Somewhat Well”

OLD

NEWStatisticallysignificantdifference (p<.05)

Page 30: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

How well has this course in natural science helped you to..?

“Course helped to broaden your world view.”

60

71

0102030405060708090100

OLDNEW

Statisticallysignificantdifference (p<.05)

Percentage of students who marked “Extremely” or “Somewhat Well”

Page 31: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

How well has this course in natural science helped you to:OLD FORMAT NEW FORMAT

1) Course helped to know SAMEmore about particular topicsin science.

2) Course helped to increase 6 Highest to lowestyour general understanding of rankedthe development of science over time.

3) Course helped to increase your SAMEgeneral understanding of scientificknowledge.

4) Broaden your world view. 2

5) Course helped to increase 4your general understanding ofthe methods scientists use.

6) Course helped to increase 5your understanding and appreciation

of natural phenomena.

Page 32: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

Student Comments on Attitudinal Surveys

Old Format versus New Format

Page 33: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

Old Format:

Most common focusedon the amount of material and the shortage of time to absorb material

New Format:

Generally more positive (stimulated myinterest, liked instructor, etc.) but still mixed(too easy, too hard)

Page 34: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

Instructor Evaluation Questionnaire (IEQ) Findings

Statistically significant more positive ratings for the new format classes on five items:

•How much do you think you learned from this course?•The instructor was well prepared for class.•The instructor’s assignments were helpful to my learning.•The instructor encouraged students to ask questions

an/or express their ideas.•How would you rate this course overall?

Page 35: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

Faculty Feedback/Perspectives

Page 36: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

Positive Outcomes of New Format:•Smaller class size allowed faculty to know the

students better.

•Individual control over the pace made it easier toengage the students and know their needs.

•Adopting a style more in tune with individual instructors led to greater creativity and flexibility.

•Teaching outside one’s discipline forced the instructorto pay closer attention to topics and where students were having difficulties.

Page 37: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

Negative Outcomes of New Format

•Teaching with other faculty members with expertisewas very informative in old format.

•Access to demonstration materials and expertise was easyunder the old format.

•Student exposure to the details of a subject has been sacrificedin favor of faculty autonomy.

•The teaching of each of the subject areas with the Nat. Sciencecourse would be served better by having an expert in the fieldteaching the subject.

Page 38: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

Neutral Outcomes of New Format

•Although the relative proportion of the subjectshas been shifted, the total amount of the material covered remains the same.

Page 39: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

RECOMMENDATIONS FOR CONSIDERATION

Recommendation Aimed at the Overall Course Format

The course should continue to be taught in individualsections rather than in a large lecture format.

Page 40: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

Recommendations Aimed at Course Improvement

Regardless of the course’s format, the followingsuggestions are offered for course improvement.

Page 41: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

STRATEGIES FOR COURSE COHESION(enhancing communication, sharing resources and expertise)

•Continue regular meetings of the Natural Science faculty and those interested in teaching the course.

•Catalog and share equipment resources, films, etc.

•Create a website of syllabi, assignments, demonstrations, etc. that faculty can see what others are doing in order to improvecommunication and faculty development related to the course.

•Continue to offer workshops on various pedagogical strategies(e.g., facilitating the development of critical thinking and/orquantitative reasoning science, strengthening students’ understandingof the methods scientists use and the broadening of worldviews throughthe study of science)

Page 42: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

STRATEGIES FOR ASSESSMENT

•Encourage and communicate the results of classroom assessment activities.

•Devise course-embedded assessments of cognitive skills, knowledge gains, quantitative reasoning, etc. and share acrosssections to diagnose student strengths and weaknesses.

•Continue assessing periodically attitudinal measures for students(use the existing instrument or develop new ones).

•Re-administer the existing attitudinal survey to see if the morepositive attitudes found in the “new” format have persisted.

•Monitor course summary IEQ data.

Page 43: An Assessment of Two Different Teaching Formats for ... · • Faculty Feedback/Perspectives. Additional Assessments and Initiatives •Analysis of Instructor Evaluation Questionnaire

STRATEGIES FOR RECRUITING FACULTY

•Gather feedback from faculty on what would help orient them to the class; assess the needs of those considering teaching the course.