Embed Size (px)
DESCRIPTION
What Students Know and What They Don't Know - Course Technology Computing Conference Presenter: Cheryl Reindl-Johnson, Jennifer Day & Jennifer Romero, Sinclair Community College We have been hearing for years that our jobs will be obsolete as students enter college knowing more about computers than their college faculty members know. At Sinclair Community College we have NOT found that to be the case. In fact, we have struggled with success and retention rates in our Computer Concepts and Applications course. We offer roughly 80 sections of our BIS 1120 Computer Concepts and Applications course each semester, enrolling approximately 4,000 students per year. We realize that there are myriad reasons why students are not successful in a course, but to begin to find a solution to address low success and retention rates in this high enrollment course, we felt we needed to do some analysis of our student population. We are using SAM to administer a pre-test to all students enrolled in a section of the course during the second week of the semester, so that we can better analyze student knowledge of the material when they arrive in our classes, and collect information on student mastery of the material at the end of the semester using the same content in a post-test. SAM's reporting features allow us to analyze section level results to compare: Day classes that tend to include more “traditional” students who are straight out of high school, to evening classes which tend to include more non-traditional students; Face-to-face to online sections; Full semester classes to 12-week or 8-week sections; Sections taught by full-time faculty versus adjunct faculty. When the pre-test and post-test are scheduled by our SAM faculty administrator, we can also analyze the data at the question level (frequency analysis) across all sections to examine: What content most students (70% or more) come to the class knowing (specific skills that are used across applications and groups of skills by application); What content most students do not know when they begin the class; What content most students struggle with at the end of the class.
Citation preview
What Students Know and Don’t KnowJennifer Day, Cheryl Reindl-Johnson, and Jennifer RomeroSinclair Community College
The Issues
• Belief that students come to college knowing MS Office from high school classes and/or work experience.
• BIS 1120 Computer Concepts and Applications - high enrollment course (4275 per year), but low success rates (53.7%).
BIS 1120
• Three semester-hour, 16-week course covers computer concepts and introductory Word, Excel, Access, PPT.
• Custom New Perspectives text and SAM.• A variety of approaches to explore low success rate
over the years.• Pre-exam/Post-exam approach
Student Enrollment
4%
58%
3%4%1%
29%
Location
CVCCDaytonELCHHLCPCLCWWW
Exam Breakdown• 70 Items in 75 Minutes– 6 tasks – Windows 7– 13 tasks – Word– 14 tasks – Excel– 14 tasks – Access– 14 tasks – PowerPoint– 5 MC – Browser & Email Basics– 4 MC – Computer Concepts
A computer interprets every signal as either “on” or “off” using numbers known as _____________
1 2 3 4
25% 25%25%25%
1. millibytes2. binary digits3. ASCIIs4. hexadecimals
The rules that establish an orderly transfer of data between sender and receive are called
1 2 3 4
25% 25%25%25%
1. protocols2. channels3. NICs4. priorities
________ changes every time you add or remove hardware; therefore, it is often referred to as semipermanent memory.
1 2 3 4
25% 25%25%25%
1. BIOS2. CMOS3. ROM4. The CPU cache
If a Web page provides an RSS feed, the orange Feeds button appears on the _______________.
1 2 3 4
25% 25%25%25%
1. Command bar2. Status bar3. Address bar4. Favorites Feeds tab
Online (29%) vs. Face-to-Face (71%)
Population # Students Average Min MaxOnline (Full Term) – 225 80% 11% 99%Online (12-Week) 13 84% 67% 94%Online (8-Week Combined) 74 79% 19% 99%
Online (A-Term) 21 84% 49% 99%Online (B-Term) 53 78% 19% 99%
F2F (On Campus) 532 79% 6% 99%F2F (Off Campus) 137 76% 20% 97%F2F (8-Week Term) 7 85% 70% 96%
The RESULTS
Exam Questions• Common questions missed among high scorers
– Multiple choice questions• CMOS question• A URL’s ___ is a standardized procedure the computers use to exchange
data (protocol)• RSS Feed question
– Task Modify the style of the selected table to create a new table style option named “MODEL” with the Header Row using Italic Font and apply it to the table • click Format as Table drop down arrow, right click selected style (modify is
dim) – select “Duplicate…” – that opens dialog box which then you select Header Row and click Format, select Italics, save, click OK, then select Table drop down arrow and apply formatting
13Fall Pretest Frequency Analysis• Multiple choice questions missed on the pre-exam
were also missed on the post-exam. • Task EX2870 no students answered correctly on pre-
exam. Frequently missed (87%) on post-exam.• Exam average – 34%
14Spring Pretest Analysis
• Multiple choice questions missed on pretest are the same three missed on fall pretest.
• Task EX2870 (Modify table) most missed question on exam (3 students).
• Exam average 33%
Scores – Fall 2013 – Post-Exam– 835 registered for sections did not take exam– 108 scores removed because of irregularities
• 990 Scores used from Post exam• 55 sections– 21 taught by FT Faculty (5 faculty members)– 34 taught by Adjunct faculty (23 faculty members)
Average by Application
Subject # TasksPre-testSP14
Pre-Test F13
Post-Test F13
Improvement
MS Access 14 24% 24% 86% 62%
MS Excel 14 19% 19% 68% 49%
MS PowerPoint
14 39% 40% 82% 42%
MS Windows 7 6 60% 59% 88% 29%
MS Word 13 33% 34% 83% 49%
Overall Average
62 32% 32% 80% 48%
High ScorersStudents Pretest
AveragePost Test Average
Women 10 37.2% 98%Men 15 46.4% 98%Total Students 25 41.8% 98%
Day vs. Evening Sections
Students Average Min MaxDay 511 79% 6% 99%Evening 117 76% 14% 99%
TimeShortest time = highest scores Highest time = lowest scores
• 41 student took 75 mins (max) – 10 (24%) scored 50% or – 28 (68%) did not pass
Score Time94% 0:12:5697% 0:15:3896% 0:15:5897% 0:16:1694% 0:16:34
Full-time v. Adjunct
• 59% students taught by adjuncts, 41% by Full-time• Overall Average = 79%– Adjunct Average = 78%– Full-time Average = 80%
• 9 of 56 sections average below 75% - 7 were taught by adjuncts – 2 of those brand new faculty –
• Highest class average =86%, lowest 65%
Highest Average by Faculty
• Highest average 86% – Day (FT - online) - 21 students with only 1 student not
passing post-exam (scored 64%)- 10 of 21 scored 90% or higher.
– Paranjpe (Adjunct - 8 week A term) - 2 of 15 did not pass (57% and 70%) – 9 of 15 scored 90% or higher
– Aungst – (FT – F2F) 2 of 10 students did not pass (57% and 74%) – 6 of 10 scored 90% or higher
Post-Exam Statistics
• 70.4% overall passed (75% or higher)• 73% male passed• 68% female passed
Post-Exam by Gender
Male – 423 (42.7%)
A - 57 (13.5%) B – 142 (33.6%) C - 109 (25.8%)D – 66 (15.6%)F – 49 (11.6%)
13%
34%
26%
16%
12%
A B C D F
Post-Exam by Gender
Female – 567 (57.3%)A - 65 (11.5%) B - 141 (24.9%) C - 182 (32.1%)D – 102 (18%)F – 77 (13.6%)
11%
25%
32%
18%
14%
A B C D F
Lessons Learned
• More faculty involvement in test creation• Better communication about significance of exam
with adjunct faculty• Naming/scheduling of exam by Superuser • SAM paths allow students to focus on what students
don’t know
Next Steps
• Involved faculty reporting other data (not SAM data)• Improving the pre/post exam to focus on course
objectives • Department changing course content to focus only
on apps
