Embed Size (px)
Citation preview
1
Facilitation and Assessment of Student Learning
A Guide for Faculty, Staff, Administrators, and Students
2
TABLE OF CONTENTS
Part One: An Overview 4
Active Learning 6
Brain‐Based Learning 9
Assessment at SCC 12
Establishing a Culture of Assessment 13
Philosophy of Assessment 14
Career and Technical Program Assessment 15
LAS Assessment 16
Classroom Assessment 17
Program Assessment Documentation 17
The Six Column Assessment Plan 18‐19
Example Assessment Grid 19
LAS Assessment
Part Two: Tools of Assessment 20
Direct and Indirect Measures 20
Capstone Courses 20
Portfolios 21
Rubrics 22
Application to Program Assessment 24
Analytic Rubric Example 24‐25
Holistic Rubric Example 26‐27
Direct Program Assessment Measures 27
Indirect Program Assessment Measures 29
Classroom Assessment Techniques 29
Course‐Related Knowledge and Skills 30‐33
Attitudes and Values 33‐34
Reactions to Class Activities 34‐36
3
Part Three: References and Appendices
References 36
Appendices 37
Appendix I (Glossary) 38
Appendix II (definitions) 40
Appendix III (Assessment Presentation for Students) 41
Appendix IV (Blank Outline) 43
Appendix V (Answer Key for Blank Outline) 45
Appendix VI (CAT Training) 47
Appendix VII (LAS Rubrics) 48‐52
4
PART ONE: AN OVERVIEW
Overview
Assessment of student learning, with an emphasis on assessing program and institutional core compe‐
tencies, has been an initiative of South Central College (SCC) over the past several years. Assessment of
student learning is tied to the broader realization that institutions of higher education need to be ac‐
countable for outcomes and should be working to improve student success by focusing on learning as
well as teaching. The emphasis of any institution of higher education should be on how to strengthen
and enhance the ability of graduates to transfer their learning to an ever‐changing workplace and soci‐
ety in which they live.
The assessment of student learning in the classroom, in the courses, and at the program level has direct
ties to active learning. Through the various assessment strategies, students become actively engaged in
the learning process by providing valuable feedback that impacts instructional delivery and course con‐
tent. In addition, students learn to take responsibility for their learning and to use self‐assessment tech‐
niques that further engage them in the learning process. As a result, students become partners in the
learning process. Current developments in learning and brain theories support the active involvement
of students in the learning process.
This handbook is the result of faculty requests for a guide that can be referenced as assessment activities
are developed at the course and program levels. It should be used in conjunction with the Online Assess‐
ment Training Tool. This handbook starts with an explanation of the roles and duties of the Student
Learning Coordinator and the Student Learning Committee. The handbook continues onward with a brief
overview of active and brain‐based learning that establishes a theoretical basis for assessment activities.
This overview is followed by a review of the college’s assessment policy and plan, including directions for
completing the six‐column assessment model to be used for documenting program assessment activi‐
ties. It also examines LAS assessment and the process therein. Specific program and classroom assess‐
ment techniques and activities are presented in the next sections. The appendices contain summaries of
classroom assessment activities; a blank outline for student presentations is included.
The goal of assessment at South Central College is to develop a culture of assessment. Assessment of student learning at the various levels should become automatic and the results utilized to do the follow‐ing: enhance learning through continual improvement of curricula, instruction, and student support ac‐tivities; promote program modification; and impact the budgeting process. To be truly effective, the en‐tire assessment process must continue to be faculty driven. It is the hope of the Student Learning Com‐mittee that this handbook will make it easier for faculty to engage in assessment activities that enhance student learning. A culture that focuses on continual improvement and transformation can be an excit‐ing and rewarding one in which to learn and teach. Learning is a fundamental human activity that leads to creativity and satisfaction. Assessment is another dimension of a basic human need.
5
Student Learning Coordinator
The Student Learning Coordinator facilitates and oversees all college assessment activities, including providing training and support to campus personnel and departments to assist in the development and implementation of effective assessment of student learning initiatives. In addition, provision of assis‐tance to faculty and staff in the collection, analysis, and practical application of their data falls upon the coordinator. Other duties include:
Collecting and maintaining records of annual assessment programs for all academic areas in both paper and electronic formats.
Providing leadership for the Student Learning Committee Recruiting new members for the committee annually Providing annual schedule of SLC Meetings to all committee members Providing monthly agendas for meetings Maintaining minutes
Maintaining documentation of all assessment of student learning initiatives (workshops, meet‐ings, conference attendance, presentations, special projects, etc.).
Assisting with the HLC on‐site visit. Providing leadership for the HLC Assessment for Student Learning Academy Mentor/Mentee
Program. Facilitating the review process for all assessment programs and the creation of the annual re‐
port prepared for AASC review. Maintenance of assessment website information for internal staff. Facilitating implementation of the Student Learning Committee college‐wide plan for student
learning outcomes assessment. Assisting the Student Learning Commitee in creating an annual report that summarizes, ana‐
lyzes, and evaluates the past year’s student learning outcomes, assessment activities and ac‐complishments, and outlines necessary improvements and adjustments to the assessment ef‐fort.
Student Learning Committee
The Student Learning Committee oversees campus‐wide assessment activities and examines their effec‐tiveness. They also promote a culture of continuous improvement in assessment at the institution, pro‐gram, and course level. The SLC also communicates assessment efforts to faculty, staff, students, and the public, in addition to the monthly report made to the Academic Affair and Standards Committee. Other roles include: Educate faculty, staff, students, and the public on the importance and meaning of assessment. Promote discussion college‐wide on assessment activities and initiatives. Provide continuous support and review on the administration, data collection, and outcome‐
reporting. Create an annual report that summarizes, analyzes, and evaluates the past year’s student learning
outcomes assessment activities and accomplishments. Implement necessary improvements and adjustments to the assessment effort.
Develop, implement, and evaluate college‐wide Assessment of Student Learning Plan.
6
ACTIVE LEARNING
Figure 1
Over the past several years, there has been a growing awareness of the importance of actively engag‐ing learners in the learning process. Active learning is most broadly defined as “anything that involves students in doing things and thinking about the things they are doing” (Huba and Freed, 2000). With active learning, students use more of their senses than just listening; they are involved in higher‐order thinking, are engaged in activities such as reading, discussion, and writing, and are encouraged to con‐tinue learning outside of the classroom. It should be noted that while active learning includes small group learning and physical engagement, it also includes self‐reflection, self‐monitoring, critical think‐ing, and association of new knowledge with existing knowledge (Cross, 2003). Another dimension of active learning is assessment. Classroom and program assessments are enhanced when students are actively involved in the design, implementation, and analysis phases. Traditionally, education has been teacher‐centered rather than learner‐centered. In a teacher‐centered learning environment, the focus is on the teacher. In many respects, the teacher works and learns while the students are passively engaged. Active learning seeks to reverse this environment by focusing on the learners and actively engaging them in the process intellectually, emotionally, and sometimes physically. Active learning proponents suggest that by having learners actively engaged in the learning process they will learn faster (accelerated learning) and retain more of the knowledge and skills that are being acquired. The two education paradigms are contrasted in Figure 1.1.
7
Traditional Paradigm
Active Learning Paradigm
Teacher‐centered Student‐centered
Instructor as sage on stage Instructor as coach/mentor
Learners passive Learners active
Students viewed as learners Instructors and students learn together
Emphasis on transmission of knowledge Emphasis on the construction of knowledge
Unemotional Utilizes emotions
Evaluations of learning by instructor Evaluation of learner jointly by instructor and learners
Teaching and assessing different activities Teaching and assessing viewed as supporting activities
Assessment used for evaluation Assessment used to promote and diagnose learning
Emphasis on competition Emphasis on cooperation and collaboration
Emphasis on traditional intelligence (linguistic and math/logical)
Emphasis on multiple intelligences
Figure 1.1 Traditional vs. Active Learning
Work done by the National Training Laboratories (NTL) in the 1960’s underscores the importance of ac‐tive learning to acquisition and retention (Sousa, 2001). NTL researchers found that retention rates at 24 hours and beyond are directly related to the amount of engagement by the learners during the ini‐tial learning. Retention rates are extremely low for passive activities such as listening, reading, and watching. Retention increases as students become actively involved. Students, who immediately use the material, typically involving a number of senses and thought processes, remember the most. These findings are presented in Figure 1.2.
8
Figure 1.2
A comprehensive technical and Liberal Arts education tends to involve the learners through its emphasis on classroom, lab, shop, and clinical learning activities. Classroom instruction with technical education tends to be more traditionally based. In addition, technical education relies heavily on occupational analysis to determine program and course content. As a result, instruc‐tion is directed toward helping the students master the instructional objectives that are closely related to tasks performed on the job. From its beginnings, technical education has borrowed heavily from behavioral psychology to establish learning environments that prepare the student for the workplace. The emphasis on the transfer of skills, knowledge, and attitudes to the workplace has tended to produce a tradi‐tional, teacher‐centered approach to instruction. Classroom instruction in the Liberal Arts and Sciences is more often a mix of traditional and non‐traditional teaching strategies, including lecture, small group work, and discussion groups. Con‐tent is often theoretical in nature, with goals being the understanding and application of theo‐ries to real‐world issues. The educational emphasis is on building knowledge that helps one to better understand the complex world around them and aids in transfer to four‐year colleges. A comprehensive technical and Liberal Arts education can benefit from active learning strategies and methodologies in the classroom to aid student learning. These strate‐gies and methods include the following:
• cooperative learning strategies • case studies • games • jig‐saw assignments • simulations • oral exams • debates • visualizations materials, displays • construction of models and props • projects • role‐playing • themes, posters, pictograms
9
BRAIN‐BASED LEARNING
Going hand‐in‐hand with the active learning movement is the renewed emphasis on brain‐based learn‐ing. The 1990’s were designated as the decade of the brain. As a result, there has been a considerable amount of research conducted using the latest technology to advance the understanding of the brain and its many functions, including learning. Brain based learning research supports the active learning paradigm by providing insights into how the brain functions. This knowledge is useful in helping instructors to better understand how learners as‐similate and store information. Insights regarding the involvement of the entire body in the learning process, including movement, the role of the emotions, and how the brain physiologically transforms the information taken in by the senses into memories are providing the rationale for the use of active learning strategies. Learning is a complex activity that is impacted by a number of environmental, psychological, personal, and physiological factors as illustrated in Figure 1.3. Brain‐based learning research supports the active learning paradigm by providing insights into how the brain functions. This knowledge is useful in help‐ing instructors to better understand how learners assimilate and store information.
Figure 1.3
10
Current thinking emphasizes learning as a mind/brain/body activity rather than a function exclusively of the mind. The importance of physical movement to learning is being explored by a number of research‐ers. Ratey (2001) notes that movement and memory are closely linked because they are both coordi‐nated by the frontal lobe of the brain. Sousa (2000) points out that physical movement decreases as grade level increases and is typically non‐existent in most postsecondary classrooms. He suggests that movement incorporated into the classroom environment can enhance learning by energizing the stu‐dents and introducing novelty into the classroom. Movement can include stretching exercises at mid‐point during a lecture session; using directed activi‐ties such as think‐pair‐share, where students mentally summarize a portion of the lesson and then find a partner in the group to share their insights; role‐playing; using the body in a physical way to learn and remember a difficult concept; and acting out words. Physical activity provides an outlet for kinesthetic learners who need to be active to learn. A second area of inquiry that has been a focus of research this past decade is the role of emotions in the learning process. The role of emotions has been underestimated in the traditional approach to educa‐tion. It has been known for a long time that survival is the top priority for an individual. Unconsciously and consciously the human brain is constantly monitoring the environment for danger. Typically, physi‐cal survival is not an issue in the classroom. However, survival can be extended to psychological survival. Thus, it is important to create an environment in the classroom that is supportive and safe, physically and psychologically, where students can devote their energy to learning rather than survival. Figure 1.4 illus‐trates the priority that the brain gives to the functions of security, pleasure, and learning.
Figure 1.4
11
In addition to survival and security, our brains are also designed to seek pleasure. Again, this is not a new concept. Zull (2002) addresses its importance to the classroom. He notes that individuals get pleasure from physical movement or anticipated movement. In classrooms, labs, and shops, positive emotions (pleasure) can result from movement, achieving goals, solving problems, completing pro‐jects, and producing tangible results. He also notes that a facet of pleasure and security is the need of the human brain to have a sense of control over actions. This need for control is an integral part of the emotional make‐up of the individual. In the classroom, active learning provides students with the op‐portunity to be involved in the learning process and to exercise some control over the learning envi‐ronment. This active involvement engages the brain in a positive way and makes the instructional process more pleasurable. Classroom activities that allow students to exhibit some control over their learning include choices in terms of assignments and learning contracts. The positive emotions that result from the active learning strategies strengthen the neural networks that are responsible for memory. Assessment activities are a part of the active learning process. The various assessment strategies, such as portfolios and project work, actively engage students. The knowledge of results and reaching goals that are documented through the use of rubrics encourage positive emotions. Classroom Assessment Techniques (CATs) provide students with the means to be actively involved in assessing their own learning and to have some input into how the class is structured and delivered. Having a sense of con‐trol provides for ownership and emotions that are conducive to good learning. When the results of program assessment are shared, students engage in positive ways that enhance learning. In summary, assessment, active‐learning, and brain compatible instruction are linked. Student learn‐ing is enhanced as learners become actively and emotionally engaged.
12
13
ESTABLISHING A CULTURE OF ASSESSMENT
It is the goal of South Central College to continue to establish a culture of assessment. As such, the college continues to support assessment activities by sending faculty and staff to assessment work‐shops, by providing an office to coordinate assessment efforts, and by providing a range of informa‐tional/training sessions for faculty and staff. Simultaneously, the college supports student awareness and involvement in the assessment process.
Institutional Context for Assessment of Student Learning The diagram below, Figure 2.1, illustrates that the assessment of student learning is a member of a family of efforts at SCC. These efforts ensure that the most effective, efficient, and meaningful infor‐mation is available to assist faculty and staff in evolving programs specifically and the college generally to best realize SCC’s mission and vision.
Figure 2.1
14
Philosophy of Assessment College‐wide comprehensive, systematic and continuous assessment is critical to achieving our college mission and pursuing our vision. We believe assessment practices are a reflection of the SCC values and further the basic aims and purposes of higher education. Through assessment, the college will be better able to develop the talents and skills of its students and staff to the fullest extent possible and make a positive difference in their lives. In this context, SCC’s assessment model will promote student success, program improvement, and effective and efficient use of resources.
Figure 2.2
The primary outcome of the assessment of student learning is the improvement of curricula, programs, and instruction to further enhance and assure student success. The college’s assessment model pro‐motes student success through the improvement of student services, the effective and efficient use of resources, and most importantly, the improvement of instructional programs. Assessment activities do not just center on the testing of student learning; they also include discussions about what should be assessed, how assessment takes place, and how the assessment results can serve to improve student learning. SCC’s assessment plan covers each of these areas. The inter‐relationship of assessment to the various levels is presented in Figure 2.2.
15
Career and Technical Program Assessment South Central College’s Assessment Plan provides a coherent and effective blueprint for the system‐atic improvement of student learning at the program level. Program assessment focuses on the stu‐dent learning that is taking place. SCC strives to prepare students with quality higher education for the world of successful employment and a desire for continuous and life‐long learning, as well as for possible transfer to four‐year colleges. In the process of preparing students for completion of a cer‐tification, diploma, or degree, the college has established Program Core Competencies (PCC’s) spe‐cific to career and technical programs, and six Institutional Core Competencies (ICC's) that are to be mastered by the student before the completion of their program.
Figure 2.3
The PCC's are the fundamental career specific abilities that a student must master in order to successfully complete a program in preparation for entry into the workforce. The PCC's are es‐tablished by the program faculty under the di‐rection of the program advisory committee. PCC's are assessed in a variety of ways as deter‐mined by the faculty in the respective program areas. With program assessment, student learn‐ing is assessed using direct and indirect meas‐ures.
The results from these measures are then analyzed. The process is complete when the loop is closed and changes and modifications to the curricula are made. The process is then repeated. Such improve‐ment of student learning is consistent with the college’s mission of providing a comprehensive higher education for employment and continuous learning. Figure 2.3 illustrates the steps included in the pro‐gram assessment process. As a part of the program assessment process, faculty members in each career and technical program determine program‐specific student learning outcomes to be measured in an annual Program Assess‐ment Plan to assess actual student learning outcomes. The results of these assessment strategies are analyzed and used to:
1. Enhance student learning through the improvement of curricula, instruction, and student sup‐
port activities 2. Modify the program as needed. 3. Impact the budgeting process for each program.
16
Liberal Arts and Sciences (LAS) Assessment South Central College’s Assessment Plan also provides a coherent and effective blueprint for the sys‐tematic improvement of student learning at the program level of LAS. Assessment across the LAS is in many ways similar to that done in the career and technical areas of the college. The PCCs utilized by the career and technical programs are replaced in LAS by learning outcomes based upon the Minnesota Transfer Curriculum (MNTC). These learning outcomes ensure equality in all LAS courses system‐wide in MNSCU, which aids in the transfer of college credits. They are assessed in a variety of ways at the class and program level, culminating in an assessment of student works parked in eFolio by LAS faculty utilizing rubrics to determine whether educational efforts have been successful and driving any needed changes. These rubrics can be found in Appendix IX. As with the Career and Technical programs, the LAS also has a focus that includes the ICC’s. These core competencies are examined yearly by testing of students with 45 or more completed college credits.
These competencies help to prepare students in a broader way for the world of work and society. Stu‐dents build these competencies through course work; they may also build them through involvement in student organizations, service learning projects, and internships. Individual career and technical pro‐grams have identified areas within the curricula where the institutional core competencies are devel‐oped as shown in Figure 2.4. ICC’s are assessed as part of the assessment of general education and across programs of study.
17
Classroom Assessment Assessment also occurs at the course level when student progress in achieving course objectives is evalu‐ated through objective and performance tests. Classroom Assessment Techniques (CATs) are used in a formative way to quickly assess student mastery or to obtain learner reactions. CATs are discussed in a later section.
PROGRAM ASSESSMENT DOCUMENTATION
As a part of the program assessment process, faculty members in each career and technical program de‐termine program‐specific student learning outcomes to be measured in an annual Program Assessment Plan to assess actual student learning outcomes. A six‐column form and a summary page is utilized for this process.
Directions for Completing the Six‐Column Assessment Plan for Career and Technical Pro‐grams The standardized SCC Program Assessment Record is a six‐column model, as shown in the “Resources & Examples” section. Program faculty members use this document to record the program efforts in as‐sessment. The assessment record serves as a guide for faculty members to organize, document, and report results of student assessment; plus it provides the platform to “close the loop.” The results of assessment strategies are used to indicate how well students are learning the knowledge, skills, and at‐titudes that are critical for them to succeed at an entry level in their chosen career field. The assess‐ment results are used in making changes for improvement in the program and in our instruction. This tool enables faculty members in their respective programs to focus on a learner‐centered approach to education. Below are the steps in completing the six‐column model Program Assessment Record. This record can be completed electronically. The form has been created in Word format and can be accessed on the shared drive.
General Assessment Record Information
a. Fill in Program name (Assessment Record For:____________ )
b. Indicate the academic year that will be covered by the assessment record (example 2009 – 2010)
c. Identify the date the plan was submitted to the Academic Dean on or before required date.
d. Plan Approval Date – leave blank
e. Identify participating faculty members in your program area
f. Sign on the line
18
EXAMPLES OF DOCUMENTS
Sample Six Column Assessment Grid:
19
Liberal Arts and Sciences (LAS) Assessment Assessment in the LAS divisions incorporates the use of eFolio. Students post works considered to be of high quality for any of the ten MnTC goal areas in eFolio as they work through the semester. At the end of the year, works are randomly selected to be evaluated, based upon rubrics created for each MnTC goal area. The works selected are reviewed by faculty member (s) from that goal area, to the extent possible. The rubrics utilized can be viewed in Appendix VII.
20
PART II: TOOLS OF ASSESSMENT
DIRECT MEASURES OF STUDENT LEARNING
CAPSTONE COURSES Definition: A capstone course occurs at or near the end of the program of study. It is designed to integrate the major program competencies that have been developed through other courses and learning activi‐ties. Application to Program Assessment: The objectives of a capstone course are directly tied to the program competencies. The capstone course should be designed to measure how well the students have mastered the program competen‐cies through their previous work in other courses and learning experiences such as clinicals or intern‐ships. Assessment Strategies: Evaluation of the course objectives could be a shared activity in multiple instructor programs. Instruc‐tors may be assigned to portions of the course or multiple evaluations could be used. In terms of pro‐gram assessment, it is important to evaluate the level of mastery on each objective for the entire class by averaging the performance by all of the learners on the various projects and activities. Lower per‐formance by the group on a particular measure could indicate the need to review previous coursework and learning activities to address deficiencies. It is important that the graded work be tied directly to course objectives that measure student proficiency on program core competencies.
21
PORTFOLIOS Definition: A portfolio is a collection of student work, written materials or products that exhibit the student’s progress and achievement over a period of time. A portfolio is a collection of works, written materi‐als, or products. Electronic portfolios have an additional utilization in that they can be used by stu‐dents as they apply for jobs. Portfolios can be developed for a particular course or could be developed over a portion of or an entire program of study. Portfolios are excellent examples of active learning, especially when stu‐dents are involved in the selection of work included and when students are involved in their evalua‐tion. Two types of portfolios are the best work portfolio that showcases the student’s best work and the growth portfolio that demonstrates the development of the student’s skill over time. The growth portfolio shows baseline work and drafts of work leading up to the final product, instructor evalua‐tions, and examples of best work (Martin‐Kniep, 2002; Rotheiser, Bower, and Stevahn, 2000). Portfo‐lios may be part of a public exhibition or show. Application to Program Assessment: A portfolio can be used to assess student learning in a particular course or over a portion of an entire program. For program assessment purposes, a portfolio that includes work over a significant portion of the student’s course work should be considered. However, portfolios, as part of a capstone course or in courses that reflect mastery of a program core competency, could be appropriate. LAS assessment utilizes portfolios in assessment. Specifically, eFolio is utilized as a docking station for student portfolios. These are subsequently reviewed by Las faculty as a form of pogrammatic assess‐ment. Evaluation of the portfolios by outside experts like an employer panel or advisory committee members, evaluation by other instructors in the program, or faculty‐team evaluations should be considered to obtain the best analysis. In addition to providing students with an individual grade in the case of a course, instructors and outside evaluators should analyze the performance of the entire class on each evaluation criterion. Criteria that receive lower average scores should be analyzed to determine what might be contrib‐uting to the lower average scores. Rubrics would be helpful in providing consistency to the evalua‐tion process.
22
RUBRICS Definition: A rubric is a rating scale that provides a detailed description of expected outcomes. It can be viewed as the scoring rules or criteria that will be used to evaluate student work. Rubrics can be used to evaluate student performance or projects. Rubrics differ from scoring sheets, checklists, and rating scales in that the criteria are detailed for each level and dimension of performance. There are two types of rubrics (Martin‐Kniep, 2000). Analytic scoring rubrics include descriptions of performance for each level of performance for each dimension or attribute. An analytic rubric consists of rows of attributes or dimensions and columns for levels of performance. An example of an analytical rubric can be found on page 31. Holistic rubrics are used to assign a single score to a project or student activity. A short detailed de‐scription describes specific attributes that are expected for each score. An example of a holistic rubric appears on pages 32‐33. Uses: Rubrics can be used at the course assessment level to evaluate assignments and at the program as‐sessment level to evaluate activities such as portfolios, projects, and internships. Rubrics are powerful learning tools. They help instructors to clarify what is expected from students in a consistent manner. Students can use them to determine expectations and what constitutes exem‐plary work. The rubrics can be used to monitor student work as students proceed through the devel‐opment of their work or prepare presentations and demonstrations. The advantages of using rubrics in assessment include the following:
allowing assessment to be more consistent clarifying the criteria for evaluation in specific terms showing the student what is expected and how it will be evaluated providing the instructor with feedback about instructional effectiveness providing students with a basis for peer evaluation
Construction: When developing a rubric, consider the following factors: 1. Elements of high quality work need to be identified: “What do the students need to master?” The
answer to this question will determine the number of rows needed on an analytic rubric. 2. Levels of performance need to be determined. This will determine the number of columns needed
(examples include: Exemplary – Proficient – Acceptable – Unacceptable; Superior – Satisfactory – Needs Improvement).
3. Point values or rating scheme needs to be determined. Each column will be assigned a point value or letter grade. Some dimensions could be weighted differently due to importance or significance.
4. Clear, concise descriptions must be written for each level of each dimension.
23
Levels of Performance
Criteria Exemplary 4 points
Proficient 3 points
Acceptable 2 points
Unacceptable 1 points
Criterion 1 Descriptors Descriptors Descriptors Descriptors
Criterion 2 Descriptors Descriptors Descriptors Descriptors
Criterion 3 Descriptors Descriptors Descriptors Descriptors
Components and Qualities: All rubrics have a title, levels of performance, elements/dimensions to be assessed, and descriptors that detail the expected performance. An objective that the rubric is being used to assess could be included as well. Levels of Performance – Levels are arranged in a continuum of mastery from little to accomplished. The distance between the levels should be more‐or‐less equal. More levels will allow for more dif‐ferentiation of skill. Points can be assigned to the levels or designations such as Novice‐Practitioner‐Expert might be used. Elements/dimensions – The criteria to be assessed should include all the elements that need to be mastered by the student to complete the activity. The elements/dimensions should be of equal im‐portance and integral to the mastery of the content or demonstration of proficiency. (Elements/dimensions can be weighted to differentiate importance.) They can be grouped under general head‐ings to provide further clarity and aid student understanding. Descriptors – Descriptors should be written using qualitative words with specific statements that indicate to students what is expected in their work rather than what is missing in terms of per‐formance. Evaluative judgments and relative language (for example, some mistakes in punctua‐tion – few mistakes in punctuation – no mistakes in punctuation) should be avoided. Descriptors that focus on specific behaviors or outcomes, such as measurement, could be appropri‐ate. Space should be provided for additional comments. Students can be involved in the development of rubrics as well. Students can be good sources of feedback, and they can make contributions to the content. Students could develop rubrics to evaluate each other as part of a project. .
24
Other considerations when developing rubrics include the following:
descriptors should be specific descriptors should be precise descriptors should not rely excessively on quantitative terms like several, numerous, and most descriptors should describe qualitative differences in performance at the levels of quality in‐
stead of suggesting that lower levels are less than higher levels descriptors can be tied to quantitative industry standards rubrics are a work in progress rubrics can be refined through peer review and student input.
As faculty developing a rubric, answering the following six questions should be helpful (Huba, 2000):
1. What criteria/essential elements must be used to ensure that quality is sufficiently high? 2. How many levels of achievement are necessary to give students meaningful feedback? 3. What rating scale should be used? 4. What is a clear description of performance at each level of achievement? 5. Is the format user friendly? 6. What needs improvement (after using)?
Other tips: Periodically update rubrics. Constant revisions over time will produce rubrics that are clear and concise. Rubrics can be shared. Faculty within a program can jointly develop rubrics that are used by all instructors teaching a particular class. Application to Program Assessment: Rubrics have a variety of uses. They are a useful tool for evaluating capstone projects, portfolios, final projects, internships, and student performance. Rubrics also have great utility within courses to deter‐mine levels of student performance.
Master Intermediate Novice
Rubric Maker Rubric Maker Rubric Maker
25
Rows – Elements stu-dents need to measure
Descriptors – Specific state-ments about qual-ity of work at each level
Elements/Dimension – Attributes, qualties, behaviors that are used to evaluate performance or the project/product.
Each element/dimension clearly iden‐tifies and explains the teacher’s main ideas, expectations, and re‐quired components of the assignment.
Each dimension identi‐fies the teacher’s main ideas and required com‐ponents of the assign‐ment.
Dimensions are too gen‐eral, some dimensions are missing.
Descriptors – The statements used to describe perform‐ance. Rows— Elements students need to measure
Descriptors are written in concise and clear terms that completely describe what is ex‐pected from the students at each level. Emphasis is on precise, concrete, and descrip‐tive statements with all levels written in terms of what the student has included (vs. what is missing). There is a descriptor for each element at each level.
Descriptors are written in clear terms that con‐vey teacher expecta‐tions. Descriptors include some vague terms. There is a descriptor for each level. Descriptors— Specific statements about quality of work at each level
Descriptors are vague and difficult to understand, leaving too much to stu‐dent interpretation. Descriptors focus on what is missing. Value laden terms – good or vague terms – somewhat satisfac‐tory. Descriptors are missing.
Levels – The scale of the rubric (eg. in this rubric, 1 through 3).
Descriptors provide scaffolding for students by describing an evenly graduated progression toward excellence. Level titles use terms that show respect and dignity for the learner/user while accurately critiquing performance on the task. The number of levels is dictated by concrete, nontrivial differences in student performance.
Descriptors at adjacent levels progress in even steps toward achieve‐ment. Level titles focus learner/user on the grade and identify the levels of achievement (eg. proficient). The number of levels is determined by external constraints (grades, out‐side agencies) rather than distinctions in stu‐dent performance.
Unequal difference be‐tween the descriptors at adjacent levels hinders stu‐dent self‐evaluation and correction. Level titles focus learner/user on grade (4, 3, 2, 1). The number of levels does not permit sufficient dis‐tinction in quality of stu‐dent work.
26
HOLISTIC RUBRIC EXAMPLE
Learning Styles Summary Paper Your paper will be evaluated using the following criteria as they relate to construction and content. This activity is worth 25 points in its entirety. A. Construction – 5 points possible 5 Points –
Good organization – Introduction – Body – Conclusion Paragraphs – well constructed – topic sentence followed by well developed idea. Appropriate
length Spelling and grammar – 1 error maximum Sentence structure – complete sentences, subject/verb agreement Punctuation – appropriate use of colons, commas, and semicolons, 1 error maximum Format – double‐spaced, 1 inch margins, 12 point font, meets or exceeds 2 page minimum length
4 Points –
Good organization ‐ Introduction – Body – Conclusion Paragraphs – well constructed – topic sentence followed by well developed idea. Appropriate
length Spelling & grammar – 2 errors maximum Sentence structure – complete sentences, subject/verb agreement Punctuation – 2 error maximum Format – double‐spaced, 1 inch margins, 12 point font, meets or exceeds 2 page minimum length
3 Points –
Good organization Paragraphs ‐‐ need to be combined, lengthened, or shortened Spelling and grammar – 3 errors maximum Sentence structure – 3 errors maximum Punctuation – 3 errors maximum Format – double‐spaced, 1 inch margins, 12 point font, meets or exceeds 2 page minimum length
2 Points –
Organization ‐‐ need to be clearer/sharper, hard to differentiate sections of paper Paragraphs ‐‐ need to be combined, lengthened, or shortened Spelling and grammar – 4 errors maximum Sentence structure – 4 errors maximum �Punctuation – 4 errors maximum Format – problem with spacing, 1 inch margins, length less than 2 page minimum
27
1 Point – Organization ‐‐ lacking an introduction and/or conclusion Paragraphs – excessively short, unorganized Spelling and grammar – 4 or more errors Sentence structure – 4 or more errors Punctuation – 4 or more errors Format – excessively below 2 page minimum
B. Content – 20 points 18 ‐ 20 Points – A The personal results on each inventory (Gregorac Style Delineator, hemispheric inventory, modalities, and multiple intelligences) are presented and explained. There is good evidence that the assigned read‐ings have been read and understood through the written narrative (i.e., there is an explanation of what the results mean). The second section of the paper describes how knowledge of the inventories will enhance your teaching and student learning. In this section of the paper, the impact of this knowledge will have on materials selection, presentations, classroom environment, and learner activities are clearly presented. Applica‐tion demonstrates good reflection and specifics on how this information will be used. Comments com‐municate a learner‐centered approach to instruction. 16 ‐ 17 Points – B Personal results are presented. Commentary reflects an above average understanding of the inven‐tories. It is evident that the on‐line materials have been read. The second portion of the paper addresses the instructional applications in a more general way. Depth of reflection is not as evident. Some specific examples are used. The focus is on helping learners. 14 ‐ 15 Points – C Personal results are presented with little reflection and reflect a minimum amount of understand‐ing. It would appear that the on‐line materials have not been read or adequately understood. The second portion of the paper addresses the utilization of the knowledge of learning styles in a basic and superficial way.
DIRECT MEASURES OF STUDENT LEARNING
Exams Measures: Comprehensive exams, state board exams, normed tests, final projects, final papers, and performance tests can be used effectively to evaluate student learning for program assessment. Comprehensive tests are developed by the program faculty to measure student learning on the cognitive portions of courses. To be used for program assessment purposes, the exam is designed to measure student mas‐tery of the program core competencies. Instructors can then analyze group performance on each of the competencies to determine instruction that might need improvement or change.
28
Students in several programs are required to take state board exams for licensure purposes. Certifica‐tions also exist in other program areas. Group pass rates provide only an indirect measure of student learning. Exam results that provide a breakdown of student performance on the various areas tested provide a direct measure of student learning. Scores averaged for the group across each content area on the exam can be used to analyze instructional effectiveness. In some cases, such as in nursing, a pre‐paratory exam can be used to obtain scores on the various dimensions of the test when the results on the various sections of the exam are not available. Normed exams provided by vendors like the National Occupational Competency Testing Institute (NOCTI) are another option for measuring student achievement. An advantage to these exams is that the results are normed. Thus, comparisons on student learning can be made to other groups of stu‐dents. It is critical that the competencies measured by the test correlate to the program core compe‐tencies. Two examples of a NOCTI summary of results are found on page 35. Final projects, papers, and performances can be good direct measures of student mastery of program core competencies. To be effective, the activities must be designed specifically to measure the pro‐gram core competencies. The assessment is enhanced when outside experts or faculty teams evaluate the activities, thus removing rater bias (a person who rates student outcomes with a prejudiced view). Rubrics can be extremely helpful to ensure consistency and to document the evaluation.
29
INDIRECT MEASURES OF STUDENT LEARNING
Definition:
Indirect measures are indicators rather than direct measures of student learning. Indirect meas‐ures are appropriate in that they can indicate the general direction that instruction and learning activities are taking in helping students to master the program core competencies. Indirect measures are often easier to obtain since the survey method is frequently used. Indirect meas‐ures do not measure the learning itself, as direct measures do (Huba, 2000).
Types:
Placement rates. For technical programs, placement into related jobs can be an indicator that employers find the program graduates to be prepared and ready for employment.
Student surveys. Asking students to reflect on their learning as it relates to mastery of the core competencies can also be an indicator of student learning. The measure is indirect because the first‐hand observation/measurement of student performance is lacking.
Employer surveys. Employer surveys offer another avenue for feedback on student mastery of program core competencies. Employers rate graduates or interns on their success on the job. These ratings can then be used as an indicator of student learning. Faculty can then analyze the data to develop a general understanding of how the students are mastering the program core competencies.
Examples of indirect measures used at SCC appear on the next three pages. The Institutional Core Competency survey that appears on page 37 is given to graduates as part of the exit inter‐view process. The results are analyzed by the general education faculty and the Assessment Committee. A program specific core competency survey appears on page 38. This indirect measure, which was piloted fall semester 2002, asked students to self‐assess their skill levels at the conclusion of their program and to assess the influence that the college had on the devel‐opment of the competencies. A summary of results appears on page 39 and includes an expla‐nation of results.
CLASSROOM ASSESSMENT TECHNIQUES‐ BASICS
Classroom Assessment Techniques (CATs) are informal assessment techniques that are used throughout the semester to provide valuable feedback to the learner and to the instructor. By using CATs, the in‐structor can modify the course during the semester to enhance student learning. The benefits are four‐fold. CATs can be used to assess student understanding, application, creativity, and problem‐solving and critical thinking skills. Secondly, CATs can be used to measure learner satisfaction with classroom in‐struction and class activities. Thirdly, CATs can be used as learning tools to encourage deeper learning, reflection, and integration of the material. Finally, CATs promote active learning through learner reflec‐tion, increased self‐awareness, involvement, and empowerment. The formative process is presented in Figure 3.1
30
Figure 3.1
The utilization of CATs is a five‐step process, as presented in Figure 3.1. The instructor plans what CAT is appropriate for the situation. The planning might include preparation of questionnaires, power point questions, or overhead transparencies. The instructor administers the CAT. The instructor then analyzes the data and reports the results to the class. The instructor modifies the course and/or cur‐riculum based on the data collected. Learners may be included in deciding what changes should be implemented. Several different CATs are described on the following pages. More detailed information on a number of these strategies can be found in Cross and Angelo’s Classroom Assessment Techniques: A Handbook for College Teachers (1993, San Francisco: Jossey‐Bass). These techniques can be easily modified to meet assessment objectives in a variety of settings. Creative new CATs that are germane to the content area or to the course can easily be developed by the instructor. I. Assessing Course‐Related Knowledge and Skills
A. Understanding
31
Empty Outlines – An empty or partially completed outline is given to the students at the beginning of a class or at the conclusion of a class to assess mastery of the content of homework. Students are given a small amount of time to complete the outline either individually or as a group. The outlines are collected and evaluated by the instructor to determine gaps in student understanding.
Memory Matrix – Students are given a matrix with column headings and row de‐scriptions. The cells of the matrix are left blank. Students fill in the cells. The in‐structor collects the matrix and analyzes the completed cells to determine student mastery and understanding. This approach helps the instructor to assess content mastery and student organization skills.
Minute Paper – At the conclusion of the class, the students are asked to write the main points that were presented during the class or describe how the class infor‐mation would be utilized and applied. Responses are collected and reviewed by the instructor. The CAT should take approximately 60 seconds for written student response.
Muddiest Point – At the end of the class session, students are asked to write down class‐related questions that they have or statements about what is unclear. These are collected and summarized by the instructor, who then provides clarifica‐tion during the next class session.
Diagrams, Flow Charts – At the end of class, students are asked to reconstruct from memory the steps to a procedure or process. Students submit their work for instructor analysis.
Ungraded Quizzes – Students are given a quiz at the end of the session that will be collected but is not graded. The instructor analyzes the quizzes to check for stu‐dent understanding.
32
B. Assessing Critical Thinking
Categorizing Grid – Students are given a grid with two or three categories and a scrambled list of terms or concepts. The students are given a limited amount of time in which to place the terms or concepts into correct categories. The instructor looks for areas of misunderstanding based on placement of their terms or concepts into the correct categories.
Pros and Cons – Students are asked to list the pros and cons, advantages and disadvantages of a con‐cept or topic. The instructor collects and analyzes based on the ability of the students to see two sides of an issue.
C. Application/Performance
Application Cards ‐ After presenting a concept, theory, or procedures, students are asked to write down two or three real world applications on a note card or half sheet of paper. These are collected and evaluated by the instructor.
Student‐Generated Test Questions – Students are asked, individually or in groups, to develop ques‐tions that will be used on a test. Students/groups are given specific content areas to develop the questions.
Directed Paraphrasing – Students are asked to summarize in their own words a specific section of ma‐terial with a specific audience in mind. Nursing students, for example, might be asked to describe a specific set of procedures for a patient. The instructor collects and analyzes their summaries for un‐derstanding and correct applications.
D. Creative Thinking
Approximate Analogies – The instructor prepares sets of analogies that require the students to com‐plete a second analogy. Example: Washington is to the Revolutionary War as ____ is to the Civil War.
These are presented to the students on the overhead or using power point. The students write down their responses on 3x5 cards or half‐sheets of paper. The instructor collects and analyzes the responses.
One‐Sentence Summary – Students are asked to write a one‐sentence summary of a topical area pre‐sented in class or from assigned readings. These are collected and analyzed.
Concept Maps – Students are asked to organize concepts about a given topic by visually/graphically showing the relationship between the various components. These maps can be developed individually or by groups. The concept maps are shared in class and then turned in for analysis by the instructor.
33
Word Journal – Students summarize a given amount of material in one word and then write a para‐graph discussing why they chose the word. These are collected by the instructor and analyzed.
Metaphors – A metaphor is a Figure of speech in which a word or phrase is used in place of another to show likeness or relationship between the two concepts. Common metaphors include brainstorming, ivory tower, and basket case. Students use a metaphor to summarize a concept or learning activity. The metaphors can be shared with the class or can be written down and turned in to the instructor. Meta‐phors can be analyzed to determine the level of student understanding and creative ability.
E. Problem Solving
Problem Recognition – Students are given examples or situations on the overhead or using power point from the discipline. Student must analyze the situation to determine the problem. There should be one problem per event. The students write down their diagnosis to be turned in for instructor analysis.
What’s the Principle – Students are presented problems. They are asked to determine what principles from the lesson are used to solve the problem. The problems can be presented through written materi‐als or using the overhead/power point. Responses are collected for instructor analysis.
II. Assessing Attitudes and Values
A. Attitudes/Value Awareness
Opinion Polls – Students are invited to share their opinions regarding an event or issue by raising their hands or using a scale to register the strength of their opinion (five fingers being strong opinion, fist being a strong disagreement). Instructor and the class can gain a quick understanding of how the class feels on a particular issue.
Double‐Entry Journals – Students divide a page vertically with a line to form two columns. In column one, the students write down key concepts, principles or generalizations from the assigned material. In column two, they record their reactions to the items in column one. These are submitted to the in‐structor for analysis.
Profiles – Students write a one‐page paper profiling a person from their field of study, a historical Fig‐ure, or role model who they admire. Students explain why individuals were selected and summarize the qualities that make the persons exemplary. The profiles are selected for analysis.
34
B. Self‐Awareness as Learners
Interest Checklists – At the beginning of the course, students are given a checklist of topics that will be covered in the course or in a unit of instruction. Students are asked to rank their knowledge of skills with and/or interest in the topic. These are submitted anonymously to the instructor for analysis.
Goal Ranking – Students list several learning goals that they would like to achieve throughout the course. The students then rank the goals as to personal importance. The instructor collects the goals to gain a better understanding of the class.
Learning Styles – The instructor uses a learning styles inventory to help students assess their learning style. The instructor collects the results for analysis. The results are discussed with the students.
C. Personal Learning
Study‐Time Logs – Student keep a log of the time that they spend preparing for class using an instruc‐tor‐developed log sheet. The instructor should review what is to be included on the log. Students can compare their logs with other students. The instructor should collect their logs for analysis.
Punctuated Lectures – Students listen to a portion of the lecture. The instructor stops, has the students reflect on what has been discussed, and has the students write a short summary, which can be shared with another student or collected by the instructor for analysis. A shortened varia‐tion of this CAT could be a think‐pair‐share activity where lectures are punctuated with segments where students reflect on what has been said, pair with another student, and share their under‐standings.
Diagnostic Learning Logs – Students keep an extended two‐column log where they record concepts that they understood and problems accurately solved in the left column and they record things not understood or problems that they are unable to solve in the right column. Students are asked to di‐agnose their learning strengths and weaknesses. The results are analyzed by the instructor.
III. Assessing Reactions to Class Activities
A. Reactions to Instruction
35
Chain Notes – Students respond to a question about the class. A sheet of paper is passed around for students to write down their comments. A variation would be to have the students write down their comments on 3x5 cards which are then placed in an envelope. The instructor analyzes the comments and reports to the class.
Electronic Feedback – The CAT is a variation of chain notes. The instructor poses a question electroni‐cally about the class and students respond by the same means.
Student Advisory Panel – The instructor meets with a small group of representatives of the class. The students provide input to the instructor based on feedback given to them by fellow students. The in‐structor and the students develop responses or jointly come up with solutions. The student representa‐tives share in the responsibility of promoting the changes.
Critical Incidents – The instructor asks the students questions to which the students respond with spe‐cific examples of behavior. The instructor analyzes these “critical incidents” as to expectations, con‐cerns, and satisfaction. Examples of questions includes the following:
“At what moment did you feel most in tune with what was happening in class?”
“At what moment did you feel most frustrated?” “What one thing did the teacher do that helped your
learning the most?” “What teacher behavior annoyed you the most? “What student behavior annoyed you most?”
5 and 5’s – An outside facilitator meets with the class at the request of the instructor who is not present for the activity. The facilitator divides the class into small groups of 4 to 5 students. The students are asked to reflect on their reactions to the class – instruction, activities, and assignments – and to identify five strengths and five areas that could be improved. After 10 to 15 minutes, each group reports to the large group. The facilitator records student comments and, through consensus, the class agrees to five strengths and five areas that can be improved. The strengths and areas of improvement are then pri‐vately given to the instructor by the facilitator. Student anonymity is ensured. The instructor then discusses the findings with the class and makes modifications with student input.
B. Reactions to Class Activities/Assignments
Reading Rating Sheets – Students rate the course readings using instructor‐prepared assessment forms. Questions should be brief, limited to five questions that are rated on a scale or yes/no choice. Space may be provided for written comment.
36
Assignment Assessment – Students are asked to assign value to the assignments that they are given as a part of the class. The instructor chooses an assignment and asks two or three leading questions to which the students respond in writing. Questions could includes the following:
“How has the assignment contributed to your learning?” “How did the assignment meet your learning style needs?’ “How would you change the assignment so that it will be more beneficial?”
The instructor collects the responses for analysis.
Exam Evaluations – This CAT will help to direct student learning through student analysis of how the testing procedures used in the class helps students improve the way they learn the material. The in‐structor, either as part of the test itself or as a follow‐up, asks the students to assess the testing proc‐ess. Examples of questions includes the following: “Which part of the test was the best measure of your learning and why?” “Which part of the test was the fairest assessment of what you know? Why?” “What type of test should be used during the rest of the semester? Why?” The teacher analyzes the collected comments.
37
References:
Cross, K.P. (2003). Techniques for promoting active learning. Phoenix: League for Innovation in the Community College.
Huba, M.E. & Freed, J.E. (2000). Learner‐centered assessment on college cam‐
puses: Shifting the focus from teaching to learning. Needham Heights, MA: Allyn & Bacon.
Martin‐Kniep, C.O. (2000). Becoming a better teacher: Eight innovations that
work. Alexandria, VA: ASCD. Ratey, J.J. (2001). A user’s guide to the brain. New York: Pantheon Books. Rolheiser, C., Bower, B., & Stevahn, L. (2000). The portfolio organizer: Suc‐
ceeding with portfolios in your classroom. Alexandria, VA: ASCD. Sousa, D.A. (2001). How the brain learns. Thousand Oaks, CA: Corwin Press,
Inc. Zull, J.E. (2002). The art of changing the brain. Sterling, VA: Stylus.
Appendix I
GLOSSARY Active learning: When students participate, cooperate, collaborate in their assessment of what is learned, they reinforce their grasp of the course content and strengthen their own skills at self learning and assessment. The students’ focus more clearly, feel more confident that they can suc‐ceed, and are likely to do better in their coursework. Analytic scoring rubric: A rubric including descriptions of performance for each level of perform‐ance for each dimension or attribute. An analytic rubric consists of rows of attributes or dimen‐sions and columns for levels of performance. Assessment: The act of collecting information about individuals or groups of individuals in order to understand them better.
38
Classroom assessment: An approach designed to help teachers find out what students are learn‐ing in the classroom and how well they are learning it. This approach is learner‐centered, teacher‐directed, mutually beneficial, formative, context‐specific, ongoing, and firmly rooted in good practice. CATs: Classroom Assessment Techniques are a systematic approach to formative evaluation, and are simple tools for collecting data on student learning in order to improve their learning. These are "feed‐back devices," instruments that faculty use to find out how much, how well, and even how students are learning. Direct assessment measures: Evaluating student learning outcomes based on faculty controlled as‐sessment measures. Examples include: portfolios, exams, internship evaluations, standardized test‐ing, and performances evaluated with rubrics. These strategies measure learning directly.
Evaluation: A judgment regarding the quality or worth of the assessment results. These are usu‐ally based on multiple sources of assessment information (for example, "The information we col‐lected indicates that students are performing above expectations."). Generalized Rubric: A rubric that can be used to score performance on a large number of re‐lated tasks (for example, to score problem‐solving and communication skills on any math prob‐lem‐solving problem). Holistic rubric: A rubric used to assign a single score to a project or student activity. A short de‐tailed description provides specific attributes that are expected for each score. Indicator: A more specific description of an outcome in terms of observable and assessable behav‐iors. An indicator specifies what a person who possesses the qualities articulated in an outcome understands or can do. For example, a student may demonstrate his or her understanding of prob‐lem solving by finding a solution to an accounting problem. The solution is an indictor. Indirect assessment measures: Strategies that ask for reflection or opinion about the learning that has taken place. With indirect measures, student behavior/work is not observed or evalu‐ated. Examples of indirect measures include: graduate program core competency surveys, evaluations by employers of graduates, the SCC Core Competency survey, and placement rates of graduate. Norm‐referenced assessment: An assessment designed to reveal how an individual student's per‐formance or test result ranks or compares to that of an appropriate peer group. Performance assessment: Direct, systematic observation of actual student performances and rat‐ing those performances according to pre‐established performance criteria. Portfolio: A purposeful, integrated collection of student work showing effort, progress, and/or degree of proficiency.
39
Reliability: An indication of the consistency of scores across evaluators, over time, or across different versions of the test. An assessment is considered reliable when the same answers receive the same score no matter when the assessment occurs or how or who does the scoring or when students re‐ceive the same scores no matter which version of the test they took. Rubric: An established and written‐down set of criteria for scoring or rating student performance on tests, portfolios, writing samples, or other performance tasks. Scale: The range of scores possible on an individual item or task. Performance assessment items are typically scored on a 4‐ to 6‐ point scale, compared to a scale of 2 (right/wrong) on multiple‐choice items. Standardized assessments: Assessments that are administered and scored in exactly the same way for all students. Traditional standardized tests are typically mass produced and machine scored and are designed to measure skills and knowledge that are thought to be taught to all students in a fairly stan‐dardized way. Performance assessments can also be standardized if they are administered and scored in the same way for all students. Standardization is an important consideration if comparisons are to be made between scores of different individuals or groups. An example of a standardized assessment instrument is the Collegiate Assessment of Academix ) Standards (performance): An established level of achievement, quality of performance, or degree of proficiency expected of students. Examples include a cutoff score on a multiple‐choice test or an ex‐pected benchmark performance on a performance assessment, like a minimum score of 70 on CDL be‐hind the wheel test. Task‐specific rubric/scoring: A scoring guide or rubric that can only be used with a single exercise or performance task. A new rubric is developed for each task (for example, aligning the front‐end in Auto Service, demonstrating input technology in Office Technology, setting up and operating screen printing equipment safely in Graphics Production, and performing CPR in the Emergency Medical Training program).
40
APPENDIX II
This presentation is available for faculty to use with students
41
42
APPENDIX III
Blank Outline – Assessment Name: _________________ Alias – Cheat Sheet Date: ___________________ a.k.a. – Swiss Cheese Worksheet Students complete the outline below while participating in the Assessment Lesson Presentation. 1. What is assessment?________________________________________ 2. How do we know if learning is successful?
a) _______________________________________________ b) _______________________________________________ c) _______________________________________________ d) _______________________________________________
3. Why do we assess learning at South Central College? • Guarantee ______________ • Ensures currency in our __________________ • We value student __________________ • For ________________ Improvement 4. Who does assessment? S_________________
E________________ F___________________
5. Where does assessment occur? (List three areas that work in our program)
• • •
6. What are the (6) SCC Institutional Core Competencies? _________________________ _____________________________ _________________________ _____________________________ _________________________ _____________________________
43
7. When do we assess learning at SCC? (Circle on the continuum line below when you think assessment occurs)
Day 1 Mid Program Program End
8. Draw lines from the asterisks by the assessment measures to connect the appropriate type
of assessment (bulleted) technique. Hint‐2 CATs & 2 PATs Classroom Assessment Techniques *Portfolios
*Blank Outlines Performance Assessment Techniques *Internships
*Concept Maps
9. Define a “DIRECT” measure of assessment. 10. Define an “INDIRECT” measure of assessment. 11. Roles of student and faculty in the assessment process – connect the appropriate role to
the student and faculty by drawing a line to each.
Student role …is to directly assess learning by establishing core competencies, creating an assessment plan with strategies, collect and document results of assessment, discuss those results, make changes in the program based on results, utilize CATs periodically.
Faculty role …is to give feedback regarding learning,
identify program & institutional core competencies and learn them and at program completion demonstrate the ability to perform at a level necessary for success in the industry.
44
APPENDIX IV
Blank Outline – Assessment Students complete the outline below while participating in the Assessment Lesson Presentation. 1. What is assessment? Knowing if learning is successful 2. How do we know if learning is successful?
a. Are students learning what we intend for them to learn? b. Perform the intended instructional outcomes c. Demonstrate achievement of program competencies at program completion d. Document, evaluate and change to improve
3. Why do we assess learning at South Central College?
Guarantee quality Insures currency in our graduates We value student learning For Program Improvement
4. Who does assessment? Students Employers Faculty 5. Where does assessment occur? (List three areas that work in our program)
classroom, internship, lab, clinicals, on‐ line, shop, job site, at program end
6. What are the (6) SCC Institutional Core Competencies?
Communication, Math Logic, Critical Thinking, Citizenship, Professionalism, Technological Lit‐eracy
7. When do we assess learning at SCC? (Circle on the continuum line below when you
think assessment occurs)
45
8. Draw lines from the asterisks by the assessment measures to connect the appropriate type of assessment (bulleted) technique. Hint‐2 CATs & 2 PATs Classroom Assessment Techniques *Portfolios
*Blank Outlines Program Assessment Techniques *Internships *Concept Maps 9. Define a “DIRECT” measure of assessment. evaluating students learning outcomes based
on faculty controlled assessment measures. For example: portfolios, exams, internship evaluations, standardized testing, performances evaluated with rubrics. These strategies measure learning directly.
10. Define an “INDIRECT” measure of assessment. strategies that don’t measure student learning. Evaluation of student outcomes based on assessment measures requested but not controlled by faculty. For example, graduate program core competency surveys,
evaluations by employers of graduates, or the SCC Core Competency survey, or placement rates of graduates.
11. Roles of student and faculty in the assessment process – connect the appropriate role to the student and faculty by drawing a line to each.
Student role …is to directly assess learning by establishing core com‐
petencies, creating an assessment plan with strategies, collect and document results of assessment, discuss those results, make changes in the program based on re‐sults, utilize CATs periodically.
Faculty role …is to give feedback regarding learning, identify program & institutional core competencies and learn them and at program completion demonstrate the ability to perform at a level necessary for success in the indus‐try.
46
APPENDIX 5
Classroom Assessment Techniques (CATs) are a necessity for faculty to determine learning outcomes. The following set of slides key in on the basics of CATs. An interactive presentation is available for faculty to become “more friendly” with CATs. Contact the campus assessment director for further
47
48
APPENDIX VI
49
50
51
52
53
