COMPASS Guide to Effective Student Placement and Retention in

COMPASS Guide toEffective Student

Placement and Retentionin Language Arts

Setting the Right Course for College Success

i

CONTENTS

Section 1: Introduction....................................................................................................... 1

Section 2: How to Select Which Test(s) to Use ................................................................ 2

COMPASS Placement Tests Overview ............................................................................. 2

COMPASS Writing Skills Placement Test ...................................................................... 2

COMPASS e-Write Tests ............................................................................................... 3

COMPASS Reading Placement Test.............................................................................. 5

COMPASS Diagnostic Tests Overview.............................................................................. 6

COMPASS Writing Skills Diagnostic Tests..................................................................... 6

COMPASS Reading Diagnostic Tests ............................................................................ 6

How to Determine Which Test(s) to Administer ................................................................. 8

Test Selection & Content Alignment Overview ............................................................... 8

Align Course Content with COMPASS Placement Tests: Selection & Configuration ...... 9

Align Course Content with COMPASS Diagnostic Tests: Selection & Configuration..... 17

Section 3: COMPASS Placement Tests Setup & Routing Rules ................................... 18

COMPASS Reading Routing Rules.............................................................................. 18

COMPASS Writing Skills & e-Write Routing Rules ....................................................... 20

Section 4: COMPASS Placement Tests & Multiple Measures........................................ 23

Best Practice Focus on Multiple Measures for Writing Courses....................................... 23

Best Practice Focus on Multiple Measures for English Language Arts Courses .............. 25

Section 5: COMPASS Placement Test Cutoff Scores..................................................... 27

Setting Cutoff Scores for COMPASS Placement Tests.................................................... 27

Set Stage 1 Cutoff Scores Using National Data............................................................ 27

Set Stage 1 Cutoff Scores Using Local Content Alignment & Local Data ..................... 28

Evaluate & Adjust Stage 1: Stage 2 Cutoff Scores ....................................................... 28

Cutoff Scores for COMPASS Diagnostic Tests................................................................ 29

ii

CONTENTS (continued)

Section 6: Effective Retention in English Language Arts.............................................. 30

Connecting Students to the Campus Resources ............................................................. 30

Better Advising through COMPASS Demographics: Survey Options............................... 30

Section 7: Improving Student Performance in English Language Arts ........................ 33

Section 8: English Language Arts Program Evaluation with COMPASS Research Reports.............................................................................................................................. 34

References ........................................................................................................................ 35

Appendix A: How to Select Which Test(s) to Use ......................................................... 36

Appendix B: Setting COMPASS Placement Test Stage 1 Cutoff Scores ...................... 39

Step 1—Align Course Expectations with COMPASS Tests ............................................. 39

Step 2—Review Course Alignment with COMPASS Tests & Local Test Results............. 42

Step 3—Develop Placement Messages Based on Cutoff Scores .................................... 54

Step 4—Validate Stage 1 Cutoff Scores .......................................................................... 60

1

Section 1: Introduction

A recent survey* of educators at two-year public institutions nationwide identified the delivery of course placement services, academic advising, and learning support services as critical to helping students persist in their studies and achieve academic success. Recognizing this need for strong course placement and advising support services, ACT developed the Computer-adaptive Placement Assessment and Support System (COMPASS) to assist with the delivery of these services. COMPASS is a comprehensive, computer-adaptive testing program that quickly and accurately assesses students’ skill levels in reading, writing skills, writing production, mathematics, and ESL; provides the information you need to place students in appropriate courses; and connects students to the campus resources they need to achieve their academic goals.

The COMPASS system provides English language arts placement tests in reading, writing skills, and writing production. In addition, the system includes three (3) reading diagnostics tests covering key concepts in the areas of reading comprehension, vocabulary, and reading habits. The COMPASS system also includes eight (8) writing skills diagnostic tests covering critical concepts related to punctuation, verb formation and agreement, usage, relationships of clauses, shifts in construction, organization, spelling, and capitalization. Lastly, the COMPASS system offers routing across placement tests (e.g., from COMPASS Writing Skills to COMPASS e-Write) and from placement tests to diagnostic tests (e.g., from the COMPASS Writing Skills Placement Test to COMPASS Writing Skills Diagnostic Tests) to allow significant performance evaluation capabilities. Over 1,000 postsecondary institutions use COMPASS to help their students start their English language arts studies on a solid footing.

This document provides an overview of the COMPASS Reading, COMPASS Writing Skills, and COMPASS e-Write Placement Tests, along with suggestions on how to align COMPASS test scores with the prerequisites you have established for your English language arts courses. The goal of the COMPASS placement system is to assist you in ensuring that students are placed appropriately, increasing the likelihood of student success. This document also provides suggestions regarding developing placement messages and using COMPASS to connect students with appropriate courses and additional campus resources.

* What Works In Student Retention? – Two-Year Public Colleges: http://www.act.org/path/postsec/droptables/pdf/TwoYearPublic.pdf

2

Section 2: How to Select Which Test(s) to Use

COMPASS Placement Tests Overview

COMPASS Writing Skills Placement Test The COMPASS Writing Skills Placement Test is designed to help determine whether a student possesses the writing skills and knowledge needed to succeed in a typical entry-level college composition course. Examinees are presented with a passage on-screen and are asked to read it while looking for problems in grammar, usage, and style. Upon finding an error, students can replace the portion of text with one of five answer options presented. Writing Skills Test passages are presented to the examinee as an unbroken whole, with no indication of where errors are located. To accommodate the task for computer-based testing, the test passages are divided into a series of segments. Because examinees can choose to edit any portion of the passage, every part of the text is included within these segments, and no part of the text is contained in more than one segment. There is a test item for each segment of the text so that an item with five answer options will appear no matter where an examinee chooses to revise the text. Of the five answer options, option “A” always reproduces the original text segment. If the segment selected by the examinee contains no error, then the correct alternative would be option “A.” Allowing students to select and correct any part of the passage broadens the task from simple recognition of the most plausible alternative to a more generative error-identification exercise. In addition to the items that correspond to passage segments, the COMPASS Writing Skills Placement Test has one or two multiple-choice items that appear after the examinee is finished revising the passage. These items pose global questions related to the passage. COMPASS Writing Skills Placement Test items are of two general categories: usage/mechanics and rhetorical skills. Each of these general categories is composed of the subcategories listed below.

• Usage/Mechanic Items⎯Items in this category are directed at the surface-level characteristics of writing, as exemplified in three major subcategories: punctuation, basic grammar and usage, and sentence structure.

• Rhetorical Skills Items⎯Items in this category deal with misplaced, omitted, or

superfluous commas; colons; semicolons; dashes; parentheses; apostrophes; question marks; periods; and exclamation points.

Additional Information⎯Additional detailed information regarding the COMPASS Writing Skills Placement Test content can be found in Chapter 7 of the COMPASS Reference Manual, which is available within “Welcome” screen of the online COMPASS system Help feature. Detailed technical information regarding COMPASS Writing Skills (e.g., reliabilities, standard errors of measurement) can be found in Chapter 12 of the COMPASS Reference Manual. Sample COMPASS Writing Skills Placement Test items are available from the ACT website at http://www.act.org/compass/sample/pdf/writing.pdf. In addition, COMPASS Placement Test Sample Item Review Sets include specifics on content and sample items for COMPASS tests. Review sets are secure materials and are available by contacting ACT Educational Services at 319/337-1054 or emailing Educational Services at [email protected].

3

COMPASS e-Write Tests For the COMPASS e-Write 2–8 and 2–12 direct writing assessments, examinees are provided with a writing task (prompt) that is framed within a familiar context. This might be a community or school setting where a problem or issue related to that setting is presented. Overall, the structure of the COMPASS e-Write prompts is a single-paragraph format that includes:

• Sentence 1: Initial presentation of the setting, an issue or problem, and the person or the group considering how to address the issue or problem.

• Sentence 2: Additional information about the setting, the issue or problem, and the outcome the person or group wants to achieve.

• Sentences 3 and 4: Two different proposals for addressing the issue or problem.

• Sentence 5: Instructions asking the writer to argue for one of the two proposals by providing a rationale for why the alternative they have chosen will be more likely to achieve the desired outcome.

This type of prompt requires that the examinee take a position and offer a solution that is supported with specific examples or evidence regarding the position taken. The COMPASS e-Write prompts were developed to ensure that the format of the prompt is accessible and familiar to all levels of English-speaking students. COMPASS e-Write Scoring⎯The purpose of the COMPASS e-Write scoring system is to assess a student’s performance given a first-draft writing situation. COMPASS e-Write 2–8 and 2–12 tests are scored using “holistic” scoring. Holistic scoring is based on reviewing features of writing and providing a score based on a general or holistic impression of the response. The primary scores assigned for COMPASS e-Write 2–8 and e-Write 2–12 are holistic scores. For COMPASS e-Write 2–8, holistic scores are reported on a scale of 2–8. For COMPASS e-Write 2–12, holistic scores are reported on a scale of 2–12. Each score point reflects a student’s ability to perform the skills identified in the respective COMPASS e-Write holistic scoring guides. Responses are evaluated according to how well a student:

• Formulates a clear and focused position on the issue defined in the prompt

• Supports that position with reasons and evidence appropriate to the position taken and the specified concerns of the audience

• Develops the argument in a coherent and logical manner

• Expresses ideas using clear, effective language A student obtains lower scores for not taking a position on the specified issue, not supporting that position with reasons and evidence, not developing the argument, or not expressing those ideas using clear, effective language. A student who does not respond to the prompt is assigned an “unscoreable” code rather than a score.

4

Secondary COMPASS e-Write scores are subscores. Subscores are based on assigning a score associated with a specific feature of writing. Subscores provide users with some additional information regarding direct writing performance; subscores may be used to support instruction. COMPASS e-Write 2–8 and 2–12 subscores align with the following scoring rubric scales: subscores of 1–4 for COMPASS e-Write 2–8 and subscores of 1–6 for COMPASS e-Write 2–12. The five subscores for both COMPASS e-Write 2–8 and 2–12 components are:

• Focus

• Content

• Organization

• Style

• Conventions Please note that the primary holistic scores and the secondary subscores are assigned independently based on the respective holistic scoring rubrics and the respective subscore rubrics. That is, while there is a relationship between the holistic scores and the subscores in terms of general features of writing, subscores for the COMPASS e-Write components are not used to derive the holistic score. Separate scoring occurs for the holistic score versus the subscores. Additional Information⎯Additional detailed information regarding the COMPASS e-Write components (i.e., 2–8 and 2–12 score scales) content can be found in Chapter 8 of the COMPASS Reference Manual, which is available within “Welcome” screen of the online COMPASS system Help feature. Detailed information regarding COMPASS e-Write development and scoring reliability can be found in Chapter 16 of the COMPASS Reference Manual. In addition, the COMPASS/ESL e-Write Answers to Frequently Asked Questions (FAQ) offers in-depth information specific to the direct writing components. This FAQ is available through an ACT Educational Services regional office near you: http://www.act.org/contacts/field.html. In addition, the COMPASS e-Write 2–12 Sample Essay Responses include specifics on content for COMPASS e-Write tests, with detailed information related to the COMPASS e-Write 2–12 holistic scoring rubric (i.e., the content specifics related to this test) and sample essays with scoring explanations. This document is available through an ACT Educational Services regional office near you: http://www.act.org/contacts/field.html. COMPASS e-Write scoring information is also available from the ACT website at http://www.act.org/compass/tests/writingessay.html.

5

COMPASS Reading Placement Test The COMPASS Reading Placement Test emphasizes an examinee’s ability to construct meaning from what is read, focusing on items that assess reading comprehension. COMPASS Reading passages are of five types:

• Prose Fiction passages emphasize the narration of events and revelation of character.

• Humanities passages describe or analyze ideas or works of art and craft.

• Social Sciences passages present information gathered by research.

• Natural Sciences passages present a science topic along with an explanation of its significance.

• Practical Reading passages present information relevant to vocational or technical courses.

The majority of the COMPASS Reading passages are excerpts from copyrighted material, and the remaining passages are original works written by contracted item writers. The reading level of all passages is approximately equal to what a student encounters in the first year of college; much of the excerpted passage material comes from books, essays, journals, and magazines commonly used in entry-level college courses. The average length of the passages is approximately 240 standard words. All operational passages are accompanied by up to five reading comprehension items. These reading comprehension items are of two general categories: referring items and reasoning items.

• Referring Items⎯Items in this category pose questions about material explicitly stated in a passage (e.g., recognizing the explicitly stated main idea of a passage, recognizing the explicitly stated main idea of a paragraph, recognizing explicit evidence presented in support of a claim).

• Reasoning Items⎯Items in this category assess readers’ proficiencies at making

appropriate inferences, developing a critical understanding of the text, and determining the specific meanings of difficult, unfamiliar, or ambiguous words based on context (e.g., inferring the main idea of a passage, inferring the main idea of a paragraph, recognizing logical fallacies, recognizing various points of view).

Additional Information⎯Additional detailed information regarding the COMPASS Reading Placement Test can be found in Chapter 6 of the COMPASS Reference Manual, which is available within “Welcome” screen of the online COMPASS system Help feature. Detailed information regarding COMPASS Writing Skills technical characteristics (e.g., reliabilities, standard errors of measurement) can be found in Chapter 12 of the COMPASS Reference Manual. Sample COMPASS Reading Placement Test items are available from the ACT website at http://www.act.org/compass/sample/pdf/reading.pdf. In addition, COMPASS Placement Test Sample Item Review Sets include specifics on content and sample items for COMPASS tests. Review sets are secure materials and are available by contacting ACT Educational Services at 319/337-1054 or emailing Educational Services at [email protected].

6

COMPASS Diagnostic Tests Overview

COMPASS Writing Skills Diagnostic Tests COMPASS Writing Skills Diagnostic Tests are designed to assess students’ relative strengths and weaknesses in eight writing skills areas. When used with developmental writers, the Writing Skills Diagnostic Tests provide specific information about students’ writing skills in these eight areas with separate scores for each domain. The eight diagnostic domains are:

• Punctuation

• Verb Formation and Agreement

• Usage

• Relationships of Clauses

• Shifts in Construction

• Organization

• Spelling

• Capitalization Each domain consists of an item pool from which items are adaptively selected and presented. The Writing Skills Diagnostic Tests use a variety of item types, including Cloze, multiple choice, sentence recombination, and capitalization. All items are discrete (i.e., a single item) and consist of one or more sentences that focus on one writing skill. Additional Information⎯Additional detailed information regarding the COMPASS Writing Skills Diagnostic Tests can be found in Chapter 7 of the COMPASS Reference Manual, which is available within “Welcome” screen of the online COMPASS system Help feature. COMPASS Reading Diagnostic Tests The COMPASS Reading Diagnostic Tests evaluate students' skill levels and provide student-level feedback in three areas: Reading Comprehension Diagnostic Test—This test is designed for developmental readers and provides teachers and students with specific information about students’ reading comprehension skills. The test adaptively selects passages and items from passages across three content areas: humanities, social sciences, and natural sciences. Each passage is accompanied by four to seven test items. Two levels of reading comprehension skills are assessed: (1) locating information that is explicitly presented in text (Explicit Detail) and (2) drawing conclusions from information implicit in the passage (Implicit Information). Vocabulary Diagnostic Test—This test provides instructors and students with specific information about students’ abilities to determine the meaning of vocabulary words in context. The test is composed of one- or two-sentence items adaptively selected from the item pool. Each item has one word missing, and students are asked to select the vocabulary word that works best within the context of the sentences. The vocabulary words included in the test cover a broad range of difficulty.

7

Difficulty ranges are as follows:

• Low difficulty (grades 4 to 6)

• Moderate difficulty (grades 8 to 10)

• High difficulty (grade 12 and beyond) The Vocabulary Diagnostic Test includes words from four parts of speech:

• Noun (approximately 33 percent of the pool)

• Verb (approximately 33 percent of the pool)

• Adjective (approximately 16 percent of the pool)

• Adverb (approximately 17 percent of the pool) Reader Profile—The COMPASS Reader Profile is a survey designed to provide specific information about students’ reading habits. The survey asks from 9 to15 multiple-choice questions that focus on the following areas:

• Students’ enjoyment of reading

• Kinds of materials students typically read

• Amount of time students spend reading

• Students’ main purpose for reading

• Students’ perceptions of their reading ability The number of questions asked varies based on responses to previous items. Student responses to Reader Profile questions are supplemental and do not affect scores on any other reading test. A report can be generated that indicates student responses to each survey question. Additional Information⎯Additional detailed information regarding the COMPASS Reading Diagnostic Tests can be found in Chapter 6 of the COMPASS Reference Manual, which is available within “Welcome” screen of the online COMPASS system Help feature.

8

How to Determine Which Test(s) to Administer

Test Selection & Content Alignment Overview The Standards for Educational and Psychological Testing (AERA, NCME, APA, 1999) describes professional standards for educational assessment and test user responsibilities regarding the selection and use of tests.

• Standard 11.4 states: “The test user should have a clear rationale for the intended use of the test or evaluation procedure in terms of its validity and contribution to the assessment and decision-making process.”

• Standard 11.6 states: “Test users should verify periodically that their interpretations of test data continue to be appropriate given any significant changes in their population of test takers, their modes of test administration, and their purposes in testing.”

The above citations have been included to underscore the need for analyzing and validating the selection of course placement tests based on local needs and purpose. To this end, there are two broad categories of questions to consider and specific questions associated with these categories to answer:

1. What are you currently doing to meet your course placement needs?

• Which tests are being used for specific courses (e.g., COMPASS tests, other)?

• How were these tests selected? What were the original analyses associated with the selection of these instruments?

• How were existing cutoff scores and placement messages determined?

• Have there been changes in local curriculum that require an adjustment to the current course placement model (i.e., tests, cutoff scores, and placement messages used)?

• What information do you have regarding the success of the current course placement model (e.g., research, faculty feedback, other validity evidence)?

2. What do you want to do differently and why?

• What information do you have that contributes to concerns about the existing course placement model (e.g., institutional research data, course placement analysis, faculty feedback)?

• Should you investigate an additional test? For example, is there reason to believe that testing writing production would be beneficial for local course placement?

• What information will an additional test provide and to what degree does the content of this test align with local curriculum and instruction?

It is important to review the above when considering course placement assessment models. This analysis allows you to consider institutional goals and sets the stage for selecting tests that align with local curriculum to support these goals.

9

Align Course Content with COMPASS Placement Tests: Selection & Configuration Unlike the COMPASS Mathematics Placement Tests, the COMPASS Reading Placement Test, the COMPASS Writing Skills Placement Test, and the COMPASS e-Write components are administered from single, broad content domains. That is, rather than allowing routing across specific subtests (e.g., from Prealgebra to Algebra to College Algebra), the COMPASS English language arts tests are designed as stand-alone assessments that can be “bundled” together as an institution-specific test package. This difference in COMPASS system configuration was a deliberate decision due to the nature of English language arts course offerings at postsecondary institutions. For example, if colleges offer a reading-specific course, the ACT recommendation would be to administer the COMPASS Reading Placement Test. However, postsecondary institutions do not typically offer reading-specific courses; nevertheless, for many institutions, reading skills are an important component in evaluating performance for local English language arts course placement. In addition, emphasis on reading skills may vary depending on the level of course within a particular curriculum, with stronger emphasis on reading skills in lower-level courses. Site-specific curriculum may also vary according to emphasis on specific types of writing skills. For example, for many colleges, English language arts courses include an emphasis on both the ability to identify errors in writing and writing production. However, emphasis on these two writing skill components may also vary depending on local curriculum and course sequencing within that curriculum. A postsecondary institution may need to focus placement testing decisions using both the COMPASS Writing Skills Placement Test and COMPASS e-Write depending on course-specific emphasis. Overall, the most critical component in English language arts placement testing is based on analyzing local curriculum terms of specific performance expectations. The selection of COMPASS Placement Tests and the configuration of site-specific English language arts test packages should be based on local curriculum, course content, and performance expectations. Making test selection judgments is best done with the participation of faculty. The following steps outline a process for analyzing local course content and aligning these with COMPASS components.

10

Step 1—Describe local English language arts course content. If there are multiple courses that articulate a course sequence, describe local English language arts content as this relates to the ELA sequence. Figure 1 provides an example of an articulated ELA course sequence, with content information listed for each of the sample courses. Worksheet 1 in Appendix A provides a space for you to detail course content for your college.

Developmental English 1

Lower-level, non-credit-bearing developmental course focused on developing abilities in:

• Reading comprehension focused on basic text (literal comprehension building toward inferential reading skills)

• English language mechanics: grammar, spelling, sentence structure

• Writing: Sentence and paragraph development


Upper-level, non-credit-bearing developmental course focused on developing abilities in:

• Reading comprehension transitioning to college text (building inferential comprehension and critical reading skills)

• English language skills review (grammar, syntax, and mechanics)

• Introduction to revision and editing

• Writing: short essay and informal writing

• Introduction to the essay form (e.g., audience, purpose, thesis, supporting ideas, development, transitions, logical progression, style)

Freshman English

Introductory credit-bearing English course focused on developing abilities in:

• Analysis and response to a variety of texts

• Writing for exposition and argumentation

• Study of the essay form (e.g., audience and purpose, thesis statement and supporting ideas, rhetorical strategies, methods of development, transitions and logical progression of ideas, style)

• Summarizing, analyzing, and synthesizing information from sources

• Revision and editing strategies

• Introduction to research and building research skills

Figure 1

Sample Course Content & Sequence

11

Step 2—Review the local course content described in Worksheet 1. Based on this worksheet, you should have described the Reading and Writing course content for each local English language arts course under examination. Before we continue analyzing content alignment relative to selecting COMPASS Reading and Writing tests, a critical step is to analyze local course content horizontally and vertically. The questions that follow are intended to guide this analysis and assist colleges in determining whether there is agreement on what is being taught and how it is being taught:

• Horizontal Alignment: Are each of the courses listed in Worksheet 1 being taught the same way within the institution (within the campus, across campuses)? Is there horizontal alignment in terms of course content (e.g., course objectives, syllabus)?

• Vertical Alignment: Does the course content listed in Worksheet 1 from a lower-level

course to the next course level describe a logical sequence of content? Is there vertical alignment in terms of a lower-level course providing the foundation skills needed for success at the next course level?

A coherent system of education requires a coherent curriculum, with courses that are aligned horizontally and vertically. At the most basic level, this curriculum alignment analysis is intended to engage faculty in a curriculum mapping exercise to validate current course content by ensuring agreement on course content scope (e.g., syllabus) and sequence. Figure 2 illustrates a curriculum alignment analysis model that may be useful for colleges to use in analyzing feedback gathered in Step 1 (Worksheet 1) and reviewed in Step 2.

Figure 2 Curriculum Alignment Analysis: Feedback Loop

12

Step 3—Consider English language arts content contributions for specific courses. For this step, you need to consider:

• Course content focus as this relates to reading and writing

• Writing course content focus as this relates to specific writing strands: 1) ability to identify and correct writing errors and 2) ability to produce writing

While the writing strands listed above are closely related, they tap into different skill sets. Ability to Identify and Correct Errors—Foundation writing skills focus on procedural knowledge that aligns with the ability to identify errors in written text, skills that are related to recognizing language usage, punctuation, grammar, sentence structure, and organization errors and editing text to correct these errors. Local instructional objectives for writing likely include procedural components. Using the language of Bloom’s cognitive taxonomy (Bloom, 1956), the ability to identify and correct errors is more closely aligned with knowledge and comprehension or, using a revised version of Bloom’s taxonomy (Anderson & Krathwohl, 2001), remembering and understanding—skills that are associated with retrieving, recalling, recognizing, identifying, and classifying information. Ability to Produce Writing—Writing production focuses on demonstration. Producing writing includes procedural knowledge components, but also incorporates higher-order product generating abilities that move beyond recognizing and correcting. Local instructional objectives for essay writing likely focus on both procedural knowledge and the ability to apply procedural and higher-order thinking skills to the creation of an essay. Again, using the language of Bloom’s taxonomy, the ability to produce writing is more closely aligned with application, analysis, synthesis, and evaluation or, using the revised Bloom model, applying, analyzing, evaluating, and creating—skills that are associated with demonstrating, comparing, contrasting, arguing, defending, supporting, and developing. It is important to note that the writing strands and associated skills described above are not mutually exclusive: writing as a construct appears to be informed and improved by both “knowing” and “doing.” And since writing instruction varies by school, course, and course sequence, it is important to understand that the level of course may dictate whether instruction is more focused on procedural knowledge or on the ability to generate writing. Local faculty has the in-depth knowledge of the content (curriculum) and instructional focus to determine what is taught and how the focus on course content is distributed (e.g., amount of a quarter or semester devoted to a specific area). Using the feedback captured in Step 1, your analyses in Step 2, and the information outlined above, determine the percentage of focus for local course content. Figure 3 lists content information and percentage of content focus for each of the sample courses listed in Figure 1. Figure 4 translates the content and content percentages to a worksheet example. Use the examples provided by Figures 3 and 4 to detail course content and associated content percentages for your college. Worksheet 2 in Appendix A provides space for you to describe local course content and content focus.

13


Lower-level, non-credit-bearing developmental course focused on developing abilities in:

• Developmental English 1 Course Content Focus: Course content for sample Developmental English Course 1 in Figure 1 focused primarily on basic reading comprehension skills using lower-level texts. Writing is focused on basic language skills and mechanics. Writing production focuses on the sentence and paragraph level and does not yet approach the ability to construct an essay.

• Developmental English 1 Course Content Percentages: Approximately 40 percent of this course focuses on reading, while approximately 60 percent of this course focuses on the ability to identify errors in writing.


Upper-level, non-credit-bearing developmental course focused on developing abilities in:

• Developmental English 2 Course Content Focus: Course content for sample Developmental English Course 2 in Figure 1 transitions to reading that focuses on college text and building inferential skills. For this course, there is a review of basic language skills, with a focus on revision and editing. In terms of writing production, students are introduced to the essay form and expectations.

• Developmental English 2 Course Content Percentages: Approximately 40 percent of this course focuses on reading, while approximately 60 percent of this course focuses on the ability to identify errors in writing.

Freshman English

Introductory credit-bearing English course focused on developing abilities in:

• Freshman English Course Content Focus: Course content for sample Freshman English in Figure 1 moves toward critical reading skills across the disciplines. There is still focus on the ability to identify errors and revise written text. However, a primary focus is on writing production, working to develop additional understanding and sophistication in terms of essay development, analysis, and rhetorical strategies.

• Freshman English Course Content Percentages: Approximately 20 percent of this course focuses on reading. Approximately 20 percent of this course focuses on the ability to identify errors in writing. However, because much of this course is devoted to writing production and study of the essay form, approximately 60 percent of this course focuses on writing production.

Figure 3 Sample Course Content & Percentage of Focus: Example 1

14

Figure 4 Sample Course Content & Percentage of Focus: Example 2

15

Step 4—Determine prerequisite skills for each English language arts course. Prerequisite skills describe the concepts a student needs to understand or have mastered to be successful in a specific English language arts course. At the most basic level, literacy competence is gained through fundamental prerequisite skills. For example, prerequisite skills for reading include phonological and phonemic awareness, vocabulary development, understanding of the alphabetic principle, and print knowledge. These foundation skills are critical in the learning hierarchy of literacy. In the context of postsecondary education, prerequisite skills are typically characterized as “course prerequisites.” Prerequisites identify the previous course(s) and associated skills or body of knowledge essential for student success in the course. Prerequisite skills are those skills that closely correspond with objectives for the previous course in a sequence. Prerequisite course objectives define the content or expected body of knowledge for subsequent courses, and prerequisite course grades define the resulting performance, which, in turn, provides important information about the likelihood of student success in subsequent courses. In the case of courses at the beginning of a course sequence, “prerequisite skills” information is often gleaned from local admission requirements and associated indices (e.g., admission test score, placement test score, high school grades). However, this information may not provide the level of specificity necessary to articulate prerequisite skills or performance expectations. In these cases, articulating and validating prerequisites for course success should be defined in the context of initial performance expectations within an established scope and sequence of English language arts instruction. Step 5—Select COMPASS English language arts placement tests. Based on the analyses you completed in Steps 1 through 4, you now have in-depth, validated information regarding site-specific courses, course sequencing, horizontal and vertical content alignment, and the content focus percentages associated with each core English language arts course offered locally. You can use this course content information to select COMPASS English language arts placement tests. Table 1 outlines the selection of COMPASS Placement Tests based on the sample curriculum and content focus illustrated in Figures 3 and 4.

16

Table 1 Sample Selection & Configuration of COMPASS Placement Tests

Sample Course Recommended COMPASS Tests

Model 1: Stand-alone Tests &

Discrete Scores *

Model 2: Combined Tests &

Multi-measure Composite Score *


(including prerequisite skills and course objectives)

Use

COMPASS Reading and

COMPASS Writing Skills

If used as stand-alone tests, the placement decision model would include weighting the COMPASS Writing Skills score more than the COMPASS Reading score (i.e., 60 percent and 40 percent).

If used as a single test package and set up as multiple measures within the system, weight COMPASS Reading at 40 percent and COMPASS Writing Skills at 60 percent.



Use COMPASS Reading


COMPASS e-Write

If used as stand-alone tests, the placement decision model would include weighting the COMPASS Writing Skills and the COMPASS Reading scores equally and using COMPASS e-Write as an additional measure.

If used as a single test package and set up as multiple measures within the system, weight COMPASS Reading at 40 percent, COMPASS Writing Skills at 40 percent, and COMPASS e-Write at 20 percent.

Freshman English


Use COMPASS Reading


COMPASS e-Write

If used as stand-alone tests, the placement decision model would include weighting COMPASS e-Write most heavily and using COMPASS Writing Skills and the COMPASS Reading scores equally as additional measures.

If used as a single test package and set up as multiple measures within the system, weight COMPASS Reading at 20 percent, COMPASS Writing Skills at 20 percent, and COMPASS e-Write at 60 percent.

* Weights should be based on the relative importance of prerequisite skills. Use the site-specific information you developed using Worksheet 2 to guide the COMPASS English language arts test selection and configuration for your local courses.

17

Align Course Content with COMPASS Diagnostic Tests: Selection & Configuration The ACT National Curriculum Survey® for 2005–2006 includes feedback from just over 7,100 educators in English/writing; approximately 3,400 of these were educators who taught postsecondary English at developmental and credit-bearing course levels. Analyses of survey results reveal differences between secondary and postsecondary judgments of what is most important for success in English and writing. Results suggest that postsecondary instructors are more concerned with the fundamental elements of writing than are high school teachers. Commentary provided by postsecondary instructors focused on the importance of mechanics at the postsecondary level. Postsecondary instructors reported having to reteach basic elements of writing before they could move on to the critical thinking/reading components required for their courses. Given this survey feedback, it is apparent that postsecondary faculty and staff require additional tools to assist in evaluating English language arts knowledge and skills and targeting specific areas of instructional need. The COMPASS Reading and Writing Skills Diagnostic Tests offer postsecondary users with varying levels and types of assessment that can be configured to meet local needs relative to a continuum of student readiness. In the previous section, we examined how to analyze local courses to select and configure COMPASS English language arts placement tests. However, there may also be cases where you have determined that additional diagnostic information would be helpful. The most commonly occurring reason for using diagnostic tests is to inform instruction either at the student level or the course level. In general, COMPASS Reading and Writing Skills Diagnostic Tests are used with students receiving a score below a user-defined cutoff on the Placement Tests. These students may be routed to one or more of the Diagnostic Tests for additional assessment of specific areas, providing a powerful developmental-level assessment that can impact both individual learning and instructional efficacy. General suggestions regarding the use of COMPASS Diagnostic Test scores include developing messages to students for each diagnostic test. These messages would include references to specific units or chapters of materials (e.g., PLATO) to help develop needed skills, along with suggested steps or strategies, resource location information, web page information, costs, etc., for accessing local resources. Diagnosing Students—At the student level, diagnostic testing is typically used to identify the instructional intervention necessary to support student learning and success. This instructional intervention may include targeted brush-up instruction, special courses, tutoring, and other one-on-one services that are available locally. The COMPASS Reading and Writing Skills Diagnostic Tests are designed to be used at lower developmental course levels to assess students’ relative strengths and weaknesses in specific skill areas. Diagnosing Courses—For courses, diagnostic testing may be used to identify specific instructional changes necessary to support local curriculum. For example, if your college is experiencing ongoing and significant student retention problems or specific disconnects within a course sequence, using COMPASS Reading or Writing Skills Diagnostic Tests allows you to review results for a class or a course to better understand whether there is an instructional delivery disconnect. One approach for this use of the COMPASS Reading and Writing Skills Diagnostic Tests would be to use these as both pre-test and post-test components to examine learning.

18

Section 3: COMPASS Placement Tests Setup & Routing Rules

With COMPASS, you can specify which content areas are to be included in a specific test package. You can specify “routing rules” to guide the presentation of various COMPASS English language arts components based on local curriculum requirements and student performance. The COMPASS software comes preloaded with standard test packages, but you can also build your own. For example, based on local curriculum, you can build a test package that includes the COMPASS Reading Placement Test, the COMPASS Writing Skills Placement Test, and a COMPASS e-Write component. This flexibility helps ensure that your COMPASS tests are appropriate to the English language arts courses at your institution. COMPASS Reading Routing Rules The COMPASS Reading Placement Test uses five passage types: prose fiction, humanities, social sciences, natural sciences, and practical reading. The test administrator is able to indicate which of these passage types will be administered, as well as the order in which they will be presented. In the COMPASS Reading Placement Test, test depth and precision are also controlled in by specifying test length. There are two options for test length in the COMPASS Reading Placement Test: standard and maximum. Average test depth and precision increase when the test length is set at maximum. If examinees score poorly on the COMPASS Reading Placement Test, they can be optionally routed to Reading Diagnostic Tests. The Reading Diagnostic Tests do not affect a student’s COMPASS Reading Placement Test score. Rather, the diagnostic tests are intended to provide additional student-level performance information that can be used to evaluate instructional needs. Students who take these tests have a separate statement appended to their placement scores indicating their performance on the diagnostics. The COMPASS Reading Comprehension Diagnostic Test and the Vocabulary Diagnostic Test provide estimates of proficiency based on items specifically aligned with these subskills. Test depth and precision for these diagnostic tests are controlled in the same way as for the COMPASS Reading Placement Test (i.e., by setting test length). The Reading Comprehension Diagnostic Test can be set at either standard or maximum test length, and the Vocabulary Diagnostic test can be set at standard, extended, or maximum test length. Examinees can be routed from the Reading Placement Test into one or more of the three subtests: Reading Comprehension Diagnostic, Vocabulary Diagnostic, and the COMPASS Reader Profile. Figure 5 shows the setup for routing from the COMPASS Reading Placement Test to the COMPASS Reading Diagnostic Tests. Figure 6 shows a sample routing table for the Reading Diagnostic Tests.

19

Figure 5 Sample Routing from COMPASS Reading Placement to

COMPASS Reading Diagnostic Tests

Figure 6 Sample Routing Table for COMPASS Reading Diagnostic Tests

20

COMPASS Writing Skills & e-Write Routing Rules The COMPASS/ESL system has two components related to native English speaker writing skills: COMPASS Writing Skills and direct writing assessments, which include COMPASS e-Write 2–8 or 2–12. The COMPASS Writing Skills Placement Test is designed to give an efficient assessment of specific writing skills to help determine students’ readiness for entry-level college English courses. Students who perform poorly on the test would likely be placed into a developmental English or writing course; students who perform well would be placed into the standard entry-level course or higher. Test depth and precision are controlled in the COMPASS Writing Skills Placement Test by specifying test length, either standard or maximum. For the COMPASS Writing Skills Placement test, setting test length determines the number of essays (plus their associated items) that will be presented to the student. COMPASS e-Write complements the COMPASS Writing Skills Test by providing additional information about students’ direct writing abilities, including the ability to formulate a position on specified issue, support that position, focus on an audience, develop an argument, and express ideas using clear, effective language. In addition to the placement tests, there are eight COMPASS Writing Skills diagnostic domains intended for use with English-speaking students who do not score at or above the cutoff score for placement into a standard entry-level English course. Users may select which of the diagnostic tests to administer. All COMPASS Writing Skills Diagnostic Tests provide estimates of proficiency within a specific writing domain; test depth and test precision for the diagnostic domains are controlled by setting test length. Examinees can be routed to the COMPASS Writing Skills Diagnostic Tests from the Writing Skills Placement COMPASS Writing Skills depending on the student’s score in the Writing Skills Placement test. Typically, low-ability examinees would be routed to one or more of the eight diagnostics. Figure 7 shows the setup for routing from the COMPASS Writing Skills Placement Test to the COMPASS Writing Skills Diagnostic Tests. Figure 8 shows a sample routing table for the Writing Skills Diagnostic Tests. Figure 7 also illustrates the setup for routing from COMPASS Writing Skills into one of the two COMPASS e-Write tests. The COMPASS system allows the user to combine both the multiple-choice Writing Skills measure and a COMPASS e-Write direct writing component into one seamless testing session. Please note that while Figure 7 illustrates routing to both the diagnostic tests and e-Write, it is possible to route to only one of these tests. In addition, Figure 9 illustrates the setup for routing from COMPASS e-Write to the COMPASS Writing Skills Placement Test.

21

Figure 7

Sample Routing from COMPASS Writing Skills Placement to COMPASS Writing Skills Diagnostic Tests & COMPASS e-Write

Figure 8

Sample Routing Table for the COMPASS Writing Skills Diagnostics

22

Figure 9

Sample Routing from COMPASS Writing Skills Placement to COMPASS Writing Skills Diagnostic Tests

23

Section 4: COMPASS Placement Tests & Multiple Measures

COMPASS offers various types of assessments for English language arts course placement, including the COMPASS Reading Skills Placement Test (focusing on reading comprehension skills), the COMPASS Writing Skills Placement Test (focusing on the ability to identify errors in writing), and COMPASS e-Write 2–8 or 2–12 (focusing on writing production). All of these skill sets may be important components in postsecondary English curriculum and instruction, depending on your local needs. Because you need to be able to differentiate performance for placement purposes based on specific local content requirements, one answer is the use of multiple measures for English language arts assessment. For COMPASS users that are attempting to place students into multiple English language arts levels (e.g., a sequence of developmental and credit-bearing courses), the use of multiple measures is critical to achieving the level of precision necessary. Best Practice Focus on Multiple Measures for Writing Courses

Some postsecondary institutions offer courses specifically focused on writing instruction. In these cases, users are particularly interested in evaluating performance on writing-related tests. ACT offers COMPASS Writing Skills and COMPASS e-Write assessments to measure writing ability for local placement purposes. The COMPASS e-Write direct writing assessments were developed as complementary tests to be used in conjunction with the COMPASS Writing Skills multiple-choice test. COMPASS e-Write components were developed because many postsecondary writing and English language arts programs consider writing production to be a cornerstone of curriculum and instruction, especially at the credit-bearing level. Therefore, the assessment of writing production can be an important component in differentiating ability for ELA course placement. However, the use of only a direct writing assessment may not yield sufficient information for placement decisions. An ACT best practice recommendation for writing course placement is to use COMPASS e-Write components as companion tests with the COMPASS Writing Skills Placement Test to assess both an examinee’s ability to identify errors in writing and produce writing. Using COMPASS e-Write in conjunction with COMPASS Writing Skills allows a greater level of precision for local placement decisions. The complementary nature of the two tests can offset the clustering effects associated with a direct writing assessment and offer additional writing performance differentiation, For COMPASS users that are attempting to place students into multiple course levels that begin with a sequence of developmental courses and move to credit-bearing courses, the use of multiple measures is critical to achieving the type of writing skills information and level of precision necessary. As a best practice for writing course placement, ACT recommends starting examinees in COMPASS Writing Skills and routing examinees to COMPASS e-Write (either 2–8 or 2–12) as companion assessments such that both assessments are administered in one sitting.

24

The use of multiple measures for writing assessment may be accomplished by using COMPASS Writing Skills and COMPASS e-Write scores as “stand-alone” scores (e.g., two discrete measures). This means that placement decisions are based on two independent scores. However, it is also possible to use the multiple-measures feature within the COMPASS system to combine two tests into a single, composite score. The steps in setting up the multiple-measures feature are described in the COMPASS Internet Version “Help” interface. If users choose to use the multiple-measures feature available within the COMPASS system, ACT recommends using COMPASS Writing Skills as the primary measure and COMPASS e-Write as the secondary measure. In the COMPASS system multiple-measures setup, the primary measure is the determiner of the composite score scale. Using COMPASS Writing Skills as the primary measure and COMPASS e-Write as the secondary measure means the direct writing score would be automatically transformed to the multiple-choice score scale so that the final multiple-measures composite score scale would extend to 99 points. This multiple-measures composite score approach for English language arts course placement allows for a greater number of sore points to be used in making placement decisions. Please note that if you use the COMPASS multiple-measures feature, the Standard Individual Report (SIR) will include 1) a COMPASS Writing Skills score, 2) a COMPASS e-Write score, and 3) a multiple-measures composite score for both the multiple-choice test and the direct writing assessment. Also note that the multiple-measures feature allows users to weight measures based on site-specific content needs (e.g., more focus on the ability to identify errors in writing or more focus on writing production). Figure 10 illustrates the multiple-measures setup interface in the COMPASS system. In this example, only COMPASS Writing Skills and COMPASS e-Write have been used and each test is equally weighted.

Figure 10 Writing Multiple Measures Setup

The strength of these COMPASS multiple-measure models for writing is in combining different assessment approaches to better evaluate and discern performance; multiple measures provide

25

better discrimination in performance. Combined assessment approaches for writing also allows for more in-depth prescriptive information than would otherwise be available for assessing individual strengths and weaknesses. Best Practice Focus on Multiple Measures for English Language Arts Courses

As outlined previously, ACT recommends using a combination of tests that best align with local courses as a best practice for English language arts placement. Some postsecondary institutions offer English language arts courses that focus on both reading and writing instruction. In these cases, users may choose to use COMPASS Reading, COMPASS Writing Skills, and COMPASS e-Write to support effective ELA course placement. The use of multiple measures for ELA course placement may be accomplished by using COMPASS Reading, Writing Skills, and e-Write scores as either “stand-alone” scores (e.g., three discrete measures) where placement decisions are based on three independent scores. However, it is also possible to use the multiple-measures feature within the COMPASS system to combine three tests into a composite score. These two models for multiple measures testing were outlined in Table 1 in Section 2. Using the multiple-measures capability, it is possible to start examinees in COMPASS Reading, route them into COMPASS Writing Skills, and then route them into COMPASS e-Write such that three assessments are administered in one sitting. Scores from the selected tests can be weighted to reflect prerequisite skills (see Table 1). The steps in setting up the multiple-measures feature are described in the COMPASS Internet Version “Help” interface. If users choose to use the multiple-measures feature available within the COMPASS system, ACT recommends using one of the multiple-choice tests as the primary measure such that any COMPASS e-Write score would be automatically transformed to the multiple-choice score scale. Using this approach, the final multiple-measures composite score scale would extend to 99 points, allowing a greater number of sore points to be used in making placement decisions. Please note that if you use the COMPASS multiple-measures feature with three discrete COMPASS tests, the Standard Individual Report (SIR) will include 1) a COMPASS Reading score, 2) a COMPASS Writing Skills score, 3) a COMPASS e-Write score, and 4) a multiple-measures composite score for the combined assessments. Also note that the multiple-measures feature allows users to weight measures based on site-specific content needs. Figures 11 and 12 illustrate the multiple measures feature available in the COMPASS system. Figure 11 shows a test package that includes COMPASS Reading and COMPASS Writing Skills, with routing from COMPASS Writing Skills into COMPASS e-Write. Figure 12 shows the detail of the multiple measures setup for these combined tests.

26

Figure 11 Sample Multiple Measures Setup: Combining Tests

Figure 12 Sample Multiple Measures Setup: Weighting Tests

27

Section 5: COMPASS Placement Test Cutoff Scores

Setting Cutoff Scores for COMPASS Placement Tests

The COMPASS system can quickly and accurately assess students’ skill levels in reading and writing. Once a student completes a COMPASS test, the system immediately provides student-level results in the form of a Student Advising Report. This report includes the student’s score on each test area completed and a course placement recommendation based on these scores. Typically course placement messages inform students which English course he or she should take and how to register for it. The key to helping students achieve academic success is to use their COMPASS scores to place them in the most appropriate English courses.

Institutions often make placement decisions on the basis of cutoff scores. A cutoff score for a particular course is the minimum score a student needs to be adequately prepared to succeed in the course. ACT refers to the initial cutoff scores as “Stage 1” cutoff scores. Stage 1 cutoff scores may be based on national COMPASS data or local content alignment (i.e., local course content expectations) and local data (i.e., a norm-referenced alignment of score distributions). ACT recommends that you evaluate the effectiveness of your Stage 1 cutoff scores after you have been able to collect data on students’ success in particular courses, and use this information to establish more refined “Stage 2” cutoff scores.

Set Stage 1 Cutoff Scores Using National Data To help you get started, ACT has developed preliminary cutoff scores for COMPASS Reading, Writing Skills, and e-Write using national data. Table 2 includes suggested Stage 1 cutoff scores based on national data; users may also use COMPASS national norms to set cutoff scores. The COMPASS system comes pre-loaded with suggested Stage 1 cutoff scores for COMPASS Reading and Writing Skills. For e-Write 2-8 and e-Write 2-12, the cut score values included in the system are for illustration purposes only and are not considered to be "Stage 1" recommended values. Based on recent research, ACT recommends the insertion of the suggested Stage 1 cut scores shown for e-Write 2-8 an d e-Write 2-12 in Table 2 below.

Table 2 Suggested COMPASS Stage 1 Cutoff Scores Based on National Data

COMPASS Scores Course Recommendations

Reading

0–60 Reading Development 1

61–80 Reading Development 2

81–100 No Reading Required

Writing Skills

0–37 Developmental English 1


70–100 Freshman English

COMPASS e-Write 2–8


5 Developmental English 2


COMPASS e-Write 2–12




28

Set Stage 1 Cutoff Scores Using Local Content Alignment & Local Data While setting Stage 1 cutoff scores using national data is an efficient and effective way to set preliminary cutoff scores, cutoff scores based on national data may or may not be appropriate for your institution. For example, national COMPASS cutoff scores have been represented based on three levels of English language arts courses. If your site has more or fewer courses, the national model may not align with your needs. Likewise, if your student population differs markedly from the national COMPASS sample, you may also see differences that are directly related to the local testing population. You may order a site-specific COMPASS Entering Student Descriptive Report (ESDR) from ACT to aid you in determining whether the characteristics of students at your college align with national data. The ESDR describes the characteristics, needs, plans, and skills of students at your college. These data can be compared with national COMPASS characteristics. The ESDR also includes options for creating subgroup reports for groups of students of special interest. In order for you to use local COMPASS Placement Test results to set Stage 1 cutoff scores, it will be necessary for your college to administer the selected COMPASS tests to a reasonable sample of students for each English language arts course. It is important that the sample selected represents each of the courses for which you wish to set cutoff scores; it is also important that the sample size be sufficient to yield meaningful results. ACT recommends a sample of at least two classes (e.g., 40 students) for each course. Overall, ACT recommends that you evaluate the approaches to setting cutoff scores to determine the best method for your institution. Setting English language arts course placement cutoff scores requires monitoring and adjusting based on students' performance. Ideally this should be done in conjunction with trained researchers in your institution's Institutional Research department. However, ACT has developed worksheets that outline approaches for setting local placement cutoff scores. Appendix B provides detailed guidelines for analyzing local course content, aligning courses with COMPASS Reading, Writing Skills, and e-Write tests and test results, and developing cutoff scores and placement messages. Appendix B also contains worksheets to assist you in these important cutoff score analyses and deliberations. Evaluate & Adjust Stage 1: Stage 2 Cutoff Scores ACT recommends that you evaluate the effectiveness of your Stage 1 cutoff scores after you have been able to collect data on students’ success in particular courses, and use this information to establish more refined “Stage 2” cutoff scores. If Stage 1 cutoff scores were set locally based on national COMPASS data, this Stage 2 adjustment can be accomplished by using data collected during a “Stage 1 piloting” phase of implementation. The analyses to provide the information needed for this Stage 2 cut score adjustment process may be accomplished by local college staff or they may be completed through use of the Course Placement Service available from ACT. The “success rate” for a given course is the percentage of students placed into that course who received a grade of C or higher. If the success rate for a particular course is higher or lower than desired, you may consider adjusting the cutoff score accordingly. For example, if your department targeted a 60 percent success rate for an introductory English course (i.e., 60 percent of enrolled students receive a C or higher grade), but the observed success rate was 48 percent, you may

29

want to either raise the cutoff score or strongly recommend specific review or tutoring services to students at or slightly above the cut score being used. A follow-up study of the student success rate under the new cutoff score would be highly recommended. Cutoff Scores for COMPASS Diagnostic Tests

Cutoff scores for COMPASS Reading or Writing Diagnostic Tests may include scores for the three ability levels, with associated messages regarding local resources or strategies for study. Diagnostic test cutoff scores should be determined locally; however, the following suggestions may be helpful as possible starting points:

• Low (0–40): Fully work through the related units/chapters; use the high school or college learning center tutoring services

• Mid-range (41–65/75): Take chapter quiz in the related units/chapters; work through sections needing review; use the high school or college learning center tutoring services

• Upper-range (65/75–100): Review materials or go on to next area needing additional work Your college may consider offering or requiring retesting to confirm that material has been mastered.

30

Section 6: Effective Retention in English Language Arts

Placing students in the proper courses is half the battle. The other half is ensuring that they are aware of and have access to the academic and advising resources available on your campus. COMPASS can help in this regard as well. Connecting Students to the Campus Resources

In addition to advising students what courses they should take and how to register, many English departments use the course placement messages to connect students to the campus resources they need to improve their chances for success. These messages can be customized based on students’ scores. If a student’s COMPASS score falls just above a certain cutoff score (perhaps 5 to 10 score points), that student will be one of the weaker students in the course and at higher risk of failure. Messages tailored to this type of student’s needs may include information about tutorial services, the location and hours of operation of the campus language lab, online courseware and resources, etc. Better Advising through COMPASS Demographics: Survey Options

COMPASS can provide academic advisors with much more information than a single test score. The demographic section of COMPASS includes pre-formed and locally developed demographic items regarding the students’ English language arts background and needs. This information can be used to advise individual students more effectively and to evaluate and enhance advising programs through COMPASS research reports. Pre-formed Items⎯The items include individual student responses that are reported in the COMPASS Student Advising Report. These items are also summarized at the campus, state, and national level in the Entering Student Descriptive Report and the Returning Student Retention Report. Items particularly relevant to English include the following:

Item 8 Type of High School Certificate (includes name of high school attended and year of graduation)

Item 11 Overall High School Grade Point Average Item 12 Courses Completed and Grades Earned Item 14 Career Goal Item 16 Educational Program or Major Item 22 Would Like Help with Study Skills and English Skills

31

Local Items⎯You may develop up to 40 local items at no additional cost. Individual student responses are reported on the COMPASS Student Advising Report and summarized at the campus level in the Entering Student Descriptive Report and Returning Student Retention Report. Due to the local nature of the items, no national response data is available. Suggestions for items related to English include the following:

Local Item 1: What is the last English course you completed during high school? [Use local course names that area students will connect accurately to their high school English courses.]

A. Introductory English B. Intermediate English C. Advanced Placement English D. Not Sure

Local Item 2: What is the last English course you completed after high school?

[Use local course names that area students will connect accurately to their high school English courses.]

A. Introductory English B. Intermediate English C. Advanced English D. Have not taken English courses after high school

Local Item 3: What is the final grade (approximately) you received in the last English

course you completed?

A. A, A+, A- B. B, B+, B- C. C, C+, C- D. D+ or lower E. Other or Not Sure

Local Item 4: How long ago did you complete your last English course?

A. One year ago or less B. One to two years ago C. Two to five years ago D. Five or more years ago

Local Item 5: For the areas that you have studied, how would you rate your English skills at

this time?

A. Fairly strong, ready to go on in next course B. Somewhat rusty, but could rebuild with help C. Fairly weak, need help in rebuilding skills for next course D. Not sure

32

Local Item 6: If you were to participate in an English refresher experience, what type of approach would you prefer most?

A. Work with an English tutor at the college at my own speed B. Work with computer software to review English skills at my own speed C. Take a complete course in a regular college classroom with a group of

other students working on the same skills D. Some other approach E. Not sure

Local Item 7: CCC is considering the development and delivery of an “English study skills”

seminar for students (2 or 3 sessions of 2 hours each; no tuition charge, text book costs of $xx). Would you be interested in participating?

33

Section 7: Improving Student Performance in English Language Arts

The following suggestions may help create an overall context at your institution that is more conducive to successful course placement and retention in English language arts courses.

• Deliver all new student assessment services within a “success planning” approach as an expression of the mission and service orientation of the college. As part of the invitation to participate in the Success Planning Services for New Students, provide new students with a leaflet introducing the student to the “Success Planning Services” of the college, including the testing process and services. Include examples of COMPASS test items (available on the ACT website at http://www.act.org/compass/sample/index.html) in the leaflet, with references to the English advising and instructional support services provided by your college.

• In all communications to potential new students, actively recommend early involvement

in the admissions and assessment process to allow more time for best advising and support services. Also recommend more time for involvement in “skill brush-up” activities followed by retesting, with the aim of moving “rusty” students one course upward in the placement ladder when possible.

• For students whose COMPASS scores fall somewhat below the cutoff score selected by

the college as necessary to enter the next course (perhaps 8–10 score points), include a message on the COMPASS Student Score Report that notes this fact and describes what the student can do by way of skills “brush-up” followed by retesting, with the student’s new score level being used to make a revised course placement recommendation.

• Consider the use of the COMPASS Reading or Writing Skills Diagnostic Tests to pin-

point areas of weakness and instructional needs. Link the results of the diagnostics to the development of a study plan for the student (e.g., include references to local resources, such as PLATO and where they are available at your college).

• Consider delivering “pre-planning assessment” services to juniors in targeted area

feeder high schools, with the objective of maximizing the quality of career and educational planning and related course planning by students as they plan for their senior year of high school. Also, consider similar adult outreach services, perhaps in a seminar approach at area businesses that include assessment and advising services. These approaches can deliver highly supportive student development information and experiences for the participants, with positive community service and student recruitment benefits for the college.

34

Section 8: English Language Arts Program Evaluation with COMPASS Research Reports

COMPASS offers a rich variety of research reports that you can use to evaluate and improve your mathematics course placement and advising. • Entering Student Descriptive Report⎯This report describes the characteristics, needs,

plans, and skills of students who initiate the "becoming a student" process. The report includes options for creating subgroup reports for groups of students of special interest.

• Returning Student Retention Report⎯This report identifies the retention patterns occurring

at the institution and identifies the student characteristics that are related to those retention patterns. This report includes options for creating subgroup reports.

• Course Placement Service⎯ This ACT services describes and evaluates the outcomes of

course placement practices on campus, providing information to assist in the setting or fine-tuning of placement cutoff scores. This service also provides an option for developing placement cutoff score linkages between pairs of placement measures from different tests (COMPASS/ACT, COMPASS/ASSET, COMPASS/local instruments).

• Under-prepared Student Follow-up Report⎯This report identifies the success of under-

prepared students in standard courses following their participation in preparation or development courses.

For more information on the COMPASS Reading, COMPASS Writings Skills, and COMPASS e-Write Tests, please visit our website at www.act.org/compass/ or contact your regional COMPASS representative, as indicated on the back cover of this document.

35

References

Ackerman, T.A. and Smith, P.L. (1988). A comparison of the information provided by essay, multiple-choice, and free-response writing tests. Applied Psychological Measurement, Volume 12, 117–128; http://apm.sagepub.com/cgi/content/abstract/12/2/117 ACT National Curriculum Survey® 2005–2006; http://www.act.org/research/curricsurvey.html. American Psychological Association. (1999). Standards for Educational and Psychological Testing. Washington, DC: The committee to develop Standards of the American Research Association, American Psychological Association, National Council on Measurement in Education. Anderson, L.W., and Krathwohl (Eds.). (2001). A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives. New York: Longman. Behar, Linda S. (1994). An empirical analysis of curriculum domains: implications for program development and evaluation. Peabody Journal of Education, Volume 69, No. 4, Our Evolving Curriculum, Part 2, 100–112; http://www.jstor.org/pss/1492695 Bloom, Benjamin S. and Krathwohl, David R. (1956). Taxonomy of educational objectives: The classification of educational goals, by a committee of college and university examiners. Handbook 1: Cognitive domain. New York, Longmans. Jonassen, David H.; Tessmer, Martin; and Hannum, Wallace H. (1999). Task Analysis Methods for Instructional Design. Philadelphia: Lawrence Erlbaum Associates. Krathwohl, D.R. (2002). A revision of bloom's taxonomy: An overview. Theory into Practice, 41 (4), 212–218. Sawyer, R. and Schiel, J. (2000). Posttesting students to assess the effectiveness of remedial instruction in college. ACT Research Report: http://www.act.org/research/researchers/reports/

36

Appendix A: How to Select Which Test(s) to Use

Worksheet 1

Course Content & Sequence for _______________________________________ (insert name of college)

Course 1: ________________________________________________ (insert name/number of course)

Select One: Developmental Course 1 Credit-bearing Course 1

Developmental Course 2 Credit-bearing Course 2

Reading Focus: ___________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ Writing Focus: ____________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________





37

Worksheet 1 (continued)

Course Content & Sequence for _______________________________________ (insert name of college)









38

Worksheet 2

Course Content & Percentage for _______________________________________ (insert name of college)

39

Appendix B: Setting COMPASS Placement Test Stage 1 Cutoff Scores

Setting cutoff scores using local content alignment and COMPASS data requires specific steps for success: Step 1—Align Course Expectations with COMPASS Tests

When setting cutoff scores and developing associated COMPASS placement messages, it is extremely important that English language arts instructors are involved in analyzing each course in terms of performance expectations. Faculty familiar with students and course-specific content need to describe what students should know and be able to do in terms of entry-level ability or minimum competencies. Faculty needs to align student proficiency levels, as indicated by their COMPASS scores, with the skill levels required for entering the various courses in your English curriculum. An important question to be answered as part of this analysis is:

What are the entry-level performance expectations for this course and how do these expectations align with COMPASS Reading content, COMPASS Writing Skills content, and/or COMPASS e-Write proficiency descriptions?

The steps completed using Worksheets 1 and 2 in the context of “How to Select Which Test(s) to Use” will assist in this local content alignment analysis. However, you may need to revisit COMPASS content information outlined in this guide (i.e., Section 2). Additional in-depth content information is available in the following resources:

• COMPASS Internet Version Reference Manual⎯Includes specifics on content for all COMPASS tests. This is available as part of the “Help” feature within the COMPASS test administration interface. This is also available at http://www.act.org/compass/resources.html.

• COMPASS Placement Test Sample Item Review Sets⎯Includes specifics on

content and sample items for all COMPASS tests. The review sets are secure materials and are available by contacting ACT Educational Services at 319/337-1054 or emailing Educational Services at [email protected].

• Frequently Asked Questions about COMPASS e-Write & ESL e-Write⎯ Includes

specifics on COMPASS e-Write tests, with detailed information regarding COMPASS e-Write 2–8 and 2–12 holistic scoring rubrics. This document is available through an ACT regional office: http://www.act.org/contacts/field.html.

• COMPASS e-Write 2–12 Sample Essay Responses⎯ Includes specifics on

COMPASS e-Write, with detailed information regarding the COMPASS e-Write 2–12 holistic scoring rubric and sample essays with scoring explanations. This document is available through an ACT regional office: http://www.act.org/contacts/field.html.

Worksheet 3 provides space to revisit the course content alignment previously described (i.e., Worksheets 1 and 2) and list the COMPASS Placement Tests associated with specific English language arts courses. Please note that the content described in Worksheet 3 should articulate course scope and sequence.

40

Worksheet 3

Course Content Alignment with COMPASS Tests for

_______________________________________ (insert name of college)

English Language Arts Course Name, Number & Course Description

COMPASS Placement Test(s) Aligned with Course

(see Worksheets 1 & 2)

Course Name: _______________________________ / #: ______ Reading Focus (Scope): _________________________________

_____________________________________________________

_____________________________________________________

_____________________________________________________

Writing Focus (Scope): __________________________________

_____________________________________________________

_____________________________________________________

_____________________________________________________


_____________________________________________________

_____________________________________________________

_____________________________________________________


_____________________________________________________

_____________________________________________________

_____________________________________________________

41

Worksheet 3 (continued)

Course Content Alignment with COMPASS Tests for


English Language Arts Course Name, Number & Course Description

COMPASS Placement Test(s) Aligned with Course

(see Worksheets 1 & 2)


_____________________________________________________

_____________________________________________________

_____________________________________________________


_____________________________________________________

_____________________________________________________

_____________________________________________________


_____________________________________________________

_____________________________________________________

_____________________________________________________


_____________________________________________________

_____________________________________________________

_____________________________________________________

42

Step 2—Review Course Alignment with COMPASS Tests & Local Test Results

Review course-specific expectations outlined in Worksheet 3 in conjunction with local data (e.g., score frequency distributions) to inform the selection of cutoff scores. Use Worksheets 4A and 4B to review local COMPASS Reading and local COMPASS Writing Skills data for your college and compare these data to the national data listed. Use the cumulative percentages listed in the first column to guide the selection of local cutoff scores. Local COMPASS Reading and Writing Skills information is available from ACT as part of a local Entering Student Descriptive Report (ESDR).

Worksheet 4A

COMPASS Reading Score Distributions for


Cumulative %

National COMPASS

Reading Score at or below

Local COMPASS Reading Score

at or below (ESDR)

Course Number Course Name

5% 46

10% 51

15% 63

20% 67

25% 70

30% 74

35% 76

40% 79

45% 80

50% 82

55% 84

60% 86

65% 87

70% 89

75% 91

80% 92

85% 94

90% 96

95% 99

100% 99

43

Worksheet 4B

COMPASS Writing Skills Score Distributions for


Cumulative %

National COMPASS Writing

Skills Score at or below

Local COMPASS Writing Skills

Score at or below (ESDR)


5% 10

10% 18

15% 26

20% 35

25% 42

30% 51

35% 56

40% 60

45% 65

50% 70

55% 74

60% 79

65% 82

70% 87

75% 89

80% 93

85% 94

90% 97

95% 99

100% 99

44

Worksheets 4C & 4D

COMPASS e-Write 2–8 Proficiency Level & Course Alignment for


ACT has developed detailed proficiency descriptors to define the skills a student at each proficiency level is likely to demonstrate for COMPASS e-Write 2–8. These worksheets are designed to help you align e-Write proficiency levels with English language arts courses at your institution. Again, the question to keep in mind as you complete the worksheets that follow is:

What are the entry-level performance or proficiency expectations for this course and how do these expectations align with the COMPASS e-Write 2–8 or the proficiency descriptions listed?

• Use the information captured in Worksheet 3 to evaluate and record proficiency

alignment decisions.

• Please note that proficiency levels are cumulative, with each level assuming the inclusion of material at previous levels. Please reference local English language arts course descriptions to fill in the COMPASS e-Write worksheet appropriate for your college and align your courses with the appropriate COMPASS e-Write holistic scores.

• Use Worksheet 4C to record local course alignment with COMPASS e-Write 2–8 proficiency description.

• Use Worksheet 4D to review local COMPASS e-Write 2–8 data for your college and compare these data to the national data listed. Use the cumulative percentages listed in the first column to guide the selection of local cutoff scores. Local COMPASS e-Write 2–8 information is available from ACT as part of a local Entering Student Descriptive Report (ESDR).

45

Worksheet 4C



COMPASS e-Write Score

Ranges

COMPASS e-Write 2–8 Holistic Score

Proficiency Descriptions

ELA Course Number

English Language Arts Course Name

1–2

The response shows an inadequately developed sense of purpose, audience, and situation. Although the writer attempts to address the topic defined in the prompt, the response displays more than one of the following significant problems: Much of the style and language may be inappropriate for the occasion; focus may be unclear or unsustained; support is very minimal; sentences may be poorly constructed; word choice may be imprecise; or there may be many errors in usage and mechanics.

3–4 The response shows a partially developed sense of purpose, audience, and situation. The writer takes a position on the issue defined in the prompt and attempts to support that position, but there may be little elaboration or explanation. Focus may be unclear and not entirely sustained. Some effort to organize and sequence ideas is apparent, but organization may lack coherence. A limited control of language is apparent: word choice may be imprecise; sentences may be poorly constructed or confusing; and there may be many errors in usage and mechanics.

46

Worksheet 4C (continued)




Ranges



ELA Course Number


5–6 The response shows a developed sense of purpose, audience, and situation. The writer takes a position on the issue defined in the prompt with some elaboration or explanation. Focus is clear and generally maintained. Organization is generally clear. A competency with language is apparent: word choice and sentences are generally clear though there may be some errors in sentence structure, usage, and mechanics.

7–8 The response shows a thoughtful and well-developed sense of purpose, audience, and situation. The writer takes a position on the issue defined in the prompt with well-developed elaboration or explanation. Focus is clear and consistently maintained. Organization is unified and coherent. Good command of the language is apparent: word choice is precise; sentences are well structured and varied; and there are few errors in usage and mechanics.

47

Worksheet 4D

COMPASS e-Write 2–8 Score Distributions for


Cumulative %

National COMPASS e-Write

2–8 Score at or below

Local COMPASS e-Write 2–8 Score

at or below (ESDR)


1% 2

3% 3

14% 4

54% 5

94% 6

100% 7

100% 8

48

Worksheets 4E & 4F



ACT has developed detailed proficiency descriptors to define the skills a student at each proficiency level is likely to demonstrate for COMPASS e-Write 2–12. These worksheets are designed to help you align e-Write proficiency levels with English language arts courses at your institution. Again, the question to keep in mind as you complete the worksheets that follow is:

What are the entry-level performance or proficiency expectations for this course and how do these expectations align with the COMPASS e-Write 2–12 or the proficiency descriptions listed?

• Use the information captured in Worksheet 3 to evaluate and record proficiency

alignment decisions.

• Please note that proficiency levels are cumulative, with each level assuming the inclusion of material at previous levels. Please reference local English language arts course descriptions to fill in the COMPASS e-Write worksheet appropriate for your college and align your courses with the appropriate COMPASS e-Write holistic scores.

• Use Worksheet 4E to record local course alignment with COMPASS e-Write 2–12 proficiency description.

• Use Worksheet 4F to review local COMPASS e-Write 2–12 data for your college and compare these data to the national data listed. Use the cumulative percentages listed in the first column to guide the selection of local cutoff scores. Local COMPASS e-Write 2–12 information is available from ACT as part of a local Entering Student Descriptive Report (ESDR).

49

Worksheet 4E




Ranges



ELA Course Number


1–2

The response shows an inadequately developed sense of purpose, audience, and situation. These responses show a failed attempt to engage the issue defined in the prompt, and the response displays more than one of the following significant problems: focus on the stated position may be unclear or unsustained; support is lacking or not relevant; much of the style and language may be inappropriate for the occasion, with a very poor control of language: sentences may be poorly constructed and incomplete, word choice may be imprecise, or there may be so many severe errors in usage and mechanics that the writer’s ideas are very difficult to follow.

50

Worksheet 4E (continued)




Ranges



ELA Course Number


3–4 The response shows a poorly developed sense of purpose, audience, and situation. While the writer takes a position on the issue defined in the prompt, the response shows significant problems in one or more of the following areas, making the writer’s ideas often difficult to follow: focus on the stated position may be unclear or unsustained; support may be extremely minimal; organization may lack clear movement or connectedness; much of the style and language may be inappropriate for the occasion, with a weak control of language: sentences may be poorly constructed or incomplete, word choice may be imprecise, or there may be a pattern of errors in usage and mechanics that significantly interfere with meaning.

51





Ranges



ELA Course Number


5–6 The response shows a partially developed sense of purpose, audience, and situation. The writer takes a position on the issue defined in the prompt and attempts to support that position, but with only a little elaboration or explanation. The writer maintains a general focus on the stated position, with minor digressions. Organization is clear enough to follow without difficulty. A limited control of language is apparent: word choice may be imprecise, sentences may be poorly constructed or confusing, and there may be numerous errors in usage and mechanics.

7–8 The response shows a developed sense of purpose, audience, and situation. The writer takes a position on the issue defined in the prompt and supports that position with some elaboration or explanation. Focus on the stated position is clear and generally maintained. Organization is generally clear. A competency with language is apparent: word choice and sentence structures are generally clear and appropriate, though there may be some errors in sentence structure, usage, and mechanics.

52





Ranges



ELA Course Number


9–10

The response shows a well-developed sense of purpose, audience, and situation. The writer takes a position on the issue defined in the prompt and supports that position with moderate elaboration or explanation. Focus on the stated position is clear and consistent. Organization is unified and coherent. A command of language is apparent: word choice and sentence structures are generally varied, precise, and appropriate, though there may be a few errors in sentence structure, usage, and mechanics.

11–12 The response shows a thoughtful and well-developed sense of purpose, audience, and situation. The writer takes a position on the issue defined in the prompt and supports that position with extensive elaboration or explanation. Focus on the stated position is sharp and consistently maintained. Organization is unified and coherent. Outstanding command of language is apparent: word choice is precise, sentences are well structured and varied, and there are few errors in usage and mechanics.

53

Worksheet 4F

COMPASS e-Write 2–12 Score Distributions for


Cumulative %

National COMPASS e-Write

2–12 Score at or below

Local COMPASS e-Write 2–12 Score

at or below (ESDR)


1% 2

3% 3

9% 4

18% 5

44% 6

63% 7

89% 8

96% 9

99% 10

100% 11

100% 12

54

Step 3—Develop Placement Messages Based on Cutoff Scores

At the core of the student placement framework is the placement message that is associated with a student’s score. Use Stage 1 cutoff scores to develop placement messages. Both the score and the associated placement message are displayed on a student’s score report. While the COMPASS software includes default placement messages for all COMPASS/ESL tests, these default messages are intended to provide only a model for users. Placement messages associated with each cutoff score (or cutoff score range) typically include information such as what course the student should take, where to register, and how to access academic support programs and services. To ensure that placement messages align with institution-level needs and requirements, local staff should customize placement messages to either indicate assignment to a specific course (e.g., “Recommend placement in Writing Level 90”) or convey a general impression of performance (e.g., “Recommend placement in an introductory writing course to strengthen your written communication skills”). Placement messages (up to eight lines) can be created for each Placement Test, score range, and Major Group. A different placement message is assigned for each user-defined score range. A complete set of placement messages, called a message group, covers all contingencies within a given content area or domain by collectively assigning placement messages to each possible score point. Figures B1 and B2 illustrate the pre-loaded cutoff scores and sample placement messages for COMPASS Writing Skills and COMPASS Reading.

Figure B1 COMPASS Writing Skills Placement Message Screen

55

Figure B2 COMPASS Reading Placement Message Screen

• Worksheet 5 contains sample COMPASS Writing Skills placement messages, with space for you to revise these messages to align with local course needs.

• Worksheet 6 contains sample COMPASS Reading placement messages, with space

for you to revise these messages to align with local course needs.

• Worksheet 7 contains sample COMPASS e-Write 2–8 placement messages, with space for you to revise these messages to align with local course needs.

• Worksheet 8 contains sample COMPASS e-Write 2–12 placement messages, with

space for you to revise these messages to align with local course needs.

56

Worksheet 5

COMPASS Writing Skills Cutoff Score Ranges & Placement Messages for

_______________________________________ (insert name of course)

If the score is… show this message in the score report:

0 37 Sample placement message: Enroll in ENG 080: English Development I (3 credits; course does not apply to AA or AS degree; description appears on page 201 of the college catalog). To help you succeed, be sure to use the "Tips for Success in English Courses" ideas (see blue Success Tips document) and the Writing and Study Skills materials in the Learning Center, Johnson Hall, Room 101 (no charge; 8 am - 7 pm, M-F, call 333-1211 for more information on these services). Also talk with your advisor if you have questions about course choices.

38 69 Sample placement message: Enroll in ENG 090; English Development II (3 credits; course does not apply to AA or AS degree; description appears on page 201 of the college catalog). To help you succeed, be sure to use the "Tips for Success in English Courses" ideas (see blue Success Tips document) and the Writing and Study Skills materials in the Learning Center, Johnson Hall, Room 101 (no charge; 8 am - 7 pm, M-F; call 333-1211 for more information on these services). Also talk with your advisor if you have questions about course choices.

70 100 Sample placement message: Enroll in ENG 101 Freshman Composition (3 credits; course can be applied to the AA or AS degree; course description appears on page 201 of the college catalog). To help you succeed, be sure to use the "Tips for Success in English Courses" ideas (see blue Success Tips document) and the Writing and Study Skills materials in the Learning Center, Johnson Hall, Room 101 (no charge; 8 am - 7 pm, M-F; call 333-1211 for more information on these services). Also talk with your advisor if you have questions about the course choices.


Course 1:________________________________

Course 2:________________________________

Course 3:________________________________

Course 4:________________________________

57

Worksheet 6

COMPASS Reading Cutoff Score Ranges & Placement Messages for



0 60 Sample placement message: Enroll in RDG 080: Reading Development I (3 credits; not a college transfer course; course description appears on page 208 of the college catalog). To help you succeed, be sure to use the "Tips for Success in Reading Courses" ideas (see blue Success Tips document) and the Reading and Study Skills materials in the Learning Center, Johnson Hall, Room 101 (no charge; 8 am - 7 pm, M-F; call 333-1211 for more information on these services). Also talk with your advisor if you have questions about course choices.

61 80 Sample placement message: Enroll in RDG 090: Reading Development II (3 credits; not a college transfer course; course description appears on page 208 of the college catalog). To help you succeed, be sure to use the "Tips for Success in Reading Courses" ideas (see blue Success Tips document) and the Reading and Study Skills materials in the Learning Center, Johnson Hall, Room 101 (no charge; 8 am - 7 pm, M-F; call 333-1211 for more information on these services). Also talk with your advisor if you have questions about course choices.

81 100 Sample placement message: Enroll in RDG 080: Reading Development I (3 credits; not a college transfer course; course description appears on page 208 of the college catalog). To help you succeed, be sure to use the "Tips for Success in Reading Courses" ideas (see blue Success Tips document) and the Reading and Study Skills materials in the learning Center, Johnson Hall, Room 101 (no charge; 8 am - 7 pm, M-F; call 333-1211 for more information on these services). Also talk with your advisor if you have questions about course choices.


Course 1:________________________________

Course 2:________________________________

Course 3:________________________________

Course 4:________________________________

58

Worksheet 7

COMPASS e-Write 2–8 Score Ranges & Placement Messages for



0 4 Sample placement message: Enroll in Developmental English 1.


6 8 Sample placement message: Enroll in Freshman English.

9 90 This score is out of range. Please check with your advisor.

91 91 The response was off topic.

92 92 The response was judged to be unscoreable (e.g., blank response, unintelligible response, or a response written in a language other than English).



Course 1:________________________________

Course 2:________________________________

Course 3:________________________________

Course 4:________________________________

59

Worksheet 8

COMPASS e-Write 2–12 Score Ranges & Placement Messages for





8 12 Sample placement message: Enroll in Freshman English.


91 91 The response was off topic.

92 92 The response was judged to be unscoreable (e.g., blank response, unintelligible response, or a response written in a language other than English).



Course 1:________________________________

Course 2:________________________________

Course 3:________________________________

Course 4:________________________________

60

Step 4—Validate Stage 1 Cutoff Scores

Use expectations associated with local curriculum to validate cutoff scores. Once a COMPASS English language arts test package has been implemented, it is important that you validate placement results using faculty feedback or use faculty judgments to adjust cutoff scores. Colleges can use faculty ratings to evaluate placement decisions using the following rating scale:

5. Should definitely be placed in a higher level course.

4. Might have the ability to do well in a higher level course.

3. Is appropriately placed.

2. Might be better placed in a lower level course.

1. Should definitely be placed in a lower level course. Based on faculty feedback on the above, you can better judge the validity of local placement decisions. Should faculty ratings indicate that a change is needed to improve alignment between course content expectations and COMPASS test results, an adjustment to cutoff scores should be made. When there is uncertainty as to how to use results to make placement decision adjustments, colleges have ACT Course Placement Service resources available to support a data-driven fine-tuning of cutoff scores.

ACT Offices

Postsecondary Services—Assessment Programs

National Headquarters500 ACT DriveP.O. Box 168

Iowa City, Iowa 52243-0168Telephone: 319/337-1000

Washington, DC OfficeOne Dupont Circle NW, Suite 340

Washington, DC 20036-1170Telephone: 202/223-2318

Hunt Valley OfficeExecutive Plaza I, Suite 200

11350 McCormick RoadHunt Valley, Maryland 21031-1002

Telephone: 410/584-8000

National Center for Educational Accountability

4030-2 West Braker LaneAustin, Texas 78759-5329Telephone: 512/320-1800

WEST REGION

Denver Office3131 South Vaughn Way, Suite 218

Aurora, Colorado 80014-3507Telephone: 303/337-3273

Sacramento Office2880 Sunrise Boulevard, Suite 214

Rancho Cordova, California 95742-6549Telephone: 916/631-9200

MIDWEST REGION

Chicago Office300 Knightsbridge Parkway, Suite 300

Lincolnshire, Illinois 60069-9498Telephone: 847/634-2560

Ohio Office700 Taylor Road, Suite 210Gahanna, Ohio 43230-3318Telephone: 614/470-9828

Michigan Office1001 Centennial Way, Suite 400Lansing, Michigan 48917-8249

Telephone: 517/327-5919

SOUTHWEST REGION

Austin Office8303 MoPac Expressway N, Suite A-110

Austin, Texas 78759-8369Telephone: 512/345-1949

EAST REGION

Albany Office4 Pine West Plaza, Suite 403Albany, New York 12205-5564

Telephone: 518/869-7378

Atlanta Office3355 Lenox Road NE, Suite 320

Atlanta, Georgia 30326-1332Telephone: 404/231-1952

Florida Office1315 East Lafayette Street, Suite A

Tallahassee, Florida 32301-4757Telephone: 850/878-2729

Field Offices

Olathe Office1365 North Winchester StreetOlathe, Kansas 66061-5880Telephone: 913/768-1696

© 2008 ACT, Inc. All rights reserved. 11928 *0170A2090*

Documents

COMPASS Guide to Effective Student Placement and Retention in