Introduction to Formative Tools

Rural Schools ConferenceKim Rathke, Julie Quinn, and Jared Wright

Utah State Office of Education

Summer, 2013 2

What is the most emphasized indicator of success in your school?

Formative Tools

Interim AssessmentSummative Assessment

Benchmark Assessment

Teacher-led Classroom Assessment

Teacher EvaluationAccountability Student Reflection and ownership of learning

Teacher Reflection

Analysis of student work

What informs data conversations in your school?

Formative Tools

Interim AssessmentSummative Assessment

Benchmark Assessment

Teacher-led Classroom Assessment

Teacher EvaluationAccountability Student Reflection and ownership of learning

Teacher Reflection

Analysis of student work

What is the purpose of the assessment?

With Assessment, Purpose Is Everything

We assess for two reasons:1) to gather evidence to inform instructional decisions and2) to encourage students to try to learn

Stiggins, R. (2008). Assessment Manifesto:

A Call for the Development of Balanced

Assessment Systems. Portland, OR:

ETS Assessment Training Institute.

FIRST Components (Teacher Tools)Tool Access and Support Features Timeline

“Old” UTIPS 12 regional serversE.g., Myutips.org,

Ucutips.org

Available through July 31, 2014

UTIPS Core Central server currently located at UEN

New servers come online at USOE August 1, 2014

E.g., District.utips.org

Help Desk: help@utips.org

Documentation: http://utipscore.wikispaces.com/

Teacher-created items and tests

Share tests to groups

Current State StandardsELA, Math, Science, CTE, Health, all content areas

State item pool was realigned to current state standards – no new items

have been added

Available through at least June 30, 2015

6

FIRST Components (Teacher Tools)Tool Access and Support Features Timeline

Learning Point Navigator(Formerly

SAGE Formative)

http://sageportal.org

Help Desk: sagehelpdesk@air.org or 855-570-7239

Documentation: User Guides -

http://sageportal.org/resources/?section=1-general-family

Tutorials -

http://sageportal.org/resources/?section=1-general-family

Students log in with SSID and password – can log in from home

and create their own assignments, separate from teacher-created

assignments

Math, Science, ELA onlyMultiple Item Types

(MC, CR, technology-enhanced (haptic))

Teachers create assessments for their own classrooms to support instruction

Shared assessments supported beginning fall 2014

Instructional support materials

School and district level libraries, sharing between teachers coming fall 2014

Students loaded from UTREx

May, 2014Teacher-created items, can share

to school or district library

Fall, 2014Ad-hoc groups can be created to

share assignments between

COMPOSE(Formerly Utah

Write)

Access and support TBA…

Online writing with AI scoring for grades 3-12

Available to all content teachers, grades 3-12

Launches September, 2014

Interim and Benchmark Assessments

Tool Purpose Features Timeline

SAGE InterimOPTIONAL

Provide “growth”

Predictive of SAGE Summative

Computer Adaptive

Full mirror of theSAGE Summative assessment

(blueprint, length)

Immediate Results flow to Learning Point Navigator

Fall Interim

Winter Interim

Students may take one interim during the fall and/or one interim during the winter

DIBELSREQUIRED

Reading on grade level determination

“Amplify” available for online data collection (LEA option)

Ongoing

ShmoopOPTIONAL

ACT Prep Toolhttp://www.shmoop.com/utah/

Sample questions and lesson plans

Ongoing

Explore, PlanOPTIONAL

Predictive of ACT Aligned to ACT, provides indicators of

strengths/weaknesses

Ongoing

What is “Formative” Assessment?

(FAST) State Collaborative on Assessment and Student Standards (SCASS) of the Council of Chief State School Officers (CCSSO):

Formative assessment is a process used by teachers and students during instruction that provides feedback to adjust ongoing teaching

and learning to improve students’ achievement of intended instructional outcomes. The formative assessment process involves

gathering evidence and providing feedback to adjust ongoing teaching moves and learning tactics.

What is “Formative” Assessment?Similarly, an assessment is an important part of the formative-

assessment process, but it is only that—a part of the formative-assessment process. The entire process involves decisions about when to test and what to test, selection or construction of suitable assessment procedures, judgments about whether assessment-elicited evidence should lead to

adjustments, and choices about the nature of any adjustments. Assessments are a key component of the formative-

assessment process, but they are not the entire process.Popham, W. J. (2011, February 22).

Formative Assessment-A Process, Not a Test. Retrieved June 23, 2014, from http://www.edweek.org/ew/articles/2011/02/23/21popham.h30.html?tkn=PSCCGmSb/B5QkuTaRS6t7BoT2I7Q/Q9NdgMl

Elements Of anAssessment Event

1. Is the assessment high-quality?2. Do teachers have access to the data?3. Do teachers use the data?4. Do students have access to the data?5. Do students use the data?

Summer, 2013 11

Four Major Assessment Types

Daily Events – Things that happen everyday in the normal activity of a classroom

Periodic Activities – Structured or planned classroom activities

Formal Assessments – Documented evidence about something is gathered Typically student understanding is what the evidence points to

External Assessments – Evidence of student understanding is gathered for purposes outside of the classroom

Data-Informed Instruction

Good Assessment Should generate good data

Good data Should generate good discussion and conclusions

Good conclusions Should generate good actions

UsingLearning Point

NavigatorLPN

Key Terminology

• Item: an assignment activity that requires a student response (a question).• Activity: an individual component of a resource—a lesson, a practice question,

or a quiz question• Assignment: a collection of student activity resources (a quiz or test)• Stimulus: a component in a resource that provides context for an activity—

includes reading passages, images, graphs, tables, video or audio clips.• Stem: The question or prompt in an activity that students must respond to.• Content Editor: The tool in the Activity Builder that is used to align activities to

standards, and to create and edit the content for activities and stimuli.

Learning Point Navigator

Formerly “SAGE Formative”

Three main tools—grades K-12, Math, Science and ELA ONLY • Formative test bank for ELA, math, and science—to inform instruction

• Includes items types that look and feel like the SAGE Summative item types• Activity builder – educator-created items, must align to math, science or ELA

standards (multiple choice – single and multiple select; constructed response)• Lesson plans—to enhance instruction and professional development

Learning Point Navigator

• The landing page for educators provides access to the five major sections of the site:

• Browse Resources• Assignments• Activity Builder• Classes & Students• Reports

Browse Resources

• Resources are aligned to ELA, Math, and Science standards and benchmarks.

• Resources are also categorized by ELA, Math, and Science content areas, grade level, library, material type, and learning modality. These categories are organized as filter boxes on the search panel.

• Student activities can be added to assignments. • Lesson plans and professional development aids are not designed for

students and cannot be added to assignments.

Browse Resources

Browse Resources Search Panel (With Filters Selected)

Browse Resources: Resource Search

Sorting search results Previewing resources Marking resources as favorites Rating and reviewing resources

Viewing related resources Removing resources Copying resources to the Activity Builder Adding resources to the Resource Cart

The Resources Search Results list displays the resource(s) that meet the search parameters you selected. This results section provides you with several options:

Browse Resources: Previewing• You can preview and navigate through resources to see how they will look to students. You can also use the

student tools, mark resources as favorites, add resources to the Resource Cart, and print resources.

• On the Resources Search Results page, click the arrow button next to the name of the resource. The Resource Preview will load.

Resource Preview Page

Previewing a Resource

Browse Resources: Marking Favorites• You can mark resources as favorites, allowing you to easily find them later. Resources can be

“favorited” from the Resources Search Results page or from the Resource Preview page.

To view a list of only your favorite resources in the search results section, click the [Favorites] button next to “View by.” If you have not marked any resources as favorites, no resources will be displayed.

To view all the resources in the search results, click the [All] button next to “View by.”

Adding Resources• The Resource Cart allows educators to temporarily store resources. Resources that have been added to the

Resource Cart can be combined into a new resource or saved as an assignment. Added resources will remain in the resource cart until they are removed or they are saved as a resource or assignment.

Saving Resources as Assignments • Once resources have been saved in the Resource Cart as an assignment, the Add to Assignment

page will load. On the Add to Assignment page, you can choose to add the resources to a new or existing assignment for your students.

Saving Resources as Assignments1. In the Assignment Name field, enter the assignment’s name.

2. Click inside the Start Date field. A pair of calendars will pop up. Click the date when the assignment should start. Repeat the process for the Due Date field.

Saving Resources as Assignments3. In the Directions field, enter the directions for the assignment.4. You can adjust the proficiency level cut scores. Click and drag the

sliders on the Proficiency scale.

Saving Resources as Assignments

5. To preview the assignment, click the [Preview Assignment] button. To finish and assign to students, click the [Save] button. To exit this page without creating the assignment, click the [Cancel] button.

Adding Resources to Existing Assignments

The process for adding a resource to an existing assignment is similar to the process for adding a resource to a new assignment.

• On the Add to Assignment page, click the “Select Assignment” drop-down list, and select the assignment that you want to add the resource to.

Creating Your Own Activities

Using the Activity Builder

Activity Builder Enables users to develop their own activities using the Content Editor and assign them to students

Creating an Activity• To create an activity, click the [+ Create New Activity] button above

the Activities table.

Creating an Activity• The details page of the Content Editor will open. You are creating one

question so name the activity as a question.• In the Name field, enter a name for the activity.

Creating an Activity • To align the activity with a standard, click the [Add Alignment] button. • The “Grades” drop-down menu will appear. • Continue to select options from the drop-down menus until you can click the

[Add] button.

Creating an Activity• Click the [Save] button and then click the [Open Content Editor]

button. The content authoring page of the Content Editor will open.

Adding a Stem• Step 1: In the Activity Canvas, click [Item Body].

Adding a Stem• Step 2: In the Elements Pane, click the [Stem] element. The Content

Editor text box will open.

Adding a Stem• Step 3: In the editing area of the Content Editor text box, enter the

text for the stem. • Step 4: When you are finished, click [Save]. The Stem box will appear

in the Activity Canvas.

Adding Response Choices• Step 5: In the Activity Canvas, click the blue [Insert] bar below the

Stem box. • Step 6: In the Elements Pane, click the [Response Choices] element.

Adding Response Options Step 7: In the Activity Canvas, click [Response Choices]. en Step 8: In the Elements Pane, click [Response Option] element. The Content

Editor text box will open. Step 9: In the editing area, enter the response option. Step 10: Click the [Save] button. The Response Option will be added to the

Activity Canvas. To add additional Response Options, you will click on every other blue bar.

Adding Response Feedback Step 11: To add feedback for the response option, click the blue [Insert] bar

below the Response Option box. Step 12: In the Elements Pane, click the [Response Feedback] element. The

Content Editor text box will open. Step 13: In the editing area, enter the response feedback.

Adding Response Feedback Step 14: When you are finished, click the [Save] button. The Response Feedback box will be added to the Activity Canvas.

To add multiple response options to the activity, repeat steps for each response option.

Adding a Multiple Choice Rubric To access the rubric, click the [File] button above the Elements Pane and select

“Multiple Choice Rubric.” The rubric window will open. Click the correct response option(s). The correct answer(s) will be marked in the

rubric. When you are finished configuring the rubric, click the [Save] button. When you have finished creating the activity, click the [Save] button located in

the bottom right corner of the Content Editor page.

Reviewing a Multiple Choice Rubric To view the rubric, click the [File] button above the Elements Pane

and select “Multiple Choice Rubric.” The rubric window will open. The correct answer will be selected. When you have finished reviewing the rubric, click the [Save] button

located in the bottom right corner of the Content Editor tool.

Adding a Text Response Area After adding a stem to the Activity Canvas, click the blue [Insert] bar

directly below the stem box. Click the [Text Response Area] element. The Text Response Area box

will be added to the Activity Canvas.

Adding a Human Rubric To access the rubric, click the [File] button above the Elements Pane

and select “Human Rubric.” The Content Editor text box will open. Enter the hand-scoring rubric

and then click [Save]. The Content Editor text box will open. Enter the hand-scoring rubric

and then click [Save].

Reviewing a Human Rubric To view the rubric, click the [File] button above the Elements Pane

and select “Human Rubric.” The Content Editor text box will open. When you have finished reviewing the rubric, click the [Save] button

located in the bottom right corner of the Content Editor tool.

Hand Scoring Allows users to manually enter scores for certain activities in student assignments.

If an assignment requires hand scoring, the Actions toolbar on the Assignments page will include a [Hand Score] button.

Viewing an Activity You can preview an activity by clicking the View button to the left of

the activity on the Activities table.

Publishing an Activity as a Resource

• Navigate to the Activities table on the Activity Builder page.• Click the [<<] button in the Actions column. The Actions toolbar will

expand.

Publishing an Activity as a Resource

1. Click the [Publish] button in the Actions toolbar. The “Create New Resource” page will open.

2. In the Name field, enter a name for the resource.3. In the Description field, enter a description for the resource.

Create New Resource Page: Managing Standards

• The Aligned Standards section displays the standards that are aligned to the activities in the resource. To remove a standard, uncheck the checkbox next to that standard. Note: A resource must be aligned to at least one standard. If there is only one standard listed in this section, you cannot uncheck it.

Select Library to Publish to Window

• When you are finished, click the [Save and Publish] button in the bottom right corner of the page. If you are a School Educator, the resource will automatically be added to My Library. If you are an Administrative Educator, the “Select a Library to Publish to” window will open. Click the name of the library you want to publish the resource to and then click [Publish]. The resource will be added to the selected library. Note: This step only applies to Administrative Educators.

Where Do I Find My Activities?

• After you have published your activity, you will find each individual question under Browse Resources in your own library.

• To locate them quickly, make sure you check only the box titled “My Library.”

• To assign your activities, follow the same steps used to assign resources from the formative test bank.

Creating Common Assignments

LEAs/Administrators OnlyTeachers Fall 2014

Common Assignments Provides ability for school or LEA level users to develop common assignments

that can be assigned by teachers in their school or district. Stored in an entity library that any School Educator belonging to that entity can

access.

Creating a Common Assignment

• Administrative Educators can create common assignments, but they cannot assign them directly to students.

1. In the Name field, enter a name for the common assignment.

Creating a Common Assignment

1. Click inside the Assignment Window Open field. A pair of calendars will pop up.

2. Click the date when the assignment should become available.

3. Click inside the Assignment Window Close field. A pair of calendars will pop up.

4. Click the date when the assignment’s availability should end.

Creating a Common Assignment

5. In the Directions field, enter directions for the assignment.

Creating a Common Assignment

6. To adjust the proficiency level cut scores for the assignment, click and drag the sliders on the Proficiency Level scale. These levels determine which student scores will be considered “not proficient,” “needs improvement,” and “proficient.”

Creating a Common Assignment7. Optional: To add a calculator to the assignment, select an option from the

“Select a Calculator” drop-down menu.8. Optional: To upload a file to the assignment, click the [Select File] button and

then select a file from your computer. It’s important to note that only pdf files should be uploaded because students can edit others.

Creating a Common Assignment9. To manage the resources included in the common assignment, click [Manage Resources]. The

Assigned Resources window will open.

10. When you are finished creating the resource, click the [Save] button in the bottom right corner of the Create New Assignment page. Remind Admin Educators that their assignments are saved in the LEA Library if they need to go back to review or edit.

Reporting withLearning Point

Navigator

Common Assignment Reports• Assignment Summary and Item Analysis Reports available• Display data only for the selected assignment at the selected library

level• Available at the state, LEA, and school level for Administrative

Educators.• These reports are similar to assignment reports that school educators

can view however they are only available to administrative educators and can only be accessed from the assignments page.

Common Assignment Summary Reports

• Provide overall assignment information for the selected common assignment at the selected level.

• Available at the state, LEA, and school level.

Common Assignment Item Analysis Reports

• Available for common assignments at various library levels.• Provide information on how students responded to each quiz item

included in the selected common assignment.

Educator-Level Reporting

•Three types of reports• Assignment Summary Reports• Benchmark Proficiency Reports—not available for

common assignments until Fall 2014• Item Analysis Reports

Educator-Level Reporting

• Who? . . . Who do you want to see data for? All rosters, a single roster, only students given the assignment, or an individual student?

• What? . . . What data do you want to view? Assignment results, benchmark results, or item-specific information?

• When? . . . At what point in time do you want to view data? Do you want to view performance during a specific window in time? Do you want to view performance on a single assignment? How do you want to handle items not answered or attempted by students?”

Educator-Level Reporting• When you click [View Reports], the main Reports Widget will display.

This widget allows you to select the parameters for each available report dimension: Who, What, and When. This widget is also available from within each report.

Educator-Level Reporting

Sample Assignment Summary Report Screen for a Selected Assignment

Educator-Level ReportingBenchmark Proficiency Reports provide information on students’ proficiency for each benchmark associated with the selected assignment(s). These reports display students’ benchmark proficiency based on their response to activities. Activities can be aligned to one or more benchmarks. ONLY available at educator level until Fall 2014

Educator-Level ReportingBenchmark Proficiency Reports provide information on students’ proficiency for each benchmark associated with the selected assignment(s). These reports display students’ benchmark proficiency based on their response to activities. Activities can be aligned to one or more benchmarks. ONLY available at educator level until Fall 2014

New FeaturesExpected in Fall 2014

New Features

• Simplified item authoring tool—tabs added to reduce mouse clicks• Peer-to-Peer sharing—Educators can share assignments• .csv Export of individual reports• Assignment print feature—Educators will be able to print an entire

assignment• Answer key: Option to see answer key for all machine-scored items• UTIPS upload: Ability to upload ELA, math, and science items from

teacher-created UTIPS files into educator libraries

New Features

• Accessibility features:• Masking• Text-to-Speech capability in Chrome browser and iOs• iPads will be able to download a browser for TTS• Creating a Chrome plug-in for PCs• Ability to expand and contract passages