Embed Size (px)
Measuring Progress
A Consortium Workshop on Assessing Students for Technological
Literacy
About
Center to Advance the Teaching of Technology & Science
It is the professional development arm of ITEA Initiatives are directed toward:
Development and dissemination of standards-based curriculum materials and supporting resources
Teacher enhancement Research projects Consortium of states
The CATTS Consortium is a cooperative means by which states can efficiently accomplish work that contributes to their respective state plans
Typical Development Process
Identify the course, product, or service Conduct the research Present an outline to the CATTS Consortium Develop the course, product, or service Solicit field reviews Conduct teacher workshops in Consortium states
and/or train trainers at ITEA Conference Use for one year in Consortium states Revise, print, and produce compact disks Disseminate
Center to Advance the Teaching of Technology & Science
Standards for Technological Literacy
• Nature of Technology
• Technology and Society
• Design
• Abilities for a Technological World
• The Designed World
A National Model
Elementary• Models for Introducing Technology: A Standards - based Guide
Middle School• Exploring Technology• Invention & Innovation• Technological Systems
High School• Foundations of Technology• Technology Assessment• Issues in Technology• Engineering Design
K
12
Courses Under Development
Engineering Model Course Guide High School
Invention and Innovation Model Course Guide Middle School
Center to Advance the Teaching of Technology & Science
Contents
Chapter 1•Learning, Memory, and a Study of TechnologyChapter 2• Designing - From the Beginning• The Natural World• The Human-Made World• Tools & Safety• Engineering Portfolio & Journals• Classroom Store• The Designed WorldChapter 3• Resources
Center to Advance the Teaching of Technology & Science
Supported by the National Science Foundation
This Four Project will focus on:• Designing Thematic Units for Grades 5 & 6• Developing Teaching & Learning Resources• Disseminating Workshops & Distance Learning
Design & Innovation Units• Ways Inventions & Innovations are Developed and• How They Affect Us Personally, Socially, & Economically
Technological Systems Units• Ways Systems are Developed, Produced, Controlled, & Assessed• Medical, Agriculture-Biological, Energy-Power, Manufacturing, Communication, Construction
Workshop Objectives
At the end of this workshop, participants will be able to: Effectively assess their students for technological
literacy using standards-based assessment strategies. Apply the results of effective student assessment to
improve teaching and learning and to address issues of accountability.
Teach others about how to effectively assess their students for technological literacy using standards-based assessment strategies.
Professional Development Considerations
Help participants design or select the PDPromote incremental changeProvide for collaborationCreate an atmosphere for open discussionCreate a climate of sustained supportEstablish a rapport were adults have both
permission and expectation to share the responsibility for their learning
NASA Professional Development Benchmark Study (NASA, 2002)
About Adult Learners
Tend to be self-directedHave a rich reservoir of experience that can
serve as a resource for learningHave a life-, task-, or problem-centered
orientation to learning as opposed to a subject matter orientation
Generally motivated to learn due to internal or intrinsic factors
NASA Professional Development Benchmark Study (NASA, 2002)
Adult Learners
Used to getting things done on their own
Technologically literateResponsiveFocusedLifelong learnersAmbitiousFearless
NASA Professional Development Benchmark Study (NASA, 2002)
Workshop Overview
Introduction to assessing students for technological literacy
What to assessHow to assessHow to gather evidence of learningOrganizing the standards/assessment-
based learning environmentHelping others with the assessment
processAssessing the workshop
Center to Advance the Teaching of Technology & Science
Contents
Section One: IntroductionSection Two: Standards, Benchmarks, and Enduring ConceptsSection Three: Assessment, Criteria and Evidence-Gathering ToolsSection Four: Managing the Assessment Process
Notebook Contents
Measuring Progress: Assessing Students for Technological Literacy (Process of Development)
Strategies & Activities, Script AlsoPower Point SlidesStandards for Technological LiteracyState Standards & MaterialsWorksheetsEvaluation
Assessment
Assessment is the systematic, multi-step process of collecting information on student learning, understanding, and capability and using that information to inform instruction and enhance learning.
Assessment
Curriculum assessment and student assessment are different
Curriculum assessment measures the curriculum against frameworks and standards
Student assessment measures student understandings against standards
Why Assess Students for Technological Literacy?
Primarily:To improve students’ learningTo improve teaching effectiveness
And as a result, there are other benefits too!
Center to Advance the Teaching of Technology & Science
Advancing Excellence in Technological Literacy:
• Student Assessment• Professional Development• Program Standards
Technology for All Americans Project
Student Assessment STL
Standards-basedClear purposePrinciples of good assessmentWorld beyond school contextsApplication of results
Standards-based means:
Standards for Technological Literacy: Content for the Study of Technology (STL)
There are others, but we’ll discuss those later
Standards-based (STL)
Students will develop an understanding of The Nature of TechnologyStudents will develop an understanding of Technology and SocietyStudents will develop an understanding of
DesignStudents will develop Abilities for a Technological WorldStudents will develop an understanding of The Designed World
Clarity of Purpose
Focus on significant understandings, what is really important to know, be able to do, and truly understand
Establish clearly defined expectations so it will be clear when they are met
Principles of Good Assessment
Assess often Formative Summative
Use a variety of measures Consider learning styles Consider students’ abilities to demonstrate
understandings Match the evidence gathering tool to the
measurement task Use teacher, peer, and self-assessment
Provide timely feedback To students and teachers Not just right or wrong, but how to
improveMaintain security
Establish equity Make expectations clear Treat all students with the same respect Use contexts that have meaning to all
students Be objective; don’t fulfil biased expectations Don’t be predisposed with color, gender,
social class, or the way they talk or look Don’t use assessment as a sorting tool
World Contexts
Couch assessment in everyday situations that are familiar to students(also answers the question of, “yes, but
when am I ever going to use this stuff?”)Be sure that the contexts are
actually familiar to all students
Applications of Results
Communicating achievement and performance to students
Making instructional and program or curriculum decisions
Assigning grades and communicating with parents & guardians
Marketing and promotionCommunicating with other
stakeholders
The Nature of Technology
Students will develop an understanding of The Nature of Technology. This includes acquiring knowledge of:1. The characteristics and scope of
technology.2. The core concepts of technology.3. The relationships among
technologies and the connections between technology and other fields.
Students will develop an understanding of Technology and Society. This includes learning about:4. The cultural, social, economic, and political
effects of technology.5. The effects of technology on the
environment.6. The role of society in the development and
use of technology.7. The influence of technology on history.
Technology and Society
Students will develop an understanding of Design. This includes knowing about: 8. The attributes of design. 9. Engineering design.10. The role of troubleshooting,
research and development, invention and innovation, and experimentation in problem solving.
Design
Students will develop Abilities for a Technological World. This includes becoming able to:11. Apply the design process.12. Use and maintain technological
products and systems.13. Assess the impact of products and
systems.
Abilities for a Technological World
Students will develop an understanding of The Designed World. This includes selecting and using:14. Medical technologies.15. Agricultural and related biotechnologies.16. Energy and power technologies.17. Information and communication
technologies.18. Transportation technologies.19. Manufacturing technologies.20. Construction technologies.
The Designed World
Emotional intelligenceTeam workResponsibilityWriting & mathematics applicationsReadingScience connectionsEconomics, government, globalizationSCANS
Other Important Content, e.g.
SCANS: Skills & Competencies
Competencies Resources Interpersonal Information Systems Technology
Foundation & Personal Qualities Basic Skills Thinking Skills Personal Qualities
A Technological Studies Continuum
Grade Levels
K-2 3-5 6-8 9 10 11 12 13 14 15 16 Graduate|_________________________________________________________________________|
Standards for Technological Literacy (All Levels)
|_________________|Industry Standards (NSSB)
|______________________| Professional Standards (Accreditation)
|________| Professional
Standards for Technological Literacy
•Characteristics & Scope of Technology•Core Concepts of Technology•Relationship of Technology & Other Subjects
•Social, Economic, & Political Effects of Technology•Effects of Technology on the Environment•Role of Society in Technology•Influence of Technology on History
•Attributes of Design•Engineering Design•Trouble Shooting, R&D, Invention & Innovation, Problem Solving
•Apply the Design Process•Use & Maintenance of Technological Systems•Assess Impacts of Technology
•Medical Technologies•Agricultural & Biological Technologies•Energy & Power Technologies•Information & Communication Technologies•Transportation Technologies•Manufacturing Technologies•Construction Technologies
Other Standards
Related to all of Technology•ABET•SCANS•NSSB•Mathematics•Science•English / Language Arts•Social Studies•Etc.
Related to Construction•DPI, DWD, AGC, MCUBC•Milwaukee Area Const. Adv. Comm. •Nat. Center Const. Educ. & Research•Am. Institute of Constructors
Pre - K Life-long Learning
Understanding Defined
Understanding is “the capacity to apply knowledge to new situations in appropriate ways.” (Howard Gardner)
Enduring Concepts: These are: Big (broad with universal
applications) Important (significant) Profound (deep, thoughtful) Lasting (durable through time)Ideas about technology.
So What’s Worthy of Understanding?
“Enduring” understandings (Wiggins & McTighe, 1998):
Represent a big idea having enduring value beyond the classroom.
Reside at the heart of the discipline (involve “doing” the subject).
Require uncoverage (of abstract or often misunderstood ideas).
Offer potential for engaging students.
Enduring Understandings
Understanding
In this book, (Understanding by Design, 1998) we (Wiggins and McTighe) use “understand” to mean that a student has something more than just textbook knowledge and skill – that a student really “gets it.” Understanding involves sophisticated insights and abilities, reflected in varied performances and contexts.
Students need to make conscious sense and apt use of the knowledge they are acquiring as well as the principles underlying it (Wiggins and McTighe (1998).
Wiggins and McTighe (1998) go on to say that understanding involves the abstract and conceptual, not merely the concrete and discrete: concepts, generalizations,
theories, and mental links between facts.
Standards-based & Traditional
Standards-based AssessmentApply Standards for Technological
Literacy Establish expectations/outcomes to
determine what we teachAssess for enduring understandings
that will be lasting through time and apply in different situations
Assess often and provide continuous feedback to students and teachers
Assess students using multiple measures
Assess to improve the learning and teaching process
Assess to improve the Standards for Technological Literacy
Assess to ensure accuracy of content
Traditional AssessmentApply curriculum models that
focus on craft knowledge and skills
Teach our favorite content and then rationalize that we are attaining standards and objectives
Measure for specific content knowledge and craft skills
Assess at the end of a term or when the project is finished
Assess using limited measures such as paper/pencil tests and completed projects
Assess students to determine a grade
Curriculum Development Considerations
Technology Education Delivery
Curriculum (standards-based)
Instruction Assessment
Technology Education Development
Curriculum (standards-based) Assessment Instruction
Measuring Progress: . . .
Standards and Benchmarks - lead to big, important, and
Enduring Concepts - that lead to
Selection of Evidence Gathering Tools
Assessment Criteria - that influence and the design of
Teaching Strategies - that drive the selection of
Learning Activities - that are monitored by
Assessment - that informs
A Standards-based Curriculum Approach with Embedded Assessment
Format
Standard: Engineering DesignBenchmarks: Design Process; Modeling; Testing; Prototyping;
EvaluatingEnduring Concept: Problem-solving/OptimizationSub-concepts: Defining the Challenge Proposing Solutions Testing Possibilities Applying the DesignAssessment Criteria & Levels of Understanding:Learning Activity: Working from a design brief, students will build
a working model of a fish farmEvidence Gathering: Describe, list, sketch, model, brainstorm,
plan, draw,engage experts,etc.
Course or Unit: ________________________________________
Standard(s): __________________________________________
Grade Level: (K-2) (3-5) (6-8) (9-12) ______________________
Enduring Concepts:1. ___________________________ 3. ______________________2. ___________________________ 4. ______________________ An Enduring Concept is a large, important, profound, and lasting idea we want
students to understand. An Understanding is “the capacity to apply knowledge to new situations in
appropriate ways.” (Howard Gardner)
Sub-concepts:1. ___________________________ 3. ______________________2. ___________________________ 4. ______________________
Rubric:Enduring Concept goes here: __________________________
Sub-conceptsLevels
Target3
Draft2
Unacceptable1
Learning Activity with Embedded Assessment (Evidence Gathering):
Activity
Purpose: Introduce STL & State Standards, Objectives, and Materials
Consider: Teachers have students build a computer from component parts, interface the device with input & output devices, and network machines with others
Connect: The learning activity with standards or objectives. Which standards might be addressed?
Activity
Purpose: Introduce Enduring ConceptsConsider: Teachers have students build a computer
from component parts, interface the device with input & output devices, and network machines with others
Connect: Using the benchmarks, connect the learning activity with standards or objectives and enduring concepts. Which standards and enduring concepts might be addressed? What big and important ideas were these teachers addressing?
Purpose: Apply the use of standards, benchmarks, and enduring concepts
Select: A favorite course, identify the standards being addressed, and using the benchmarks, identify the enduring concepts.
Select: One enduring concept to be assessedWorksheet: Blue worksheet in notebook
Activity (Part 2)
Problem Solving/Optimization
Element
Levels
Defining theChallenge
Proposing Solutions Testing Possibilities Applying the Design
3The challenge is veryclear, specific, andfocused
Sketches, drawings, anddescriptions suggest awide range ofpossibilities
Effective drawings,descriptions, andprototypes of severalpossibilities
An effective drawing,model, portfolio, andpresentation of acreative and workablesolution
2The challenge issomewhat specific andfocused
Sketches, drawings, anddescriptions show some“out-of-the-box”possibilities
Somewhat effectivedrawings and prototypesof a few possibilities
A reasonably effectivedrawing, model,portfolio, andpresentation of a some-what creative solution
1The challenge needs tobe more clearly statedand focused
Proposals suggest alimited number ofpossibilities
Minimally effectivedrawing and one prototypeof a possible solution
A drawing, model,portfolio, andpresentation of asolution
Activity
Purpose: Write assessment criteriaUsing: The standard and selected
enduring conceptReduce: The enduring concept into 3 -
6 component parts (sub-concepts)Enter: Those sub-concepts across the
top of the rubric
Enduring Concept
Sub-
LevelSub-concept Sub-concept Sub-concept Sub-concept
3
2
1
Enduring Concept
Sub-concepts
Levels
Sub-concept Sub-concept Sub-concept Sub-concept
3
2
1
Assessment Criteria and Evidence Gathering
Assessment criteria help to determine if a student understands
Evidence is how a student demonstrates understanding, e.g., performs on a test
In all cases, make it clear to the student what is expected and measure accordingly
Embed the assessment into learning activities
Problem Solving/Optimization
Element
Levels
Defining theChallenge
Proposing Solutions Testing Possibilities Applying the Design
3The challenge is veryclear, specific, andfocused
Sketches, drawings, anddescriptions suggest awide range ofpossibilities
Effective drawings,descriptions, andprototypes of severalpossibilities
An effective drawing,model, portfolio, andpresentation of acreative and workablesolution
2The challenge issomewhat specific andfocused
Sketches, drawings, anddescriptions show some“out-of-the-box”possibilities
Somewhat effectivedrawings and prototypesof a few possibilities
A reasonably effectivedrawing, model,portfolio, andpresentation of a some-what creative solution
1The challenge needs tobe more clearly statedand focused
Proposals suggest alimited number ofpossibilities
Minimally effectivedrawing and one prototypeof a possible solution
A drawing, model,portfolio, andpresentation of asolution
Grades 9-1220 Standards and Corresponding Benchmarks
24 (estimated) Enduring ConceptsSorted into Compatible Groups then Labeled as Courses
Group of 6Enduring Concepts
Group of 6Enduring Concepts
Group of 6Enduring Concepts
Group of 6Enduring Concepts
x six x sixx six x six
OR
Engineering Course
List the Standards being AddressedSix (estimated) Enduring Concepts18-24 Sub-concepts
Technology Program
Course Examples
Middle School Model Courses Exploring Technology Invention & Innovation Technology Systems
High School Model Courses Foundations of Technology Technology Assessment Issues in Technology Engineering
Teaching Strategy
Problem solvingModeling and prototypingRealizations/products/projectsCooperative learningModular teachingOut-of-school learning
Measuring Progress
A Consortium Workshop on Assessing Students for Technological
Literacy
Bismarck, North DakotaMarch 28 & 29, 2003
Course or Unit: ________________________________________
Standard(s): __________________________________________
Grade Level: (K-2) (3-5) (6-8) (9-12) ______________________
Enduring Concepts:1. ___________________________ 3. ______________________2. ___________________________ 4. ______________________ An Enduring Concept is a large, important, profound, and lasting idea we want
students to understand. An Understanding is “the capacity to apply knowledge to new situations in
appropriate ways.” (Howard Gardner)
Sub-concepts:1. ___________________________ 3. ______________________2. ___________________________ 4. ______________________
Rubric:Enduring Concept goes here: __________________________
Sub-conceptsLevels
Target3
Draft2
Unacceptable1
Learning Activity with Embedded Assessment (Evidence Gathering):
Learning Activities
Have meaning and purposeAre selected or developed after assessment
criteria have been writtenEnable students to experience content that
will lead to understanding important concepts that are driven by standards
Enable students to “do the subject,” “do technology”
Have assessment embedded whenever possible
Center to Advance the Teaching of Technology & Science
Digital Libraryfor Technological
Literacy
It’s more than just another Web-site!
National Digital Library for Technological Literacy
Innovation Curriculum Online Resources
International Technology Education Association
Eisenhower National Clearinghouse
Innovation Curriculum Online Resources
NSF digital library collection Manages technological literacy resourcesInformed by Standards for Technological
LiteracyAvailable Winter 2002
Technology: A Definition
“Know-how that extends human capability”
A common misconception is that technology is only computers, networking, and the like
Standards for Technological Literacy:Content for the Study of Technology
Categories:Nature of TechnologyTechnology and SocietyDesignAbilities for a Technological WorldThe Designed World
Published by ITEASupported by NSF & NASAApproved by NRCEndorsed by NAE
NDLTL Project Goals
Create an electronic system to connect users with technological literacy resources.
Monitor and maintain the digital library collection.
Build and sustain the digital library.Promote digital library tools and
services to relevant professional communities.
Standards-basedMetadata
Grade LevelsK-2, 3-5, 6-8, 9-12
Nature of Technology
Technology & Society
Design
Technological Systems
Resources Essential toTechnology
Resources CurriculumCoursesActivitiesUnits
MediaTextsInternet linksGraphicsSoftware
Collection Matrix
Status of the Project
Currently loading the system using selection criteria
Pilot within the next month followed by field testing
Available January 2003Operational Fall 2003
Evidence Gathering Tools
Portfolios and JournalsScenariosOpen-ended QuestioningModels and PrototypesRealizations/Products/ProjectsObservationsDiscussions/InterviewsConcept MappingPresentations and DebatesPaper/pencil Tests
Portfolios & Journals
Similar in that they are collections of information (evidence), usually at the end of a major activity or unit
Different in that: Portfolios are collects of physical
objects, but do include writings Journals are written logs of daily or
weekly progress on a significant project
Scenarios
Stories about possibilitiesBased on data, trends, interactions,
and futures
Open-ended Questioning
Formats (Freedman, 1994) Analysis (examine variables of a situation) Comparison (examine similarities and
differences) Description (explain or report) Evaluation (collect & analyze data and make
judgments) Fiction (stories) Problem solving (define, experiment, conclude)
Models and Prototypes
Models tend to be: Idea generators Examples of possibilities Communication tools
Prototypes tend to be: Examples of a final solution
Realizations/Products/Projects
Why not just call it what it is, a project
Turning ideas into realizations Concept drawing or description Prototypes System or process to do something
Embedding assessment
Observation
Observation is seeing and moreWorks well in a laboratory setting
where process is importantUse a form to record observations
Large and small group student discussions
Teachers ask questions of students as they work
Students interview resource persons
Discussion/Interview
Concept Map
1 2b
a A B
C
Issue
Concept Mapping Example
Issue: Air pollution in an urban area as a result of heavy automotive traffic. With this being the case, then what?
Fuel efficient vehicles
Hybrid Vehicles
Mass transit
Smaller, lighter vehicles
Fuel efficient engines
Safety is compromised
More commuters choose to drive
If this, then what?
Universal Systems Model
Resources
Input Process Output
Feedback
Systems Example
Input: Need for electrical energyResource Input: An appropriate fuel, e.g.,
fossil fuel (coal)Process: Use coal to heat water to produce
steam to drive a turbine to turn a generator to produce electricity
Output: Electricity, new problems & challenges, potentially new knowledge
Feedback: Considering the outputs, what adjustments need to be made?
Tests
Paper/pencil tests still have a placeThey usually include multiple choice
and true-falseAlternative assessment usually means
alternative to paper/pencil testingAuthentic usually mean that the test
is presented in a context that exists outside of school
Course or Unit: ________________________________________
Standard(s): __________________________________________
Grade Level: (K-2) (3-5) (6-8) (9-12) ______________________
Enduring Concepts:1. ___________________________ 3. ______________________2. ___________________________ 4. ______________________ An Enduring Concept is a large, important, profound, and lasting idea we want
students to understand. An Understanding is “the capacity to apply knowledge to new situations in
appropriate ways.” (Howard Gardner)
Sub-concepts:1. ___________________________ 3. ______________________2. ___________________________ 4. ______________________
Rubric:Enduring Concept goes here: __________________________
Sub-conceptsLevels
Target3
Draft2
Unacceptable1
Learning Activity with Embedded Assessment (Evidence Gathering):
Creating an Equitable Environment
Assess against criteria, use a variety of tools, do it often, and provide feedback
Make expectations clear to studentsTreat students objectively, regardless of color,
gender, social class, or the way they talk or look
Treat all students with equal seriousnessProvide accommodations as necessaryUse out-of-school contexts that are familiar to
students
Applying Assessment Results
Improving learning and teachingReporting student achievementEvaluating program effectivenessCommunicating with stakeholdersInfluencing state and national
assessments
Assessing the Assessment Process
Using Student Assessment Standards for Technological Literacy
Student assessment is consistent with STLThe assessment purpose is clearly definedThe assessment process applies accepted
assessment principlesAssessment is authentic and contextualAssessment results are applied
