Assessing for College and Career Readiness:

Higher Order Thinking Skills (HOTS)

Teaching and Assessing Higher Order Thinking Skills for K-12 TeachersDAY THREE: September 27, 2012Follow-up Session 2 (Online)Personalized, Domain-specific HOTSComments to, and from othersTopic(s) for Performance-based HOTS-aligned Projects

Todays emphasis: Instructional StrategiesCritical ThinkingProblem-SolvingAnalytic ReasoningPre-requisites (Lower Order Thinking Skills)The mastery of content and lower order thinking are particularly important prerequisites to higher order thinking. Any lesser degree of learning of prerequisites will result in puzzlement, delay, inefficient trial and error at best, and in failure, frustration, or termination of effort at worst

- Gagne, Briggs and Wagner, 1988To do Higher-Order Thinking, students must have something to think ABOUT3Pre-requisites (Lower Order Thinking Skills)This division of basic facts from higher-order thinking runs against common sense. How middle schoolers may apprehend historical thinking without delving into the factual details of another time and place far from their own, is a mystery.

- Mark Bauerlein, The Dumbest Generation, 2009Pre-requisitesLower Order Thinking Skills

cognitive strategies comprehension concept classification discriminations routine rule using simple analysis simple application

Examples: Underlining main ideas, outlining, paraphrasing, mnemonic devices to recall informationHigher Order Thinking Skills complex analysis creative thinking critical thinking decision making evaluation logical thinking meta-cognitive thinking problem solving reflective thinking scientific experimentation scientific inquiry synthesis systems analysisPre-requisitesIt is important that students acquire sufficient declarative knowledge (e.g., concepts and facts) and procedural knowledge (e.g., strategies and algorithms) within a specific content domain before they can be expected to engage in complex reasoning processes.

-- Marzano, 1992What Higher-Order Thinking is NOT:HOTS are NOT simply an extensive collection of Lower-Order Thinking Skills.

Consider an Example (Science):Essential Question: Why do some substances burn, and other substances do not?Not Necessarily Higher-Order Thinking

Barriers to Higher-Order Thinking

Higher-Order Thinking and Reading in the Content AreaAntoine-Laurent Lavoisier(Adapted from The Ten Most Beautiful Experiments, by George Johnson, Alfred A. Knopf, 2008)Why do some substances burn, while other substances do not? In the 1700s, Antoine Lavoisier and most other chemists accepted the notion that some kinds of matter burned because of a mysterious substance called phlogiston (flow-ji-ston). The reason things burned was that they were rich in phlogiston, and as they were consumed they released this fire stuff into the air. Set a piece of wood aflame and it would stop burning only when its phlogiston was spent, leaving behind a pile of ash. Wood, it logically followed, was made of phlogiston and ash. Likewise, heating a metal under an intense flame, a process called calcination, left a whitish brittle substance, or calx. Metal was thus composed of phlogiston and calx. The process also worked the other way around. Calx, it was recognized, resembled the crude ores mined from the ground, which were refined or reduced by heating them next to a piece of charcoal. The charcoal emitted phlogiston, which combined with the calx to recover the lustrous metal.

With phlogiston, scientists had a consistent explanation for combustion, calcination, and reduction. Chemistry suddenly made sense. There was however a problem: the calx left behind after calcination weighed more than the original metal. How could removing phlogiston leave something heavier?

Developmental Progression:Lower Elementary Grades K-2Upper ElementaryGrades 3-5Middle SchoolGrades 6-8High SchoolGrades 9-12Common Theme: Argumentation(From our Previous Example an exit outcome):

Students should be able to evaluate arguments, identifying the strengths and weaknesses of claims in light of the evidence and reasoning used to support them. Students should also be able to construct valid arguments, and support them with evidence.Developmental Progression:ArgumentationYoung students can begin by constructing an argument for their own interpretation of the phenomena they observe and of any data they collect. They need support to go beyond simply making claimsthat is, to include reasons or references to evidence and to begin to distinguish evidence from opinion. As they grow in their ability to construct arguments, students can draw on a wider range of reasons or evidence, so that their arguments become more sophisticated. In addition, they should be expected to discern what aspects of the evidence are potentially significant for supporting or refuting a particular argument. Students should begin learning to critique by asking questions about their own findings and those of others. Later, they should be expected to identify possible weaknesses in either the data or an argument and explain why their criticism is justified. As they become more adept at arguing and critiquing, they should be introduced to the language needed to talk about argument, such as claim, reason, data, etc.

Framework for K-12 Science Education, National Academy of SciencesDevelopmental Progression:ArgumentationLower Elementary (Grades K-2): Students should

Know that people are more likely to believe your ideas if you can give good reasons for them.

Ask How do you know? when appropriate, and to attempt reasonable answers when others ask them the same question.

Adapted from Benchmarks for Science Literacy, AAASDevelopmental Progression:ArgumentationUpper Elementary (Grades 3-5): Students should

Support their statements with facts found in books, articles and other sources.Seek better reasons for believing something than Everybody knows that, or Just because.Recognize when comparisons or analogies used in an argument might not be correct or fair.e.g. an electric wire is like a garden hose, Earth is like a peach

Adapted from Benchmarks for Science Literacy, AAASDevelopmental Progression:ArgumentationMiddle School (Grades 6-8): Students should

Be aware that there may be more than one good way to interpret a given set of facts.Be skeptical of arguments based on small samples of data, biased data, or vague reasons.Criticize arguments where fact and opinion are intermingled, or where conclusions dont logically follow from the evidence.

Adapted from Benchmarks for Science Literacy, AAASDevelopmental Progression:ArgumentationHigh School (Grades 9-12): Students should

Criticize arguments based on faulty, incomplete, or misleading use of evidence.Know that convincing arguments need true supporting statements and valid connections between them.Identify the hidden assumptions behind an argument to assess its validity.Suggest alternative ways of interpreting the evidence in an argument.

Adapted from Benchmarks for Science Literacy, AAAS

High-Impact Instructional StrategiesThe 5-E Model of Inquiry applied to Critical ThinkingClaim/Evidence/Reasoning applied to Problem-SolvingAdditional Strategies applied to Argumentation and Analytic ReasoningHigh-Impact Instructional StrategiesInquiry focuses on the engagement of students to generate and pursue the answers to questions through careful observation and reflection. It is a multifaceted activity that involves making observations; posing questions; examining other sources of information to see what is already known in light of experimental evidence: using tools to gather, analyze, and interpret data; proposing answers, explanations, and predictions; and communicating the results. Inquiry requires identification of assumptions, use of critical and logical thinking, and considerations of alternative explanations. It is far more flexible than, for example, the rigid sequence of steps commonly depicted in textbooks as the scientific method. It is much more than just doing experiments.

Llewellyn, Teaching Science Through Inquiry, NSTA Press 2005The 5E Learning CycleLearning involves making sense of 1) prior experiences, and 2) new first-hand explorations.EngageExploreExplainElaborateEvaluateThe 5E Learning CycleEngage: A scientist has made the claim, For all types of animals, an average lifetime is about one billion heartbeats.

Suppose you wanted to test this idea. Which questions need to be answered? What data do you need?

The 5E Learning CycleEngage: For all animals, a lifetime is about one billion heartbeats.Exploring the Data:AnimalWeight (kg)Avg. Heart Rate (beats/minute)Avg. Life SpanCat215015Dog59015Hamster0.064503Chicken1.527515Monkey519015Horse1,2004440Cow8006522Pig1507025Rabbit12059Elephant50,0003070Giraffe9006520Whale120,0002080What do you notice? What patterns or relationships do you observe? How could you best represent these patterns or relationships?22The 5E Learning Cycle

The 5E Learning Cycle

The 5E Learning Cycle

The 5E Learning CycleAnimalWeight (kg)Avg. Heart Rate (beats/minute)Avg. Life SpanAvg. Heart Beats per Lifetime (billions)Human9060702.21Cat2150151.18Dog590150.71Hamster0.0645030.71Chicken1.5275152.17Monkey5190151.50Horse1,20044400.93Cow80065220.75Pig15070250.92Rabbit120590.97Elephant50,00030701.10Giraffe90065200.68Whale120,00020800.84Black Bear16055250.72Galapagos Tortoise26061770.56Hummingbird0.01126053.31Frog0.052230.035Instructional Strategies: Claim Evidence Reasoning (CER):Claim: An assertion or conclusion that addresses the original question or problem.Evidence: The facts, data, measurements or calculations from an investigation that support the claim.Appropriate: The evidence is relevant to the question or problem; andSufficient: There should be enough evidence to convince others of the claim.Reasoning: Link the evidence to the claim, explain why the evidence supports the claim.Instructional Strategies: C-E-R and Problem SolvingExample: A Disease Outbreak (Huang & Bayona, 2004)

Background:

An outbreak (epidemic) of gastroenteritis occurred in Greenport, a suburban neighborhood, on the evening of April 28. A total of 89 people went to the emergency departments of the three local hospitals during that evening. No more cases were reported afterward. These patients complained of headache, fever, nausea, vomiting and diarrhea. The disease was severe enough in 19 patients to require hospitalization for rehydration. Gastroenteritis outbreaks like this are usually caused by the consumption of a contaminated or poisoned meal. Meal contamination can often be caused by pathogenic viruses or bacteria. However, acute outbreaks are more often produced by toxins from bacteria such as Staphylococcius spp., Clostridium perfringens, Salmonella spp. and Vibrio cholerae. Food poisoning can also be caused by chemicals or heavy metals, such as copper, cadmium or zinc, or by shellfish toxins.Instructional Strategies: Problem SolvingExample: A Disease Outbreak (Huang & Bayona, 2004)

Identify the problem to be solved, the goal to be reached, or the conclusion to be drawn:

Identify the source of contamination that caused the outbreak of gastroenteritis (you will present your claim, evidence and reasoning); andSuggest a plan to prevent future outbreaks.Instructional Strategies: Problem SolvingExample: A Disease Outbreak (Huang & Bayona, 2004)

Inquire: What are some initial questions you have?1.2.3.

Develop a plan: What are the steps you will take to find answers to your questions?1.2.3.

Instructional Strategies: Problem SolvingExample: A Disease Outbreak

Instructional Strategies: Problem SolvingExample: A Disease Outbreak

Instructional Strategies: Problem SolvingExample: The Epidemic Curve

From what you know about the human digestive system, when was the source meal eaten? e.g. Staphylococcus aureus poisoning is due to a toxin that is pre-formed in food before it is eaten. It causes vomiting within 1 to 6 hours after eating the contaminated food. 33Instructional Strategies: Problem SolvingExample: The epidemic team investigated the places where affected persons, their relatives and neighbors ate that day.

Instructional Strategies: Problem SolvingExample: The epidemic team investigated the places where affected persons, their relatives and neighbors ate that day.

Instructional Strategies: Problem SolvingExample: The epidemic team investigated the places where affected persons, their relatives and neighbors ate that day.

Instructional Strategies: Problem SolvingExample: Once the implicated place was determined, the investigation centered on the food. The following table includes the food items served in that place April 28:

Instructional Strategies: Problem SolvingExample: Once the implicated place was determined, the investigation centered on the food. The following table includes the food items served in that place April 28:

Instructional Strategies: Problem SolvingNone of the kitchen personnel were ill. The names of the kitchen personnel and their participation in the food preparation are as follows: Manuel prepared the beef burritos and the potatoes, John prepared the salad and the fruit, Sally served all dishes except the ice cream, and Jane prepared the cheeseburgers and served the ice cream. The ice cream was a commercial brand and was bought at a nearby supermarket.

Given that the epidemic team worked fast enough and the implicated meal(s) was(were) identified before all food leftovers were discarded, food samples from some meal leftovers were taken to the laboratory. In addition, stool samples were taken from the kitchen personnel who prepared or handled each different food item. The laboratory confirmed that Salmonella toxin was present in some of the food samples and that one of the kitchen personnel had the same Salmonella species. Furthermore, the Salmonella species found in the food and the kitchen worker was the same species found in stool samples of the patients.Instructional Strategies: Problem Solving

Instructional Strategies: Claim Evidence Reasoning (CER):Claim: An assertion or conclusion that addresses the original question or problem.Evidence: The facts, data, measurements or calculations from an investigation that support the claim.Appropriate: The evidence is relevant to the question or problem; andSufficient: There should be enough evidence to convince others of the claim.Reasoning: Link the evidence to the claim, explain why the evidence supports the claim.Methods and Strategies to Enhance Higher Order Thinking SkillsClear Instructions and Explanations;Modeling of thinking skills;Examples of Applied Thinking;Feedback on student thinking processes;Transitions (scaffolding) from initial support (e.g. teaching an algorithm specified set of steps for problem-solving) to student-centered learning (e.g. heuristics widely applicable problem solving strategies).Combination of Direct Instruction with Guided PracticeEffective Questioning Strategies pose paradoxes, dilemmas, novel problemsFrom King, Goodson & Rohani, 1998Instructional Strategies:Analytic Reasoning - ArgumentationAn argument is a collective series of statements to establish a definite proposition

http://www.youtube.com/watch?v=yTl9zYS3_dc Instructional Strategies:Analytic Reasoning - ArgumentationArgument: A set of sentences such that

one of them is said to be true (the conclusion); andthe others are being offered as reasons (premises) for believing the truth of the conclusion.

Words as clues: if, then, since, therefore,Instructional Strategies:Analytic Reasoning - ArgumentationArguments: Examples Syllogisms (3-part)Identify the premises and conclusions

All men are mortal.Socrates is a man.Therefore, Socrates is mortal.

Saginaw is north of Detroit.Detroit is north of Toledo.Therefore, Saginaw is north of Toledo.

Instructional Strategies:Analytic Reasoning - ArgumentationArguments: Examples Syllogisms (3-part)Identify the premises and conclusions

All men are mortal. (premise)Socrates is a man. (premise)Therefore, Socrates is mortal. (conclusion)

Saginaw is north of Detroit. (p)Detroit is north of Toledo. (p)Therefore, Saginaw is north of Toledo. (c)

Instructional Strategies:Analytic Reasoning - ArgumentationArguments: Examples Syllogisms (3-part)Identify the premises and conclusions

John is a bachelor.John is single.All bachelors are single.

All apples grow on trees.All fruits grow on trees.All apples are fruits.

Instructional Strategies:Analytic Reasoning - ArgumentationArguments: Examples Syllogisms (3-part)Identify the premises and conclusions

John is a bachelor. (P)John is single. (C)All bachelors are single. (P)

All apples grow on trees. (C)All fruits grow on trees. (P)All apples are fruits. (P)

Instructional Strategies:Analytic Reasoning - ArgumentationArguments: Examples (not stated as syllogisms)Identify the premises and conclusions

The members of a Smith family are Aaron, Bob and Char. Aaron is left-handed. Bob is left-handed. Char is left-handed. Therefore, all members of the Smith family are left-handed.

Im expected at the office at 8:00 am each weekday. Since my drive takes half an hour, if I leave home by 7:30 am, I should make it to work on time.

Instructional Strategies:Analytic Reasoning - ArgumentationArguments:

Arguments are neither true nor false;Premises, or assertions, are either true or false;Arguments are either strong or weakA strong argument:All the premises are true; andThe conclusion follows from the premises.A good argument preserves truth.Instructional Strategies:Analytic Reasoning - ArgumentationTypes of Arguments:

Deductive: The truth of the premises guarantees the truth of the conclusion.Inductive: The truth of the premises makes the truth of the conclusion more probable.Inductive arguments rely on the assumption that the future will be like the past (the uniformity of nature).Instructional Strategies:Analytic Reasoning - ArgumentationTypes of Arguments: Which are Deductive, and which are Inductive?

Susan is either a Democrat or a Republican. Since shes not a Democrat, she must be a Republican.It has snowed in Michigan every December in recorded history. Therefore, it will snow in Michigan this December.Dom Perignon is a champagne, so it must be made in France.Instructional Strategies:Analytic Reasoning - ArgumentationTypes of Arguments: Which are Deductive, and which are Inductive?

Susan is either a Democrat or a Republican. Since shes not a Democrat, she must be a Republican.(Deductive)It has snowed in Michigan every December in recorded history. Therefore, it will snow in Michigan this December.(Inductive)Dom Perignon is a champagne, so it must be made in France.(Inductive)Instructional Strategies:Analytic Reasoning - ArgumentationArguments: Validity vs. Truth When an argument is valid, then if its premises are true, we can be certain that its conclusion is also true.

Valid arguments may contain false premises:All birds can fly. All fish live in the ocean.Penguins are birds. Whales are fish.Therefore, penguins can fly. Whales live in the ocean.Instructional Strategies:Analytic Reasoning - ArgumentationArguments: Using a Truth TableIs the Argument Strong?True PremisesFalse PremisesValid Argument

STRONGWEAKInvalid Argument

WEAKWEAKInstructional Strategies:Analytic Reasoning - ArgumentationTypes of Inductive Arguments:

Arguments from GeneralizationCausal ArgumentsArguments from AnalogyArguments from Authority

All of these arguments CAN be valid and strong. But we must consider some questions.Instructional Strategies:Analytic Reasoning - ArgumentationArguments from Generalization:

My Father-in-Law bought a car in Florida and got a great deal. Florida must be a great place to buy cars!

Helen is the Republican candidate, so she will be opposed to gun control.

Questions to pose:How large was the sample size?How representative was the sample?Are there any counterexamples?Instructional Strategies:Analytic Reasoning - ArgumentationCausal Arguments:

Married people live longer than single people. So being married causes you to live longer.

I washed my car yesterday, so itll probably rain today.

Questions to pose:How strong is the correlation?Does the causal relationship make sense?What causes what?Instructional Strategies:Analytic Reasoning - ArgumentationArguments from Analogy:

The universe is like a pocket-watch. Pocket watches have designers. Therefore the universe must have a designer. (William Paley, 18th century)

Questions to pose:How similar are the two things being compared?Are there any counter-examples?

Instructional Strategies:Analytic Reasoning - ArgumentationArguments from Authority:

According to a research study, 93% of scientists favor the use of live animals in scientific research.

Questions to pose:How were the data gathered? How were the questions worded?Who are the experts? Are they qualified in the appropriate field?Are these experts impartial? Are there any potential sources of bias?Are there any counter-examples (experts with contrary opinions)?

Instructional Strategies:Analytic Reasoning - ArgumentationLogical Fallacies:Slippery SlopeBlack or WhiteAd hominem / GeneticBandwagonAppeal to NatureAppeal to EmotionCherry-picking DataMiddle GroundEtc

Instructional Strategies:Analytic Reasoning - ArgumentationIssue: Hydraulic Fracturing

Tip-of-the-Mitt Watershed Council and National Wildlife Federation

vs.

EnergyFromShale.orgInstructional Strategies:Analytic Reasoning - ArgumentationEvaluating Arguments:

Conclusion: _______________Premise #1 : _______________Premise #2 : _______________Hidden Premise (inference) #1 : ______________Hidden Premise (inference) #2 : ______________Instructional Strategies:Analytic Reasoning - ArgumentationEvaluating Arguments: Questions

What general types of arguments are presented?Inductive or Deductive?Arguments from Generalization, Authority, Causation, Analogy, other?What additional questions need to be asked (and answered)?Do you observe any potential Logical Fallacies?How strong is the argument?Are the premises accurate?Is there a valid connection between premises and conclusion?Methods and Strategies to Enhance Higher Order Thinking SkillsClear Instructions and Explanations;Modeling of thinking skills;Examples of Applied Thinking;Feedback on student thinking processes;Transitions (scaffolding) from initial support (e.g. teaching an algorithm specified set of steps for problem-solving) to student-centered learning (e.g. heuristics widely applicable problem solving strategies).Combination of Direct Instruction with Guided PracticeEffective Questioning Strategies pose paradoxes, dilemmas, novel problemsFrom King, Goodson & Rohani, 1998END OF DAY THREESession 4: OnlineDevelop HOTS-aligned Performance-based Project(s)Identify the standard(s) to which the project is aligned;Use some of the sample High-Impact Instructional Strategies weve seen today.Provide feedback/critique of other participants posted Project(s)Begin thinking about how the student work from your project(s) will be evaluated and scored.END OF DAY THREESession 5: Face-to-face October 25th 4:30-7:30 pm

Assessing HOTS with Scoring RubricsExamining Student Work