Upload
raymond-hunter
View
217
Download
0
Embed Size (px)
Citation preview
CPSC 699
Scientific Evidence
Message
Computer Science research requires scientific evidence and well design research plan even in the presence of uncertainty
Lecture Outline
History of algorithms Problem solving Ways to deal with uncertainty:
assumptions, paradoxesScientific evidenceExperimental design
Algorithm Definition
Algorithm is a procedure that consists of a finite set of instructions which, given an input from some set of possible inputs, enable to obtain an output
Donald Knuth: “Computer Science is the Study of Algorithms”
Some simple examplesSuppose you and I have the same amount of
money. How much should I give you so that you have $10 more than I do?
A dealer bought a book for $7, sold for $8, bought for $9, sold for $10. How much profit did he make?
Solution provides an algorithm.
AssumptionsEach problem can be solved under a set of specific
assumptionsExamples:
Weight is given in kilogramsActivity takes place on Earth (Newton’s laws apply)When pouring the water into container, it does not
evaporate
“Which one is older?”Logical problem (Raymond Smullyan) A brother and a sister were asked who is older. The
brothers said “I am older”. The sister said “I am younger”. At least one of them lied. Who is older? Under what assumption the solution works?
Solution
If we assume they cannot be of the same age, then they both lied and the sister is older.
A pizza puzzle
How do you cut a pizza onto 8 pieces with 3 cuts. What is the minimum number of slices you can get with 3 cuts? Which assumptions did you use?
SolutionA piece is not necessarily should have a
cheese on it (be the top one).You can cut horizontally.All cuts are distinct.
Logical problemsR. Smullyan “The riddle of Scheherazade”1. Think of a yes/no question that will forceyou to tell the truth.
2. Think of a yes/no question that will forceyou to tell the truth.
SolutionsSolution 1. Will you answer “Yes” to this question?Solution 2. Will you answer “No” to this question?The first solution is obtained as a result of cognitive
process.The second solution can be obtained by analogy.
Problem solving
Applying specific rulesBy analogy (similar problem)Abstraction (generalization)Intuition (non-ordinary solution)
The biggest assumption“Math is perfect??”In 1931 Kurt Godel came up with a startling discovery
that mathematical truth can not be completely formalized. I.e. given the set of axioms and rules of inference there are some statements that cannot be proven true or false inside this system.
This is called: Incompleteness theorem
ParadoxesThis leads to paradoxes:“This sentence is false”“You have no reason to believe this sentence”Do you believe it?
How do you know?Many ways we come
to believe thingsOften, we get to
wrong conclusionsLook at
Gilovich, How we know what isn’t so, 1991.
Huff, How to lie with statistics, rev 1993.
Campbell and Stanley, Experimental and quasi-experimental designs for research, 1963.
Polya, How to Solve it, 1945
Cognitive People often see patterns in random data
“images in the clouds”we then mistakenly assume the pattern is
significantwe are also good at explaining patterns we
believe are there.
The hot handMany basketball fans and players believe in
the hot hand shooting success comes in streaks
No statistical evidenceConsider the sequence
OXXXOXXXOXXOOOXOOXXOOrandom62% of subject believe shows streaks
Illusion of validity
Anecdotal evidence –seen in TV ads“I followed the new x
diet and lost 25 lbs.”need more
informationlook at all four
quadrants
Lost Weight
anecdotal
evidence in
advertising
no lostweight
no diet diet
CognitiveSeeing what we expect to see
biased evaluationlike evidence/research that supports what we
already believeExamples
scrutinizing contrary evidence more closelygamblers’ rationalization of lossespolarizes people of opposing viewpoints (eg.,
capital punishment)
Social Seeing what we want to see
motivational determinants of beliefThe Lake Wobegon effect
“… the women are strong, the men are goodlooking, and all the children are above average.” (Garrison Keillor)
Examplesthe large majority of the general publicbelieves that they are
more intelligentmore fair-mindedless prejudicedmore skilled at driving than the average person
Social Believing what we are told
the biasing effects of second hand informationWhere there is smoke there is fire?
if you hear it often enough, it must be trueAlterations in recounted stories
it was this big
How to use statisticsHuff, “How to Lie with Statistics”, Norton,
New York, 1954.Who says so?How does he know?What is missing?Did somebody change the subject?Does it make sense?
examples (1)U of C 101
Higher Grades have been achieved by students who attended U of C 101 according to the research. Overall GPA of frosh who did not attend U of C 101: 2.26 Overall GPA of frosh who did attend U of C 101: 2.68
Can we know anything?
Yes, withExperimentation plus statisticsCampbell and Stanley show how to design
experiments that have validity
Experiment designOne-shot case study
X One-group pretest-posttest design
O X OPre-test-post-test control group design
R O X OR O O
ValidityInternal
does an experiment give a valid answer to the problem posed? How many times and in which settings?
Externalcan the results of an experiment transfer to
other contexts?Uncertainty
do the results have anything to do with conditions that occur in the real world?
3 research questionsEarly
what will happen if …?
what happens when …?
Middledoes this predict
what will happen?End
what causes this to happen
i.e., a causal explanation
Experiments and research questionsexperiments that do not have an underlying research
question areface value experimentsanything learned about a research question is
incidentalOne should try to have experiment address
underlying research question and critically compare results: under various settings, with variable size data sets, with different algorithms
Case studies
Where ideas come fromInternal Talk to other grads in your lab Get demonstrations of what they are doing Experiment with any software that's built by
others in your group so you get your hands dirty in the area (e.g., in the lab or downloadable from the net) that let you get your hands dirty in the area
External Read lots - conferences, journals, etc. Mark the articles you like so you can go back and
review it. Or copy the abstract and put it in a binder for later review.
Attend a key conference in your area if you can
Good area choiceHow do you decide what is a good area? Talk to your supervisor first. Some have specialized
areas of interests and focused projects in mind that they will want you to work on. Others may be willing to consider projects outside their direct interest.
It should be personally exciting and interesting to you. You will be working on this for a long time.
It should be rich in scope It should be topical i.e., something of relevance It should be related to your supervisor's experiences
Once you found an areaResearch the area to gain exposure to it. In particular,
collect all interesting material you find in a binder. photocopy /save good articlesmaintain a list of full references of these articles as a
bibliography better yet, create an annotated bibliography where
you briefly note what is interesting about them keep a folder or sketchbook or notebook where you
can collect ideas or snippets related to this area maintain a list of links to interesting web sites
Message
Computer Science research requires scientific evidence and well design research plan even in the presence of uncertainty
SourcesHistory of Mathematicshttp://www-groups.dcs.st-andrews.ac.uk/~history/Chronology/index.htmlFun Math http://www.cut-the-knot.com/content.html
R. Smullyan “The riddle of Scheherazade”Gilovich, How we know what isn’t so,
1991.Huff, How to lie with statistics, rev
1993.