The Green Lab - [07-B] Hypothesis Testing

Vrije Universiteit Amsterdam

Hypothesis: a formal statement about a phenomenon

● Null hypothesis H0

: no real trends or patterns in the

experiment setting (aka ~Q)

● Alternative hypothesis Ha: there are real trends or

patterns in the experiment setting (aka Q)


● We aim at verifying the null hypothesis (~Q)○ we test the null hypothesis H

0

We can reject the null hypothesis → we can draw conclusions

This comes from Popper (1959): any statement in a scientific

field is true until anybody can contradict it

● Aiming at verifying Q is WRONG○ Provides no insight on the conjecture


●

●

●



●○

"P-value in statistical significance testing" by User:Repapetilto @ Wikipedia & User:Chen-Pan Liao @ Wikipedia - https://en.wikipedia.org/wiki/File:P_value.png. Licensed under CC BY-SA 3.0 via Wikimedia Commons


●

●


●○

○ α

●○

○ β



●○ β

●



● α○ α○

● β○○

●


●

●

●○○

■ α


●

●

○

●○○


α

0.38


α

● α○

●

0.38

α


●○

●○○

●○


●

●


●

○

●


● Analysis and Interpretation: Chapter 10

Experimentation in SE, Wohlin et al. (Blackboard)

● Hypothesis Testing: Chapter 10, Section 3

Experimentation in SE, Wohlin et al. (Blackboard)

● "It's the effect size, stupid" (Coe, R.)

● Plenty of online resources\tutorials

https://www.google.com/url?q=http://www.leeds.ac.uk/educol/documents/00002182.htm&sa=D&ust=1477923780021000&usg=AFQjCNE2IKPu3ibxnZCd9eSuMZBn-WcH_Q

https://www.google.com/url?q=http://www.leeds.ac.uk/educol/documents/00002182.htm&sa=D&ust=1477923780021000&usg=AFQjCNE2IKPu3ibxnZCd9eSuMZBn-WcH_Q


● Coin toss example and other content from Empirical

Methods in Software Engineering, Marco Torchiano,

Politecnico di Torino - http://softeng.polito.it/EMSE/

● Effect Size from Basics of Empirical Software Engineering,

Massimiliano Di Penta, University of Sannio, Italy

https://www.google.com/url?q=http://softeng.polito.it/EMSE/&sa=D&ust=1477923780364000&usg=AFQjCNF0pM3JKCTF5zhQW30YZSHTtIAIIA


[email protected]

[email protected]

mailto:[email protected]




Education

The Green Lab - [07-B] Hypothesis Testing