Difference between revisions of "Hypothesis Testing"
(27 intermediate revisions by the same user not shown) | |||
Line 2: | Line 2: | ||
<center>H. Kemal Ilter<br>2020</center> | <center>H. Kemal Ilter<br>2020</center> | ||
+ | |||
+ | ==Concepts== | ||
# The ''null hypothesis'' {{#tag:math|H_0}} is a claim about the value of a population parameter. The ''alternate hypothesis'' {{#tag:math|H_1}} is a claim opposite to {{#tag:math|H_0}}. | # The ''null hypothesis'' {{#tag:math|H_0}} is a claim about the value of a population parameter. The ''alternate hypothesis'' {{#tag:math|H_1}} is a claim opposite to {{#tag:math|H_0}}. | ||
Line 7: | Line 9: | ||
# A ''test statistic'' is a function of the sample data on which the decision is to be based. | # A ''test statistic'' is a function of the sample data on which the decision is to be based. | ||
# A ''rejection region'' is the set of all values of a test statistic for which {{#tag:math|H_0}} is rejected. | # A ''rejection region'' is the set of all values of a test statistic for which {{#tag:math|H_0}} is rejected. | ||
− | # ''Type I error'': you reject {{#tag:math|H_0}} when {{#tag:math|H_0}} is true. {{#tag:math|P(\text{Type I error}) = P (reject | + | # ''Type I error'': you reject {{#tag:math|H_0}} when {{#tag:math|H_0}} is true. {{#tag:math|P(\text{Type I error}) = P(\text{reject }H_0 \mid H_0\text{ true}) = \alpha}}. The resulting {{#tag:math|\alpha}} is called the significance level of the test and the corresponding test is called a level {{#tag:math|\alpha}} test. We will use test procedures that give {{#tag:math|\alpha}} less than a specified level (0.05 or 0.01). |
+ | |||
+ | ==A Problem<ref>M. K. Chung's lecture notes, 2003.</ref>== | ||
+ | |||
+ | I believe that dogs are as smart as people. Assume IQ of a dog follows {{#tag:math|X_i \sim N(\mu,10^2)}}. IQ of 10 dogs are measured: 30, 25, 70, 110, 40, 80, 50, 60, 100, 60. We want to test if dogs are as smart as people by testing | ||
+ | |||
+ | <center>{{#tag:math|H_0 : \mu = 100 \text{ vs. } H_1 : \mu < 100}}.</center> | ||
+ | |||
+ | |||
+ | One reasonable thing one may try is to see how high the sample mean is. | ||
+ | |||
+ | <syntaxhighlight lang="r"> | ||
+ | > x<-c(30, 25, 70, 110, 40, 80, 50, 60, 100, 60) | ||
+ | > mean(x) | ||
+ | [1] 62.5 | ||
+ | </syntaxhighlight> | ||
+ | |||
+ | |||
+ | Since the average IQ of 10 dogs are lower than 100, one would be inclined to reject {{#tag:math|H_0}}. | ||
+ | |||
+ | Let {{#tag:math|\bar{X} }} be a test statistic and {{#tag:math|R = (−∞,90]}} to be a rejection region. Let’s compute the probability of making Type I error based on this testing procedure. Under the assumption {{#tag:math|H_0}} is true, | ||
+ | |||
+ | <center>{{#tag:math|X_i \sim N(100,10^2)}}</center> | ||
+ | |||
+ | |||
+ | Under this condition, {{#tag:math|\bar{X} \sim N(100, 10)}} and | ||
+ | |||
+ | <center>{{#tag:math|\alpha = P(\bar{X} \leq 90) }}</center> | ||
+ | |||
+ | |||
+ | <syntaxhighlight lang="r"> | ||
+ | > pnorm(90,100,sqrt(10)) | ||
+ | [1] 0.0007827011 | ||
+ | </syntaxhighlight> | ||
+ | |||
+ | |||
+ | By using this test procedure, it is highly unlikely to make Type I error. Let’s see what happens when we change the rejection region. | ||
+ | |||
+ | When {{#tag:math|R = (−∞,95], \alpha = P(\bar{X} \leq 95) }}. | ||
+ | |||
+ | <syntaxhighlight lang="r"> | ||
+ | > pnorm(95,100,sqrt(10)) | ||
+ | [1] 0.05692315 | ||
+ | </syntaxhighlight> | ||
+ | |||
+ | |||
+ | When {{#tag:math|R = (−∞,99], \alpha = P(\bar{X} \leq 99) }}. | ||
+ | |||
+ | <syntaxhighlight lang="r"> | ||
+ | > pnorm(99,100,sqrt(10)) | ||
+ | [1] 0.3759148 | ||
+ | </syntaxhighlight> | ||
+ | |||
+ | |||
+ | The test procedure based on rejecting {{#tag:math|H_0 \text{ if } \bar{X} \leq 99}} will produce huge Type I error. | ||
+ | |||
+ | ==Decisions in Hypothesis Testing== | ||
+ | |||
+ | <small> | ||
+ | {| class="wikitable" | ||
+ | ! Decision | ||
+ | ! H<sub>0</sub> false | ||
+ | ! H<sub>0</sub> true | ||
+ | |- | ||
+ | | Reject H<sub>0</sub> | ||
+ | | Correct.<br>We can be 95% sure that we made a right decision, because in our case; ({{#tag:math|p = 1 - \alpha }}) 1 – .05 = .95. | ||
+ | | Type I error = a level, p = a, probability of error is commonly set at .05. | ||
+ | |- | ||
+ | | Fail to reject H<sub>0</sub> | ||
+ | | Type II error, b level = maximum accepted probability is suggested to be set to .20. | ||
+ | | Correct. Probability: p = 1 – b. In our case 1 – .20 = .80. We would make a right decision based on our analyses 80% of the time. | ||
+ | |} | ||
+ | </small> | ||
{{References}} | {{References}} | ||
[[Category:Blog]] | [[Category:Blog]] |
I believe that dogs are as smart as people. Assume IQ of a dog follows [math]X_i \sim N(\mu,10^2)[/math]. IQ of 10 dogs are measured: 30, 25, 70, 110, 40, 80, 50, 60, 100, 60. We want to test if dogs are as smart as people by testing
One reasonable thing one may try is to see how high the sample mean is.
> x<-c(30, 25, 70, 110, 40, 80, 50, 60, 100, 60)
> mean(x)
[1] 62.5
Since the average IQ of 10 dogs are lower than 100, one would be inclined to reject [math]H_0[/math].
Let [math]\bar{X} [/math] be a test statistic and [math]R = (−∞,90][/math] to be a rejection region. Let’s compute the probability of making Type I error based on this testing procedure. Under the assumption [math]H_0[/math] is true,
Under this condition, [math]\bar{X} \sim N(100, 10)[/math] and
> pnorm(90,100,sqrt(10))
[1] 0.0007827011
By using this test procedure, it is highly unlikely to make Type I error. Let’s see what happens when we change the rejection region.
When [math]R = (−∞,95], \alpha = P(\bar{X} \leq 95) [/math].
> pnorm(95,100,sqrt(10))
[1] 0.05692315
When [math]R = (−∞,99], \alpha = P(\bar{X} \leq 99) [/math].
> pnorm(99,100,sqrt(10))
[1] 0.3759148
The test procedure based on rejecting [math]H_0 \text{ if } \bar{X} \leq 99[/math] will produce huge Type I error.
Decision | H0 false | H0 true |
---|---|---|
Reject H0 | Correct. We can be 95% sure that we made a right decision, because in our case; ([math]p = 1 - \alpha [/math]) 1 – .05 = .95. |
Type I error = a level, p = a, probability of error is commonly set at .05. |
Fail to reject H0 | Type II error, b level = maximum accepted probability is suggested to be set to .20. | Correct. Probability: p = 1 – b. In our case 1 – .20 = .80. We would make a right decision based on our analyses 80% of the time. |
References
hkilter.com by H. K. Ilter is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License
© 2020 H. K. Ilter