Hypothesis Testing

H. Kemal Ilter
2020

Concepts

1. The null hypothesis [math]H_0[/math] is a claim about the value of a population parameter. The alternate hypothesis [math]H_1[/math] is a claim opposite to [math]H_0[/math].
2. A test of hypothesis is a method for using sample data to decide whether to reject [math]H_0[/math]. [math]H_0[/math] will be assumed to be true until the sample evidence suggest otherwise.
3. A test statistic is a function of the sample data on which the decision is to be based.
4. A rejection region is the set of all values of a test statistic for which [math]H_0[/math] is rejected.
5. Type I error: you reject [math]H_0[/math] when [math]H_0[/math] is true. [math]P(\text{Type I error}) = P(\text{reject }H_0 \mid H_0\text{ true}) = \alpha[/math]. The resulting [math]\alpha[/math] is called the significance level of the test and the corresponding test is called a level [math]\alpha[/math] test. We will use test procedures that give [math]\alpha[/math] less than a specified level (0.05 or 0.01).

A Problem

I believe that dogs are as smart as people. Assume IQ of a dog follows [math]X_i \sim N(\mu,10^2)[/math]. IQ of 10 dogs are measured: 30, 25, 70, 110, 40, 80, 50, 60, 100, 60. We want to test if dogs are as smart as people by testing

[math]H_0 : \mu = 100 \text{ vs. } H_1 : \mu \lt 100[/math].

One reasonable thing one may try is to see how high the sample mean is.

> x<-c(30, 25, 70, 110, 40, 80, 50, 60, 100, 60)
> mean(x)
[1] 62.5

Since the average IQ of 10 dogs are lower than 100, one would be inclined to reject [math]H_0[/math].

Let [math]\bar{X} [/math] be a test statistic and [math]R = (−∞,90][/math] to be a rejection region. Let’s compute the probability of making Type I error based on this testing procedure. Under the assumption [math]H_0[/math] is true,

[math]X_i \sim N(100,10^2)[/math]

Under this condition, [math]\bar{X} \sim N(100, 10)[/math] and

[math]\alpha = P(\bar{X} \leq 90) [/math]

> pnorm(90,100,sqrt(10))
[1] 0.0007827011

By using this test procedure, it is highly unlikely to make Type I error. Let’s see what happens when we change the rejection region.

When [math]R = (−∞,95], \alpha = P(\bar{X} \leq 95) [/math].

> pnorm(95,100,sqrt(10))
[1] 0.05692315

When [math]R = (−∞,99], \alpha = P(\bar{X} \leq 99) [/math].

> pnorm(99,100,sqrt(10))
[1] 0.3759148

The test procedure based on rejecting [math]H_0 \text{ if } \bar{X} \leq 99[/math] will produce huge Type I error.

Decisions in Hypothesis Testing

Decision	H0 false	H0 true
Reject H0	Correct, [math]p = 1 - \alpha [/math] p = 1 – a. We can be 95% sure that we made a right decision, because p = 1 – a. In our case: 1 – .05 = .95.	Type I error = a level, p = a, probability of error is commonly set at .05.
Fail to reject H0	Type II error, b level = maximum accepted probability is suggested to be set to .20.	Correct. Probability: p = 1 – b. In our case 1 – .20 = .80. We would make a right decision based on our analyses 80% of the time.

---

References