Hypothesis Test for the Difference of Two Population Proportions

In this article we will go through the steps necessary to perform a hypothesis test, or test of significance, for the difference of two population proportions. This allows us to compare two unknown proportions and infer if they are not equal to each other or if one is greater than another.

Hypothesis Test Overview and Background

Before we go into the specifics of our hypothesis test, we will look at the framework of hypothesis tests. In a test of significance we attempt to show that a statement concerning the value of a population parameter (or sometimes the nature of the population itself) is likely to be true. 

We amass evidence for this statement by conducting a statistical sample. We calculate a statistic from this sample. The value of this statistic is what we use to determine the truth of the original statement. This process contains uncertainty, however we are able to quantify this uncertainty

The overall process for a hypothesis test is given by the list below:

1. Make sure that the conditions that are necessary for our test are satisfied.
2. Clearly state the null and alternative hypotheses. The alternative hypothesis may involve a one-sided or a two-sided test. We should also determine the level of significance, which will be denoted by the Greek letter alpha.
3. Calculate the test statistic. The type of statistic that we use depends upon the particular test that we are conducting. The calculation relies upon our statistical sample. 
4. Calculate the p-value. The test statistic can be translated into a p-value. A p-value is the probability of chance alone producing the value of our test statistic under the assumption that the null hypothesis is true. The overall rule is that the smaller the p-value, the greater the evidence against the null hypothesis.
5. Draw a conclusion. Finally we use the value of alpha that was already selected as a threshold value. The decision rule is that If the p-value is less than or equal to alpha, then we reject the null hypothesis. Otherwise we fail to reject the null hypothesis.

Now that we have seen the framework for a hypothesis test, we will see the specifics for a hypothesis test for the difference of two population proportions. 

The Conditions

A hypothesis test for the difference of two population proportions requires that the following conditions are met: 

• We have two simple random samples from large populations. Here "large" means that the population is at least 20 times larger than the size of the sample. The sample sizes will be denoted by n₁ and n₂.
• The individuals in our samples have been chosen independently of one another. The populations themselves must also be independent.
• There are at least 10 successes and 10 failures in both of our samples.

Read More

Confidence Interval for the Difference of Two Population Proportions

By Courtney Taylor

As long as these conditions have been satisfied, we can continue with our hypothesis test.

The Null and Alternative Hypotheses

Now we need to consider the hypotheses for our test of significance. The null hypothesis is our statement of no effect. In this particular type of hypothesis test our null hypothesis is that there is no difference between the two population proportions. We can write this as H₀: p₁ = p₂.

The alternative hypothesis is one of three possibilities, depending upon the specifics of what we are testing for: 

• H_a: p₁ is greater than p₂. This is a one-tailed or one-sided test.
• H_a: p₁ is less than p₂. This is also one-sided test.
• H_a: p₁ is not equal to p₂. This is a two-tailed or two-sided test.

As always, in order to be cautious, we should use the two-sided alternative hypothesis if we do not have a direction in mind before we obtain our sample. The reason for doing this is that it is harder to reject the null hypothesis with a two-sided test.

The three hypotheses can be rewritten by stating how p₁ - p₂ is related to the value zero. To be more specific, the null hypothesis would become H₀:p₁ - p₂ = 0. The potential alternative hypotheses would be written as:

• H_a: p₁ - p₂ > 0 is equivalent to the statement "p₁ is greater than p₂."
• H_a: p₁ - p₂ < 0 is equivalent to the statement "p₁ is less than p₂."
• H_a: p₁ - p₂  ≠ 0 is equivalent to the statement "p₁ is not equal to p₂."

This equivalent formulation actually shows us a little bit more of what is happening behind the scenes. What we are doing in this hypothesis test is turning the two parameters p₁ and p₂ into the single parameter p₁ - p_2. We then test this new parameter against the value zero. 

The Test Statistic

The formula for the test statistic is given in the image above.  An explanation of each of the terms follows:

• The sample from the first population has size n_1. The number of successes from this sample (which is not directly seen in the formula above) is k_1.
• The sample from the second population has size n_2. The number of successes from this sample is k_2.
• The sample proportions are p₁-hat = k₁ / n₁ and p₂-hat = k₂ / n₂ .
• We then combine or pool the successes from both of these samples and obtain:                         p-hat = ( k₁ + k₂) / ( n₁ + n₂).

As always, be careful with order of operations when calculating. Everything underneath the radical must be calculated before taking the square root.

The P-Value

The next step is to calculate the p-value that corresponds to our test statistic. We use a standard normal distribution for our statistic and consult a table of values or use statistical software. 

The details of our p-value calculation depend upon the alternative hypothesis we are using:

• For H_a: p₁ - p₂ > 0, we calculate the proportion of the normal distribution that is greater than Z.
• For H_a: p₁ - p₂ < 0, we calculate the proportion of the normal distribution that is less than Z.
• For H_a: p₁ - p₂  ≠ 0, we calculate the proportion of the normal distribution that is greater than |Z|, the absolute value of Z. After this, to account for the fact that we have a two-tailed test, we double the proportion. 

Decision Rule

Now we make a decision on whether to reject the null hypothesis (and thereby accept the alternative), or to fail to reject the null hypothesis. We make this decision by comparing our p-value to the level of significance alpha.

• If the p-value is less than or equal to alpha, then we reject the null hypothesis. This means that we have a statistically significant result and that we are going to accept the alternative hypothesis.
• If the p-value is greater than alpha, then we fail to reject the null hypothesis. This does not prove that the null hypothesis is true. Instead it means that we did not obtain convincing enough evidence to reject the null hypothesis. 

Special Note

The confidence interval for the difference of two population proportions does not pool the successes, whereas the hypothesis test does. The reason for this is that our null hypothesis assumes that p₁ - p₂ = 0. The confidence interval does not assume this. Some statisticians do not pool the successes for this hypothesis test, and instead use a slightly modified version of the above test statistic.