Dataplot Vol 1 Vol 2

# DIFFERENCE OF PORPORTION CONFIDENCE LIMITS

Name:
DIFFERENCE OF PROPORTION CONFIDENCE LIMITS
Type:
Analysis Command
Purpose:
Generates a confidence interval for the difference between two proportions.
Description:
Given a set of N1 observations in a variable X1 and a set of N2 observations in a variable X2, we can compute the proportion of successes in each sample as p1 and p2. We can then compute the difference of the proportions as P1 - P2. In addition, the DIFFERENCE OF PROPORTION CONFIDENCE LIMITS command computes a confidence interval for the difference between the two proportions of successes.

For most applications, you typically define success as a "1" and failure as a "0". However, in Dataplot you can also specify the range of values that constitute success (all other values denote failure) by entering the command

ANOP LIMITS <lower limit> <upper limit>

The default limits are 0.5 and 1.5 so that if your data contains 0's and 1's this command can be omitted.

Several methods have been proposed for the confidence limits for the difference between binomial proportions. The following methods are currently supported in Dataplot (other methods are available)

• WALD (or NORMAL APPROXIMATION)

The Wald two-sided confidence interval is

$$(\hat{p}_1 - \hat{p}_2) \pm z_{\alpha/2} \sqrt{\frac{\hat{p}_1 (1 - \hat{p}_1)}{n_1} + \frac{\hat{p}_2 (1 - \hat{p}_2)}{n_2}}$$

where

$$n_1$$ is the number of observations in sample one
$$n_2$$ is the number of observations in sample two
$$\hat{p}_1$$ is the proportion of successses in sample one
$$\hat{p}_2$$ is the proportion of successses in sample two
$$z_{\alpha}$$ is the percent point function of the normal distribution

This method is commonly used. However, Agresti and Caffo pointed out that this method does not always perform well in the sense that the actual coverage probabilities can be less than (and often substantially less than) the nominal coverage probabilities.

Agresti and Caffo recommend the following confidence limit

$$(\tilde{p}_1 - \tilde{p}_2) \pm z_{\alpha/2} \sqrt{\frac{\tilde{p}_1 (1 - \tilde{p}_1)}{n_1+2} + \frac{\tilde{p}_2 (1 - \tilde{p}_2)}{n_2+2}}$$

where

$$\tilde{p}_1 = \frac{X_1 + 1}{n_1 + 2}$$ where $$X_1$$ is the number of successes in sample one
$$\tilde{p}_2 = \frac{X_2 + 1}{n_2 + 2}$$ where $$X_2$$ is the number of successes in sample two

The justification and derivation of this interval is given in the Agresti Caffo paper. This interval improves the coverage probabilities of the standard Wald interval.

• CARLIN AND LOUIS BAYESIAN

Carlin and Louis propsed the following interval based on a Bayesian derivation

$$(\tilde{p}_1 - \tilde{p}_2) \pm z_{\alpha/2} \sqrt{\frac{\tilde{p}_1 (1 - \tilde{p}_1)}{n_1+3} + \frac{\tilde{p}_2 (1 - \tilde{p}_2)}{n_2+3}}$$

where $$\tilde{p}_1$$ and $$\tilde{p}_2$$ are defined as they are for the Agresti-Caffo interval.

This interval also improves upon the Wald interval. It is slightly less conservative with sometimes slightly narrower intervals than the Agresti-Caffo interval. However, it can also at times have poorer coverage probabilities than the Agresti-Caffo interval.

To specify the method to use, enter the command

SET DIFFERENCE OF BINOMIAL METHOD <WALD/ADJUSTED WALD/BAYESIAN>

The default is the adjusted Wald (Agresti-Caffo) interval.

Dataplot computes this inverval for a number of different probability levels.

Syntax:
DIFFERENCE OF PROPORTION CONFIDENCE LIMITS <y1> <y2>
<SUBSET/EXCEPT/FOR qualification>
where <y1> is the first response variable;
<y2> is the second response variable;
and where the <SUBSET/EXCEPT/FOR qualification> is optional.
Examples:
ANOP LIMITS 0.50 1.5
DIFFERENCE OF PROPORTION CONFIDENCE LIMITS Y1 Y2

SET DIFFERENCE OF BINOMIAL METHOD ADJUSTED WALD DIFFERENCE OF PROPORTION CONFIDENCE LIMITS Y1 Y2 ...
SUBSET TAG > 2

Note:
A table of confidence intervals is printed for alpha levels of 50.0, 75.0, 90.0, 95.0, 99.0, 99.9, 99.99, and 99.999. The sample size, sample number of successes, and sample proportion of successes are also printed.
Note:
The following statistics are also supported

LET ALPHA = <value>
LET A = DIFF OF BINOMIAL PROPORTIONS LOWER CONFIDENCE ...
LIMIT Y1 Y2
LET A = DIFF OF BINOMIAL PROPORTIONS UPPER CONFIDENCE ...
LIMIT Y1 Y2

In addition to the above LET commands, built-in statistics are supported for about 20+ different commands (enter HELP STATISTICS for details).

Default:
The adjusted Wald (Agresti-Caffo) interval is the default
Synonyms:
None
Related Commands:
 ANOP LIMITS = Specify success region for proportions. PROPORTION CONFIDENCE LIMITS = Compute a proportions confidence interval. ANOP PLOT = Generate an analysis of proportions plot. CONFIDENCE LIMITS = Generate the confidence limits for the mean.
References:
Snedecor and Cochran (1989), "Statistical Methods", Eigth Edition, Iowa State University Press, pp. 125-128.

Agresti and Caffo (2000), "Simple and Effective Confidence Intervals for Proportions and Differences of Proportions Result From Adding Two Successes and Two Failures", The American Statistician, Vol. 54, No. 4, pp. 280-288.

Carlin and Louis (1996), "Bayes and Empirical Bayes Methods for Data Analysis", Chapman and Hall.

Applications:
Categorical Data Analysis
Implementation Date:
1999/05
Program:

let n1 = 40
let nsuc1 = 8
let y1 = 0 for i = 1 1 n1
let y1 = 1 for i = 1 1 nsuc1
.
let n2 = 30
let nsuc2 = 11
let y2 = 0 for i = 1 1 n2
let y2 = 1 for i = 1 1 nsuc2
.
.          Now do proportions confidence interval
.
set write decimals 5
let alpha = 0.95
.
set difference of binomial method wald
difference of proportion confidence interval y1 y2
set difference of binomial method adjusted wald
difference of proportion confidence interval y1 y2
set difference of binomial method bayesian
difference of proportion confidence interval y1 y2

This command generates the following output.

Difference of Proportions Two-Sided Confidence Limits
(Wald Method)

First Response Variable:  Y1
Second Response Variable: Y2

Sample 1:
Number of Observations:                  40
Number of Successes:                     8
Proportion of Successes:                 0.20000

Sample 2:
Number of Observations:                  30
Number of Successes:                     11
Proportion of Successes:                 0.36667

Difference Between Proportions:          -0.16667

------------------------------------------
Confidence          Lower          Upper
Value (%)          Limit          Limit
------------------------------------------
50.000       -0.23975       -0.09358
75.000       -0.29131       -0.04202
90.000       -0.34489        0.01156
95.000       -0.37904        0.04570
99.000       -0.44577        0.11244
99.900       -0.52321        0.18988
99.990       -0.58823        0.25490
99.999       -0.64529        0.31195

Difference of Proportions Two-Sided Confidence Limits

First Response Variable:  Y1
Second Response Variable: Y2

Sample 1:
Number of Observations:                  40
Number of Successes:                     8
Proportion of Successes:                 0.20000

Sample 2:
Number of Observations:                  30
Number of Successes:                     11
Proportion of Successes:                 0.36667

Difference Between Proportions:          -0.16667

------------------------------------------
Confidence          Lower          Upper
Value (%)          Limit          Limit
------------------------------------------
50.000       -0.23252       -0.08891
75.000       -0.28318       -0.03825
90.000       -0.33582        0.01439
95.000       -0.36936        0.04794
99.000       -0.43493        0.11350
99.900       -0.51101        0.18958
99.990       -0.57489        0.25346
99.999       -0.63095        0.30952

Difference of Proportions Two-Sided Confidence Limits
(Carlin and Louis Bayesian Method)

First Response Variable:  Y1
Second Response Variable: Y2

Sample 1:
Number of Observations:                  40
Number of Successes:                     8
Proportion of Successes:                 0.20000

Sample 2:
Number of Observations:                  30
Number of Successes:                     11
Proportion of Successes:                 0.36667

Difference Between Proportions:          -0.16667

------------------------------------------
Confidence          Lower          Upper
Value (%)          Limit          Limit
------------------------------------------
50.000       -0.23151       -0.08992
75.000       -0.28146       -0.03997
90.000       -0.33337        0.01194
95.000       -0.36644        0.04501
99.000       -0.43109        0.10966
99.900       -0.50611        0.18468
99.990       -0.56909        0.24766
99.999       -0.62437        0.30294


NIST is an agency of the U.S. Commerce Department.

Date created: 06/05/2001
Last updated: 11/21/2017