SED navigation bar go to SED home page go to Dataplot home page go to NIST home page SED Home Page SED Staff SED Projects SED Products and Publications Search SED Pages
Dataplot Vol 1 Vol 2

DIFFERENCE OF PORPORTION CONFIDENCE LIMITS

Name:
    DIFFERENCE OF PROPORTION CONFIDENCE LIMITS
Type:
    Analysis Command
Purpose:
    Generates a confidence interval for the difference between two proportions.
Description:
    Given a set of N1 observations in a variable X1 and a set of N2 observations in a variable X2, we can compute the proportion of successes in each sample as p1 and p2. We can then compute the difference of the proportions as P1 - P2. In addition, the DIFFERENCE OF PROPORTION CONFIDENCE LIMITS command computes a confidence interval for the difference between the two proportions of successes.

    For most applications, you typically define success as a "1" and failure as a "0". However, in Dataplot you can also specify the range of values that constitute success (all other values denote failure) by entering the command

      ANOP LIMITS <lower limit> <upper limit>

    The default limits are 0.5 and 1.5 so that if your data contains 0's and 1's this command can be omitted.

    Several methods have been proposed for the confidence limits for the difference between binomial proportions. The following methods are currently supported in Dataplot (other methods are available)

    • WALD (or NORMAL APPROXIMATION)

      The Wald two-sided confidence interval is

        \( (\hat{p}_1 - \hat{p}_2) \pm z_{\alpha/2} \sqrt{\frac{\hat{p}_1 (1 - \hat{p}_1)}{n_1} + \frac{\hat{p}_2 (1 - \hat{p}_2)}{n_2}} \)

      where

        \( n_1 \) is the number of observations in sample one
        \( n_2 \) is the number of observations in sample two
        \( \hat{p}_1 \) is the proportion of successses in sample one
        \( \hat{p}_2 \) is the proportion of successses in sample two
        \( z_{\alpha} \) is the percent point function of the normal distribution

      This method is commonly used. However, Agresti and Caffo pointed out that this method does not always perform well in the sense that the actual coverage probabilities can be less than (and often substantially less than) the nominal coverage probabilities.

    • ADJUSTED WALD (or AGRESTI-CAFFO)

      Agresti and Caffo recommend the following confidence limit

        \( (\tilde{p}_1 - \tilde{p}_2) \pm z_{\alpha/2} \sqrt{\frac{\tilde{p}_1 (1 - \tilde{p}_1)}{n_1+2} + \frac{\tilde{p}_2 (1 - \tilde{p}_2)}{n_2+2}} \)

      where

        \( \tilde{p}_1 = \frac{X_1 + 1}{n_1 + 2} \) where \( X_1 \) is the number of successes in sample one
        \( \tilde{p}_2 = \frac{X_2 + 1}{n_2 + 2} \) where \( X_2 \) is the number of successes in sample two

      The justification and derivation of this interval is given in the Agresti Caffo paper. This interval improves the coverage probabilities of the standard Wald interval.

    • CARLIN AND LOUIS BAYESIAN

      Carlin and Louis propsed the following interval based on a Bayesian derivation

        \( (\tilde{p}_1 - \tilde{p}_2) \pm z_{\alpha/2} \sqrt{\frac{\tilde{p}_1 (1 - \tilde{p}_1)}{n_1+3} + \frac{\tilde{p}_2 (1 - \tilde{p}_2)}{n_2+3}} \)

      where \( \tilde{p}_1 \) and \( \tilde{p}_2 \) are defined as they are for the Agresti-Caffo interval.

      This interval also improves upon the Wald interval. It is slightly less conservative with sometimes slightly narrower intervals than the Agresti-Caffo interval. However, it can also at times have poorer coverage probabilities than the Agresti-Caffo interval.

    To specify the method to use, enter the command

      SET DIFFERENCE OF BINOMIAL METHOD <WALD/ADJUSTED WALD/BAYESIAN>

    The default is the adjusted Wald (Agresti-Caffo) interval.

    Dataplot computes this inverval for a number of different probability levels.

Syntax:
    DIFFERENCE OF PROPORTION CONFIDENCE LIMITS <y1> <y2>
                            <SUBSET/EXCEPT/FOR qualification>
    where <y1> is the first response variable;
                <y2> is the second response variable;
    and where the <SUBSET/EXCEPT/FOR qualification> is optional.
Examples:
    ANOP LIMITS 0.50 1.5
    DIFFERENCE OF PROPORTION CONFIDENCE LIMITS Y1 Y2

    SET DIFFERENCE OF BINOMIAL METHOD ADJUSTED WALD DIFFERENCE OF PROPORTION CONFIDENCE LIMITS Y1 Y2 ...
                SUBSET TAG > 2

Note:
    A table of confidence intervals is printed for alpha levels of 50.0, 75.0, 90.0, 95.0, 99.0, 99.9, 99.99, and 99.999. The sample size, sample number of successes, and sample proportion of successes are also printed.
Note:
    The following statistics are also supported

      LET ALPHA = <value>
      LET A = DIFF OF BINOMIAL PROPORTIONS LOWER CONFIDENCE ...
                      LIMIT Y1 Y2
      LET A = DIFF OF BINOMIAL PROPORTIONS UPPER CONFIDENCE ...
                      LIMIT Y1 Y2

    In addition to the above LET commands, built-in statistics are supported for about 20+ different commands (enter HELP STATISTICS for details).

Default:
    The adjusted Wald (Agresti-Caffo) interval is the default
Synonyms:
    None
Related Commands: References:
    Snedecor and Cochran (1989), "Statistical Methods", Eigth Edition, Iowa State University Press, pp. 125-128.

    Agresti and Caffo (2000), "Simple and Effective Confidence Intervals for Proportions and Differences of Proportions Result From Adding Two Successes and Two Failures", The American Statistician, Vol. 54, No. 4, pp. 280-288.

    Carlin and Louis (1996), "Bayes and Empirical Bayes Methods for Data Analysis", Chapman and Hall.

Applications:
    Categorical Data Analysis
Implementation Date:
    1999/05
    2017/11: Added the adjusted Wald (Agresti-Caffo) and Bayesian intervals
    2017/11: Made the adjusted Wald (Agresti-Caffo) interval the default
Program:
     
    let n1 = 40
    let nsuc1 = 8
    let y1 = 0 for i = 1 1 n1
    let y1 = 1 for i = 1 1 nsuc1
    .
    let n2 = 30
    let nsuc2 = 11
    let y2 = 0 for i = 1 1 n2
    let y2 = 1 for i = 1 1 nsuc2
    .
    .          Now do proportions confidence interval
    .
    set write decimals 5
    let alpha = 0.95
    .
    set difference of binomial method wald
    difference of proportion confidence interval y1 y2
    set difference of binomial method adjusted wald
    difference of proportion confidence interval y1 y2
    set difference of binomial method bayesian
    difference of proportion confidence interval y1 y2
        
    This command generates the following output.
     
                 Difference of Proportions Two-Sided Confidence Limits
                                     (Wald Method)
      
     First Response Variable:  Y1
     Second Response Variable: Y2
      
     Sample 1:
     Number of Observations:                  40
     Number of Successes:                     8
     Proportion of Successes:                 0.20000
      
     Sample 2:
     Number of Observations:                  30
     Number of Successes:                     11
     Proportion of Successes:                 0.36667
      
     Difference Between Proportions:          -0.16667
      
      
      
     ------------------------------------------
       Confidence          Lower          Upper
        Value (%)          Limit          Limit
     ------------------------------------------
           50.000       -0.23975       -0.09358
           75.000       -0.29131       -0.04202
           90.000       -0.34489        0.01156
           95.000       -0.37904        0.04570
           99.000       -0.44577        0.11244
           99.900       -0.52321        0.18988
           99.990       -0.58823        0.25490
           99.999       -0.64529        0.31195
      
      
                 Difference of Proportions Two-Sided Confidence Limits
                        (Agresti-Caffo (Adjusted Wald) Method)
      
     First Response Variable:  Y1
     Second Response Variable: Y2
      
     Sample 1:
     Number of Observations:                  40
     Number of Successes:                     8
     Proportion of Successes:                 0.20000
      
     Sample 2:
     Number of Observations:                  30
     Number of Successes:                     11
     Proportion of Successes:                 0.36667
      
     Difference Between Proportions:          -0.16667
      
      
      
     ------------------------------------------
       Confidence          Lower          Upper
        Value (%)          Limit          Limit
     ------------------------------------------
           50.000       -0.23252       -0.08891
           75.000       -0.28318       -0.03825
           90.000       -0.33582        0.01439
           95.000       -0.36936        0.04794
           99.000       -0.43493        0.11350
           99.900       -0.51101        0.18958
           99.990       -0.57489        0.25346
           99.999       -0.63095        0.30952
      
      
                 Difference of Proportions Two-Sided Confidence Limits
                          (Carlin and Louis Bayesian Method)
      
     First Response Variable:  Y1
     Second Response Variable: Y2
      
     Sample 1:
     Number of Observations:                  40
     Number of Successes:                     8
     Proportion of Successes:                 0.20000
      
     Sample 2:
     Number of Observations:                  30
     Number of Successes:                     11
     Proportion of Successes:                 0.36667
      
     Difference Between Proportions:          -0.16667
      
      
      
     ------------------------------------------
       Confidence          Lower          Upper
        Value (%)          Limit          Limit
     ------------------------------------------
           50.000       -0.23151       -0.08992
           75.000       -0.28146       -0.03997
           90.000       -0.33337        0.01194
           95.000       -0.36644        0.04501
           99.000       -0.43109        0.10966
           99.900       -0.50611        0.18468
           99.990       -0.56909        0.24766
           99.999       -0.62437        0.30294
        
Date created: 06/05/2001
Last updated: 12/11/2023

Please email comments on this WWW page to alan.heckert@nist.gov.