CONSENSUS MEAN PLOT

CONSENSUS MEAN PLOT

Graphics Command

Compute an estimate of a consensus mean, and the associated uncertainty, based on the data from multiple laboratories or multiple methods and present the results in a graphical format.

The problem of determining a consensus mean based on the data from two or more laboratories (or from using two or more methods from the same laboratory) is a common one at NIST and other measurement laboratories. There are a number of approaches to this problem. The Dataplot CONSENSUS MEANS command implements a number of the more commonly used approaches. Note that Dataplot computes estimates for a variety of methods and does not specify which is the most appropriate method for a given data set. Consult with a statistician for guidance on which method is most appropriate for your data.

The CONSENSUS MEAN PLOT command performs a consensus means analysis and in addition to the textual output it presents the results in a graphical format. The details of the consensus means analysis are discussed in the documentation for the CONSENSUS MEANS command (enter HELP CONSENSUS MEANS for details).

The graph consists of two parts.

1. The left hand side of the plot displays the consensus means estimates with the associated uncertainty obtained by the various methods. You can specify the desired uncertainty to plot by entering one of the following command
   SET CONSENSUS MEAN PLOT ERROR ...
               ONE STANDARD ERRORS
   SET CONSENSUS MEAN PLOT ERROR ...
               TWO STANDARD ERRORS
   SET CONSENSUS MEAN PLOT ERROR ...
               CONFIDENCE INTERVAL
   

   The default is to use TWO STANDARD ERRORS. If CONFIDENCE INTERVAL is used, then 95% confidence intervals will be plotted.

2. The right hand side of the plot displays the lab means with their associated uncertainty. The SET CONSENSUS MEAN PLOT ERROR will also specify the uncertainty to use for the lab data.

This plot allows you to compare the consensus values and associated uncertainties obtained by the different methods. It also allows you to compare the consensus values against the lab data.

The methods are plotted in the following order:

1. Mandel-Paule
2. Modified Mandel-Paule
3. Vangel-Rukhin
4. DerSimonian-Laird (original variance)
5. DerSimonian-Laird (HHD variance)
6. DerSimonian-Laird (minmax variance)
7. DerSimonian-Laird (bootstrap variance)
8. Graybill-Deal
9. Fairweather (Fairweather variance)
10. Fairweather (Cox variance)
11. Fairweather (minmax variance)
12. Generalized Confidence Intervals
13. Grand Mean
14. Mean of Means
15. Bounds on bias (BOB)
16. Schiller-Eberhardt
17. Bayesian Consensus Procedure
18. Median of Means
19. Huber Mean of Means

In most cases, you will probably want only a subset of these methods. You can use the following commands to specify which methods to include in the consensus means analysis. This is demonstrated in the program samples below.

SET MANDEL PAULE <ON/OFF>	- default is ON
SET MODIFIED MANDEL PAULE <ON/OFF>	- default is ON
SET VANGEL RUHKIN <ON/OFF>	- default is ON
SET DERSIMONIAN LAIRD <ON/OFF>	- default is ON
SET DERSIMONIAN LAIRD HHD <ON/OFF>	- default is ON
SET DERSIMONIAN LAIRD MINMAX <ON/OFF>	- default is OFF
SET DERSIMONIAN LAIRD BOOTSTRAP <ON/OFF>	- default is OFF
SET GRAYBILL DEAL <ON/OFF>	- default is ON
SET GENERALIZED CONFIDENCE INTERVAL <ON/OFF>	- default is ON
SET FAIRWEATHER <ON/OFF>	- default is OFF
SET MEAN OF MEANS <ON/OFF>	- default is ON
SET GRAND MEAN <ON/OFF>	- default is ON
SET BOB <ON/OFF>	- default is ON
SET SCHILLER EBERHARDT <ON/OFF>	- default is OFF
SET BAYESIAN CONSENSUS PROCEDURE <ON/OFF>	- default is OFF
SET MEDIAN OF MEANS <ON/OFF>	- default is OFF
SET HUBER MEAN OF MEANS <ON/OFF>	- default is OFF

CONSENSUS MEAN PLOT <y> <tag> <SUBSET/EXCEPT/FOR qualification>
where <y> is the response variable;
            <tag> is the group-id variable;
and where the <SUBSET/EXCEPT/FOR qualification> is optional.

This is used for the raw data case.

CONSENSUS MEAN PLOT <ymean> <ysd> <ni>
                        <SUBSET/EXCEPT/FOR qualification>
where <ymean> is the variable containing the lab means;
            <ysd> is the variable containing the lab standard deviations;
            <ni> is the variable containing the lab sample sizes;
and where the <SUBSET/EXCEPT/FOR qualification> is optional.

This is used for the summary data case.

CONSENSUS MEAN PLOT <ymean> <ysd> <ni> <labid>
                        <SUBSET/EXCEPT/FOR qualification>
where <ymean> is the variable containing the lab means;
            <ysd> is the variable containing the lab standard deviations;
            <ni> is the variable containing the lab sample sizes;
            <labid> is the variable containing the lab-id's (numeric);
and where the <SUBSET/EXCEPT/FOR qualification> is optional.

This is used for the summary data case. The is used in the labeling of the textual output. It is not used in the computation of the consensus means or in the plot.

CONSENSUS MEAN PLOT Y X
CONSENSUS MEAN PLOT Y TAG
CONSENSUS MEAN PLOT Y TAG SUBSET TAG >= 2
CONSENSUS MEAN PLOT Y TAG SUBSET TAG = 1 TO 6
CONSENSUS MEAN PLOT YMEAN YSD NI

You can omit the lab data portion of the plot by entering the command
SET CONSENSUS MEAN PLOT DATA OFF

To reset the default of showing the lab data on the plot, enter

SET CONSENSUS MEAN PLOT DATA ON

This is demonstrated in the Program 2 example below.

You can optionally sort the methods by the width of the uncertainty interval (from smallest to largest) on the plot by entering the command
SET CONSENSUS MEAN PLOT SORTED ON

To reset the default of not sorting the methods, enter

SET CONSENSUS MEAN PLOT SORTED OFF

This is demonstrated in the Program 3 example below. The program example shows how to read DPST3F.DAT to retrieve the sorted order of the methods.

In some cases, an outlying laboratory may distort the plot (i.e., you cannot distinguish the data from the other non-outlying labs). In these cases, you may want to omit the lab from the plot while including it in the analysis (you can use the SUBSET clause to omit the lab both from the plot and the analysis).

In order to specify one or more labs to omit from the plot (but not the analysis), enter the command

SET CONSENSUS MEAN PLOT OMIT LABS <lab1> ... <labk>

where <lab1> ... <labk> is a list of 1 to 10 labs to be omitted from the plot.

To reset the default of all labs being plotted, enter

SET CONSENSUS MEAN PLOT OMIT LABS

This option was added 12/2016.

Similarly, you can omit methods from the plot (but not the analysis) with the commands

SET CONSENSUS MEAN PLOT OMIT METHOD ONE <method>
SET CONSENSUS MEAN PLOT OMIT METHOD TWO <method>
SET CONSENSUS MEAN PLOT OMIT METHOD THREE <method>


When extreme outliers are included in the analysis, the results for some methods may distort the plot.

This option was added 04/2017.

You can use the CHARACTER and LINE commands to control the appearance of the plot. Specifically,
1. Trace 1 is a line connecting the estimated means for each of the methods.
2. Traces 2, 3, and 4 are the estimated means, lower interval, and upper interval, respectively, within a given method.
3. Trace 5 is line connecting the lower and upper interval within a single method.
This is demonstrated in the example program below.

The default is to put the method results on the left side of the plot and the lab data results on the right side of the plot. To reverse this (i.e., lab data results on the left and the method results on the right side), enter the command
SET CONSENSUS MEAN PLOT DATA LEFT

To reset the default, enter the commnd

SET CONSENSUS MEAN PLOT DATA RIGHT

This example is demonstrated in the Program 4 example below.

None

There are no synonyms for this command.

CONSENSUS MEANS	= Perform a consensus means analysis.
BOX PLOT	= Generate a box plot.
MEAN PLOT	= Generate a mean plot.
SD PLOT	= Generate a standard deviation plot.
YOUDEN PLOT	= Generate a Youden plot.
ANOVA	= Perform an analysis of variance.

DerSimonian and Laird (1986), "Meta-analysis in Clinical Trials", Controlled Clinical Trials, 7, pp. 177-188.

Graybill and Deal (1959), "Combining Unbiased Estimators", Biometrics, 15, pp. 543-550.

M. S. Levenson, D. L. Banks, K. R. Eberhardt, L. M. Gill, W. F. Guthrie, H. K. Liu, M. G. Vangel, J. H. Yen, and N. F. Zhang (2000), "An ISO GUM Approach to Combining Results from Multiple Methods", Journal of Research of the National Institute of Standards and Technology, Volume 105, Number 4.

John Mandel and Robert Paule (1970), "Interlaboratory Evaluation of a Material with Unequal Number of Replicates", Analytical Chemistry, 42, pp. 1194-1197.

Robert Paule and John Mandel (1982), "Consensus Values and Weighting Factors", Journal of Research of the National Bureau of Standards, 87, pp. 377-385.

Andrew Rukhin (2009), "Weighted Means Statistics in Interlaboratory Studies", Metrologia, Vol. 46, pp. 323-331.

Andrew Ruhkin (2003), "Two Procedures of Meta-analysis in Clinical Trials and Interlaboratory Studies", Tatra Mountains Mathematical Publications, 26, pp. 155-168.

Andrew Ruhkin and Mark Vangel (1998), "Estimation of a Common Mean and Weighted Means Statistics", Journal of the American Statistical Association, Vol. 93, No. 441.

Andrew Ruhkin, B. Biggerstaff, and Mark Vangel (2000), "Restricted Maximum Likelihood Estimation of a Common Mean and Mandel-Paule Algorithm", Journal of Statistical Planning and Inference, 83, pp. 319-330.

Mark Vangel and Andrew Ruhkin (1999), "Maximum Likelihood Analysis for Heteroscedastic One-Way Random Effects ANOVA in Interlaboratory Studies", Biometrics 55, 129-136.

Susannah Schiller and Keith Eberhardt (1991), "Combining Data from Independent Analysis Methods", Spectrochimica, ACTA 46 (12).

Susannah Schiller (1996), "Standard Reference Materials: Statistical Aspects of the Certification of Chemical SRMs", NIST SP 260-125, NIST, Gaithersburg, MD.

Bimal Kumar Sinha (1985), "Unbiased Estimation of the Variance of the Graybill-Deal Estimator of the Common Mean of Several Normal Populations", The Canadian Journal of Statistics, Vol. 13, No. 3, pp. 243-247.

Nien-Fan Zhang (2006), "The Uncertainty Associated with The Weighted Mean of Measurement Data", Metrologia, 43, PP. 195-204.

Hagwood and Guthrie (2006), "Combining Data in Small Multiple-Methods Studies", Technometrics, Vol. 48, No. 2.

Iyer, Wang, and Matthew (2004), "Models and Confidence Intervals for True Values in Interlaboratory Trials", Journal of the American Statistical Association, Vol. 99, No. 468, pp. 1060-1071.

Fairweather (1972), "A Method for Obtaining an Exact Confidence Interval for the Common Mean of Several Normal Populations", Applied Statistics, Vol. 21, pp. 229-233.

Cox (2002), "The Evaluation of Key Comparison Data", Metrologia, Vol. 39, pp. 589-595.

Standard Reference Materials, Interlaboratory Studies

2001/08
2011/10: Redesigned plot to incorporate the lab data
2016/12: Support for SET CONSENSUS MEAN PLOT OMIT LABS
2017/03: Support for SET CONSENSUS MEAN PLOT OMIT METHOD
2017/07: Changed the default for Schiller-Eberhardt, Fairweather, and Bayesian consensus procedure to OFF
2019/08: Added the command SET CONSENSUS MEAN PLOT DATA

 
. Step 1: Read the data
.
SKIP 25
READ STUTZ86.DAT ALITE JUNK2 JUNK3 JUNK4 JUNK5 LABID
.
. Step 2: Specify methods to include on the plot and define
.         strings containing method id's.  Define settings
.         for DerSimonian-Laird bootstrap.
.
set write decimals 5
set modified mandel paule off
set schiller eberhardt off
set mean of means off
set grand mean off
set graybill deal off
set generalized confidence interval off
set fairweather off
set bayesian consensus procedure off
set dersimonian laird minmax off
set dersimonian laird bootstrap on
.
let string sx1 = MP
let string sx2 = VR
let string sx3 = DSLcr()Original
let string sx4 = DSLcr()HHD
let string sx5 = DSLcr()Bootstrap
let string sx6 = BOB
let nmeth = 6
.
seed 21307
set random number generator fibonacci congruential
bootstrap samples 100000
.
. Step 3: Set plot control features
.
.         Settings for plot characters/lines
.
line solid all
character blank all
character fill on
character hw 1 0.75
character circle
line      blank
.
.         Settings for labels
.
case asis
title asis
title offset 2
tic mark label case asis
title Consensus Means Plot
y1label Response
.
let nlab = unique labid
loop for k = 1 1 nlab
     let icnt = k + nmeth
     let string sx^icnt = ^k
end of loop
.
let ntot = nmeth + nlab
xlimits 1 ntot
major xtic mark number ntot
minor xtic mark number 0
tic mark offset units data
xtic mark offset 0.5   0.5
x1tic mark label format group label
let igx = group label sx1 to sx^ntot
x1tic mark label content igx
.
. Step 4: Generate the plot
.
set consensus mean plot error two standard errors
feedback off
consensus mean plot alite labid
.
. Step 5: Post plot labelling
.
line dotted
drawdsds  6.5 20  6.5 90
justification center
moveds 3.5 5
text Consensus Method: Two Standard Errors
moveds 9 5
text Laboratorycr()meansp()+/-2*sd/sqrt(n)
    

 
. This version suppresses the lab data portion of the plot
.
. Step 1: Read the data
.
SKIP 25
READ STUTZ86.DAT ALITE JUNK2 JUNK3 JUNK4 JUNK5 LABID
.
. Step 2: Specify methods to include on the plot and define
.         strings containing method id's.  Define settings
.         for DerSimonian-Laird bootstrap.
.
set write decimals 5
set modified mandel paule off
set schiller eberhardt off
set mean of means off
set grand mean off
set graybill deal off
set generalized confidence interval off
set fairweather off
set bayesian consensus procedure off
set dersimonian laird minmax off
set dersimonian laird bootstrap on
.
let string sx1 = MP
let string sx2 = VR
let string sx3 = DSLcr()Original
let string sx4 = DSLcr()HHD
let string sx5 = DSLcr()Bootstrap
let string sx6 = BOB
let nmeth = 6
.
seed 21307
set random number generator fibonacci congruential
bootstrap samples 100000
.
. Step 3: Set plot control features
.
.         Settings for plot characters/lines
.
line solid all
character blank all
character fill on
character hw 1 0.75
character circle
line      blank
.
.         Settings for labels
.
case asis
title asis
title offset 2
tic mark label case asis
title Consensus Means Plot
y1label Response
.
xlimits 1 nmeth
major xtic mark number nmeth
minor xtic mark number 0
tic mark offset units data
xtic mark offset 0.5   0.5
x1tic mark label format group label
let igx = group label sx1 to sx^nmeth
x1tic mark label content igx
x1label Consensus Method: Two Standard Errors
x1label displacement 15
.
title Consensus Means Plot - Lab Data Omitted
set consensus mean plot data off
feedback off
consensus mean plot alite labid
    

 
. Step 1: Read the data
.
SKIP 25
READ STUTZ86.DAT ALITE JUNK2 JUNK3 JUNK4 JUNK5 LABID
.
. Step 2: Specify methods to include on the plot and define
.         strings containing method id's.  Define settings
.         for DerSimonian-Laird bootstrap.
.
set write decimals 5
set modified mandel paule off
set schiller eberhardt off
set mean of means off
set grand mean off
set graybill deal off
set generalized confidence interval off
set fairweather off
set bayesian consensus procedure off
set dersimonian laird minmax off
set dersimonian laird bootstrap on
.
let string sx1 = MP
let string sx2 = VR
let string sx3 = DSLcr()Original
let string sx4 = DSLcr()HHD
let string sx5 = DSLcr()Boot
let string sx6 = BOB
let nmeth = 6
.
seed 21307
set random number generator fibonacci congruential
bootstrap samples 100000
.
. Step 3: Set plot control features
.
.         Settings for plot characters/lines
.
line solid all
character blank all
character fill on
character hw 1 0.75
character circle
line      blank
.
.         Settings for labels
.
case asis
title asis
title offset 2
tic mark label case asis
y1label Response
.
let nlab = unique labid
loop for k = 1 1 nlab
     let icnt = k + nmeth
     let string sx^icnt = ^k
end of loop
.
let ntot = nmeth + nlab
xlimits 1 ntot
major xtic mark number ntot
minor xtic mark number 0
tic mark offset units data
xtic mark offset 0.5   0.5
x1tic mark label format group label
let igx = group label sx1 to sx^ntot
x1tic mark label content igx
.
. Step 4: Generate the plot
.
loop for k = 1 1 ntot
    let string sxnew^k = ^sx^k
end of loop
.
.         Generate dummy plot, read dpst3f.dat to obtain
.         sorted labels
.
set consensus mean plot sorted on
set consensus mean plot error two standard errors
.
device 1 off
device 2 off
print off
consensus mean plot alite labid
skip 0
read dpst3f.dat indx
loop for k = 1 1 nmeth
    let icnt = indx(k)
    let string sxnew^k = ^sx^icnt
end of loop
let igx = group label sxnew1 to sxnew^ntot
x1tic mark label content igx
device 1 on
device 2 on
printing on
.
.         Now generate plot with sorted labels
.
title Consensus Means Plot - Method Sorted by Width of Expanded Uncertainty
feedback off
consensus mean plot alite labid
.
. Step 5: Post plot labelling
.
line dotted
drawdsds  6.5 20  6.5 90
justification center
moveds 3.5 5
text Consensus Method: Two Standard Errors
moveds 9 5
text Laboratorycr()meansp()+/-2*sd/sqrt(n)
    

 
. Step 1:   Read the data
.
SKIP 25
READ STUTZ86.DAT ALITE JUNK2 JUNK3 JUNK4 JUNK5 LABID
.
. Step 2:   Define consensus means options
.
set write decimals 5
set bob on
set dersimonian laird bootstrap on
set dersimonian laird minmax off
set modified mandel paule off
set schiller eberhardt off
set mean of means off
set grand mean off
set graybill deal off
set generalized confidence interval off
set fairweather off
set bayesian consensus procedure off
set random number generator fibonacci congruential
seed 55631
bootstrap samples 100000
.
.  iflagm  = 1   => 95% Confidence Interval
.          = 2   => Two Standard Errors Confidence Interval
.          = 3   => One Standard Errors Confidence Interval
.
let iflagm = 2
if iflagm = 1
   set consensus mean plot error confidence intervals
else if iflagm = 2
   set consensus mean plot error two standard errors
else if iflagm = 3
   set consensus mean plot error one standard errors
end of if
.
.
. Step 3:   Define strings containing Method id's
.
let nlab = unique labid
loop for k = 1 1 nlab
     let string sx^k = ^k
end of loop
.
let icnt = nlab + 1
let string sx^icnt = MP
let icnt = icnt + 1
let string sx^icnt = VR
let icnt = icnt + 1
let string sx^icnt = DSLcr()Original
let icnt = icnt + 1
let string sx^icnt = DSLcr()HHD
let icnt = icnt + 1
let string sx^icnt = DSLcr()Bootstrap
let icnt = icnt + 1
let string sx^icnt = BOB
let nmeth = 6
.
. Step 4:   Set plot control options
.
line solid all
character blank all
character fill on
character hw 1 0.75
character circle
line      blank
.
case asis
title case asis
label case asis
tic mark label case asis
title offset 2
title Consensus Means Plot - With Data, Unsorted
y1label Percentage of Alite
.
let ntot = nmeth + nlab
xlimits 1 ntot
major xtic mark number ntot
minor xtic mark number 0
tic mark offset units data
xtic mark offset 0.5   0.5
x1tic mark label format group label
let igx = group label sx1 to sx^ntot
x1tic mark label content igx
.
. Step 5:   Generate the consensus mean plot
.
set consensus mean plot data left
. printing off
capture screen on
capture cmplot.out
consensus mean plot alite labid
end of capture
. printing on
.
line dotted
drawdsds  5.5 20  5.5 90
.
justification center
moveds 8.5 8
if iflagm = 1
   text Consensus Method:cr()95% Confidence Limits
else if iflagm = 2
   text Consensus Method:cr()Two Standard Errors
else if iflagm = 3
   text Consensus Method:cr()One Standard Error
end of if
moveds 2.5 8
text Method/Laboratorycr()meansp()+/-2*sd/sqrt(n)