![]() |
CONSENSUS MEANName:
There are a number of approaches to this problem. The Dataplot CONSENSUS MEANS command computes estimates for a variety of methods and does not specify which is the most appropriate method for a given data set. Consult with a statistician for guidance on which method is most appropriate for your data.
In this case, the consensus mean is simply the grand mean of all the data and a confidence interval for the consensus mean is simply the standard t-based condidence interval:
![]()
where The assumption of no lab effect is unrealistic in almost all cases. However, we include the grand mean method as a reference point as it gives an indication of how including the lab effects the estimate of the consensus mean and its uncertainty.
For this method, we compute the mean for each of the k
laboratories. Then we compute
![]() The limitations of this method are discussed in the "An ISO GUM Approach to Combining Results from Multiple Methods" paper (see the Reference section). For this method, the consensus mean estimate is an equi-weighted mean with no regard to possible differences in within-lab variation or within-lab sample sizes. The advantages of this method are that it is robust and simple to compute. The primary disadvantage is that no consideration is given to possible differences in the within-lab variation and sample sizes.
![]()
![]()
with independent Gaussian errors
Unbiased estimates of the within lab means and variances
![]()
When the variaces
![]()
with
![]() In practice, these within lab variances are unknown and so the true wi are also unknown. The Graybill-Deal method is based on this model. In the Graybill-Deal model, the estimate of the consensus mean is
![]() Dataplot supports four methods for computing the variance of the Graybill-Deal consensus mean.
Dataplot currently generates confidence intervals for the Graybill-Deal method using a method proposed by Rukhin (private communication). This method generates conservative intervals. The Graybill-Deal approach has the following limitations
![]()
where there are i = 1, ..., k labs and
j = 1, .... ni observations for each
lab. In this model,
For convenience, define the following terms:
The Mandel-Paule, modified Mandel-Paule, maximum likelihood (ML), DerSimonian-Laird, and generalized confidence interval methods are based on this model. We will discuss each of these in turn.
Answers to the above questions will determine how to appropriately weight the labs. The consensus mean will be a weighted mean of the lab means. The weighting can be either fixed (i.e., equal weights) or variable where the variable weights can be based on both engineering and statistical considerations. If the engineering decision is made to treat all labs as equal in importance, then from a statistical point of view the analysis consists primarily of the following two steps:
An additional third step is to carry out formal statistical tests to identify potentially outlying labs. A statistically unsolvable question that persists here is that just because a lab appears "different" does not necessarily mean that the lab is wrong (i.e., biased). The spectre that all of the consistent labs being self-behaved but biased is a real possibility which can only be solved by engineering judgement.
where <y> is a response variable; <tag> is a lab id variable; and where the <SUBSET/EXCEPT/FOR qualification> is optional. This syntax computes the consensus means based on the raw data.
<SUBSET/EXCEPT/FOR qualification> where <ymean> is a variable containing the lab means; <ysd> is a variable containing the lab standard deviations; <ni> is a variable containing the lab sample sizes; and where the <SUBSET/EXCEPT/FOR qualification> is optional. This syntax computes the consensus means based on the lab means, standard deviations, and sample sizes.
CONSENSUS MEANS Y1 GROUP SUBSET GROUP > 2 CONSENSUS MEANS YMEAN YSD NI
The following variables are written to the file dpst1f.dat. These are the statistics for the labs.
The following variables are written to the file dpst2f.dat. This is the information contained in table 2 of the CONSENSUS MEAN output. These variables can be used to make plots of the consensus mean results.
The following variables are written to the file dpst3f.dat. This is the information contained in table 3 of the CONSENSUS MEAN output. These variables can be used to generate plots of the consensus mean results.
The following variables are written to the file dpst4f.dat. This is the information contained in table 4 of the CONSENSUS MEAN output. These variables can be used to generate plots of the consensus mean results.
The following variables are written to the file dpst5f.dat.
If you want to use an exponential format (E15.7), enter
HELP CAPTURE LATEX for details.
If you have raw data, you can enter one of the following
LET A = DERSIMONIAN LAIRD STANDARD ERROR Y X LET A = DERSIMONIAN LAIRD HHD Y X LET A = DERSIMONIAN LAIRD MINMAX Y X LET A = MANDEL PAULE Y X LET A = MANDEL PAULE STANDARD ERROR Y X LET A = MODIFIED MANDEL PAULE Y X LET A = MODIFIED MANDEL PAULE STANDARD ERROR Y X LET A = VANGEL RUKHIN Y X LET A = VANGEL RUKHIN STANDARD ERROR Y X LET A = GENERALIZED CONFIDENCE INTERVAL Y X LET A = GENERALIZED CONFIDENCE INTERVAL STANDARD ERROR Y X LET A = BOB Y X LET A = BOB STANDARD ERROR Y X LET A = BCP Y X LET A = BCP STANDARD ERROR Y X LET A = MEAN OF MEANS Y X LET A = MEAN OF MEANS STANDARD ERROR Y X LET A = FAIRWEATHER Y X LET A = FAIRWEATHER STANDARD ERROR Y X LET A = SCHILLER-EBERHARDT Y X LET A = SCHILLER-EBERHARDT STANDARD ERROR Y X LET A = GRAYBILL DEAL Y X LET A = GRAYBILL DEAL SINHA STANDARD ERROR Y X LET A = GRAYBILL DEAL NAIVE STANDARD ERROR Y X LET A = GRAYBILL DEAL ZHANG ONE STANDARD ERROR Y X LET A = GRAYBILL DEAL ZHANG TWO STANDARD ERROR Y X If you have summary data, you can enter one of the following
LET A = SUMMARY DERSIMONIAN LAIRD STANDARD ERROR MEAN SD N LET A = SUMMARY DERSIMONIAN LAIRD HHD MEAN SD N LET A = SUMMARY DERSIMONIAN LAIRD MINMAX MEAN SD N LET A = SUMMARY MANDEL PAULE MEAN SD N LET A = SUMMARY MANDEL PAULE STANDARD ERROR ... MEAN SD N LET A = SUMMARY MODIFIED MANDEL PAULE MEAN SD N LET A = SUMMARY MODIFIED MANDEL PAULE STANDARD ERROR MEAN SD N LET A = SUMMARY VANGEL RUKHIN MEAN SD N LET A = SUMMARY VANGEL RUKHIN STANDARD ERROR MEAN SD N LET A = SUMMARY GENERALIZED CONFIDENCE INTERVAL MEAN SD N LET A = SUMMARY GENERALIZED CONFIDENCE INTERVAL ... STANDARD ERROR MEAN SD N LET A = SUMMARY BOB MEAN SD N LET A = SUMMARY BOB STANDARD ERROR MEAN SD N LET A = SUMMARY BCP MEAN SD N LET A = SUMMARY BCP STANDARD ERROR MEAN SD N LET A = SUMMARY MEAN OF MEANS MEAN SD N LET A = SUMMARY MEAN OF MEANS STANDARD ERROR MEAN SD N LET A = SUMMARY FAIRWEATHER MEAN SD N LET A = SUMMARY FAIRWEATHER STANDARD ERROR MEAN SD N LET A = SUMMARY SCHILLER-EBERHARDT MEAN SD N LET A = SUMMARY SCHILLER-EBERHARDT STANDARD ERROR MEAN SD N LET A = SUMMARY GRAYBILL DEAL MEAN SD N LET A = SUMMARY GRAYBILL DEAL SINHA STANDARD ERROR MEAN SD N LET A = SUMMARY GRAYBILL DEAL NAIVE STANDARD ERROR MEAN SD N LET A = SUMMARY GRAYBILL DEAL ZHANG ONE STANDARD ERROR ... MEAN SD N LET A = SUMMARY GRAYBILL DEAL ZHANG TWO STANDARD ERROR ... MEAN SD N Dataplot statistics can be used in a number of other commands. For details, enter
For the SUMMARY cases, bootstrapping is not currently supported. However, we anticipate adding this capability in a subsequent release.
Graybill and Deal (1959), "Combining Unbiased Estimators", Biometrics, 15, pp. 543-550. M. S. Levenson, D. L. Banks, K. R. Eberhardt, L. M. Gill, W. F. Guthrie, H. K. Liu, M. G. Vangel, J. H. Yen, and N. F. Zhang (2000), "An ISO GUM Approach to Combining Results from Multiple Methods", Journal of Research of the National Institute of Standards and Technology, Volume 105, Number 4. John Mandel and Robert Paule (1970), "Interlaboratory Evaluation of a Material with Unequal Number of Replicates", Analytical Chemistry, 42, pp. 1194-1197. Robert Paule and John Mandel (1982), "Consensus Values and Weighting Factors", Journal of Research of the National Bureau of Standards, 87, pp. 377-385. Andrew Ruhkin (2003), "Two Procedures of Meta-analysis in Clinical Trials and Interlaboratory Studies", Tatra Mountains Mathematical Publications, 26, pp. 155-168. Andrew Ruhkin and Mark Vangel (1998), "Estimation of a Common Mean and Weighted Means Statistics", Journal of the American Statistical Association, Vol. 93, No. 441. Andrew Ruhkin, B. Biggerstaff, and Mark Vangel (2000), "Restricted Maximum Likelihood Estimation of a Common Mean and Mandel-Paule Algorithm", Journal of Statistical Planning and Inference, 83, pp. 319-330. Susannah Schiller and Keith Eberhardt (1991), "Combining Data from Independent Analysis Methods", Spectrochimica, ACTA 46 (12). Susannah Schiller (1996), "Standard Reference Materials: Statistical Aspects of the Certification of Chemical SRMs", NIST SP 260-125, NIST, Gaithersburg, MD. Bimal Kumar Sinha (1985), "Unbiased Estimation of the Variance of the Graybill-Deal Estimator of the Common Mean of Several Normal Populations", The Canadian Journal of Statistics, Vol. 13, No. 3, pp. 243-247. Mark Vangel and Andrew Ruhkin (1999), "Maximum Likelihood Analysis for Heteroscedastic One-Way Random Effects ANOVA in Interlaboratory Studies", Biometrics 55, 129-136. Nien-Fan Zhang (2006), "The Uncertainty Associated with The Weighted Mean of Measurement Data", Metrologia, 43, PP. 195-204.
2002/10: Support for Latex and HTML output 2006/3: Reformat output for consistency and clarity Add Tables 3 and 4 to the output Updated the Graybill-Deal method Added the DerSimonian-Laird method Added the generalized confidence intervals method Added support for Rich Text Format (RTF) output Added support for SET WRITE DECIMALS SKIP 25 READ STUTZ86.DAT ALITE JUNK2 JUNK3 JUNK4 JUNK5 LABID . CONSENSUS MEANS ALITE LABIDThe following output is generated: Consensus Means Analysis (Full Sample Case) Data Summary: Response Variable: ALITE Lab-ID Variable: LABID Total Number of Observations: 46 Grand Mean: 57.2260857 Grand Standard Deviation: 1.4274194 Total Number of Labs: 5 Minimum Lab Mean: 56.5000000 Maximum Lab Mean: 61.1999969 Minimum Lab SD: 0.1414219 Maximum Lab SD: 1.6800299 Within Lab (pooled) SD: 0.8369111 Within Lab (pooled) Variance: 0.7004202 Table 1: Summary Statistics by Lab Standard Lab Standard Deviation ID n(i) Mean Variance Deviation of Mean ---------------------------------------------------------------------------- 1 36 56.7527771 0.5522779 0.7431540 0.1238590 2 4 58.4249992 2.8225005 1.6800299 0.8400150 3 2 56.5000000 0.1799991 0.4242630 0.2999992 4 2 60.0999985 0.0200002 0.1414219 0.1000004 5 2 61.1999969 0.7200009 0.8485287 0.6000004 ---------------------------------------------------------------------------- 1. Method: Mandel-Paule Estimate of (unscaled) Consensus Mean: 58.5663223 Estimate of (scaled) Consensus Mean: 0.4396437 Between Lab Variance (unscaled): 4.0465660 Between Lab SD (unscaled): 2.0116079 Between Lab Variance (scaled): 0.1831857 Standard Deviation of Consensus Mean: 0.8317266 Standard Uncertainty (k = 1): 0.8317266 Expanded Uncertainty (k = 2): 1.6634532 Expanded Uncertainty (k = 1.9599645): 1.6301546 Normal PPF of 0.975: 1.9599645 Lower 95% (normal) Confidence Limit: 56.9361687 Upper 95% (normal) Confidence Limit: 60.1964760 Note: Mandel-Paule Best Usage: 6 or More Labs 2. Method: Modified Mandel-Paule Estimate of (unscaled) Consensus Mean: 58.5590630 Estimate of (scaled) Consensus Mean: 0.4380985 Between Lab Variance (unscaled): 3.2046051 Between Lab SD (unscaled): 1.7901411 Between Lab Variance (scaled): 0.1450706 Standard Deviation of Consensus Mean: 0.8338748 Standard Uncertainty (k = 1): 0.8338748 Expanded Uncertainty (k = 2): 1.6677495 Expanded Uncertainty (k = 1.9599645): 1.6343650 Normal PPF of 0.975: 1.9599645 Lower 95% (normal) Confidence Limit: 56.9246979 Upper 95% (normal) Confidence Limit: 60.1934280 Note: Modified Mandel-Paule Best Usage: 6 or More Labs 3. Method: Vangel-Rukhin Maximum Likelihood Estimate of (unscaled) Consensus Mean: 58.5534592 Estimate of (scaled) Consensus Mean: 0.4369068 Between Lab Variance (unscaled): 3.2312329 Between Lab SD (unscaled): 1.7975631 Between Lab Variance (scaled): 0.1462760 Standard Deviation of Consensus Mean: 0.8306379 Standard Uncertainty (k = 1): 0.8306379 Expanded Uncertainty (k = 2): 1.6612757 Expanded Uncertainty (k = 1.9599645): 1.6280208 Normal PPF of 0.975: 1.9599645 Lower 95% (normal) Confidence Limit: 56.9254379 Upper 95% (normal) Confidence Limit: 60.1814804 Note: Vangel-Rukhin Maximum Likelihood Best Usage: 6 or More Labs 4. Method: BOB (Bound on Bias) Estimate of Consensus Mean: 58.5955544 Within Lab Uncertainty: 0.2173445 Between Lab Uncertainty: 1.3567723 Standard Uncertainty (k = 1): 1.3740704 Expanded Uncertainty (k = 2): 2.7481408 Lower 95% (k = 2) Confidence Limit: 55.8474121 Upper 95% (k = 2) Confidence Limit: 61.3436966 Note: BOB Best Usage: 5 or Fewer Labs 5. Method: Schiller-Eberhardt Estimate of Consensus Mean: 58.5908279 Estimate of Variance of Mean: 0.0169179 Bias Allowance: 2.6091690 Sigmah (heterogeneity): 0.0000000 Degrees of Freedom for Sigmah: 1 Standard Uncertainty (k = 1): 2.7392378 Expanded Uncertainty (k = 2): 2.8693065 Expanded Uncertainty (k = 2.3645761): 2.9167265 Degrees of Freedom: 7 t Percent Point Value (alpha = 0.05): 2.3645761 Lower 95% Confidence Limit: 55.6741028 Upper 95% Confidence Limit: 61.5075531 Note: Schiller-Eberhardt Best Usage: 5 or Fewer Labs 6. Method: Mean of Means Mean of Lab Means: 58.5955544 Standard Deviation of Lab Means: 2.0532134 Standard Uncertainty (sd/sqrt(n)): 0.9182249 SD of Consensus Mean (sd/sqrt(n)): 0.9182249 Standard Uncertainty (k = 1): 0.9182249 Expanded Uncertainty (k = 2): 1.8364499 Expanded Uncertainty (k = 2.7764461): 2.5494020 Degrees of Freedom: 4 t Percent Point Value (alpha = 0.05): 2.7764461 Lower 95% (t-value) Confidence Limit: 56.0461540 Upper 95% (t-value) Confidence Limit: 61.1449547 Note: Mean of Means Best Usage: Any Number of Labs 7. Method: Graybill-Deal Estimate of Consensus Mean: 58.6732941 Estimate of Variance (Sinha): 0.0128360 Estimate of Variance (Naive): 0.0055405 Standard Uncertainty (Sinha) (k = 1): 0.1132961 Expanded Uncertainty (Sinha) (k = 2): 0.2265923 Lower 95% (Rukhin) Confidence Limit: 56.2782021 Upper 95% (Rukhin) Confidence Limit: 61.0683861 Note: Graybill-Deal Best Usage: Any Number of Labs, but no Between Lab Variance 8. Method: Grand Mean (No Lab Effect) Mean of All Data: 57.2260857 Standard Deviation of All Data: 2.0532134 SD of Consensus Mean (sd/sqrt(n)): 0.3027298 Standard Uncertainty (k = 1): 0.3027298 Expanded Uncertainty (k = 2): 0.6054596 Expanded Uncertainty (k = 2.0141039): 0.6097292 Degrees of Freedom: 45 t Percent Point Value (alpha = 0.05): 2.0141039 Lower 95% (t-value) Confidence Limit: 56.6163559 Upper 95% (t-value) Confidence Limit: 57.8358154 Note: Grand Mean Best Usage: Any Number of Labs, but no Lab-to-Lab Differences 9. Method: Generalized Confidence Intervals Estimate of Consensus Mean: 58.4525528 Standard Uncertainty (k = 1): 1.2792627 Expanded Uncertainty (k = 2): 2.5585253 Lower 95% (Simulation) Confidence Limit: 55.9661942 Upper 95% (Simulation) Confidence Limit: 61.0020752 Note: Generalized Confidence Interval Best Usage: Any Number of Labs, but no Between Lab Variance 10. Method: DerSimonian Laird Estimate of Consensus Mean: 58.5719872 Estimate of Variance of Consensus Mean: 0.8636000 Estimate of Between-Lab Variance: 5.0619205 Standard Uncertainty (k = 1): 0.9293008 Expanded Uncertainty (k = 2): 1.8586016 Degrees of Freedom: 4 t Percent Point Value: 2.7764461 Lower 95% (t-value) Confidence Limit: 55.9918327 Upper 95% (t-value) Confidence Limit: 61.1521416 Lower 95% (Rukhin) Confidence Limit: 56.0077209 Upper 95% (Rukhin) Confidence Limit: 61.1362534 Note: DerSimonian-Laird Best Usage: Any Number of Labs Table 2: 95% Confidence Limits Consensus Lower Upper Method Mean Limit Limit -------------------------------------------------------------------------- 1. Mandel-Paule 58.5663223 56.9361677 60.1964770 2. Modified Mandel-Paule 58.5590630 56.9246980 60.1934279 3. Vangel-Rukhin ML 58.5534592 56.9254384 60.1814799 4. BOB 58.5955544 55.8474121 61.3436966 5. Schiller-Eberhardt 58.5908279 55.6741015 61.5075544 6. Mean of Means 58.5955544 56.0461540 61.1449547 7. Graybill-Deal 58.6732941 56.2782006 61.0683875 8. Grand Mean 57.2260857 56.6163559 57.8358154 9. Generalized CI 58.4525528 55.9661958 61.0020750 10. DerSimonian-Laird (t) 58.5719872 55.9918327 61.1521416 (Rukhin) 58.5719872 56.0077202 61.1362541 -------------------------------------------------------------------------- Table 3: Standard Uncertainties (k = 1) Standard Relative Consensus Uncertainty Standard Method Mean (k = 1) Uncertainty (%) -------------------------------------------------------------------------- 1. Mandel-Paule 58.5663223 0.8317266 1.4201448 2. Modified Mandel-Paule 58.5590630 0.8338748 1.4239892 3. Vangel-Rukhin ML 58.5534592 0.8306379 1.4185975 4. BOB 58.5955544 1.3740704 2.3450079 5. Schiller-Eberhardt 58.5908279 2.7392378 4.6751986 6. Mean of Means 58.5955544 0.9182249 1.5670557 7. Graybill-Deal 58.6732941 0.1132961 0.1930966 8. Grand Mean 57.2260857 0.3027298 0.5290066 9. Generalized CI 58.4525528 1.2792627 2.1885488 10. DerSimonian-Laird 58.5719872 0.9293008 1.5865959 -------------------------------------------------------------------------- Table 4: Expanded Uncertainties (k = 2) Expanded Relative Consensus Uncertainty Expanded Method Mean (k = 2) Uncertainty (%) -------------------------------------------------------------------------- 1. Mandel-Paule 58.5663223 1.6634532 2.8402896 2. Modified Mandel-Paule 58.5590630 1.6677495 2.8479784 3. Vangel-Rukhin ML 58.5534592 1.6612757 2.8371949 4. BOB 58.5955544 2.7481408 4.6900158 5. Schiller-Eberhardt 58.5908279 2.8693066 4.8971944 6. Mean of Means 58.5955544 1.8364499 3.1341114 7. Graybill-Deal 58.6732941 0.2265923 0.3861932 8. Grand Mean 57.2260857 0.6054596 1.0580132 9. Generalized CI 58.4525528 2.5585253 4.3770976 10. DerSimonian-Laird 58.5719872 1.8586016 3.1731918 --------------------------------------------------------------------------
|
Privacy
Policy/Security Notice
NIST is an agency of the U.S.
Commerce Department.
Date created: 6/5/2001 |