PROFICIENCY TEST

PROFICIENCY TEST

Analysis Command

Generate the tables for either a one-sample or a two-sample proficiency test as defined by the ASTM E 2489 - 06 standard.

The following document
"Standard Practice for Statistical Analysis of One-Sample and Two-Sample Proficiency Testing Programs", ASTM International, 100 Barr Harbor Drive, PO BOX C700, West Conshohoceken, PA 19428-2959, USA.

describes a methodology for performing either a one-sample or a two-sample proficiency test.

Proficiency testing is the use of interlaboratory comparisons for the determination of laboratory testing or measurement performance. The methods in the E 2489- 06 standard provide direction for assessing and categorizing the performances of individual laboratories based on the relative likelihood of occurence of their test results.

The standard recommends a minimum of 10 laboratories and states that it is desirable to have 30 or more participating laboratories.

Proficiency testing programs typically have a larger number of participants than an interlaboratory test. This results in a wider variation of test conditions, so a proficiency test can provide more information regarding the precision of test results that may be expected when a test method is used in the general testing community.

In this standard, the median is used as the consensus value. The measure of variablility is the interquartile range. In this standard, the interquartile range is defined as the difference between the upper hinge and the lower hinge (this is slightly different than the standard definition of the interquartile range as the difference between the 75th percentile and the 25th percentile).

The lower hinge is the median of the points less than or equal to the median and the upper hinge is the median of the points greater than or equal to the median.

The inner fence is the value equal to the upper (lower) hinge of the data set plus (upper) or minus (lower) 1.5 times the interquartile range.

The outer fence is the value equal to the upper (lower) hinge of the data set plus (upper) or minus (lower) 3.0 times the interquartile range.

A test result that is between the lower inner fence and upper inner fence is labeled as "typical". A test result that is between the inner and outer fence values is labeled as "unusual". A test result that is beyond the outer fence values is labeled as "extremely unusual".

These statistics are used because they are both simple and robust. Note that the above values are used in generating a box plot.

The data consists of:
1. A response variable containing measurements on a sample
2. A lab-id variable

For a one-sample proficiency analysis, each lab reports a single test result.

This E 2489 one-sample proficiency analysis generates the three tables documented in the above document:

1. The test results sorted by lab-id.

   The purpose of this table is to make it easy to identify the results for a given laboratory.

2. The test results sorted in descending order with the median and lower and upper hinges marked. Each lab's result is categorized as "extremely unusual", "unusual", or typical.

   The purpose of this table is to show the range and distribution of the test results.

3. The test results sorted in descending order (as in table 2). However, the data are divided into bins.

   The purpose of this table is to show the range and distribution of the test results.

In addition to the tables, the standard also recommends complementing the tables with a dot plot. These are also known as dot diagrams or strip plots. In Dataplot, these are referred to as strip plots. Enter the command HELP STRIP PLOT for details on generating these plots in Dataplot. The first program example below demonstrates this plot.

A strip plot is an alternative to a histogram for displaying univariate data. The x-axis contains the value of the test result and the y-axis is simply a constant value. If two or more test results have the same value, the points are stacked vertically. You can draw the points are drawn as filled circles. Alternatively, you can draw the points drawn as the lab-id (this is useful for identifying outlying labs).

You can also generate the strip plot with the data divided into bins (you can specify the bin width and the starting and ending bin limits). In this form, the vertical axis will represent the number of occurences. This form of the strip plot is essentially a histogram.

Although the E 2489 - 06 standard does not explicitly talk about box plots, these can also be a useful complement to the tables since the box plot is a graphical representation of table 2.

The data consists of:
1. A response variable containing measurements on the first sample
2. A response variable containing measurements on the second sample
3. A lab-id variable

For a two-sample proficiency analysis, each lab reports exactly two test results (i.e., a single measurement on each sample).

The random error quantity is defined as

(X - Y) - (X_med -Y_med)

where X and Y denote the test results for sample one and sample two, respectively, and X_med and Y_med denote the medians of sample one and sample two, respectively.

This E 2489 two-sample proficiency analysis generates the three tables documented in the above document:

1. The test results for both samples sorted by lab-id.

   The purpose of this table is to make it easy to identify the results for a given laboratory.

2. The test results sorted in descending order of the random error quantity with the median and lower and upper hinges marked. The random error quantity for each lab's result is categorized as "extremely unusual", "unusual", or "typical".

   The purpose of this table is to show the range and distribution of the random error quantities.

3. The test results sorted in descending order of sample two with the median and lower and upper hinges marked. The test results for each sample are categorized as "extremely unusual", "unusual", or "typical".

   The purpose of this table is to show the range and distribution of the test results for each sample.

The standard also recommends complementing the tables with a Youden plot. This is demonstrated in the second program example below.

ONE SAMPLE PROFICIENCY TEST <y> <labid>
                        <SUBSET/EXCEPT/FOR qualification>
where <y> is a response variable;
            <labid> is a lab id variable;
and where the <SUBSET/EXCEPT/FOR qualification> is optional.

TWO SAMPLE PROFICIENCY TEST <y1> <y2> <labid>
                        <SUBSET/EXCEPT/FOR qualification>
where <y1> is the first response variable;
            <y2> is the second response variable;
            <labid> is a lab id variable;
and where the <SUBSET/EXCEPT/FOR qualification> is optional.

ONE SAMPLE PROFICIENCY TEST Y LABID
TWO SAMPLE PROFICIENCY TEST Y1 Y2 LABID

You can use the CAPTURE HTML command to generate these tables in HTML format. You can use the CAPTURE LATEX command to generate these tables in Latex format. You can use the CAPTURE RTF command to generate these tables in Rich Text Format (RTF).

In table 2 of the one sample proficiency test, if multiple laboratories have the same value, only the first laboratory id is printed along with the number of occurrences of that value.

For the "Unusual" and "Extremely Unusual" categories, it may be desirable to print the individual laboratory id's in the table.

To specify that for both the "Unusual" and "Extremely Unusual" categories all laborotories will be identified, enter

SET ONE SAMPLE PROFICIENCY TEST IDENTIFY LAB UNUSUAL

To specify that for only the "Extremely Unusual" category all laborotories will be identified, enter

SET ONE SAMPLE PROFICIENCY TEST IDENTIFY LAB EXTREMELY UNUSUAL

To reset the default, enter

SET ONE SAMPLE PROFICIENCY TEST IDENTIFY LAB DEFAULT

None

None

E691 INTERLAB	= Perform an E691 interlaboratory analysis.
STRIP PLOT	= Generate a strip plot.
BOX PLOT	= Generate a box plot.
YOUDEN PLOT	= Generate a Youden plot.
CAPTURE HTML	= Generate output in HTML format.
CAPTURE LATEX	= Generate output in Latex format.
CAPTURE RTF	= Generate output in RTF format.

"Standard Practice for Statistical Analysis of One-Sample and Two-Sample Proficiency Testing Programs", ASTM International, 100 Barr Harbor Drive, PO BOX C700, West Conshohoceken, PA 19428-2959, USA.

Interlaboratory Studies

2009/01
2019/08: Added the SET ONE SAMPLE PROFICIENCY TEST IDENTIFY LAB command

 
.
.  Read the data
.
SKIP 25
READ E2489A.DAT LABID Y
.
.  Generate the tables to the screen
.
ONE SAMPLE PROFICIENCY TEST Y LABID
.
.  Now generate the tables in RTF format (for import into Word)
.
CAPTURE RTF ONE.RTF
ONE SAMPLE PROFICIENCY TEST Y LABID
END OF CAPTURE
.
.  Generate the strip plot for the raw (unbinned) data
.
LABEL CASE ASIS
TITLE CASE ASIS
Y1LABEL Number of Occurences
X1LABEL Data Values
TITLE Dot Diagram for Original Data
Y1TIC MARKS OFF
Y1TIC MARK LABELS OFF
Y2FRAME OFF
X2FRAME OFF
YLIMITS 0 2
MAJOR YTIC MARK NUMBER 3
MINOR YTIC MARK NUMBER 0
SET STRIP PLOT STYLE STACK
SET STRIP PLOT INCREMENT 0.1
LET STRING SYMBL = CIRCLE
IF SYMBL = CIRCLE
   CHARACTER CIRCLE ALL
   CHARACTER FILL ON ALL
   CHARACTER HW 1 0.75 ALL
END OF IF
IF SYMBL = LABID
   CHARACTER AUTOMATIC LABIDSRT
END OF IF
LINE BLANK ALL
.
STRIP PLOT Y
.
Y1TIC MARKS ON
Y1TIC MARK LABELS ON
.
.  Generate the strip plot for the binned data
.
CLASS LOWER 0.5
CLASS UPPER 5
CLASS WIDTH 0.1
FRAME CORNER COORDINATES 15 40 85 70
Y1LABEL Number of Occurences
X1LABEL Data Values
TITLE Dot Diagram for Binned Data
Y2FRAME OFF
X2FRAME OFF
LET MAXFREQ = MAXIMUM Y2
LET NUMTIC = MAXFREQ + 1
YLIMITS 0 MAXFREQ
MAJOR YTIC MARK NUMBER NUMTIC
MINOR YTIC MARK NUMBER 0
Y1TIC OFFSET 1 1
SET STRIP PLOT STYLE STACK
SET STRIP PLOT INCREMENT 1
CHARACTER CIRCLE ALL
CHARACTER FILL ON ALL
CHARACTER HW 1 0.75 ALL
LINE BLANK ALL
.
STRIP PLOT Y2 X2
.
FRAME CORNER COORDINATES
CHAR FILL OFF ALL
CHAR BLANK ALL
CHAR HW
LINE SOLID ALL
LIMITS
FRAME ON
MAJOR TIC MARK NUMBER
MINOR TIC MARK NUMBER
.
.  Now generate a box plot
.
Y1LABEL Test Results
X1LABEL
TITLE Box Plot for Proficiency Data
XLIMITS 0 2
XTIC MARKS OFF
XTIC MARK LABELS OFF
CHARACTER BOX PLOT
LINE BOX PLOT
FENCES ON
.
BOX PLOT Y
XLIMITS
X1TIC MARKS ON
X1TIC MARK LABELS ON
    

View the output from this command.

 
.
.  Read the data
.
SKIP 25
READ E2489B.DAT Y1 Y2 LABID
.
.  Generate the tables to the screen
.
TWO SAMPLE PROFICIENCY TEST Y1 Y2 LABID
.
.  Now generate the tables in RTF format (for import into Word)
.
CAPTURE RTF TWO.RTF
TWO SAMPLE PROFICIENCY TEST Y1 Y2 LABID
END OF CAPTURE
.
.  Generate the Youden plot
.
LET STRING SYMBL = LABID
IF SYMBL = CIRCLE
   CHARACTER CIRCLE ALL
   CHARACTER FILL ON ALL
   CHARACTER HW 1 0.75 ALL
END OF IF
IF SYMBL = BOX
   CHARACTER BOX ALL
   CHARACTER FILL ON ALL
   CHARACTER HW 1 0.75 ALL
END OF IF
IF SYMBL = DIAMOND
   CHARACTER DIAMOND ALL
   CHARACTER FILL ON ALL
   CHARACTER HW 1 0.75 ALL
END OF IF
IF SYMBL = LABID
   LET LABIDSRT = LABID
   LET YSORT = SORTC Y LABIDSRT
   CHARACTER HW 2 1.50 ALL
   CHARACTER AUTOMATIC LABIDSRT
END OF IF
LINE BLANK ALL
.
LABEL CASE ASIS
TITLE CASE ASIS
TITLE OFFSET 2
X1LABEL Test Results for Sample One
Y1LABEL Test Results for Sample Two
TITLE Youden Plot of Test Results
.
LIMITS 0 5
TIC OFFSET UNITS DATA
TIC MARK OFFSET 0 0.5
.
YOUDEN PLOT Y2 Y1 LABID
.
PROBE FX1MIN
LET AX1 = PROBEVAL
PROBE FX1MAX
LET AX2 = PROBEVAL
PROBE FY1MIN
LET AY1 = PROBEVAL
PROBE FY1MAX
LET AY2 = PROBEVAL
.
LINE SOLID
DRAWDATA AX1 MEDX AX2 MEDX
DRAWDATA MEDY AY1 MEDY AY2
LINE DASH
DRAWDATA AX1 AY1 AX2 AY2
LINE SOLID
    

View the output from this command.