SED navigation bar go to SED home page go to Dataplot home page go to NIST home page SED Home Page SED Staff SED Projects SED Products and Publications Search SED Pages
Dataplot Vol 1 Vol 2

PROFICIENCY TEST

Name:
    PROFICIENCY TEST
Type:
    Analysis Command
Purpose:
    Generate the tables for either a one-sample or a two-sample proficiency test as defined by the ASTM E 2489 - 06 standard.
Description:
    The following document

      "Standard Practice for Statistical Analysis of One-Sample and Two-Sample Proficiency Testing Programs", ASTM International, 100 Barr Harbor Drive, PO BOX C700, West Conshohoceken, PA 19428-2959, USA.

    describes a methodology for performing either a one-sample or a two-sample proficiency test.

    Proficiency testing is the use of interlaboratory comparisons for the determination of laboratory testing or measurement performance. The methods in the E 2489- 06 standard provide direction for assessing and categorizing the performances of individual laboratories based on the relative likelihood of occurence of their test results.

    The standard recommends a minimum of 10 laboratories and states that it is desirable to have 30 or more participating laboratories.

    Proficiency testing programs typically have a larger number of participants than an interlaboratory test. This results in a wider variation of test conditions, so a proficiency test can provide more information regarding the precision of test results that may be expected when a test method is used in the general testing community.

    In this standard, the median is used as the consensus value. The measure of variablility is the interquartile range. In this standard, the interquartile range is defined as the difference between the upper hinge and the lower hinge (this is slightly different than the standard definition of the interquartile range as the difference between the 75th percentile and the 25th percentile).

    The lower hinge is the median of the points less than or equal to the median and the upper hinge is the median of the points greater than or equal to the median.

    The inner fence is the value equal to the upper (lower) hinge of the data set plus (upper) or minus (lower) 1.5 times the interquartile range.

    The outer fence is the value equal to the upper (lower) hinge of the data set plus (upper) or minus (lower) 3.0 times the interquartile range.

    A test result that is between the lower inner fence and upper inner fence is labeled as "typical". A test result that is between the inner and outer fence values is labeled as "unusual". A test result that is beyond the outer fence values is labeled as "extremely unusual".

    These statistics are used because they are both simple and robust. Note that the above values are used in generating a box plot.

Description of One-Sample Proficiency Test:
    The data consists of:

    1. A response variable containing measurements on a sample
    2. A lab-id variable

    For a one-sample proficiency analysis, each lab reports a single test result.

    This E 2489 one-sample proficiency analysis generates the three tables documented in the above document:

    1. The test results sorted by lab-id.

      The purpose of this table is to make it easy to identify the results for a given laboratory.

    2. The test results sorted in descending order with the median and lower and upper hinges marked. Each lab's result is categorized as "extremely unusual", "unusual", or typical.

      The purpose of this table is to show the range and distribution of the test results.

    3. The test results sorted in descending order (as in table 2). However, the data are divided into bins.

      The purpose of this table is to show the range and distribution of the test results.

    In addition to the tables, the standard also recommends complementing the tables with a dot plot. These are also known as dot diagrams or strip plots. In Dataplot, these are referred to as strip plots. Enter the command HELP STRIP PLOT for details on generating these plots in Dataplot. The first program example below demonstrates this plot.

    A strip plot is an alternative to a histogram for displaying univariate data. The x-axis contains the value of the test result and the y-axis is simply a constant value. If two or more test results have the same value, the points are stacked vertically. You can draw the points are drawn as filled circles. Alternatively, you can draw the points drawn as the lab-id (this is useful for identifying outlying labs).

    You can also generate the strip plot with the data divided into bins (you can specify the bin width and the starting and ending bin limits). In this form, the vertical axis will represent the number of occurences. This form of the strip plot is essentially a histogram.

    Although the E 2489 - 06 standard does not explicitly talk about box plots, these can also be a useful complement to the tables since the box plot is a graphical representation of table 2.

Description of Two-Sample Proficiency Test:
    The data consists of:

    1. A response variable containing measurements on the first sample
    2. A response variable containing measurements on the second sample
    3. A lab-id variable

    For a two-sample proficiency analysis, each lab reports exactly two test results (i.e., a single measurement on each sample).

    The random error quantity is defined as

      (X - Y) - (Xmed -Ymed)

    where X and Y denote the test results for sample one and sample two, respectively, and Xmed and Ymed denote the medians of sample one and sample two, respectively.

    This E 2489 two-sample proficiency analysis generates the three tables documented in the above document:

    1. The test results for both samples sorted by lab-id.

      The purpose of this table is to make it easy to identify the results for a given laboratory.

    2. The test results sorted in descending order of the random error quantity with the median and lower and upper hinges marked. The random error quantity for each lab's result is categorized as "extremely unusual", "unusual", or "typical".

      The purpose of this table is to show the range and distribution of the random error quantities.

    3. The test results sorted in descending order of sample two with the median and lower and upper hinges marked. The test results for each sample are categorized as "extremely unusual", "unusual", or "typical".

      The purpose of this table is to show the range and distribution of the test results for each sample.

    The standard also recommends complementing the tables with a Youden plot. This is demonstrated in the second program example below.

Syntax 1:
    ONE SAMPLE PROFICIENCY TEST <y> <labid>
                            <SUBSET/EXCEPT/FOR qualification>
    where <y> is a response variable;
                <labid> is a lab id variable;
    and where the <SUBSET/EXCEPT/FOR qualification> is optional.
Syntax 2:
    TWO SAMPLE PROFICIENCY TEST <y1> <y2> <labid>
                            <SUBSET/EXCEPT/FOR qualification>
    where <y1> is the first response variable;
                <y2> is the second response variable;
                <labid> is a lab id variable;
    and where the <SUBSET/EXCEPT/FOR qualification> is optional.
Examples:
    ONE SAMPLE PROFICIENCY TEST Y LABID
    TWO SAMPLE PROFICIENCY TEST Y1 Y2 LABID
Note:
    You can use the CAPTURE HTML command to generate these tables in HTML format. You can use the CAPTURE LATEX command to generate these tables in Latex format. You can use the CAPTURE RTF command to generate these tables in Rich Text Format (RTF).
Note:
    In table 2 of the one sample proficiency test, if multiple laboratories have the same value, only the first laboratory id is printed along with the number of occurrences of that value.

    For the "Unusual" and "Extremely Unusual" categories, it may be desirable to print the individual laboratory id's in the table.

    To specify that for both the "Unusual" and "Extremely Unusual" categories all laborotories will be identified, enter

      SET ONE SAMPLE PROFICIENCY TEST IDENTIFY LAB UNUSUAL

    To specify that for only the "Extremely Unusual" category all laborotories will be identified, enter

      SET ONE SAMPLE PROFICIENCY TEST IDENTIFY LAB EXTREMELY UNUSUAL

    To reset the default, enter

      SET ONE SAMPLE PROFICIENCY TEST IDENTIFY LAB DEFAULT
Default:
    None
Synonyms:
    None
Related Commands: Reference:
    "Standard Practice for Statistical Analysis of One-Sample and Two-Sample Proficiency Testing Programs", ASTM International, 100 Barr Harbor Drive, PO BOX C700, West Conshohoceken, PA 19428-2959, USA.
Applications:
    Interlaboratory Studies
Implementation Date:
    2009/01
    2019/08: Added the SET ONE SAMPLE PROFICIENCY TEST IDENTIFY LAB command
Program 1:
     
    .
    .  Read the data
    .
    SKIP 25
    READ E2489A.DAT LABID Y
    .
    .  Generate the tables to the screen
    .
    ONE SAMPLE PROFICIENCY TEST Y LABID
    .
    .  Now generate the tables in RTF format (for import into Word)
    .
    CAPTURE RTF ONE.RTF
    ONE SAMPLE PROFICIENCY TEST Y LABID
    END OF CAPTURE
    .
    .  Generate the strip plot for the raw (unbinned) data
    .
    LABEL CASE ASIS
    TITLE CASE ASIS
    Y1LABEL Number of Occurences
    X1LABEL Data Values
    TITLE Dot Diagram for Original Data
    Y1TIC MARKS OFF
    Y1TIC MARK LABELS OFF
    Y2FRAME OFF
    X2FRAME OFF
    YLIMITS 0 2
    MAJOR YTIC MARK NUMBER 3
    MINOR YTIC MARK NUMBER 0
    SET STRIP PLOT STYLE STACK
    SET STRIP PLOT INCREMENT 0.1
    LET STRING SYMBL = CIRCLE
    IF SYMBL = CIRCLE
       CHARACTER CIRCLE ALL
       CHARACTER FILL ON ALL
       CHARACTER HW 1 0.75 ALL
    END OF IF
    IF SYMBL = LABID
       CHARACTER AUTOMATIC LABIDSRT
    END OF IF
    LINE BLANK ALL
    .
    STRIP PLOT Y
    .
    Y1TIC MARKS ON
    Y1TIC MARK LABELS ON
    .
    .  Generate the strip plot for the binned data
    .
    CLASS LOWER 0.5
    CLASS UPPER 5
    CLASS WIDTH 0.1
    FRAME CORNER COORDINATES 15 40 85 70
    Y1LABEL Number of Occurences
    X1LABEL Data Values
    TITLE Dot Diagram for Binned Data
    Y2FRAME OFF
    X2FRAME OFF
    LET MAXFREQ = MAXIMUM Y2
    LET NUMTIC = MAXFREQ + 1
    YLIMITS 0 MAXFREQ
    MAJOR YTIC MARK NUMBER NUMTIC
    MINOR YTIC MARK NUMBER 0
    Y1TIC OFFSET 1 1
    SET STRIP PLOT STYLE STACK
    SET STRIP PLOT INCREMENT 1
    CHARACTER CIRCLE ALL
    CHARACTER FILL ON ALL
    CHARACTER HW 1 0.75 ALL
    LINE BLANK ALL
    .
    STRIP PLOT Y2 X2
    .
    FRAME CORNER COORDINATES
    CHAR FILL OFF ALL
    CHAR BLANK ALL
    CHAR HW
    LINE SOLID ALL
    LIMITS
    FRAME ON
    MAJOR TIC MARK NUMBER
    MINOR TIC MARK NUMBER
    .
    .  Now generate a box plot
    .
    Y1LABEL Test Results
    X1LABEL
    TITLE Box Plot for Proficiency Data
    XLIMITS 0 2
    XTIC MARKS OFF
    XTIC MARK LABELS OFF
    CHARACTER BOX PLOT
    LINE BOX PLOT
    FENCES ON
    .
    BOX PLOT Y
    XLIMITS
    X1TIC MARKS ON
    X1TIC MARK LABELS ON
        
    View the output from this command.

    plot generated by sample program

    plot generated by sample program

    plot generated by sample program

Program 2:
     
    .
    .  Read the data
    .
    SKIP 25
    READ E2489B.DAT Y1 Y2 LABID
    .
    .  Generate the tables to the screen
    .
    TWO SAMPLE PROFICIENCY TEST Y1 Y2 LABID
    .
    .  Now generate the tables in RTF format (for import into Word)
    .
    CAPTURE RTF TWO.RTF
    TWO SAMPLE PROFICIENCY TEST Y1 Y2 LABID
    END OF CAPTURE
    .
    .  Generate the Youden plot
    .
    LET STRING SYMBL = LABID
    IF SYMBL = CIRCLE
       CHARACTER CIRCLE ALL
       CHARACTER FILL ON ALL
       CHARACTER HW 1 0.75 ALL
    END OF IF
    IF SYMBL = BOX
       CHARACTER BOX ALL
       CHARACTER FILL ON ALL
       CHARACTER HW 1 0.75 ALL
    END OF IF
    IF SYMBL = DIAMOND
       CHARACTER DIAMOND ALL
       CHARACTER FILL ON ALL
       CHARACTER HW 1 0.75 ALL
    END OF IF
    IF SYMBL = LABID
       LET LABIDSRT = LABID
       LET YSORT = SORTC Y LABIDSRT
       CHARACTER HW 2 1.50 ALL
       CHARACTER AUTOMATIC LABIDSRT
    END OF IF
    LINE BLANK ALL
    .
    LABEL CASE ASIS
    TITLE CASE ASIS
    TITLE OFFSET 2
    X1LABEL Test Results for Sample One
    Y1LABEL Test Results for Sample Two
    TITLE Youden Plot of Test Results
    .
    LIMITS 0 5
    TIC OFFSET UNITS DATA
    TIC MARK OFFSET 0 0.5
    .
    YOUDEN PLOT Y2 Y1 LABID
    .
    PROBE FX1MIN
    LET AX1 = PROBEVAL
    PROBE FX1MAX
    LET AX2 = PROBEVAL
    PROBE FY1MIN
    LET AY1 = PROBEVAL
    PROBE FY1MAX
    LET AY2 = PROBEVAL
    .
    LINE SOLID
    DRAWDATA AX1 MEDX AX2 MEDX
    DRAWDATA MEDY AY1 MEDY AY2
    LINE DASH
    DRAWDATA AX1 AY1 AX2 AY2
    LINE SOLID
        
    View the output from this command.

    plot generated by sample program

Date created: 01/14/2009
Last updated: 12/11/2023

Please email comments on this WWW page to alan.heckert@nist.gov.