F TEST

F TEST

Analysis Command

Perform a two sample F test that the ratio of two standard deviations is equal to 1 (i.e., the standard deviations are equal).

This F test can be either an upper tailed test or a two-sided test.

The two-sided hypothesis test is:

H0:	\( \frac{\sigma_1} {\sigma_2} = 1 \)
Ha:	\( \frac{\sigma_1} {\sigma_2} \ne 1 \)
Test Statistic:	\( \mbox{F} = \frac{s_{1}^{2}} {s_{2}^{2}} \) with \( s_1 \) and \( s_2 \) denoting the sample standard deviations
Alpha:	Typically set to .05
Critical Region:	\( \mbox{F} < f(\alpha/2)(\nu_1,\nu_2) \) or \( \mbox{F} > f(1-\alpha/2)(\nu_1,\nu_2) \) and where the critical region is determined from the F distribution function with \( \nu_1-1 \) and \( \nu_2-1 \) degrees of freedom
Conclusion:	Reject the null hypothesis if F in critical region

Note that for the upper tailed test, the larger of \( s_1 \) and \( s_2 \) is in the numerator and the smaller is in the denominator of the test statistic.

The upper tailed hypothesis test is:

H0:	\( \frac{max(\sigma_1,\sigma_2)} {min(\sigma_1,\sigma_2)} = 1 \)
Ha:	\( \frac{max(\sigma_1,\sigma_2)} {min(\sigma_1,\sigma_2)} > 1 \)
Test Statistic:	\( \mbox{F} = \frac{max(s_1,s_2)^2} {min(s_1,s_2)^2} \) with \( s_1 \) and \( s_2 \) denoting the sample standard deviations
Alpha:	Typically set to .05
Critical Region:	\( \mbox{F} > f(1-\alpha,\nu_1,\nu_2) \) where the critical region is determined from the F distribution function with \( \nu_1 - 1\) and \( \nu_2 - 1\) degrees of freedom
Conclusion:	Reject null hypothesis if F in critical region

To specify the upper tailed test (the default), enter

SET F TEST TYPE UPPER TAILED

To specify the two-tailed test, enter

SET F TEST TYPE TWO SIDED

The test conclusions are given for several values of alpha.

Note that the F test is known to be quite senstive to departures from normality. There have been several proposals in the literature to make this test more robust. This is discussed further in the "Note:" section below.

F TEST <y1> <y2>             <SUBSET/EXCEPT/FOR qualification>
where <y1> is the first response variable;
            <y2> is the second response variable;
and where the <SUBSET/EXCEPT/FOR qualification> is optional.

F TEST <y1> ... <yk>             <SUBSET/EXCEPT/FOR qualification>
where <y1> ... <yk> is a list of two or more response variables;
and where the <SUBSET/EXCEPT/FOR qualification> is optional.

This syntax performs all the two-way F tests for the listed variables. This syntax supports the TO syntax.

F TEST Y1 Y2
F TEST Y1 Y2 Y3
F TEST Y1 TO Y10
F TEST Y1 Y2 SUBSET Y2 > 0


SET F TEST TYPE TWO SIDED
F TEST Y1 Y2


SET F TEST TYPE UPPER TAILED
F TEST Y1 Y2

The F test is known to be quite sensitive to departures from normality.

Shoemaker (2003) proposed the following adjustment to the degrees of freedom to make the F test more robust against non-normality.

The degrees of freedom are

\( r_{i} = \frac{2 n_{i}} {\frac{\mu_{4}}{\sigma^{4}} - \frac{n_{i}-3}{n_{i}-1}} \)

where

\( \mu_{4} = \frac{\sum_{i=1}^{2}{\sum_{j=1}^{n_i} {(y_{ij} - \bar{y}_{i})^{4}}}} {n_{1} + n_{2}} \)

\( \sigma^{2} = \frac{(n_{1} - 1) s_{1}^{2} + (n_{2} - 1) s_{2}^{2}} {n_{1} + n_{2}} \)

with \( y_{ij} \), \( n_{i} \), \( s_{i} \) and \( \bar{y}_{i} \) denoting the data, the sample size, the standard deviation and the mean, respectively, of the i-th variable.

This typically results in fractional degrees of freedom. The Dataplot FCDF, FPDF and FPPF routines were updated to support fractional degrees of freedom (fractional degrees of freedom less than 1 are set to 1).

The justification and derivation of this modification is given in Shoemaker's paper. Simulations by Shoemaker indicated that the standard F test tends to be too conservative for light tailed distributions and too liberal for heavy tailed and skewed distributions. Using the Shoemaker modifications for degrees of freedom generally had good coverage properties with the exception of heavily skewed distributions with small samples where it was still too liberal.

Bonett (2006) suggested that rounding the numerator degrees of freedom up to the nearest integer degrees of freedom and rounding the denominator degrees of freedom down to the nearest integer degrees of freedom (but not less than 1) prevented some anomalous behavior while reducing coverage performance only slightly.

Bonett proposed the following statistic:

\( \exp{ \frac{ \ln{ \frac{c s_{1}^{2}} {s_{2}^{2}} }} {se}} \)

where

\( \gamma_{4} \)	=	\( (n_1 + n_2) \frac{ \sum_{i=1}^{2}{\sum_{j=1}^{n_i}{(y_{ij} - m_{i})^{4}}}} {(\sum_{i=1}^{2}{\sum_{j=1}^{n_i}{(y_{ij} - \mu_{i})^{2}}} )^{2}} \)
\( m_{i} \)	=	sample trimmed mean of the i-th variable with trimming proportion \( \frac{1}{\sqrt{2 (n{i} - 4)}} \)
\( \bar{y_{i}} \)	=	sample mean of the i-th variable
se	=	\( \sqrt{ \frac{\gamma_{4} - k_{1}} {n_{1} - 1} + \frac{\gamma_{4} - k_{2}} {n_{2} - 1} } \)
\( k_1 \)	=	\( \frac{n_{1}-3}{n_{1}} \)
\( k_2 \)	=	\( \frac{n_{2}-3}{n_{2}} \)
c	=	\( \frac{\frac{n_{1}} {n_{1} - \Phi^{-1}(\alpha/2)}} {\frac{n_{2}} {n_{2} - \Phi^{-1}(\alpha/2)}} \)
\( \Phi^{-1} \)	=	the percent point function of the normal distribution

The parameter c is a small sample adjustment to help equalize the tail probabilities. It equals 1 when n1 and n2 equal and approaches 1 as n1 and n2 get large. For unequal sample sizes, the value of the test statistic will vary slightly depending on the value of alpha.

The above formula is for the variance. For the standard deviation, take the square root.

This test statistic is compared to a standard normal distribution.

The justification and derivation of this test is given in the Bonett paper. Based on his simulations, he claims this method improves somewhat on the Shoemaker modification for heavy-tailed (particularly skewed) distributions.

To specify Shoemaker's modification with fractional degrees of freedom, enter

SET F TEST DEGREES OF FREEDOM SHOEMAKER

To specify Shoemaker's modification with integral degrees of freedom as suggested by Bonett, enter

SET F TEST DEGREES OF FREEDOM SHOEMAKER ROUNDED

To specify Bonett's method, enter

SET F TEST DEGREES OF FREEDOM BONETT

Shoemaker suggests that when more than two groups of data are available, it can improve accuracy to use all of the groups in the estimates of the pooled location (mu4) and pooled variance (sigma**2) instead of just the two groups being tested. That is,
\( \mu_{4} = \frac{\sum_{i=1}^{k}{\sum_{j=1}^{n_i} {(y_{ij} - \bar{y}_{i})^{4}}}} {n_{1} + n_{2} + ... + n_{k}} \)

\( \sigma^{2} = \frac{(n_{1} - 1) s_{1}^{2} + (n_{2} - 1) s_{2}^{2} + ... + (n_{k} - 1) s_{k}} {n_{1} + n_{2} + ... + n_{k}} \)

To use these values, enter the commands

SET SHOEMAKER F TEST POOLED MU <value>
SET SHOEMAKER F TEST POOLED VARIANCE <value>


To reset the default of Dataplot computing these from the two groups being tested, enter

SET SHOEMAKER F TEST POOLED MU
SET SHOEMAKER F TEST POOLED VARIANCE 0

The following parameters are saved after the F test is performed.

STATVAL	-	value of the test statistic
STATCDF	-	CDF of the test statistic
PVALUE	-	p-value of the test statistic
STATNU1	-	degrees of the freedom of the numerator variable
STATNU2	-	degrees of the freedom of the denominator variable
POOLSD	-	pooled standard deviation

For an upper tailed test, the following parameters are saved

CUTUPP50	-	upper critical value for 50% test
CUTUPP75	-	upper critical value for 50% test
CUTUPP90	-	upper critical value for 90% test
CUTUPP95	-	upper critical value for 95% test
CUTUPP99	-	upper critical value for 99% test
CUTUP999	-	upper critical value for 99.9% test

For a two sided test, the following are saved

CUTLOW80	-	lower critical value for 80% test
CUTUPP80	-	upper critical value for 80% test
CUTLOW90	-	lower critical value for 90% test
CUTUPP90	-	upper critical value for 90% test
CUTLOW95	-	lower critical value for 95% test
CUTUPP95	-	upper critical value for 95% test
CUTLOW99	-	lower critical value for 99% test
CUTUPP99	-	upper critical value for 99% test
CUTLW999	-	lower critical value for 99.9% test
CUTUP999	-	upper critical value for 99.9% test

If the Bonett method is specified, a two sided interval will always be used. Also, STATNU1, STATNU2 and POOLSD will not be saved.

In addition to the F TEST command, the following commands can also be used:
LET STATVAL = F TEST Y1 Y2
LET STATCDF = F TEST CDF Y1 Y2
LET PVALUE = F TEST PVALUE Y1 Y2


In addition to the above LET commands, built-in statistics are supported for 30+ different commands (enter HELP STATISTICS for details).

The RATIO OF STANDARD DEVIATIONS CONFIDENCE LIMITS can be used to generate the corresponding confidence limits.

An upper tailed test without the Shoemaker or Bonett modifications is performed.

None

RATIO OF SD CONFIDENCE LIMITS	=	Compute the confidence limtis for the ratio of two standard deviations.
SIEGEL TUKEY TEST	=	Perform a ranks based test that two standard deviations are equal.
BARTLETT TEST	=	Perform a k-sample Bartlett test for homogeneous variances.
LEVENE TEST	=	Perform a k-sample Levene test for homogeneous variances.
SQUARED RANKS TEST	=	Perform a k-sample squared ranks test for homogeneous variances.
KLOTZ TEST	=	Perform a k-sample Klotz test for homogeneous variances.
SD CONFIDENCE LIMITS	=	Compute the confidence limits for the standard deviation.
CHI-SQUARE TEST	=	Performs a one sample chi-square test that the standard deviation is equal to a given value.
T TEST	=	Performs a two-sample t test for equal means.
CONFIDENCE LIMITS	=	Compute the confidence limits for the mean of a sample.
STANDARD DEVIATION	=	Computes the standard deviation of a variable.

F tests are discussed in most introductory statistics books.

Shoemaker (2003), "Fixing the F-Test for Equal Variances," The American Statistician, Vol. 57, pp. 105-114.

Bonett (2006), "Robust Confidence Intervals for a Ratio of Standard Deviations," Applied Pyschological Measurement, Vol. 30, No. 5, pp. 432-439.

Confirmatory Data Analysis

1994/02
2023/06: Added support for Shoemaker and Bonett modifications
2023/06: Added support for two sided tests

 
    
 
SKIP 25
READ AUTO83B.DAT Y1 Y2
DELETE Y2 SUBSET Y2 < 0
.
F TEST Y1 Y2
    
    The following output is generated
    
    
             Two Sample F-Test for Equal Standard Deviations
  
 First Response Variable:  Y1
 Second Response Variable: Y2
  
 H0: Sigma1/Sigma2 = 1
 Ha: Sigma1/Sigma2 > 1
  
 Sample One Summary Statistics:
 Number of Observations:                             249
 Sample Mean:                                   20.14458
 Sample Standard Deviation:                      6.41470
  
 Sample Two Summary Statistics:
 Number of Observations:                              79
 Sample Mean:                                   30.48101
 Sample Standard Deviation:                      6.10771
  
 Test:
 Standard Deviation (Numerator):                 6.41470
 Standard Deviation (Denomerator):               6.10771
 Degrees of Freedom (Numerator):                     248
 Degrees of Freedom (Denomerator):                    78
 Pooled Standard Deviation:                      6.34260
 F-Test Statistic Value:                         1.10305
 F-Test CDF Value:                               0.69032
 F-Test P-Value:                                 0.30968
  
  
             Conclusions (Upper 1-Tailed Test)
  
 H0: sigma1/sigma2 = 1; Ha: sigma1/sigma2 > 1
 ------------------------------------------------------------
                                                         Null
    Significance           Test       Critical     Hypothesis
           Level      Statistic    Region (>=)     Conclusion
 ------------------------------------------------------------
           50.0%        1.10305        1.00590         REJECT
           75.0%        1.10305        1.14047         ACCEPT
           90.0%        1.10305        1.28015         ACCEPT
           95.0%        1.10305        1.37347         ACCEPT
           97.5%        1.10305        1.46106         ACCEPT
           99.0%        1.10305        1.57145         ACCEPT
           99.9%        1.10305        1.83565         ACCEPT
    
    
    

 
. Step 1:   Read the data
.
skip 25
read shoemake.dat bottom mid surface
let y x = stack bottom mid surface
let temp1 = cross tabulate mean y x
let temp2 = (y - temp1)**4
let num = sum temp2
let den = size y
let mu4 = num/den
let y1var = variance bottom
let y2var = variance mid
let y3var = variance surface
let n1 = size bottom
let n2 = size mid
let n3 = size surface
let num   = (n1 - 1)*y1var + (n2 -1)*y2var + (n3 - 1)*y3var
let poolvar = num/den
.
. Step 2:   Generate a box plot
.
character box plot
line box plot
xlimits 1 3
major xtic mark number 3
minor xtic mark number 0
xtic mark offset 0.5 0.5
x1tic mark label format alpha
x1tic mark label content Bottom Middepth Surface
tic mark label case asis
set box plot fences on
box plot y x



.
. Step 3:   Default F-test
.
set f test degrees of freedom default
print "Standard F Test"
f test bottom mid
let statval = f test        bottom mid
let statcdf = f test cdf    bottom mid
let pvalue  = f test pvalue bottom mid
print statval statcdf pvalue

The following output is generated

Standard F Test
  
             Two Sample F-Test for Equal Standard Deviations
  
 First Response Variable:  BOTTOM
 Second Response Variable: MID
  
 H0: Sigma1/Sigma2 = 1
 Ha: Sigma1/Sigma2 > 1
  
 Sample One Summary Statistics:
 Number of Observations:                              10
 Sample Mean:                                    6.02100
 Sample Standard Deviation:                      1.58184
  
 Sample Two Summary Statistics:
 Number of Observations:                              10
 Sample Mean:                                    5.01900
 Sample Standard Deviation:                      1.10440
  
 Test:
 Standard Deviation (Numerator):                 1.58184
 Standard Deviation (Denomerator):               1.10440
 Degrees of Freedom (Numerator):                       9
 Degrees of Freedom (Denomerator):                     9
 Pooled Standard Deviation:                      1.36417
 F-Test Statistic Value:                         2.05152
 F-Test CDF Value:                               0.85031
 F-Test P-Value:                                 0.14969
  
  
             Conclusions (Upper 1-Tailed Test)
  
 H0: sigma1/sigma2 = 1; Ha: sigma1/sigma2 > 1
 ------------------------------------------------------------
                                                         Null
    Significance           Test       Critical     Hypothesis
           Level      Statistic    Region (>=)     Conclusion
 ------------------------------------------------------------
           50.0%        2.05152        1.00000         REJECT
           75.0%        2.05152        1.59090         REJECT
           90.0%        2.05152        2.44034         ACCEPT
           95.0%        2.05152        3.17889         ACCEPT
           97.5%        2.05152        4.02599         ACCEPT
           99.0%        2.05152        5.35113         ACCEPT
           99.9%        2.05152       10.10663         ACCEPT
  
  
 THE COMPUTED VALUE OF THE CONSTANT STATVAL       =    2.051516
  
  
 THE COMPUTED VALUE OF THE CONSTANT STATCDF       =   0.8503082
  
  
 THE COMPUTED VALUE OF THE CONSTANT PVALUE        =   0.1496918
  

 PARAMETERS AND CONSTANTS--

    STATVAL --        2.05152
    STATCDF --        0.85031
    PVALUE  --        0.14969

.
. Step 4:   Shoemaker F-test
.
set f test degrees of freedom shoemaker
print "Shoemaker degrees of freedom F Test"
f test bottom mid
print "Use pooled location and variance for Shoemaker"
set shoemaker f test pooled variance poolvar
set shoemaker f test pooled mu       mu4
f test bottom mid
print "Shoemaker rounded degrees of freedom F Test"
set f test degrees of freedom shoemaker rounded
f test bottom mid
set shoemaker f test pooled variance 0
set shoemaker f test pooled mu

The following output is generated

Shoemaker degrees of freedom F Test
  
             Two Sample F-Test for Equal Standard Deviations
             (Use Shoemaker Modifications to Degrees of Freedom)
  
 First Response Variable:  BOTTOM
 Second Response Variable: MID
  
 H0: Sigma1/Sigma2 = 1
 Ha: Sigma1/Sigma2 > 1
  
 Sample One Summary Statistics:
 Number of Observations:                              10
 Sample Mean:                                    6.02100
 Sample Standard Deviation:                      1.58184
  
 Sample Two Summary Statistics:
 Number of Observations:                              10
 Sample Mean:                                    5.01900
 Sample Standard Deviation:                      1.10440
  
 Test:
 Standard Deviation (Numerator):                 1.58184
 Standard Deviation (Denomerator):               1.10440
 Degrees of Freedom (Numerator):                12.42315
 Degrees of Freedom (Denomerator):              12.42315
 Pooled Standard Deviation:                      1.36417
 F-Test Statistic Value:                         2.05152
 F-Test CDF Value:                               0.89036
 F-Test P-Value:                                 0.10964
  
  
             Conclusions (Upper 1-Tailed Test)
  
 H0: sigma1/sigma2 = 1; Ha: sigma1/sigma2 > 1
 ------------------------------------------------------------
                                                         Null
    Significance           Test       Critical     Hypothesis
           Level      Statistic    Region (>=)     Conclusion
 ------------------------------------------------------------
           50.0%        2.05152        1.00000         REJECT
           75.0%        2.05152        1.47952         REJECT
           90.0%        2.05152        2.11767         ACCEPT
           95.0%        2.05152        2.63791         ACCEPT
           97.5%        2.05152        3.20501         ACCEPT
           99.0%        2.05152        4.04346         ACCEPT
           99.9%        2.05152        6.73615         ACCEPT
  
Use pooled location and variance for Shoemaker
  
 THE FORTRAN COMMON SCALAR SHOEF    HAS JUST BEEN SET TO   0.1251562E+01
  
 THE FORTRAN COMMON SCALAR SHOEF    HAS JUST BEEN SET TO   0.4585221E+01
  
             Two Sample F-Test for Equal Standard Deviations
             (Use Shoemaker Modifications to Degrees of Freedom)
  
 First Response Variable:  BOTTOM
 Second Response Variable: MID
  
 H0: Sigma1/Sigma2 = 1
 Ha: Sigma1/Sigma2 > 1
  
 Sample One Summary Statistics:
 Number of Observations:                              10
 Sample Mean:                                    6.02100
 Sample Standard Deviation:                      1.58184
  
 Sample Two Summary Statistics:
 Number of Observations:                              10
 Sample Mean:                                    5.01900
 Sample Standard Deviation:                      1.10440
  
 Test:
 Standard Deviation (Numerator):                 1.58184
 Standard Deviation (Denomerator):               1.10440
 Degrees of Freedom (Numerator):                 9.30474
 Degrees of Freedom (Denomerator):               9.30474
 Pooled Standard Deviation:                      1.36417
 F-Test Statistic Value:                         2.05152
 F-Test CDF Value:                               0.85454
 F-Test P-Value:                                 0.14546
  
  
             Conclusions (Upper 1-Tailed Test)
  
 H0: sigma1/sigma2 = 1; Ha: sigma1/sigma2 > 1
 ------------------------------------------------------------
                                                         Null
    Significance           Test       Critical     Hypothesis
           Level      Statistic    Region (>=)     Conclusion
 ------------------------------------------------------------
           50.0%        2.05152        1.00000         REJECT
           75.0%        2.05152        1.57799         REJECT
           90.0%        2.05152        2.40166         ACCEPT
           95.0%        2.05152        3.11256         ACCEPT
           97.5%        2.05152        3.92318         ACCEPT
           99.0%        2.05152        5.18299         ACCEPT
           99.9%        2.05152        9.64550         ACCEPT
  
Shoemaker rounded degrees of freedom F Test
  
 THE FORTRAN COMMON CHARACTER VARIABLE F   TEST HAS JUST BEEN SET TO SHO2
  
             Two Sample F-Test for Equal Standard Deviations
             (Shoemaker Modifications with Rounded Degrees of Freedom)
  
 First Response Variable:  BOTTOM
 Second Response Variable: MID
  
 H0: Sigma1/Sigma2 = 1
 Ha: Sigma1/Sigma2 > 1
  
 Sample One Summary Statistics:
 Number of Observations:                              10
 Sample Mean:                                    6.02100
 Sample Standard Deviation:                      1.58184
  
 Sample Two Summary Statistics:
 Number of Observations:                              10
 Sample Mean:                                    5.01900
 Sample Standard Deviation:                      1.10440
  
 Test:
 Standard Deviation (Numerator):                 1.58184
 Standard Deviation (Denomerator):               1.10440
 Degrees of Freedom (Numerator):                      10
 Degrees of Freedom (Denomerator):                     9
 Pooled Standard Deviation:                      1.37649
 F-Test Statistic Value:                         2.05152
 F-Test CDF Value:                               0.85275
 F-Test P-Value:                                 0.14725
  
  
             Conclusions (Upper 1-Tailed Test)
  
 H0: sigma1/sigma2 = 1; Ha: sigma1/sigma2 > 1
 ------------------------------------------------------------
                                                         Null
    Significance           Test       Critical     Hypothesis
           Level      Statistic    Region (>=)     Conclusion
 ------------------------------------------------------------
           50.0%        2.05152        1.00774         REJECT
           75.0%        2.05152        1.58634         REJECT
           90.0%        2.05152        2.41632         ACCEPT
           95.0%        2.05152        3.13728         ACCEPT
           97.5%        2.05152        3.96387         ACCEPT
           99.0%        2.05152        5.25654         ACCEPT
           99.9%        2.05152        9.89430         ACCEPT

.
. Step 5:   Bonett
.
set f test degrees of freedom bonett
print "Bonett method"
f test bottom mid
    

The following output is generated

Bonett method
  
             Two Sample F-Test for Equal Standard Deviations
                     (Bonett Method for Robustness)
  
 First Response Variable:  BOTTOM
 Second Response Variable: MID
  
 H0: Sigma1/Sigma2 = 1
 Ha: Sigma1/Sigma2 > 1
  
 Sample One Summary Statistics:
 Number of Observations:                              10
 Sample Mean:                                    6.02100
 Sample Standard Deviation:                      1.58184
  
 Sample Two Summary Statistics:
 Number of Observations:                              10
 Sample Mean:                                    5.01900
 Sample Standard Deviation:                      1.10440
  
 Test:
 Standard Deviation (Numerator):                 1.58184
 Standard Deviation (Denomerator):               1.10440
 Test Statistic Value (alpha = 0.05):            3.23288
 Test CDF Value (alpha = 0.05):                  0.99939
 Test P-Value (alpha = 0.05):                    0.00123
  
  
             Conclusions
  
 H0: sigma1/sigma2 = 1; Ha: sigma1/sigma2 <> 1
 ---------------------------------------------------------------------------
                                         Lower          Upper           Null
    Significance           Test       Critical       Critical     Hypothesis
           Level      Statistic    Region (<=)    Region (>=)     Conclusion
 ---------------------------------------------------------------------------
           50.0%        3.23288       -0.67449        0.67449         REJECT
           80.0%        3.23288       -1.28155        1.28155         REJECT
           90.0%        3.23288       -1.64485        1.64485         REJECT
           95.0%        3.23288       -1.95996        1.95996         REJECT
           99.0%        3.23288       -2.57583        2.57583         REJECT
           99.9%        3.23288       -3.29053        3.29053         ACCEPT