By Thomas Murphy

Jul 26, 2016

The Benefits of E691

Q: ASTM International standard E691, Practice for Conducting an Interlaboratory Study to Determine the Precision of a Test Method, recommends a particular design, but other ASTM committees suggest alternative designs. What are the benefits of the E691 design?

A. According to section A21.2 of Form and Style for ASTM Standards, an interlaboratory study is mandated for all ASTM International test methods. The purpose of the ILS is to estimate the precision of an ASTM test method, specifically the repeatability and reproducibility. Both concepts have been discussed previously in DataPoints (see the March/April, May/June and July/August 2009 columns in SN). An ILS is, in fact, a designed experiment (i.e., an operation carried out under controlled conditions to estimate particular effects).

A few ASTM International committees have well-organized ILS programs for their test methods and have supporting standards and subcommittees for this purpose, including statistical resources for data analysis. However, many other ASTM committees have limited resources and experience with ILS work, and the ASTM staff-supported ILS Program, which assists committees in conducting their studies, often recommends the E691 design.

What Is the E691 Design?

The E691 design involves two factors (controlled variables): laboratories and materials. The E691 design matrix is a two-way table with labs as rows and materials as columns (Table 1). Each row-column combination (called a "cell") represents a group of two or more test results carried out under repeatability conditions (same operator, same instrument, same day, etc., for each of the materials). The ILS design consists of p laboratories, each running n tests on each of m materials, for a total of N = pnm test results in the study.

Table 1 - E691 design matrix for eight laboratories, five materials, and three replicates per cell1

1Each "x" represents a test result.

The key number in the ILS design is the number of laboratories because this determines how well the reproducibility precision is estimated. The recommended minimum number of labs is six, but the more labs that can be enrolled the better. With p = 6 labs, the uncertainty in the reproducibility standard deviation is roughly 62 to 245 percent of the estimate and with p = 30 labs (the recommended number), the uncertainty reduces to 80 to 134 percent of the estimated value, both uncertainties stated at 95 percent confidence.

The number of replicate test results in each cell should be constant within the matrix for ease of statistical analysis, but n = 2 to 5 replicates per cell is recommended, depending on the cost and the logistics of carrying out the multiple tests under repeatability conditions. For n = 2, the repeatability uncertainty is roughly the same as for the reproducibility. For small numbers of labs in the ILS, the repeatability uncertainty can be reduced to some extent by increasing n, but the reproducibility uncertainty will not be affected.

The number of materials is dependent on the objectives of the study. If the precision of a measured property is to be estimated over a range of levels, then at least m = 3 materials covering the range should be included in the design. However, the number of materials in the design should be kept to a minimum since each lab will be committed to running a total of nm test results. An important consideration for each material is homogeneity and stability of the individual samples prepared from the bulk material.

What Are Other ILS Designs?

Many committees have their own standards but use the E691 design. Other committees have standards with more elaborate ILS designs, which may include other factors such as different analysts, days or instruments (Table 2). Many of these standards give procedures for the statistical analysis of data from such designs, which seek to estimate some form of intermediate precision in addition to repeatability and reproducibility. This additional objective is permitted in A21.2.4 of Form and Style, but it is usually better accomplished in a different type of study, which will be discussed in a future DataPoints article.

Table 2 - Cell detail for a more complex ILS design

For a good example of these more elaborate designs and their analysis, including additional statistical procedures, see the comprehensive practice of Committee D02 on Petroleum Products and Lubricants, D6300, Practice for Determination of Precision and Bias Data for Use in Test Methods for Petroleum Products and Lubricants.

What Are the E691 Design Benefits?

The primary benefit of the E691 design lies in its simplicity, both in the execution of the protocol and analysis of the data. According to Phillip Godorov, director of the ASTM ILS Program (phone: 610-832-9715; pgodorov@astm.org), a more simply designed study, such as suggested by E691, facilitates the recruitment and retention of laboratory participants, and it can help ensure better laboratory compliance with the ILS protocol. The statistical analysis of data from an E691 design is easy to conduct, since a spreadsheet template will usually suffice for developing the precision estimates and the data diagnostics, even with one or two missing values. As two of the early developers of ILS standards by Committee E11 on Quality and Statistics stated in a July 1988 SN article, "Simplest is best."1 The E691 design fills that requirement and is thus well suited for use by many ASTM committees in their interlaboratory studies.

Reference

1. Lashof, T.W., and Mandel, John, "Round Robins," ASTM Standardization News, Vol. 16, No. 7, July 1988, p. 42.

Thomas Murphy, T.D. Murphy Statistical Consulting LLC, is chairman of Subcommittee E11.20 on Test Method Evaluation and Quality Control, a part of ASTM Committee E11 on Quality and Statistics.

Dean V. Neubauer, Corning Inc., coordinates the DataPoints column; an ASTM International fellow, he is vice chairman of Committee E11 on Quality and Statistics, chairman of Subcommittee E11.30 on Statistical Quality Control and chairman of E11.90.03 on Publications.

Statistics play an important role in the ASTM International standards you write, and a panel of experts is ready to answer your questions about how to use statistical principles in ASTM standards. Please send your questions to SN Editor in Chief Maryann Gorman at mgorman@astm.org or ASTM International, 100 Barr Harbor Drive, P.O. Box C700, West Conshohocken, PA 19428-2959.