The Intermediate Precision of ASTM International Test Methods
How to Use, Measure and Control IP
Q: What is the intermediate precision of a test method? How is it measured and used?
A: Precision is the closeness of agreement between independent test results obtained under stipulated conditions. As implied by its name, intermediate precision is determined under conditions that are intermediate between repeatability and reproducibility conditions, which represent the two extreme conditions for determining test method precision. Repeatability conditions stipulate that the test results are obtained by the same method on identical test items in the same laboratory by the same operator using the same apparatus within short periods of time. Reproducibility conditions stipulate that the same method be conducted on identical test items in different laboratories, which necessarily involves different operators and apparatus, as well as differences in other factors such as laboratory environment, management and quality control policies, and even differing interpretations of the test method procedure itself.
Intermediate precision conditions allow for the varying of factors such as operators and apparatus over longer periods of time within a single laboratory, whereas repeatability conditions attempt to hold these factors constant. Historically, specific intermediate conditions have been defined by varying some, but not all, of the potential factors causing variability (see Table 1). E177, Practice for Use of the Terms Precision and Bias in ASTM Test Methods, defines two such conditions as single operator-apparatus, multi-day precision, and multi-operator, single day-apparatus precision. ISO 5725-3, Accuracy (Trueness and Precision) of Measurement Methods and Results, Part 3: Intermediate Measures of the Precision of a Standard Measurement Method, lists IP conditions based on various combinations of time, calibration, operator and equipment. However, the current trend leans toward a single definition of an intermediate condition that reflects changing levels of all known factors that could influence test method variation within a laboratory, including longer time periods of weeks or months instead of days. In this latter definition, IP is simply the long-term within-laboratory precision of a test method.
Measuring and Controlling Intermediate Precision
E177 and ISO 5725 both state that IP is likely to be more characteristic of a laboratory than of a test method because IP is likely to vary more between laboratories than repeatability. In particular, ISO 5725 recommends against the estimation of IP from an interlaboratory study for this reason. Another reason is that an ILS is conducted over a relatively short time period (2-3 days) within each laboratory and thus tends to underestimate IP in all of the laboratories participating in the study.
A recommended approach for estimating IP is to use a control sample program, in which samples of a homogenous and stable material are repeatedly introduced into the laboratory as routine samples for analysis and are assigned to the usual mix of operators and apparatus within the laboratory on a regular schedule over a longer time period. The resulting data will give a realistic estimate of the test method IP standard deviation for that laboratory.
Data from a control sample program can also be put to use to improve the test method performance within the laboratory, in other words, to reduce the IP standard deviation. If test results are well documented as to the operator, apparatus and other factors, the data can be stratified by operator and apparatus unit to detect possible biases due to these and other factors. Operator contribution to variation can then be reduced by corrective training if there is an indication of large biases among operators. Apparatus variation can be reduced by improved calibration and maintenance programs. Batches of materials and reagents used in the test method can be monitored for consistency, and their quality can be improved through incoming inspection and working with the suppliers.
Two ASTM International standards that describe control sample programs and their data analysis are D6299, Practice for Applying Statistical Quality Assurance and Control Charting Techniques to Evaluate Analytical Measurement System Performance, and E2554, Practice for Estimating and Monitoring the Uncertainty of Test Results of a Test Method in a Single Laboratory Using a Control Sample Program. For ongoing control, these standards recommend that the control sample data should be monitored by a control chart to detect any special cause variation entering the system.
Use of Precision Estimates
Examples of the use of repeatability limits and reproducibility limits to check agreement of test results on the same material have been given before in DataPoints (see the March/April 2009 SN column, “What Are Repeatability and Reproducibility? Part 1: A D02 Viewpoint for Laboratories”). An example of the use of an IP limit appears as a Bonus Q&A following the May/June 2009 SN DataPoints column.
The IP standard deviation is the proper measurement variability estimate to use when designing studies involving data collected over time, such as bench/pilot scale development projects involving a series of experiments, running assessment of product stability under storage for many months or clinical data in long-term treatment of disease. (The repeatability standard deviation only applies when there is a designed experiment where all submitted samples are analyzed together.)
Lastly, the IP standard deviation may be used by the laboratory as its estimate of measurement uncertainty (see Section A22 of Form and Style for ASTM Standards) and will serve to satisfy the uncertainty requirement in ISO/IEC 17025, General Requirements for the Competence of Testing and Calibration Laboratories.
Thomas Murphy, T.D. Murphy Statistical Consulting LLC, Morristown, N.J., is chair of Subcommittee E11.20 on Test Method Evaluation and Quality Control, part of ASTM Committee E11 on Quality and Statistics, an ASTM fellow and a past E11 chairman.
Dean V. Neubauer, Corning Inc., Corning, N.Y., coordinates the DataPoints column; an ASTM International fellow, he is vice chairman of Committee E11 on Quality and Statistics, chairman of Subcommittee E11.30 on Statistical Quality Control and chairman of E11.90.03 on Publications.