What Are Repeatability and Reproducibility?
Part 1: A D02 Viewpoint for Laboratories1
Q: Many ASTM standard test methods contain repeatability and reproducibility statements and values. What is the conceptual difference between repeatability and reproducibility, and how can this information be applied?
A. A standard test method prescribes a fixed set of equipment requirements, measurement methodologies and procedural protocols on how to produce a measurement for a specific property of interest. In a perfect world, if the same material is repeatedly tested using the same test method, whether by the same operator in the same laboratory using the same apparatus or by different laboratories and operators using different apparatus meeting the specified requirements, all measurements generated will be numerically identical.
However, we do not live in a perfect world. Repeated execution of the same test method on the same material, whether by the same operator in the same laboratory using the same apparatus or by different operators in different laboratories using apparatus of similar design, will not always yield numerically identical results. Minor differences will exist between test results due to variations in factors that are inherent in the test method.
For instance, different operators may prepare the test material in slightly different manners. Different apparatus, while designed to the same specification, may differ slightly in performance. The quality of reagents may differ slightly between laboratories. The variation in environmental conditions, for example, humidity, temperature, pressure, within or between laboratories, may affect instrumentation responses. Variation attributable to factors considered inherent in the test method is generally referred as common-cause variation and can be thought of conceptually as the inherent “imperfections” associated with the test method. In statistical nomenclature, this inherent “imperfection” is known as precision although some might prefer to think of it as imprecision.
Intuitively, the precision associated with multiple test results obtained by the same operator on the same test material using the same apparatus and test method within a very short time should be less than the variation associated with an equivalent number of test results, where each test result is produced by a different laboratory on the same test material using the same test method. This intuitive expectation comes from recognizing that, in the former scenario, the number of factors that can affect the precision of the test results is less than the latter. Therefore, precision must be associated with specific conditions.
Repeatability and reproducibility are standardized terms2 adopted by ASTM and other standardization organizations, and they are associated with the precision of measurements generated on the same material using the same test method under specific conditions. Table 1, Conditions for Precision, outlines three different sets of specific conditions associated with precision as defined in ASTM E177, Practice for Use of the Terms Precision and Bias in ASTM Test Methods, and E456, Terminology Relating to Quality and Statistics.
Precision associated with repeatability and reproducibility conditions is quantified by the repeatability and reproducibility limits published in the precision and bias section of an ASTM test method. A nonstatistical interpretation of these values is that these are the maximum difference between two results obtained under specified conditions that can be attributed to the test method precision. Hence, the published repeatability and reproducibility limits (r, R) can be used as decision limits to support or challenge the validity of the assumption that both test results have been produced on the same material in a correct manner under the associated specific conditions. It should be noted that r and R may not necessarily remain constant throughout the measurement range or material types included in the scope of the test method. Repeatability and reproducibility limits are derived statistically from specially designed studies. Various standard practices prescribing specific requirements associated with these studies have been developed and adopted by various industries to address their specific needs. ASTM standards C802, D2777, E180, E691, D6300, and ISO 4259 and ISO 57253 are examples of these standard practices. While the statistical approaches and philosophies may appear different to the practitioner, these practices are all based on similar statistical principles, and the interpretation and application of the derived end results are essentially the same.
The application of r and R can best be illustrated with the following two examples.
Example 1 — DIESEL FUEL AND D7039
Repeatability (r) = 0.23*()0.5 = 0.23*(15.5)0.5 = 0.91
Reproducibility (R) = 0.5*()0.5
where = average of two results (Note: Limits are dependent on the sulfur level).
Example 2 — GASOLINE AND D2700
The repeatability and reproducibility values (between 80.0 and 90.0) as published in ASTM D2700 are:
Next issue: Part 2 — The practical application of statistics standards for repeatability and reproducibility in interlaboratory studies.
Alex T. Lau, Engineering Services Canada, is chair of Subcommittee D02.94 on Quality Assurance and Statistics, part of ASTM Committee D02 on Petroleum Products and Lubricants, and a contributing member of Committee E11 on Quality and Statistics.
Dean Neubauer is the DataPoints column coordinator and E11.90.03 publications chair.