|
Guidance on Precision and Bias / Overview ASTM test methods are frequently used in determining performance properties of materials cited in specifications. A statement addressing precision and bias is required in an ASTM test method so that when a component operator properly performs the test, the results will be found satisfactory for juding the compliance of a material with the specification. A precision statement expresses the variability that resulted when the test method was used in an interlaboratory study (ILS) involving six or more laboratories. A statement on precision allows potential users of the test method to assess in general terms its usefulness in proposed applications. Repeatability and Reproducibility are the two measurements used to express the precision of a test method. Repeatability addresses variability between independent test results gathered from within a single laboratory (intralaboratory testing). Reproducibility addresses variability among single test results gathered from different laboratories (interlaboratory testing). Guidance for developing the precision statement can be found in Section A21 of the Form & Style Manual. Bias is a systematic error that contributes to the difference between the mean of a large number of test results and an accepted reference value. In other words, the average of all test results obtained from the ILS for a specific material is compared with the accepted reference value for that material. If no accepted reference value is available, bias cannot be calculated. In cases where the direction or bounds on the bias, or both, can be estimated, the information should be included in the bias statement. For additional information on developing a bias statement when an accepted reference value is not available, refer to Section A21 of the Form & Style Manual. Prior to running a full interlaboratory study (ILS) that may be costly and time consuming, ruggedness testing is often used to pinpoint the variables that are associated with the performance of a test method and that impact the variability of the precision. If a ruggedness test is not performed prior to the actual interlaboratory study, the participants in the study may not be aware of the variables that directly impact the results of performing the method. The ruggedness test should be confined to a single laboratory in order to maximize the visibility of any variables. Examples of variables to consider include: sampling, conditioning, pressure, temperature, and relative humidity. For additional guidance, please refer to ASTM Standard E1169 Standard Guide for Conducting Ruggedness Tests. The ILS or .round robin. is performed to generate data that is used to develop the precision statement included in a test method. Typically, testing protocols and data report forms are provided to each laboratory along with appropriate test sample(s). All labs conduct the test and return their data to a source that will statistically analyze the data and develop a research report addressing repeatability and reproducibility. It is encouraged that the ILS be conducted while the new standard is being developed so that the precision statement can be included prior to approval. For additional information on conducting an ILS, refer to E691 Standard Practice for Conducting an Interlaboratory Study to Develop the Precision of a Test Method and the E691 software. A research report includes the key information from the ILS including the raw data. Upon approval of the ballot item, ASTM will assign a research report number and include the number in a note in the standard so that interested users can obtain a copy from ASTM International Headquarters. A research report should be organized according to The Guide for the Format of a Research Report For additional information, please visit the Committee E11 Quality and Statistics Homepage. |
||