Using Proficiency Test Data to Set Uncertainty Budgets for Test Methods

by Dean Flinchbaugh

Through ASTM’s proficiency testing program, committee E01 has developed a model for using proficiency test data to set uncertainty budgets for test methods. Dean Flinchbaugh explains how this model will benefit standards developers, analytical laboratories, and even manufacturers.

Committee E01 on Analytical Chemistry for Metals, Ores and Related Materials is developing protocols for using proficiency test data from its Plain Carbon and Low Alloy Steel and Stainless Steel Proficiency Test Programs to set data quality objectives and uncertainty budgets for its new test methods. The development of the model, supported by actual laboratory data, was described in a technical paper1 presented at the E01 Workshop at The Pittsburgh Conference, New Orleans, La., in March.

Since that workshop, the model was slightly extended and converted into an ASTM Standard Practice for Establishing an Uncertainty Budget for the Chemical Analysis of Metals, Ores, and Related Materials, which is now undergoing balloting in E01.

How the Model Works

The model plots the log of the robust standard deviations from the proficiency test database vs. the log of the concentration of the analyte, independent of analyte or the test method. The resulting power-fit straight line defines the data quality being achieved by competent laboratories. Therefore, it predicts the uncertainty that laboratories can tolerate in their measurements in order to achieve acceptable z-scores in proficiency tests. It also follows that new ASTM test methods must be capable of producing data of equivalent quality if they are to be accepted by those who provide and use test results.

Once the total allowable uncertainty is established, it is easy to identify the major steps in the procedure that contribute to the error and sub-divide or “budget” that error among those major contributing steps. Now we have a typical uncertainty budget that can be used to evaluate the ruggedness of sub-portions of a candidate test method. The result of this evaluation, completed during interlaboratory testing, is to be included in the standard test method. The uncertainty budget, confirmed by interlaboratory testing, will become the criteria of acceptance to be met by laboratories that wish to use the new ASTM standard test method in their work. These changes, once adopted, will enable E01 to write ISO 170252-compliant, performance-based standard test methods. A practice describing how to write performance based methods is under development.

Note that this model does not change the existing interlaboratory test program now required by the Form and Style Manual for ASTM Standards, but it does add some validation steps. The significant change is that it will now make use of the interlaboratory test to set performance expectations of those who use ASTM test methods. This concept of demonstrating performance rather than self-declaring that a procedure was followed makes it easier to verify that a given test was carried out in accordance with an ASTM standard test method and makes the performance-based ASTM test method more compliant and more easily auditable to ISO 17025. There are many potential advantages to the analytical chemistry community, to ASTM, proficiency test providers, and to accrediting bodies as a whole if Committee E01 can implement this model successfully.

How the Model Helps

Analytical laboratories will benefit by having access to standard test methods that can be adopted and validated by following simple, prescriptive procedures and will be compliant with ISO 17025. Laboratories will have the confidence that they are capable of successfully participating in proficiency tests, will be able to report state-of-the-art uncertainties to their clients, and should pass methods-related accreditation audits without difficulty. They will also have a very simple model for communicating their uncertainties to clients and accreditation bodies.

Analytical chemistry standards-writing committees will be able to write performance-based methods having quantitative acceptance criteria rather than self-certification that a given procedure was followed. Laboratories should be more willing to participate in test method development work because they will be able to use their interlaboratory test data to validate their use of the method.

Specification-writing committees will be able to write clearer, more precise instructions for accepting or rejecting material when compositional test results are near specification limits. Knowing the uncertainty associated with a given measurement will eliminate the need to apply “product tolerance” extensions to specification limits to allow for assumed levels of measurement error. Also, they will be able to develop uncertainties for sampling practices, once the uncertainties of the test methods are known. This will ultimately make it possible to assign uncertainties to manufacturing processes, such as steelmaking, and then use this information to research improvements in products and processes. //

1 Dean A. Flinchbaugh, Larry F. Crawford, and David Bradley, “The Use of ASTM Proficiency Test Data to Set Measurement Quality Objectives and to Establish Measurement Uncertainties in Analytical Chemistry Laboratories”, presented at Pittcon on March 4, 2001 and accepted for publication in Accreditation and Quality Assurance.

2 ISO 17025, General Requirements for the Competence of Testing and Calibration Laboratories