Using Proficiency Test Data to Set Uncertainty Budgets for Test
by Dean Flinchbaugh
Through ASTMs proficiency testing program, committee E01 has developed a model for using proficiency test data to set
uncertainty budgets for test methods. Dean Flinchbaugh explains
how this model will benefit standards developers, analytical laboratories,
and even manufacturers.
Committee E01 on Analytical Chemistry for Metals, Ores and Related
Materials is developing protocols for using proficiency test data
from its Plain Carbon and Low Alloy Steel and Stainless Steel Proficiency Test Programs to set data quality objectives and
uncertainty budgets for its new test methods. The development
of the model, supported by actual laboratory data, was described
in a technical paper1 presented at the E01 Workshop at The Pittsburgh
Conference, New Orleans, La., in March.
Since that workshop, the model was slightly extended and converted
into an ASTM Standard Practice for Establishing an Uncertainty
Budget for the Chemical Analysis of Metals, Ores, and Related
Materials, which is now undergoing balloting in E01.
How the Model Works
The model plots the log of the robust standard deviations from
the proficiency test database vs. the log of the concentration
of the analyte, independent of analyte or the test method. The
resulting power-fit straight line defines the data quality being
achieved by competent laboratories. Therefore, it predicts the
uncertainty that laboratories can tolerate in their measurements
in order to achieve acceptable z-scores in proficiency tests.
It also follows that new ASTM test methods must be capable of
producing data of equivalent quality if they are to be accepted
by those who provide and use test results.
Once the total allowable uncertainty is established, it is easy
to identify the major steps in the procedure that contribute to
the error and sub-divide or budget that error among those major
contributing steps. Now we have a typical uncertainty budget that
can be used to evaluate the ruggedness of sub-portions of a candidate
test method. The result of this evaluation, completed during interlaboratory
testing, is to be included in the standard test method. The uncertainty
budget, confirmed by interlaboratory testing, will become the
criteria of acceptance to be met by laboratories that wish to
use the new ASTM standard test method in their work. These changes,
once adopted, will enable E01 to write ISO 170252-compliant, performance-based
standard test methods. A practice describing how to write performance
based methods is under development.
Note that this model does not change the existing interlaboratory
test program now required by the Form and Style Manual for ASTM Standards, but it does add some validation steps. The significant change
is that it will now make use of the interlaboratory test to set
performance expectations of those who use ASTM test methods. This
concept of demonstrating performance rather than self-declaring
that a procedure was followed makes it easier to verify that a
given test was carried out in accordance with an ASTM standard
test method and makes the performance-based ASTM test method more
compliant and more easily auditable to ISO 17025. There are many
potential advantages to the analytical chemistry community, to
ASTM, proficiency test providers, and to accrediting bodies as
a whole if Committee E01 can implement this model successfully.
How the Model Helps
Analytical laboratories will benefit by having access to standard
test methods that can be adopted and validated by following simple,
prescriptive procedures and will be compliant with ISO 17025.
Laboratories will have the confidence that they are capable of
successfully participating in proficiency tests, will be able
to report state-of-the-art uncertainties to their clients, and
should pass methods-related accreditation audits without difficulty.
They will also have a very simple model for communicating their
uncertainties to clients and accreditation bodies.
Analytical chemistry standards-writing committees will be able
to write performance-based methods having quantitative acceptance
criteria rather than self-certification that a given procedure
was followed. Laboratories should be more willing to participate
in test method development work because they will be able to use
their interlaboratory test data to validate their use of the method.
Specification-writing committees will be able to write clearer,
more precise instructions for accepting or rejecting material
when compositional test results are near specification limits.
Knowing the uncertainty associated with a given measurement will
eliminate the need to apply product tolerance extensions to
specification limits to allow for assumed levels of measurement
error. Also, they will be able to develop uncertainties for sampling
practices, once the uncertainties of the test methods are known.
This will ultimately make it possible to assign uncertainties
to manufacturing processes, such as steelmaking, and then use
this information to research improvements in products and processes.
1 Dean A. Flinchbaugh, Larry F. Crawford, and David Bradley, The
Use of ASTM Proficiency Test Data to Set Measurement Quality Objectives
and to Establish Measurement Uncertainties in Analytical Chemistry
Laboratories, presented at Pittcon on March 4, 2001 and accepted
for publication in Accreditation and Quality Assurance.
2 ISO 17025, General Requirements for the Competence of Testing
and Calibration Laboratories