Bookmark and Share

Standardization News Search
Feature

ASTM Proficiency Testing Programs

Statistical Quality Assurance Tools for Laboratories

by David Bradley

ASTM International is a developer and publisher of standards that are referenced and used worldwide. Of primary interest to ASTM Proficiency Testing Programs are test methods, as these are the documents used by laboratories to obtain results that impact laboratory performance. This article provides a general overview of ASTM PTPs and the importance of these programs to participating laboratories and the sponsoring ASTM technical committees.

ASTM PTPs have grown significantly over the past years with units of laboratory participation exceeding 2,000 labs, including over 40 percent non-continental U.S. laboratory participation, reflecting the international use and acceptance of ASTM PTPs. The success of the program is a direct reflection of the support and technical guidance provided by dedicated ASTM technical committee members.

ASTM Proficiency Testing Programs are designed to provide participating laboratories with a statistical quality assurance tool that enables them to assess their performance compared to other labs participating in the same program. ASTM coordinates the preparation and distribution of test samples (through the utilization of outside contractors) and provides each laboratory with a detailed set of test instructions, listing of methods to be used, and a data report form for recording results and other requested information. Upon completion of testing, each laboratory returns their data to ASTM. ASTM compiles the data and prepares a statistical summary report that includes all lab data (each lab is identified with a randomized lab code to maintain confidentiality), charts plotting test results, and other relevant information. Final reports are provided to participating labs within approximately one month of completing the data collection.

The Early Years

In 1991, the ASTM board of directors approved a business plan for the management of PTP by ASTM staff. The concept was not new to ASTM at that time, as there were ongoing programs being conducted through the support and guidance of ASTM technical committees. These included D02’s National Exchange Group, which monitors laboratory performance in testing engine knock, its Test Monitoring Center, and other regional testing programs in the petroleum testing community; and the Cement and Concrete Reference Laboratory of Committees C01 on Cement and C09 on Concrete and Concrete Aggregates.

In 1993, ASTM launched five new programs, four of which were from Committee D02 on Petroleum Products and Lubricants and one from Committee E01 on Analytical Chemistry for Metals, Ores, and Related Materials. Over the years PTP has expanded to its present offering of 30 programs including those in the areas of petroleum products and lubricant testing (sponsored by Committee D02), metals testing (chemical analysis (E01) and mechanical properties (E28 on Mechanical Testing)), gold in bullion by cupellation (E01), textiles (D13 on Textiles), plastics (D20 on Plastics), and engine coolant testing (D15 on Engine Coolants). The frequency of testing in these programs varies depending upon relevant industry needs. Programs are generally offered twice a year, three times a year, or quarterly; two programs are offered monthly.

Program Development

When a need is expressed for a new program, the concept is reviewed at ASTM and then presented to the appropriate executive subcommittee of the relevant ASTM technical committee for review. The executive subcommittee reviews the proposal to determine the value of the program to industry and the committee’s ability to provide technical support. If the results of this review are positive, a small working group is created to develop a program scope, identify proposed methods to be included, type of samples to be tested, and a brief overview description. ASTM staff will utilize this information and conduct a limited market study to determine the level of acceptance and need for the program. When the market survey indicates a need for an ASTM administered program, then ASTM staff and the technical committee will work together to finalize the program structure. A minimum of 30 labs is needed to start a program in order to provide a minimum basis for administering programs and providing a significant amount of data.

ASTM Proficiency Testing Programs are designed to operate according to ASTM Standard E 1301, Guide for Proficiency Testing by Interlaboratory Comparisons, and to specific program needs approved through the sponsoring technical committees.

ASTM Headquarters provides the administrative and management support to facilitate program operations. This includes direct interface with technical committees, conducting marketing surveys on program needs and designs, managing contracts, accounting and legal review, obtaining test materials, program marketing, database programming, preparing and submitting statistical summary reports to participants and technical committees, and providing management oversight for the programs.

ASTM does not provide certification of laboratory participation or rankings of laboratories based on performance. The data, as displayed in the final reports, enables a lab to evaluate its own performance against the other labs. Draft final reports are also circulated to technical contacts from the committee for review prior to distribution to participants. This additional review adds technical integrity to the program and to published reports since those with technical knowledge of the methods review the data.

Benefits to Labs and Technical Committees

PTPs provide many services to laboratories and to the technical committees responsible for the related standards used in the program. Laboratories can periodically assess and benchmark their capabilities in conducting a test when their data are compared to the consensus average generated from all data submitted by the participating labs. The programs are designed to allow laboratories to satisfy the requirement for proficiency testing program participation as part of their accreditation (for the methods included in the programs that fall under the scope of their accreditation). Laboratories can also use the programs as a training tool to monitor the performance of laboratory technicians. Some laboratories may use their results to demonstrate their testing capabilities to their clients and customers. For organizations that have multiple labs in numerous locations, the program provides a collective comparison of their organizational data against outside laboratories.

A value-added benefit provided by most programs is the feedback the technical subcommittees provide to the participating laboratories on testing issues and techniques, and concern over data identified as outliers. The intent is to help laboratories maintain or improve their understanding of and performance in conducting ASTM test methods. This dialogue and exchange of information is funneled through ASTM Headquarters so as not to release lab code identifications to technical subcommittees. As part of this review process, the technical subcommittees have also asked ASTM to collect additional information from participating labs, which may pertain to the methods themselves, the handling of samples, units of measure, or other information.

There are discussions on the use and acceptance of PTP data as part of an ASTM standard’s precision statement within the sponsoring technical subcommittees. Some people feel that using this information is appropriate for precision studies while others view the PTPs’ assessment of the laboratory’s ability to generate a test result vs. the technical subcommittee’s intent to validate a method as two separate entities and purposes. Regardless, the significance of the subcommittee in reviewing these data and any questions raised on the methods are important to the integrity of the methods. Modifications to existing standards have already occurred as a result of questions raised and data submitted by participating laboratories.

The ASTM technical committees sponsoring programs receive the benefit of obtaining real-world data for their standards. ASTM provides copies of final reports and other program information to the technical committees. The data remains confidential and is provided as a review of the data against expected results and of test method accuracy.

Electronic Enhancement

ASTM is now improving program operations to electronically enhance data entry and summary report distributions. By mid-year 2004, all PTP data report forms will begin appearing on the ASTM Web site, allowing labs to enter their own data directly into the database. Fields on the data forms will be validated where appropriate to provide uniformity in the data reporting. Participants will receive an e-mail notification announcing that the form is electronically posted and providing directions on how to access it and submit data. This process will greatly enhance data entry and position ASTM to shorten the time in producing and publishing final reports.

A second phase to this project, scheduled for late 2004 and into 2005, is the electronic distribution and availability of reports to program participants and technical committees.

The intent of this phase will be to provide access to data summary tables within days after the deadline date of submitting data. The proposed content of these reports will be reviewed with relevant committees and program participants.

Participation in the program is open to any lab that can conduct the relevant testing within their laboratory. For more specific information on current programs and participation fees, visit our Web page at www.astm.org/STATQA. For more information on establishing new programs please contact David Bradley, ASTM (phone: 610/832-9681) or Anne McKlindon, ASTM (phone: 610/832-9688). //

Copyright 2004, ASTM International

David Bradley is director of the Proficiency Testing Program at ASTM.

Sidebar
Committee D02 on Petroleum Products and Lubricants has established Aids for the Analyst for several of its test methods. These one- to two-page documents are prepared by people expert in the analytical techniques and provide an overview of the key attributes in the method, of which labs must be aware when conducting the test. Each laboratory in the program has access to this information. In a few cases, the Aids for the Analyst have been inserted in the D02 standards themselves. D02 distributes a newsletter twice each year that includes program status updates and any newly developed Aids for the Analyst. D02 developed this checklist to assist laboratories in identifying why statistical outliers may have occurred and/or how to improve precision. This checklist is published in each D02 summary report and contains a note that there may be additional ways to improve performance. Variations of this listing are either already included or are being reviewed for inclusion in other committee-sponsored ASTM PTP summary reports.

A Checklist for Investigating the Root Cause of Unsatisfactory Analytical Performance (Developed by D02.92 at their June 1998 meeting)

1. Check the results for typos, calculation errors, and transcription errors.
2. Reanalyze the sample; check for repeatability.
3. Check the sample for homogeneity or contamination, and that a representative specimen has been analyzed.
4. Review the test method and ensure that the latest version of the ASTM test method is being used. Check the procedure step-by-step with the analyst.
5. Check the instrument calibration.
6. Check the statistical quality control chart to see if the problem has been developing earlier.
7. Check the quality of the reagents and standards used, and whether they are expired or contaminated.
8. Check the equipment for proper operation against vendor’s operating manual.
9. Call the vendor for maintenance and/or repairs.
10. After the problem has been resolved, analyze a certified reference material if one is available, or the laboratory quality control sample to ascertain that the analytical operation is under control.
11. Provide training to new analysts and, if necessary, refresher training to experienced analysts.
12. Document the incident and the learnings for use in future similar problems.