|Phillip Godorov assumed his position as ASTM International’s Interlaboratory Study Program manager in March 2005. With his undergraduate degree from the Johns Hopkins University, he is currently pursuing his M.B.A. at Rutgers University. He has more than 15 years of scientific technical and laboratory experience, including serving as the director of quality assurance for the Philadelphia Water Department, Bureau of Laboratory Services.
For more information on the ILS program, or how to register a study, please contact Phillip Godorov (phone: 610/832-9715).
ASTM International’s Interlaboratory Study Program
Standards without documented precision run the risk of appearing arbitrary. A validated precision statement ensures a standard’s technical reliability and enhances its scientific integrity. The most effective way to develop precision statements is through the creation and execution of interlaboratory studies. It is not unusual for hundreds of hours of work to go into the planning, execution, and administration of one large study. Studies are costly and, as a result, have kept some ASTM technical committees from validating the precision of their standards for years.
ASTM International has ventured into new territory. Embracing its penchant for offering a high level of service in their committed pursuit of excellence in standards development, ASTM’s board of directors adopted plans in the fall of 2004 to launch an Interlaboratory Study Program. Responding to the need for standards in the marketplace to be of known and documented quality, a $4,000,000 commitment was made for the next five years to support the effort. This commitment means ASTM will be able to assist those technical committees for which the prospect of implementing an interlaboratory study was daunting or impossible. The goal is to enhance the quality of ASTM test methods by providing administrative assistance to the technical committees as they develop precision statements supported by high-quality laboratory data.
The program began in April with the limited involvement of a few committees working on one or two standards at a time. In time, virtually all committees, and the test methods that they determine to be of the highest priority, will be viewed as potential program participants. Some of the work will even be applicable to more than one standard.
Deliberate to the design of the ILS Pilot Program was the inclusion of a diversity of committee backgrounds, and it is expected that within a year a majority of committees will benefit from some combination of the various supports offered.
Why Interlaboratory Studies?
The ILS Program will help ASTM to ensure that every one of its test methods is validated by a precision and bias statement that quantifies exactly what a user of the standard can expect to achieve in terms of repeatability (what one laboratory analyzing the same sample multiple times expects to see as its range of acceptable answers) and reproducibility (the difference that can be expected if the same sample were analyzed using the same standard in multiple laboratories). The end result is that users, producers, and consumers will all have greater confidence in the standards that they reference.
Unless the repeatability and reproducibility of a standard are known, a user of that standard would have no idea how to interpret variability in the data. It may be attributable to the normal deviations that occur from lab to lab, or it may be the sort of variability that reasonably can be expected from the technologies in the method itself. Accurate precision statements, derived from data produced by laboratories using up-to-date technologies, clearly let the user know how much variability is “normal” within a lab, and when comparing results from multiple labs.
Test methods that demonstrate excessively high levels of variability within a single laboratory (as determined by each committee) may indicate the need for technical revision to improve or update the technologies referenced in the standard. As technologies change, precision, accuracy, and method sensitivity are often improved. Keeping standards relevant in today’s marketplace is one of ASTM International’s many strengths. If the majority of the variability is seen from lab to lab, a committee may wish to enhance the level of detail contained in the procedural sections of the standard in order to better ensure uniformity in the application of the test method.
Designing the Program
As ASTM set out on the course of formalizing its assistance to members contemplating an ILS, they realized studies of this magnitude might not always be easy. Committed to giving their consensus standards community their full support, ASTM moved ahead full force. ASTM has committed funding for an initial five-year program of interlaboratory studies and standards verification. This past spring, all ASTM committees were notified of the board of directors’ commitment and plans to launch the ASTM ILS Program. Dedicated staff has been hired to oversee the project. Databases have been created to assist in processing and maintaining the interlaboratory studies facilitated by ASTM International, and Web development has been initiated to provide an online interface for those ASTM International members wishing to register their interlaboratory studies for technical, administrative, or financial assistance. These events put ASTM right on track with the planned timeline for launching the program.
A pilot program was initiated to enable ASTM staff to explore, through direct committee interaction, more about the needs of the members and technical committees and how best to build and develop the program. This was essential given the diverse community of committees developing standards under the ASTM umbrella. This project is still under way, and has yielded much valuable information, changing ASTM’s thinking about how to be of most service to the membership. ASTM has relied heavily on the committees’ collective knowledge and patience to help guide and shape the development of the forms, procedures, and methodologies that will be used in later studies.
As the program moves forward, all committees will be invited to submit projects to be considered for inclusion in the ILS Program. ASTM is optimistic that participation will be widespread. Greater flexibility will be engineered into the program than was originally envisioned; committees will be given more options as to exactly how they wish to be supported. The various forms of assistance that will be made available to committees include ILS design, planning, material procurement, sample preparation and distribution, data collection and data processing.
As more studies are launched, a searchable database will be established that will ease the process of identifying potential laboratories, suppliers and distributors. By carefully “tagging” every ILS participant in the database, it would be possible to go back into previous studies and compile reports listing, for example, “all of the distributors that have worked on interlaboratory studies for Committee D28 on Activated Carbon,” or “all of the suppliers of material for studies initiated by Committee C14 on Glass and Glass Products.” With this ability, the management of new ILS Programs, especially those registered by less experienced members, would be greatly facilitated.
The development of an integrated registration process was also broken down into a series of phases. To get the program started, a completely manual registration process was adopted for the initial phase. All information was collected on hardcopy forms that were assembled with the input of several ASTM International staff managers who were familiar with the round-robin studies performed by their committees in the past. The first enhancement to that system was basically an electronic version of the hard-copy forms, still fairly inflexible, but an improvement in that the forms and responses could be easily e-mailed to the technical contacts. The scope and quantity of information collected was still limited by the design of the template. The third design phase of the registration segment encompasses the interface that one will see when going online to the ASTM International Web site, logging in to MyASTM, and proceeding to register an interlaboratory study. The site has been designed to be user-friendly, an intuitive interface to help registrants convey not only the scope, but also some of the details of their proposed study.
Some committees require more extensive assistance in designing an experiment that will provide useful data for the determination of a precision statement, while remaining very aware of the laboratory resources that are being volunteered in support of their efforts. Other committees have several active members with significant experience in statistics and design of experiments, however, they may require assistance in identifying participants that are both willing and able to join a planned ILS during the proposed timeframe.
As the ILS Program is currently designed, once a test method is identified by a subcommittee as lacking a complete or up-to-date precision statement, and it has been given a high level of priority by the committee’s executive subcommittee or its designee, a technical contact would be assigned. The technical contact is the committee representative that will be most heavily involved in the design and implementation of the ILS and works with ASTM staff on everything from initial registration, formalization of instructions, and creation of data sheets, to answering questions on methodology from the labs and reviewing the final data.
Together with the ASTM ILS staff, the technical contact would determine the need to identify additional resources for the study, and review each potential material and participant for overall appropriateness. Should there be any question as to the usability of a particular sample type or participant, the decision to include or exclude would often be left up to the judgment of the technical contact. Once the design is set, the ILS staff can perform many of the time-consuming administrative functions that would ordinarily fall on the task group charged with the study, such as confirming participation of the laboratories, suppliers, distributor, and statistician, or generating electronic packets consisting of the test method to be utilized by the laboratories, the data reporting form, special shipping and handling requests, and the cover letter explaining the purpose and anticipated timeline for the study.
ASTM Committee F23 on Protective Clothing volunteered to be the first participant in the pilot program, with an interlaboratory study conducted to develop a precision and bias statement for a proposed revision to F 739, Test Method for Resistance of Protective Clothing Materials to Permeation by Liquids or Gases Under Conditions of Continuous Contact. Phillip Godorov worked closely with technical contacts Arthur Schwope and Jason Khayat to implement the study while ensuring continuous feedback with the participating laboratories. Usable data were obtained from eight of the 14 originally proposed labs, and the results were presented to Subcommittee F23.30 on Chemicals for discussion during its June meeting in Reno, Nev.
Since then, nine more studies have been registered in the pilot program. These studies represent a variety of committees, interests, and technologies, including studies conducted by Committee E28 on Mechanical Testing of Rockwell and Brinell hardness, the shear and tensile strength testing of construction adhesives conducted by Committee D14 on Adhesives, and the study of adhesion-in-peel of elastomeric joint sealants done by Committee C24 on Building Seals and Sealants. From each new study more is learned about how best to satisfy the needs of the various committees. Other committees proposed for pilot study involvement include C16 on Thermal Insulation, D05 on Coal and Coke, D16 on Aromatic Hydrocarbons and Related Chemicals, E08 on Fatigue and Fracture, and E33 on Environmental Acoustics.
Essential to the successful implementation of an interlaboratory study will be finding an appropriate mix of materials and suppliers to adequately represent the different test levels within the method’s applicable range. When choosing test materials, it will be vital to consider the level of difficulty normally associated with procurement as well as the relative expense of one sample material versus a comparable alternative, not to mention the distribution issues associated with many different types of samples. Sensitivity to heat, cold, light, air, moisture or jarring (among other things) could mean that specialty packaging and distribution are necessary. The time-consuming task of locating an available distributor with the ability to handle the preparation and distribution of ILS samples would be yet another reason to call upon ASTM International’s ILS staff for assistance. ASTM’s background puts us in the ideal position to facilitate an arrangement like the ILS Program.
The initiation of the Interlaboratory Study Program is an example of ASTM International’s commitment to providing superior service to its membership and the community at large through tireless efforts to provide a supportive and professional atmosphere within which the goal of developing high-quality standards can best be accomplished. //