Requirements for ILS
Types of Support from ASTM
Precision and Bias Statements
How do I start an ILS program with ASTM?
To start an ILS program with ASTM, the technical contact for the study must first register a program. To register a program, select the following link: http://myastm.astm.org/CUSTOMERS/ils.cgi. After selecting "Register a new Interlaboratory Study", you will be prompted to enter the appropriate information for your ILS. Shortly after the program is registered, the ASTM ILS staff will contact you to begin providing support. Registration of an ILS program ensures full administrative support from the ILS team.
What information does one need to register an ILS?
When registering an ILS, it would be helpful to know which test method(s) will be included in the study, the materials or samples to be tested, the names of the supplier(s), the distributor, the number of replicates to be reported, and the name of your committee's statistical support person (if available). You will also need a copy of the standard test method or draft procedure. (ASTM will supply a copy of the standard test method to you if needed.) Please fill out the registration with as much information as you have and then submit to ASTM. If you are not sure of all the details for your study, ASTM can assist you in designing your study.
Do I need to register a work item before I have registered an ILS?
Yes, before you register an ILS program you will need to register a work item. (You will be asked for this information on the first page of the ILS registration.) A Work Item (WK) may be a new standard or a revision to an existing standard that is under development by a committee. The Work Item notifies the public that work is being done to revise a standard or to write a new one. ASTM publishes the Work Item descriptions in the interest of openness, as well as to solicit input from interested stakeholders who may not be members of the committee. To learn how to register a work item select the following link http://www.astm.org/ISP/PDFS/WorkItemReg.pdf.
What is the difference between pilot testing and ruggedness testing?
Often used during the standard test method development, a ruggedness test is a screening procedure for investigating the effects of variations in environmental or other conditions in order to determine how control of such test conditions should be specified in the written description of the method. For example, the temperature of the laboratory or of a heating device used in the test may have an effect that cannot be ignored in some cases but may be much less important in others. In a ruggedness test, deliberate variations in temperature would be introduced to establish the allowable limits on control of temperature. A ruggedness test is done before a pilot test to determine the variables that will need to be considered in the study.
Once all of the controllable variables in your test method have been considered and addressed in the standard, it is strongly recommended that a pilot test be conducted. Before investing considerable laboratory time and expense in the full scale ILS, it is usually wise to conduct a pilot run in at least two laboratories, with one or two material(s) for several reasons:
1. to ensure that the study directions are clearly understood
2. to disclose any weaknesses in the protocol or the test method
3. to serve as a familiarization procedure for those without sufficient experience with the method
4. to provide the committee an opportunity to refine the method prior to conducting the full ILS
How many laboratories, samples and replicates does a committee need to run an ILS?
For guidance on the number of laboratories refer to E691 Under no circumstances should the final statement of precision of a test method be based on acceptable test results for each material from fewer than 6 laboratories. This would require that the ILS begin with 8 or more laboratories in order to allow for attrition.
For guidance on the number of samples refer to E691 An ILS of a test method should include at least three materials representing different test levels, and for development of broadly applicable precision statements, six or more materials should be included in the study.
For guidance on the number of replicates refer to E691 The number may be as small as two when there is little danger that a test unit will be lost or questionable test results obtained, or as many as ten when test results are apt to vary considerably.
Do the ILS participants need to be ASTM members to participate in an ILS?
No, participants of the ILS do not need to be members of ASTM. Participants however should be familiar with the testing that is being completed.
Can a single company with laboratories at multiple locations consider each lab as a separate testing site?
Yes, as long as each data set is generated by a laboratory with its own set of environmental conditions, calibrations, and equipment conditions and is then compared to other laboratories with their own set of variables, the separate testing site requirement is met. To promote a normal distribution of data, it is recommended that no more than half of the participating labs come from any one organization.
If a committee wants to run an ILS and there are less than six labs in the industry that perform the test, should an ILS still be done?
Yes, an ILS should still be completed even if the recommended labs are not available. It is possible to generate a precision statement with less that six participating laboratories. The bottom line is that some precision information is preferable to no precision information.
If a committee is unable to find labs to complete the ILS, can ASTM assist?
Yes, ILS can assist in finding labs. First a lab solicitation will be sent to the committee looking for volunteers and then if more labs are still needed, ILS will reach out to the industry. Participants do not have to be ASTM members to take part in the study.
Can ASTM collect my laboratory data and compute the statistics?
Once a program is registered with ILS, we prefer to collect the data from the labs. We have an online data report form that is sent out via an email to the lab participants. In the email, a hyperlink is provided which leads to a private webpage for that lab. After entering their data into the form the lab would select the submit button and the data will be sent directly to ASTM. Once all of the labs have submitted their data to ASTM, ILS is then able to run the data through ASTM's ILS Statistical Software. When we are ready to generate the research report, the data will be pulled directly from the software.
What do I do with data from an old ILS? Can ASTM help?
ILS is happy to assist with any data from studies previously completed. To get the process started, please register a program by selecting the following link: http://myastm.astm.org/CUSTOMERS/ils.cgi. After selecting "Register a new Interlaboratory Study", you will be prompted to enter the appropriate information for your ILS. Shortly after the program is registered, the ASTM ILS staff will contact you to begin providing support. Once the program is registered, please send the raw data to ILS@astm.org referencing the standard designation and ILS number (given to you once the ILS program is registered and submitted).
What should I do with round robin (interlaboratory study) data? Does ASTM keep it on file?
Once a round robin has been completed and the information has been balloted into the test method, the data shall be submitted to ASTM to be kept on file as a research report. A number will be assigned to the research report and a footnote will appear in the precision and bias section of the test method referencing the specific research report.
What parts of the ILS can ASTM provide financial assistance with?
ASTM has funds available to assist with purchasing samples, sample preparation and distribution. After a proposal of costs is presented, ASTM will work with the committee to determine how much assistance can be provided. ASTM gratefully acknowledges the analyzing laboratories and the technical contacts for volunteering their time to the ILS efforts.
What support can the ILS Program provide to the committees?
• Design an interlaboratory study
• Identify potential samples
• Solicit volunteer laboratories
• Find available suppliers
• Contract with distributors
• Reviewing laboratory instructions
• Reimburse distribution expenses
• Collect and analyze data
• Produce a draft precision statement
• Compile information for the research report
• Give recognition to participating laboratories
When a committee is getting ready to ballot a new test method, do they need to have a precision and bias statement?
Yes, when balloting a new test method, repeatability is required in the precision and bias section. Per the ASTM Form and Style Manual Section A21.2.2 Every test method shall contain a statement (1) regarding the precision of test results obtained in the same laboratory under specifically defined conditions of within-laboratory variability (repeatability conditions). To produce repeatability conditions, one lab should test a range of samples numerous times. The recommended amount of replicates is as many as 10-20.
Is a precision and bias required in a standard test method?
Per the ASTM Form and Style Manual Section A21 , precision and bias is required in a test method.
A21. Precision and Bias (Mandatory)
A21.2 Statement of Precision (Mandatory):
A21.2.1 Precision is the closeness of agreement between test results obtained under prescribed conditions. A statement on precision allows potential users of the test method to assess in general terms its usefulness in proposed applications. A statement on precision is not intended to contain values that can be duplicated in every user's laboratory. Instead the statement provides guidelines as to the kind of variability that can be expected between test results when the test method is used in one or more reasonably competent laboratories.
A21.2.2 Every test method shall contain a statement (1) regarding the precision of test results obtained in the same laboratory under specifically defined conditions of within-laboratory variability (repeatability conditions), and (2) regarding the precision of test results obtained in different laboratories (reproducibility conditions). Use a statement such as the following:
Precision — The repeatability standard deviation has been determined to be (insert the test values and corresponding repeatability values). The reproducibility standard deviation has been determined to be (insert test values and corresponding reproducibility values).
When a Practice is being turned into a Standard Test Method, does it still need to have a precision and bias statement?
Per the Form and Style Manual Section A21.2.2:
Every test method shall contain a statement (1) regarding the precision of test results obtained in the same laboratory under specifically defined conditions of within-laboratory variability (repeatability conditions), and (2) regarding the precision of test results obtained in different laboratories (reproducibility conditions).
How long should a committee wait after the development of a standard test method to include a precision and bias statement?
It is recommended that ruggedness testing and an interlaboratory testing program be completed before approval of a new standard test method. If a committee determines that a delay is necessary before conducting a full ILS to produce reproducibility, a temporary statement addressing repeatability conditions is permitted for five years. Included in the statement shall be language regarding the anticipated date of actual availability of reproducibility data.
What happens if we find an old standard test method without a precision and bias statement? Is it grandfathered in?
The Form and Style for ASTM Standards mandates that all test methods contain a precision and bias statement. If a test method is discovered that does not contain this section, an evaluation should be immediately performed to determine the utility of the document and then steps should be undertaken to address the inclusion of a precision and bias statement.
How do I submit a ballot item?
To submit an item for ballot, simply log on to your "MyASTM" page, then select "Submit Item for Ballot" from the "My Tools" section on the left-hand side of the page. You will be prompted to enter the appropriate information. If you have any questions please contact your committee's Administrative Assistant.
When is a research report required?
Per the Form and Style Manual Section A29:
Research Reports (Mandatory for Precision and Bias Statements Producing Numerical Results) A29.1 Where numerical data have been generated to establish the precision and bias of a test method, a research report is required.
What parts should be included in the research report?
2. List of Participating Laboratories- Including the laboratory name, technical contact, address, email address and phone number.
3. Description of Samples- Including the supplier(s) and distributor information.
4. Laboratory Study Instructions- All instructions/guidelines provided to the participants.
5. Description of Equipment/Apparatus.
6. Data Report Forms- Provide a copy of the raw data
7. Statistical Data Summary - A summary of the statistics calculated from the data returned by the participating laboratories.
8. Precision and Bias Statement - Taken directly from the revised standard.
RR:C21-1005 is an example of an acceptable research report.
How/when/to whom do I submit a research report?
Research reports should be submitted to ILS at ILS@astm.org before the item goes to ballot. The Research report needs to be completed at that time, so that a research report number can be assigned and inserted into the approved standard.
Do research reports have to be balloted along with the standard test method?
Research reports are not usually attached to the ballot. For a copy of a research report linked to a precision and bias ballot item, members should contact their committee administrative assistant or ILS at ILS@astm.org.
When will my research report be assigned a number?
A research report will be given a number once the ballot item to which it is linked passes the balloting process and is approved.
How can I purchase a copy of a research report?
To purchase a research report, go to ASTM's website, www.astm.org, or contact ASTM Customer Service by phone at 610-832-9585 (8:30 a.m. - 4:30 p.m. Eastern U.S.R. Standard Time, Monday through Friday) or by email at firstname.lastname@example.org.
Where should the footnote for a research report on the precision and bias statement be placed?
Footnotes for the precision and bias statement should be cited in the precision and bias section. There should be no footnotes for the precision and bias statement in the scope, reference documents or any other sections of the standard.
Click here to see an example footnote.