Search ASTM
A01 STEEL, STAINLESS STEEL AND RELATED ALLOYS A04 IRON CASTINGS A05 METALLIC-COATED IRON AND STEEL PRODUCTS B01 ELECTRICAL CONDUCTORS B05 COPPER AND COPPER ALLOYS B07 LIGHT METALS AND ALLOYS C01 CEMENT C04 VITRIFIED CLAY PIPE C07 LIME AND LIMESTONE C09 CONCRETE AND CONCRETE AGGREGATES C11 GYPSUM AND RELATED BUILDING MATERIALS AND SYSTEMS C12 MORTARS AND GROUTS FOR UNIT MASONRY C13 CONCRETE PIPE C14 GLASS AND GLASS PRODUCTS C15 MANUFACTURED MASONRY UNITS C16 THERMAL INSULATION C17 FIBER-REINFORCED CEMENT PRODUCTS C18 DIMENSION STONE C21 CERAMIC WHITEWARES AND RELATED PRODUCTS C24 BUILDING SEALS AND SEALANTS C27 PRECAST CONCRETE PRODUCTS D01 PAINT AND RELATED COATINGS, MATERIALS, AND APPLICATIONS D04 ROAD AND PAVING MATERIALS D07 WOOD D08 ROOFING AND WATERPROOFING D09 ELECTRICAL AND ELECTRONIC INSULATING MATERIALS D11 RUBBER D14 ADHESIVES D18 SOIL AND ROCK D20 PLASTICS D35 GEOSYNTHETICS E05 FIRE STANDARDS E06 PERFORMANCE OF BUILDINGS E33 BUILDING AND ENVIRONMENTAL ACOUSTICS E36 ACCREDITATION & CERTIFICATION E57 3D IMAGING SYSTEMS E60 SUSTAINABILITY F01 ELECTRONICS F06 RESILIENT FLOOR COVERINGS F13 PEDESTRIAN/WALKWAY SAFETY AND FOOTWEAR F16 FASTENERS F17 PLASTIC PIPING SYSTEMS F33 DETENTION AND CORRECTIONAL FACILITIES F36 TECHNOLOGY AND UNDERGROUND UTILITIES G03 WEATHERING AND DURABILITY C14 GLASS AND GLASS PRODUCTS C21 CERAMIC WHITEWARES AND RELATED PRODUCTS D01 PAINT AND RELATED COATINGS, MATERIALS, AND APPLICATIONS D06 D09 ELECTRICAL AND ELECTRONIC INSULATING MATERIALS D10 PACKAGING D11 RUBBER D12 SOAPS AND OTHER DETERGENTS D13 TEXTILES D14 ADHESIVES D15 ENGINE COOLANTS AND RELATED FLUIDS D20 PLASTICS D21 POLISHES D31 LEATHER E12 COLOR AND APPEARANCE E18 SENSORY EVALUATION E20 TEMPERATURE MEASUREMENT E35 PESTICIDES, ANTIMICROBIALS, AND ALTERNATIVE CONTROL AGENTS E41 LABORATORY APPARATUS E53 ASSET MANAGEMENT E57 3D IMAGING SYSTEMS F02 FLEXIBLE BARRIER PACKAGING F05 BUSINESS IMAGING PRODUCTS F06 RESILIENT FLOOR COVERINGS F08 SPORTS EQUIPMENT, PLAYING SURFACES, AND FACILITIES F09 TIRES F10 LIVESTOCK, MEAT, AND POULTRY EVALUATION SYSTEMS F11 VACUUM CLEANERS F13 PEDESTRIAN/WALKWAY SAFETY AND FOOTWEAR F14 FENCES F15 CONSUMER PRODUCTS F16 FASTENERS F24 AMUSEMENT RIDES AND DEVICES F26 FOOD SERVICE EQUIPMENT F27 SNOW SKIING F37 LIGHT SPORT AIRCRAFT F43 LANGUAGE SERVICES AND PRODUCTS F44 GENERAL AVIATION AIRCRAFT A01 STEEL, STAINLESS STEEL AND RELATED ALLOYS A04 IRON CASTINGS A05 METALLIC-COATED IRON AND STEEL PRODUCTS A06 MAGNETIC PROPERTIES B01 ELECTRICAL CONDUCTORS B02 NONFERROUS METALS AND ALLOYS B05 COPPER AND COPPER ALLOYS B07 LIGHT METALS AND ALLOYS B08 METALLIC AND INORGANIC COATINGS B09 METAL POWDERS AND METAL POWDER PRODUCTS B10 REACTIVE AND REFRACTORY METALS AND ALLOYS C03 CHEMICAL-RESISTANT NONMETALLIC MATERIALS C08 REFRACTORIES C28 ADVANCED CERAMICS D01 PAINT AND RELATED COATINGS, MATERIALS, AND APPLICATIONS D20 PLASTICS D30 COMPOSITE MATERIALS E01 ANALYTICAL CHEMISTRY FOR METALS, ORES, AND RELATED MATERIALS E04 METALLOGRAPHY E07 NONDESTRUCTIVE TESTING E08 FATIGUE AND FRACTURE E12 COLOR AND APPEARANCE E13 MOLECULAR SPECTROSCOPY AND SEPARATION SCIENCE E28 MECHANICAL TESTING E29 PARTICLE AND SPRAY CHARACTERIZATION E37 THERMAL MEASUREMENTS E42 SURFACE ANALYSIS F01 ELECTRONICS F34 ROLLING ELEMENT BEARINGS F40 DECLARABLE SUBSTANCES IN MATERIALS F42 ADDITIVE MANUFACTURING TECHNOLOGIES G01 CORROSION OF METALS G03 WEATHERING AND DURABILITY D21 POLISHES D26 HALOGENATED ORGANIC SOLVENTS AND FIRE EXTINGUISHING AGENTS D33 PROTECTIVE COATING AND LINING WORK FOR POWER GENERATION FACILITIES E05 FIRE STANDARDS E27 HAZARD POTENTIAL OF CHEMICALS E30 FORENSIC SCIENCES E34 OCCUPATIONAL HEALTH AND SAFETY E35 PESTICIDES, ANTIMICROBIALS, AND ALTERNATIVE CONTROL AGENTS E52 FORENSIC PSYCHOPHYSIOLOGY E54 HOMELAND SECURITY APPLICATIONS E58 FORENSIC ENGINEERING F06 RESILIENT FLOOR COVERINGS F08 SPORTS EQUIPMENT, PLAYING SURFACES, AND FACILITIES F10 LIVESTOCK, MEAT, AND POULTRY EVALUATION SYSTEMS F12 SECURITY SYSTEMS AND EQUIPMENT F13 PEDESTRIAN/WALKWAY SAFETY AND FOOTWEAR F15 CONSUMER PRODUCTS F18 ELECTRICAL PROTECTIVE EQUIPMENT FOR WORKERS F23 PERSONAL PROTECTIVE CLOTHING AND EQUIPMENT F26 FOOD SERVICE EQUIPMENT F32 SEARCH AND RESCUE F33 DETENTION AND CORRECTIONAL FACILITIES G04 COMPATIBILITY AND SENSITIVITY OF MATERIALS IN OXYGEN ENRICHED ATMOSPHERES D08 ROOFING AND WATERPROOFING D18 SOIL AND ROCK D19 WATER D20 PLASTICS D22 AIR QUALITY D34 WASTE MANAGEMENT D35 GEOSYNTHETICS E06 PERFORMANCE OF BUILDINGS E44 SOLAR, GEOTHERMAL AND OTHER ALTERNATIVE ENERGY SOURCES E47 E48 BIOENERGY AND INDUSTRIAL CHEMICALS FROM BIOMASS E50 ENVIRONMENTAL ASSESSMENT, RISK MANAGEMENT AND CORRECTIVE ACTION E60 SUSTAINABILITY F20 HAZARDOUS SUBSTANCES AND OIL SPILL RESPONSE F40 DECLARABLE SUBSTANCES IN MATERIALS G02 WEAR AND EROSION B01 ELECTRICAL CONDUCTORS C26 NUCLEAR FUEL CYCLE D02 PETROLEUM PRODUCTS, LIQUID FUELS, AND LUBRICANTS D03 GASEOUS FUELS D05 COAL AND COKE D19 WATER D27 ELECTRICAL INSULATING LIQUIDS AND GASES D33 PROTECTIVE COATING AND LINING WORK FOR POWER GENERATION FACILITIES E10 NUCLEAR TECHNOLOGY AND APPLICATIONS E44 SOLAR, GEOTHERMAL AND OTHER ALTERNATIVE ENERGY SOURCES E48 BIOENERGY AND INDUSTRIAL CHEMICALS FROM BIOMASS A01 STEEL, STAINLESS STEEL AND RELATED ALLOYS C01 CEMENT C09 CONCRETE AND CONCRETE AGGREGATES D02 PETROLEUM PRODUCTS, LIQUID FUELS, AND LUBRICANTS D03 GASEOUS FUELS D04 ROAD AND PAVING MATERIALS D15 ENGINE COOLANTS AND RELATED FLUIDS D18 SOIL AND ROCK D24 CARBON BLACK D35 GEOSYNTHETICS E12 COLOR AND APPEARANCE E17 VEHICLE - PAVEMENT SYSTEMS E21 SPACE SIMULATION AND APPLICATIONS OF SPACE TECHNOLOGY E36 ACCREDITATION & CERTIFICATION E57 3D IMAGING SYSTEMS F03 GASKETS F07 AEROSPACE AND AIRCRAFT F09 TIRES F16 FASTENERS F25 SHIPS AND MARINE TECHNOLOGY F37 LIGHT SPORT AIRCRAFT F38 UNMANNED AIRCRAFT SYSTEMS F39 AIRCRAFT SYSTEMS F41 UNMANNED MARITIME VEHICLE SYSTEMS (UMVS) F44 GENERAL AVIATION AIRCRAFT F45 DRIVERLESS AUTOMATIC GUIDED INDUSTRIAL VEHICLES D10 PACKAGING D11 RUBBER E31 HEALTHCARE INFORMATICS E35 PESTICIDES, ANTIMICROBIALS, AND ALTERNATIVE CONTROL AGENTS E54 HOMELAND SECURITY APPLICATIONS E55 MANUFACTURE OF PHARMACEUTICAL PRODUCTS E56 NANOTECHNOLOGY F02 FLEXIBLE BARRIER PACKAGING F04 MEDICAL AND SURGICAL MATERIALS AND DEVICES F29 ANESTHETIC AND RESPIRATORY EQUIPMENT F30 EMERGENCY MEDICAL SERVICES G04 COMPATIBILITY AND SENSITIVITY OF MATERIALS IN OXYGEN ENRICHED ATMOSPHERES C07 LIME AND LIMESTONE D14 ADHESIVES D16 AROMATIC HYDROCARBONS AND RELATED CHEMICALS D20 PLASTICS D26 HALOGENATED ORGANIC SOLVENTS AND FIRE EXTINGUISHING AGENTS D28 ACTIVATED CARBON D32 CATALYSTS E13 MOLECULAR SPECTROSCOPY AND SEPARATION SCIENCE E15 INDUSTRIAL AND SPECIALTY CHEMICALS E27 HAZARD POTENTIAL OF CHEMICALS E35 PESTICIDES, ANTIMICROBIALS, AND ALTERNATIVE CONTROL AGENTS F40 DECLARABLE SUBSTANCES IN MATERIALS E11 QUALITY AND STATISTICS E36 ACCREDITATION & CERTIFICATION E43 SI PRACTICE E55 MANUFACTURE OF PHARMACEUTICAL PRODUCTS E56 NANOTECHNOLOGY F42 ADDITIVE MANUFACTURING TECHNOLOGIES
Bookmark and Share

DataPoints

DataPoints

Test Method (Im)precision

How to Account for It

Q: How can you account for test method (im)precision in the adjudication of a petroleum product specification conformance dispute?

A. In the petroleum industry, commercial petroleum product specifications are generally articulated in terms of maximum or minimum limits as measured by specifically referenced standardized test methods (STM) developed by ASTM International or other standards development organizations. The current de facto industry practice is that an aliquot taken from a homogenous batch of finished product will be tested once using the specification-referenced method by the supplier, and the product is deemed to be conforming and fit for release if this single test result meets the specification limit.

Since we do not live in a perfect world, replicate execution of the same STM on the same material by different operators in different laboratories will not always yield numerically identical results.1 Hence, there will be occasions where a supplier will ship a product based on a single conforming test result, but a retest by the customer at the receiving facilities will yield a nonconforming result.

This article introduces an objective protocol in ASTM D3244 on how to address the product conformance issue when faced with two conflicting results as described in the above scenario.

D3244, Practice for Utilization of Test Data to Determine Conformance with Specifications, is under the jurisdiction of ASTM D02.94, Coordinating Subcommittee on Quality Assurance and Statistics, a part of ASTM Committee D02 on Petroleum Products and Lubricants. The standard covers guidelines and statistical methodologies with which two parties, usually a supplier and a receiver, can compare and combine independently obtained test results to obtain an assigned test value for the purpose of resolving a product quality dispute. The technique for determining the acceptance limit, against which the ATV is compared, is an integral part of this protocol. The protocol applies only to STMs with published repeatability and reproducibility limits that conform to requirements of Form and Style for ASTM Standards.

A brief overview of the D3244 protocol is described below to handle the case when there are two results: One is conforming (usually from the supplier) and the other is nonconforming (usually from the receiver or an independent third party retest). Each result is obtained from a single application of the specification-referenced STM by the supplier’s and receiver’s (or independent) laboratory, respectively.

The D3244 protocol calls for the following steps:

1. The supplier and receiver agree a priori on the probability of acceptance, P, if the true value of the property is exactly at the specification limit. This P value is required for the calculation of the AL in step 3.

2. Compare the difference between the two results to the published reproducibility, R, of the STM. If this difference is less than or equal to R, then calculate the ATV by taking the average of both results. If not, reject both results and obtain a new set of results using a mutually agreed upon aliquot. Repeat step 2.

3. Calculate the AL based on the ATV in step 2, the P value agreed upon in step 1, and R for the STM using Equation 2 in the practice. If the ATV meets the AL, the product is accepted in accordance with the P value agreed upon in step 1. Otherwise the product is rejected.

Determination of the P value in step 1 is guided by the mutually agreed upon criticality of the specification. In the absence of an agreement to the contrary, practice D3244 recommends the following:

  • For noncritical specifications, set the AL such that there is 95 percent probability of product acceptance if the true value of the property is exactly at the specification limit value.
  • For critical specifications, set the AL such that there is 5 percent probability of product acceptance if the true value of the property is exactly at the specification limit value.

Practice D3244 defines critical specifications as those specifications that, due to the product characteristic or the end use of the product, or both, require that the receiver have a high degree of assurance that the true value of the product property actually meets or exceeds the quality level indicated by the specification limit value. Noncritical specifications are defined as those that only require reasonable assurance that the product property is not substantially poorer than indicated by the specification limits.

It should be noted that for P = 0.05 (critical specification), the AL will actually be numerically inside the specification limit values, which will result in a lower consumer’s risk of unknowingly accepting nonconforming product. For P = 0.95, the AL will be outside the specification limit values, which will result in a lower supplier’s risk of falsely rejecting conforming product.

When P = 0.5, the AL coincides exactly with the specification limit. This means that there is a 50 percent probability that the product will be accepted if the true value of the property is exactly at the specification limit. The practical implication with P = 0.5 is that the receiver and supplier equally share the risk associated with test method (im)precision.

This is also the delineation point between critical and noncritical specification as chosen by the practice.

As a prerequisite for acceptance for lab test results to be used in the calculation of ATV, practice D3244 also requires the following:

  • Long-term standard deviation for the STM, as practiced by each lab, for material typical of the product in dispute, is statistically indistinguishable or better than the published method standard deviation under reproducibility conditions.
  • Each lab must be able to demonstrate, by way of results from proficiency testing programs, a lack of a systemic bias in the execution of the STM in question.

Both requirements can be substantiated by in-house quality control programs involving appropriate statistical control charts that meet the requirements of D6299, Practice for Applying Statistical Quality Assurance and Control Charting Techniques to Evaluate Analytical Measurement System Performance. Practice D6299 is also under the jurisdiction of D02.94.

Interested readers are encouraged to study the examples in the annex of practice D3244.

Reference

1. Lau, Alex, “What Are Repeatability and Reproducibility? Part 1: A D02 Viewpoint for Laboratories,” ASTM Standardization News, March/April 2009, pp. 15-18.

Alex T. Lau, TCL-Consulting, Whitby, Ontario, Canada, is chairman of Subcommittees D02.94 on Quality Assurance and Statistics and D02.01B on Precision of Combustion Characteristics Test Methods, which are part of ASTM Committee D02 on Petroleum Products and Lubricants. An ASTM International fellow, Lau is also a member of Committees E11 on Quality and Statistics, E36 on Accreditation and Certification and F08 on Sports Equipment and Facilities.

Dean V. Neubauer, Corning Inc., Corning, N.Y., coordinates the DataPoints column; an ASTM International fellow, he is chairman of Committee E11 on Quality and Statistics and chairman of E11.90.03 on Publications.

This article appears in the issue of Standardization News.