Search ASTM
A01 STEEL, STAINLESS STEEL AND RELATED ALLOYS A04 IRON CASTINGS A05 METALLIC-COATED IRON AND STEEL PRODUCTS B01 ELECTRICAL CONDUCTORS B05 COPPER AND COPPER ALLOYS B07 LIGHT METALS AND ALLOYS C01 CEMENT C04 VITRIFIED CLAY PIPE C07 LIME AND LIMESTONE C09 CONCRETE AND CONCRETE AGGREGATES C11 GYPSUM AND RELATED BUILDING MATERIALS AND SYSTEMS C12 MORTARS AND GROUTS FOR UNIT MASONRY C13 CONCRETE PIPE C14 GLASS AND GLASS PRODUCTS C15 MANUFACTURED MASONRY UNITS C16 THERMAL INSULATION C17 FIBER-REINFORCED CEMENT PRODUCTS C18 DIMENSION STONE C21 CERAMIC WHITEWARES AND RELATED PRODUCTS C24 BUILDING SEALS AND SEALANTS C27 PRECAST CONCRETE PRODUCTS D01 PAINT AND RELATED COATINGS, MATERIALS, AND APPLICATIONS D04 ROAD AND PAVING MATERIALS D07 WOOD D08 ROOFING AND WATERPROOFING D09 ELECTRICAL AND ELECTRONIC INSULATING MATERIALS D11 RUBBER D14 ADHESIVES D18 SOIL AND ROCK D20 PLASTICS D35 GEOSYNTHETICS E05 FIRE STANDARDS E06 PERFORMANCE OF BUILDINGS E33 BUILDING AND ENVIRONMENTAL ACOUSTICS E36 ACCREDITATION & CERTIFICATION E57 3D IMAGING SYSTEMS E60 SUSTAINABILITY F01 ELECTRONICS F06 RESILIENT FLOOR COVERINGS F13 PEDESTRIAN/WALKWAY SAFETY AND FOOTWEAR F16 FASTENERS F17 PLASTIC PIPING SYSTEMS F33 DETENTION AND CORRECTIONAL FACILITIES F36 TECHNOLOGY AND UNDERGROUND UTILITIES G03 WEATHERING AND DURABILITY C14 GLASS AND GLASS PRODUCTS C21 CERAMIC WHITEWARES AND RELATED PRODUCTS D01 PAINT AND RELATED COATINGS, MATERIALS, AND APPLICATIONS D06 D09 ELECTRICAL AND ELECTRONIC INSULATING MATERIALS D10 PACKAGING D11 RUBBER D12 SOAPS AND OTHER DETERGENTS D13 TEXTILES D14 ADHESIVES D15 ENGINE COOLANTS AND RELATED FLUIDS D20 PLASTICS D21 POLISHES D31 LEATHER E12 COLOR AND APPEARANCE E18 SENSORY EVALUATION E20 TEMPERATURE MEASUREMENT E35 PESTICIDES, ANTIMICROBIALS, AND ALTERNATIVE CONTROL AGENTS E41 LABORATORY APPARATUS E53 ASSET MANAGEMENT E57 3D IMAGING SYSTEMS F02 FLEXIBLE BARRIER PACKAGING F05 BUSINESS IMAGING PRODUCTS F06 RESILIENT FLOOR COVERINGS F08 SPORTS EQUIPMENT, PLAYING SURFACES, AND FACILITIES F09 TIRES F10 LIVESTOCK, MEAT, AND POULTRY EVALUATION SYSTEMS F11 VACUUM CLEANERS F13 PEDESTRIAN/WALKWAY SAFETY AND FOOTWEAR F14 FENCES F15 CONSUMER PRODUCTS F16 FASTENERS F24 AMUSEMENT RIDES AND DEVICES F26 FOOD SERVICE EQUIPMENT F27 SNOW SKIING F37 LIGHT SPORT AIRCRAFT F43 LANGUAGE SERVICES AND PRODUCTS F44 GENERAL AVIATION AIRCRAFT A01 STEEL, STAINLESS STEEL AND RELATED ALLOYS A04 IRON CASTINGS A05 METALLIC-COATED IRON AND STEEL PRODUCTS A06 MAGNETIC PROPERTIES B01 ELECTRICAL CONDUCTORS B02 NONFERROUS METALS AND ALLOYS B05 COPPER AND COPPER ALLOYS B07 LIGHT METALS AND ALLOYS B08 METALLIC AND INORGANIC COATINGS B09 METAL POWDERS AND METAL POWDER PRODUCTS B10 REACTIVE AND REFRACTORY METALS AND ALLOYS C03 CHEMICAL-RESISTANT NONMETALLIC MATERIALS C08 REFRACTORIES C28 ADVANCED CERAMICS D01 PAINT AND RELATED COATINGS, MATERIALS, AND APPLICATIONS D20 PLASTICS D30 COMPOSITE MATERIALS E01 ANALYTICAL CHEMISTRY FOR METALS, ORES, AND RELATED MATERIALS E04 METALLOGRAPHY E07 NONDESTRUCTIVE TESTING E08 FATIGUE AND FRACTURE E12 COLOR AND APPEARANCE E13 MOLECULAR SPECTROSCOPY AND SEPARATION SCIENCE E28 MECHANICAL TESTING E29 PARTICLE AND SPRAY CHARACTERIZATION E37 THERMAL MEASUREMENTS E42 SURFACE ANALYSIS F01 ELECTRONICS F34 ROLLING ELEMENT BEARINGS F40 DECLARABLE SUBSTANCES IN MATERIALS F42 ADDITIVE MANUFACTURING TECHNOLOGIES G01 CORROSION OF METALS G03 WEATHERING AND DURABILITY D21 POLISHES D26 HALOGENATED ORGANIC SOLVENTS AND FIRE EXTINGUISHING AGENTS D33 PROTECTIVE COATING AND LINING WORK FOR POWER GENERATION FACILITIES E05 FIRE STANDARDS E27 HAZARD POTENTIAL OF CHEMICALS E30 FORENSIC SCIENCES E34 OCCUPATIONAL HEALTH AND SAFETY E35 PESTICIDES, ANTIMICROBIALS, AND ALTERNATIVE CONTROL AGENTS E52 FORENSIC PSYCHOPHYSIOLOGY E54 HOMELAND SECURITY APPLICATIONS E58 FORENSIC ENGINEERING F06 RESILIENT FLOOR COVERINGS F08 SPORTS EQUIPMENT, PLAYING SURFACES, AND FACILITIES F10 LIVESTOCK, MEAT, AND POULTRY EVALUATION SYSTEMS F12 SECURITY SYSTEMS AND EQUIPMENT F13 PEDESTRIAN/WALKWAY SAFETY AND FOOTWEAR F15 CONSUMER PRODUCTS F18 ELECTRICAL PROTECTIVE EQUIPMENT FOR WORKERS F23 PERSONAL PROTECTIVE CLOTHING AND EQUIPMENT F26 FOOD SERVICE EQUIPMENT F32 SEARCH AND RESCUE F33 DETENTION AND CORRECTIONAL FACILITIES G04 COMPATIBILITY AND SENSITIVITY OF MATERIALS IN OXYGEN ENRICHED ATMOSPHERES D08 ROOFING AND WATERPROOFING D18 SOIL AND ROCK D19 WATER D20 PLASTICS D22 AIR QUALITY D34 WASTE MANAGEMENT D35 GEOSYNTHETICS E06 PERFORMANCE OF BUILDINGS E44 SOLAR, GEOTHERMAL AND OTHER ALTERNATIVE ENERGY SOURCES E47 E48 BIOENERGY AND INDUSTRIAL CHEMICALS FROM BIOMASS E50 ENVIRONMENTAL ASSESSMENT, RISK MANAGEMENT AND CORRECTIVE ACTION E60 SUSTAINABILITY F20 HAZARDOUS SUBSTANCES AND OIL SPILL RESPONSE F40 DECLARABLE SUBSTANCES IN MATERIALS G02 WEAR AND EROSION B01 ELECTRICAL CONDUCTORS C26 NUCLEAR FUEL CYCLE D02 PETROLEUM PRODUCTS, LIQUID FUELS, AND LUBRICANTS D03 GASEOUS FUELS D05 COAL AND COKE D19 WATER D27 ELECTRICAL INSULATING LIQUIDS AND GASES D33 PROTECTIVE COATING AND LINING WORK FOR POWER GENERATION FACILITIES E10 NUCLEAR TECHNOLOGY AND APPLICATIONS E44 SOLAR, GEOTHERMAL AND OTHER ALTERNATIVE ENERGY SOURCES E48 BIOENERGY AND INDUSTRIAL CHEMICALS FROM BIOMASS A01 STEEL, STAINLESS STEEL AND RELATED ALLOYS C01 CEMENT C09 CONCRETE AND CONCRETE AGGREGATES D02 PETROLEUM PRODUCTS, LIQUID FUELS, AND LUBRICANTS D03 GASEOUS FUELS D04 ROAD AND PAVING MATERIALS D15 ENGINE COOLANTS AND RELATED FLUIDS D18 SOIL AND ROCK D24 CARBON BLACK D35 GEOSYNTHETICS E12 COLOR AND APPEARANCE E17 VEHICLE - PAVEMENT SYSTEMS E21 SPACE SIMULATION AND APPLICATIONS OF SPACE TECHNOLOGY E36 ACCREDITATION & CERTIFICATION E57 3D IMAGING SYSTEMS F03 GASKETS F07 AEROSPACE AND AIRCRAFT F09 TIRES F16 FASTENERS F25 SHIPS AND MARINE TECHNOLOGY F37 LIGHT SPORT AIRCRAFT F38 UNMANNED AIRCRAFT SYSTEMS F39 AIRCRAFT SYSTEMS F41 UNMANNED MARITIME VEHICLE SYSTEMS (UMVS) F44 GENERAL AVIATION AIRCRAFT F45 DRIVERLESS AUTOMATIC GUIDED INDUSTRIAL VEHICLES D10 PACKAGING D11 RUBBER E31 HEALTHCARE INFORMATICS E35 PESTICIDES, ANTIMICROBIALS, AND ALTERNATIVE CONTROL AGENTS E54 HOMELAND SECURITY APPLICATIONS E55 MANUFACTURE OF PHARMACEUTICAL PRODUCTS E56 NANOTECHNOLOGY F02 FLEXIBLE BARRIER PACKAGING F04 MEDICAL AND SURGICAL MATERIALS AND DEVICES F29 ANESTHETIC AND RESPIRATORY EQUIPMENT F30 EMERGENCY MEDICAL SERVICES G04 COMPATIBILITY AND SENSITIVITY OF MATERIALS IN OXYGEN ENRICHED ATMOSPHERES C07 LIME AND LIMESTONE D14 ADHESIVES D16 AROMATIC HYDROCARBONS AND RELATED CHEMICALS D20 PLASTICS D26 HALOGENATED ORGANIC SOLVENTS AND FIRE EXTINGUISHING AGENTS D28 ACTIVATED CARBON D32 CATALYSTS E13 MOLECULAR SPECTROSCOPY AND SEPARATION SCIENCE E15 INDUSTRIAL AND SPECIALTY CHEMICALS E27 HAZARD POTENTIAL OF CHEMICALS E35 PESTICIDES, ANTIMICROBIALS, AND ALTERNATIVE CONTROL AGENTS F40 DECLARABLE SUBSTANCES IN MATERIALS E11 QUALITY AND STATISTICS E36 ACCREDITATION & CERTIFICATION E43 SI PRACTICE E55 MANUFACTURE OF PHARMACEUTICAL PRODUCTS E56 NANOTECHNOLOGY F42 ADDITIVE MANUFACTURING TECHNOLOGIES
Bookmark and Share

DataPoints

DataPoints

Setting Control Limits

The Roles of Data and Subject Matter Experts

Q: How are limits set?

A. Setting limits depends on what you are doing. Since the purpose of a limit and the methodology for constructing a limit are specific to its application, this article will not attempt to describe the solution for a specific problem. Instead, this article describes considerations made when someone is asked to produce a limit, set of limits or system of limits.

Consequences of Exceeding the Limit

The primary considerations are the consequences and planned reaction for exceeding the limit. This may be negotiated with the client as potential limits and flexibility in reaction are balanced. The negotiation may reveal what is driving the request for the limit: usually process control, process monitoring for quality assurance and quality control, regulation, physical science or customer specifications. An observation that falls outside a limit is perhaps out of control or OOC (i.e., exceeds control limits), out of trend or OOT (i.e., exceeds trend limits), out of specification or OOS (i.e., exceeds specification limits) or perhaps just interesting (i.e., exceeds alert limits). The reaction to an observation that falls outside a limit may be reporting only, but generally there is a stronger consequence to company resources (e.g., compliance) or an operational consequence such as a change to the process (e.g., process control). The reader should attempt to match the methodology for constructing the limit to expected consequences within the context of the client’s request.

Interaction with Experts

Data-driven decisions require substantive knowledge of the context and mechanism(s) for generating the data. In psychiatric research, a subject may be excluded from an analysis used to set limits on normative psychophysiological function due to nutritional or medical history, yet included in another analysis that compares individual results to those limits. In pharmaceutical manufacturing, a test result observed by a QC laboratory may be excluded from the process history used to calculate control limits if the batch record confirms that the material tested was not representative of the process.

Input from subject matter experts is needed when an initial limit is required for a new production process or when an updated limit is required following a major change to an existing process. Two years of process history is desirable, yet uncommon, for calculating control limits. If the process is new, few sources of variation will have the opportunity to manifest in the small number of observations, n, available for the calculation. Consequently, limits regularly require adjustment as the number of data points (information) increases. Input from subject matter experts is needed to characterize processes with few observations (small n) due to infrequent output (e.g., four batches per year). A calculated limit may be adjusted by a process engineer or a physical scientist to a value known by other means as meeting production needs.

Data, Frequency and Inference

Limits on a process may be set at the minimum and maximum of observed process history (two years recommended). The minimum and maximum are data points, but they also have a frequency interpretation: the 0th and 100th percentiles, where n = 100 observations. With sufficient information about a process (large n), other percentiles of the data may be calculated directly or estimated using nonparametric procedures.1 The 0.25th and 99.75th percentiles capture extremes at the rate of 1 in 400 each and may provide a useful screen. Alert limits (e.g., minimal reaction) and action limits (e.g., strong reaction) may be staged percentiles, such as the 95th and 99th percentiles (or 5th and 1st percentiles, respectively).

Percentiles used as limits may be computed by statistical inference. For example, it is known by the central limit theorem that process observations will conform to the normal (Gaussian) distribution over time if the observations (e.g., QC lab test results) are constructed as means or totals (sums). It is also known that this actually works.2 With normality, it is known that kσ limits (calculated as the product of a multiplier k and an estimated standard deviation s, or ks) cover about 95 percent of possible results when k = 2, and 99.73 percent when k = 3, assuming that you have an adequate estimate s of the total variance σ. Without normality, the limits widen to obtain similar coverage: At least (1 - 1/k2) x 100% of the data fall within k standard deviations of the mean, symmetry not required (Chebyshev’s theorem), providing at least 94 percent coverage with k = 4 and at least 96 percentage coverage with k = 6. Inferential confidence bounds meeting highly prescribed criteria might be found in such books as Hahn and Meeker’s Statistical Intervals: A Guide for Practitioners.3

Automation or Manual Production of Calculated Limits

It is easy to automate the routine calculations needed to maintain a system of limits using all available data. However, input from subject matter experts regarding exclusions and process issues cannot be automated. A system of limits for an automated significance editing procedure called the SignEdit System4 is planned for agricultural surveys in the Research and Development Division of the National Agricultural Statistics Service, U.S. Department of Agriculture. The SignEdit System includes a system of action limits and alert limits on process behavior. For farms included in every reporting period of a survey cycle, current values are compared to previously reported values to check for outliers that might reflect processing errors. A ratio comparison attributed to Hidiroglou and Berthelot of Statistics Canada was used.5 The distributional properties of the H-B effects are unknown, so a manual approach was used to set initial limits across roughly one year of process history for development purposes.6 For purposes of automation, action limits at the 0.25th and 99.75th percentiles were planned with annual recalculation based on two years of process history. The action limits indicate an automated restriction of OOT farm reports from the donor pool for donor imputation. The alert limits were planned at the 1st and 99th percentiles. The alert limits indicate an automated notification for manual review where the HB effects will be studied further and where the performance of the action limits may be evaluated. Automation was adopted as hundreds of limits were required to control potential outliers from each survey question of each participating survey.

References

1. Natrella, Mary, Experimental Statistics: National Bureau of Standards Handbook 91, Washington, D.C., U.S. Department of Commerce, 1963.

2. Shewhart, Walter, Economic Control of Quality of Manufactured Product, Bell Telephone Laboratories Series, New York, D. Van Nostrand Co. Inc., 1931.

3. Hahn, Gerald, and Meeker, William, Statistical Intervals: A Guide for Practitioners, Hoboken, John Wiley and Sons, 1991.

4. Kosler, Joseph S., “Survey Process Control with Significance Editing,” Conference Proceedings: Fourth International Conference on Establishment Surveys, June 11-14, 2012, Montreal, Canada, American Statistical Association.

5. LaTouche, Michel, and Berthelot, Jean-Marie, “Use of a Score Function to Prioritize and Limit Recontacts in Editing Business Surveys,” Journal of Official Statistics, Vol. 8, No. 3, Sept. 1992, pp. 389-400.

6. Tukey, John, Exploratory Data Analysis, John Wiley and Sons, 1977.

Joseph S. Kosler, Ph.D., a statistician in the Research and Development Division of the National Agricultural Statistics Service at the U.S. Department of Agriculture in Fairfax, Va., is the secretary of ASTM Committee E11 on Quality and Statistics and an appointee to U.S. Food and Drug Administration advisory committees on clinical pharmacology and pharmaceutical science.

Dean V. Neubauer, Corning Inc., Corning, N.Y., is an ASTM fellow; he serves as chairman of Committee E11 on Quality and Statistics, chairman of E11.90.03 on Publications and coordinator of the DataPoints column.

This article appears in the issue of Standardization News.