|Geraldine Cheok is a research engineer at the National Institute of Standards and Technology. She received her master’s degree in civil engineering from the University of Maryland in 1986. She received the Department of Commerce Silver Medal in 2001 and the Bronze Medal in 1997 and 2004, and the American Concrete Institute’s Structural Research Award in 1997.
Alan Lytle leads the Construction Metrology and Automation Group at the National Institute of Standards and Technology. Prior to joining NIST, Lytle served in the United States Navy and then worked for a company that produced laser-based positioning technology. He was a recipient of the 2004 Department of Commerce Bronze Medal.
||Performance Evaluation of 3D Imaging Systems
Recently, the National Institute of Standards and Technology, as the United States’ primary measurement laboratory, launched an assessment of the U.S. Measurement System. The aim is to determine whether this vital infrastructure can effectively address multiplying needs for ever-more exacting and reliable measurement tools and associated services. The development of standard test methods for three-dimensional imaging systems is a recognized USMS need. The FIATECH consortium1 has also recognized this need and, in partnership with NIST, has established the Laser Scanning Measurement Assurance Project.
Although the technology for most 3D imaging systems has existed for several decades, the use of these systems has become more established or accepted only in the past 10 years. 3D imaging systems are used to rapidly capture, in thousands of measurements per second, 3D information on a scene or object. Systems include laser scanners, LADARs (laser detection and ranging), laser radars, 3D range cameras, and 3D flash LADARs. The applications for 3D imaging systems span widely varying industrial and government sectors such as construction, manufacturing (e.g., aerospace and automotive), homeland security, military, law enforcement (e.g., forensics), and transportation safety, and they continue to grow. In the construction industry, the anticipation is that these systems will eventually either replace or merge with total stations (survey instruments that measure distances and angles).
Despite their growing prevalence, there are still no standard test protocols for evaluating the performance of 3D imaging systems or best practices or methods for assessing the accuracy of their derived output (e.g., 3D models, volumes, geometric dimensions). This lack of standard test methods is inhibiting the wider industry acceptance of these systems. Standard test methods for the performance evaluation of 3D imaging systems will provide a basis for fair comparisons, reduce the confusion regarding terminology, and increase user confidence in the applications of these systems. Best practices will improve product quality and will ensure a minimum level of performance.
Summary of NIST’s Efforts
Between 2003 and 2006, NIST held four workshops2,3 to address the need for standard methods for the performance evaluation of 3D imaging systems. The workshops were attended by manufacturers, users, and researchers from academia and government agencies from around the world.
The workshop participants agreed that standard terminology and methods for assessment and evaluation of the 3D imaging systems were required. The latter includes both evaluation of instrument characteristics (e.g., range error, resolution) and assessment of the accuracy of the end-product, for example, the volume of material moved at a construction site.
Additionally, a neutral facility for evaluation/calibration was needed. The consensus was that a single facility that would encompass the entire range of 3D imaging systems would be impractical. Therefore, a minimum of three kinds of testing facilities was deemed necessary:
• A small, climatecontrolled indoor facility for highly accurate, short range instruments (<10 m);
• A medium sized, climate-controlled indoor facility for instruments with ranges up to 50 m;
• An outdoor testing area for long-range instruments and for testing in a more realistic environment.
In the course of these workshops, two draft documents were developed a terminology pre-standard and a ranging protocol pre-standard. These documents were presented at the 2006 workshop and can serve as the starting documents in a formal standardization process.
Also at the 2006 workshop, participants selected ASTM International as the standards setting body for 3D imaging systems, and ASTM Committee E57 on 3D Imaging Systems was established on June 7, 2006.
NIST Indoor Facilities
Figure 14 shows the small, indoor, artifact-based facility at NIST. The purpose of this facility is to develop test protocols and metrics for the evaluation of 3D imaging systems. At present, temperature and humidity are continuously monitored and recorded but are not controlled within this facility. Prototype artifacts developed for use in this facility include 152- and 203-millimetre (6- and 8-inch) diameter spheres, a 610 mm (24 in) diameter slotted disc, and a stair artifact. Figure 2 shows pictures and some scans of these artifacts. A 3 m ball bar (Figure 3) was also manufactured, which could be used in the field to determine if an instrument was within specified tolerance. The ball bar consists of a carbon-fiber reinforced tube with a 152-mm (6 in) diameter spherically mounted reflector at each end. Diffuse targets of known reflectivity (2 to 99 percent) are also available for indoor and outdoor use.
In addition to the artifact-based facility, an indoor 1D ranging facility5 is also available at NIST (Figure 4), which can provide reference measurements with an uncertainty of 10 µm ± 0.5 µm/m (10 µm ± 0.5 ppm); the uncertainty consists of a constant component and a length-dependent component. This facility would fall under the medium range facility described above. The facility is temperature- and humidity-controlled, and the barometric pressure is monitored. A rail system is used to position the targets up to a maximum distance of 61 m. Other artifacts available for use in this facility are 102-mm (4-in) diameter titanium spheres and 102-mm (4-in) diameter spherically mounted retroreflectors.
Future work includes planning and establishing an outdoor benchmark facility for the evaluation of 3D imaging systems at longer ranges.
3D imaging systems can fundamentally change large-scale metrology as applied to capital construction projects, manufacturing and transportation, enabling measurable improvements in project delivery, product quality, and safety. The ultimate benefits would include shorter delivery schedules, improved product quality, and increased worker safety. Standard test methods to evaluate the performance of 3D imaging systems will greatly increase market confidence and acceptance and, therefore, the use of these systems. The development of these standards will be under the auspices of ASTM Committee E57 on 3D Imaging Systems. //
1 A nonprofit consortium focused on fast-track development and deployment of technologies to substantially improve how capital projects and facilities are designed, engineered, built and maintained.
2 Cheok, G. S., Ed. , “Proceedings of the 2nd NIST LADAR Performance Evaluation Workshop March 15 - 16, 2005,” NISTIR 7266, National Institute of Standards and Technology, Gaithersburg, Md., October. www.bfrl.nist.gov/861/CMAG/ LADAR_workshop/index.htm
3 Cheok, G. S., Ed. , “Proceedings of the LADAR Calibration Facility Workshop, June 12-13, 2003,” NISTIR 7054, National Institute of Standards and Technology, Gaithersburg, Md., October. www.bfrl. nist.gov/861/CMAG/LADAR_
4 Certain company products are shown in the figure. In no case does such an identification imply recommendation or endorsement by the National Institute of Standards and Technology, nor does it imply that the products are necessarily the best available for the purpose.
5 For more information about this facility, contact Steven Phillips, steven. email@example.com.