ASTM E2566 - 17a

    Standard Test Method for Evaluating Response Robot Sensing: Visual Acuity

    Active Standard ASTM E2566 | Developed by Subcommittee: E54.09

    Book of Standards Volume: 15.08


      Format Pages Price  
    PDF 16 $52.00   ADD TO CART
    Hardcopy (shipping and handling) 16 $52.00   ADD TO CART
    Standard + Redline PDF Bundle 32 $62.40   ADD TO CART


    Significance and Use

    5.1 Various levels of visual acuity are essential when remotely operating robots in unstructured and often hazardous environments. Missions typically include establishing situational awareness, finding available paths, maneuvering through obstacles, identifying objects of interest, and performing detailed inspections. This test method measures robot system far-field and near-field visual acuity which are applicable to virtually every mission. These quantitative measures of performance provide a common language that allows robot users to better understand and express their own requirements and improve the way visual sensing capabilities are specified.

    5.2 Multiple cameras could be incorporated into remotely operated robotic systems since a single camera is unlikely to be effective for all aspects of a mission. For example, cameras with zoom lenses are often used for far-field tasks. Cameras with close focus capabilities are often used for near-field tasks. Wide-angle lenses are often used for driving and obstacle avoidance. This test method characterizes each onboard camera to understand overall system capabilities.

    5.3 This test method provides a way to unambiguously specify robot requirements in terms of the related measures of visual acuity and field of view. This helps quantify the trade-offs and general usefulness of optical versus digital zoom cameras and fixed versus variable focus lenses. The visual acuity charts can also help provide quantitative measures of performance within other test methods and training scenarios. See Figs. 2-4 for illustrations.

    FIG. 2 This Baseline Image is Used for Purposes of Comparisons Below

    This Baseline Image is Used for Purposes of Comparisons
 Below

    FIG. 3 Three Images of the Same Scene with the Same Image Resolution. Top Row Shows Field of View Increasing from Left to Right (the image “zooms out”) While Bottom Row Shows Acuity Decreasing (features of the same size become harder to clearly observe)

    Three Images of the Same Scene with the Same Image
 Resolution. Top Row Shows Field of View Increasing from Left to Right
 (the image “zooms out”) While Bottom Row Shows Acuity
 Decreasing (features of the same size become harder to clearly observe)

    FIG. 4 Three Images of the Same Scene with the Same Field of View. The Top Row Shows the Field of View is Unchanged While Bottom Row Shows Both Resolution and Acuity Increasing (features become clearer)

    Three Images of the Same Scene with the Same Field
 of View. The Top Row Shows the Field of View is Unchanged While Bottom
 Row Shows Both Resolution and Acuity Increasing (features become clearer)

    5.4 This test method helps evaluate the effect of illumination on visual acuity. In dark environments, robots typically need to illuminate the scene to be effective. Far-field objects downrange require much greater light intensity than near-field objects close to the robot. Variable illumination helps ensure the scene is neither too dark nor overwhelmingly lighted so as to thwart the camera’s ability to discern visual details (so-called “washout” of the image). Variable illumination is especially important when quickly transitioning from far-field to near-field and back again.

    5.5 Key features of response robots are that they are remotely operated from safe standoff distances, deployable at operational tempos, capable of operating in complex environments, sufficiently hardened against harsh environments, reliable and field serviceable, durable or cost-effectively disposable, and equipped with operational safeguards. As such, a major advantage of using robots in response operations is to enhance the safety and effectiveness of responders or soldiers.

    5.6 This test method aligns user expectations with actual capabilities to understand the inherent trade-offs in deployable systems at any given cost. For example, an increase in image resolution typically results in improved field of view or acuity, but not necessarily both. An increase in both may not be possible for robots of a desired weight, endurance, or cost. Appropriate levels of understanding can help ensure that requirement specifications are articulated within the limit of current capabilities.

    5.7 This test method provides a tangible representation of essential robot capabilities with quantifiable measures of performance. When considered with other related test methods in the suite, it facilitates communication among communities of robot users and manufacturers. As such, this test method can be used to:

    5.7.1 Inspire technical innovation and guide manufacturers toward implementing combinations of capabilities necessary to perform essential mission tasks.

    5.7.2 Measure and compare essential robot capabilities. This test method can establish the reliability of the system to perform specified tasks, highlight break-through capabilities, and encourage hardening of developmental systems.

    5.7.3 Inform purchasing decisions, conduct acceptance testing, and align deployment objectives with statistically significant robot capabilities data captured through repeated testing and comparison of quantitative results.

    5.7.4 Focus operator training and measure proficiency as a repeatable practice task that exercises actuators, sensors, and operator interfaces. The test method can be embedded into training scenarios to capture and compare quantitative scores even within uncontrolled environmental variables. This can help develop, maintain, measure, and track very perishable skills over time and enable comparisons across squads, regions, or national averages.

    5.8 Although this test method was developed as part of a suite of sensing tests for response robots, it may be applicable to other domains. Different user communities can set their own thresholds of acceptable performance within the test method for various mission requirements.

    5.9 It is recommended that users of this test method consider their particular robot requirements when interpreting the test results. The capability evaluated in this test method alone shall be interpreted according to the scope of this test method and shall not be considered as an overall indication of the capability of the robot’s mobility subsystem nor of the entire robotic system. A single test method only captures the specified single aspect of a robot’s capabilities. A more complete characterization of a robot’s capabilities requires test results from a wider set of test methods.

    1. Scope

    1.1 The purpose of this test method is to specify the apparatuses, procedures, and performance metrics necessary to quantitatively measure a robot’s visual acuity as displayed to a remote operator or vision algorithm. The primary performance metric for this test method shall be a robot’s possession of such a capability with a specified statistical significance level.

    1.2 Secondary performance metrics are the robot’s field of view and aspect ratio.

    1.3 This test method can also be used to measure the operator proficiency in performing the specified task. The corresponding performance metric may be the number of completed task repetitions per minute over an assigned time period ranging from 10 to 30 minutes.

    1.4 This test method is a part of the sensing suite of response robot test methods, but this test method is stand-alone and complete. This test method applies to systems operated remotely from a standoff distance appropriate for the intended mission. The system includes a remote operator in control of all functionality and any assistive features or autonomous behaviors that improve the effectiveness or efficiency of the overall system.

    1.5 The apparatus, specified in Section 6, can only test a limited range of a robot’s capabilities. When the robot has been tested through the limit or limits of the apparatus, a note shall be associated with the results indicating that the robot’s actual capability may be outside of the limit or limits imposed by the test apparatus. For example, the robot could exceed the capabilities of the printing process used to create the charts used in the apparatus.

    1.6 Performing Location—This test method may be performed anywhere the specified apparatuses and environmental conditions can be implemented.

    1.7 Units—The values stated in either SI units or inch-pound units are to be regarded separately as standard. The values stated in each system may not be exact equivalents; therefore, each system shall be used independently of the other. Combining values from the two systems may result in nonconformance with the standard. Both units are referenced to facilitate acquisition of materials internationally and minimize fabrication costs.

    1.8 This standard does not purport to address all of the safety concerns, if any, associated with its use. It is the responsibility of the user of this standard to establish appropriate safety, health, and environmental practices and determine the applicability of regulatory limitations prior to use.

    1.9 This international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for the Development of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade (TBT) Committee.


    2. Referenced Documents (purchase separately) The documents listed below are referenced within the subject standard but are not provided as part of the standard.

    ASTM Standards

    E2521 Terminology for Evaluating Response Robot Capabilities

    E2592 Practice for Evaluating Response Robot Capabilities: Logistics: Packaging for Urban Search and Rescue Task Force Equipment Caches

    Additional Standards

    ISO 12233 PhotographyElectronic Still Picture ImagingResolution and Spatial Frequency Responses Available from International Organization for Standardization (ISO), ISO Central Secretariat, BIBC II, Chemin de Blandonnet 8, CP 401, 1214 Vernier, Geneva, Switzerland, http://www.iso.org.

    ISO/IEC 18004:2015 InformationAutomatic Identification and Data Capture TechniquesQR Code Barcode Symbology Specification


    ICS Code

    ICS Number Code 13.200 (Accident and disaster control)

    Referencing This Standard
    Link Here
    Link to Active (This link will always route to the current Active version of the standard.)

    DOI: 10.1520/E2566-17A

    Citation Format

    ASTM E2566-17a, Standard Test Method for Evaluating Response Robot Sensing: Visual Acuity, ASTM International, West Conshohocken, PA, 2017, www.astm.org

    Back to Top