New Test Method for Performance of machine vision systems for bin-picking systems
1. Scope
This standard will be used to measure the performance of machine vision systems that can measure the 6 degrees-of-freedom (6DOF) position and orientation (pose) of objects, in a bin of identical or similar objects, for the purpose of guiding a robot to pick up one or more of the objects. The performance of the machine vision system may be defined in terms of the following attributes: - Pose uncertainty: the uncertainty in the 6D pose of a predefined object as measured by the machine vision system. The positional uncertainty may be measured in mm and angular uncertainty may be measured in degrees. - Precision: the repeatability at which the vision system can determine the 6D pose of a predefined object. The directional precision is measured in terms of mm and angular precision in terms of degrees. - Reliability: the robustness of the vision system in face of complicated situations. The reliability is determined in terms of percentage of successful detection for the predefined situations listed below: - Simple slippage: The pose of the object slips along the object itself - Compound slippage: The pose of the object slips among similar and nested objects - Partial object error: error in pose estimation due to partial occlusion of an object - Coffee cup error: Pose estimation error caused by symmetry of the object - Banana symmetry error: Pose estimation error caused for objects that look almost identical when rotated 180 either in plane or off plane - Transparency error: Pose estimation error caused by transparency of the object. (i.e. how transparent an object can be before the system can no longer correctly detect it?) - Reflectiveness error: Pose estimation error caused by reflectiveness of an object. (i.e. how reflective an object can be before the system can no longer correctly detect it?)
Keywords
3D Vision; 4D Vision; Random Bin-picking; Belt-picking, Robotic Guidance
Rationale
Use of 3D or 4D vision for the purpose of robotic guidance in industrial environments is a relatively new phenomenon. There are a large variety of solutions in the market and in the absence of a worldwide accepted standard, it is hard to compare the capabilities of different solutions and to find the most suitable one for the task at hand. At the moment, users have no choice but to go through extensive, time-consuming tests to either validate or reject a solution for use in their operations. This standard can be used for comparing existing solutions before any test is needed and enables the user to pick, or at least shortlist, the solutions that suit their needs and to make an informed decision faster. The users of this standard will be system integrators or manufacturing companies who are interested in robotic automation for the purpose of sorting, assembling, and packaging.
