Austin Weber // Senior Editor // webera@bnpmedia.com

Picking and placing parts is one of the most common robotic applications. However, there is often a trade-off between high accuracy for a repetitive motion and reliability in an unstructured environment.

To teach robots how to move objects into an organized arrangement, engineers at the Massachusetts Institute of Technology (MIT) have developed a system called SimPLE (Simulation to Pick, Localize, and placE). Given only a model of the object, it generates training data by sampling grasps in simulation.

“SimPLE is a learning-based [tool] to pick, regrasp and place objects precisely that doesn’t require any prior real-world experience with the objects,” says Maria Bauza, Ph.D., a recent MIT grad who worked on the project at MIT’s Manipulation and Mechanisms Laboratory (MCube) along with lab director Alberto Rodriguez, Ph.D., and other students.

“SimPLE relies on three main components, which are developed in simulation,” explains Bauza. “First, a task-aware grasping module selects an object that is stable, observable and favorable to placing. Then, a visuotactile perception module fuses vision and touch to localize the object with high precision. Finally, a planning module computes the best path to the goal position.”

The combination of these three modules allows the robot to compute robust and efficient plans to manipulate a wide range of objects with high precision, while requiring no previous interaction with them in the real world.

According to Bauza, SimPLE is one of the first studies that utilizes tactile feedback for a complex manipulation task. “During our experiments, we found that both tactile [feedback] and vision are necessary to achieve the best performance, which suggest the importance of considering visuotactile perception in robotics applications,” she explains.

“Visuotactile perception is the ability to combine vision and touch sensing to perceive objects,” says Bauza. “In our case, this means combining vision and touch to precisely estimate the pose (position and orientation) of objects in order to manipulate them.”

Bauza and her colleagues used a dual-arm YuMi machine from ABB Robotics that was equipped with a pair of WSG-32 grippers from Weiss Robotics GmbH.

“Traditionally, robotics research has focused on single-arm setups with parallel-jaw grippers,” explains Bauza. “While this setting is usually sufficient to perform grasping on objects, it is often insufficient to efficiently perform precise pick-and-place.

“Dual-arm robots, while more challenging to control, offer a wider task space enabling the machine to easily regrasp objects and reorient them as required by the final placing configuration,” Bauza points out.

“On each gripper, we attached two fingers,” says Bauza. “For the arm that produces the first grasp, these fingers are [equipped] with tactile sensors called GelSlim V3, which are the final evolution of a family of tactile sensors developed at MCube. On the other arm, we use nonsensorized fingers made with the same shape and materials.”

Bauza and her colleagues also used machine vision technology to process information from a depth camera and the signal, in the form of images, produced by two tactile sensors in the arm that performs the first grasp on an object. “The algorithm we devised for perception works equally for both sensing modalities,” she explains.

“The main idea behind it is to leverage simulation to learn an embedding space where two images encoded in this embedding space are close, if the object poses that generated them are close,” says Bauza. “As a result, to obtain the distribution of possible objects positions given a new image from the depth camera or from a tactile sensor, we just need to encode that image and compare its embedding against a dense set of possible images previously generated in simulation.”