Cooperative push-grasping in clutter via deep reinforcement learning

Grasping tasks in real-world scenarios, where objects are often tightly packed, present significant challenges. A collaborative push-grasping strategy based on deep reinforcement learning offers a potential approach for handling cluttered object grasping. However, the exploration of extensive workspace reduces training efficiency and generates numerous ineffective or even harmful actions. Additionally, the robot only receives reward from the environment when it successfully grasps an object or its push changes the workspace, which makes it difficult to precisely evaluate the effectiveness of the actions. To address these issues, a novel collaborative push-grasping method based on deep Q-learning is proposed in this paper. Specifically, we design the mask to eliminate the unnecessary exploration regions. The mask allows the robot to focus on safe and meaningful regions. Moreover, we propose a novel reward function to finely evaluate the effectiveness of the actions. For push, the degree of dispersion of objects caused by the push is assessed by quantifying variation in the total contour length of all objects. For grasp, its robustness is evaluated by testing the robot’s ability to maintain the grasp despite introduced disturbance. Based on these action evaluations, a hierarchical reward function is designed. Extensive experiments conducted in both simulation and real-world demonstrate that our method quickly learns more effective and precise actions, achieving superior action efficiency and success rates compared to existing approaches. Notably, our model can be transferred to real-world applications without any fine-tuning. A supplementary video is available at https://www.youtube.com/watch?v=ACiuZxtKVM4.