Hahne, F., Prasad, V., Chalvatzaki, G., Peters, J., & Kshirsagar, A. (2026).
Task-Aware Bimanual Affordance Prediction via VLM-Guided Semantic-Geometric Reasoning

ICRA 2026 Workshop on Semantics for Reliable Robot Autonomy
Full text
Share

Abstract

Bimanual manipulation requires joint reasoning over object affordances and arm allocation, a challenge for geometry-only planners. To address this, we propose a hierarchical framework leveraging Vision-Language Models (VLMs) for task-aware bimanual affordance prediction without category-specific training. Our approach fuses multi-view RGB-D data to generate global 6-DoF grasps, which the VLM filters to determine task-relevant contact regions and optimal arm assignments. Evaluated on a dual-arm robot across nine real-world tasks, including tool use and human handovers, our approach significantly outperforms existing baselines, demonstrating that VLM-guided semantic reasoning enables highly reliable bimanual manipulation in unstructured environments.