Efficient Data Collection for Robotic Manipulation via Compositional Generalization

📦[Review] Efficient Data Collection for Robotic Manipulation via Compositional Generalization

Meta

author:: Jensen Gao, Annie Xie, Ted Xiao, Chelsea Finn, Dorsa Sadigh

---

[논문 리뷰] 강동규
Reviewed by Kade Kang (devkade12@gmail.com)
Reviewed:: 2025-12-22, 52week

First-Pass

Data collection has become an increasingly important problem in robotic manipulation, yet there still lacks much understanding of how to effectively collect data to facilitate broad generalization. Recent works on large-scale robotic data collection typically vary many environmental factors of variation (e.g., object types, table textures) during data collection, to cover a diverse range of scenarios. However, they do not explicitly account for the possible compositional abilities of policies trained on the data. If robot policies can compose environmental factors from their data to succeed when encountering unseen factor combinations, we can exploit this to avoid collecting data for situations that composition would address. To investigate this possibility, we conduct thorough empirical studies both in simulation and on a real robot that compare data collection strategies and assess whether visual imitation learning policies can compose environmental factors. We find that policies do exhibit composition, although leveraging prior robotic datasets is critical for this on a real robot. We use these insights to propose better in-domain data collection strategies that exploit composition, which can induce better generalization than naive approaches for the same amount of effort during data collection. We further demonstrate that a real robot policy trained on data from such a strategy achieves a success rate of 77.5% when transferred to entirely new environments that encompass unseen combinations of environmental factors, whereas policies trained using data collected without accounting for environmental variation fail to transfer effectively, with a success rate of only 2.5%. We provide videos at this http URL.

• Paper : Efficient Data Collection for Robotic Manipulation via Compositional Generalization

Second-Pass

Third-Pass

• 해당 논문은 정책이 다양한 환경 요인에 대한 조합에 일반화가 가능함을 보인다. 그렇다면 이 정책은 요인들간의 관계를 가지고 있을까?(e.g. A 가 B 위에 있다) 만약 가지고 있다면, 관계를 명시적으로 추출하거나 분석할 수 있는 방법은 무엇인가? 명시적으로 추출할 수 있다면, 새로운 요인 조합, 새로운 형태의 관계도 일반화할 수 있지 않을까?
• Prior data 가 compositional generalization 에 결정적인 역할을 하는데, 이때 prior data 가 로봇 정책에 부여한 이점이 무엇일까? 일반화가 될 수 있도록 지식을 전이한다면, 어떤 지식을 전이할까?
• 고려하지 못한 요인 (방해물, 조명 변화 등) 에 대해 선행 데이터가 강건성을 부여한다고 논문에서 언급하는데, 로봇이 이전에 전혀 정의되지 않고 새롭게 발생하는 환경 요인에 대해서도 compositional generalization 을 발휘할 수 있을까?