RoboAgent can efficiently acquire a wide diversity of non-trivial skills and can generalize them to diverse unseen scenarios.
RoboAgent Towards Sample Efficient Robot Manipulation with Semantic Augmentations and Action Chunking
Trained merely on 7500 trajectories, we are demonstrating a universal RoboAgent that can exhibit a diverse set of 12 non-trivial manipulation skills (beyond picking/pushing, including articulated object manipulation and object re-orientation) across 38 tasks and can generalize them to 100s of diverse unseen scenarios (involving unseen objects, unseen tasks, and to completely unseen kitchens). RoboAgent can also evolve its capabilities with new experiences.
RoboAgent can exhibit 12 skills across 38 tasks
Towards a universal robotic agent A causality dilemma: The grand aim of having a single robot that can manipulate arbitrary objects in diverse settings has been a distant goal for several decades. This is in-part because of the paucity of diverse robotics datasets to train such agents, at the same time absence of generic agents than can generate such dataset. Escaping the vicious circle: To escape this vicious circle our focus is on developing an efficient paradigm that can deliver a universal agent capable acquiring multiple skills under a practical data budget and generalizing them to diverse unseen situations.
RoboAgent is a culmination of effort spanning over two years. It builds on the following modular and recompensable ingredients - RoboPen - a distributed robotics infrastructure build with commodity hardware capable of long term uninterrupted operations. RoboHive - a unified framework for robot learning across simulation and real-world operations. RoboSet - a high quality dataset representing multiple skills with everyday objects in diverse scenarios. MT-ACT - an efficient language conditioned multi-task offline imitation learning framework that multiplies offline datasets by creating a diverse collection of semantic augmentations over the existing robot’s experiences and employs a novel policy architecture with efficient action representation to recover performant policies under a data budget.
Glasp is a social web highlighter that people can highlight and organize quotes and thoughts from the web, and access other like-minded people’s learning.