IPAB Workshop - Josh Smith, Sebastian Starke, Theodoros Stouraitis

Theodoros Stouraitis

 

Title: Partner Adaptive Dyadic collaborative Manipulation through Informed Hybrid Bilevel Optimization

 

Abstract: Effective collaboration is based on the individuals' ability to adapt their policy to their partner's actions. This work provides a principled formalism to address online adaptation in joint planning problems such as Dyadic collaborative Manipulation (DcM) scenarios by representing the human as task space wrenches. We propose an efficient bilevel formulation which combines informed search methods with trajectory optimization, enabling robotic agents to adapt their policy on-the-fly in accordance to changing intentions of the human partner. This method is the first to empower agents with the ability to plan online in hybrid spaces; optimizing over discrete contact locations, contact sequence patterns, continuous trajectories, and force profiles for co-manipulation tasks. This is particularly important in large object manipulation scenarios that require grasp-hold and plan changes. We demonstrate in simulation and hardware the efficacy of the bilevel optimization by investigating the effect of robot policy changes in response to real-time alterations of the human's intention, including goal changes, eminent grasp switches, as well as optimal dyadic interactions to realize the joint task.

 

Sebastian Starke

 

Title: Neural State Machine for Goal-Directed Character Control In this talk, I will present about our recent progress in AI-driven character control and animation. While learning raw locomotion controllers for bipeds and quadrupeds has been successfully explored in previous years, this research now focuses on the next challenges involving interaction with objects in the environment. The difficulty in such tasks is due to complex planning with periodic and non-periodic motions reacting to the scene geometry in order to precisely position and orient the character. We propose Neural State Machine, a novel data-driven framework to guide characters to perform actions in a goal-driven control scheme, learned in an end-to-end supervised fashion. Our proposed deep auto-regressive framework enables modeling of multi-modal scene interaction behaviors purely from data, and produces a series of movements and transitions through the state space to reach the goal in the desired goal state. In addition, to allow characters to adapt to a wide range of geometry such as different shapes of furniture and obstacles, we developed an efficient data augmentation scheme that enriches the motion manifold based on the original context of motion, does not require any relabeling, and also does not increase the data size for training. We demonstrate the versatility of our model with various scene interaction tasks such as sitting on a chair, avoiding obstacles, opening and entering through a door, and picking and carrying objects

Josh Smith

Title: Adaptive Model Learning and Control with Adaptive Dynamics Library

 

Abstract: Dynamics Identification is essential for work on highly dynamic motions, robots, and/or tasks that require the ability to control the amount of force put on the environment. In this particular talk I will briefly go through the current essence of this area of work covering some detail in the types of models that are used (Parametric, Non-Parametric and

Semi-parametric) with their respective strengths and weaknesses. I shall also be showing some work in progress that aims to solve some of the issues that will be highlighted. A simple simulated robot will also be shown that demonstrates the ability of the adaptive controllers demonstrating their abilities to track trajectories whilst learning the models.