Closed Loop Influence

In a human-robot interaction setting, humans inevitably have flawed or biased understanding of the environment around them. They may not be sure about extrinsic factors of the world (e.g. a robot arm's joint friction), or their own internal goals/preferences (e.g. how the robot's arm should be configured). Humans often have a long learning curve when learning to operate a robotic arm with precision and dexterity. What if we made the robot more aware of the human's understanding of the world to proactively guide the human in completing a task?

In our Closed-Loop Influence approach, we setup a two-phase process of estimation and influence. In the estimation phase, we model the human's constantly changing understanding of reality using a sequence model, and in the influence phase, we use classical control to assist the human w.r.t. their perceptions.

Below is our approach's performance in a simulated human-robot environment.

Note that reducing Action Prediction Error is our actual objective, but we are also able to reduce Internal State Prediction Error! We do not explicitly train on the latter, because we do not have access to a human's internal state in real-world settings.

Here is a high-level overview of what each file does:

• build_human_action_set.py - generates the set of all possible actions a human can take in our discretized action space.
• dare.py - the math behind the Discrete Algebraic Riccati Equation (DARE), which underlies our optimal control.
• data_gen.py - generates synthetic state-action trajectories of a human interacting with a robot.
• device.py - sets whether GPU/CPU should be used for all components of the project.
• env_setup.py - sets the values of our control matrices.
• estimation.py - performs the entire estimation process.
• human.py - outlines the human/robot environment for OpenAI Gym.
• lqr.py - uses dare.py to derive optimal controls that the human would take.
• models.py - defines different models that can be used in the estimation process.
• train_mlp.ipynb - interactive environment to train a vanilla neural network-based estimator of the human's internal state.
• train_transformer.ipynb - interactive environment to train a Transformer-based estimator of the human's internal state.