Skip to content

SenseAct: A computational framework for developing real-world robot learning tasks

License

Notifications You must be signed in to change notification settings

kindredresearch/SenseAct

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

SenseAct: A computational framework for real-world robot learning tasks

License

This repository provides the implementation of several reinforcement learning tasks with multiple real-world robots. These tasks come with an interface similar to OpenAI-Gym so that learning algorithms can be plugged in easily and in a uniform manner across tasks. All the tasks here are implemented based on a computational framework of robot-agent communication proposed by Mahmood et al. (2018a), which we call SenseAct. In this computational framework, agent and environment-related computations are ordered and distributed among multiple concurrent processes in a specific way. By doing so, SenseAct enables the following:

  • Timely communication between the learning agent and multiple robotic devices with reduced latency,
  • Easy and systematic design of robotic tasks for reinforcement learning agents,
  • Facilitate reproducible real-world reinforcement learning.

This repository provides the following real-world robotic tasks, which are proposed by Mahmood et al. (2018b) as benchmark tasks for reinforcement learning algorithms:

Universal-Robots (UR) robotic arms:

Tested on UR Software v. 3.3.4.310

UR-Reacher-2
UR-Reacher-2
UR-Reacher-6
UR-Reacher-6

Dynamixel (DXL) actuators:

Currently we only support MX-64AT.

DXL-Reacher
DXL-Reacher
DXL-Tracker
DXL-Tracker

iRobot Create 2 robots:

Create-Mover
Create-Mover
Create-Docker
Create-Docker

Mahmood et al. (2018b) provide extensive results comparing multiple reinforcement learning algorithms on the above tasks, and Mahmood et al. (2018a) show the effect of different task-setup elements in learning. Their results can be reproduced by using this repository (see documentation for more information).

Versions

The branch master is the latest official release and dev is current development branch.

Installation

SenseAct uses Python3 (>=3.5), and all other requirements are automatically installed via pip.

On Linux and Mac OS X, run the following:

  1. git clone https://github.com/kindredresearch/SenseAct.git
  2. cd SenseAct
  3. pip install -e . or pip3 install -e . depends on your setup

To replicate experimental results from the paper please install the tag v0.1.1 from the git repo.

  1. git fetch --all --tags
  2. git checkout tags/v0.1.1

Additional instruction for installing OpenAI Baselines needed for running the advanced examples is given in the corresponding readme.

Additional installation steps for Dynamixel-based tasks (Linux only)

Dynamixels can be controlled by drivers written using either ctypes by Robotis or pyserial, which can be chosen by passing either True (ctypes) or False (pyserial) as an argument to the use_ctypes_driver parameter of a Dynamixel-based task (e.g., see examples/advanced/dxl_reacher.py). We found the ctypes-based driver to provide substantially more timely and precise communication compared to the pyserial-based one.

In order to use the CType-based driver, we need to install gcc and relevant packages for compiling the C libraries:

sudo apt-get install gcc-5 build-essential gcc-multilib g++-multilib

Then run the following script to download and compile the Dynamixel driver C libraries:

sudo bash setup_dxl.sh

For additional setup and troubleshooting information regarding Dynamixels, please see DXL Docs.

Tests

You can check whether SenseAct is installed correctly by running the included unit tests.

cd SenseAct
python -m unittest discover -b

Support

Installation problems? Feature requests? General questions?

Acknowledgments

This project is developed by the Kindred AI Research team. Rupam Mahmood, Dmytro Korenkevych, and Brent Komer originally developed the computational framework and the UR tasks. William Ma developed the Create 2 tasks and contributed substantially by adding new features to SenseAct. Gautham Vasan developed the DXL tasks. Francois Hogan developed the simulated task.

James Bergstra provided support and guidance throughout the development. Adrian Martin, Scott Rostrup, and Jonathan Yep developed the pyserial DXL driver for a Kindred project, which was used for the SenseAct DXL Communicator. Daniel Snider, Oliver Limoyo, Dylan Ashley, and Craig Sherstan tested the framework, provided thoughtful suggestions, and confirmed the reproducibility of learning by running experiments on real robots.

Citing SenseAct

For the SenseAct computational framework and the UR-Reacher tasks, please cite Mahmood et al. (2018a). For the DXL and the Create 2 tasks, please cite Mahmood et al. (2018b).