1. Initial Setup

This project uses Unity as a base of operations and connects to ROS to communicate to the robot. Since the Franka Emika Panda robot cannot be operated using ROS on Windows, ROS# is used to establish connection between Unity and and ROS, which is running on an Ubuntu virtual machine. The initial setup consists of:

1. Creating and setting up the virtual machine environment.
2. Installing Ubuntu on the virtual machine.
3. Installing ROS on Ubuntu.
4. Installing franka_ros on Ubuntu.
5. Installing Unity.

Ubuntu-Side Set-up

1. The tutorial on how to install Ubuntu on Oracle VM can be found here.

You should install Ubuntu 20.04 as starting with 22.04, ROS2 is preferred.

2. Install ROS on your current Ubuntu VM and follow these tutorials to get acquainted with creating a workspace.
3. Follow the instructions on the ROS# wiki to configure network settings and rosbridge-server.

MoveIt!

While installing MoveIt! is not necessary, it is recommended to get acquainted with ROS and path planning.

• The MoveIt! tutorials use Franka Emika Panda by default so the robot model is already configured with the installation of the tutorials.
• Following these tutorials also ensures that the important packages such as catkin and wstool that may have been skipped over in the ROS tutorials are installed.
• While real-time arm servoing with MoveIt! is possible through the servo_server node, it's not used for the teleoperation of the robot (updated on 13.04.2022).

franka_ros

franka_ros is an integration of the Franka Emika Panda libraries into ROS and I've slightly modified the franka_gazebo example that uses the cartesian_impedance_example_controller to achieve end effector tracking. This will be better explained in the wrist tracking sections.

• For the complete installation, following the tutorial on the Franka Emika Franka Control Interface page is enough.

Windows-Side Setup

• Follow the instructions on Unity downloads page.

Establishing Connection Between Unity and ROS

The connection between Unity and ROS is mainly carried out by two options: ROS# and Unity Robotics Hub, which is built on ROS# with additional features. While working, I've used ROS# because it seemed easier to use and more malleable but there are other examples that uses URH as well.