One UR10 with gripper & Leap Motion

image

This section is the continuation of phases 1, 2 and 3, where the integration of Leap Motion into the system takes place, i.e. phases 4 and 5. In the previous phases, the robots were controlled by sending the previously defined trajectories to perform the Pick & Place, but now this is done by Leap Motion, controlled by a person who sends the commands to the robot.

For this purpose, part of the Leap Motion repository is directly modified and adapted to properly control the simulated robot. The idea is identical to what was presented above, but the data input is the information provided by Leap Motion. This information needs to be properly processed and adapted for use with the API that provides it.

:book: Implementing the manipulator with Leap Motion

There are a few things to keep in mind when implementing this:

  • Determining the position of the robot using the API of the move_group node.
  • The working environment of the robot.
  • The working environment of Leap Motion.

Since we want the robot to follow the movements of Leap Motion, we will implement a controller whose workspace is contained in the robot’s workspace.

:computer: Create package

cd ~/MultiCobot-UR10-Gripper/src/multirobot/two_arm_moveit
catkin_create_pkg two_arm_moveit_leap_motion rospy
cd two_arm_moveit_leap_motion
mkdir scripts
cd scripts
cp ~/MultiCobot-UR10-Gripper/src/multirobot/two_arm_no_moveit/two_arm_no_moveit_leap_motion/scripts/leap_interface.py .
touch ur10_1_lm_robot_manipulator.py
touch ur10_2_lm_robot_manipulator.py
touch sender.py
touch leap_interface.py

:computer: Modified from the original repository.

The following files were created: leapcobotright.msg and leapcobotleft.msg in the msg directory of the Leap Motion repository:

[...]
## Generate messages in the 'msg' folder
add_message_files(
  FILES
  Arm.msg
  Bone.msg
  Finger.msg
  Gesture.msg
  Hand.msg
  Human.msg

  # For backwards compatibility with the old driver files
  leap.msg
  leapros.msg
  leapcobotright.msg
  leapcobotleft.msg
)
[...]

:computer: Tests with the Gazebo simulator

At least 4 terminals are needed to run the tests, although they can be reduced in the startup files to run them automatically. But to better visualize the information sent and for debugging, it was left like this.

  • Terminal 1: 
    cd ~/MultiCobot-UR10-Gripper
source devel/setup.bash
roslaunch one_arm_moveit_gazebo ur10_joint_limited.launch
  • Terminal 2: 
    cd ~/MultiCobot-UR10-Gripper
source devel/setup.bash
roslaunch one_arm_moveit_manipulator one_arm_moveit_execution.launch
  • Terminal 3: 
    cd ~/MultiCobot-UR10-Gripper
source devel/setup.bash
rosrun one_arm_moveit_leap_motion sender.py
  • Terminal 4: 
    cd ~/MultiCobot-UR10-Gripper
source devel/setup.bash
rosrun one_arm_moveit_leap_motion lm_robot_manipulator.py
  • Terminal 5 (in case the Leap Motion device fails): 
    sudo service leapd restart

Updated: