Gripper feedback

As I watched participants in the pilot study, the dependence on cameras is still very high. The system as it was was not developed enough to support a camera-less operation. In particular, the gripper had no feedback on whether it was actually closed all the way or just partway. The virtual display was only able to show the commanded state of the gripper, as that is all the servo controller (and in fact the servos themselves) give.

The result of this lack of feedback is that the operator must depend on other sources to examine the state of the system. One of the experiment parameters I had set was to use the 3d model by itself and a birds-eye-view camera. The up-close side-view camera was disabled. So the only way someone could check to see if they successfully grabbed an object was to lift it up until the gripper was in the camera view and see if the object was there. More often than not, they missed, and so had to try again.

As a result, I found a relatively simple way to get position feedback from these hobbyist servos. Now when the gripper misses an object, it shows up completely closed, and when it successfully grasps an object, it shows up halfway open. I think this will help quite a bit.

Virtual situation awareness

In running the pilot study for my experiment, one of the participants ended up rather forcefully bashing the gripper-mounted camera into an obstacle. 

The interface display consisted of a shaded point cloud representing the 3d environment around the robot arm, a virtual model of the robot arm, and a few video streams showing views from cameras on the arm and off to the side of the arm.  The virtual arm display did not include a representation for the camera and rigging to mount the camera, which things are about half the size of the total arm. 

At the time of the collision, the participant was focused mostly on the 3d virtual display and not the camera views.  As such, he thought he had plenty of space before colliding with the obstacle.  Of course one fix would be to add the camera and rigging to the virtual model that is displayed.