Filtering the 3D scan

For the 3D scan display, we don't necessarily want to show every point that the camera sees.  For example, the robot arm itself.  We already display a graphical version of the arm that we draw based on outputs from the arm.  When the arm and 3d points overlap, it's impossible to see exactly what's going on.

We can also remove things like the table the whole setup sits on and replace it with a plane.  This will be easier to see, and give more contrast to items of interest, such as the blocks or pipe cleaners we will be using.  The ground plane is simple enough to filter... since the coordinate system for the robot arm is what's used in the actual program.  Anything below zero altitude is simply eliminated.

In the case of the robot arm, though, it's a little bit trickier.  There are two ways I can think of to filter out the points.  The first would be to put bounding boxes around the  larger parts of the arm, then move the bounding boxes along with the arm and test to see if points are inside.  The second method is to use OpenGL to render my relatively detailed model of the robot arm from the viewpoint of the stereo camera and in that way determine which pixels correspond to the robot arm.  We have a mapping between 3D points and image pixels, so that's not the hard part.  The second way is a little bit more nebulous to me... I'm not sure how difficult it will be.  I do think the second way is better, though.  Certainly faster, and probably more accurate as well.  We can get a "closer trim" with the second method.