Kinect Targeting

The ideas introduced in this tutorial have not been verified to be according to FIRST rules. Please verify with the rule manual on any modifications or unique usage of items provided in the FIRST kit of parts.

This tutorial assumes you are using a Kinect with a PC on your FIRST robot and that you are interested in an autonomous guidance mechanism to keep track of the goal targets. The targets are made of 2" reto-reflective tape which will reflect light back to the source. Given a camera placed near to that light source it should pick up a much larger amount of light than the surrounding area and make it easier for you to extract out the square target for further processing. While this can work reasonably well in the visible light spectrum it is much more advantageous to use the infrared or IR spectrum as that spectrum will have much less noise (i.e. light reflected that is not part of the desired square).

Luckily the Kinect has an IR camera built into it. The way the device works is to project thousands of IR dots into the field of view of the Kinect and based on where those dots appear it can deduce depth. In our case we are not as interested in the depth since the distance to the target can be accurately determined by the size of the target. What we would like to do instead is create a better way to ensure only the target is extracted from the scene.

The Kinect's projector is very powerful and will produce a spotted pattern in the IR domain. If we instead blur this projection by placing a filter in front of the projector it will instead produce a more even IR illumination. We used a piece of packing foam taped ontop of the projector to create this blurring. Effectively, this makes the Kinect an IR camera.

With this dispersed IR illumination you can now point the Kinect at the retro-reflective target and isolate just the squares despite bright areas in the visible spectrum. This is a great trick to use to remove a lot of unwanted noise pixels just using equipment that you already have.

What is interesting is that using the higher resolution of the Kinect will increase the IR projector power and reveal an even better image. The following is a 1280x1024 image taken from the IR camera on the Kinect and scaled back down to 640x480 so that you can compare it against the one above.

These images are taken from a distance of 36 ft which is well beyond the depth range of the Kinect but thanks to the retro-reflective tape the targets are easily enhanced. Just be sure to blur the IR projector well enough otherwise you will see more speckles that may cause detection failure when processing for a 'rectangle' shape later on.

The IR image does contain parts of the rest of the scene so it is still important to process the results. For example, we used the Color Balance module to generate the following image by maximizing the Contrast. You can see that the other details of the room are being seen somewhat by the IR camera.

You can also purchase additional IR illuminators that can further enhance the image. Team #2410 Metal Mustang Robotics has reported very high quality results from using an IR LED and a Lens to enhance the IR light received by the Kinect. The brighter the illumination the further the distance you will be able to see the target and the more changes you remove any other image artifacts that may cause successive processing to get confused. Thanks Team 2410!!

Using the following  robofile and a couple of images from Team #2410 using the above IR lighting we get good results.

To try this out yourself:

1. Download RoboRealm
2. Install the Kinect Drivers
3. Run RoboRealm and add the Kinect module to the pipeline
4. Once this is activated select the IR radio button and you should see the IR pattern the Kinect generates
5. Use a semi-transparent piece of waxpaper, packing foam, etc. and tape that over the lens that has a red spot in the middle of it. Its the one isolated by itself (the other two being the IR camera and RGB camera).
6. Once you have blurred things enough you should be able to see objects in IR without all the spots present. Move the Kinect to view the retro-reflective tape and see what that looks like up close and from a distance.