Lego NXT Vision Ball Picker

(1.1 MB) Video from the point of view of the Lego NXT as it sees the ball and cone. This is the original unprocessed video.

(1.4 MB) The processed video as seen from the Tribot's perspective. Notice the switch from blue ball to red cone tracking!

Problems

It is worth pointing out some of the failures that we experienced during the creating of this tutorial. As one can learn much from mistakes we chose to share some of the issues this current robot configuration has.

1. Perhaps the most difficult issue is keeping the robot together. Grabbing, bumping, and jerking can reduce the solidity of the Lego robot and it tends to start becoming lose which causes the camera mount to drop or raise slightly. This changes the field of view of the camera. Giving a quick squeeze to the camera mount once in a while ensured that the mounting of the gripper claw was stable and that the camera could receive the best field of view.

2. The wireless camera+receiver can be very noisy. In fact many trials were terminated due to the camera image suddenly becoming violently colorized with jagged lines everyone (the kind of screen one sees in a highly noisy electrical environment). We will even go as far as creating a module around this issue to stop the robot when it detects this scenario. It only effects a couple frames at a time but often the ball is only seen for a couple frames and thus the robot can easily miss the detection.

3. Camera battery life is an issue. Once the battery becomes less powerful the image tends to become darker and darker. It is bad enough to have to deal with the specula lighting that is produced on shiny surfaces but we also had to deal with the camera light being reduced. Luckily using the flicker reduction and tweaking the threshold values the robot could still see the ball in an image where a human would have great difficulty in seeing it. In this case software was able to compensate.

4. The Lego NXT bluetooth is great for going wireless but we found that the delay in motor control can become too great for near real-time control. From the video you can see how the ball is almost overshot but then corrects itself. This is in part due to the induced delays inherent in the system of which Bluetooth is a factor. Regardless, reducing speed is the best way to solve this issue.

Your turn ...

Want to try this yourself? Download the Ball Picker .robo file to run an example yourself. Note that you need RoboRealm installed to run this file. Clicking on any of the modules will bring up that interface and allow you to customize it for yourself. Try getting the robot to track a green ball instead of a blue one ...

The End

That's all folks. We hope you've enjoyed this little adventure into an application of machine vision processing and have inspired you to download a free trial of our software.

If you have any questions or comments about this tutorial please feel free to contact us.

Have a nice day!