Eating Breadcrumbs

 Download the Breadcrumbs robofile.

1. This demo shows how to steer the Roomba to pickup crumbs or small dark objects that along a path. The demo will turn on the main Roomba brush and vacuum in order to attempt to pickup those crumbs. This demo works best on a light carpet with dark objects. It will NOT work well on a multi-colored (i.e. Persian) carpet as it assumes an even colored background. The demo requires that the camera is pointed down at the floor so that it can only see the immediate floor in front of it.

2. First double click on the Roomba module in the processing pipeline and change the serial COM port to the one you are using. Note that if you happen to have a serial port on your Netbook then the COM port will most likely be from COM1 to COM4. If you are using a USB to serial converter then it can be a much higher number. RoboRealm will only display those COM ports that are active within the system so the COM port selection should be limited to only a few selections. If you are not sure on which COM port is being used start with the highest one and test while moving to lower ones.

   If you have already configured the Roomba in a previous robofile then the COM port should already be configured for your robot assuming the "Remember as default" checkbox in the Roomba GUI interface is set.
   

3. If the camera view is not already present press the Camera button in the main RoboRealm interface to show you the USB webcam image. If you have multiple cameras installed or other imaging devices you may need to press the Options button->Video Tab and select the camera from the dropdown list that shows all the imaging devices installed in the system.

4. Once you get the COM port and camera set you should see the image from the Roomba in the main RoboRealm GUI. From here you can place the Roomba in front of a trail of crumbs it will start to follow that trail which sucking up the breadcrumbs. Be sure to check the camera image to ensure that the crumbs are showing up correctly in the webcam image.

5. We perform this task by first evening out the lighting in the image using the Flatten module. This helps to reduce the effects of shadows, highlights and reflections that may be on the ground or on the breadcrumb itself.

6. Next we segment the image (divide it into blob objects) by collecting together like colored pixels into connected objects of the same mean color using the Flood Fill module. The result of this module is a cartoon like image which we can now process as objects instead of individual pixels. Being able to process the images using object (or connected pixels) we can determine attributes that we could not on individual pixels such as the size of an object (i.e. how many pixels comprise that object).

7. By removing objects that are too bright, too small, too big, those touching the border or those that are not well defined (i.e. their edges are soft) we can eliminate all but the most distinct crumbs which we should focus on and cause the robot to turn. This filtering is performed by the Blob Filter module which analyses blobs and calculates attributes/statistics that we can filter against.
8. Now we want to know the center of gravity of the trail which is a good indication of where we should move the robot in order to suck up all the breadcrumbs. We do this using the Center of Gravity module which identifies the center of all the crumbs in view and thus is a good target to move towards to keep on the trail.
   

9. Finally we now need to tell the Roomba how to move in order to stay on the trail by targeting this COG. We do this by understanding on what side of the image the identified target is. If the desired target point is right of the center of the image we need to move the robot right, whilst if it is on the left we need to move left. If we create this behavior the robot will move in the direction of the line. The VBscript module allows us to write some VBScript to set the motor values of the Roomba based on the X coordinate of the target point. The more the X location is to the right the harder we should turn LEFT and vice versa for the other side. The VBScript module produces the left and right motor variables that contains the suggested direction that the Roomba should move. The Roomba module then grabs this value and passes it to the Roomba robot to complete the cycle.