Video of the marble maze being autonomously solved even with a large roll error.
Results
The following are image captures of the final route as determined by the image processing and route
planning. The white square is ontop of the ball, the red square is the location closest to the
destination red square and the purple squares are the direction changes due to the blue stops in
the maze. The path is shown in red that proceeds from the white square to the red square. Note that
this planning is performed after every move in case the ball rolls in an irregular direction
or bounces off from the blue/gray blocks due to roll momentum.
Issues
Alignment
The alignment of the square maze is key to the correct processing of the maze. With minor misalignment
some of the blue squares may bleed into neighboring areas and cause a false obstacle to be
detected. This may or may not cause an issue with the maze depending on if the found route depends
on that obstacle. If the solution found
requires that the blue block exists then it may cause the system to oscillate around
a nonexistent solution. In addition, the false detection of the blue block may cause the
maze to be defined as unsolvable (i.e. the image will be set to all white).
Lighting
Other issues are due to lighting. Lego blocks and surfaces are actually very reflective and
can cause substantial lighting halos that cause the detected color to become over saturated
and not detected. This is an issue for all detected objects (blue blocks, white ball, and red
destination object). To counteract this issue the maze is tilted slightly i.e. is not perfectly
horizontal when the overhead camera takes the maze picture. This slight tilt helps to reduce the glare typically
produced by overhead lighting.
Robustness
Despite many of the inaccuracies that can develop in the system (such as the calibration going out of
phase) the system is quite robust in that after each move of the ball the entire course is
recalculated in order to know the next move to make. Due to this errors that happen due to lighting
or even the ball not rolling correctly (as seen in the above video) the end goal is still accomplished
as any move will change the dynamics of the system. As long as the system performs more correct
than wrong analysis the goal should be achieved.
Failsafe
In some cases the ball may accidentally get stuck in a strange location between the Lego blocks. In this
case the system detects this situation to see if the ball coordinate is different than the last
time. If the ball has not moved the VBScript module will command both the pan and tilt servos to move
with a larger than normal value in an attempt to "free" the stuck ball. Note that when this happens
the goal direction is ignored until the ball is found to be free and moving again. This process, while
rare, helps to ensure the system has failsafe.
Pipeline
The following is the processing pipeline as seen within RoboRealm to accomplish the marble maze solution.
Notes:
1. You can enable the Parallax controller module if you have the Parallax servo controller. (The robofile
has that module disabled on load) If not you
will need to replace that module with your appropriate controller. This configured servo controller
module assumes that you have two servos attached to the controller to move the tilt and pan of a platform
like the Marble Maze.
2. The speed of the system is currently constrained by the time it takes for the marble ball to roll from
one side of the maze to the other. Currently the algorithm waits a set amount of time regardless of how
long the ball actually needs to roll. An improvement would be to monitor the image and right the
maze back to horizontal once no more movement is detected, i.e. the ball has stopped rolling ... but we will
leave that improvement to you!
3. The core of the path finding algorithm is in the VBScript module. The VBScript essentially processes the
image as a 14x14 pixel array accessed directly in VBScript. Within this code there is a lot of additional
code used to generate the final graphic seen above. Keep in mind while looking at this code that a lot of it
is not necessary for the actual functioning of the algorithm. Only the very next move needs to be
store and
not the entire path. The entire path is recorded, however, in order to show graphically what route is currently being
attempted.
Your turn ...
Want to try this yourself? Following is the robofile that should launch RoboRealm with the original image
we used in this tutorial. You can click on each of the steps in the pipeline to review what the processing
looks like up to that point. Note that if you disable the first conditional you can highlight each successive
modules to see how the image processing is performed. The first conditional in the pipeline is to ensure that
the system only updates the ball path when
needed (i.e. the servos have completed their move) such that the system can easily run on a slower platform.
That's all folks. We hope you've enjoyed this little adventure into an application of machine vision processing
and have inspired you to download a free trial of our software.
If you have any questions or comments about this tutorial please feel free to
contact us.
Download robofile