This Digital Reader tutorial shows how to use a camera to read a digital device that may not have
a computer interface. This is especially useful for older devices that do not provide
an interface to modern computers but do have a human visual readout.
In this tutorial we use a regular air-condition sensor that one may find in any office or home. In the
setup we use a DLink DCS900 camera mainly because it has an attachment that allows a large tripod to
be connected to the camera and can transmit the image wirelessly to the base computer that is interpreting
the image viewed.
First left us have a look at what the camera is seeing in order to begin our analysis ...
On viewing the image it become immediately apparent that the lighting is not ideal in this situation. Even
for humans the digital numbers (in this case 77.0) are hard to read.
The first task is to segment the digits from the rest of the scene. Note that there are many ways
to accomplish this task and the specific modules you may use may differ from this approach.
To begin our segmentation we need to remove the lighting issue. To do this we use the Adaptive_Threshold
module. Adding this module converts the above image to:
The parameters of the adaptive thresholding should be set to approximately the width of one of the digits
in order to extract out the digit. We found 20 to be an appropriate window size.
One can easily see that the adaptive threshold does a great job of eliminating any lighting issues and highlights
the digits from the background.
But you can notice we've still got some cleanup to do ....
First off you will notice that within the image the digits are black. In order to process them correctly
going forward we need to be manipulating white objects as RoboRealm considers white pixels as foreground and
black ones as background. All the following modules use this assumption as the basis for their processing. To
remedy this situation we simply invert the image. Using the negative module.
Now that the digits are in the correct color we can continue by removing all blobs (connected pixel groups)
that are smaller than 70 pixels. The choice of 70 is largely arbitrary but it should be a number where most
of the smaller insignificant objects are removed. The actual size will naturally depend on your situation.
The digits are now appearing more clearly but due to the original low lighting that the image was taken in the
border of the digits is not very smooth. To smooth things out a little we use
the Smooth Hull module to round out the digits' shapes.
But you will notice that we still have a rather big issue. The digits themselves are broken into two blobs. In order
to best match blobs we need to combine the digits into a single blob ...
To merge the blobs we need to expand all white parts of the image until they touch with nearby parts. An
easy way to do this is with a combination of erode and dilate
modules. We first perform a 2 pixel dilation in all directions.
This helps to connect the digits but in some situations a thin line still separates the digits. If we continue
to use the dilate module we risk the chance of digits being merged into each other. So instead of using
the dilate module to dilate in all directions we simply specify the expansion to only happen in the vertical
Now that the digits have been merged we reduce them closer to their original size by using the erode
module with a 2 pixel erosion.
This reduces the digits down to a more visually appealing size.
Once the digits have been segmented from the original image we now need to understand the digit properties
as represented by blobs to help eliminate other objects in the image that are not digits. This helps to reduce
the clutter in the final stages and provides a more stable result.
You can notice that the digits appear within a square rectangle. This rectangle was initially formed by the
white casing in contrast to the lower light LCD part of the panel. We can use this characteristic to better
focus on just the digits. To do this filtering we use the Blob Filter module
with a single 'inner' parameter.
You will notice that while helpful there are still two triangles left in the image. We could use the blob
filter to further filter based on triangles and invert that selection to remove them but instead we
will use the final step to eliminate those objects.
To recognize which digit is which, we have to compare the resulting objects with a known database of objects.
This known database is simply a folder that contains images of each of the separate shapes to be recognized. The folder
we used contains the following images:
This database of images was generated by cutting the final image generated up to this point and manually
separating each digit into its own file. This was done by pressing CTRL-C inside RoboRealm when the appropriate
image was seen and pasting the image (CTRL-V) into our favorite paint program to then crop and save each
image into its own file. Note that the image filenames were specified as 0.gif, 1.gif, 2.gif, etc. in order
to make the image to numeric translation easy.
This database was then trained by the Shape Matching module which
matches each resulting object to a single file in the database. The confidence and size filters within the
Shape_Match module were then used to trim out any bad matches such as those against the remaining triangles.
Note that the digits displayed at the top of each green box is actually the filename of the image matched. Since
we named the database images with the digit they represent it is easy to translate the image name into the
actual number. In order to do the final grouping we need to use a little VBScript to combine the
separate objects together to create the final number.
Each recognized shape now represents a single digit. We need to use the shape information to create a single
"temperature" variable that contains the final number. We do this by using the VBScript module to
combine the information produced by the Shape_Match module in the SHAPE array.
' get a reference to the shapes array information
' that was created by the shape_match module
shapes = GetArrayVariable("SHAPES")
names = GetStrVariable("SHAPES_PATH")
' temp will hold the final temperature number
temp = ""
if isArray(shapes) then
' be sure we have found enough digits
if ubound(shapes) >= 26 then
' step through all objects and merge
' the number they represent
for i=0 to ubound(shapes)-1 step 9
nstart1 = shapes(i+7)
temp = temp & mid(names, nstart1+1, 1)
' we artificially put back in the period
' as we know where it should be
if i = 9 then
temp = temp & "."
' and finally create a variable that represents
' the text that we see. Note that this only
' gets set when enough digits are recognized
setvariable "temperature", temp
On executing this script a new variable "temperature" is produced that contains the recognized temperature
(362 KB) 10 second video of the original temperature images.
(688 KB) 10 second video of the processed results of the temperature images.
Notice how the shapes wiggle in the processed video but seem stable in the original? This is due to low lighting noise that causes the adaptive
threshold (first module) to make different decisions as to what is part of a digit and what is not. As this
decision affects the final shapes of the digits they will appear to wiggle when viewed as live video. You will also
notice that at times the digits will not match or accidentally become merged with other digits that cause the
matches to fail. Again this is due to lighting creating noise in the final results.
The following is the processing pipeline as seen within RoboRealm to accomplish the digital reading.
1. You will need to train the Shape_Match module on a folder that contains the database of
images to be matched. You can do this by copying all the images into a single folder, edit the
Shape_Match module, press the train button, change the path to your folder and press start.
2. You can view the final "temperature" variable by adding in a Watch_Variables module or by
editing the VBScript module and focusing on the second variable column.
Your turn ...
Want to try this yourself? Following is the robofile that should launch RoboRealm with the original image
we used in this tutorial. You can click on each of the steps in the pipeline to review what the processing
looks like up to that point.
Download the Digital Reader robofile. Don't forget
that you may need the folder of digits if you want to replicate exactly what we did. You can download that
That's all folks. We hope you've enjoyed this little adventure into an application of machine vision processing
and have inspired you to download a free trial of our software.
If you have any questions or comments about this tutorial please feel free to
Have a nice day!
|Reading Digital Displays Related Forum Posts||Last post||Posts||Views|
Digital reader issue
i have an issue with making everything right by your tutorial. On the screen you can see tha...