Blob Tracking

The Blob Tracking module provides a way to track blobs (collections of pixels) from one image to the next. Often as part of any object tracking solution it is necessary to identify not only that object moved but which object was which when comparing the current frame and the last frame.

The Blob Tracking module will label each blob with a specific id that will be attached to the same or similar blob in the next frame. What defines a blob as being similar in two images depends on how you have configured the Blob Tracking module.

Note that two images are needed for the blob tracking to work. The first is an image that defines the blobs. This is basically an image that is the result of a threshold, flood fill, segment colors, etc. and normally has many fewer unique colored pixels that the original image. The second image is the actual original image that was used to create the blob image. This image is needed in order to create better statistics than what the blob image would define. For example, a blob image would contain a single color for an entire blob, whereas the original image may have different colors that would better identify the blob in successive images. Thus one image is used to define what is a blob and the second is used to understand the features of that defined blob.

Interface

Instructions

1. Blob Pixels - Select the image that represents connected blobs. This is normally an image that has very few colors (segmented) such that same color pixels can be used to define a blob.

2. Raw Pixels - Select the image that contains the original raw pixel values that was used to define the blobs. This provides the raw pixel information needed to calculate the tracking features.

3. Tracking Features - Select the required features that are used to determine a blobs similarity from one image to the next. Try to use as few features as possible for your application. What features are required will depend on your application and what is being tracked. Selecting the appropriate checkboxes will add that feature into the similarity checking. Note that if the feature is of less importance but still valid you can decrease the weight to something <1.0. Note that the weights can be above 1.0 if you want to instead strengthen a particular feature with regards to all the others. Again, what weights are needed for what features will depend on your project requirements.

Location - Similarity is defined as the proximity or closeness of two blobs. I.e. a blob in the current image close to a blob in the previous image means they are similar.

Size - Similarity is defined as the area in pixels of two blobs. I.e. a blob in the current image with the same size as a blob in the previous image means they are similar.

Color - Similarity is defined as the pixel color of two blobs. I.e. a blob in the current image with the same color as a blob in the previous image means they are similar.

Aspect - Similarity is defined as the aspect ratio (width/height) of two blobs. I.e. a blob in the current image with the same aspect ratio of a blob in the previous image means they are similar.

Orientation - Similarity is defined as the orientation/angle of two blobs. I.e. a blob in the current image with the same orientation (second order moment) of a blob in the previous image means they are similar.

Shape - Similarity is defined as the shape comparison of two blobs. I.e. a blob in the current image with the same shape as a blob in the previous image means they are similar.

Histogram - Similarity is defined as the color histogram of difference of two blobs. I.e. a blob in the current image with the same color distribution as a blob in the previous image means they are similar.

4. Match Threshold - Specify the match similarity threshold which defines if a successful match is made between a current blob and a blob in the last image. If the similarity score between the blobs is above the threshold then the match is made, otherwise the blob is considered to be new and not connected to a previous blob.

5. Precision - When tracking a blob an array of a blobs position is noted. The precision defines how many samples are added to the trail of blob positions. Increasing the precision will shorten the blob's trail list and remove smaller movements caused be noise. Decreasing the precision will increase the length of the blob's trail. See the variables list below ...

6. Present After Frames - To avoid having blobs that appear momentary to being treated as an active blob the Present After Frames number will require a blob to be present for X number of frames in order to be declared present and start being tracked.

7. Present After Pixel - To avoid stable blobs that do not move from bring tracked the Present After Pixels number prevents blobs that remain within X number of pixels from their first location from being tracked. Thus if you use 10 then any blob needs to move more than 10 pixels (relative to their center) in order for the blob to start being tracked.

8. Absent After Frames - When tracking a blob it might suddenly disappear due to a momentary glitch or image disturbance. It is very desirable not to have the blob be reassigned a new id just due to a quick disappearance. The Absent frame number provides a way for a blob to be remembered for a couple of frames in order to allow the blob to be reacquired and retain its current id. By setting the number higher than zero you allow the module to remember blobs in previous frames in hope to preserve the id. Setting the number too high may cause blobs to stay in memory around for too long a time and then be reassigned to new blobs appearing close to the departure point of a previous blob.

9. Graphic Color - The color of the overlaid id and trail (connecting blob arrow).

10. Display Trail - If you want to see how a blob has moved in previous frames select the Display Trail checkbox which will enable that graphic to show.

11. Trail Length - Select how many samples you want the trail length to be. The longer the trail the further in the past the blob's position can be seen.

12. Display Blob Id - Each tracked blob is provided a unique id that identifies that blob from image to image. This id can be displayed ontop of each blob for viewing purposes.

13. Font Size - Select the font size of the displayed blob id.

14. Display As Annotation - Select if you want the graphics to be draw after all processing has been completed. If this is NOT selected then the next module in the processing pipeline will see the graphic as if it were part of the image and process it accordingly.

Notes

1. There is a maximum limit of 100 tracked positions. I.e. a trail cannot be more than 100 samples.

2. There is a maximum blob tracking count of 8192. If you have more than 100 blobs in the current image you will need to reduce that number down by either filtering the image or increasing the segmentation amount.

Example

Blob_Tracking after movement detection on Freeway

Variables

Each new image will create a bunch of variables that reflect the current tracking status. There are two types, blobs that are currently being tracked and those that have stopped tracking (most likely the blob is no longer present in the image). All the following are arrays that contain the current tracking state. Use the Watch Variable module to see these being created and modified per image.