Blob Tracking

The Blob Tracking module provides a way to track blobs (collections of pixels) from one image to the next. Often as part of any object tracking solution it is necessary to identify not only that object moved but which object was which when comparing the current frame and the last frame.

The Blob Tracking module will label each blob with a specific id that will be attached to the same or similar blob in the next frame. What defines a blob as being similar in two images depends on how you have configured the Blob Tracking module.

Note that two images are needed for the blob tracking to work. The first is an image that defines the blobs. This is basically an image that is the result of a threshold, flood fill, segment colors, etc. and normally has many fewer unique colored pixels that the original image. The second image is the actual original image that was used to create the blob image. This image is needed in order to create better statistics than what the blob image would define. For example, a blob image would contain a single color for an entire blob, whereas the original image may have different colors that would better identify the blob in successive images. Thus one image is used to define what is a blob and the second is used to understand the features of that defined blob.

Interface

Instructions

1. Blob Pixels - Select the image that represents connected blobs. This is normally an image that has very few colors (segmented) such that same color pixels can be used to define a blob.

2. Raw Pixels - Select the image that contains the original raw pixel values that was used to define the blobs. This provides the raw pixel information needed to calculate the tracking features.

3. Tracking Features - Select the required features that are used to determine a blobs similarity from one image to the next. Try to use as few features as possible for your application. What features are required will depend on your application and what is being tracked. Selecting the appropriate checkboxes will add that feature into the similarity checking. Note that if the feature is of less importance but still valid you can decrease the weight to something <1.0. Note that the weights can be above 1.0 if you want to instead strengthen a particular feature with regards to all the others. Again, what weights are needed for what features will depend on your project requirements.

Overlap - Similarity is defined as the overlapping of two blobs. I.e. some blob pixels in the current image are in the same location as in the previous image.

Location - Similarity is defined as the proximity or closeness of two blobs. I.e. a blob in the current image close to a blob in the previous image means they are similar.

Size - Similarity is defined as the area in pixels of two blobs. I.e. a blob in the current image with the same size as a blob in the previous image means they are similar.

Color - Similarity is defined as the pixel color of two blobs. I.e. a blob in the current image with the same color as a blob in the previous image means they are similar.

Aspect - Similarity is defined as the aspect ratio (width/height) of two blobs. I.e. a blob in the current image with the same aspect ratio of a blob in the previous image means they are similar.

Orientation - Similarity is defined as the orientation/angle of two blobs. I.e. a blob in the current image with the same orientation (second order moment) of a blob in the previous image means they are similar.

Shape - Similarity is defined as the shape comparison of two blobs. I.e. a blob in the current image with the same shape as a blob in the previous image means they are similar.

Histogram - Similarity is defined as the color histogram of difference of two blobs. I.e. a blob in the current image with the same color distribution as a blob in the previous image means they are similar.

4. Score Threshold - Specify the match similarity threshold which defines if a successful match is made between a current blob and a blob in the last image. If the similarity score between the blobs is above the threshold then the match is made, otherwise the blob is considered to be new and not connected to a previous blob.

5. Length Limit - Specify the length limit that connected blobs can move in any direction from one frame to the next. If you have slow moving objects you can lower the length limit to ensure that no spurious connections over that limit are made. You can enter zero to ignore this limit.

6. Left, Right, Down, Up - Specify the length limit that connected blobs can move from one frame to the next in the appropriate axis. Right is positive X, Up is positive Y. If you have slow horizontal moving objects you can lower the X length limit to ensure that no spurious connections over that limit are made. Both directions (positive and negative) are supported to allow right (positive X direction) but limited left (set negative to -1) movement. You can enter zero to ignore any limits and allow for as much change in distance as needed.

7. Precision - When tracking a blob an array of a blobs position is noted. The precision defines how many samples are added to the trail of blob positions. Increasing the precision will shorten the blob's trail list and remove smaller movements caused be noise. Decreasing the precision will increase the length of the blob's trail. See the variables list below ...

8. Present After Frames - To avoid having blobs that appear momentary to being treated as an active blob the Present After Frames number will require a blob to be present for X number of frames in order to be declared present and start being tracked.

9. Present After Pixel - To avoid stable blobs that do not move from bring tracked the Present After Pixels number prevents blobs that remain within X number of pixels from their first location from being tracked. Thus if you use 10 then any blob needs to move more than 10 pixels (relative to their center) in order for the blob to start being tracked.

10. Absent After Frames - When tracking a blob it might suddenly disappear due to a momentary glitch or image disturbance. It is very desirable not to have the blob be reassigned a new id just due to a quick disappearance. The Absent frame number provides a way for a blob to be remembered for a couple of frames in order to allow the blob to be reacquired and retain its current id. By setting the number higher than zero you allow the module to remember blobs in previous frames in hope to preserve the id. Setting the number too high may cause blobs to stay in memory around for too long a time and then be reassigned to new blobs appearing close to the departure point of a previous blob.

11. Reset Tracking - Specify a variable whose value will be monitored to indentify restarting of a video stream. For example, in the Media_Reader module the VIDEO_FRAME variable indicates which frame in the video is currently being viewed. When this counter resets below a previous value the Blob_Tracking module knows to also reset tracking information since the video looped.

12. Remove Un-Tracked - You can chose to remove those blobs that are present in the image but are not being tracked (perhaps they are just stationary objects).

12. Single Color/Labelled Color - The color of the overlaid id and trail (connecting blob arrow). If you select Labelled Color each track and blob will use a unique color (similar to Blob Label).

13. Display Trail - If you want to see how a blob has moved in previous frames select the Display Trail checkbox which will enable that graphic to show.

14. Samples - Select how many samples you want the trail length to be. The longer the trail the further in the past the blob's position can be seen.

15. Display Blob Id - Each tracked blob is provided a unique id that identifies that blob from image to image. This id can be displayed ontop of each blob for viewing purposes.

16. Font Size - Select the font size of the displayed blob id.

17. Display As Annotation - Select if you want the graphics to be draw after all processing has been completed. If this is NOT selected then the next module in the processing pipeline will see the graphic as if it were part of the image and process it accordingly.

Notes

1. There is a maximum limit of 100 tracked positions. I.e. a trail cannot be more than 100 samples.

2. There is a maximum blob tracking count of 8192. If you have more than 100 blobs in the current image you will need to reduce that number down by either filtering the image or increasing the segmentation amount.

Example

Blob_Tracking after movement detection on Freeway

Variables

Each new image will create a bunch of variables that reflect the current tracking status. There are two types, blobs that are currently being tracked and those that have stopped tracking (most likely the blob is no longer present in the image). All the following are arrays that contain the current tracking state. Use the Watch Variable module to see these being created and modified per image.

BLOB_TRACKING - an array of id1,x1,y1,id2,x2,y2,... of all objects that are currently being tracked
BLOB_TRACKING_IDS - an array of ids that are currently being tracked
BLOB_TRACKING_BOUNDING_COORDINATES - p1x, p1y, p2x, p2y, p3x, p3y, p4x, p4y coordinates of the Bounding
	Box of the tracked blob. Note that this is an ARRAY with 8 numbers per blob.
BLOB_TRACKING_%ID% - for each id being tracked a variable holding an array of past COGs is created
BLOB_TRACKING_%ID%_X - the current X cog coordinate for the blob being tracked
BLOB_TRACKING_%ID%_Y - the current Y cog coordinate for the blob being tracked
BLOB_TRACKING_%ID%_MIN_X - the smallest X bounding box coordinate for the blob being tracked
BLOB_TRACKING_%ID%_MIN_Y - the smallest Y bounding box coordinate for the blob being tracked
BLOB_TRACKING_%ID%_MAX_X - the largest X bounding box coordinate for the blob being tracked
BLOB_TRACKING_%ID%_MAX_Y - the largest Y bounding box coordinate for the blob being tracked
BLOB_TRACKING_LOST_IDS - contains an array of ids that stopped tracking. I.e. a blob being tracked has suddenly disappeared.
BLOB_TRACKING_LOST_%ID% - for each lost id a variable hold an array of past COGs is created.