Visual Targeting #1

This tutorial assumes that you have acquired an image of some form and need to process it to determine the location of the upper targets. This information would be used to determine the rotation of the robot to center it for shooting.

Let's assume the following images are to be processed.

#1 Original #2 Original #3 Original

The first thing we want to process the image for the reto-reflective tape. By separating the strips from the rest of the image we can proceed to isolate and extract just that object. First we use the Adaptive Threshold module to convert it into a black and white image.

#1 Threshold #2 Threshold #3 Threshold

We can see that using that module extracts the curved strips nicely. The next step is to identify the specific ratio of the thicker strip against thinner one. This can be done using the Line Pattern module with the pattern specifying how the strips appear to a single vertical scan line.

#1 Pattern #2 Pattern #3 Pattern

The result of the Line Pattern module is an X and Y coordinate of the center of all detected patterns. We can now use this to determine the needed results.

#1 Arrow #2 Arrow #3 Arrow

An additional green arrow has been added to show the offset from center of screen to the X,Y coordinate of the found target. The coordinates can be used to understand which direction to move the robot. This iterative feedback is more of an approach than a specific calculation. We can use the visual feedback to constantly tell us how to change our current state in order to achieve a better position. This method requires that the camera sensor be quickly processed and feed new information to actuators that will then update the robot position which will then again be processed by the camera in near real time. This iterative approach does not require any precise calculations that may or may not change during the competition due to worn equipment or lower battery levels.

The actual values fed to the actuator can be very coarse (neutral, near right, right, extreme right, etc.) since over time the values are integrated based on the feedback of the camera and reaction time of the robot.

To determine distance we can use the size of the detected pattern relative to the source pattern. The detected size of the pattern in image #1 is 1000 or 100.0%. This makes sense since that is the image the source pattern was read from (i.e. 10 6 9 10 9). For #2 and #3 we have different sizes that correspond to how much further or closer the target is based on what we used as the source pattern. If you have a desired distance in order to launch your ball you can use that distance as the source pattern and move the robot such that the pattern size is as close to 1000 as you can get it. This will ensure that the source pattern size and the detected pattern size are very close which implies the same distance.

For those that have the ability to target at different distances, one can use the source pattern 10 17 31 32 10 which is the target size in inches * 4. Note the 10's are just borders so we ignore them when coming up with this pattern. The inner 3 numbers are therefore 17 31 32 divided by 8 are approximately 2 4 4 which is the specification of the targets, i.e. lowest strip is 2" thick separated from the upper 4" thick strip by a 4" gap.

From this knowledge and the known field of view of the camera we can determine a distance from the camera to the target using:

list = GetArrayVariable("LINE_PATTERN")
if isArray(list) then
  if ubound(list) > 0 then

		' based on a line pattern in 8 x actual inches
		' of 10 17 31 32 10 = 100

    targetPixelHeight = (list(3)/10)
    targetSamples = 0

    ' calibrated for an Axis camera
    imageHeight = GetVariable("IMAGE_HEIGHT")
    cameraFieldOfView = 47.5
    targetHeight = 100.0

		' determine distance in 8 x inches
    totalDistance = (((targetHeight*imageHeight)/targetPixelHeight)/2)/_

		' convert to ft (12 inch per ft * 8 inch multiplier) = 96
		totalDistance = CInt((totalDistance*100)/96)/100

		' save it for use in other modules
    SetVariable "Distance", totalDistance

  end if
end if

To try this out yourself:

  1. Download RoboRealm
  2. Install and Run RoboRealm
  3. Load and Run the following  robofile which should produce the above results.

If you have any problems with your images and can't figure things out, let us know and post it in the forum.