loading

GETTING STARTED


Introduction

The RoboRealm application contains all the tools you need for processing video images for use in robotic applications. It is easy enough for the casual user who wants to experiment with video processing, yet powerful enough for the serious robotic scientist who needs to integrate a vision system into their project.

The heart of the video processor is the image processing pipeline that allows multiple modules (aka image processing filters) to be added in succession to generate the desired results. Combined with the programming logic and control interfaces one can create a visual processing loop that moves a robot according to what it sees. To get an idea on how the application works please watch our Introduction to RoboRealm video.

To try RoboRealm with your own camera and see what it can do  Click here to load a slideshow configuration that will cycle through various filters.

Main RoboRealm Interface

The main interface shows the live video, available functions and the processing pipeline. Adding modules into the pipeline will immediately begin processing of the image. You can reorder the modules in the pipeline by using the up and down yellow arrow buttons.

Theory

The main purpose of RoboRealm is to translate what a camera sees into meaningful numbers in order to cause a reaction to what the image contains. This requires the processing of an image into a few or a single number that is meaningful in the current context of whatever project you have in mind. As each project has vastly different requirements with different attributes that need to be detected, RoboRealm is written in a flexible approach where the basic tools are provided to you in the form of modules in order to be combined into a relevant combination. For example, if your task is to track a red ball then the color red can be picked up by a color filter module as the first step. But if you wanted to track any type of ball then you would not use the red color as an attribute but instead a round shape detector. The reason you would ever want to use a color tracker instead of just using a shape tracker is that the color tracker may be more reliable and repeatable in some environments.

For example, if a red circle is only partially seen a color tracker would easily pick up half of the red ball but a circle detector might not due to the incomplete circle. The choice of which modules to use comes with experience and is very dependant on the environment being sampled. For this reason we regularly will ask for images in the forum as apposed to a description as images will reveal many attributes such as image quality, lighting, color, etc. in the environment.

Original ImageRed IdentifiedCircle Identified

The best way to accomplish your processing task is to learn from the tutorials and post an example image in the forum where we and others can help provide meaningful modules that might accomplish the task.

Installation Instructions

  1. You should have received an email containing a download link to the RoboRealm.zip application after completing the email registration process. Run the downloaded RoboRealm.exe file. This will launch the installation program.

  2. Once installed Double-click on RoboRealm.exe in the installed folder or look for RoboRealm from you Window Start button, Programs menu, RoboRealm menu. Once you have done this you should see the main interface. If the application fails to execute you can try executing RoboRealm.exe while holding down the CTRL key. Holding down the CTRL key tells RoboRealm NOT to try to connect to any cameras or run any existing configurations which allows RoboRealm to start in a safe default mode.

  3. If your webcam image does not appear in the main image area click on the Options Button->Video tab and select your camera in the dropdown selection. This should cause RoboRealm to start previewing the image in the main RoboRealm interface. Also be sure that the Camera button is pressed in order to activate the camera. If it is not, press the camera button and then use the Options button->Video tab to select the appropriate camera.

Hello World

Once RoboRealm is running and you have connected to and can see your webcam image you can now start inserting in modules that will affect the currently viewed image. You do this by using either the Contents, Index or Search tab in the upper left corner of the main interface. They all display the same modules but from a different perspective. Selecting a module and then pressing insert will insert that module into the processing pipeline. (You can also double click on a module to automatically insert it into the pipeline). Once a module is added into the pipeline it will start to alter the image in some way to extract or highlight certain features (color, shape, texture, etc.) By adding in more modules additional attributes can be filtered from the image more precisely in order to identify a particular object or image quality.

Note that you can also use Firewire Cameras, Internet Cameras, Video files (Media_Reader) or just static images (you can drag static images into RoboRealm) as apposed to using a webcam.

The processing of images using the inserted modules happens as quickly as possible and is limited by the speed of the camera in use and the CPU requirements of the inserted modules. The sequence of processing an image starts with an image capture (for example a webcam) and then proceeds by passing the image to each of the inserted modules one at a time in the sequence they appear in the processing pipeline. At the completion of all modules the resulting image is displayed (the exception to this is if a module is selected in which case the image up to and including that module is displayed to aid in debugging the pipeline). Once this image is displayed the processing loop begins again with the next new image. Note that if the camera does not provide a new image to RoboRealm then RoboRealm will STOP processing and wait for a new image as reprocessing an already processed image will not add any new information into the system. Thus the pipeline with no inserted modules will run at the frame rate of the webcamera. Adding a module will slow the looping speed a little depending on the processing requirements of the module. Different modules will have different CPU requirements as each module is different in terms of complexity and usage.

The pipeline will execute only when new information is provided. This includes a new image from a webcam or when a change is made to modules on a static image. Thus the frames per second will be limited by the rate of the attached camera.

To ensure that changes in the UI (User Interface) do not update while processing an image the UI will only update inbetween pipeline iterations. This means if you have a very slow pipeline the interface will not appear to update until the pipeline has completed. This can have the appearance of slowing down or freezing the application on slow executing pipelines.

You can also pay attention to the gray numbers on the right side of the pipeline. Those numbers indicate the number of milliseconds that each module takes. Those modules with high numbers are the cause of the pipeline slowdown. Often these modules are ones that are attempting to connect to external devices and have not yet established a connection. Other modules such as the Stereo module will require significant processing resources depending on the size of the image being processed.

The pipeline will iterate/execute twice. Once to process the image and a second time to create any annotations ontop of that processed data. This ensures that annotations written ontop of the image do NOT affect processing of successive modules. Because of this some modules may cause the annotations to not draw correctly.

For example, adding in the RGB Filter module to the above image will cause the image to appear as


Fatal error: Uncaught ArgumentCountError: Too few arguments to function robofile(), 1 passed in /homepages/10/d91033323/htdocs/RoboRealm/help/Getting Started.php on line 128 and exactly 2 expected in /homepages/10/d91033323/htdocs/RoboRealm/header.php:5 Stack trace: #0 /homepages/10/d91033323/htdocs/RoboRealm/help/Getting Started.php(128): robofile('help/main_red') #1 {main} thrown in /homepages/10/d91033323/htdocs/RoboRealm/header.php on line 5