RoboRealm Test Page

The RoboRealm Test page provides a way to test multiple static images in a pipeline while reporting any differences from previous tests. This procedure allows for an automated way of regression testing against a large number of images without having to manual ensure every image results.

Testing any combination of modules needs to happen over many images in order to determine if the solution is stable. The problem is that one can make changes to the pipeline which cause the solution to be LESS correct than the current solution. The Test interface allows you to make modifications to the pipeline and rerun all previous images through that new pipeline and verify that images that already have produced the right result continue to do so.

The process for this interface is to first select a folder that contains many images. Once specified these images are listed in the clickable interface on the right. You then specify what variable needs to be checked for each image. This variable would be generated by your current pipeline, i.e. maybe a variable that holds the amount of Red color in an image. The specific variable and value within would be dependant on your current pipeline. You then proceed through all the images starting with the first by pressing PASS or FAIL. These buttons are located at the bottom of the image list. Pressing PASS will record the variable results as being positive, FAIL as negative and SKIP as ignored. Once the entire list has been manually reviewed once, you can then press the Start Tests button to automatically reprocess the Result Images in the current pipeline while comparing the variable results to the previous recorded results.

For example, if you used the RGB_Filter module and selected the RGB_FILTER_RED_COUNT variable in the Test interface the PASS and FAIL buttons would record this red count as valid (PASS) or invalid (FAIL) amounts. If you then eliminated this module from the pipeline and reran all tests everything would appear as FAILED since the variable would no longer be created based on the image content in the pipeline. In this manner you can validate that changes to the pipeline either change or do not change previously desired results.

• Folder - Specifies the folder that contains all the images to test. This will also be the location of where RoboRealm will store a roborealm_test.dat file that contains the following configuration.
• Results - Once you specify the Folder, the Results list will immediately list all the images in that folder. By clicking on each image that will cause that image to be loaded into RoboRealm.
• Check Variables - Check if the test should verify a variables value each time a test is performed. Select that variable into the Check Variables list and specify what Tolerance value is allowed.
• Group Size - If the variable contains a list of values, the group size specifies how long that array/list should be and that all values should also be checked.
• Value Tolerance Percent - Specifies that if a variable changes as a percent of the correct value this will trigger a negative test.
• Value Tolerance Units - Specifies that if a variable changes +- the specified number that will trigger a negative test.
• Messages - Provides some feedback on variable differences between current and recorded.
• Total Tests - The number of images tested
• Passed - The number of correct tests as compared with previous values
• Percent Passed - Radio of passed tests relative to total tests.
• Failed - The number of failed tests
• Test Duration - Number of seconds the total tests took. This can be used as a measure of performance.

New Post

Related Forum Posts	Last post	Posts	Views
qfeqfeq qfeq...	3 year	0	679
Vb script to determine ball in region I need help to determine ball position in 4 quadrant, which is quadrant 1 to quadrant 4 by using the line as  i attach...	7 year	2	2180
Test Module (Sorry, this belongs to test module, but I cannot select it, therefore the post will probably not show up there. No posts there ...	8 year	2	2443