|
Eye Tracking
Kamlesh Thakur from India [32 posts] |
13 year
|
|
I have a Trossen Robotics 'RoboTurret Vision Tracking' kit with me. The Kit is successfully interfaced with RoboRealm and able to track the color objects. I would like to know is any API available in RoboRealm to track the eye image within a moving video captured by the turret vision system?
|
|
|
|
Anonymous |
13 year
|
Kamlesh,
That depends on how close the camera is to the eye. When really close tracking a circle for the pupil works really well. Further away you have to track the face and from that deduce where the eye is ... but you cannot tell where a person is looking as it is just too far away.
Given your turret hardware and the fact that the camera probably can't focus on a very close eye I don't think that eye tracking is possible with RR.
STeven.
|
|
|
|
Kamlesh Thakur from India [32 posts] |
13 year
|
Hi STeven,
Thanks for the comments.
For eye tracking application, it is assumed that the person will be sitting in front of the Camera within a distance of 1 meter. Also, the person is expected to look at the camera with front facial position. So the Camera will always have an eye image with front face.
Secondly, the eye tracking is to be done with front facial position within a confined area of 1 meter X 1 meter with a depth of 1 meter.
The Camera mounted on the Turret is LifeCam VX-800 with a resolution of 640X480. I am planning to use a high resolution camera (having same weight) with an optical zoom.
Under these conditions will it be possible to track the eyes using RR?
|
|
|
|
Anonymous |
13 year
|
Kamlesh,
I assume you already have the camera and have tested the setup? Can you snap a couple pictures of what you would consider normal usage of the system and we can send you our opinion and a test robofile that may do the trick.
You may also take an image at the expected zoom and then one at a maximum zoom (with the eye/head in the image center).
That's probably the best way to answer your question.
Thanks,
STeven.
|
|
|
|
Kamlesh Thakur from India [32 posts] |
13 year
|
Steven,
Sending you a snap of sample picture. Usually, the zoom in this image would be a typical operating condition to track an eye (either right or left eye). Subsequently, I'll send you an image sequence (probably a video file) considering all possible zoom conditions.
In the meantime, if you can provide me some guidelines to build an eye tracking algorithm using Roborealm then it would be a great help to begin with.
|
|
|
|
Anonymous |
13 year
|
|
What about the AVM recognition program. That would recognize the eye just fine.
|
|
|
|
Kamlesh Thakur from India [32 posts] |
13 year
|
Hi,
Thanks for a suggestion to try AVM for an eye tracking!
I tried using AVM demo with the user interface (GUI) i.e. Recognition.exe installation.Upon selecting an eye object from a turret camera captured image, the demo software is tracking an eye object.
I have following queries regarding AVM Plugin:
1. I have a licensed copy (purchased along with Robo Turret vision tracking starter kit) of Roborealm. Is AVM Plugin included in this version. If not then how can I get this plugin?
2.Will it be possible to control the Robo Turret PTU (Pan-Tilt Unit) for camera positioning w.r.t. eye object tracking using AVM navigator? If yes then what could be the steps to command the PTU motors for Camera Positioning using AVM Navigator?
3. Is it possible to use AVM plugin (Navigator) directly within Roborealm for eye tracking and Turret PTU camera positioning?
4. I have OpenCV 1.0 & OpenCV 2.0 Installation on my PC. I tried to build the Navigator package (Navigator_src_en.zip) in MS Visual C ++ 2008. However, I am getting an error "cxcored.dll file not found". There is no dll named cxcored.dll in OpenCV, instead it could be cxcore.dll. Please send your opinion in this regard.
5. Is it possible to build Navigator package solution (Navigator_src_en.zip) in MS Visual C ++ 2008 using OpenCV2.0 instead of OpenCV1.0. If yes then what should be the include and dll file paths to be added in build options?
|
|
|
|
EDV [328 posts] |
13 year
|
>> 3. Is it possible to use AVM plugin (Navigator) directly within Roborealm for eye tracking and Turret PTU camera positioning?
You can try to use variables described below for this purpose:
NV_OBJECTS_TOTAL - total number of recognized objects
NV_ARR_OBJ_IDX - index (identifier) of object
NV_ARR_OBJ_RECT_X - left-top corner X coordinate of recognized object
NV_ARR_OBJ_RECT_Y - left-top corner Y coordinate of recognized object
NV_ARR_OBJ_RECT_W - width of recognized object
NV_ARR_OBJ_RECT_H - height of recognized object
NV_ARR_OBJ_SIM - similarity rate (0...1) of recognized object
See "Using of AVM plugin in RoboRealm" thread for more details:
http://forums.trossenrobotics.com/showthread.php?t=4764
>> 5. Is it possible to build Navigator package solution (Navigator_src_en.zip) in MS Visual C ++ 2008 using OpenCV2.0 instead of OpenCV1.0. If yes then what should be the include and dll file paths to be added in build options?
You should use OpenCV1.0 for compilation of Navigator package solution.
See here for more details:
http://forums.trossenrobotics.com/showthread.php?t=3510&page=2
|
|
|
|
Anonymous |
13 year
|
>>1. No, the AVM module is not part of the Robo Turret PTU but can be purchased separately on this website for $9.00.
>>2. Not directly. You would need an application that can interface with the AVM system and then send commands in the right format to the Turret. There are examples on how to do this for both systems but you will have to check the respective sites on how to do that. Naturally, RR makes this integration seamless.
STeven.
|
|
|
|
Anonymous |
13 year
|
Given the above image, you can also track the eyeballs directly. See the robofile below. May be useful if you want to know the general direction of where someone is looking or to detect blinks.
This, however, should be tested on a live image to see how stable this solution is. Moving the eyeballs to the extreme left or right may cause the circle detection to fail.
STeven.
 program.robo
|
|
|
|
Kamlesh Thakur from India [32 posts] |
13 year
|
Hi,
Thanks for your comments.
Regarding queries in my last post, there is one correction in query no.4.
Upon compiling the Navigator package (Navigator_src_en.zip) in MS Visual C ++ 2008, I am getting Link error "cxcored.lib - file not found". There is no file named cxcored.lib in install directory. Can you suggest how to resolve this linker error?
|
|
|
|
EDV [328 posts] |
13 year
|
|
You should compile OpenCV1.0 in debug mode. Open opencv\_make\opencv.dsw (in case of MSVC 6.0 or MSVC.NET 2008, in the latter case they will be converted to MSVC.NET 2008 project files). When compilation will be done the file "cxcored.lib" will appear in "opencv\lib" folder.
|
|
|
|
Kamlesh Thakur from India [32 posts] |
13 year
|
Thanks for your comments.
I purchased the AVM Module plugin from Roborealm website last Saturday.
Now, I am able to use the Navigator Plugin to track the eye (front face position) under Navigator Mode. As a next steps, I plan to drive the Turret PTU motors using the Control Variables from AVM Navigator.
As suggested by EDV, I thought of using the variables described below for this purpose:
NV_OBJECTS_TOTAL - total number of recognized objects
NV_ARR_OBJ_IDX - index (identifier) of object
NV_ARR_OBJ_RECT_X - left-top corner X coordinate of recognized object
NV_ARR_OBJ_RECT_Y - left-top corner Y coordinate of recognized object
NV_ARR_OBJ_RECT_W - width of recognized object
NV_ARR_OBJ_RECT_H - height of recognized object
NV_ARR_OBJ_SIM - similarity rate (0...1) of recognized object
However, I am not sure about generating a position command (0 to 255) for the Pan and Tilt mortor positioning using these variables. Is any sample program.robo file available so as to understand the position command generation using the above variables to begin with?
|
|
|
|
EDV [328 posts] |
13 year
|
I think you should use VBScript program like this:
' Get turret control variables
turret_v = GetVariable("TURRET_V_CONTROL")
turret_h = GetVariable("TURRET_H_CONTROL")
nvObjectsTotal = GetVariable("NV_OBJECTS_TOTAL")
if nvObjectsTotal>0 then ' If any object was found
' Get image size
img_w = GetVariable("IMAGE_WIDTH")
img_h = GetVariable("IMAGE_HEIGHT")
' Get array variables of recognized objects
nvArrObjRectX = GetArrayVariable("NV_ARR_OBJ_RECT_X")
nvArrObjRectY = GetArrayVariable("NV_ARR_OBJ_RECT_Y")
nvArrObjRectW = GetArrayVariable("NV_ARR_OBJ_RECT_W")
nvArrObjRectH = GetArrayVariable("NV_ARR_OBJ_RECT_H")
' Get center coordinates of first object from array
obj_x = nvArrObjRectX(0) + nvArrObjRectW(0)/2
obj_y = nvArrObjRectY(0) - nvArrObjRectH(0)/2
' Get difference between object and screen centers
dX = img_w/2 - obj_x
dY = img_h/2 - obj_y
if dX > 40 and turret_h > -128 then
' The object is at left side
turret_h = turret_h - 1
end if
if dX < -40 and turret_h < 127 then
' The object is at right side
turret_h = turret_h + 1
end if
if dY > 40 and turret_v > -128 then
' The object is at the bottom
turret_v = turret_v - 1
end if
if dY < -40 and turret_v < 127 then
' The object is at the top
turret_v = turret_v + 1
end if
end if
turret_v128 = turret_v + 128
turret_h128 = turret_h + 128
' Set turret control variables
SetVariable "TURRET_V_CONTROL", turret_v
SetVariable "TURRET_H_CONTROL", turret_h
SetVariable "TURRET_V_128", turret_v128
SetVariable "TURRET_H_128", turret_h128
=============================
TURRET_H_128 - horizontal turret control
TURRET_V_128 - vertical turret control
See attached program below.
program.robo
|
|
|
|
Kamlesh Thakur from India [32 posts] |
13 year
|
Hi,
I used the use VBScript program and now am able to see that turret PTU is able to track an eye image (using AVM Navigator mode).
Thanks once again for the solution.
Next step in my project work is to capture the tracked eye image especially for an IRIS recognition database (minimum 150 pixel resolution for IRIS image) under different lighting conditions. I'll try this and keep you all posted about my findings and queries, if any.
|
|
This Thread