|
Suggestions for eye tracking algorithms
Zachary Lim from Malaysia [12 posts] |
12 year
|
Hi guys. I am working on a project to control a robot with my eyes. It is done with LEGO NXT. So far I have tried playing around with the modules and managed to get a rough program to detect the darkness of the eyeball. However it is not very good and does not manage detect the whole eyeball, usually partially, and sometimes the dark detected merges with the outlines of the eye.
Does anyone have a good suggestion for an eye tracking algorithm which can detect the eyeballs fully and smoothly such that the eyeballs can be separated from merging with the outline?
ps. I would also like to know how to detect eye blinks. Thanks!
|
|
|
|
Anonymous from United States [28 posts] |
12 year
|
Zachary,
Can you post an image of what you're working with from the camera?
I would think that a close up view of your eyes would need to be found, and you should be able to break it up using something like the Canny Line module, and the the Circle module to find the pupils.
You may need to create a reference for the 'center' point of each eye, and then reference the COG of the circles you find after you have calibrated.
You can use the VbScript module to check where the eye currently is compared to the calibration that you took originally, and then use these values to determine what you need to happen.
If the circle disappears, I would assume this could be a 'blink' trigger.
|
|
|
|
Zachary Lim from Malaysia [12 posts] |
12 year
|
Hi Necrolyte,
I have inserted a picture of what i'm working with. It is a close-up view of the eye. This will be the image projected when I mount the webcam onto my head, so that the viewpoint of the camera will remain the same regardless of head movement.
Will the circles probe be able to detect a partial circle? I fear that sometimes if the eyelid is covering a small tip of the eyeball, the probe won't detect it as a circle.
Also, I see that many RR users tend to like VBscript. Is there a difference compared to CScript or PythonScript? I would like to use C if possible because I am most familiar with it.
I am having problems sending messages over the LEGO NXT module via USB. It seems that there is no problem sending over the FPS variable as my NXT can read it. The problem comes with user-defined variables. For instance, I set a variable "message" to have the value 2, but tried to send it over. The NXT did not read. I have checked RR and made sure that the type of message sent was of number type, and I have made sure the receiving program in the NXT was receiving integer type messages.
What seems to be the problem here?
|
|
|
|
John Wood from United Kingdom [67 posts] |
12 year
|
Hi,
Had a simlar problem sending data from RR on the serial lead to communicate with the PIC and then relised that the PIC required me to process the Ascii value.
As you mentioned you sent dec 2 then you may have to process Ascii 50..
John
|
|
|
|
Anonymous from United States [28 posts] |
12 year
|
Zachary,
Here's a quick .robo file that I made up, you may need to delete the LoadImage and just allow it to hook into your camera. You'll see how the Canny line detector will show a partial circle around the center of the eye, with this, using the Circle module will overlay an entire circle on that, then blob track and find the Center of Gravity. As you move your eye left or right, and even up and down, the CoG should change.
As far as serial communications are going, I second what John said, make sure you are reading Ascii values on the PIC, as well as possibly sending a carriage return or line feed behind it.
Hope this helps!
 program.robo
|
|
|
|
Anonymous |
12 year
|
|
|
|
|
Zachary Lim from Malaysia [12 posts] |
12 year
|
Necolyte, thanks for the robo file. I have managed to get the canny line detector working with my own program. However the effectiveness of the settings would differ from place to place and need tweaking with the parameters. For example, lighting would be an issue.
John, the NXT is not run by PIC controller, and therefore I am not using the serial module for it. There is a special module for LEGO NXT in Roborealm. However I am trying to get it to run with NXC. There is a problem when I send the values of variables as they do not show up in my NXT. When I tried to run the LEGO NXT sample program on the module's page, it doesn't show up on-screen.
Is there any solution for this problem?
|
|
|
|
John Wood from United Kingdom [67 posts] |
12 year
|
Hi Zachary
Sorry I can not help as I am not familiar with either but I am sure there will be an answer from someone more acquainted with the unit.
I am on with boat tracking and counting using a PIC driving the servos and the stepper motors for FOCUS/ZOOM on the cam for the different programmed routines.
RR is one great program.
Cheers and good luck
John
|
|
|
|
Zachary Lim from Malaysia [12 posts] |
12 year
|
Thanks John, good luck to you too on your project.
STeven, I have managed to get RR sending messages in the form of numbers to my NXT. The receiver program is written with NXC. However the current problem is, when the value is 256, it displays "056". I think this problem is rather similar to one of the questions asked in this thread (second last post):
http://www.roborealm.com/forum/index.php?thread_id=4123
Unfortunately, the poster has not received a reply with a solution. I am refering to many forum threads but so far have not found any solution yet.
Regards,
Zachary.
|
|
|
|
Zachary Lim from Malaysia [12 posts] |
12 year
|
I have found the reason for this:
When running a pipeline analyzing a fixed picture of a red ball(which the COG_X and COG_Y of the ball never changes), I looked closely at the NXT screen and noticed the 0 "prefix" was constantly blinking. I rewrote my program so that the program would break out of the loop as soon as it receives its first message. The values displayed were exactly the same values that I was trying to send over. The problem with 0 appearing in front would most likely be that the refresh rate while sending messages is extremely fast. It keeps juggling between the value 0 and the value of the number I sent over.
Is there a way to slow down the refresh rate while sending messages so that I can see the number before it gets overwritten?
|
|
|
|
Anonymous |
12 year
|
Zach,
I don't think it is based on the send rate, that should not be a problem. The issue is in the format being used. Most likely that 0 is being read as the end of a string and incorrectly being displayed. Have you played with the types setting in the NXT RR GUI window?
Also, can you include a sample NXC app where you are seeing this behavior? Most likely the module is not quite compatible with that since it was only tested with the Lego Gblocks.
STeven.
|
|
|
|
Zachary Lim from Malaysia [12 posts] |
12 year
|
STeven,
Here is my NXC source code for a receiver program that displays the value of the value sent over via the LEGO NXT module:
#define INBOX 2
task main()
{
int i;
while(true) {
ReceiveRemoteNumber(INBOX, true, i);
NumOut(0, LCD_LINE1, i);
}
}
It seems that no matter what class I specify the message type to be, there seems to be a 0 prefix in front.
I have written another program which tests whether the value of data received is the actual value or if the value was tampered with by the 0 in front. Turns out that it was actually receiving the correct value all along. Here is the source code, I set it to receive message only once when the value wasn't 0 and to display it out for me to see.
task main() {
int i;
string iStr;
while(i == 0) {
ReceiveRemoteNumber(0, true, i);
}
iStr = NumToStr(i);
while(true) {
TextOut(0, LCD_LINE1, iStr);
NumOut(0, LCD_LINE2, i);
}
}
|
|
|
|
Zachary Lim from Malaysia [12 posts] |
12 year
|
|
Also, please note that occasionally when I try to run RR it crashes. However it is okay after restarting my computer.
|
|
|
|
Anonymous |
12 year
|
Zach,
I'm not very familiar with NXC but it seems that the receive is just not waiting for a non-zero number and returning immediately. See if there is a function similar to ReceiveRemoteNumber will wait for a number or just use the code that you have ... assuming you never need to send a 0 it should work.
STeven.
|
|
This Thread