|
Framerate issues while object tracking
Tyberius from United States [8 posts] |
16 year
|
So... I'm running a pan/tilt camera setup with 2 servos controlled by an SSC-32. I'm using RGB_Filter, Mean Filter, COG, SSC-32 Module and of course the VBscript module. My code seems to work fine, however the pan/tilt tracking is suffering from jerky movements, and I'm thinking this is due to the very poor framerate I'm getting while processing.
Few factors involved: I'm using a pico-itx 1ghz C7 via mainboard/cpu. I don't think this is the bottleneck as its only using about 20% of the cpu when running the process. However, I am using a fairly cheap webcam, its rated 640x480 @ 30fps, but I'm running it in at 320x240 in Roborealm... thing is, even without any pipelines loaded it's only getting about 7-8fps on straight video. When I apply the modules to run the object tracking, its dipping down to 0.9-1.1 fps, making my tracking pretty jerky.
My question is, is this a limitation of my webcams? Or is it a limitation of the software? Just wanted to know if this performance is normal before I go out and spend more money on more expensive cameras.
Thanks a ton!
|
|
|
|
Ricardo Sampaio from Portugal [32 posts] |
16 year
|
I have the same board (1GB DDR2 677mhz) and have suffer the same problem, but not that low FPS 10 -17.
Most likely the problem is 80% the camera and 20% the software. But if you have a 0.9-1.1fps then u have some problem in your SO because i tried very complex modules and never happen to drop to that values. Have you test the same camera in a better PC?
|
|
|
|
Tyberius from United States [8 posts] |
16 year
|
I think I've figured it out actually. There appears to be a performance problem with the SSC-32 module. When I disable that particular module, my FPS goes right back to normal, as soon as I enable it to send commands to my SSC-32, my FPS dips to about 1fps.
Anyone else had similar issues with the SSC-32 module? I know someone on the lynxmotion boards had mentioned jerky movement when trying to do the same thing.
|
|
|
|
Tyberius from United States [1 posts] |
16 year
|
Still running into the same issues... I've played around with this quite a bit (different PCs, different cameras) and still am encountering the same issue.
I'm almost 100% convinced theres something wrong with the SSC32 module, as soon as I enable it my FPS drops to about 1 FPS. Anyway, losing hope here =/
I'm attaching the .robo I'm using... if anyone else has had success doing pan/tilt tracking with an SSC32, I'd appreciate you sharing your code and/or .robo =)
Any help would rock, thanks!
 program.robo
|
|
|
|
Anonymous |
16 year
|
Tyberius,
It sounds like your camera is probably fine ... its the communication to the SSC that is having an issue. We noticed a similar behavior when the servos would draw too much current and cause the board to suddenly reset and relax the servos creating a very jerky movement.
Try running the system without the actual servos attached and see if that makes a difference.
The problem is that the SSC module is probably losing connection to the SSC board and trying to reconnect ASAP. This reconnection is what is slowing the system down ... you'll note that your CPU is probably NOT taxed which would indicate that there is more speed possible.
The slow fps would not create jerky movements unless your object that you are tracking is darting across the screen ... so we're guessing that it is something to do with the com with the board.
You may also try a slower baud rate and/or a different serial cable to try to stabilize the communication.
Let us know how it goes.
STeven.
|
|
|
|
Anonymous |
16 year
|
How can you change the baud rate when it requires 115.2k for PC communicatrions?
|
|
|
|
SN96 from United States [9 posts] |
16 year
|
I have tried Tyberius script and I can't get it to work with the SSC-32 beyond 1 fps.
If I load up one of my .robo files that uses only one servo for panning, it runs at 14.7 fps.
|
|
|
|
Anonymous |
16 year
|
Yes, I've already tried different baud rates... and my SSC-32 running its own logic power supply so that it wont lose connectivity when the servos power up (even though this shouldnt be an issue, it works fine when Im moving all 15 servos on my robot without losing power on batteries).
It appears to be an issue with my code, its somehow lagging out Roborealm, as I am able to get SN96's code to work... I'm not sure why my code is doing that. Could you check it out (posted above) Steven? Thanks!
|
|
|
|
SN96 from United States [9 posts] |
16 year
|
OK, we have narrowed it down to a problem with RoboRealm. Either it's a bug with the Script module OR the SSC-32 Module. Using the script bellow, the following was observed:
Both the x and the y servo channels selected in the SSC-32 module will yield 1 fps.
If ONLY the x channel is enabled in the SSC-32 module, frame rates are at 14.7 fps.
If ONLY the y channel is selected, the frame rates are at 14.7 fps.
If BOTH channels are selected in the SSC-32 module, the frame rates bog down to 1 fps.
Each channel by them selves works flawlessly, but as soon as you try to use both channels together, there is a serious problem. The script works fine since each channel was tested individually with no problems.
Here is the code:
' first check that something moving is in view
if GetVariable("COG_BOX_SIZE") > 10 then
' get the current screen width so we can calibrate movement
width = GetVariable("IMAGE_WIDTH")
height = GetVariable("IMAGE_HEIGHT")
end if
' get the current center of movement
x = GetVariable("COG_X")
y = GetVariable("COG_Y")
' scale the image space into the servo space
x = CInt((x/width)*2000)
y = CInt((y/height)*2000)
'---------------------
' Select variable "servo_position" for use in the servo controller for normal movent.
' Open SSC-32 Module and from the drop down list, select "servo_position" for normal
' operation.
SetVariable "servo_position_x", x
SetVariable "servo_position_y", y
'----------------------
|
|
|
|
Anonymous |
16 year
|
Thanks for the great debugging ... that really helps. It does seem that something is going wrong with the SSC module. We'll review this entire thread and try out some experiments to see if we can replicate the issue. We should be able to get back to y'all sometime tomorrow/late this evening.
If anyone else can verify the above with any additional information let us know!
STeven.
|
|
|
|
Anonymous |
16 year
|
Thanks for the response Steven. SN96 has been lending me a hand on this, so I don't have a ton to add as he pretty much summed up our problem, however I will provide the original code I was using as another troubleshooting variable to verify against:
**************************************************
'320x240 resolution camera
' initialize starting servo values
pan = GetVariable("PAN_SERVO")
tilt = GetVariable("TILT_SERVO")
' get the size (width or height) of the current bounding box
size = GetVariable("COG_BOX_SIZE")
' if it is equal to "" then no object was detected
if size <> "" then
' get the horizontal center of gravity
cogX = GetVariable("COG_X")
' pan left
if cogX < 140 then
pan = pan - 50
end if
' pan right
if cogX > 180 then
pan = pan + 50
end if
' get the vertical center of gravity
cogY = GetVariable("COG_Y")
' tilt down
if cogY < 100 then
tilt = tilt - 50
' tilt up
end if
if cogY > 140 then
tilt = tilt + 50
end if
SetVariable "PAN_SERVO", pan
SetVariable "TILT_SERVO", tilt
end if
**************************************************
This code does essentially the same thing that the code SN96 does (slightly different manner though) but ends up having the same exact issue with FPS.
|
|
|
|
SN96 from United States [9 posts] |
16 year
|
Thanks Steve!
I hate to do this but if you are going to take a look at the SSC-32 Module, I might as well provide one more bug.
Its a small glitch, but when you first open the SSC-32 module after strating up the RR app, the servo channel lables are incorrect. To correct it, you have to click the Bank #1 tab then reselect the bank #2 tab and all is good. I have attached two screens of this and I hope it shows up.
[1] [2]
|
|
|
|
Anonymous |
16 year
|
Ok, folks, we found a couple issues and have corrected all that we could duplicate. Please download v 1.8.6.3 for the fixes.
1. There was an issue with the feedback of the SSC. We had the command incorrectly specified for multiple servos and the serial connection was being closed and reopened after the returned error was detected. This caused the slowdown you all saw.
2. We added in a checkbox to NOT receive feedback at all .. this is nice if you don't care about the feedback and just want more speed.
3. We also added in a enable and disable all button to quickly config all the channels.
4. There was also an error detected when selecting the variables from the dropdown that some experienced. This has also been corrected in this and all other control modules.
5. The attached robofiles above were almost right except that you will need to limit the extents of the variables so that when the object disappears and reappears the values do not have to "catch up". Attached is the one of the corrected robofiles.
SN96, do you still see the bank startup issue on this new version?
Enjoy!
STeven.
program.robo
|
|
|
|
Anonymous |
16 year
|
And here's the second robofile.
They are almost the same thing but with a slightly different flavor.
Note that we tested this using Lynxmotion's Pan/Tilt kit with two Hitech servos attached to channel 1 and 2.
Let us know how it goes!
STeven.
program.robo
|
|
|
|
SN96 from United States [9 posts] |
16 year
|
Thanks Steve!!
Both .robo files work and the second flavor is nice and smooth.
All fixed on my end, Andrew still needs to try it on his end to see if it will work.
The only problem I could see that still remains is the Bank # tabs still defaults to Tab#2 with the same numbering. :( But the most important stuff was fixed.
|
|
|
|
Anonymous |
16 year
|
Oh, and we also added in a enable/disable for the analog/digital inputs that was being requested each time too. Leaving this disabled will also reduce the time/bandwidth requirements which should keep things fast.
Despite all of that, we still get about 27-30fps with everything turned on with the servo feedback fix ... so the enable/disable should just be a bonus.
STeven.
|
|
|
|
SN96 from United States [9 posts] |
16 year
|
Great job with this release! The new options make it 200x better. My biped has a lot of servos and these options make it SO much easier to configure the various channels.
The response time is instant! Watching the separate servos react put a big smile on my face! :D
|
|
|
|
Tyberius from United States [8 posts] |
16 year
|
Thanks a ton for all your hard work Steven, can't tell you how much I appreciate it. It's funny that an opensource software platform has far better support then any retail software I've ever encountered :P (I work in IT, and I'm jaded too)
I only had a few minutes to test things out... but I can already tell the issue appears to be resolved. GREAT WORK!
The ONLY thing I can give as feedback, is it seems Roborealm is still a bit weird with the SSC-32, I loaded up the program and was only getting 1fps... had to power cycle the SSC-32 with roborealm running to get the FPS back up to normal. Once I did that it was working fine, but it seems when things are initially loaded, something goes weird with the module until you turn off the ssc-32, and then turn it back on. I don't think its the power up disconnect as I have the ssc-32's logic voltage running on a separate power supply.
Regardless of that small glitch, things are working great. Thank you so much for your effort on this Steve, and thanks to SN96 (Mike) for his help with it as well. I'll post up a video of my robot running this new roborealm version once I get everything dialed in.
Steven- In case you're interested, I'm using roborealm for the following robotics project:
http://www.lynxmotion.net/viewtopic.php?t=3004
It's a Johnny 5 like robot with an onboard pico-itx PC running windows xp :D
Next step with roborealm is making a "follow the ball" program, similar to the project you have posted on this site.
|
|
|
|
Anonymous |
16 year
|
Thanks all for the feedback! Glad we could help out.
SN96, the bank issue is now also fixed and will be released in the next upload ... most likely the in the next couple of days.
Tyberius, WOW, fantastic robot. I think I have a new background desktop image! Nice work on the robot. Can't wait to see more videos. Good to see that Jim is also aware of what you're up to. They're great guys over there at Lynxmotion!
We'll look into the startup issue ... if you notice any other characteristic of that error or if anyone else experiences it please let us know!
Thanks all!
STeven.
|
|
|
|
Anonymous |
16 year
|
As far as the startup issue goes, here's what I know so far. SN96 mentioned it to me when I was first having FPS issues... however power cycling the ssc-32 didn't work for me at the time, as I was running two servos and that was addressed in this last version. So, the startup issue occurred before the patch, and still is occurring for both of us.
Basically, I startup roborealm with my ssc-32 already powered on, load up my tracking program, and my fps drops to 1. Previously, power cycling the ssc-32 did nothing for me, but since you have fixed that bug, if I power cycle the ssc-32, it will then function 100% normally at normal fps. SN96 was and still is running into this only using 1 servo as well as 2 now with the patch. I'll try to get more info on this today and test it, to see if I can find any other characteristics.
|
|
|
|
SN96 from United States [9 posts] |
16 year
|
I tried it on my end to see if the frame rates would drop to 1fps when loading up RR.
I powered on the SSC-32 and THEN loaded RoboRealm and it did not drop any frame rates.
If I leave the SSC-32 card turned off, close RR, and then reopen it, the Frame rates drop to 1fps. As soon as I power ON the SSC-32, the frame rates dive back up to the max. Now If I turn off the SSC-32 card a second time, the frame rates remain unchanged, it still functions at ~14.8 fps.
It seems to working ok on my end. I just wanted to provide my experience to help the debug process.
|
|
|
|
SN96 from United States [9 posts] |
16 year
|
Here's an interface I am working on. I wanted to try out the line tool and it can make nice overlays!
[Interface]
|
|
|
|
Tyberius from United States [8 posts] |
16 year
|
"I powered on the SSC-32 and THEN loaded RoboRealm and it did not drop any frame rates. "
See, that doesn't seem to work for me, I thought thats what wasnt working before for you as well?
If I have my SSC-32 on and then fire up Roborealm, I have to then power cycle the SSC-32 to get normal frames.
|
|
|
|
SN96 from United States [9 posts] |
16 year
|
My apologies for the confusion. I have tried so many things that I my self got confused as to what's going on, but going back and trying to duplicate your problem, where the SSC-32 is on, and then starting RR, it just doesn't cause my frame rates to drop.
Before the patch, it was always @ 1fps when the SSC-32 module was being used and would only "free up" when I deleted the SSC-32 module from the pipeline. Since the patch, I don't have any issues at all. I'm using an old Logitech web cam that has low resolution and I really don't know if that's what's contributing to your problems.
I do notice when I start RR first with the SSC-32 off, that the frame rates are 1fps which I would expect since the SSC-32 is not on.
Regardless of how things are working on my end, you have a different problem. I wish I could figure out why you have different results compared to mine. The camera should have no effect on RR and the SSC-32 card. I wonder if the serial ports are an issue between the camera and the SSC-32? I don't know, just a wild guess.
|
|
|
|
RNA LABS
Les Fortner from United States [15 posts] |
16 year
|
WoW Tyberius,
We checked out the vids, and the post, and we love your work!
I thought of something quick, when I was reading about you voice recognition, and speech, and I thought that I may throw out an idea. I don't know if you have ever checked out verbots, (www.verbots.com I think, but do not quote me) but you can get this full blown for I think 10 dollars or something. (it may need some stripping later), and download it, it may give you the "HUMAN LIKE" edge that you want for your J5... From what little bit that I have seen, from on of my associates, it looks like it could be quite versitile, and it seems to be quite intellegent, it can run other probrams or scripts, right away, so it may be worth further investigation...
Hang in there brother, your doing a great job!
|
|
|
|
Tyberius from United States [8 posts] |
16 year
|
Thanks for the link! That is pretty much what I've been looking for, most excellent.
Thanks for the kind words =)
|
|
|
|
RNA LABS
Les Fortner from United States [15 posts] |
16 year
|
Tyberius,
No Prob... Let me know how it works out, I would be greatly appreciative. It seems as though the interface is great, but much of it is not needed in an application such as we need, but the structure of the program is really rather great. After only a few moments with the software, I liked it, but I have not had time to see if I can use the SDK, to eliminate alot of the un-needed stuff associated with the interface, and My associates wonder if we would be able to access only certain types of verbose, based on specific input from an array of variables. I would love to hear anything back from you, in this area.
Thanks for your post, I look forward to great things!
Les
RNA LABS
Les6772@hotmail.com
|
|
|
|
Anonymous |
15 year
|
Sninctown,
Your issue does not appear to be related to this post as the SSC by Scott Edwards does not provide servo feedback ... perhaps you can post your robofile that you are using? We are unable to replicate this problem with the SSC and the slowdown might be due to something else in the pipeline.
Thanks,
STeven.
|
|
|
|
Dave from United States [84 posts] |
15 year
|
I've been following this thread hopen to find a solution to the frame rate problem. On my one computer here my frame rate keeps dropping from 10, 9, 8, 7, so on and so on after a period of time. The only way to get the fram rate back up is to either stop/restart RoboRealm else go to the "Video Format" under the "Video" tab.
I just find it rater odd being teo computers are running the same version of Roboream, and using the same cameras, yet after awhile the one starts to lose fps. but don't know why. Sometimes it stays at 10 fps a long time, even over night, while other times it might be only an hour before its starts to drop.
I know starting another program while RR is running will pull the fps down, but it will coime back up when the event starts or when its over. I'm not sur if when the fps drop if someone has connected pulling the fps down and it doesn't recover or what? But something is causing it to get pulled down and not recovering to its original 10 fps. It just slowly keeps dropping and seen it as low as 3 fps when I caught it. Never had this probelm on the other computer. Both cams and RR are set up the same on both computers. Any ideas?
Dave
|
|
|
|
SN96 from United States [9 posts] |
15 year
|
|
It's been a while since I messed with the SSC-32 and RR. I recall the issue related to servo feedback problems. Are your servos connected? Did you try turning OFF servo feedback?
|
|
|
|
Anonymous |
15 year
|
Dave,
Have you checked your CPU performance monitor when you get the slow FPS? Is the CPU really maxed? What does your memory look like?
Also, how many devices do you have connected to the USB bus on your computer?
Also, what kind of lighting do you have when you notice the slower FPS? Darker scenes will slow the FPS so be sure there is LOTS of light.
STeven.
|
|
This Thread
