 |
navigation by computer vision
hemant from India [15 posts] |
8 year
|
am writing a simple vbscript for navigation of car by computer vision ie. image processing
i want to send serial commands forward , left and right .
but i cannot understant where to right the serial commands in the serial module and how !!
pls help !
|
|
|
|
Steven Gentner from United States [1446 posts] |
8 year
|
What serial protocol are you using? I.e. what commands (like 'f' for forward, 'b' for back) do you need to send over serial?
STeven.
|
|
|
|
hemant from India [15 posts] |
8 year
|
I am using 'l' for left , 'r' for right , 'f' for forward and 's' for stop.
we have used floor finder module to detect the road in vlc module and center of gravity module to find the correction required .
I am an amateur and have jus started the coding .
I have attached the corresponding robofile.
Pls suggest the obvious changes Steven .
thank you
 program.robo
|
|
|
|
hemant from India [15 posts] |
8 year
|
steven
please omit the attached robofile .
consider the one attached here.
thank you
program.robo
|
|
|
|
Steven Gentner from United States [1446 posts] |
8 year
|
Your robofile looks about right. You may need a newline after the move command in the serial module. So that would be more like
[move]<cr>
but that will now depend on what is receiving the signal on the other end of the serial cable. Do you have access to that device or are you following an existing protocol?
STeven.
|
|
 |
Arvind from India [6 posts] |
8 year
|
|
Hi Steven, I am partner of Hemant and we are working together. We have access to the device, its a robotic car n takes serial commands as input. But before calibrating that, I want u to suggest ways on making my image processing module more robust. Like I still have no idea on how to deal with turns and obstacle detection. Please guide us in this regard.
|
|
|
|
Steven Gentner from United States [1446 posts] |
8 year
|
Hemant,
That's certainly possible but you will have to record some video to test. Any vision project should generate many hours of video or thousands of images in order to get each situation represented in the testing of any solution. Do you have such videos of turning and obstacle avoidance? I.e. manually drive the car and record video, then see if the automated solution will make the same direction recommendations.
I would highly recommend *NOT* to just try on a couple images, connect up the car and start testing! You may find that it is very unpredictable and cause not only damage to the car but injury to yourselves.
STeven.
|
|
|
|
Arvind from India [6 posts] |
8 year
|
|
Hi Steven, Well I got your point regarding trial and error and we are working on that. I wanna know that can I use neural network or any learning algorithm or precisely speaking, do I have such option in RR.
|
|
|
|
Steven Gentner from United States [1446 posts] |
8 year
|
Arvind,
If you are looking for neural network based learning packages you may be better off with a platform that focuses on that particular method. While we do have some neural network like techniques at our disposal we have often found that using other methods in image detection/recognition/etc. are much superior in terms of speed than your basic ANN type structure.
I would suggest you research what the Darpa Grand Challenge systems were using. Vision is certainly a component but RoboRealm does not include many of the additional computation modules you would need in a project like what you are describing. Keep in mind this is very active area of research in Google and Uber at the moment!
STeven.
|
|
|
|
hemant from India [15 posts] |
8 year
|
hi steven,
is there any way where we can find the speed of the car using rr ?
the cam is mounted on the car itself .
thank you
|
|
|
|
Steven Gentner from United States [273 posts] |
8 year
|
Not just using a standard forward facing camera ... or at least not very accurately. There are ways of determining Time to Contact which can translate to speed but that's not very accurate. Detecting speed from zoom isn't very accurate. If you place the camera point straight down on the floor like a mouse camera that's possible to calculate speed. Pointing it to the right or left is better but not as good as seeing the ground moving.
Alternatively, just using other sensors like the built in odometer or adding encoders will do the trick. With some older cars you can even tap into the odometer signal (typically an analog wave that changes frequency based on speed) which is the least mechanical difficulty but with a bit more software to determine frequency from the sampled signal.
STeven.
|
|
|
|
Anonymous |
8 year
|
|
|
|
|
Arvind from India [6 posts] |
8 year
|
Hi Steven,
I want to know, what are the advantages of using stereo-vision in navigation? Does it mean that I can get distance or depth of any object in order to avoid collision and related things? And please tell me if I can integrate feeds of camera and gps simultaneously to guide an autonomous car.
|
|
|
|
Steven Gentner from United States [1446 posts] |
8 year
|
Arvind,
"I can get distance or depth of any object in order to avoid collision and related things" - yes, to some extent. The quality will depend on a lot of factors ... mainly the amount of texture in the image.
Yes you can integrate camera and gps feeds to create a better autonomous car. This form of sensor fusion is very common and documented extensively on the web using other sensors. I know that Stanley from the original DARPA Grand challenges used LIDAR and camera data fused to look for valid pathways.
STeven.
|
|
|
|
Arvind from India [6 posts] |
8 year
|
|
Thanks Steven for your last reply. Please tell which stereo vision camera I should use to integrate successfully with RR? I would like to start with most economic one.
|
|
|
|
Steven Gentner from United States [1446 posts] |
8 year
|
We're not aware of any good economical stereo cameras. Typically we just purchase two reasonably good quality cameras and mount them as perfectly as possible in a stereo rig. From there, tweaking the heck out of the software can get you reasonable enough results to avoid large obstacles (but if a rabbit runs out in front ... oh well!) This also only works for relatively slow moving objects ... ie. we avoid needing to sync the cameras.
The problem is that it is very expensive to calibrate, sync and secure two cameras in such a way as to not have to re-calibrate frequently. Point Gray with their bumblebee cameras have been available for a long time and are good quality if you can afford them. We've never actually played with one since they are way out of even our budget.
http://www.ptgrey.com//bumblebee2-firewire-stereo-vision-camera-systems
Perhaps post on stackoverflow to see if a more general audience has advice. Or review the choices others have made on their way to using stereo images.
Note, Minoru is an option but we've not liked the results it has ... the two images are way off. Maybe they improved the calibration since we last tried it.
http://fukushima.web.nitech.ac.jp/fukushima/minoru/minoru3D-e.html
You may also see if you can get the newest playstation stereo camera to work. Does not come with a standard USB connection wired correctly and no drivers (yet) for Windows. Lots of hacking needed for that solution .. unless someone has come out with a tutorial since we last looked.
STeven.
|
|
|
|
hemant from India [15 posts] |
8 year
|
steven,
the first link you sent us of the bumblebee camera from point gray , we have it.
the device specification is model no. BB2-03S2C-60.
now how do we connect this to the laptop on which RR is present.??
hemant
|
|
|
|
Steven Gentner from United States [1446 posts] |
8 year
|
Hemant,
In theory that is a 1394 camera. If you don't have one of those firewire ports on your machine (doubtful) you will have to buy an adapter of some sort to convert a firewire port to a USB or ethernet or something like that. We used to use PCMCIA cards to do that.
Once that is connected the Firewire_Camera module in *theory* would be able to connect to the camera. The ** is because this has not been tested with this camera so it will probably not work. Regardless you still need the hardware in order to connect the camera to your computer for any app to access it.
STeven.
|
|
|
|
hemant from India [15 posts] |
8 year
|
steven,
we have detected the road ahead using canny !
now can you please help us as to how to check the correction in the road ahead as to curving right or left ?
we had used floor finder module , and then the cog module,checked it with the middle of the image and accordingly through serial module gave forward , left and right .
i want to use the canny module .. but the correction part after detecting edge is the problem . please suggest us how.
thank you
hemant
|
|
|
|
Steven Gentner from United States [1446 posts] |
8 year
|
Hemant,
Detecting the road with Canny works but you will have a lot of other non-road detection. The process now is to eliminate those edges that are not relevant to the actual road such that you can narrow down the detected edges to just the road. This can be done in several ways. One way is to filter edges by the angle of that line ... I'm not sure if this will work on your images since the values will depend on where the camera is placed on the car.
Note, while asphalt roads typically have sharp edges dirt roads are a different story. Especially since asphalt roads typically have paint marks (yellow/white lines) which are easier to detect. Are you intending to use this for off road type applications?
Perhaps you can paste an image or two here so I can get a better idea on your camera view/angle.
Thanks,
STeven.
|
|
|
|
hemant from India [15 posts] |
8 year
|
steven,
say the detection part is done that is the relevant edges. now how do we check for the correction whether to turn left or right depending on the road curve. the camera is placed at the centre of the car say 3 feet above the ground. the images i will send them soon to you. but the query is what after detection. which modules to be used for correction ?
hemant
|
|
|
|
Steven Gentner from United States [1446 posts] |
8 year
|
You would have to use different parts of the detected road to determine different factors. For example, you would look at the middle of the image (ie. towards the horizon) for where the road is. If it is straight ahead, you can increase the car speed, if it is very much to the left or right you would need to slow down the car. For direction, you would need to look a bit before the car to determine if you are in the right lane and how much you need to turn.
Again, you will have to test a huge database of images/videos in order to get the right calibration between the detected road and the actual car speed/direction.
STeven.
|
|
|
|
Arvind from India [6 posts] |
8 year
|
|
Hi Steven, I checked AVM navigation module in AVM_Q3Mod. I liked the concept of it but please help me out in using this in an autonomous car. I give serial commands to car for movements via serial module. In AVM_Q3Mod its just a simulation using robotdrv module, but I want it to work on my car. I hope you got want I want to achieve.
|
|
|
|
hemant from India [15 posts] |
8 year
|
hi steven,
am attaching this program where am using canny and hough.
please help me with the way I have used display line module.
I want the line to give me the correction when the road turns.
so ive used calculate angle module .. please see and tell me ur recommendations to better it .
ty
hemant
program.robo
|
|
|
|
Steven Gentner from United States [273 posts] |
8 year
|
Hemant,
Do you have the DSCN9048.mp4 video? If its not too big you can zip it and upload it here ... otherwise a dropbox or similar link might be needed. Its hard to understand what you need without seeing the video ... or at least one image frame from that video.
Note, the robofiles don't automatically embed video files due to their size.
Thanks,
STeven.
|
|
|
|
hemant from India [15 posts] |
8 year
|
steven ,
We have uploaded the video @
** https://www.youtube.com/watch?v=KPR8zR445iA&feature=youtu.be **
.. it is a bit too fast because it was captured on manually driven fast moving car.
Problem - 1. How to find the intersection of the two hough lines detecting the edge of the road ?
2. How to calculate the angle from the bottom center of the screen to the point where the hoghh lines intersect ?
will be very grateful !
thank you
hemant
|
|
|
|
Steven Gentner from United States [1446 posts] |
8 year
|
|
|
This Thread