|
Detecting Peaks from Kinect Depth Map
Max Hendrikse from New Zealand [1 posts] |
9 year
|
Hi there,
I'm trying to count kiwifruit as they move past on a conveyor belt. I had a bit of success with a regular webcam but had a hard time separating large groups of fruit from each other reliably.
I thought I would try it with a Kinect so I can use the shape of the fruit to distinguish them by looking for peaks. The problem I'm having is finding a suitable module to find these peaks. The "Peak and Valley Probe" looks like what I want but it only works along a line. Any suggestions?
I've attached a snaphot of from the (I'll only be using the depth, the RGB is just there to show what we're actually looking at).
|
|
|
|
Steven Gentner from United States [1446 posts] |
9 year
|
Max,
Yes, the Kinect is good for this type of analysis. We've worked with Potatoes in the past in this fashion.
Can you attach and image that is a grayscale version of your image? What you want is not to use the colored version as grayscale works better when processing. The trick is that you don't want to just detect a specific height (or in your case color) as some of the Kiwis will be slightly lower but should still be connected. Instead you are looking for distinct separate blobs that are large enough to be significant (see the noise in the upper right corner!) to be counted.
While the peak/valley seems to be what you want it will not work correctly as it assumes a very sharp peak/valley. Because the fruit is not quite pointy you may have many peaks for a single fruit.
If you can attach a couple of the images showing the grayscale depth (i.e. Grayscale radio buton) we can send back a file that will count the kiwis based on similar techniques used in the past.
We've also tried in the past with a regular webcam. This is possible but you need to ensure some sort of vibratory process before counting ensures that the fruit is not ontop of each other. This will still be an issue with the Kinect in that fruit directly ontop of each other will obscure the count but it will at least distinguish when they overlap a little.
You may also find
http://www.roborealm.com/forum/index.php?thread_id=4995#5
to be of interest. Not using a Kinect but the process they attempted will probably interest you.
Thanks,
STeven.
|
|
|
|
Max Hendrikse from New Zealand [2 posts] |
9 year
|
Hi Steven,
Thanks for your ideas. I have my counting working pretty well using a similar process to what you're describing. I'm using the grayscale image (I just attached the colour one to my previous post as it's easier to visualise). I crop the image which removes the noise in the corners (this is a piece of steel and the Kinect doesn't deal too well with shiny things). I found it's much easier to differentiate between different fruit if the kinect is only looking at the tops of the fruit (with a threshold). I'm using erode to split them but it’s best if they are very unlikely to be touching in the 1st place. The hard part is tuning the threshold so that the small fruit are seen while the blobs created from the large fruit are as separated as possible. Also, if I was to go with this approach I would need something better than a straight threshold since the fruit on the sides of the belt are further from the kinect than those in the middle. I've attached this robo file.
It's for this reason that I'm keen to look for peaks. It seems like it should be more adaptive for small vs large fruit. I've had some success using the peak valley probe. I don't get multiple peaks per fruit because I can use "PV Isolation" to ignore them. Because I am only looking at a small range of distance (the grayscale image changes from black to white over around 50mm), the peaks are actually very sharp, so they can be found easily.
The issue I'm having now is a bit weird (maybe it's a bug or I'm misunderstanding something). I'm using the peak valley probe to mark the peaks with a dot. I then remove the background image and I want to use blob tracking to track and count these dots. However, none of the blob functions seem to work correctly on the dots. The blob tracking module only seems to execute for a fraction of a second and only if I select the line above it, then select the blob tracking module. It is not tracking continuously. I tried updating to the latest version of Roborealm, but the only thing this changed is that the peak valley probe now also draws a line on the image. I’ve attached a robofile with of this as well.
That potato counting thread looks interesting. We make machines that do what he is talking about with the fruit on rollers (www.compacsort.com). We have a whole team that does the vision stuff, I am just prototyping mechanical designs and have recently started using roborealm with a webcam quite a bit for rough tests.
Thanks for your support,
Max
 program.robo
|
|
|
|
Steven Gentner from United States [1446 posts] |
9 year
|
Max,
There is a module that does 'peak detection' probably better suited to what you are doing than the peak/valley probe. Its called Non-Maxima Supression. Its a rather technical name as its a very common module in many vision algorithms and thus the name has stuck. Basically it looks at a specific pixel and if it is not the highest intensity pixel in a defined neighborhood it will eliminate that pixel. So in an ideal sense, a single pixel would remain for each peak in your image. In reality this doesn't quite happen so you first need to blur the image to reduce the peak noise and then apply this module.
Assuming that you will then need to expand the peak back to its original size WITHOUT connecting each fruit, we can use another module to create a border between peaks and then remove that border from the original masked image ... this creates a boundry between peaks that should be reasonable.
Attached is a robofile that does all this on your original image. Naturally many more test images are needed but this should get you a bit further along.
STeven.
program.robo
|
|
|
|
Steven Gentner from United States [1446 posts] |
9 year
|
Oh, and you will have to download the latest version for this robofile to work.
STeven.
|
|
|
|
Max Hendrikse from New Zealand [2 posts] |
9 year
|
|
Great, thanks for that. When I get some time I'll give this a go and post my results in case some else stumbles upon this thread.
|
|
This Thread