 |
Object counting
Sebastian Johnson from Sweden [9 posts] |
15 year
|
I am a total noob when it comes to RoboRealm and programming in general. But i am eager to learn. I wonder if somebody can point me in the right direction in a project concerning people counting. I have a USB camera mounted in the cealing of a shop. It look straight down on the floor beneath. What i want is to identify what probably is a person moving across the floor. When they reach a defined line and cross it i want the crossing to be counted. The counting should be bi directional so crossing in one direction will increase the "in" value, and crossing in the other, the "out" value. If several people pass at the same time and even in different directions they all should be counted individually. A person must be identified as just a person so that for example a shopping cart wont be counted as another person or cause the counting to fail because the two blobs are somewhat interconnected. What filters etc should i look at to do this (if it can be done at all). Is RR able to output data to some kind of rudimentary database or even for example trigger some event on the serial port when a confirmed count is done? I know it is not popular to ask these thing without first investigate it deeper myself, but it would probably save a lot of time if You pushed me the right way regarding what filters etc to use. The floor between the cam is a single color (grey). What would be the best way to isolate a person? Would it be a shape thing? Im thinking head+shoulders should be universally usable to identify a person. But there will be some distortion in this if a person enters far off center of the camera view. Could this be compensated? Would it be an idea to use some form of image bank containing samples of what a person might look like as a reference for the search for moving people? Can RoboRealm do that? ANY kind of suggestions are highly appreciated.
|
|
|
|
Anonymous |
15 year
|
Sebastian,
The first step is to isolate the individual from the background and from perhaps other individuals. A quick check with the Movement module
http://www.roborealm.com/help/Movement.php
would help to do this ... but it is only a start to get and idea of what movement looks like.
Be sure to use the Previous 500 or 1000 frames which will ensure that a background is built up over time and the difference between that background and the current image is what is seen as movement. Note that if someone stops and stands for ~16 seconds then they will become part of the background. (This assumes 30 fps). You can increase that number if needed (for example 1000 is 33 seconds).
This will give you a kind of blob that is the person moving through the space. You may need to dilate or close (morphology routines) that blob in order to connect it as a single blob.
Once you've had a chance to see this the next step would be to understand what people walking next or close to each other look like. Would it be possible to post a video somewhere that shows a single person and then two people and then three people walking close to each other? Also including one with the shopping cart would be helpful for further analysis.
This video may show that the movement blobs become connected together. At that point you'd need to work with one of the splitter modules that will break blobs at appropriate places (see Watershed). This may or may not work depending on what the final blobs look like.
At this point it would be best to try the above and then proceed as many unknowns have already been introduced.
Once the people have been identified it is a simple matter to count them as their COG moves across a certain point in the image. This count data can then be exported in many different ways (email, ftp, http, file, API, etc.) What database system are you talking to that would require this information? Perhaps RR can already talk directly to it or if not a module can be added to do so.
Looking forward to seeing the videos. You can use something like
http://files-upload.com/
to post the videos and then include a link back here.
Thanks for the donation!
STeven.
|
|
|
|
Sebastian Johnson from Sweden [9 posts] |
15 year
|
Thanks for Your suggestions. I am currently working with an actual avi from the counting camera in a live setting where people walk in and out all the time. I will uppload the clip to http://www.panopticon.nu/counter.rar in a few minutes. The clip unrared is 35MB in size and perfect for testing various filters trying to isolate humans. I have found the following difficult:
1. The image is "noisy" from the MPEG4 compression, i.e. the image is "crawling". I think using MJPEG would help, but many of my cameras are using MPEG4, some with poor result. How to filter out the digital artifacts?
2. I find the Background removal filter to help clean out the image, but problems from point one remains.
3. As You can see in the clip there is a pattern in the floor. This is actually a scraping mat for cleaning shoes. Here it causes problems as You can see. What to do..?
4. There is various levels of light entering thru the glass doors, and this seems to mess up the image too. How to get rid of the problem of varying light?
What is most effective in this project; to use grayscale or color images?
Is there a way to even more effectively remove the background (parts of the image that never moves).
Any hints is appreciated!
|
|
|
|
Anonymous |
15 year
|
Sebastian,
We will answer all these issues bit by bit just in case you are continuing to work.
1. The issue you have is due to interlacing. You need to get a very high quality image from the camera and then run the deinterlacing module. This will help to fix those jaggies that you are seeing. Note that once you compress it to JPEG or MPEG the deinterlace will NOT work very well due to the blurring that JPEG and MPEG have.
2. Try using the Mean filter to smooth out the image. It is like a Gaussian filter but much faster. This will also take care of #1 somewhat as a mean filter will also soften the interlaced movement.
3. Our initial tests seem to be ok with the floor mat. (Most likely due to mean filter).
4. Still an issue ... will get back to you on this one.
We will continue doing some tests on the video (thanks for that!) and get back to you early next week as to what we have tried and our best implementation.
STeven.
|
|
|
|
Sebastian Johnson from Sweden [9 posts] |
15 year
|
Thanks for Your reply. I am aware of the poor de-interlacing. I plan to use progressive cameras only in the future. The clip is actually reduced in quality to the original video. I will get a clip directly from the source without added compression. I will also look at what de-interlacing settings I used for this camera (i can adjust de-interlace). If You ever need some type of camera for other tests You can borrow from me. I have lots of them :-).
I have another idea about how to clean up the image; i just add a 940nm high pass IR filter to the cam and illuminate the entire camera footprint with steady 940nm light. This will eliminate ALL problems with varying light conditions and probably make it easier to get a clean image to process. However, it will be B/W. And I dont know if that is effective.
Well, looking forward to hear from You again. Have a nice weekend and thanks for Your help. This will keep me busy for a long while :-). Love the software!
The new better quality clip will be found at http://www.panopticon.nu/counter2.rar tomorrow.
|
|
|
|
Sebastian Johnson from Sweden [9 posts] |
15 year
|
|
Any more ideas about how to solve this so a fairy accurate counting can be done?
|
|
|
|
Anonymous |
15 year
|
YES! We have tried quite a few techniques with moderate success. The issue is in the grid like flooring that when people move over with dark coats become merged with the floor. A texture based segmentation technique (added to the movement module) helped a little but it still not quite right. We've got a couple more ideas that we would like to try but will also post a quick and easy technique just to give you something more to play with.
Seems that segmenting people from an arbitrary background is quite difficult. We will also investigate if it is possible just to focus on the head instead of the entire body to see if that can be extracted out easier. The head follows an ellipse model quite nicely and is almost never obstructed (except for hats ) so that might help a bit. More later ....
STeven.
|
|
|
|
Sebastian Johnson from Sweden [9 posts] |
15 year
|
|
I am so happy to hear that! Looking forward to hear about Your findings. I have spent hours with RR and havent been able to separate the people from other people or the background in any effective way. But it is fun experimenting with RR so i dont mind :-).
|
|
|
|
Anonymous |
13 year
|
|
hi, I want to setup a PEOPLE COUNTER like this (setup a CCTV on top), have any idea can do it now??
|
|
|
|
Anonymous |
13 year
|
to count IN and OUT number!!!
Thanks!
|
|
|
|
Anonymous |
13 year
|
Got a couple images or a video? The ground, lighting, distance to camera, etc. may cause a different technique to be used.
Also keep in mind that if people are too close to each other they will be counted as a single person unless you can provide a secondary sensor (like depth, laser, etc).
STeven.
|
|
This Thread
