I have spent the last few days working on getting the Kinect to function the way that I want it to. Since I did not write the program that I am using, I’ve looked through it a bit to see how it works. It looks like It scans for a blob of pixels that is around the same color and at a depth of less than 600 units. This works pretty well if you want your hand to stay in the same plane, but the goal of using the kinect is to be able to move your hand forward and backwards as well as left, right, up, and down.

Getting the actual depth value was pretty easy, the kinect returns a black and white image where the value of each pixel is the depth. All I did was get the coordinates of the center of the blob, then get the value of the depth image at that point. The obvious first thing to do is to increase the range of the depth. As you increase the range though, it stars to detect things in the background that you don’t want it to detect, in my case there is a tool-chest that it keeps finding the center of. Another problem is that the kinect has a pretty big minimum range, as you can see in the video. When I get to close, it turns black, indicating that it can no longer read the depth.

Here is a video showing the program detecting outlines and also showing the depth image (in color, not black and white):

https://youtu.be/celzrqAwF8g