Monday, May 23, 2011

Change Set 5994

There is a new version of the source code available: 5994

- Added a build script (execute with build.bat, you might have to adjust the path)
- New property for HandDataSourceSettings: MinimalPointsInContour
- DetectCenterOfPalm = true by default
- Increased the speed of the palm finder a bit

(I did not publish a new binary release, I won't do that for every check in)

Saturday, May 14, 2011

Testing Hand and Finger Detection

Until now the project source code on CodePlex was missing an important part: Tests!

In the newest commit (5603) I've added two integration tests that cover the whole production chain (but do not rely on OpenNI to provide the data) from depth data to clustering to hand detection.

Test 1: No hand is visible
 
Test 2: A hand with 5 fingers must be detected

Friday, May 13, 2011

Quick Start

I've added a small quick start documentation to the CodePlex page:

http://candescentnui.codeplex.com/documentation

Quickstart

First of all you need to create an OpenNIFacade. This object is used to interact with the OpenNI Framework (www.openni.org).

var facade = new OpenNIFacade("config.xml");

A sample config xml is checked in and must specify at least one Image and one Depth Node.

How to create a cluster data source

The next step is to create a cluster data source. But first you need an DepthDataSource (actually an IDepthPointerDataSource, because it returns only the pointer to where the depth data is located in memory).

var depthDataSource = new DepthPointerDataSource(facade.GetDepthGenerator());

With this data source you can create the cluster data source:

var clusterDataSource = new ClusterDataSource(depthDataSource);

This code will use the default setting values. You can also pass in your own settings object:

var clusterDataSource = new ClusterDataSource(depthDataSource,
    new ClusterDataSourceSettings());

To get notified after new data is available, you can register the NewDataAvailable event:

clusterDataSource.NewDataAvailable +=
    new NewDataHandler<ClusterData>(clusterDataSource_NewDataAvailable);

...

void clusterDataSource_NewDataAvailable(ClusterData data)
{
    for(int index = 0;index < data.Count;index++) 
    {
        var cluster = data.Clusters[index];
        Console.WriteLine(string.Format("Cluster {0}: {1} / {2}",
            index, cluster.X, cluster.Y));
    }
}

How to create a hand data source

To create a hand data source you first need an IClusterDataSource (see previous chapter).

var clusterDataSource = ...

var handDataSource = new HandDataSource(clusterDataSource);
handDataSource.NewDataAvailable +=
    new NewDataHandler<HandCollection>(handDataSource_NewDataAvailable);

...
void handDataSource_NewDataAvailable(HandCollection data)
{
    for (int index = 0; index < data.Count; index++) 
    {
        var hand = data.Hands[index];
        Console.WriteLine(string.Format("Fingers on hand {0}: {1}",
            index, hand.FingerPointCount));
    }
}

Sunday, May 8, 2011

Finger Detection on CodePlex

Over the weekend I decided to release an initial version of the finger detection algorithm. You are welcome to download the newest version from CodePlex (http://candescentnui.codeplex.com).

video

I'd be happy to receive feedback to see what you use it for!

Please note: It is not as well refactored as the custering part and I'm still working on it.

Some restrictions:
  • Finger points only have 2D coordinates (z is always 0)
  • Finger points are not ordered. It can happen that they swap position in the array between frames
  • There can be false positives for the finger points
  • Detecting the center of the palm is currently too slow, it is disabled by default
Guidance on how to position your hand
It works best if you position your hand flat to the viewing angle of the kinect. It's also helpful if not too much of the arm is within the processed depth segment; it decreases the chance of false positives.

 
Correct hand position

The hand is too close
The hand is too far or the cluster is not compact
False positive in the lower left corner
Properties
There is a new settings window that allows you to change settings while the program is running.
Settings window

Wednesday, May 4, 2011

Microsoft Research Kinect Page

Microsoft has published a teaser page to advertise their Kinect SDK.

http://research.microsoft.com/en-us/um/redmond/projects/kinectsdk/

According to this page it will offer:

  • Audio processing
  • Sound source localization
  • Depth, raw image and audio data
  • Skeleton tracking
  • Samples and documentation

Your Plans with Hand and Finger Tracking?

Today I would like to ask you what you plan to do with hand and finger tracking?

Image manipulation? A game? ...?

I hope to release a first version by the end of this or next week.