Wednesday, August 17, 2011

Candescent NUI (9237)

For those who were waiting on a binary version, it's now available for download here.

For details please read my last post.

Sunday, August 14, 2011

KinectMultiTouchDevice for WPF 4

Today I've published the KinectMultiTouchDevice class on CodePlex. It allows you to leverage the multi-touch support added by Microsoft to WPF 4 and combining it with Kinect! (Here's an introduction blog post that covers much of multi-touch in WPF 4).

Multi-touch with Kinect and WPF 4

The update comes with two simple WPF samples in a new Candescent NUI WPF Samples project.
It also works with the WPF samples of Microsoft Surface 2.0 SDK, but you'll have to add the KinectTouchDevice for each demo project manually.

Screen capture of the multi-touch sample project

All fingers of both hands can be used simultaneously. Though it rarely makes sense to use more than 1 or two per hand.

Please note: I've only published the source code, I'll add a binary release soon. The sample only runs with OpenNI for the moment.

Technical details
The class CCT.NUI.Touch.KinectMultiTouchDevice can be found in the CCT.NUI.HandTracking.dll.

In a WPF 4 window it can be initialized like this (in the loaded event, you must should not add this code in the constructor).

 this.factory = new OpenNIDataSourceFactory("config.xml");  
 var handDataSource = new HandDataSource(this.factory.CreateClusterDataSource());  
 this.multiTouchDevice = new KinectMultiTouchDevice(handDataSource, this);  
 handDataSource.Start();

The device identifiers are created the following way: The ID is always a 2 digit decimal number where the first number is the hand ID and the second number the finger ID. Finger IDs are assigned in order of appearance and do not identify a specific finger (for example index).

There are some glitches here and there, I'll keep working on it. For now I'm looking forward to see what you'll do with it!

Other changes in changeset 9161
  • New mapping algorithm
    Hands and fingers are mapped between two frames and keep the same ID until they disappear. IDs are reused! That means if Hand 1 is removed from view the next hand that appears will get ID 1 again. The same is true for the fingers.
  • Fingers and Hands implement the ILocatable interface
  • HandData.Id is now int instead of guid
  • Finger points get IDs. They are assigned in order of appearance, so you can't assume that 1 is always the thumb. Also the IDs are reused.
  • Option to show depth view in Candescent StartMenu
  • Exit button in Candescent Start Menu

Sunday, August 7, 2011

Kinect Multitouch Device for WPF 4 (Preview)

I'm working on a KinectMultiTouchDevice class which will create WPF 4 TouchDevices for touch input.

Here's a code snippet that shows how you will be able to intialize the device (inside the load method of a WPF 4 window):

 this.factory = new OpenNIDataSourceFactory("config.xml");  
 var handDataSource = new HandDataSource(this.factory.CreateClusterDataSource());  
 this.multiTouchDevice = new KinectMultiTouchDevice(handDataSource, this);  
 handDataSource.Start();  

Then you can use the following events of the UIElement class:
  • public event EventHandler<TouchEventArgs> TouchEnter;
  • public event EventHandler<TouchEventArgs> TouchLeave;
  • public event EventHandler<TouchEventArgs> TouchMove;
  • public event EventHandler<TouchEventArgs> TouchUp;
video
This is a small sample that draws lines between two touch move events. It's based on WpfTouchEventsSample that can be found here


video
In this video multiple finger devices are used simultaneously (the lines are always drawn between the last two reported points)

I'm also working on a more sophisticated sample that I can share together with the KinectMultiTouchDevice class.

    Wednesday, August 3, 2011

    OpenNI Upgrade

    With changeset 8918 I've updated OpenNI to the newest unstable version (1.3.2.3). All the changes are internal to Candescent NUI and don't influence it's public interface.

    I've also updated to the newest driver Avin2 SensorKinect. There you can find instructions on the installation order (OpenNI, then the driver, then NITE).

    I've installed the 32-bit version on my 64-bit Windows PC without problems (this time).