Monday, January 30, 2012

Efficient model-based 3D tracking of hand articulations using Kinect

Now this is really, really cool! It's what I've been thinking about for some time now, but haven't started to work on because it looks like a ton of work.

Instead of just trying to detect the fingertips, this approach uses a 3D model of the hand and then tries to orient and bend it to fit with the data from the Kinect. Because it's a complete model of the hand you not only find the positions for the fingertips, but also all other joints. This makes my work more than obsolete ;-)

I. Oikonomidis, N. Kyriazis and A.A. Argyros, "Efficient model-based 3D tracking of hand articulations using Kinect", to appear in Proceedings of the 22nd British Machine Vision Conference, BMVC 2011, University of Dundee, UK, Aug. 29-Sep. 1, 2011.

Here's the link to the project site (more videos)

Sunday, January 29, 2012

Candescent NUI 12515

You can download the new version here: [binary] or [source]

  • NITE hand tracking with finger detection (see CCT.NUI.Samples)
  • Mouse controller application is now part of the binary release
  • New options in the mouse controller application

Monday, January 23, 2012

Combining NITE hand tracking with finger detection

I'm working on combining PrimeSense NITE middleware with my finger detection (what does NITE stand for, anybody knows?). The advantage is that the algorithm doesn't use a fixed depth segment but just the area around the tracked hand. Like this other objects can be closer but the hand is still correctly processed.

You can waive your hand to tell NITE to start tracking it. Candescent NUI then performs finger detection on the part of the depth map where the hand is located.

Saturday, January 21, 2012

Candescent NUI 12392

Here's the new version of Candescent NUI. You can download the [binary] or [source]

Threading with OpenNI
There is now a single thread updating all OpenNI generators. There used to be exceptions when running the depth and image generator datasources in parallel.

Mouse Controller
I've implemented an example on how to control the mouse. There are several modes you can play with. It's not that good yet. It's checked in as separate project, it's not added to the binary yet.

(The recording program shows the mouse in hourglass mode in the first part, don't know why...)

Pin Code Interface Source Code
The source code is now on CodePlex. You can find it in the WPF samples project.

Thursday, January 19, 2012

Quick Notes

I'm having problems when I create two datasources where one is waiting for an ImageGenerator and the other is waiting for a DepthGenerator. It seems when two different threads are accessing two different generators this can mess up OpenNI.

I'll work on a new version that will only use a single thread for all OpenNI based datasources.

Also coming up as sample: Mouse Control with Candescent NUI

Sunday, January 15, 2012

Gesture Controlled Pin Code Interface - Video

I found this video on YouTube. It was made with Adobe After Effects. I took it as inspiration for a little UI that I've made and animated with WPF. It's a gesture controlled pin code interface.

It won't win a usability award, but I think it looks cool and actually does something.

I'll publish the code in the near future as sample of Candescent NUI.

Friday, January 13, 2012

ASUS Xtion pro

As I've blogged here, I've recently got a Xtion pro depth sensor.

It's working nicely. And since I'm only using the depth data for the hand detection I can use it. It's a lot smaller than the Kinect, so it's easier to take with me.

But I guess if I'd have to buy one I'd get the Kinect or the Asus Xtion pro live.

Saturday, January 7, 2012

Upgrading to stable OpenNI

Today is release day :-)

I've upgraded OpenNI to the newest stable version ( without problems.

Candescent NUI release is 12234:  [binary] or [source]

Candescent NUI 12230

I've published Candescent NUI version 12230. You can download it here: [binary] or [source]

There are lots of changes including a split into more projects and renames of public interfaces. So this release will break some existing code, sorry for that!

I've seperated the Kinect SDK part into its own DLL called CCT.NUI.KinectSDK. This code was in the CCT.NUI.Core project before. Like this the core project needs no more reference to Microsoft.Research.Kinect. The next step will probably be to introduce a seperate project for OpenNI.

Until now it was necessary to first create a bitmap and transform that into an image source later to be used in WPF. Now there are data sources that return WriteableBitmaps that can directly be assigned as WPF image source. The WPF sample project now uses these new data sources and also draws the layers with WPF elements.

Next step will probably be to work on performance again. Things seem to have become a bit "sluggish" over time.

Wednesday, January 4, 2012

Thanks PrimeSense!

Today I've received a parcel from Israel. It's the Asus Xtion PRO sensor I've won in an OpenNI raffle they held last year on Twitter. The thing that surprised me most was how long it took to be delivered: only 1 day!  Yesterday it was in Israel, today here in Switzerland.

Cool! Now my Kinect has a little twin :-)

So: Thanks PrimeSense!

Tuesday, January 3, 2012

Happy new year!

Happy new year to everybody and thanks for 30'000 pageviews in 2011!

Stay tuned for a new release of Candescent NUI later this week. I've been working on better support for WPF.