Friday, February 25, 2011

Grab & Drag

With the hand tracking working at a good enough level, the next step was to implement a 'grab gesture' detector. Currently it is not yet very intelligent and only uses the change of the size of the cluster as data source.

But this already enables the user to grad something on the screen and drag it around and then drop it again:

 All in all I am surprised how easy it is to do these things with 3D data at hand. The next step will be using two hands to rotate and scale the image.

Thursday, February 24, 2011

Hand tracking with k-means clustering

Today's video features a screen capture where I am tracking two hands using depth data gathered from the Kinect sensor.

Instead of applying the built-in hand tracking from OpenNI, I implemented a simple k-means clustering algorithm, which divides all points of a frame that are closer than 80 cm into one or two clusters.

The red dots are the centers of those clusters, the other colored points are the ones that are used to find the clusters. For performance gains, only every 25th point is measured.

Tuesday, February 15, 2011


Windows Forms does not allow controls to be accessed from non GUI threads. You are supposed to use the Control.Invoke method that takes a delegate as parameter which I don't find very comfortable.

Here is a simple class with two extension methods that allow you to use a Func<T> or Action as parameter.
   public static class ControlExtensions   
     public static void InvokeIfRequired(this Control control, Action action)  
       if (control.InvokeRequired)  
     public static TResult InvokeIfRequired<TResult>(this Control control, Func<TResult> func)  
       if (control.InvokeRequired)  
         return (TResult)control.Invoke(func);  
       return func();  

With these you can write methods like the ones below that can be called by the GUI thread or any other thread.

1. Action
     public void SetLoadingMessage()  
       this.InvokeIfRequired(() =>   
         this.labelMessage.Text = "Loading...";  
         this.labelMessage.Visible = true;  

2. Func<T>
     public string GetText()  
       return this.InvokeIfRequired(() => this.textBox.Text);  

Warning! These methods will throw an exception if the control was diposed.

Sunday, February 13, 2011

Virtual Keyboard

This weekend I didn't have much time to spend on the Kinect. The only thing I came up with was this (not very sophisticated) virtual keyboard:

Thursday, February 10, 2011

Gesture recognition and hand tracking with Kinect and OpenNI

This is a follow up on my recent post 'Playing with the Kinect'. Until now I was only using the depth map, but yesterday I got gesture recognition and hand tracking working:

Tuesday, February 8, 2011

Playing with the Kinect

Usually when presenting a slideshow you have to either have a handheld device to switch slides or be near the computer to use the mouse / keyboard. Maybe you also have a laser pointer to highlight things on the screen.

Wouldn't it be nice if you could use gestures to switch slides or to zoom? Or you could just point with a finger at the screen and the slideshow program would highlight the area you point at.

Last Friday I bought myself a Kinect Sensor and connected it to my PC (here is a tutorial how to do that with OpenNI).

Below is a screen capture of my first integration into Candescent Slideshows. Currently you can either go to the next slide if you move the hand (or anything else) from right to left or go to the previous slide by moving from left to right. The image is mirrored, because the Kinect is located in front of me.

Thursday, February 3, 2011

ConcurrentDictionary Extensions

With the new .NET Framework 4, Microsoft has introduced new concurrent collection classes in the System.Collection.Concurrent namespace. I had written a thread safe dictionary (that used locks), which I could replace with the new ConcurrentDictionary. However, my implementation had some useful methods which I did not want to lose, so I wrote these extension methods:
  •  bool TryRemove(TKey key)
    This method can be used when you don't care about the value that is removed.
  • TValue GetOrDefault(TKey key)
    Useful when you want to use the types default value if the key is not in the dictionary.
  •  TValue AddOrUpdate(TKey key, TValue value)
    When the value is already known and you don't want to use the Func parameter.
  • TValue GetOrAdd(TKey key, Func<TValue> valueFactory)
    When the valueFactory functions execution does not depend on the key.
  • int RemoveConditional(Predicate<TValue> condition)
    This method removes all key value pairs where the value meets a condition and returns the count of removed items. Warning: This method is not thread-safe. If one thread changes the value between the check of the condition and removal, the key is still removed.
The download contains one class with the extension methods and another with some tests.

Click here to download

First Post

Here I'm going to write down my thoughts about topics related to the development of Candescent Organizer and Slideshows and programming in general.

Let's start with some numbers about the project:
Number of lines = 59'283, number of code files = 846, number of code-generated lines = 12'003, number of user-entered blank lines = 6'324, number of user-entered comments = 1'197

Generated by this line count tool.