Monday, October 31, 2011

Test Data Collector - Preview

Here's a preview of what I've been working on over the weekend. It's a tool that lets you take snapshots of the depth data stream. You can then add additional data that describes the hand (center of palm and finger tips).


This will allow the definition of an accuracy metric that calculates how good the algorithm detects the fingers and center of palm.

Benefits
- Easier problem identification (problems in saved frames can be reproduced).
- Measuring improvements of the algorithm
- Automatic determination of settings

I've decided to create a seperate application instead of including it in the samples project.


I'll need your help building a collection of such annotated frames!

Wednesday, October 26, 2011

Test Data

After more than one month of not working with the Kinect, I've plugged it back in today! :-) I've had some ideas and I'm eager to implement them.

Meanwhile I was experimenting with TDD (test first, acceptance tests...) on some other greenfield project of mine and it worked out nicely. I'll try to write more tests for Candescent NUI too.

Here's one of the planned new features:
You can save single (or maybe whole sequences) of depth frames to the disk. These can then be loaded again and used to test the algorithm. The idea: No more guessing if a code or setting change did improve the algorithm, just run the tests and measure the accuracy!


A lot of the code is already on CodePlex, the only two tests that are checked in use it. There are only two still depth frames checked in and there is no UI for it yet. Now I want to make this more accessible to build a set of test data (all kind of hands, your help will be needed!)

Example:
depth frame 1: Left hand with 3 fingers
depth frame 2: Left hand with 2 fingers, right hand with 5 fingers
depth frame 3: No hand
...

Ideally the location of the finger points and center of palm will be manually marked for each test frame. Like this the distances of the points that the algorithm finds can be compared with the expected points.

Saturday, October 22, 2011

HoloDesk and OmniTouch

Microsoft (Research) keeps working on new stuff that uses the Kinect. Here's a video on YouTube that features a HoloDesk.



The second video by Chris Harrison is an example of a wearable interface. Of course the depth camera and projector will have to shrink a bit. It would be connected to a smartphone you have in your pocket.

 

Wednesday, October 5, 2011

Vacation

I've been on vacation during the last two weeks, sorry to those who had to wait for an answer.