C#, Kinect, Windows 8, WinRT

TCD.Mathematics

UPDATE: Source and samples for WinRT and WPF are now on GitHub: https://github.com/michaelosthege/TCD.Mathematics This is just a quick post to introduce you to TCD.Mathematics, which is a NuGet package that adds several extension methods to the System.Windows.Media.Media3D namespace. If you’re unfamiliar with this namespace, it contains Point3D, Vector3D, Line3D and Plane3D classes which can be used for three-dimensional calculations. EDIT: I just submitted an update to the NuGet package, wich adds support for WinRT. Please not that WinRT does not have neither Point3D, Vector3D, nor the System.Windows.Media.Media3D namespace. So in WinRT all classes, including substitutes for Point3D/Vector3D are located under TCD.Mathematics. EDIT: Since version 1.2.0 TCD.Mathematics is a Portable Class Library that targets everything: Silverlight 4+, .NET4+, .NET for Windows Store, Windows Phone 7+, Xbox360 – make sure you’re running version 2.1+ of NuGet Package Manager! The extension methods, which are added to the mentioned Point3D/Vector3D classes are: Vector3D.AsPoint3D() Point3D.AsVector3D() Vector3D.Normalized() Point3D.CenterOfClusterWithOtherPoint3Ds () Point3D.ProjectOnLine3D() Point3D…

C#, Kinect, UI, Windows 8

TCD.Kinect

EDIT: If you’re interested in Skeleton tracking and calculating 3D stuff, take a look at TCD.Mathematics too^^ EDIT: There’s a WPF application that I once wrote, featuring SkeletonPainter3D: Kinect Sword. There’s a Youtube video of it (http://youtu.be/2j6GCd4M1bA) and that’s the source code: http://dl.dropbox.com/u/7813771/Blog/CodeSamples/KinectSword.zip   Hello there, I’ve created another UserControl, this time for Kinect and .NET 4.5, which hopefully has the potential to save you a lot of time. It can be used to draw three-dimensional Skeletons and explore the virtual space around them in real-time. A picture may be more appropriate: The control can be constructed in XAML: xmlns:kinect=”clr-namespace:TCD.Kinect.Controls;assembly=TCD.Kinect.Controls” <kinect:SkeletonPainter3D x:Name=”painter” Background=”Black” IsAnaglyphEnabled=”False” CameraFieldOfView=”70″ CameraPosition=”0,0.5,-1″ CameraLookAt=”0,0,2″ /> And populated with Skeletons using painter.Draw(skeletons), where skeletons is an Skeleton[] from inside the SkeletonFrameReady event. You can either get the TCD.Kinect package from NuGet, or refer to the sample implementation: http://code.msdn.microsoft.com/Implementation-of-f175b025 Let me know if you have any questions or suggestions, and by the way: SkeletonPainter3D has an experimental…

C#, Kinect

Draw the Skeletons in two lines

Hello, Recently I was coding some Kinect stuff and wanted to draw the skeleton that the user can see what’s going on. In the Samples there’s some code for that, but it’s pretty annoying to copy/paste it in your own project every time, so I wrapped it up in a class called SkeletonPainter. To use it you just need an Image of either 640×480 or 320×240 and hand it over to the SkeletonPainter, along with the resolution and a background Brush of your choice: SkeletonPainter painter = new SkeletonPainter(skeletonImage, SkeletonPainterResolution.Resolution320x240, Brushes.White); Of course, you need to hand him the KinectSensor as well: painter.SwitchSensor(sensor); This way you can easily switch the KinectSensor while the painter is still active^ Have a try and tell me what you think =) Download TCD.Kinect.zip

C#, Kinect, Software, UI

KinDrive (for Kinect for Windows SDK 1.0)

Recently I migrated from Beta2 to Kinect for Windows SDK 1.0, so here’s a quick review of the new features: works w/ Kinect for Windows hardware AND the ‘old’ Kinect for XBox360 requires Kinect for Windows SDK 1.0 (or at least its runtime) tracks two players (you can play split-screen-games…) sends not only WASD keys, but also UHJK racing-mode only, no detection of additional gestures! (like the ones in the older versions) you can use the DLLs to write your own interface (and detect more gestures) –> mail me, or comment if you have questions Oh and one random thought: I think the delay you feel when playing a game with it, is due to the slowness of your hands rather than b/c of software-delay – it’s just a theory I can’t prove, but compare it with playing w/ a hardware steering wheel… Here’s the download link

C#, Kinect, Software, UI

Kinect over Sockets

There’s no Kinect support in Windows 8 Metro, so I had to come up with a network-based solution to use the tracked skeleton data. A byproduct of this are the TCD.Networking and TCD.Kinect namespaces. Both have a Server and a Client child namespace (with simple network communication or Kinect over network..). What I can make available today is a sample of how to send Kinect-skeleton-data from a WPF server application through the network (or to localhost..) to a WPF client application. While an advantage of network-based routing is that you can have multiple clients, a major disadvantage is that you can’t use the tracking-engines methods to render 3D-coordinates to a pixel coordinate. I had to make a CustomSkeletonData class as well, b/c the JSON deserialization is kind of fussy. You may use all code and libraries you find in the solution to power your own projects =) Most of it…

Kinect, Software, UI

KinDrive enhanced – working w/ every game using DirectX input

Last week – on thursday to be precise – I stumbled over a library to send input to the system on DirectX-level, called InputManager (a big different to all the tons of SendKeys()-wrappers out there!!). So great thanks to Shay from Israel for making this awesome library!! As all the rest of the architechture was up and running already, I just made a simple WPF application to show the Skeleton and listen for a gesture as well. The gesture (streching your arms in the air) causes the WASD-input to halt, a press of the F-Key (which I usually use to enter vehicles/use weapons in games..) and finally after 3.5 seconds everything continues as before. Here’s a video of the KinDrive for GTA WPF application: Then of course: GTA IV: And just for fun – Need For Speed Carbon: For those interested in DirectX-level Keyboard/Mouse input: I’ll blog about the InputManager anytime soon. My plan is to open source…

Kinect, Software, UI

KinDrive

Hey everyone, I’m happy to announce that I finally did the re-coding of KinDriver, now called KinDrive. The new version is made with the Beta 2 SDK, is a lot more stable, simpler and faster than the old one. It lacks of speech recognition and remote-controlability, but I splitted the two main features (tracking and keypressing) into separate assemblies, so it’s very easy to build a whole new application out of it! Here comes the code part: (if you aren’t interested in using coding, skip to the last paragraph)   TCD.KinDrive.Tracking.dll In this assembly there are three classes: KinDriveTracker, Point3D and DriveInformation TCD.Mathematics.Point3D is just a helper class for 3D vectors/points with some useful calculation-methods (like calculating the distance etc.) DriveInformation exposes two properties, Throttle and Steer, as well as a constructor that takes a SkeletonFrame as a parameter. Tracking and steer/throttle calculations are done by this very same constructor….

Kinect, Software, UI

The KinDriver Sources

Hey there, As a result of some requests I got, I decided to share the KinDriver sources with you =) There are 3 important things to notice: 1. You can define how keypresses are generated: there are three different methods that may or may not work with different computer games. 2. There’s built-in speech recognition, but it’s currently deactivated, b/c I don’t really know how to define grammar… 3. There’s something called “KinDriverControllerService” – it’s a WCF service exposed at http://localhost:8001/connect (you probably have to allow it in your firewall). This service can be used to send commands (activate/deactivate/custom macros..) to KinDriver. I’ve made a client for WP7, but as it’s more like a proof-of-concept, I encourage you to write a new one (there are WCF libraries for Android and even iOS I think^) you can get the sources here: http://dl.dropbox.com/u/7813771/Blog/KinDriverSources.zip If you have any further questions comment on this…

Kinect, Software, UI

Kinect tracking

UPDATE: The’re newer versions and posts about this software – if you want to test it please have a look at: http://kuchenzeit.wordpress.com/tag/kindriver/   It’s been only 5 days since MS launched the Kinect for Windows SDK and in three of these days I coded KinDriver (and another small tool I’ll write about too). KinDriver is a dynamic Kinect tracker designed to be used with racing/driving games. We’ve seen Kinect trackers designed for the use w/ games already (talking of FAAST at this point), which may work nicely with ego-shooters or adventure games, but have a huge disadvantage when it comes to driving: They only know 1 or 0, just like the real keyboard. But while you can (an do) tip keys many times in a short period of time on your keyboard, doing so with you whole body is even more exhausting than you think! So I made a tracker that…

Hardware, UI

Mounting the Kinect

Yes! I’m well-prepared for an the upcoming milestone in human-computer interaction. The expert already knows what I’m talking about. It’s the Kinect SDK. Coming as OpenSource software it will enable some of us (including me ) to make gesture-driven applications. So as a matter of prudence I assembled a mount for my Kinect this morging. Rather than standing below, the camera now sits on its throne on top of the TV screen, so it has the best point of view for one or two persons standing in front of the TV screen and interacting with the computer. Though it was a bit tricky, the ‘throne’ now perfectly fits between the wall and the back of the TV: