Augmented Reality using Kinect

Talk about Head Mounted Displays (HMDs), augmented reality, wearable computing, controller hardware, haptic feedback, motion tracking, and related topics here!
Post Reply
User avatar
Johnny-Mnemonic
Binocular Vision CONFIRMED!
Posts: 261
Joined: Sun Aug 24, 2008 5:52 am
Location: Zurich area, Switzerland
Contact:

Augmented Reality using Kinect

Post by Johnny-Mnemonic »

Here is small demo from me, using Kinect for Augmented Reality :D
[youtube-hd]http://www.youtube.com/watch?v=ezahJ2KrAOE[/youtube-hd]

Retro MRG2.2 with webcam and head-tracker attached used as AR HMD:
Image
For experimentation purpose it's resolution is quite adequate, big FOV is more important!

Image

Image

Image

Image

Hope you will like this small WIP demo guys!
Mentioned on V-Rtifacts:
http://www.vrtifacts.com/hmds/mnemonics ... ty-kinect/
Last edited by Johnny-Mnemonic on Mon Mar 07, 2011 11:55 am, edited 1 time in total.
Oculus Rift, Vuzix Wrap 920 AR!, Vuzix VR920, Liquid Image MRG 2.2, Razer Hydra, P5 Glove, Microsoft Kinect, TrackIR5, 2 x Hillcrest Labs Freespace tracker, Fujifilm finepix real 3d w3, GeForce 9800GT 1Gb, GeForce GT 430 1Gb, DELL XPS 17 l702x with GeForce 555 GT 3Gb, and good-old VFX1 setup
User avatar
Likay
Petrif-Eyed
Posts: 2913
Joined: Sat Apr 07, 2007 4:34 pm
Location: Sweden

Re: Augmented Reality using Kinect

Post by Likay »

This is cool! Are you using anything of the original microsoft algorithm or did you entirely make a new one?
Mb: Asus P5W DH Deluxe
Cpu: C2D E6600
Gb: Nvidia 7900GT + 8800GTX
3D:100" passive projector polarized setup + 22" IZ3D
Image
User avatar
Johnny-Mnemonic
Binocular Vision CONFIRMED!
Posts: 261
Joined: Sun Aug 24, 2008 5:52 am
Location: Zurich area, Switzerland
Contact:

Re: Augmented Reality using Kinect

Post by Johnny-Mnemonic »

Likay wrote:Are you using anything of the original microsoft algorithm or did you entirely make a new one?
To obtain head/hands positions I'm using FAAST software (based on OpenNI framework):
http://projects.ict.usc.edu/mxr/faast/
Which stream user joint's data via VRPN.

Still waiting for Microsoft to release official SDK.
Oculus Rift, Vuzix Wrap 920 AR!, Vuzix VR920, Liquid Image MRG 2.2, Razer Hydra, P5 Glove, Microsoft Kinect, TrackIR5, 2 x Hillcrest Labs Freespace tracker, Fujifilm finepix real 3d w3, GeForce 9800GT 1Gb, GeForce GT 430 1Gb, DELL XPS 17 l702x with GeForce 555 GT 3Gb, and good-old VFX1 setup
User avatar
cybereality
3D Angel Eyes (Moderator)
Posts: 11407
Joined: Sat Apr 12, 2008 8:18 pm

Re: Augmented Reality using Kinect

Post by cybereality »

Very cool, man. I've been itching to try something like this myself.
User avatar
Johnny-Mnemonic
Binocular Vision CONFIRMED!
Posts: 261
Joined: Sun Aug 24, 2008 5:52 am
Location: Zurich area, Switzerland
Contact:

Re: Augmented Reality using Kinect

Post by Johnny-Mnemonic »

Thank you Cybereality!
Oculus Rift, Vuzix Wrap 920 AR!, Vuzix VR920, Liquid Image MRG 2.2, Razer Hydra, P5 Glove, Microsoft Kinect, TrackIR5, 2 x Hillcrest Labs Freespace tracker, Fujifilm finepix real 3d w3, GeForce 9800GT 1Gb, GeForce GT 430 1Gb, DELL XPS 17 l702x with GeForce 555 GT 3Gb, and good-old VFX1 setup
PalmerTech
Golden Eyed Wiseman! (or woman!)
Posts: 1644
Joined: Fri Aug 21, 2009 9:06 pm

Re: Augmented Reality using Kinect

Post by PalmerTech »

Very cool! Now what would be really fun, is throwing those energy blobs. :D
User avatar
cybereality
3D Angel Eyes (Moderator)
Posts: 11407
Joined: Sat Apr 12, 2008 8:18 pm

Re: Augmented Reality using Kinect

Post by cybereality »

@Johnny-Mnemonic: Did you get the idea for the flames from Burning Chrome?
User avatar
Johnny-Mnemonic
Binocular Vision CONFIRMED!
Posts: 261
Joined: Sun Aug 24, 2008 5:52 am
Location: Zurich area, Switzerland
Contact:

Re: Augmented Reality using Kinect

Post by Johnny-Mnemonic »

PalmerTech wrote:Very cool! Now what would be really fun, is throwing those energy blobs. :D
I will add this ;)
Actually what demonstrated on video is just platform possibility, I want to make few art projects based on this.
Just need to solve a camera cable problem, it's too short and simple USB cables don't work to effectively make it longer.
MRG2.2 cable is about 7 meters long, so when I will make such cable for camera I will be able to freely move around.
cybereality wrote:@Johnny-Mnemonic: Did you get the idea for the flames from Burning Chrome?
Particularly here - no :D
But, Burning Chrome, Neuromancer and other Gibson's works had big influence on me, I even made few illustrations for them, I think it's good to post them here:

Image

Image

Image

By the way, welcome to my art-gallery, I have other cyberpunk related artworks:
http://maxim-lysak.deviantart.com/gallery/
Oculus Rift, Vuzix Wrap 920 AR!, Vuzix VR920, Liquid Image MRG 2.2, Razer Hydra, P5 Glove, Microsoft Kinect, TrackIR5, 2 x Hillcrest Labs Freespace tracker, Fujifilm finepix real 3d w3, GeForce 9800GT 1Gb, GeForce GT 430 1Gb, DELL XPS 17 l702x with GeForce 555 GT 3Gb, and good-old VFX1 setup
User avatar
android78
Certif-Eyable!
Posts: 990
Joined: Sat Dec 22, 2007 3:38 am

Re: Augmented Reality using Kinect

Post by android78 »

That's cool.
I wonder if you could use the image captured from the camera to help correct the head tracking errors.
User avatar
cybereality
3D Angel Eyes (Moderator)
Posts: 11407
Joined: Sat Apr 12, 2008 8:18 pm

Re: Augmented Reality using Kinect

Post by cybereality »

@Johnny-Mnemonic: Very nice work, man!
PalmerTech
Golden Eyed Wiseman! (or woman!)
Posts: 1644
Joined: Fri Aug 21, 2009 9:06 pm

Re: Augmented Reality using Kinect

Post by PalmerTech »

I watched you on Deviantart!

Here is a PDF I found that reminded me a bit of your project: http://www.nttdocomo.co.jp/english/bina ... _006en.pdf" onclick="window.open(this.href);return false;
User avatar
Johnny-Mnemonic
Binocular Vision CONFIRMED!
Posts: 261
Joined: Sun Aug 24, 2008 5:52 am
Location: Zurich area, Switzerland
Contact:

Re: Augmented Reality using Kinect

Post by Johnny-Mnemonic »

Microsoft now is able to track head-rotation, facial expressions and hand/finger movements with Kinect:
[youtube]http://www.youtube.com/watch?v=z8MeHNCsMKY[/youtube]

I hope they will include all those features in official SDK, would be so nice!
Oculus Rift, Vuzix Wrap 920 AR!, Vuzix VR920, Liquid Image MRG 2.2, Razer Hydra, P5 Glove, Microsoft Kinect, TrackIR5, 2 x Hillcrest Labs Freespace tracker, Fujifilm finepix real 3d w3, GeForce 9800GT 1Gb, GeForce GT 430 1Gb, DELL XPS 17 l702x with GeForce 555 GT 3Gb, and good-old VFX1 setup
bobv5
Certif-Eyed!
Posts: 529
Joined: Tue Jan 19, 2010 6:38 pm

Re: Augmented Reality using Kinect

Post by bobv5 »

Thats great, but, how will it cope with a head hidden in a hmd?
"If you have a diabolical mind, the first thing that probably came to mind is that it will make an excellent trap: how do you get off a functional omni-directional treadmill?"
User avatar
Johnny-Mnemonic
Binocular Vision CONFIRMED!
Posts: 261
Joined: Sun Aug 24, 2008 5:52 am
Location: Zurich area, Switzerland
Contact:

Re: Augmented Reality using Kinect

Post by Johnny-Mnemonic »

bobv5 wrote:Thats great, but, how will it cope with a head hidden in a hmd?
I guess finger tracking is ok :)
Head rotations and mimic should work with compact HMD's.
P.S. You can always draw a face over HMD lol :mrgreen:
Oculus Rift, Vuzix Wrap 920 AR!, Vuzix VR920, Liquid Image MRG 2.2, Razer Hydra, P5 Glove, Microsoft Kinect, TrackIR5, 2 x Hillcrest Labs Freespace tracker, Fujifilm finepix real 3d w3, GeForce 9800GT 1Gb, GeForce GT 430 1Gb, DELL XPS 17 l702x with GeForce 555 GT 3Gb, and good-old VFX1 setup
User avatar
Johnny-Mnemonic
Binocular Vision CONFIRMED!
Posts: 261
Joined: Sun Aug 24, 2008 5:52 am
Location: Zurich area, Switzerland
Contact:

Re: Augmented Reality using Kinect

Post by Johnny-Mnemonic »

You can call me crazy, but...
Image
I just received mine Wrap 920 AR :lol:

Expect reviews sometimes later!
Oculus Rift, Vuzix Wrap 920 AR!, Vuzix VR920, Liquid Image MRG 2.2, Razer Hydra, P5 Glove, Microsoft Kinect, TrackIR5, 2 x Hillcrest Labs Freespace tracker, Fujifilm finepix real 3d w3, GeForce 9800GT 1Gb, GeForce GT 430 1Gb, DELL XPS 17 l702x with GeForce 555 GT 3Gb, and good-old VFX1 setup
User avatar
cybereality
3D Angel Eyes (Moderator)
Posts: 11407
Joined: Sat Apr 12, 2008 8:18 pm

Re: Augmented Reality using Kinect

Post by cybereality »

Sweet man!
Post Reply

Return to “General VR/AR Discussion”