TurboTuscany demo: Kinect, Razer Hydra and PS Move supported

[TTakala]

Yeehaw, Grandma is back in Tuscany! This time 100% virtually and with more space to explore than ever before.

http://www.youtube.com/watch?v=-YYiTkf3sDs

TurboTuscany features a 1st person view with a Kinect controlled full-body avatar and 4 methods for six-degrees-of-freedom (6DOF) head tracking:
* Oculus Rift + Kinect
* Oculus Rift + Razer Hydra
* Oculus Rift + Razer Hydra + Kinect
* Oculus Rift + PlayStation Move (+ Kinect)



It makes a difference to see your own body in the virtual world, affecting the sense of presence. Those of you with Kinect: Try pushing and kicking stuff, or even climb the ladder with your hands. You can take steps freely inside Kinect's range, and when you need to go further, just use a wireless controller to walk or run like you would in any normal game. We are blending your Kinect captured pose with Mecanim animation, so while you're running and your feet follow a run animation clip, you can still flail around your hands and upper body as you like.
(Kinect users will need to install Win32-bit version of OpenNI 1.5.4.0. See readme for other details.)

Positional head tracking with Kinect alone is quite rough, so try the Razer+Kinect or PS Move option if you can.



Minimum requirements
* Oculus Rift
* Windows operating system (tested to work with Windows 7, should work on XP as well as on Windows 8)

Supported input devices
* Razer Hydra
* ASUS Xtion Pro, Kinect for Xbox, (Kinect for Windows?)
* PlayStation Move and PS Navigation controllers (Move.me software and PS3 required)
* Gamepad (any Unity compatible gamepad or joystick)
* Mouse and keyboard

This demo should be pretty fun to try out even with just mouse and keyboard. There's several physics based activities, and we've hidden a bunch of easter eggs in the scene.

Download links:
https://drive.google.com/file/d/0B0dcx4 ... sp=sharing
https://dl.dropboxusercontent.com/u/824 ... any101.zip
http://www.mediafire.com/?77ardrkg4okwhua
1080p version: https://drive.google.com/file/d/0B0dcx4 ... sp=sharing

This is a technical demo created with RUIS, which is a Unity addon for developing virtual reality games and applications. By the end of this month we will release a new version that has all the features that we used to create TurboTuscany demo. More info will become available here: http://ruisystem.net






Safety Warning:
Wearing head-mounted-displays while standing up, moving, walking, or jumping is dangerous to your health and potentially deadly. Authors of this software recommend that you to avoid the aforementioned actions, and if you choose to perform them anyway, you do it at your own risk. We cannot be held responsible in any way for any consequences to you, your pets, or your equipment.

[geekmaster]

Safety Warning:
Wearing head-mounted-displays while standing up, moving, walking, or jumping is dangerous to your health and potentially deadly. Authors of this software recommend that you to avoid the aforementioned actions, and if you choose to perform them anyway, you do it at your own risk. We cannot be held responsible in any way for any consequences to you, your pets, or your equipment.

Your disclaimer does not mention children or other family members.

http://www.youtube.com/watch?v=5IJXsUBdl0g

This looks like an excellent demo. Time to find a place to setup my Kinect near my computer...

[Namielus]

fantastic

[Lookforyourhands]

Wow !

This is awesome ! Thank you so much for sharing!

Edit: I was wrong, this is FRICKEN STELLAR AMAZING INCREDIBLE AND AWESOME ! ZOMG!! &%$#!@@#@$#!!!!@@#(*@

I was just practicing tai chi with my virtual body on the rooftop listening to the sound of the wind, and taking in the scenery.

The Kinect tracking is near perfect and tracks the entire body !! So immersive.

Still having some trouble getting the hydra to work correctly but this is so great !!!!!

TO the OP: Can you please tell us how to change the resolution of the demo to 1920x1080 ? Thank you.

[RoTaToR]

I was wrong, this is FRICKEN STELLAR AMAZING INCREDIBLE AND AWESOME ! ZOMG!! &%$#!@@#@$#!!!!@@#(*@

I was just practicing tai chi with my virtual body on the rooftop listening to the sound of the wind, and taking in the scenery.

The Kinect tracking is near perfect and tracks the entire body !! So immersive.

argh, F%CK, now i must buy a kinect...

Edit: 200 EUR *head --> table*

[Lookforyourhands]

lol ^ it's probably worth waiting until Kinect 2 comes out

[TTakala]

Wow !

This is awesome ! Thank you so much for sharing!

Edit: I was wrong, this is FRICKEN STELLAR AMAZING INCREDIBLE AND AWESOME ! ZOMG!! &%$#!@@#@$#!!!!@@#(*@

Thanks for your comment!

TO the OP: Can you please tell us how to change the resolution of the demo to 1920x1080 ? Thank you.

I'm not sure if that's possible in the current release since it skips the Unity resolution dialog and goes automatically to Rift's resolution. Why, do you have access to HD version of Oculus Rift or something?

[Lookforyourhands]

haha no I don't have the HD version. I WISH. I just find running the demos at 1920x1080 looks much better overall.

Also it's pretty impressive how good Kinect works for positional & body tracking at the same time. Really really cool, and also
nice not to have to strap the hydra on. If this was cleaned up and bit, and maybe the avatar wasn't so creepy it would probably be
my favorite demo yet ! It's certainly in my top 3 already.

[TTakala]

haha no I don't have the HD version. I WISH. I just find running the demos at 1920x1080 looks much better overall.

Here you go, I made a version that starts directly into 1080p:
https://docs.google.com/file/d/0B0dcx4D ... sp=sharing
The menu is made of 2D GUI elements and is a bit off now, but I expected that. Hopefully nothing else broke ^^

Also it's pretty impressive how good Kinect works for positional & body tracking at the same time. Really really cool, and also
nice not to have to strap the hydra on.

Good to hear that it works for you. Were filtering the Sith out of the head position, it's so jittery. At this point I prefer wearing PS Move or Hydra, especially when being close to virtual objects. But I'm really looking forward to add Kinect 2 support in our Unity VR toolset.

If this was cleaned up and bit, and maybe the avatar wasn't so creepy it would probably be
my favorite demo yet ! It's certainly in my top 3 already.

Hehe, we thought about the avatar but finally decided that it has to be grandma all the way! We even hid some voice clips from her in the demo..

[yoshithedog]

Unfortunately I don't have Kinect yet, but this sounds to be revolutionary.

Could you map at least walking forward and backward to some "poses" for Kinect, so that your hands would be completely controller-free and you still could go whenever you want in VR without going far away in RR?
Mapping turning would also be nice, since then you wouldn't need to worry about wrapping in OR's cables (hope the consumer version will be wireless).
These poses would have to be both unique (so that you still had freedom to do [almost] what you want while not moving or turning) and natural (least abstract, which means most immersive; using a thumbstick to walk is very abstract). They could be "analogue", so you could fine-tune the speed of walking/turning.
Any ideas?

[Lookforyourhands]

haha no I don't have the HD version. I WISH. I just find running the demos at 1920x1080 looks much better overall.

Here you go, I made a version that starts directly into 1080p:
https://docs.google.com/file/d/0B0dcx4D ... sp=sharing
The menu is made of 2D GUI elements and is a bit off now, but I expected that. Hopefully nothing else broke ^^

Also it's pretty impressive how good Kinect works for positional & body tracking at the same time. Really really cool, and also
nice not to have to strap the hydra on.

Good to hear that it works for you. Were filtering the Sith out of the head position, it's so jittery. At this point I prefer wearing PS Move or Hydra, especially when being close to virtual objects. But I'm really looking forward to add Kinect 2 support in our Unity VR toolset.

If this was cleaned up and bit, and maybe the avatar wasn't so creepy it would probably be
my favorite demo yet ! It's certainly in my top 3 already.

Hehe, we thought about the avatar but finally decided that it has to be grandma all the way! We even hid some voice clips from her in the demo..

Wow thank you very much for the updated demo ! That's very nice of you. Also, the grandma avatar is actually growing on me kind of cool lol..
This is so much fun.

[Kamus]

This all sounds very interesting... can someone with a Kinect please post a video on youtube or the video site of your choice? =D

Aslo, a question to the OP or anyone that knows the answer: Does the Kinect need to be the Windows version? or is the 360 version enough?
Thanks!

[WiredEarp]

@ RoTaToR:

argh, F%CK, now i must buy a kinect...

Edit: 200 EUR *head --> table*

The XBox version of the Kinect is way cheaper, and most people say it does exactly what the Windows one does. Hopefully it works with this demo...?

This demo looks very nice BTW!

[TTakala]

I just created a short video about the demo: http://www.youtube.com/watch?v=-YYiTkf3sDs
Another video coming up in a few days.

[TTakala]

This all sounds very interesting... can someone with a Kinect please post a video on youtube or the video site of your choice? =D

Aslo, a question to the OP or anyone that knows the answer: Does the Kinect need to be the Windows version? or is the 360 version enough?
Thanks!

See my previous reply for the video
Kinect for Xbox 360 works, and so does Asus XTION Pro. Kinect for Windows is the only one I'm not sure about.

[Xemakon]

I could probalby research this more myself, but I'm going to ask anyway;

For an xbox 360 kinect, do I just need the microsoft driver ?
http://support.xbox.com/en-US/kinect-fo ... dows-setup

Or is that only for the pc kinect? If so does anyone have link to the xb360/pc drivers?

[TTakala]

I could probalby research this more myself, but I'm going to ask anyway;

For an xbox 360 kinect, do I just need the microsoft driver ?
http://support.xbox.com/en-US/kinect-fo ... dows-setup

Or is that only for the pc kinect? If so does anyone have link to the xb360/pc drivers?

The demo has a readme file that tells you all you need to know about installing (don't install the microsoft package):

For Kinect to work, TurboTuscany demo requires Win32-bit version of OpenNI 1.5.4.0 (OpenNI 2.0 is not supported).
Follow the Windows section’s “Install OpenNI, NITE, and the Sensor Driver” guide in the below page (be sure to install Win32 version):
http://code.google.com/p/simple-openni/ ... stallation
If the above doesn’t work, follow steps 1-5 in the “Manual method” section of this guide:
http://www.brekel.com/?page_id=170
Your installation is successful (and Kinect should work in TurboTuscany demo) if you run the NiViewer example and get it to display depth image from Kinect. Otherwise you should start googling for a solution using the error message that NiViewer prints out.

[mrgrey]

I could probalby research this more myself, but I'm going to ask anyway;

For an xbox 360 kinect, do I just need the microsoft driver ?
http://support.xbox.com/en-US/kinect-fo ... dows-setup

Or is that only for the pc kinect? If so does anyone have link to the xb360/pc drivers?

The demo has a readme file that tells you all you need to know about installing (don't install the microsoft package):

For Kinect to work, TurboTuscany demo requires Win32-bit version of OpenNI 1.5.4.0 (OpenNI 2.0 is not supported).
Follow the Windows section’s “Install OpenNI, NITE, and the Sensor Driver” guide in the below page (be sure to install Win32 version):
http://code.google.com/p/simple-openni/ ... stallation
If the above doesn’t work, follow steps 1-5 in the “Manual method” section of this guide:
http://www.brekel.com/?page_id=170
Your installation is successful (and Kinect should work in TurboTuscany demo) if you run the NiViewer example and get it to display depth image from Kinect. Otherwise you should start googling for a solution using the error message that NiViewer prints out.

Really interesting demo that I wanted to try out last night with my xbox360 Kinect. I found the readme in the download and proceeded to download and install all the requisite files but no matter if I tried the first method or the manual method, I just couldn't get it working - it would not detect my Kinect. I am on windows 8 64bit if that makes any difference. After around 2 hours installing and uninstalling I gave up. I would have liked to try it with the move controller but I do not have access to move.me

[Fredz]

If there is no resolution selector at start you can use command line options to specify the resolution :
http://docs.unity3d.com/Documentation/M ... ments.html

-screen-width and -screen-height options specifically.

[TTakala]

Really interesting demo that I wanted to try out last night with my xbox360 Kinect. I found the readme in the download and proceeded to download and install all the requisite files but no matter if I tried the first method or the manual method, I just couldn't get it working - it would not detect my Kinect. I am on windows 8 64bit if that makes any difference. After around 2 hours installing and uninstalling I gave up. I would have liked to try it with the move controller but I do not have access to move.me

Sometimes OpenNI can be tricky to get to work. Sometimes it's a hardware/driver compatibility issue (Asus XTION Pro doesn't work in my Macbook's Win7 install for example).
Some people with 64-bit Windows 8 have gotten OpenNI with the following procedure (remember, the OpenNI still has to be the 32-bit version):

so windows 8 dose not like unsigned drivers,

Uninstall openni, nite, and the kinect driver.
Windows key + R to open the run prompt
shutdown.exe /r /o /f /t 00
select troubleshoot
select advanced
select windows startup and then restart
enter the option for Disable Driver Signature
reinstall openni, nite, and the kinect driver

I followed those instructions and it worked for me I suspect that only the driver needs to be re-installed, I also had to go in to device manager and make sure it was pointed at the right drivers because I had installed the oppenni sdk drivers and they don't work with processing.

Details here:
https://code.google.com/p/simple-openni ... tail?id=68

[cybereality]

Ha! Nice one.

[mrgrey]

Really interesting demo that I wanted to try out last night with my xbox360 Kinect. I found the readme in the download and proceeded to download and install all the requisite files but no matter if I tried the first method or the manual method, I just couldn't get it working - it would not detect my Kinect. I am on windows 8 64bit if that makes any difference. After around 2 hours installing and uninstalling I gave up. I would have liked to try it with the move controller but I do not have access to move.me

Sometimes OpenNI can be tricky to get to work. Sometimes it's a hardware/driver compatibility issue (Asus XTION Pro doesn't work in my Macbook's Win7 install for example).
Some people with 64-bit Windows 8 have gotten OpenNI with the following procedure (remember, the OpenNI still has to be the 32-bit version):

so windows 8 dose not like unsigned drivers,

Uninstall openni, nite, and the kinect driver.
Windows key + R to open the run prompt
shutdown.exe /r /o /f /t 00
select troubleshoot
select advanced
select windows startup and then restart
enter the option for Disable Driver Signature
reinstall openni, nite, and the kinect driver

I followed those instructions and it worked for me I suspect that only the driver needs to be re-installed, I also had to go in to device manager and make sure it was pointed at the right drivers because I had installed the oppenni sdk drivers and they don't work with processing.

Details here:
https://code.google.com/p/simple-openni ... tail?id=68

WOW, it worked! Thanks for the instructions.
It is amazing how much immersion is added with positional tracking and by being able to see your arms and feet in VR. You would not believe how long I spent crouching to look at things close up and swaying from side to side to see around barrels. Fantastic work on this!

You have given me some ideas to play with in Unity...as soon as I work out how to implement them lol

[urban]

This seems great. Wish I had a hydra. I do have a xbox kinect and ps3/move though but no idea how I would set it all up
First time I've heard of the Sony move.me app, that looks really good
But It does seem tricky to get hold of. Shame as it seems it could be really Handy for vr

[TTakala]

WOW, it worked! Thanks for the instructions.
It is amazing how much immersion is added with positional tracking and by being able to see your arms and feet in VR. You would not believe how long I spent crouching to look at things close up and swaying from side to side to see around barrels. Fantastic work on this!

You have given me some ideas to play with in Unity...as soon as I work out how to implement them lol

Excellent! Glad to hear you liked it
You don't necessary have to figure out everything by yourself: In a month or two we will release RUIS toolkit for Unity, the one which we used to create this demo. RUIS has a lot C# scripts and Unity prefabs to add all sorts of VR features, including the Kinect/Razer/PSMove head tracking, object selection and manipulation methods, and Kinect controlled avatars featured in this demo. If you're interested, keep eye on our website ( http://ruisystem.net/ ) or Facebook page: https://www.facebook.com/pages/RUIS-Pla ... 7233957298
Here's a bunch of games that students made using our toolkit:
http://www.youtube.com/watch?v=Ptq9F1nnUaI

[mrgrey]

WOW, it worked! Thanks for the instructions.
It is amazing how much immersion is added with positional tracking and by being able to see your arms and feet in VR. You would not believe how long I spent crouching to look at things close up and swaying from side to side to see around barrels. Fantastic work on this!

You have given me some ideas to play with in Unity...as soon as I work out how to implement them lol

Excellent! Glad to hear you liked it
You don't necessary have to figure out everything by yourself: In a month or two we will release RUIS toolkit for Unity, the one which we used to create this demo. RUIS has a lot C# scripts and Unity prefabs to add all sorts of VR features, including the Kinect/Razer/PSMove head tracking, object selection and manipulation methods, and Kinect controlled avatars featured in this demo. If you're interested, keep eye on our website ( http://ruisystem.net/ ) or Facebook page: https://www.facebook.com/pages/RUIS-Pla ... 7233957298
Here's a bunch of games that students made using our toolkit:
http://www.youtube.com/watch?v=Ptq9F1nnUaI

Thanks, will look forward to the release of RUIS.

[WiredEarp]

I just bought a Kinect and tried to get this going, but I didn't have any luck.

Your installation is successful (and Kinect should work in TurboTuscany demo) if you run the NiViewer example and get it to display depth image from Kinect. Otherwise you should start googling for a solution using the error message that NiViewer prints out.

I installed the 1.5, 32 bit versions of OpenNI, NITE, etc, and NIViewer runs fine - but TurboTuscany just says 'cannot find Kinect' or something like that.
Any tips on getting it going?

[WiredEarp]

I solved my problem by running TurboTuscany in 'administrator mode'. However, when I use it with the Kinect, it seems very jittery, my character seems to scuttle off in weird directions by itself, and my hands seem pretty flickery.

Is this just the way kinect is? If so, I don't see how people are getting much use out of it ;/

[TTakala]

I solved my problem by running TurboTuscany in 'administrator mode'.

Glad you got it working. What operating system are you using..?

However, when I use it with the Kinect, it seems very jittery, my character seems to scuttle off in weird directions by itself, and my hands seem pretty flickery.

Is this just the way kinect is? If so, I don't see how people are getting much use out of it ;/

Yeah, body tracking data from OpenNI is quite unreliable, and I don't believe it's much better with Microsoft Kinect SDK. Kinect 2 is coming out in a few months, and we have reasons to believe that its tracking is better and more usable.

Could you elaborate on "character seems to scuttle off in weird directions by itself"? If you use a gamepad (or Razer Hydra or PS Move) to make the Kinect controlled character walk, the forward direction is determined by where OpenNI thinks that your torso is facing. That works usually pretty well, especially if you are facing the kinect (+- 60 degrees).

[Fredz]

Kinect 2 is coming out in a few months, and we have reasons to believe that its tracking is better and more usable.

Unfortunately it doesn't seem like it'll be supported on PC : http://arstechnica.com/gaming/2013/06/m ... rk-on-pcs/

[cybereality]

The device that ships with the XboxOne will not work on PCs. But MS said there would be a separate Windows version in 2014.
http://blogs.msdn.com/b/kinectforwindow ... -year.aspx

[TTakala]

The device that ships with the XboxOne will not work on PCs. But MS said there would be a separate Windows version in 2014.
http://blogs.msdn.com/b/kinectforwindow ... -year.aspx

Wow, this went under my radar. Quite stupid for Microsoft's part.
We applied for the alpha Kinect 2, so hopefully we get our Windows Kinect 2 already in November.

[WiredEarp]

I fixed the random moving issue by unplugging all my other controllers - i think it was picking up something random from one. Also, by moving another meter or two back, I managed to reduce the flickering a little.

Its now 'cool' but still pretty unusable, mainly because the things I like to do with my hands in VR aren't possible. For example, rotating my wrists or flexing my fingers. I guess this is because the Kinect doesn't support tracking fingers etc?

[TTakala]

Its now 'cool' but still pretty unusable, mainly because the things I like to do with my hands in VR aren't possible. For example, rotating my wrists or flexing my fingers. I guess this is because the Kinect doesn't support tracking fingers etc?

Yup, no finger tracking in OpenNI or Microsoft Kinect SDK. Some people have made special software for that (close range usage), but getting that to work with Unity would be another story.

[WiredEarp]

Sweet. I'll give this demo another go tonight using the Hydra. I really like the virtual body, it reduces nausea a great deal.

[TTakala]

Sweet. I'll give this demo another go tonight using the Hydra. I really like the virtual body, it reduces nausea a great deal.

I like it too and even with the limitations of Kinect it adds a great deal of immersion.

[TheVividLight]

I have my PSMove recognized on my PC and I use it as part of my gun-troller at the moment, but when I try to use it on the demo (usign the same method) it doesnt want to work

So you're telling me no one has found a way around using the move.me server yet for an application like this? Or is there a easy (or more complicated) way to get the psmove to be recognized?

Also, I get quite a bit of drift from my rift, (always swayying to the left in a constant slow 360 degree motion) after about 10 seconds int eh game. very odd as it doesnt happen in any other game, and i cant seem to calibrate it to make it stop lol. I'll keep tinkering tonight as I'm sure I missed something or tweaked something wrong.

But in all honesty, my question was related to the psmove and the move.me software. there has to be a software solution already developed out there, as I know there are a ton of ways to get it to be used as a pointing device on PC w/o any move.me support

[TTakala]

I have my PSMove recognized on my PC and I use it as part of my gun-troller at the moment, but when I try to use it on the demo (usign the same method) it doesnt want to work

So you're telling me no one has found a way around using the move.me server yet for an application like this? Or is there a easy (or more complicated) way to get the psmove to be recognized?

Also, I get quite a bit of drift from my rift, (always swayying to the left in a constant slow 360 degree motion) after about 10 seconds int eh game. very odd as it doesnt happen in any other game, and i cant seem to calibrate it to make it stop lol. I'll keep tinkering tonight as I'm sure I missed something or tweaked something wrong.

But in all honesty, my question was related to the psmove and the move.me software. there has to be a software solution already developed out there, as I know there are a ton of ways to get it to be used as a pointing device on PC w/o any move.me support

There is MoveOnPC library that connects PS Move controllers directly to your PC via bluetooth:
https://code.google.com/p/moveonpc/

MoveOnPC does not replace Move.me, and our TurboTuscany demo can use PS Move only via Move.me software.

My understanding is that MoveOnPC can track 6DOF pose of only two simultaneous Moves (vs. four of Move.me), and that its tracking is not as good as on Move.me (I haven't tried though). It's a non-trivial problem to implement a really good sensor fusion for PS Move.

Can you tell me more about the drift? If you are using Kinect, disable the "Use Kinect For Drift Correction" from the menu (access via ESC). You can also try the Manual or Automatic Rift calibration.

[Fredz]

There is also the PS Move API that is able to track up to 5 controllers via OpenCV :
http://thp.io/2010/psmove/