Oculus Virtual Lounge (VirtuaView)

Post by **Gunshot** » Thu Sep 06, 2012 12:03 pm

Also, this whole concept reminds me so much of the book Ready Player One. real good, described as alice in wonderland meets the matrix.

Post by **FingerFlinger** » Thu Sep 06, 2012 12:30 pm

You know, I think this is a really good concept, Namielus! Unlike a generic virtual lounge, there is a specific purpose to it. And once you get people all in the same place, they can't help but talk to each other. I think that a virtual lounge by itself could be a bit awkward until people become used to it.

Post by **Namielus** » Thu Sep 06, 2012 12:56 pm

Yes I think this sort of thing needs a spesific purpose, like "we have this concept, lets go into the lounge and field test it"

Gunshot; you nailed it! Somebody give him a prize !

Post by **MrGreen** » Thu Sep 06, 2012 2:13 pm

Namielus wrote:MrGreen, thanks. I'll give some hints after I get some more guesses.
Btw, are you by any chance MrGreen, the producer?

I can't say I am!

Post by **Zaptruder** » Thu Sep 06, 2012 8:38 pm

This is of course one of the most natural extension ideas of a VR interface. Creating virtual environments for social interaction.

And it's going to be big money as well - I can forsee in the future that many businesses will create virtual offices with this sort of interaction over physical offices - with the benefits been reduced cost, allows them to cater to larger geographical area, and more office design variety and flexibility (I mean, it really doesn't even have to be tied to the normal notion of a physical office).

Some ideas that you'd want in either scenario - as a virtual lounge and or office.

Full body camera emote - cheap motion capture tech at its best. I think this one factor will change the viability of avatar communication immensely.

Software pass through - some sort of system that is able to emulate (or capture the video frames of) any program that can run on your computer normally. Allows you to use legacy software easily - but also means you don't have to reinvent the wheel for every function out there.

Of course this means you can move and drag the virtual windows and resize them all around your body in a manner that isn't possible on the real desktop - providing the application with a pretty compelling user feature right off the bat.

Post by **Gunshot** » Fri Sep 07, 2012 1:27 am

I can imagine multiple people viewing the same movie online in such a fassion would piss off the mpaa. "did they all pay for this movie!? Pirates!!" regardless I hope this works.

Post by **Okta** » Fri Sep 07, 2012 1:41 am

So.... if everyone is watching a video stream, does it halt for everyone every time the weakest link buffers?

Post by **Namielus** » Fri Sep 07, 2012 6:27 am

Okta, I guess that depends on how important it is for it to be completely synced.
It is hard to discuss/comment if people are viewing the video stream at different points in time.
For live events, i guess the person buffering would have to skip forward.

Post by **BHawthorne** » Fri Sep 07, 2012 11:58 am

MemeBox wrote:It would be possible to add 3d scans of objects into the shared space. Take a look at http://reconstructme.net/blog/, it scans objects using the kinect

What you are talking about is called photogrammetry. There is a forum that is focused on the technology at http://www.pgrammetry.com/forum/

Lots of information to digest there for recreating point clouds of real objects and locations. Also some talk about AR. Most of their stuff isn't realitime. TO go realitime you need to focus down on recent Kinfu development using the microsift Kinect and PCL. http://pointclouds.org/

Personally, I just installed Autodesk's 123D Catch on my iPhone 4S. That'll allow you to capture objects and mesh them using just your iPhone. Handy when you want to be mobile.

Would be very cool to see someone combine a Kinect and a Rift and see a true AR appliance.

Post by **FingerFlinger** » Fri Sep 07, 2012 12:28 pm

@BHawthorne I just saw that they released the iPhone version; how has your experience been with it so far? I'm going to play with it after work today.

Post by **Namielus** » Fri Sep 07, 2012 12:31 pm

Fingerflinger, If you can stand with your hands out legs spread, fingers spread, try to make a model of yourself, have someone take the pictures.

You can be the first model to walk around in the thing , I will try to get someone to rig the model so it can be animated.

Like this is optimal for rigging all your limbs. But Fingerflinger... PLEASE WEAR CLOTHES LOL!

I dont want to see a detailed model of your private parts hahhhaha

Edited to a more family friendly picture

Post by **Laserschwert** » Fri Sep 07, 2012 6:16 pm

If the virtual lounge is supposed to have the users talk to each other, what would be a low latency voice chat? What's the latency of Skype when phoning from one side of the world to the other?

Post by **Namielus** » Fri Sep 07, 2012 6:25 pm

I skype with peope in america with no real problems, I am in europe.

I think voice chat will be pretty equal to motion/interaction in terms of how high the latency can be before its a problem. The virtual lounge will be far away from a fast paced fps, so it can handle quite alot of latency.

As long as you got something like skype that has intelligent filtering to do noise cancelling and avoid echo I think its going to be fan.

Post by **Pyry** » Sat Sep 08, 2012 3:23 pm

You could use Mumble for the voice chat and build support into the application for Mumble's positional audio, so people would sound like they're talking from their virtual locations.

Post by **AugmentedR** » Tue Sep 25, 2012 3:52 pm

Namielus wrote:Fingerflinger, If you can stand with your hands out legs spread, fingers spread, try to make a model of yourself, have someone take the pictures.

You can be the first model to walk around in the thing , I will try to get someone to rig the model so it can be animated.

Like this is optimal for rigging all your limbs. But Fingerflinger... PLEASE WEAR CLOTHES LOL!

I dont want to see a detailed model of your private parts hahhhaha

Edited to a more family friendly picture

Do you use blender for this?

Post by **Additives** » Sat Sep 29, 2012 3:54 am

Feel free to completely trash this but...

Wouldn't a good place to start be a small room or building from which to develop the rest of the world?

I must, in the interests of full disclosure, say that I just read Snow Crash for the first time (shame on me for not having been exposed to it earlier) and the whole Metaverse as it is in that book seems just cool...and not to far from what the Oculus will be...seriously, go re-read the specs of the in book goggles with a page of rumoured specs for the Rift up, and it gets a little creepy.

I raise this here with the thought that creating something on that scale is way beyond one person, but it is, really, what a lot of the people on this form seem to want, at least as an option. Refer to my above statement of the 'cool'ness of the idea.

The thing is, that the people with the skills and the drive to make this a reality seem to be right here, on this forum. In this thread! A way to work together in a more intuitive manner than web communication would be an immensely powerful tool to bring together, at first, the key people needed to make all the things that we want from a VR space.

TLDR: Virtual dev studio so we can all work together to build the Metaverse...damn do i feel like this is a bit of a pipe dream. Buy you know, it would be cool.

Post by **nanicoar** » Sat Sep 29, 2012 9:37 am

I hope it will be an assembly of open source software. I'm rooting for Blender as a collaborative modeling platform, where you and others can see what you build as you build it. I don't think any one platform will dominate; instead you define a home space and bring your content into it.

For example you could open your smartphone as a remote desktop and position its UI in your home space as its position is read by its sensors.
The touch-screen would still function, so some interesting hand-eye coordination could take place but you wouldn't have to take off your goggles to call somebody. It could also be a versatile tool, used for modeling, aiming, controlling vehicles, or it could simply be used for virtual body language. Animating avatars this way could be easy.

Blender has python bindings, so development should be easy for beginners like myself. I'm currently investigating the open source Flight Gear simulator and Rigs of Rods, which is an all-purpose vehicle simulator with advanced physics, also open source.

Until the consumer version is released people will be building their virtual worlds as best they can. Then by yule next year you'll have lightsabers, tricoders and phasers in Skyrim.

Post by **Namielus** » Wed Oct 03, 2012 6:01 pm

Can somebody help me budget how much I need for hiring a freelancer just for the cinema demo + getting my mocap animation into that?

I still dont know what engine to use, but I am opening to hiring someone. I just don't know how much to expect for this work

A) Interface in game to choose content
B) correct aspect ratio/FOV/distortions/custom IPD and perhaps custom FOV for supporting more hmds
D) Head tracking
C) Being able to stream as many formats as possible to the screen as a video texture, hopefully 3D compatible.

I will make the 3D models, both rooms and characters + animations.

Post by **Namielus** » Sun Oct 14, 2012 7:28 am

Its so annoying that I don't have a Rift to test with. We have a working demo now, but it has not been optimized and there is no lighting etc.
The guy helping me had to make a clone of my model himself from scratch because my fbx export generated too many polygons, so that will also be improved.
Will find a way to export correctly with optimized polygon count.
I also made more space between the seats and the screen for this than the first renders, and it seems that was a mistake without making the size bigger in the process.
Its just too small and far off.
The space is supposed to be for a motion captured NPC to demonstrate the potential for conferences/lectures etc. Its a guy juggling lesson and will be pretty fun to watch if we get it right.
This is supposed to be like a very large home cinema, or smaller type of conference room. Obviously I will be able to make it more like a real cinema if that is desired instead.

[youtube-hd]https://www.youtube.com/watch?v=52SIrp63am0[/youtube-hd]

Does this really look like a widescreen in the Rift? It looks so narrow.
I hope someone can confirm that, and ignore the lack of distortion.

*Need to put in a ceiling and lights, and capability to fade the light out when movie starts
*Adjust the textures that are too large and get the bump mapping working
*Just add "stuff" to make it more realistic

As soon as we get the distortion right I can upload a demo, you need to convert your own video files to Bink format to be able to watch whatever you want.
The codec/format support is still the number one challenge here.

Post by **Okta** » Sun Oct 14, 2012 7:49 am

Thats cool man good work

Will it have a sit toggle button and avatar models?
Can i kick the chair in front and shake the person in fronts view?

Post by **Namielus** » Sun Oct 14, 2012 7:53 am

Working on a multiplayer part but its not a priority, to me the most important thing is figuring out how to have the broadest format support possible.
Yes there will be a toggle sit/stand , but hopefully just a simple "choose row/seat" map too so you dont have to walk around.

I will make sure the viewing angles in practice is the optimal THX recommendation when sitting in the middle seat middle row.

Post by **2EyeGuy** » Sun Oct 14, 2012 7:56 am

To get an idea how it will look on a rift, measure the total width of that image, and then put your eyes one quarter of that distance away from the screen. Of course it won't be blurry or feel so close on a real rift, but you will get a feeling of the aspect ratio and FOV.

Are you rendering at the right FOV? Remember it's not half-SBS, it's supposed to be full unsquashed SBS. So the horizontal FOV for each side should be 90 and the vertical FOV for each side should be about 110. If you're rendering at the right FOV, and the virtual screen is the right size, it should look the right size on a rift.

I'm not getting a Rift until December or January, so I can't help much more than that. Hopefully someone has a DIY rift or a prototype though and will try it.

EDIT: After watching that video, I'm pretty sure it's not right. The virtual screen will only look 16:9 on the rift if it looks 16:9 on the screenshot with your normal eyes. The Rift uses square pixels.

Post by **2EyeGuy** » Sun Oct 14, 2012 8:02 am

I can try and have a go at making a straight Direct3D version of it for you with no engine, if you want. Then you could use DirectShow to play pretty much any video format. I don't have the skills to add juggling avatars though.

Post by **Namielus** » Sun Oct 14, 2012 8:07 am

2EyeGuy wrote:Remember it's not half-SBS, it's supposed to be full unsquashed SBS.

I have hired a guy helping me that is doing all the programming, so its his job to get the fov right based on my instructions.
Never used UDK, only Unity, so I might have to start quickly learning UDK. It looks too narrow to me.
Like you said squashed. I assume this might be 90 horizontal FOV in full view squashed into one half?

2EyeGuy wrote:I can try and have a go at making a straight Direct3D version of it for you with no engine, if you want. Then you could use DirectShow to play pretty much any video format. I don't have the skills to add juggling avatars though.

what about the head tracking etc? It sounds interesting, but apart from the video formats, enginges like Unity and UDK will hopefully have built in support for those things.
I might be interested in getting help from you anyway, and I am in contact with some other people too.

Post by **marbas** » Sun Oct 14, 2012 8:16 am

@Namielus
God job, looking good. The demo doesn't seem to do the distortion and stereoscopic processing required to be Rift compatible. Are you adding that later?

Post by **Namielus** » Sun Oct 14, 2012 8:19 am

Yes, hopefully as soon as possible. I just need the programmer guy to figure out it. Going to try to fix the squeesed aspect ratio before that for a slightly better video.

Post by **2EyeGuy** » Sun Oct 14, 2012 8:33 am

Head tracking would be easier in straight Direct3D programming, since that's what the SDKs are always designed for first. Avatars, big levels with lots of rooms, outdoor areas, physics, and things like that are the hard part. Advanced lighting, reflections, and shadow effects can also be hard for people like me.

Post by **Namielus** » Sun Oct 14, 2012 9:32 am

UDK guy is telling me the vertical FOV cannot be set independently from horizontal FOV. Does this mean correct aspect ratio cant be done yet?

Post by **2EyeGuy** » Sun Oct 14, 2012 9:44 am

Probably what you want to do is either render to a viewport or a texture that's the right size for one eye (640x800), and set either the horizontal or vertical FOV to the correct value (90 or 110, I think). The FOV of the other dimension will probably be set right too automatically if UDK assumes square pixels. But I don't know because I've never used UDK.

Post by **Namielus** » Sun Oct 14, 2012 10:05 am

2EyeGuy, Lets say worst case I cant get this to work with UDK even after the oculus rift sdk, how much do I need to hire someone to do all this?
If I hire you to do whatever you are able to, and then you help me finding someone else to do the other stuff like NPC animation etc. I have some offers already from different people, but none
seem to be able to do it all.

Do you think I need 2000, 5000 , 10 000, 50k ? 100k? I really have no idea of what I need to have as a goal to get a platform going that works the way I want.

Post by **mahler** » Sun Oct 14, 2012 2:07 pm

I saw this video on the same channel

[youtube-hd]http://www.youtube.com/watch?v=D1_F18ohUso[/youtube-hd]

Is this from the same guy helping you or did you create this before?

Post by **Namielus** » Sun Oct 14, 2012 2:15 pm

Its his model, but there is no problem making graphics that level. Don't get stuck on that, this is just the test modelling.
He had to remake the whole smaller cinema model, and did it quickly. Like I said, there is no problem at all making something that size and obviously the shading/textures and lighting will be better looking. Alot better.

Post by **mahler** » Sun Oct 14, 2012 2:23 pm

Namielus wrote:Its his model, but there is no problem making graphics that level. Don't get stuck on that, this is just the test modelling.
He had to remake the whole smaller cinema model, and did it quickly. Like I said, there is no problem at all making something that size and obviously the shading/textures and lighting will be better looking. Alot better.

I understand, I was just wondering. It does give a good impression on how amazing this might become.

Post by **Namielus** » Sun Oct 14, 2012 2:25 pm

I can upload more rendered shots from my version that looks better if you want, I just made it to look a bit more "private".
That cinema is gigantic.

Post by **TheLookingGlass** » Sun Oct 14, 2012 2:33 pm

Great work Namielus! I am an avid gamer & can't wait to play games with the Rift but I am most excited to see what kind of home theater experiences developers can provide with the Rift! For example, in a virtual environment, such as your movie theater demo, screen size & shape is boundless! This means we could screen movies that were partially or fully filmed with large format cameras/film! i.e. The Dark Knight/Rises(IMAX) or How The West Was Won(Cinerama) Films like The Dark Knight or Mission Impossible:Ghost Protocol(IMAX) had to have their aspect ratios modified for home video release because of the current size/shape limitations of modern HDTVs. In a virtual environment I imagine that large format movies could be screened as originally intended and provide that same feeling of immersion that we can only get by driving to the nearest large format movie theater (rare birds these days!)

Post by **2EyeGuy** » Sun Oct 14, 2012 9:05 pm

I really have no idea how much it should cost. It could be anywhere between free and tens of thousands depending on who you hire, what you want to use their work for, and things like that.

Post by **jayoh** » Mon Oct 15, 2012 4:15 pm

some really nice progress in less than a week. keep it up!

Post by **Bretspot** » Wed Oct 17, 2012 3:25 pm

This would be great if you could feed it a youtube 3D stream and it would show the correct image for each eye.

Post by **rhinosix** » Wed Oct 17, 2012 7:07 pm

This project looks like it's coming along nicely Namielus. I'm sure something as simple as watching movies in a virtual lounge will be reason enough for a lot of people to buy a consumer Rift.

I can't wait to see the different themed environments people start to build off of this idea - drive ins, famous theatres from around the world, etc.

Branching off your project (and the virtual desktop), another simple but effective concept I was thinking of today is for playing old console games through emulators. We could make a living room from the era in which a console was released filled with nostalgia from that period.

So for the Master System/ NES you could have a suburban home with posters for 80s/90s films, action figures and toys, VHS player, products from infomercials, decor and furniture from that period. If you really wanted to go extreme you could add full details with an outdoor environment visible from the window, ambient sounds from an overhead fan, a distant lawnmower, birds, a television with old cartoons, news and commercials.

It would be a kind of meta emulator.

With hand tracking it would be nice to have all the games in boxes on a shelf which you could pick up. But even without, you could use a controller to cycle through games, select a box, and tap a button to put it in the console and start playing.

It's something which seems completely feasible. Just modelling, texturing, a virtual screen + emulator. You wouldn't need any complex mechanics or additional hardware. And I think nostalgia is a very strong emotion which people would buy a Rift for.

Time to get back to Unity tutorials...

Post by **raijinspecial** » Wed Oct 17, 2012 10:13 pm

rhinosix wrote: So for the Master System/ NES you could have a suburban home with posters for 80s/90s films, action figures and toys, VHS player, products from infomercials, decor and furniture from that period. If you really wanted to go extreme you could add full details with an outdoor environment visible from the window, ambient sounds from an overhead fan, a distant lawnmower, birds, a television with old cartoons, news and commercials.

YES!