A million dollar idea

Post Reply
User avatar
Libertine
Binocular Vision CONFIRMED!
Posts: 204
Joined: Wed Jan 11, 2012 1:06 pm

A million dollar idea

Post by Libertine »

I thought of a great idea after watching the video demonstrating the very precise Control VR demo with all the guys interacting with each other. Something felt like it was sort of missing in terms of the human to human ability to interact and have a meaningful amount of presence to one another, but i thought of the perfect idea for Oculus, especially concerning the Facebook/social end of things. Which is.... to put micro cameras within the interior of the Rift to record the expressions the eyes and eyebrows make, the entire area actually, as well as one or two under the rift (via long brackets) to record the mouth and face for smiles, frowns, laughter, etc. It would be a one piece design that simply snapped in to the main body. This along with a setup program to properly capture the parts of the face that will be hidden from the cameras once the rift is put on, such as the forehead and nose, which would then be blended together with the recording of the eyes/eyebrow area and sent all together as the finished frame to the game engine.

The setup program could begin by having you first putting the Rift on and taking a photo of the eyes/eyebrows and mouth with the face at rest, then having you take the rift off. Step 2 would be taking video of the the forehead, with the aid of an on-screen positional aid. It would instruct the user to start with their eye brows at rest, then move them up as much as they could, then in towards each other as much as they could -to capture various conditions of the forehead as the skin ripples together. Step 3 would be asking the user to center the recorded forehead image over a trapezoidally shaped area (or opposite, the trapezoid over the video). Step 4 would be a similar process with the nose.

Using this information, the software would dynamically update the photo of the forehead and nose area, the areas the cameras don't capture, based on the condition of the eye brows, most importantly their vertical position, as compared to the reference frames capture by the setup software in step 2. Same for the nose area hidden by the Rift, which would be affected by the movement of the mouth and jaw.

I say two cameras below the Rift (4 in total) because you'd probably miss the sides of the mouth and face with only one.

To get the body of the eye/eyebrow cameras out of the way, you could use mirrors.

Instead of inserting cameras within the Rift itself, you could design future Rifts to except a lightweight metal or plastic template that snapped in inside viewport body of the rift perhaps underneath'ish the cushion -that held the cameras, thus making it an optional accessory.

Just imagine the heighten ability to emotionally interact with your friends. Like your sneaking into a dungeon and a huge thing crosses your path in the distance and you look back at your friend [in India] and smile and give a HO... LY... crap look and a smile.

If you make a quadrillion bucks off of this idea, I beseech thee to send me a million, deal? I've got a bunch of OR inventions i'd like to get made, but I unfortunately don't have the faintest interest in almost anything that makes one money.... :? Although perhaps this idea isn't a new one... :cry:


Dave: "You know what?"

Carl: "Dave, i know that look, not today okay......DAVE!"

Dave: LLLLLLLLLEEEEEEEEEEEEEEEEEEEEEEEEEEEEERRRRRRRRRRROOOOOOOOOOOOOOOYYYYYYYYYYYYYYYYYY JJJJJJJJJJJJJJJJJJJJJJJEEEEEEEEEEEEEEEEEEEEEEEENNNNNNNNNNNNKKKKKKKKKIIIIIIIIIIIIIIINNNNNNNNNNNNSSSSSSSSSSSSSSSSSS!!!!!!!!!!!!!!!!!!!
Last edited by Libertine on Tue Jun 24, 2014 11:02 am, edited 1 time in total.
User avatar
colocolo
Diamond Eyed Freakazoid!
Posts: 790
Joined: Mon Jun 04, 2012 1:25 am

Re: A million dollar idea

Post by colocolo »

and what about nude capture?
Leahy
Cross Eyed!
Posts: 181
Joined: Thu Jun 07, 2012 1:36 pm
Location: France

Re: A million dollar idea

Post by Leahy »

I know the Emotiv EPOC allows rather limited facial recognition from mostly just the pair of electrodes on each side of the upper part of the face. I have wondered if they coupled electrodes to the temples out side the rift along with a cushion that could detect specific stretch points along its length if they could calibrate reliable facial recognition from that.
User avatar
GeraldT
Certif-Eyable!
Posts: 1057
Joined: Fri Jan 18, 2013 9:10 am
Location: Germany
Contact:

Re: A million dollar idea

Post by GeraldT »

not exactly a new idea though. and with eye tracking there actually already exist solutions where it has been integrated into a DK1.

but your facial expressions might be a bit changed when you wear a headset. so I guess the tracking abilities will always be limited.

tracking your mouth and eyes and putting those informations onto an avatar should be sufficient to create a believable situation I guess.
want to demo the Rift or check it out? click here
User avatar
Libertine
Binocular Vision CONFIRMED!
Posts: 204
Joined: Wed Jan 11, 2012 1:06 pm

Re: A million dollar idea

Post by Libertine »

colocolo wrote:and what about nude capture?
I'd like to see that advertisement...

Maybe we should wait for Asimov's physical assistance? (yuck what?)
FMPrime
One Eyed Hopeful
Posts: 23
Joined: Wed Sep 25, 2013 12:36 am

Re: A million dollar idea

Post by FMPrime »

I think these guys have scooped your idea.

http://www.youtube.com/watch?v=y5FM57A3My0
hast
Cross Eyed!
Posts: 100
Joined: Mon Mar 15, 2010 8:16 am

Re: A million dollar idea

Post by hast »

I think Palmer talked about a related concept in one video. The idea there was to put sensors on the mask that faces your skin and measure muscle movement and stuff like that to make a rough estimate of eye or face movement.

But if the result isn't good enough I recon you might get some serious uncanny valley effects though. :-)
User avatar
IanBruce
One Eyed Hopeful
Posts: 34
Joined: Tue Jul 01, 2014 7:29 am

Re: A million dollar idea

Post by IanBruce »

It's called surface electromyography -- you can find some papers on the subject from Boston University and others. Everybody's face is different, but I wouldn't be surprised if simple surface electrodes or cap sensors in the mask couldn't code for simple elements like eyebrows, frowns, or smiles. You get mouth movement straight from the audio channel.
User avatar
Libertine
Binocular Vision CONFIRMED!
Posts: 204
Joined: Wed Jan 11, 2012 1:06 pm

Re: A million dollar idea

Post by Libertine »

Yikes, i'd say photo-realism'ish or bust. I just don't see the point of anything less than that. Although i've never seen the results of sensors detecting muscle movement, but i've never seen convincing voice-capture-->lip movement. If Hollywood can barely do it on production films with keyframe animation, i don't see automatic software doing that well with blend-shapes, bones or whatever.


No no. I meant like this level of quality (and similar technology), in L.A. Noire:

https://www.youtube.com/watch?v=aL9wsEFohTw

You can see more here viewing some of the many interrogations in the game.

https://www.youtube.com/results?search_ ... errogation

Heres a really good example:

https://www.youtube.com/watch?v=vFZehZhEjg4

Not to say that those other technologies wouldn't work. I just think it would require a ton more research and development. Not that im any expert. But even if it took many more cameras to achieve, i think it could be worth it since i don't think the capture resolution has to be super high, keeping the costs down i would think.
WiredEarp
Golden Eyed Wiseman! (or woman!)
Posts: 1498
Joined: Fri Jul 08, 2011 11:47 pm

Re: A million dollar idea

Post by WiredEarp »

Its not a new idea, but its one that still has to be done, and will be at some point. I'm uncertain from your description whether you are talking about pushing the exact pixels you capture onto the model, or whether you are talking about using that camera data to determine what the facial positions are and applying that to an avatar. I see the avatar route as being much more practical and desirable.

Its definitely the next step in immersion IMHO - one day soon we'll look back at our expressionless avatars and people will be shocked that we once got by without expression capture. One thing you'd probably find is, if you were successful, you'd find your market would disappear in the next HMD version, once they stole your idea.
I've got a bunch of OR inventions i'd like to get made, but I unfortunately don't have the faintest interest in almost anything that makes one money.... :?
Ah, yes. I'm an ideas guy, more than an applying those ideas guy myself (there just isn't enough time to implement more than a small percentage of them). If people would even pay me 0.5% of their profits based on any of my ideas, I'd be a happy man!
User avatar
Libertine
Binocular Vision CONFIRMED!
Posts: 204
Joined: Wed Jan 11, 2012 1:06 pm

Re: A million dollar idea

Post by Libertine »

Yeah, i was talking about putting the actual video onto the model, similar to how they did it in L.A. Noire.

I think this is near the best you'd get using a standard facial rig, especially with automated signaling. This is from Mass Effect 3, a AAA game with high quality head models and textures and a heavy emphasis on dialogue and you can see it doesn't compare well to the L.A. Noire result.
https://www.youtube.com/watch?v=Vs6R14d2x9U

Using the setup or a similar concept to modify bones and blend shapes in existing avatars would work well for non-human characters though, or even humanoid ones, it just wouldn't be a "holy sh!t were in a video game together man!" sort of an experience i don't think.
WiredEarp wrote: Ah, yes. I'm an ideas guy, more than an applying those ideas guy myself (there just isn't enough time to implement more than a small percentage of them). If people would even pay me 0.5% of their profits based on any of my ideas, I'd be a happy man!
It just doesn't seem fair, when major manufacturers have "got a great idea for us? Put it down here." forms on their webpages. I have a great idea that could almost wipe out an industry (not a bad thing if our societies had a "guaranteed basic income" and job sharing/splitting) made of newly available technology put together and automated, but General Electric only give $5000 dollars at the most, period. It would make me millions but im too lazy (rather, slave to immediate gratification due to childhood depression to be more accurate) and will probably just give it away. Shame, it would be incredibly cool and make the news all over the world. Hint: long robot arms + house exterior + laser environment mapping + 3d software surface verification + ...., hrmmm...
WiredEarp
Golden Eyed Wiseman! (or woman!)
Posts: 1498
Joined: Fri Jul 08, 2011 11:47 pm

Re: A million dollar idea

Post by WiredEarp »

Yeah, i was talking about putting the actual video onto the model, similar to how they did it in L.A. Noire.
Ah, yep. I haven't seen LA Noire actually ;/ The reason I thought its better to analyze the picture instead and extract the features, is that then you can have multiple avatars, all with expressions - you aren't restricted to only using your face. TTT I don't think its necessary to have the super high fidelity you are mentioning, I think that avatars with things such as eye raises and rolls, smiling, etc, will be 'good enough' for now. However, for max quality, like if you were trying to make a VR that was as realistic as possible, then it could be superior. However, wouldn't you need to deform the geometry to match? Otherwise it might look weird from some angles. For example, you couldn't see a silhouette of the lips moving, if looking at them from side on with a bright wall as the background - if the model doesn't deform to match the lip movements the silhouetted lips would be static despite the textured lips animating over the face.

I remember first seeing this type of stuff years back at expo '88, where one of the pavilions had a statue with a projected face that spoke. Quite realistic, but I think feature extraction is the future personally. Easier to send (possibly less bandwidth even with compression of sending the whole texture), mappable to anything, etc. Lots of people would not be happy with putting their faces on their avatars in WOW, etc, so your method could prevent them from being able to appear with orc facial features, etc.
User avatar
Libertine
Binocular Vision CONFIRMED!
Posts: 204
Joined: Wed Jan 11, 2012 1:06 pm

Re: A million dollar idea

Post by Libertine »

I agree that many people wouldn't at all want their faces in games, but when i thought about this I was thinking more along the lines of the social spectrum of Facebook, where, if the Rift becomes more mainstream, with The Sims (I think that is the social game) support and perhaps 3D movies and youtube content, it would make for an amazing way for, say a mother and daughter, to connect on the weekends or some good friends to play coop together. I do think you would need be able to customize a face and also make available a set of pre-configured faces to choose from for expediency. I think it would be ideal if whoever developed the product to create functionality for both features, as i'm sure you can do avatar facial manipulation by tracking the eye brows, eye shape and mouth using the exact same video capture, capturing min/max values using another setup program.
WiredEarp
Golden Eyed Wiseman! (or woman!)
Posts: 1498
Joined: Fri Jul 08, 2011 11:47 pm

Re: A million dollar idea

Post by WiredEarp »

I agree both would be good to cover. You could also get pretty realistic avatars with a 3d scan of the user, which was then manipulated by the facial inputs.

This stuff certainly the future, its a matter of how soon, not if.
Post Reply

Return to “Oculus VR”