Augmented Reality @ HPG 2012 Conference in Paris

Talk about Head Mounted Displays (HMDs), augmented reality, wearable computing, controller hardware, haptic feedback, motion tracking, and related topics here!
Post Reply
JonOlick
One Eyed Hopeful
Posts: 7
Joined: Fri Jun 01, 2012 6:04 pm

Augmented Reality @ HPG 2012 Conference in Paris

Post by JonOlick »

Hi everyone!

I'll be doing a presentation this year @ HPG 2012 about computer vision and computer graphics. Its going to have some cool augmented reality stuff in there set in the Firefall universe (http://firefallthegame.com), so come by if your in the area and check it out! :)

- Jon
JonOlick
One Eyed Hopeful
Posts: 7
Joined: Fri Jun 01, 2012 6:04 pm

Re: Augmented Reality @ HPG 2012 Conference in Paris

Post by JonOlick »

The video from the conference was released =D

[youtube-hd]http://www.youtube.com/watch?v=yeXWmKzbFwI&hd=1[/youtube-hd]
User avatar
FingerFlinger
Sharp Eyed Eagle!
Posts: 429
Joined: Tue Feb 21, 2012 11:57 pm
Location: Irvine, CA

Re: Augmented Reality @ HPG 2012 Conference in Paris

Post by FingerFlinger »

That is rad. What are you using for head tracking? Is it just the camera on the headset addressing to the play field markers? I didn't really notice any glitching either; really cool!
zoost
One Eyed Hopeful
Posts: 29
Joined: Fri Jun 08, 2012 5:41 am

Re: Augmented Reality @ HPG 2012 Conference in Paris

Post by zoost »

Incredibele! So cool. Especially the bomb crater. Can you tell anything about the hardware set up? Much respect!
User avatar
cybereality
3D Angel Eyes (Moderator)
Posts: 11407
Joined: Sat Apr 12, 2008 8:18 pm

Re: Augmented Reality @ HPG 2012 Conference in Paris

Post by cybereality »

Very cool. Tracking looks spot on too, I did not notice any jumps at all.
User avatar
Chriky
Binocular Vision CONFIRMED!
Posts: 228
Joined: Fri Jan 27, 2012 11:24 am

Re: Augmented Reality @ HPG 2012 Conference in Paris

Post by Chriky »

The tracking is amazing! What hardware are you using? Can you make a demo video really pushing the tracking? Like, just a cube on the floor with the camera moving rapidly. I'm really impressed; the game looks pretty decent too.
zoost
One Eyed Hopeful
Posts: 29
Joined: Fri Jun 08, 2012 5:41 am

Re: Augmented Reality @ HPG 2012 Conference in Paris

Post by zoost »

I ran into this thread:

http://www.firefallthegame.com/communit ... ots.43470/

Where you wrote:
All code was written from scratch in just under one month's spare time of my brother Dave and I It uses a markerless AR system derived from PTAM (ours is better IMO). Most of the art is from the PC game (with permission), but a few are from my brother Matt. The game was developed on the PC with attention to the limitations of an iPhone or Android in mind when making it (for an easy port). In other news though Mark Kern may be setting up a deal with a certain HMD vendor - nothing official yet though =D.
Can I conclude that you made this for handheld / Phone, with no markers? Does your version of PTAM work on android and / or iphone / ipad? Looking forward dowloading it on my ipad ;-)

http://ewokrampage.wordpress.com/
JonOlick
One Eyed Hopeful
Posts: 7
Joined: Fri Jun 01, 2012 6:04 pm

Re: Augmented Reality @ HPG 2012 Conference in Paris

Post by JonOlick »

zoost wrote:I ran into this thread:

http://www.firefallthegame.com/communit ... ots.43470/

Where you wrote:
All code was written from scratch in just under one month's spare time of my brother Dave and I It uses a markerless AR system derived from PTAM (ours is better IMO). Most of the art is from the PC game (with permission), but a few are from my brother Matt. The game was developed on the PC with attention to the limitations of an iPhone or Android in mind when making it (for an easy port). In other news though Mark Kern may be setting up a deal with a certain HMD vendor - nothing official yet though =D.
Can I conclude that you made this for handheld / Phone, with no markers? Does your version of PTAM work on android and / or iphone / ipad? Looking forward dowloading it on my ipad ;-)

http://ewokrampage.wordpress.com/
yup :) All true. The tracking uses only the camera itself and with no markers what-so-ever. It most definitely can work on an iPad, but the final platform it will land on is unannounced at this time. ;) The HMD vendor is silicon micro display for future hardware models :)
JonOlick
One Eyed Hopeful
Posts: 7
Joined: Fri Jun 01, 2012 6:04 pm

Re: Augmented Reality @ HPG 2012 Conference in Paris

Post by JonOlick »

cybereality wrote:Very cool. Tracking looks spot on too, I did not notice any jumps at all.
The camera you see in that video is actually a virtual steady-cam running at about 120fps. It looks and plays absolutely awesome in person :) MTBS. For the HMD though I'm currently thinking a motion measurement device similar to the ones used in standard VR to improve inter-camera-frame tracking would be better in terms of immersion.
druidsbane
Binocular Vision CONFIRMED!
Posts: 237
Joined: Thu Jun 07, 2012 8:40 am
Location: New York
Contact:

Re: Augmented Reality @ HPG 2012 Conference in Paris

Post by druidsbane »

Any papers on what you guys did? Would it be very hard to get as good a result using just PTAM or do you need to have a steady-cam for best immersion?

Very cool demo. Is it actually fun to play this way though? I could certainly see players wearing VR headsets playing in the virtual world and then having spectators on the outside watching the game from the outside on a table or something using AR as you showed :)
Ibex 3D VR Desktop for the Oculus Rift: http://hwahba.com/ibex - https://bitbucket.org/druidsbane/ibex
JonOlick
One Eyed Hopeful
Posts: 7
Joined: Fri Jun 01, 2012 6:04 pm

Re: Augmented Reality @ HPG 2012 Conference in Paris

Post by JonOlick »

druidsbane wrote:Any papers on what you guys did? Would it be very hard to get as good a result using just PTAM or do you need to have a steady-cam for best immersion?

Very cool demo. Is it actually fun to play this way though? I could certainly see players wearing VR headsets playing in the virtual world and then having spectators on the outside watching the game from the outside on a table or something using AR as you showed :)
There is, but it doesn't look like its online yet? I don't cover the improvements over PTAM in the presentation too much as the audience wasn't a computer vision audience and probably wouldn't be too interested in the details there. Sufficed to say, PTAM is from 2007 IIRC and it has some very clear short-cuts in what they did that can be improved. First, their choice of corner detector and feature descriptor are not the best - not even close. There has been advancements in that area which significantly improve quality. I'd say while there were other improvements, those are probably the most important ones. In the coming weeks/months I'll be re-writing everything I did from scratch to incorporate some other improvements which should require less key-frames in the algorithm as well as a faster bundle adjustment (hoping for 60fps).
druidsbane
Binocular Vision CONFIRMED!
Posts: 237
Joined: Thu Jun 07, 2012 8:40 am
Location: New York
Contact:

Re: Augmented Reality @ HPG 2012 Conference in Paris

Post by druidsbane »

Very impressive. You're right, the paper isn't up yet though other slides are. In any case, was interesting reading up on what's out there. My own attempts at feature detection and the like were back in college, very static, quite mediocre and certainly a few years behind 2007 tech ;) Will definitely be following your work after this! Certainly more fun to see than sparse voxel tree stuff I read from a few years ago, no matter how useful that was ;)
Ibex 3D VR Desktop for the Oculus Rift: http://hwahba.com/ibex - https://bitbucket.org/druidsbane/ibex
User avatar
brantlew
Petrif-Eyed
Posts: 2221
Joined: Sat Sep 17, 2011 9:23 pm
Location: Menlo Park, CA

Re: Augmented Reality @ HPG 2012 Conference in Paris

Post by brantlew »

Don't know how I missed this, but that is really cool. How much environmental texture do you need to implement it well. That carpet has a lot of edges to grab onto. Would it work on something rough but homogenous like grass?
JonOlick
One Eyed Hopeful
Posts: 7
Joined: Fri Jun 01, 2012 6:04 pm

Re: Augmented Reality @ HPG 2012 Conference in Paris

Post by JonOlick »

brantlew wrote:Don't know how I missed this, but that is really cool. How much environmental texture do you need to implement it well. That carpet has a lot of edges to grab onto. Would it work on something rough but homogenous like grass?
yes, but the tracking tends to be better when you have actual things it can track well. We found that while it works well on a white carpet under normal conditions, the tracking fails at extreme angles on mostly random texture like carpet (haven't tried grass yet, but its similar to carpet). Of course there are always degenerate cases like a flat, textureless walls and in that case there isn't much any method based on a standard camera can do. Since that demo though we've improved things quite a bit in tracking and that is our primary target right now. We want the tracking to be flawless and playable under any reasonable condition.
User avatar
Chriky
Binocular Vision CONFIRMED!
Posts: 228
Joined: Fri Jan 27, 2012 11:24 am

Re: Augmented Reality @ HPG 2012 Conference in Paris

Post by Chriky »

Could you post a video that demonstrates the current limitations of the tracking? It literally looks perfect in that video I was like :woot

I imagine it would have trouble with something occluding the camera, really violent camera movements and smooth/plain backdrops (as you've said), anything else?

Also, what kind of set up does it need? Just the "move camera sideways" set up? If so have you considered using stereo cameras - could this let you do the set up stage by comparing feeds from the two cameras?

It really looks so close to a perfect solution, hope your development continues as well as it has been going so far.
User avatar
brantlew
Petrif-Eyed
Posts: 2221
Joined: Sat Sep 17, 2011 9:23 pm
Location: Menlo Park, CA

Re: Augmented Reality @ HPG 2012 Conference in Paris

Post by brantlew »

I'll add to Chriky's questions... Can it handle a 360 degree turn or does it need to be tightly calibrated on a small region?
User avatar
FingerFlinger
Sharp Eyed Eagle!
Posts: 429
Joined: Tue Feb 21, 2012 11:57 pm
Location: Irvine, CA

Re: Augmented Reality @ HPG 2012 Conference in Paris

Post by FingerFlinger »

My question is the same as brantlew's I think, but anyway: can it reacquire its absolute position from point cloud data or other means if the FOV totally changes and then returns?
Post Reply

Return to “General VR/AR Discussion”