Red Rovr Motion System (formerly Friispace)

Post by **cybereality** » Tue Jan 03, 2012 8:57 pm

@brantlew: Awesome work man! Seems to be coming along nicely. If you need some help getting the Vuzix tracker to work, I may be able to provide some code to get you started. Just PM me.

Aphradonis wrote:Are you quoting Carl Kenner? Is he banned from MTBS?

If he's banned from MTBS, I may be able to reverse that. But from what I understand, he was banned from the Vuzix forums.

Post by **brantlew** » Tue Jan 03, 2012 9:06 pm

Thanks Cyber. I'm using the Vuzix tracker now which is why I'm having the issue with GlovePie. CyberVillain is working on the Sparkfun driver and I have high hopes for that little piece of hardware. However since I only have a Vuzix HMD I will continue to have GlovePie woes (sigh)

Post by **Neil** » Tue Jan 03, 2012 10:56 pm

If he's banned from MTBS, I may be able to reverse that. But from what I understand, he was banned from the Vuzix forums.

Eugh! CarlKenner again?!?

With respect, we aren't bringing him back. We haven't banned someone from this forum in years (with the exception of spambots), and we did it because he was making an active effort to be disruptive and really bring this place down. He has political (and social) views that he felt entitled to express here at MTBS, and that's not the purpose of this site. When we politely asked him to stick to 3D and handle his inflammatory poltics elsewhere - he refused, and further blasted away in the chat window. I wasn't the one who banned him in the end - it was Sharky after Carl went on a personal tirade against him despite all his hard work on MTBS.

There is good cause for CarlKenner not to be here and other sites, and while he has undoubted expertise and passion for 3D and virtual reality, I'm not confident he could contribute to MTBS without bringing back his negativity as well. Thanks for your understanding.

Regards,
Neil

Post by **WiredEarp** » Wed Jan 04, 2012 5:55 am

Thanks Neil, I guessed it might be something like that.

Post by **brantlew** » Mon Mar 05, 2012 11:35 pm

It's been a while since I updated. As usual...work, other projects, and procrastination have slowed my progress, but I've been ramping up the last week or so and have some things to report.

I spent a bunch of time in January trying to track down some calculation errors in the computed world position of the light source. I found a couple major bugs fairly quickly that reduced the error a lot, but my calculations were still off maybe 5% at certain angles. I spent some time doing tedious measurements to determine accurate FOV of the Wii camera and characterizing lens distortion. Unfortunately, I never could find the source of the problem so I gave up and moved on. I can live with the errors for now.

Recently I have been preparing for a big data-gathering push. I realized very soon after my first live walking test that my motion tracking algorithms needed a LOT of work. It's one thing to track a light source moved smoothly by your hand. It's another thing entirely to track a light bobbing around and turning on your head. All that shaking around and turning has to be subtracted and smoothed out of the signal in order to come up with a decent character motion model. So I have been modifying the software to do some field recording of sensor data during locomotion. I will use the recorded sensor data to tease out this signal and tweak the motion tracking.

But before that can happen, I need a decent light source. So this last week has been spent trying to build some type of strong omnidirectional light source on my head. Basically I made a three sided pyramid on a pedestal and wired 6 LED's per face. The whole thing is then attached to a pair of headphones. This way the LED's sit right on top of my head and point directly back at the camera. Maybe I should have created a 4-sided pyramid but I only ordered 3 LED modules a few months ago and was working with what I had. Anyway, my hope is that the edges of the pyramids can still be detected. I wanted to encase and diffuse the whole thing in a ping pong ball, but I think it's going to be too big for that. Anybody got any bright ideas of how I can create a diffuser?

So here is a pic of my first prototype. I still need to finish the wiring and soldering but you can see the structural design.

I am eager to do some range testing (it needs to be detectable at about 100ft (30m)) and then get my data gathering underway. Unfortunately I broke one of my LED modules during construction so will have to wait until a replacement part comes in. I'll post of pic of the completed unit and give a performance update pretty soon.

Update: Well after digging around I found this thing that will hopefully work well as a diffuser (and also make me look even dorkier

)

http://www.twistncflcovers.com/item/bas ... d=16049527

Post by **cybereality** » Tue Mar 06, 2012 9:02 pm

Nice work. Am eager to see where this goes.

Post by **cadcoke5** » Tue Mar 06, 2012 9:59 pm

I have also made a few efforts to create a diffuse LED light source.

Method #1; Apply transparent "disappearing" cellophane tape. Moderate diffusion, but easy to add more and it seems to be more even than my other method.

Method #2; Grind the top of the LED to change its shape. The next step, I didn't try since I didn't have one, but a friend used a buffer to smooth its shape and make it more transparent. However, the results were not an even spread of light.

Method #3; bend the LED's different directions. However, in your case, since they are pre-made modules, they may not have allowed for enough space between the LED and the circuit board.

Joe Dunfee

Post by **brantlew** » Tue Mar 06, 2012 10:20 pm

@Joe: Luckily I found pre-built modules with "wide" angle (50 degree) LEDs, so they spread pretty well to begin with. Just to be sure I went ahead and clipped the tips of them as I have heard anecdotal evidence that it increases the angle. However, subjectively I really can't tell the difference - plus that's how I broke one of them. Aaargh!

Interesting tip about the tape. Would you use the "satin" looking kind instead of the clear? And you just stick it right on top of each LED? Simple. That worked well for you?

Post by **cadcoke5** » Wed Mar 07, 2012 5:18 pm

Would you use the "satin" looking kind instead of the clear? And you just stick it right on top of each LED?

Yes, the satin type. Of course, keep in mind that it would cut the output somewhat.

I just stuck it on top of the LED. Though, I imagine there might be some way to do it more professionally. The tape stuck onto a curved surface does not look that good. Perhaps on the flat snipped-off ones it would do OK.

Joe Dunfee

Post by **cirk2** » Wed Mar 07, 2012 9:34 pm

Maybe you could try a cylinder out of very thin paper or mate foil. Maybe you could wrinkle it in the pyramid base form to preserve trackability.

Exaple of the paper I mean:
http://www.pss-treff.de/bauen/mig3/butterbrotpapier.jpg

In germany we call it "Butterbrotpapier" which translates to "Butter-Bread paper" it is used to wrap up lunches etc. its quite thin as you can see in the picture.

Post by **brantlew** » Wed Mar 07, 2012 10:13 pm

That's a good idea.

An extension to that idea would be to take the thin paper, wrap it around a balloon, use a clear glue to harden it, and then pop the balloon to create a stiff sphere of thin paper. (paper mache I believe it's called) It could have very good diffusion properties, but it would also be fragile.

Post by **cirk2** » Thu Mar 08, 2012 8:44 am

yeah that is pap mache. The problem is that you only can use one layer of paper, normaly you user several layers of toilet paper.
but its easy to try since that kid of paper is sold in 10m rolls in every supermarket for nearly nothing.

an alternative would be frosted plexiglass:
http://www.estreetplastics.com/Frosted_ ... _s/142.htm
That would certanly give more stability. Also you can bend it with hot air, but you have to be carefull with that xD
I can't say for shure if the frosted status survives the bending, I only worked with clear material.

Post by **cybereality** » Thu Mar 08, 2012 9:39 pm

You can buy frosted window film for pretty cheap, its basically a big decal that fakes the frosted glass look. They have them at my office, they are fairly cheap.

For example:

http://www.purlfrost.com/purlfrost.php

Post by **brantlew** » Thu Mar 08, 2012 11:14 pm

A lot of good suggestions.

I ordered that bulb cover because it's cheap, durable, and requires no construction. But if that doesn't work, I will consider all of these options.

Post by **WiredEarp** » Fri Mar 09, 2012 1:40 am

Thought i'd already said this, but can't you get the same effect as frosting by lightly sanding the LED's?

Post by **brantlew** » Sun Mar 11, 2012 1:31 am

Quick update. Did some quick range testing on the prototype unit with mixed results.

Good news: Light detection during angular rotation of the unit was very stable. Those wide (50 degree) LED's work great, and I saw very little signal loss as I pitch and yaw the unit around. It looks like I won't even need a diffuser. That's a good thing because the diffuser bulb that I ordered really sucked and cut the brightness way too much.

Average news: Those little 6 LED modules are not bright enough to cover the full distance across a basketball court (100 ft). At 25 and 50 ft I get solid signals. Very intermittent at 75, and virtually no signal at 100. That's not too big of a deal however since I can just scale up using the same design. This bad boy should do the trick:

Bad news: My optical resolution (the physical distance I have to move the light to register motion in the camera) seems much worse than I imagined. I'm not sure why. Simple math told me that I should expect an optical resolution of about 1cm/pixel at 50 feet. But my quick field test suggested that it was more like 10cm/pixel !! (I still need to do a more rigorous test to confirm this) That's really bad news because if true, then it means that I cannot use the optical signal alone to detect motion smoothly. Not a total deal-breaker - just means that I will need to augment the signal with an inertial sensor, but I had hoped to keep the design simple by just relying on optical.

Post by **FingerFlinger** » Sun Mar 11, 2012 1:54 am

I know you've already got it sorted out, but as far as LED diffusers go, I've found that gluing one of these beads to the end of the bulb works quite well.

Post by **cybereality** » Sun Mar 11, 2012 12:07 pm

@brantlew: Cool.

Post by **brantlew** » Sat Apr 28, 2012 6:34 am

Help. Need mechanical solution for keeping a set of headphones steady. Even without extra weight attached none of the headphones I have tried clamp my head tightly enough, and so they tend to dislodge and shift when I move my head up and down. With the extra weight I am adding to the top, the problem is even worse. So far I haven't figured out a solution. I have considered using a cloth headband and somehow sewing or attaching it to the sides of the headphones so that it will give me a little stability, but I'm not particularly fond of the idea. Maybe some type of plastic ring or helmet - like the liner of a hard-hat. Ideally something simpler though. You guys that work on HMD mounting solutions have probably come across something that would be helpful. Any ideas ?

Post by **cybereality** » Sat Apr 28, 2012 11:24 am

In-ear headphones?

Post by **brantlew** » Sat Apr 28, 2012 1:31 pm

I'm mounting bulky motion tracking equipment on top of the headphones. I guess I don't have to mount it on headphones - per se, but it is a convenient location. Earphones will work fine, but I still need a stable mounting point on the top of my head. I might also be able to get away with a mount point on the back of my head (maybe counterbalanced with HMD). Any suggestions?

Post by **brantlew** » Tue May 08, 2012 12:21 pm

Project Video Update:

https://www.youtube.com/watch?v=wpwN23wPeKg

I have not talked much lately about this project, but I have been working like crazy on it. There have been a lot of technical problems solved and some significant redesigns in order to get it to the point where I can do field-testing. The video shows at what stage of development I am at. I have an untethered and functional back-top unit now, and the system has been fully scaled. This only shows a basketball court, but I have done field testing on a football field and it scales to that size as well.

The motion detection algorithms are still prototype and need a lot of work. There are some obvious flaws and delays that are clearly visible in the video. But the motion control works well enough now for me to continue developing boundary enforcement and redirected motion (none of which is shown in this video). I am particularly satisfied with how well the independent head/torso motion is working. This game (theHunter) does NOT support independent head motion - just mouse-look and WASD - so all of the walking and looking around that you see is derived from that basic interface.

I am not going to go into too many details about the technology, because I am thinking about turning it into a commercial product. The timeline is difficult for me to estimate because there is still a lot of R&D and some incredibly difficult problems to solve, but I would like to get it to market by the end of the year. Maybe some beta testing to forum members would be available earlier.

In the meantime, I've still got tons of development to go. Also, I bought SkyRim when it first came out, but I have not allowed myself to play it, because I knew it would suck my time away and distract me. My goal is to alpha-test this technology with SkyRim. Perfect timing for my ST1080 to show up as well

Post by **FingerFlinger** » Tue May 08, 2012 7:31 pm

That is Rad! How is the level of immersion at this stage? This is close to my ideal VR setup.

Post by **WiredEarp** » Tue May 08, 2012 7:40 pm

Looks great! Can it only track the one set of sensors currently?
The immersion must be way better with the ability to walk around.

Post by **brantlew** » Tue May 08, 2012 9:49 pm

@FingerFlinger: Right now the immersion is not good because it's so dangerous! I'm usually testing this outside at nighttime, and without boundary enforcement I am constantly in fear of smashing into goal posts and other obstacles. I've twice come within half a meter of crashing right into a pole. The first time I tried to film this was on a bright and sunny day so I had to use a blinder. I ended up smashing right into my camera and nearly broke it. There are some other frustrating glitches that take me "out of the moment" but all-in-all it's pretty cool. I think once I get boundary enforcement working and the ST1080 and/or the Oculus Rift, I will be be able to relax and really get in to it.

@WiredEarp: If you mean does it currently only track a single point, then yes.
There are some very subtle things that add to the immersion in a big way. Though it doesn't match your head bob in 6D, it does mimic the slight bounces of your head as you walk which is something you don't experience on a stationary system. Also the "quasi"-independent head motion is a real winner. It seems to add to your environmental awareness to be able to turn your head and scan the surroundings as you walk in a straight line. You start to realize how bogus the movement in a FPS is - all those strafe motions and the start/stop/scan habits that you pick up. Independent head motion is more natural and informative.

Post by **WiredEarp** » Tue May 08, 2012 10:10 pm

For a starting point, you could make it just play a tone when approaching a border - that would be way easier than integrating visual cues.

For a game that doesn't support proper independent head motion, it really looked pretty good! I agree about the little head movements adding a lot to immersion.

I'd pay good money for a system like this that could support 12 players locations. I'd only need the location/direction vectors, since I could use devices such as a Hydra in a backpack to provide head/gun/hip tracking.

Post by **brantlew** » Tue May 08, 2012 11:12 pm

WiredEarp wrote:I'd pay good money for a system like this that could support 12 players locations. I'd only need the location/direction vectors, since I could use devices such as a Hydra in a backpack to provide head/gun/hip tracking.

Getting 12 accurate x,y locations is relatively simple, but getting accurate direction vectors is really freakin' difficult because the yaw sensors are all so "drifty". But you HAVE to have perfect yaw vectors if you want to put players in the same play area. So that's the rub. Players must stay in the same reference frame or else they will start smacking into each other. Compounding the problem is the fact that not only must you have perfect yaw in the real-world, but you must have perfectly aligned yaw in the virtual world as well. In other words - a 90 degree turn in the real world corresponds to exactly a 90 degree turn in the virtual world - otherwise we have frame drift again and bruised players. Mouse emulation is inherently drifty. So without game supported absolute yaw, we're doubly screwed.

Luckily for single player the reference frame can drift all over the place without consequence. In fact, that's the whole point of redirected motion. But open environments and redirected motion are totally incompatible with shared multi-player areas. The best you can do is have a partitioned play field with each player running around in their own little area.

Post by **WiredEarp** » Tue May 08, 2012 11:38 pm

But open environments and redirected motion are totally incompatible with shared multi-player areas.

If you have accurate enough yaw you don't need redirected motion for a shared space, you can simply render the other players in the 3D world and let them avoid them in that way. That way, lots of players can share the same area.

I know what you mean about mouse emulation and single player accuracy etc, for any serious VR role mouse emulation will never cut it IMHO, so I only consider it for playing games using VR tech, not for real VR.

Post by **cybereality** » Wed May 09, 2012 5:43 pm

Looking good. Nice work.

One thing I noticed, though, is it seems you are moving very far in real life, but only moving slightly in the game. Was this intentional?

Post by **brantlew** » Wed May 09, 2012 6:47 pm

Just depends on what the default game walking speed is. Whatever the "W" key does is what I do. So if the game supports configurable speed then you can calibrate it to your walking speed. I'm also sort of "overwalking" it just to make sure the input signals are clear but I could slow down a little to match the game a bit better. Actually this game has a more realistic walking speed than most because it's designed as a simulator (of sorts). I've noticed that the default speed in SkyRim on the other hand is closer to a run.

Post by **PalmerTech** » Wed May 09, 2012 8:54 pm

Very, very cool! Cannot wait to see this completed, I would definitely buy it as a product.

Post by **brantlew** » Mon May 28, 2012 1:12 am

Quick update: Important milestone tonight. I just walked about 40 minutes (maybe 2 miles) on a virtual forest path, while actually walking in circles on a real football field without ever exiting the field or smashing into anything. So the redirected motion seems to be working. It took me about 20 minutes to fully trust the system, but once I realized that I wasn't going to hit anything, I was able to just relax and immerse into the scenery. It felt pretty cool.

Post by **cybereality** » Mon May 28, 2012 11:00 am

Very cool.

Post by **FingerFlinger** » Mon May 28, 2012 11:06 am

That is really awesome! To what degree were you able to detect that you were actually walking in a circle, as opposed to a straight line?

Post by **zalo** » Mon May 28, 2012 2:00 pm

Probably a terrible idea, but if you had a high enough resolution tracking camera, you could put an array of three IR balls on your head. They'd be arranged in a triangular pattern (duh) with one farther out than the rest (like the vertices of a slice of pizza).

This way, you can get the absolute position and direction of each player (you just have to be careful to not mix-up the players' balls).

Also, I know that Crysis supports absolute yaw (and other angles) in modding.

Post by **brantlew** » Mon May 28, 2012 4:29 pm

@zalo: I considered that at one point - using optical tracking for both body and head orientation but the resolution of a Wiimote is just not good enough. The only work around would be to mount poles to a helmet in order to enlarge the triangle, but that quickly becomes unwieldly. Some sort of custom camera could work, but it would probably be expensive to get the required resolution. PalmerTech's company has some sort of VR room that it contracts for the military that uses optical technology this way.
http://www.mtbs3d.com/phpBB/viewtopic.php?f=120&t=13779
But even with that kind of money, they don't scale to the sizes that I am looking at. So for practical reasons doing localized head tracking seems to be the best way.

@FingerFlinger: The game I use for testing (theHunter) encourages long, straight, uninterrupted walks so it is probably the worst-case scenario for this system. My testing has shown the curvature limit for imperceptible turning to be somewhere between 2 and 3 degrees/sec. This radius is a bit too large to fit well within a 150 foot square however and I end up intersecting the edge more often than I want to. I found a "sweet spot" between 3 and 4 degrees/sec that is just slightly perceptible but reduces edge collisions to a low level. I now come within 2 meters of an edge about once every 7 minutes at which point the system gives me a very noticeable "nudge" to keep me in bounds. The bad news in all of this however is that basketball courts in general are just going to be too small to implement redirected motion very well. I have done testing at 10 degrees/sec that would keep you in that space but the turning is very perceptible at that point. Of course, games that encourage more sporadic movement patterns may work better in smaller areas, but I haven't done that testing yet.

Post by **FingerFlinger** » Tue May 29, 2012 12:04 pm

Even though your test wasn't perfect for basketball court size, I think it is very encouraging. I'm sure it is possible to design levels that work very well within more constrained areas. And coupled with some extra tricks, it could be pretty robust.

Post by **brantlew** » Tue May 29, 2012 1:13 pm

I assume one of the "tricks" you are referring to is accelerated turning. It's been well documented that you can greatly overshoot or undershoot an in-game turn without the player noticing to make large angular corrections. I've been a bit apprehensive about using this approach however - mainly because I don't independently track the torso and head. So I can't differentiate between a real full-body turn or just a head turn. In my opinion the difference is significant. With a torso turn, the only frame of reference is the environment so by moving the frame you can fool the player. But a head turn has a second reference frame - your neck. You can "feel" the relative angle between your head and your body. So if you stand still and move your head from left to right, and back again - the original left view should be intact. You will sense that the environment moved because of the disparity between your eyes and your neck. Exaggerating this effect for comic purposes - you can make the environment spin around you just by shaking your head "no".

...but in a small play area, you would probably be forced to use such techniques.

Post by **FingerFlinger** » Tue May 29, 2012 7:49 pm

Yeah, I see your point. I think there is probably still some utility there, but since I mainly want to develop a theory for designing VR experiences, I guess I'll find out what the limits are.

Post by **brantlew** » Tue May 29, 2012 8:15 pm

Sounds great. Data in this area is badly needed. There are only a few research groups that have worked on this publicly and typically in a very limited fashion. The findings tend to be based on either extrapolations of small scale testing, or just flat out simulated results. The real field testing is probably all military contract and not publicly available.