Epson Moverio sandbox

Talk about Head Mounted Displays (HMDs), augmented reality, wearable computing, controller hardware, haptic feedback, motion tracking, and related topics here!
Post Reply
ruf
One Eyed Hopeful
Posts: 6
Joined: Mon Sep 17, 2012 8:34 pm

Epson Moverio sandbox

Post by ruf »

Alright, I met PalmerTech at SIGGRAPH this year, and he pointed me toward this forum. I feel like I just stepped into the lion’s den, so please be gentle. I’ll keep reading the forum and trying to get up to speed.

For lack of a better description, I’m in charge of breaking the Epson Moverio. We’ve got some fun projects that will probably look like child’s play to you guys, but the purpose is just to get folks thinking about what’s possible. Our goal is to show what can be done at the very basic level and start seeding the imagination. That being said, we’d love to progress these ideas.

Last week, we posted a video of our FPV/3rd-person control R/C crawler. I find it pretty easy to shift focus from the FPV to driving the car in the more familiar 3rd-person view. We’ve already got some of the parts to implement both headtracking and 3D. We’re still feeling our way around, so any recommendations you guys can make would be awesome.
http://youtu.be/PkLNDsTXy4Y

And today, we’ve just posted the video for our video input board which lets you pipe composite (gasp!) video into the Moverio. This leads to gaming, which of course leads to the Kinect. 5 words: Moverio Kinect Country Line Dancing. The gauntlet has been thrown, Palmer. Doom schoom.
http://youtu.be/Oww8ybGDqxk

-Joe Chen
Product Development Manager and Dancer
New Ventures
Epson America
User avatar
cybereality
3D Angel Eyes (Moderator)
Posts: 11406
Joined: Sat Apr 12, 2008 8:18 pm

Re: Epson Moverio sandbox

Post by cybereality »

Cool, man. Seems those are not the best use of the technology, would love to see more augmented reality type applications, but still interesting.
zalo
Certif-Eyed!
Posts: 661
Joined: Sun Mar 25, 2012 12:33 pm

Re: Epson Moverio sandbox

Post by zalo »

It seems to be android controlled. Strap a phone on that and get some head tracking asap!

Also, what do you think of Michael Abrash's recent post on the feasibility of Hard AR (selectively making parts of the screen completely opaque). Don't know who he is? Google him. I suspect Valve may become one of your biggest clients soon.
User avatar
brantlew
Petrif-Eyed
Posts: 2221
Joined: Sat Sep 17, 2011 9:23 pm
Location: Menlo Park, CA

Re: Epson Moverio sandbox

Post by brantlew »

I have to agree with Cyber. Cool tech but the example applications don't make a lot of sense to me. With these see-through technologies, it doesn't seem like you want the projected and background imagery at-odds with each other and competing for attention. A more compelling demonstration would either minimize the projected imagery to a HUD or something like the Google-glasses demo. Or you would show interaction between projected graphics and the real world as in an AR game. I realize that the company probably doesn't have these software assets at hand, but these alternative demonstrations give a confusing message about what the technology is useful for.
User avatar
android78
Certif-Eyable!
Posts: 990
Joined: Sat Dec 22, 2007 3:38 am

Re: Epson Moverio sandbox

Post by android78 »

The moverio seems like a pretty cool piece of kit. I'm not sure that just using it as a TV attached to your head really offers anything over a small screen, unless you have head tracking. If you combine head tracking with it, then AR/VR/FPV would be a really worth while application.
User avatar
rhinosix
Cross Eyed!
Posts: 136
Joined: Thu Aug 30, 2012 10:19 am

Re: Epson Moverio sandbox

Post by rhinosix »

Cool project. I don't think it matters if the demonstrations are immediately useful - like you said, it's just to get people thinking about what's possible. Some ideas come to a dead end, while others evolve.

It would be cool if you could minimize, reduce the size of the screen, or put it in the corner - which I imagine wouldn't be too difficult to implement.

I'm trying to think of some useful applications. I guess the main advantages are that this frees up your hands, and gets you away from the desk, while still having data in front of you. It differs from Project Glass, which is designed more for walking around to receive small popup notifications maps directions, etc.

Some ideas:

* A piano instructor application to teach you where to place your hands, and how to play.

* Or even just to display sheet music. Add a microphone and have it automatically turn the page when you reach the bottom.

* Yoga instructor which shows positions of stretches like 'Let's Yoga' for Nintendo DS. With the Kinect you could have it check your posture.

* Showing recipes and giving cooking instructions.

* Drawing instructor. Holding your pencil, the screen animates each line as you follow along.

How do they compare to these glasses? http://www.siliconmicrodisplay.com/st1080-features.html

How open/ hackable are the Moverios?
ruf
One Eyed Hopeful
Posts: 6
Joined: Mon Sep 17, 2012 8:34 pm

Re: Epson Moverio sandbox

Post by ruf »

cybereality - Thanks! Agreed. We’re just having some fun right now. We’re releasing a new video each week, and I think we’ll be showing an awesome application in a couple of weeks.

And has others have noticed, we’re limited from both a hardware and especially a software standpoint from exploring AR. We’re going to keep pushing our boundaries, and I think the video input board opens up a lot of opportunities to use outside hardware and software to really start exploring more compelling AR applications. We’re working with some really good folks in these areas, but these projects aren’t ready to show yet.

zalo - You’re correct, the handset is running Android. We’re very much interested in headtracking. We’re working on a variety of levels, from hacking on a webcam or a hobby-grade FatShark tracker, to partnering with aerospace companies to integrate our glasses with their sensor systems. I did google Michael Abrash and hard AR, and it’s definitely the holy grail. Very cool.

brantlew - Completely understood and agreed for the most part. This is down to the content itself. If the content is bright, then it will tend to grab your attention more than the background. If the content is dark, then it will pretty much appear as transparent, so the background shows through. In addition, the Moverio has a “contrast filter” which is a glorified name for clip-on sunglasses that darken the entire background and image and make the image much easier to lock onto. I will say that most people don’t use the contrast filter much after the first week (unless outdoors), because your brain adjusts and it becomes very easy to pick up the image. I’d estimate that it takes another week or so to get really good at shifting focus between the image and the background, and probably another week or so before you are actually shifting focus THROUGH the image instead of looking around the periphery.

However, for the first video, the usage is fundamentally different from AR. R/C vehicles are actually a LOT easier to control from the 3rd person perspective, but it’s cool to get the FPV view from the vehicle to see around objects, etc. Most FPV glasses lack transparency so it’s either one or the other. With Moverio you can see both and just shift your focus between the two. With a car like in our video, it’s not that big of a deal, but you should see some of these FPV pilots removing their glasses while trying to land an aircraft... That is if they don’t have auto-landing programmed in.

The other benefit is situational awareness. We presented this video in a very playful way, but there are folks that use robots and handlaunched UAV’s in dangerous environments. They’re really excited about the ability to still see what’s going on around them while monitoring the FPV from the vehicle.

And we’re just having some fun in the second video, and your intuition is spot on. We’d LOVE to show a “One Day”-like video, but we’re just not ready yet. We also want to start the general public out a little slower and build the story with something more accessible like gaming. We don’t have a lot of hardware or software assets here, but we are partnering with folks that do, and we’re excited to see how the projects progress. Great idea to integrate our projectors to create a projected environment along with the glasses image!

android78 - You’re right, and unfortunately a lot of people (within the company) still see Moverio as a tv on your head... That’s why we stuck our neck out and sought partners outside of our typical corporate relationships and started making these videos to stimulate conversation. As mentioned above, we are very interested in headtracking, and we’ll definitely keep you guys in the loop as the projects progress.

rhinosix - Thanks! This is exactly what we’re hoping for. A lot of folks want us to hit home runs every time, and that’s just not realistic in an emerging environment. We’re continuing to place our bets and invest in a variety of applications and support our partnerships to make the most of what little resources we have.

All the screen manipulation stuff is down to the software and the UI/UX. We’d love for an app developer to help us out on this front, but most of them look at our install base and politely decline. The sad thing is I think this is where AR needs the most work - UI/UX. The technology is going to get better, smaller, faster, but really the behavioral study and experimentation of what people want to see, and how they want to see it is what’s most important, and I think underemphasized. I think best thing about Project Glass is that it gets people talking about the possibilities and thinking about potential ramifications both positive and negative.

Regarding the ST1080, I’m sure it’s got it’s strengths, but it has a transparency of 10%. Kind of like wearing sunglasses all the time. I think ours is in the 90’s. I’ll have to check.

And finally, AWESOME ideas and applications. The piano keys and sheet music are something we’ve never thought of, but makes so much sense. It’s just an extension of the recipe/cookbook application. The yoga instructor is probably a little more realistic than an exercise instructor at this stage of the hardware, but I almost had to play a Zumba Kinect game instead of the Country Dance All Stars. I think I chose wisely.

How open is Moverio? As open as you want it to be. If you guys need a video input board, we can get you one. The kernel source code is also available. We know there’s certain things that we don’t have the expertise to do, but we can try our best to facilitate those who can.
zalo
Certif-Eyed!
Posts: 661
Joined: Sun Mar 25, 2012 12:33 pm

Re: Epson Moverio sandbox

Post by zalo »

I forgot to ask, what's the max framerate and latency (from signal to pixel)? I ask because If there is too much image processing going on in that box, it could slow down the signal and make your AR swim and be nauseating when overlayed with the real world.

Also, what do you think will be the max fov that your glasses can theoretically be designed to achieve? Immersive AR would be kind of lame if it didn't take up the whole lens.

Double also, what's it like switching focus between the image and the real world? Could an object look implanted in the world, or is this too much of an issue?
User avatar
android78
Certif-Eyable!
Posts: 990
Joined: Sat Dec 22, 2007 3:38 am

Re: Epson Moverio sandbox

Post by android78 »

It's a shame you can't turn the camera view off with this, but it could be interesting to see how it would be if you mounted the phone on top of the glasses:
http://www.youtube.com/watch?v=sr-C4kyybrs
Or maybe one of these with the same idea:
http://www.augmentedplanet.com/2011/02/ar-app-gun/
You could use a couple of the pads from iphone pens controlled by flexible pushrods for the shoot.
Zaptruder
Cross Eyed!
Posts: 128
Joined: Mon Aug 06, 2012 9:28 am

Re: Epson Moverio sandbox

Post by Zaptruder »

These glasses look pretty awesome... and a good alternative to the yet unreleased Google Glasses.

But in terms of developing for them - we really want to think in 'HUD Space' (the term that Michael Abrash coined in his blog post) terms.
http://blogs.valvesoftware.com/abrash/t ... part-1-vr/

i.e. forget about overlaying stuff over real world objects for now. Once mobile processors are fast enough to do DTAM, then we can start mapping and overlaying graphics in a quality fashion (as opposed to an acceptable for theorizing and research fashion).

But that actually still leaves us with a lot of possibilities of what can be done.

Personally, I think I'd be excited to use these in general - be out and about and just browsing the 'net, watching a show, without having to look down all the time. The ability to be always connected to a display, even while interacting with nature is the primary appeal of these devices, and plays well in the context of an increasingly connected/digitized world.
ruf
One Eyed Hopeful
Posts: 6
Joined: Mon Sep 17, 2012 8:34 pm

Re: Epson Moverio sandbox

Post by ruf »

zalo - I'm still trying to get this info. The engineers aren't super happy that we made the input board, so it's hard to get the latency spec for that particular input method... I'll keep digging around and asking. By my untrained eye, it's probably ~0.1s or less. I don't notice any lag when I'm playing the games, but some of you guys may be more sensitive to that. I mean like tweaky FPS gamer level of sensitivity. We've done camera stuff on it, and it doesn't "swim".

As for the FOV, we're pretty limited at 23 degrees. It has to do with our optic design. It's a very simple design which makes it manufacturable, but any increase in FOV would require a drastic increase in thickness which would directly impact weight. I'll be honest, if you want immersion, we are not what you're looking for. I like to reference a "spectrum" of immersion. VR helmet/goggles with 120 degree FOV being on one end, where it's completely immersive and a specific escapist event; and on the other end, Google Glass where it's very limited FOV, limited display capability, limited information, monocular, but always on, out of the way. We're in the middle, with what I consider pretty impressive display capability (including stereoscopic with no crosstalk), fantastic transparency, but limited FOV, kind of like a window to the world. Similar to Google Glass, the un-augmented periphery is still very much in your FOV and perception. We try our best to capture the effect of the display on camera, and I think we've done the best job to date, but it's just not the same as wearing them. Check out this creepy video by ElectricTV for reference as well:
http://www.youtube.com/watch?v=VW4B_WvU ... plpp_video

The focus shift becomes very natural and very quick. I think implanting an object in the real world could be pretty easy as shown above. We've already played with some smartphone AR apps mirroring their display into Moverio. Works OK, but could use some streamlining.

android78 - I don't quite understand what you mean by "turn off the camera view". We can "mute" the glasses, and anything that is displayed as black will appear transparent in the glasses. Oddly enough, one of my friends had that plastic iPhone gun. Pretty fun!

We are working on some pretty neat applications with a de-cased webcam mounted on the top of the glasses. I've attached an image of a modded pair that we did for a developer of an AR assisted-maintenance app for mining equipment. Very cool. He just showed it at MINExpo this week. I'll ask him if I can share the pictures from the show.

Zaptruder - Hmmm. I'll have to dig more into HUDspace. We tend to refer to our product as a HUD sometimes because of it's FOV characteristics, but we have a little more graphical capability. I agree with you that at this stage of the technology, "hard" mapping is going to be difficult and impractical outside of experimentation, but the hardware and software will of course progress to where it's just another app. I still think there is powerful relevance in the ability to overlay non-mapped/tracked, but related data into your real environment, as per the HUDspace concept. This is why I argue that the most compelling development at this point is behavioral, while the usage is still at it's infancy. If you could see anything you want in your glasses at any time, what would you want to see, and how would you want to see it? The nice part about exploring behavior is that you can experiment with it now without waiting for the hardware or software to catch up and give you perfectly mapped AR. This is one of the reasons we were at E3 this year talking to folks that develop FPS games. These guys have basically been creating HUDs forever, and they understand what their gamers want to see and how they want that data presented.

The interesting thing I've been kicking around is what happens when each eye is shown different data? Not just 3D, which is different POV data, but left eye = graphic, right eye = text. Just doing weird experiments, which of course feel completely unnatural, but may hopefully uncover some new usage insight.
You do not have the required permissions to view the files attached to this post.
Zaptruder
Cross Eyed!
Posts: 128
Joined: Mon Aug 06, 2012 9:28 am

Re: Epson Moverio sandbox

Post by Zaptruder »

ruf wrote:zalo - I'm still trying to get this info. The engineers aren't super happy that we made the input board, so it's hard to get the latency spec for that particular input method... I'll keep digging around and asking. By my untrained eye, it's probably ~0.1s or less. I don't notice any lag when I'm playing the games, but some of you guys may be more sensitive to that. I mean like tweaky FPS gamer level of sensitivity. We've done camera stuff on it, and it doesn't "swim".

As for the FOV, we're pretty limited at 23 degrees. It has to do with our optic design. It's a very simple design which makes it manufacturable, but any increase in FOV would require a drastic increase in thickness which would directly impact weight. I'll be honest, if you want immersion, we are not what you're looking for. I like to reference a "spectrum" of immersion. VR helmet/goggles with 120 degree FOV being on one end, where it's completely immersive and a specific escapist event; and on the other end, Google Glass where it's very limited FOV, limited display capability, limited information, monocular, but always on, out of the way. We're in the middle, with what I consider pretty impressive display capability (including stereoscopic with no crosstalk), fantastic transparency, but limited FOV, kind of like a window to the world. Similar to Google Glass, the un-augmented periphery is still very much in your FOV and perception. We try our best to capture the effect of the display on camera, and I think we've done the best job to date, but it's just not the same as wearing them. Check out this creepy video by ElectricTV for reference as well:
http://www.youtube.com/watch?v=VW4B_WvU ... plpp_video

The focus shift becomes very natural and very quick. I think implanting an object in the real world could be pretty easy as shown above. We've already played with some smartphone AR apps mirroring their display into Moverio. Works OK, but could use some streamlining.

android78 - I don't quite understand what you mean by "turn off the camera view". We can "mute" the glasses, and anything that is displayed as black will appear transparent in the glasses. Oddly enough, one of my friends had that plastic iPhone gun. Pretty fun!

We are working on some pretty neat applications with a de-cased webcam mounted on the top of the glasses. I've attached an image of a modded pair that we did for a developer of an AR assisted-maintenance app for mining equipment. Very cool. He just showed it at MINExpo this week. I'll ask him if I can share the pictures from the show.

Zaptruder - Hmmm. I'll have to dig more into HUDspace. We tend to refer to our product as a HUD sometimes because of it's FOV characteristics, but we have a little more graphical capability. I agree with you that at this stage of the technology, "hard" mapping is going to be difficult and impractical outside of experimentation, but the hardware and software will of course progress to where it's just another app. I still think there is powerful relevance in the ability to overlay non-mapped/tracked, but related data into your real environment, as per the HUDspace concept. This is why I argue that the most compelling development at this point is behavioral, while the usage is still at it's infancy. If you could see anything you want in your glasses at any time, what would you want to see, and how would you want to see it? The nice part about exploring behavior is that you can experiment with it now without waiting for the hardware or software to catch up and give you perfectly mapped AR. This is one of the reasons we were at E3 this year talking to folks that develop FPS games. These guys have basically been creating HUDs forever, and they understand what their gamers want to see and how they want that data presented.

The interesting thing I've been kicking around is what happens when each eye is shown different data? Not just 3D, which is different POV data, but left eye = graphic, right eye = text. Just doing weird experiments, which of course feel completely unnatural, but may hopefully uncover some new usage insight.
Basically the question to ask is - if you could have a HUD for your life... what would it include? What information would you want in your 'life HUD'?

Also, if my naieve tests with my HMZ-T1's SBS function (i.e. set it to side by side mode while viewing normal content) is any indication, the brain does not like viewing drastically disparate images. It looks like a mess.
Icbribob
One Eyed Hopeful
Posts: 5
Joined: Tue Jan 15, 2013 9:44 pm

Re: Epson Moverio sandbox

Post by Icbribob »

I see that joe has left epson for oculus. Any chance at getting a video input card still. Working on some AR laboratory notebook applications. Also some FPV ground stations.

Bob
shent1080
Cross Eyed!
Posts: 101
Joined: Wed Dec 26, 2012 7:10 am

Re: Epson Moverio sandbox

Post by shent1080 »

The best idea that i can think of is to use the device with a wacom tablet, people don't wan't to pay £800 for the cintiq (has integrated lcd screen), and many struggle to draw on a tablet that is on a table while looking up at a screen, solution, Epson glasses could give you the screen overlay and possible match up the surfaces with the use of a small bolt on camera etc.
Icbribob
One Eyed Hopeful
Posts: 5
Joined: Tue Jan 15, 2013 9:44 pm

Re: Epson Moverio sandbox

Post by Icbribob »

I would love to come up with a video control board for these that just removes the android box form the picture. Epson makes some projectors that have the same LCD screen in (I think). Check out the mg-50. A standalone box that the glasses plug into and allow it to recive hd from any source would be nice. I assume it is limited to 640x480 over the RCA card, but I have not recived mine yet, but it seems to be on the way.

Pin out specs would be really helpful.

Bob
Icbribob
One Eyed Hopeful
Posts: 5
Joined: Tue Jan 15, 2013 9:44 pm

Re: Epson Moverio sandbox

Post by Icbribob »

so i have finally given sometime to really playing with my set of glasses, while i wait for the video card. Originally I could not find many apps that would work on them, but I have since rooted them, and gotten a number of apps to install. Of particular intreat are mx player and icam. mx player allows for playing of almost all video formats., and network streaming. Icam lets you view web cam out put from a pc based cam. The unit can view 480 and 720 fine, 1080 is choppy. I did my tests with .mov files.


of other note dropbox also works which makes for nice file exchanging. Surprisingly, Hulu also works.

so far i can not get a remote pc screen sharing app, like gotomypc to work. I am working on being able to control the goggles from a pc. that would at least give mouse/keyboard control.

things that do not for gmail, google earth, VLC player. It looks like the video card is a atomv7.

I also tore it down and took pictures if people would like to see circuit boards. I m going to open the glasses in the next few days and see what the projectors look like.

bob
tcboy88
One Eyed Hopeful
Posts: 25
Joined: Thu Mar 22, 2012 7:15 am

Re: Epson Moverio sandbox

Post by tcboy88 »

I installed latest Google Playstore via .apk file
However it immediately crashed everytime I start it.
Any1 know how to install Google Playstore?
Should we use older version .apk file? (since the controller is stucked w android 2.2)
Post Reply

Return to “General VR/AR Discussion”