castAR - A Head-Mounted AR system by ex-Valve engineers

[zalo]

http://www.youtube.com/watch?v=8c5qrOqiFt4
http://technicalillusions.com/

For those of you who aren't familiar with the project:
Two 240hz picoprojectors and an optical tracking camera mounted on active shutter glasses
that project a perspective corrected 3D image onto an RFID-enabled retroreflective mat with
fiducials in the corners and tracks a wand in 6DoF... for ~$200.

I'm 90% certain this has been posted here already, but they just launched their forums.

Invade!

[cadcoke5]

First, I will mention that the product was reviewed by Gizmag.
http://www.gizmag.com/3d-augmented-real ... ire/29149/

I wonder how they are keeping each eye separate. They might simply rely on the retroreflective material to reflect the projection back to the correct eye. But, my own experience with retroreflective material is that the reflection angle is not that precise. They may be using polarization to keep the images separate. The polarization may also serve to reduce the light level of the surroundings.

They may be able to reduce cost by using LCD shutter glasses, and only a single projector. But, the brightness of the image will suffer, and those little projectors are fairly dim to begin with.

I also wonder if the retroreflective material is even necessary. You would use the multi-user ability, but then any white surface would work. A simple room with white walls could be used for an immersive environment, like a CAVE, provided the projector can put out a wide enough image.

-Joe Dunfee

[Cromfel]

My bet would be active stereo glasses, 2 miniature projectors (Very low light intensity) thus need for reflective material where most of the light is reflected back to the user eyes that are almost at the same spot as the light source. So their configuration is quite cool, I like how they overcome different problems. And the multiuser side would not work due wrong perspective.

- Sauli

[cadcoke5]

My bet would be active stereo glasses, 2 miniature projectors ....
... And the multiuser side would not work due wrong perspective.

It is possible that I am misreading your message, or that I wasn't totally clear about what I was proposing.

Since they have a projector over each eye, I really suspect that each projector projects an image for that eye. So, shutter glasses would be counterproductive, when they might simply use polarized projected light. So, simple polarized passive stereo is what I suspect they are doing.

My idea was to just use as single projector for a user, and active shutter glasses for stereo separation. That would cut the cost of the system dramatically, though at the expense of half the brightness.

I had not realized that the reflective material helps with brightness. But, they comment that it is inexpensive anyway, so I imagine it is a great thing to include for the brightness issue alone. For a larger gaming, or CAVE type room, the cloth can be put on a curtain track so that it is out of the way when not being used. The geometry of how they set it up is probably very tolerant of how the screen is positioned, since the projectors are near the user's eyes. A true CAVE environment requires that the screens be accurately located, and that position entered into the projection computer. This system does not need any of that if it is used for an immersive VR, rather than augmented VR.

-Joe Dunfee

[mickman]

I wonder if we'll see this materialise on K.S ?

End of summer is here .....

[zalo]

They're planning for mid-October.

[btkramer]

what I want to know is where are they getting 720p pico projectors that cheap? 2 of them plus the other stuff that comes with it for less than $200? How? and 240hz no less!

[virtualvision]

I wonder if it works very well projecting against dark surfaces.

That is almost certainly using LCD shutters as a means of separating the left and right eyes.

She did a lot of interesting stuff, like an @home integrated circuit foundry.

[usb247]

http://m.youtube.com/watch?feature=play ... 3HGrclGkIE

It looks like the reflective material is the secret sauce that's making it all work. If this video is any indication looks like it could be a game changer,

[redslashace]

I can't help but wonder what the resolution/ fov would be with the clip on which is supposed to make it work for vr.

[shole]

now
http://www.kickstarter.com/projects/tec ... -vr-system

[Kazioo]

Kickstarter page has been updated with specifications:

Glasses

• Less than 100 grams in weight
• Fits over most prescription glasses
• Ultra flexible micro coax cable
• Active shutters with 50% duty cycle

Projectors

• 1280 x 720 resolution per eye
• 120hz refresh rate per eye
• 24 bits of color per pixel
• 65 degree horizontal field of view
• 93% fill factor

Tracking System

• 110 degree FOV
• 120hz update rate
• 8.3ms response time
• 6 degrees of freedom
• Absolute positioning
• Over 200 unique tracking points
• 0.07mm accuracy at 1.5m

AR & VR Clip-On

• 90 degree horizontal FOV
• Very low distortion freeform optics
• 5mm by 8mm eye box
• Removable flip up shutter for AR mode

Currently it's $285 for a package that includes glasses, the AR & VR Clip-On, 1x2m surface and the Magic Wand (which has its own wide fov camera and a gyro).

[burf2000]

I am thinking of backing this, I have the rift onway, any reason not to back it?

[cadcoke5]

Since you are already expecting an experimenters version of the Rift, I suspect you will be busy enough with it.

I suspect this entire market is poised to explode in a year, in the same way that 3D printers have recently exploded. Then, there will be 100's of variations available to choose. After that, the dust will settle, and hopefully the poor products will disappear, and you will be able to choose a HMD that is a polished product.

-Joe Dunfee

[cybereality]

Somehow I am starting to doubt this thing will ever materialize with the specs they are claiming.

Maybe it will, that would be great. But it seems like a lot of very expensive tech for less than $300. I'm skeptical.

[cybereality]

Ha! Who am I kidding??? I bought one.

Still somewhat skeptical, but I'm in for the ride.

[BobCat]

Ha! Who am I kidding??? I bought one.

Still somewhat skeptical, but I'm in for the ride.

disclaimer: I am a True Believer; I biked 16 miles to show Jaron Lanier the flightsim + Star Wars chess video they posted before the Kickstarter. I also backed it at the Pro AR+VR level.

The Holy Grail of HMDs is 6 DoF tracking - they have nailed it. Apparent latency was zero. I played with the AR glasses and things look like they are stapled to the real world. The thing on the table is a thing on the table. It is rock solid.

The stereo image is vivid, the field of view is larger than my eyeglasses - and when you duck or move or turn, you get more pixels. You can paint any world you want.

I brashly told Jaron that everything everyone has gotten wrong in 3d and VR displays for the last few decades, the Cast system gets right. I stand by that statement.

You know how you always imagined a 3d hologram would look? You could walk around it and see the other side? You can do that now. And it can move, it's not a teapot sitting on the table, it's the Mad Hatter pouring a cup for Alice.

This is a stunning achievement and I lack the words to express how magical it is.

Once in a lifetime you encounter such a work of sheer genius and you are humbled; consider me humbled.

</gushing>

[GeraldT]

While I am still waiting to try it, this seems to me like a great idea to realize. As far as costs go I am sceptical too - those microprojectors must be really, really cheap these days to allow for it. The light engine like is because of the very low lumen it needs to produce - the retro reflective system really saves a ton of light.
What I am hoping for is the game going over a million so that they will be able to support mobile - that would be the point where things get really interesting. I see many simple yet awesome multiplayer games with the system.
Palmer told me the Rift will be the better system for VR, so I currently only care for AR with it. But I bet for many it will be the first step into the VR world and it certainly will help build a VR userbase too.
All in all it is great to see a second really cool and convincing product in the AR/VR space.

[zalo]

Apparent latency was zero. I played with the AR glasses and things look like they are stapled to the real world. The thing on the table is a thing on the table. It is rock solid.

I'm hoping for this in the final version, but the demo I tried at GDC didn't leave me impressed by with the latency. It felt around 100ms.

Now, I know latency in AR is extremely demanding; you literally have to superimpose a virtual image on a zero latency backdrop.
I'm also willing to grant that they appeared to be using an unmodified angrybots demo on a mid-range laptop (which could have also contributed to the latency).

Additionally, they mentioned the fluorescent lights in the building were pulsing at around 120.1hz; supposedly caused a weird strobing that Rick "fixed" before the demo started.
But that hack could have hurt the latency as well (and the brightness was actually a little worse than in the videos).


Hardware-wise, I have no doubt that they can deliver on 10-15 ms tracking latency, but it will definitely be up to the game developers to not screw that up.
Geekmaster's snap vector filtering could also mitigate this.


I'm still pumped for the system. I'm even making a game:
http://www.youtube.com/watch?v=kLqswWeDDA0
It's a bad game, watch with the sound off.

[geekmaster]

... Geekmaster's snap vector filtering could also mitigate this. ...

[zalo]

Well, kickstarter's over; they added a slew of new features before finishing, including a gyro and a magnetometer (but no accelerometer?).

Hopefully it'll make VR mode work well. Now to wait for VR Mode impressions.

[3dpmaster]

Put on two fisheye lenses to its projectors and BAM! There you have your full FOV head mounted display.

[budda]

Hi,

There is only one small thing that bothers me about the Castar setup.

The retroreflective sheet always seems to be silver or grey, which indicates it is picking up stray light from the surrounds.

This may reduce the contrast slightly despite the image quality being very good.

Maybe someone can design a 'black' retroreflective material that is highly directional and can reject ambient noisy light. Perhaps a microribbed sheet can be placed over an ordinary retroreflective sheet to acheive this effect. Much like privacy screens for computer displays.

It probably won't be needed but i am curious to know if this is possible.

Thanks.

[geekmaster]

I wonder how they are keeping each eye separate. They might simply rely on the retroreflective material to reflect the projection back to the correct eye. But, my own experience with retroreflective material is that the reflection angle is not that precise. They may be using polarization to keep the images separate. The polarization may also serve to reduce the light level of the surroundings.

The early prototypes used high-speed shutter glasses, and the kickstarter and their website still claim that to be the current design. However, the latest video interview with Brian Bruning and casAR, which Neil posted to this website, claims that they switched to polarized glasses, which gave increased brightness and doubled the effective framerate per eye.

[geekmaster]

... Maybe someone can design a 'black' retroreflective material that is highly directional and can reject ambient noisy light. Perhaps a microribbed sheet can be placed over an ordinary retroreflective sheet to acheive this effect. Much like privacy screens for computer displays.

Light shining into the glass beads illuminates the material touching the back half of the beads, illuminating that material, which then gets projected back through the glass bead and back to the viewer. That material (such as white paint on roadways with embedded glass beads) needs to be white or light gray to reflect colored light such as from a video projector.

It would be possible to add a black surface coating over the white paint that does NOT cover the the glass beads, but that would be a technical challenge and could drastically increase the cost of the retroreflective material. Trying to paint the surface black and then wipe it off the beads would cause too much paint to bleed down the sides of the beads, destroying most of the retroreflectivity. Even skin oils can damage retroreflective surfaces unless coated by a transparent reflective film, and most affordable retroreflective surfaces do not have such a protective layer.

When making your own retroreflective surfaces, it is important that all the paint thickness be equal to the bead radius, so only the back half of all the uniformly-sized beads is embedded in the white paint layer (acting as a projection screen). The front half of the beads must remain uncovered to function as a lens. It helps to keep the painted surface horizontal while applying paint and glass beads, and hang it on the walls or ceiling only after dry. Beads may cover the paint in a thick layer, with the excess to be carefully removed after the paint dries. You may be able to get some glass beads from a buddy in the local road works department. They discard a ton of it (around here) by picking up the excess with a street sweeper after repainting crosswalk markers on the roads. Even clean discards may be useful for DIY projects.

It could be a fun experiment to try applying a dry black powder that cannot wick down the sides of the glass beads, then brush off the excess. Or better, perhaps a black powder like from toner cartridges could be electrostatically applied between the beads, if conductive white paint were used to hold the beads. Then heat could fuse the toner in place. Just some ideas that may be worth trying. Please give me credit if you use them. Thanks.

[budda]

Hi,

I did the obvious thing and googled 'black retroreflector'.

Believe it or not, there is such a thing, and .... there is a person called Cylab on Youtube, who has recently been testing such materials as a projection screen.

http://www.youtube.com/watch?v=otWTLqMNGgQ

He found a black retroreflective material called Oralite 5500 that seems to work well.

This material is made by ORAFOL Europe GmbH for traffic signs and comes in various colours, black being one of them.

However it seems interesting that they do not seem to make projector screens. Ironic.

I am sure we will be hearing (and seeing) more about this stuff.

Thanks.

[geekmaster]

Hi,

I did the obvious thing and googled 'black retroreflector'.

Hmm.. That was NOT obvious to me, though usually my Google-Fu is great. Of course, I did not think to USE my Google-Fu powers!

Almost any idea can seem obvious once the inventor tells you how it works.

Thanks for the Google find!

What makes it non-obvious to me is that on traffic signs, you would think the sign parts that are BLACK in the daytime should also look BLACK at night (i.e. no reflected light). This is not logical. This does not compute...

Here is another video comparing different "black retro" samples:

http://www.youtube.com/watch?v=Hyorbqs2uIc

For castAR with projectors so close to eyes, would "hotspotting" still be an issue?

[budda]

Hi,

Since Castar uses infrared lights as position trackers, I wondered if they could be positioned on the back side of the retroreflector screen instead of on the front side.

I have experimented with IR leds placed just under some common materials, like electrical tape, and the lights clearly show through using an ordinary camera to see them.

Although its expected that a retroreflecting material would scatter most of the IR light backwards to where it came from, there should still be sufficient spaces between the glass beads so the IR light gets through to be detected on the front side.

If it works, then it would be a seamless way to integrate IR tracking into Castar.

Thanks.

[Moriarty]

Interview with Technical Illusions and new renderings of CastAR :

http://www.tomshardware.com/reviews/ar- ... 811-9.html

castAR Hardware Specs

Dual projectors
Total system resolution 2560x720
High Definition 1280x720 per eye
24 bits per pixel
120 Hz refresh rate per eye
Programmable low persistence
65 degree field of view in projected mode
90 degree field of view with optional AR/VR Clip on lenses.
Head and Motion Tracking 120 Hz absolute tracking
Sub millimeter precision
1000 Hz Inertial tracking

castAR Glasses
Transparent and see-through lenses
Ergonomic open frame design
Less than 100 g
Binaural/Stereo positional audio
Lightweight micro coax cable
Microphone

castAR Wand
120 Hz absolute tracking
Sub-millimeter precision
1000 Hz Inertial tracking
Analog thumb joystick
Trigger and 5 buttons

castAR RFID Grid
Surface tracking of RFID enabled objects
Centimeter tracking accuracy
Bidirectional communication for computer control of RFID enabled objects

[budda]

Hi,

I see a small design problem with the Castar glasses that perhaps should be looked at sooner rather than later.

The picoprojectors are directly mounted over the eyes.

I suggest the picoprojectors should be mounted slightly further outboard so that space can be made for a stereo camera pair directly above the eye centres.

I am not sure if such a move will cause significant parallax errors for the projected images, but it does add greater possibilities for recording what the viewer actually sees.

Just imagine being able to see in stereo what another person wearing Castar glasses is actually seeing.

Thanks.

[Dilip]

interesting relevant product is on sale from few days back.

http://androidandme.com/2014/05/news/ep ... available/

A nice detailed review on Tom's Hardware..

http://www.tomshardware.com/news/epson- ... 26728.html

They are also using twin projectors to project 960X540 images on glass to create see through certain inch screen
hovering certain feet away.

Epson has priced them exorbitant at 700USD. if twin pico projectors can be accommodated in 194$ Cast AR along with super precise
trackers then why only adding android touch pad and battery brought pricing to such horrible level of 700US$

Besides projectors of CastAR are HD,Where as Epson are just 960X540.Do any one know if

Projection modules in CastAR and Epson Moverio are essentially same or different?
Can Cast AR projectors be used in Epson like setup?

I have seen live demonstration of Moverio BT-100 in my City.

http://techionix.wordpress.com/

(Still building site - Please ignore typo)They did attract me despite of obvious flaws like bulky size and poor connectivity
Still it was Wow to see projected semi transparent screen suspended in air and was fairly large enough. while ability to be aware of your real
environment is huge plus.two different level of transparency attachment can be fairly helpful selecting different usage scenario.

if Jeri Ellsworth could be convinced to build something like BT-200 even without head tracker i would instantly buy it at 194$, its total steal. I would
love to carry in pocket for large screen experince of Movie and some casual games played on my mobile or tablet. Presently
Cast AR + 'Clip on' are going to priced at 194+75 = 270$ still cheaper then Moverio but dangerously near from oculus rift.

[metax]

I know this product is dead, but for DIY and research purposes, and general video projection,

WHAT material was used for the cloth? All I could find was it's cheap.
I'm sure it's some kind of retro-reflective tarp probably sold in rolls and used for outdoor items such as tents, but I swear anywhere I ask for such a high gain projection cloth some video projector screen seller jumps out of the bushes and tries to sell me his High Gain Super-Bright TripleX Silver™ projection screen or cloth with patented secret technology or something along those lines, costing hundreds of dollars.

I just want cloth like in these videos...

[cadcoke5]

I don't have a source for the cloth in their product video. The stuff that makes things retro-reflective a little crystals. You can buy that stuff, and it is like sand. It is used to sprinkle onto roadway striping. The fact that it is sand-like, is what makes using it for fabric a problem. Typically, if it is used on a jacket for safety, then it is embedded in a vinyl ribbon that is sewn onto the fabric. I suspect that is the form you will want to use.

Another option may be projection screens that are common in thrift stores. Many of the projection screens have a coarse sandy feel to them. I imagine they are retro-reflective. However, i suspect that they may not have the narrow angle of return necessary to keep the left-right images separate. If they did have such a narrow return beam, then your audience would have to sit in a very narrow area.

Another approach to the castAR system, is rather than using retro-reflective fabric to keep the images separate, is to use a LCD shutter glasses system, or one of the systems that use color notch filters. In other words, any system used in a CAVE environment would work here. Though, you would loose brightness, and the ability for multiple users to be viewing the same space. If the system does not need polarized projections, then virtually any surface would work.

If the projector light must be polarized, then you need a surface that preserves the polarization. Just white fabric won't work. Again, some thrift-store projection screens will preserve polarization. Aluminized fabric ironing board covers may also do this. Perhaps a flashlight, with a polarizing filter, and some polarized glasses would allow you to test the material in question.

Even though the castAR sytems seems to be dead, it may still has merit in a world where AR displays still are very narrow in their field of vision, and with low resolution.

-Joe

[metax]

Hey, thanks.

Crystals sound like strontium aluminate.

My tests don't need preserved polarization. Just need high gain (2.0 or above) to regain brightness lost from shutter glasses.

Cloth needs to be cheap and foldable, gonna make an inflatable dome out of it. Already have a dome made of blackback pvc which is fine for regular planetarium use but too dim for shutter glasses and a pico projectorion engine.

Because of the dimensions of the dome (3.5 meters wide, 2.5 meters high) it needs to be sewn from 14 individual parts which take up 40 square meters of fabric.

Maybe this more info can narrow down the list of things I could use as a retroreflective surface which would make sense both in terms of quality and price.

PS. I also tried experimenting with DIY PDLC shutter glasses over LCD shutter glasses to preserve 82% of the brightness but I can't get the refresh speed I need with this material so far.
There's a research paper on getting screen refresh rate speeds with PDLC shutters but the wording is a bit vague. If anyone understand how the "two finger electrodes" work and if they could work with a transparent ITO conductor instead of an aluminum reflector used for PDLC screens I can give this idea another shot.
https://www.researchgate.net/publicatio ... addressing

Using a 3 electrode pixel electrode (Fig. 1) it is possible to
generate a horizontal and a vertical electric field. One transmis-
sive indium tin oxide (ITO) electrode is on the cover glass and
two Al finger electrodes (electrode A and B) are on the back
plane of the display. The ITO electrode has a fixed potential.
When electrode A and B have the same potential different from
the potential of the ITO electrode there will be a vertical elec-
tric field across the cell gap of the display. When the difference
in potential between electrode A and the ITO electrode and be-
tween electrode B and the ITO electrode are the same in mag-
nitude but with opposite sign, the electric field in the cell gap
between electrode A and B will have a predominant horizontal
component. In this way it is possible to switch between a hor-
izontal and a vertical electric field in the cell gap between the
finger electrodes.

[cadcoke5]

In my experience, retro-reflective material was always vinyl. But, I just googled "retro reflective cloth" and this came up. 3M™ Scotchlite™ Reflective Material

I searched on e-bay for Scotchlite Fabric, and this came up. $34 for a 2 ft x 3 ft panel of the fabric. Note that the 2 ft width is what it comes on a role with. So, you can't buy a 3 ft Square cloth from this vendor. But, perhaps if you go to 3M you can find a source for larger widths.
https://www.ebay.com/itm/1-Yard-Safe-Re ... 7603!US!-1

Perhaps another approach is to use a cave-type system that uses fixed projectors. If you aim the projector at a concave mirror, you can spread the image. If your dome is inflated, then you can project from outside with multiple projectors. Though, you would want "black" rear-screen material to reduce how much an image gets washed out. If you only have one use, then the projectors behind the user can be projecting a black image, to reduce the amount of ambient light. I guess one benefit of having the projector on the user's head is that there is not as much extra light to wash out your image.

-Joe

[metax]

It's my experience that for such materials you really should go with cheap chinese unless you can afford to spend a lot. The US branded ones offer better performance but the price can jump 10x or more times. For example my blackback dome cost me $100 minus the time to sew the pieces together. Chinese offer it for as low as $650 including the blower. if you try to find an inflatable dome supplier in US or EU the smallest dome they will have will be over $3000, which makes sense since none of them are a large factory and they are highly specialized, but $3000 makes sense for an event company, for home VR use not so much.
Issue with fixed projectors is cost, time to install and align, time to edge blend in software and more processing power to render individual views and edge blend them together, again not a viable option for home VR.

What I'm working on is lower resolution than CAVE, sure, but pretty good and it takes 3 minutes to inflate a $100 dome with a $30 blower and a $300 ultra short throw pico projection engine to get up and running. The brightness is pretty good for monocular but is bad after the shutter glasses so that's why I'm looking into retro-reflective sheets now.

Sorry I meant sheet, not fabric. it doesn't have to be a fabric to be sewable. The blackback sheets used in my regular planetarium dome are synthetic, .2 mm PVC sheets. They are not even reinforced by any fabric inside like blackback banners usually are.