MTBS3D RT @IFCSummit: Adshir shows new ray tracing method at #IFCSummit. #Adshir #CloudComputing #Cloud
MTBS3D Please help @tifcagroup by completing their survey by December 16th!
MTBS3D RT @IFCSummit: #IFCSummit Visionaries Panel with @IntelSoftware @intel @AMD and @jonpeddie talks Client-to-Cloud. #CloudComputing #FutureCo
MTBS3D RT @IFCSummit: The Futurists Panel moderated by @deantak of @VentureBeat is online. #IFCSummit #CloudComputing #FutureComputing #Futurists
MTBS3D RT @IFCSummit: Daryl Sartain is Director & Worldwide Head of #XR, Displays, and Wireless Ecosystems for @Radeon at @AMD. He is also Chair o…
MTBS3D RT @IFCSummit: Arvind Kumar is a Senior Principal Engineer for @intel @IntelSoftware. At #IFCSummit he explained the workings of the Client…
MTBS3D RT @IFCSummit: Neil Schneider’s #IFCSummit opening presentation. #CloudComputing
MTBS3D RT @IFCSummit: Our #videogames in the #clienttocloud revolution is going on now featuring @playhatchglobal @AccelByteInc @awscloud and @Sha
MTBS3D RT @IFCSummit: On stage now, the #Crypto and #Blockchain markets with Professor Bebo White @bebo and Melissa Brown, Senior Director Develop…
MTBS3D RT @IFCSummit: Finishing off our busy morning is Mark Morrison, Director of Sales for @magicleap. He was talking about Building ROI in the…
MTBS3D RT @IFCSummit: Dominique Pouliquen CEO of @CintooCloud did a live demo of their #clienttocloud technology. #IFCSummit #CloudComputing #VR h…
MTBS3D RT @IFCSummit: Next up is Stefan Holzer CTO of @fyusioninc on Democratizing #3D Content for #ecommerce. #IFCSummit #FutureComputing https:/…
MTBS3D RT @IFCSummit: Enabling an Immersive World panel moderated by Kevin Krewel of @tiriasresearch with Alex Hornstein CTO of @LKGGlass Rama Kri…
MTBS3D RT @IFCSummit: Vinay Narayan VP at @htcvive starts off the morning for us here at #IFCSummit. He is speaking on how #5G is powering the nex…

The Story of Project Holodeck

Project Holodeck is a virtual reality platform designed to make effective VR experiences affordable and possible with readily available off the shelf parts and components.  Their goal is to bring 360-degree full-body virtual reality out of the research lab and into a fun, accessible consumer gaming platform.

Sharing their story and announcing their latest software release is James Iliff, Producer at Project Holodeck and member of the leadership team.

The Moment of Conception

James Iliff, Producer, Project Holodeck

Project Holodeck got started back in March 2012 at the IEEE Virtual Reality conference.  I was visiting there with Nathan Burba and Palmer Luckey as part of the Mixed Reality Lab’s exhibit, and at the time Nate was presenting a research paper on motion tracking and Palmer was working with the Lab on their head-mounted displays. 

Palmer, Nate, and I found ourselves riding in a limo with some famous researches to go check out a telepresence demonstration, and then we realized something:  What if we could get 90% of the VR immersion that you find in a research lab, for 1% of the cost?  We had been working with virtual reality installations for some time, and they were amazing - you could cover your entire body with motion capture markers and achieve full avatar embodiment in a virtual world.  Military-grade HMDs were expensive but had incredible field-of-view.  But few people had access to this experience but researchers at the top of their field and the occasional lucky intern.  Why did it have to be that way? We wanted take this incredible VR experience and do two things:  Make games for it, and make it cheap.

Then Project Holodeck was born.  Palmer had been working on his own HMD for quite some time, and he had numerous iterations (called PR1 – PR4, and others).  His new HMD was made with cheaper components, and we could combine it with consumer-facing motion tracking controllers to have a fully embodied VR play space, not unlike the concept of the Holodeck from The Next Generation!

We instantly started brainstorming all the possibilities.  A zombie apocalypse, an interstellar travel simulator, a pirate ship battle!  A steampunk-style airship game sounded perfect, because players could interact in a 20’ by 20’ play space that would be represented as the ship deck in the virtual world, while at the same time they could fly around in a massive environment and explore.  This idea of micro / macro space got around the spatial limitation problem. 

The ship deck could have cannons, a helm, altimeter, machine guns, afterburners, and all kinds of gadgets.  Players could frantically run around using various instruments and weapons, while having cannon balls and bullets whiz past their heads from enemy ships.  And it all happens hundreds of feet in the air, which is particularly titillating in virtual reality.

This is how our Wild Skies project began – the first game for Project Holodeck.

The Beginning of Modern VR Gaming

John Carmack at E3

As a forum moderator on Meant to be Seen, Palmer continued working on his HMD and shared his progress with the community.  Lo and behold, Palmer got discovered by John Carmack, and after sampling the prototype, John demonstrated the unit at E3.  Once "Oculus Rift" was coined and the Kickstarter launched; the rest is history!

The amazing talented individuals from Gaikai (Brendan and Nate) joined with Palmer and together they crafted the Rift into a beautiful product that could revolutionize the games industry.  They wrangled amazing endorsements from industry giants at Valve, Epic, and more, for the Kickstarter campaign.  I’m sure everyone here remembers what a rush that was!

Before E3 happened, Nate and I pitched Project Holodeck and Wild Skies to the Advanced Games pipeline at USC.  This was a very competitive program, and Advanced Games is considered the capstone class at the number one games school in the country.  We were thrilled to get accepted – and this gave us the opportunity to bring on incredibly talented engineers, designers, and artists to help us bring Project Holodeck to life.

It was in the summer 2012 that we met our Lead Developer Alex Silkin.  We were originally trying to accomplish avatar embodiment with four Kinects, but the algorithms simply weren’t giving us the fidelity we needed to sustain a proper avatar in VR.  The Kinect is a cool piece of hardware, don’t get me wrong, but it is more of a gestural interface than a high precision 6DOF tracking device.  With Alex we decided to switch to the Razer Hydra, which could provide precision tracking of the hands using a magnetic field.  Although the idea of complete body tracking of arms and legs had to be thrown out the window, with the Hydra we could have solid hand tracking with no occlusion issues.  Since the hands were your primary method of interacting, we realized this was the most important part of avatar embodiment anyway. 

Keep in mind this was before the Oculus Rift was widespread and long before Hydra released their own Unity plugins, and we had no SDKs to work with, so this was a massive engineering challenge to undergo.  Alex created an entire Hydra framework from the ground up, and by October we had hand tracking in VR.  We also added the PS Move controllers for absolute positional tracking, so that players could move around the 20’ by 20’ play space.   A gyroscope on the torso could tell the difference between crouching and bending down.  Lenovo laptops ran the entire experience on the back with a big battery, so you didn’t have to deal with tethering and tangled wires.  

The Hardware Wrench

We encountered numerous problems with the hardware, and it ultimately threw a massive wrench in our game design process.  An even bigger challenge than the hardware by itself was networking it with two players using client / server architecture.  I wish I could go into more details about it here, but that would be an article in itself! All in all by December we showcased Project Holodeck locally and internationally, and we were surprised and super exhilarated that this thing was actually working – and people actually had fun!

Project Holodeck in action!

A new medium of games was emerging (and it still is).  VR gaming shares a lot with traditional games, but in so many ways it is totally different and new.  Now that we had a system that worked, we realized we could make more games – not just experiences – but actual games, designed from the beginning for virtual reality.

By January, Oculus was soaring with their developer kit manufacturing pipeline, and our Advanced Games professor Laird Malamed was hired as COO.  Palmer and Laird got us two pilot run units and we integrated them with the Holodeck system, changing from the Socket HMDs that we had mounted on to ski goggles (Sockets graciously provided by Mark Bolas and the team at Mixed Reality Lab!).  The two HMDs are quite similar, but the build in ‘ski mask’ design on the Rift was much more comfortable than our hacked-on ski goggles, and the swappable lenses gave us more options when play testing at various showcases and venues.

The Film Noir Zombie Apocalypse

While showcasing at the Science Gallery in Dublin we kept getting feedback that a Zombie game would be jaw-dropping in VR.  So we decided to make that too. As Wild Skies was well underway, we started a 2nd project called Zombies on the Holodeck so we could try out a new genre with new game mechanics and spatial arrangements.

Zombies on the Holodeck!

This time we didn’t have a flying ship, so we had to have a reason for containing the players in a small play area.  I thought a ‘Hold Out’ scenario might work, where players could pile boxes and debris in order to block Zombies from entering the play area and eating them alive.  This was a tough balance and we never quite got that to work.  Turns out players don’t like bending over and picking things up in VR – its too much effort.  So instead we decided to focus on weapons.  Alex began implementing two-handed weaponry with our custom Hydra framework, and I built out a complete level with a unique grayscale film noir style.  I was joined in late January by Adrian Swanberg, who crafted unique game mechanics such as clip reloading, dynamic Zombie spawning, player damage system, and so much more.

Zombies Concept Art

It was fascinating playing with all these mechanics in VR – and there’s still so much more room to innovate.  Audio for one thing always turns out to be more important than you think at first, especially in VR.  I spent a lot of time crafting a robust audio experience with old school horror film scores, wind, thunder, rain, radios, televisions, and the like.  Our audio director Jeremy Tisser composed his own music to complement the setting. 

Quality audio made the experience so much more immersive.  During development I would often find myself sitting in a room with a Rift and headphones on for hours on end – complete sensory isolation – and start to feel cold and wet from the virtual thunderstorms.  I would pour myself a warm mug of coffee and get comfortable, only to be surprised to step outside and be hit in the face by 80 degree weather.

Zombies on the Holodeck Goes Alpha!

And now after months of development I bring you our pre-alpha demo of Zombies on the Holodeck so you can play it yourself.  This build isn’t for all the hardware we used in Project Holodeck, but you can play the game for Oculus Rift + Razer Hydra controls.  We though this would be the ideal compromise, because most people have these two devices on hand now, and we really want to emphasize the weapons gameplay.  Ultimately I’m super excited to see more games being designed specifically for Hydra + Rift, and in the future I’m looking forward to seeing even more unique VR games to emerge.

The inspiration for Zombies on the Holodeck is to explore what VR might feel like if it was combined with film conventions – specifically horror film conventions from around the time Horror was invented.  The goal is not necessarily to convince the player that they are in a real world, but to convince the player that they are inside a film.  And then, of course, there’s a bunch of zombies and weapons!

Zombies also playable in the full Holodeck system with backpack laptops, PS Move for positional tracking and torso tracking, and a 20’ by 20’ playspace to have fun (survive) in.  Yet our framework is very modular, so you can play with whatever hardware you have available (more on that below). 

We designed this demo to be most playable with Rift + Hydra controls, meaning the level is much larger to support joystick traversal as opposed to actual walking in a play space.

Download and Try it For Yourself!

Download the game:

Mega Mirror:!31hExRhB!XeFTpiyhpE_v0AKxCTxbkynGlBg9ApipoXZHCxyC1eY

How to Play

Instructions for installing, running, and playing the game can be found in the included ReadMe file.  Please give it a solid read-through as it has vital information for playing the game.

Usability Goals

We would love to hear what the VR community has to say about Razer Hydra + Oculus Rift gaming, so send us all the ideas and playtesting feedback you can muster!  You can email us at This email address is being protected from spambots. You need JavaScript enabled to view it.">This email address is being protected from spambots. You need JavaScript enabled to view it., post in this thread, or message me here on the forums. 

This is an early pre-alpha demo, so there are likely going to be numerous bugs.  Much of the art is temporary, but the game mechanics and hardware should function mostly as intended.  Here’s a link to our detailed playtest design goals we use for our own usability tests, but don’t let these hold you back:

That link also includes the report from our last formal usability test, which you may find interesting as a VR game developer. It includes a lot of detailed data about how gamers have responded to the hardware, how long it takes to adjust, common pitfalls, likes / dislikes, and more. 


Go Get'em Guys!

Thanks to Meant to be Seen and its community for being awesome, and for being a part of the VR revolution.  This is one of the most exciting times in my life and I know many fellow developers feel this way too.  Huge thanks to Palmer & Laird for getting us early partner preview units back in January so we could integrate our work with the latest hardware - and also for being such amazing mentors and friends.  And of course thank you to the whole Oculus Team!

Credits for "Zombies on the Holodeck"
James Iliff – Producer, Lead Designer
Alex Silkin – Lead Developer
Adrian Swanberg – Gameplay Programmer, Designer
Jeremy Tisser – Composer
Graham Matuszewski – Prolific Mathematics Genius
Sung J Woo – Concept Artist
Chris Yanson – Concept Artist
Zoey Ikemoto – Concept Artist
Kelly Ma - Concept Artist
Neilson Koerner-Safrata – 3D Artist