Meant to be Seen

Energy is something I made in 2009 for the iOS App Store. At was simply effects that kind of went to the beat of the music. It was an interactive music player. I have always been interested in music visualizers and stereoscopic anything. Today the hardware has caught up to what can be done with the masses hardware and this is what I have come up with. This is the evolution of the original project from 2009 mixed with a project I worked on for the oculus called Light In the Night. The oculus project did not use AR, it was a virtual skybox.

In essence Its a fireworks simulator that goes to the blasts of fireworks to the beat of your iPod Music. Your iPhone is your view finder and the fireworks live in the world as if it were magic. The coolest part is the integration of this new service called EveryPlay. While the magic is happening the experience can be recorded and optionally shared. This is an example of a users capture that was shared.

https://everyplay.com/videos/6303973

Here is a link to Energy on the App Store.

https://itunes.apple.com/us/app/energy- ... 64932?mt=8

Promo Screenshot


The app has a lot packed in it and I am proud of it with my research getting into Augmented Reality. I tried to utilize as many sensors on the device as possible. Give a try I am curious what people create and share.

Works on iPhone5 or equivalent hardware and up.

If anyone wants to ask any technical questions or has ideas for the future I can integrate feel free to post!

Thanks for checking it out.

- Matt

Nice.

I like it. It has a nice interface.

For the AR, it's critical that you match the latency of the camera to the latency of the IMU tracking.

Keep a history of the last second or two of orientation values and use old values when overlaying onto the video so it sticks and looks right.


Entertainingly enough, you could get an Oculus Rift + Latency tester and:
1. Measure the latency coming from your monitor (press the tester against your monitor while cloning)
2. lay the phone down onto the monitor and film the screen through your app while have the latency tester pressed against the phone's screen
3. Subtract the two measurements from each other to get the phone's camera latency.

Alternatively, you should just add degugging button and add orientation delay in increments of 15ms until it looks right.

Something to look forward to when Altergaze comes to fruition.... if it can be reworked into stereo.... hint hint.

Thanks for the tips.

As its written the app assumes your iPhone is locked on an imaginary ball/pivot. A virtual camera rotates based off the gyroscope in the iPhone in world space where the fireworks live. Then the camera video is put onto another camera layer on a plane and stays still. I matched the virtual camera Field of View to the iPhone Camera. The two camera layers are then blended together creating the effect with the fireworks camera being on top. I may play with the accelerometer in case there is panning or see what I can do to up the effect but if you hold your phone steady and just rotate you will see a nice effect.

I looked at the Altergaze project too on Kickstarter. There wouldn't be too much stereoscopic effect in my current method of fireworks as it uses billboarded blended particles that basically have no depth and always face the camera. What I am saying is because they are not polygonal and you can't "look" around them you won't see a sense of depth if both eyes see the same thing. Someone please correct me if I am wrong. The oculus project used other depth methods (objects in the scene) to give the fireworks a sense of scale to make them feel slightly stereoscopic. And the viewing angle was unbelievable along with the 6DOF head tracking.

Looking forward to shared creations with Energy.

- Matt

https://www.youtube.com/watch?v=SVWlEtqaau4

One of the cooler videos I snapped using Energy.

Bump for 4th of July Week! Free to Download!

https://itunes.apple.com/us/app/energy- ... 64932?mt=8

What can you create?