AI for 3D images and videos (to view in VR)

[CameraTraveler]

I've seen AI algorithms that can generate 360 images from text prompts and other AI algorithms that can create 3D depth maps from 2D images (to view in VR)

#1 Looking for a workflow that brings it together in a way that can turn text to 2D (180 or 360) > depth map > stereographic (side by side) 180/360 image.

#2 Also interested in turning a exisiting 2D 180 or 360 videos into 3D 180/360 videos.

#3 Finally curious if AI inpainting is able to extend out 16:9 landscape images into much wider but correct single point FOV 180 or 360 images?

Ideas?

[nimatoed]

You should look into Blockade Labs, they have a tool that'll do text-to-360 with depth.

My company also has an image to depth and image to 3D tool called LeiaPix Converter: http://convert.leiapix.com

We will be releasing an API in the coming weeks so any app can hit our media cloud and get back a depth map or send an image and receive back a 3D image.

We also already have 2D 360 to 3D 360 automatic conversion, but it's only available on the Lume Pad 2 tablet in the LeiaPlayer app for now.

[treeeddd]

I assume you are doing stable diffusion? In automatic1111 webui, there is an extension called depthmap, this can convert 2d images to 3d using different depthmaps. The depthmaps can be very good, and the 3d image conversion is also very good. It's very easy to do. Doing video is the same method, but each frame is converted.

This thread on avsforum uses depthmap to convert video into 3d: https://www.avsforum.com/threads/a-new- ... i.3228063/

So far these are the best methods I find to convert 2d to 3d images. Some programs like 3dcombine and https://stereo.jpn.org/eng/, can use depthmaps to convert 2d images to 3d.

There can be some post processing of the depthmap images to have less artifacts in the 3d images as well.