Artificial intelligence: a new technology allows you to make any photo 3D scene

Aside from improvements in quality and dimensions, the evolution of photography has left little room for change in the last century. As a result, a technology company set out to revolutionize the imaging market through the process of artificial intelligence (AI) to convert any two-dimensional image to 3D.

And although it is currently not very complicated to achieve a 3D image using a special camera – such as those used to create virtual reality environments – the results leave a lot to be desired when not used. the professional equipment.

The first time a snapshot was taken 75 years ago using a Polaroid camera, it was able to “capture the 3D world in a realistic 2D image,” as NVIDIA recalled in a statement. Today, the company prides itself on achieving just the opposite: “Make a collection of photos a 3D scene in seconds”.

That’s why the company’s expert on graphic plates has demonstrated a new technology called Instant Neural Radiance Field (NeRF) that trains AI algorithms to create 3D objects. from the two-dimensional image.

Known as “reverse rendering,” the process uses AI to “estimate the way light behaves in the real world by allowing researchers to reconstruct a 3D scene from a small number of 2D photos taken from different angles. “

The neural network fills the blank spaces of the 360-degree panorama and predicts the color of light coming from any direction, from any point in 3D space, for more realistic results. Nvidia says this technique works with some type of occlusion.

In the video provided by the company, one can see how the four images taken from different points of view merge into one three-dimensional image.

Instant NeRF uses in-depth learning techniques and opens the door for anyone with relatively affordable equipment to achieve a 3D image almost instantly. This makes creating content for virtual environments relatively easy. Although this is only one of the possible things this technology allows.

Possible uses of NeRF

Instant NeRF can be used for train robots and autonomous cars to understand the size and shape of objects in the real world. For this, 2D images or video sequences will be used. It can also be used in architecture and entertainment to quickly generate digital representations of real environments that creators can modify and build.

As the company defended, it was the “fastest to date”. “The model only needs a few seconds to practice with several dozen still photos, along with data about the camera angles at which they were taken, and then can render the resulting 3D scene in tens of milliseconds. , “which speeds up the process 1000x. .

The company explained about this process that “in a scene that involves people or other moving elements, the faster the pictures, the more beautiful.” In this sense, if there is too much mobility during the 2D capture process, the 3D scene generated by the AI “it will be blurry”.