The long-coveted task of reconstructing 3D geometry from images is still a
standing problem. In this paper, we build on the power of neural networks and
introduce Pix2Vex, a network trained to convert camera-captured images into 3D
geometry. We present a novel differentiable renderer ($DR$) as a forward
validation means during training. Our key insight is that $DR$s produce images
of a particular appearance, different from typical input images. Hence, we
propose adding an image-to-image translation component, converting between
these rendering styles. This translation closes the training loop, while
allowing to use minimal supervision only, without needing any 3D model as
ground truth. Unlike state-of-the-art methods, our $DR$ is $C^ınfty$ smooth
and thus does not display any discontinuities at occlusions or dis-occlusions.
Through our novel training scheme, our network can train on different types of
images, where previous work can typically only train on images of a similar
appearance to those rendered by a $DR$.
Description
[1903.11149] Pix2Vex: Image-to-Geometry Reconstruction using a Smooth Differentiable Renderer
%0 Generic
%1 petersen2019pix2vex
%A Petersen, Felix
%A Bermano, Amit H.
%A Deussen, Oliver
%A Cohen-Or, Daniel
%D 2019
%K 3d image reconstruction single view
%T Pix2Vex: Image-to-Geometry Reconstruction using a Smooth Differentiable
Renderer
%U http://arxiv.org/abs/1903.11149
%X The long-coveted task of reconstructing 3D geometry from images is still a
standing problem. In this paper, we build on the power of neural networks and
introduce Pix2Vex, a network trained to convert camera-captured images into 3D
geometry. We present a novel differentiable renderer ($DR$) as a forward
validation means during training. Our key insight is that $DR$s produce images
of a particular appearance, different from typical input images. Hence, we
propose adding an image-to-image translation component, converting between
these rendering styles. This translation closes the training loop, while
allowing to use minimal supervision only, without needing any 3D model as
ground truth. Unlike state-of-the-art methods, our $DR$ is $C^ınfty$ smooth
and thus does not display any discontinuities at occlusions or dis-occlusions.
Through our novel training scheme, our network can train on different types of
images, where previous work can typically only train on images of a similar
appearance to those rendered by a $DR$.
@misc{petersen2019pix2vex,
abstract = {The long-coveted task of reconstructing 3D geometry from images is still a
standing problem. In this paper, we build on the power of neural networks and
introduce Pix2Vex, a network trained to convert camera-captured images into 3D
geometry. We present a novel differentiable renderer ($DR$) as a forward
validation means during training. Our key insight is that $DR$s produce images
of a particular appearance, different from typical input images. Hence, we
propose adding an image-to-image translation component, converting between
these rendering styles. This translation closes the training loop, while
allowing to use minimal supervision only, without needing any 3D model as
ground truth. Unlike state-of-the-art methods, our $DR$ is $C^\infty$ smooth
and thus does not display any discontinuities at occlusions or dis-occlusions.
Through our novel training scheme, our network can train on different types of
images, where previous work can typically only train on images of a similar
appearance to those rendered by a $DR$.},
added-at = {2020-01-28T16:53:17.000+0100},
author = {Petersen, Felix and Bermano, Amit H. and Deussen, Oliver and Cohen-Or, Daniel},
biburl = {https://www.bibsonomy.org/bibtex/278a70935f5b6991340a3d67f8a3a11c6/kluger},
description = {[1903.11149] Pix2Vex: Image-to-Geometry Reconstruction using a Smooth Differentiable Renderer},
interhash = {936447b38d2f9d83435f4883221e6f33},
intrahash = {78a70935f5b6991340a3d67f8a3a11c6},
keywords = {3d image reconstruction single view},
note = {cite arxiv:1903.11149},
timestamp = {2020-01-28T16:53:17.000+0100},
title = {Pix2Vex: Image-to-Geometry Reconstruction using a Smooth Differentiable
Renderer},
url = {http://arxiv.org/abs/1903.11149},
year = 2019
}