Learning a Predictable and Generative Vector Representation for Objects

Publication
In European Conference on Computer Vision, 2016 (Spotlight Presentation)

What is a good vector representation of an object? We believe that it should be generative in 3D, in the sense that it can produce new 3D objects; as well as be predictable from 2D, in the sense that it can be perceived from 2D images. We propose a novel architecture, called the TL-embedding network, to learn an embedding space with these properties. The network consists of two components: (a) an autoencoder that ensures the representation is generative; and (b) a convolutional network that ensures the representation is predictable. This enables tackling a number of tasks including voxel prediction from 2D images and 3D model retrieval. Extensive experimental analysis demonstrates the usefulness and versatility of this embedding.

Rohit Girdhar
Rohit Girdhar
Research Scientist

My current research focuses on understanding and generating multimodal data, using minimal human supervision