Generative

InstanceDiffusion: Instance-level Control for Image Generation
SOTA instance-conditioned diffusion model for image generation.
Xudong Wang, Trevor Darrell, Sai Saketh Rambhatla, Rohit Girdhar, Ishan Misra
PDF Cite Code
InstanceDiffusion: Instance-level Control for Image Generation
Generating Illustrated Instructions
Introducing a new task of generating instructions for the task you want to solve with illustrations, and a LLM + Diffusion model based solution.
Sachit Menon, Ishan Misra, Rohit Girdhar
PDF Cite Code
Generating Illustrated Instructions
Motion-Conditioned Image Animation for Video Editing
Image animation FTW again! SOTA video editing results by animating the first frame with motion conditioning.
Wilson Yan, Andrew Brown, Pieter Abbeel, Rohit Girdhar, Samaneh Azadi
PDF Cite
Motion-Conditioned Image Animation for Video Editing
Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioning
A simple and effective approach to high-quality video generation by learning to animate high quality images.
Rohit Girdhar, Mannat Singh, Andrew Brown, Quentin Duval, Samaneh Azadi, Sai Saketh Rambhatla, Akbar Shah, Xi Yin, Devi Parikh, Ishan Misra
PDF Cite Demo
Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioning
MetaPix: Few-Shot Video Retargeting
A dataset to evaluate temporal reasoning in video models.
Jessica Lee, Deva Ramanan, Rohit Girdhar
PDF Cite Slides Video Code
MetaPix: Few-Shot Video Retargeting
Learning a Predictable and Generative Vector Representation for Objects
A single embedding space, good for both generating and understanding 3D models
Rohit Girdhar, David F. Fouhey, Mikel Rodriguez, Abhinav Gupta
PDF Cite Video Code
Learning a Predictable and Generative Vector Representation for Objects