Wilson Yan

Hi everyone! I'm a senior research scientist at Google DeepMind and a 4th year Computer Science PhD Student at UC Berkeley, where I am fortunate to be advised by Professor Pieter Abbeel.

Email  /  CV  /  Google Scholar

I am interested in generative modeling. My current research is focused on learning models for accurate video prediction and generation.

A vision-language model trained on million-length sequences of images, language, and video.

A motion-conditioned image animation approach to video editing that enabled a diverse range of edit types

We show that video prediction likelihoods can provide more dense reward signal for learning reinforcement learning agents

We learn an aligned text-image token representation without text-image data, that can then be used for downstream VQA and classification tasks using in-context learning with an LLM

A vector-quantized latent dynamics video prediction model that learns compressed representations to efficiently condition on long videos of hundreds of frames during both training and generation

An video generation architecture that leverages an object-centric transformer design for more scalable and efficient video generation

An efficient and scalable video generation architecture that first learns a VQ-VAE to compress video data, and then learns a GPT-style transformer to model discrete latent codes

Using contrastive methods to learn plannable representations for manipulating deformable objects such as rope and cloth

Performing image interpolation and semantic manipulation (e.g. hair color, smiling, gender) with a PixelCN on facial images using Fisher scores as an embedding space

Sim-to-Real transfer to learning a cloth-spreading policy on a real robot with any human demonstrations