Wilson Yan

Hi everyone! I was previously a senior research scientist at Google DeepMind. I also completed my PhD at UC Berkeley, where I was fortunate to be advised by Professor Pieter Abbeel.

Email  /  Github  /  Google Scholar

My current research is focused on world models for control.

Achieving diamonds in Minecraft using a purely offline world model-based approach

Learns variable-length tokenization for image and long video

A vision-language model trained on million-length sequences of images, language, and video.

A motion-conditioned image animation approach to video editing that enabled a diverse range of edit types

We show that video prediction likelihoods can provide more dense reward signal for learning reinforcement learning agents

We learn an aligned text-image token representation without text-image data, that can then be used for downstream VQA and classification tasks using in-context learning with an LLM

A vector-quantized latent dynamics video prediction model that learns compressed representations to efficiently condition on long videos of hundreds of frames during both training and generation

An video generation architecture that leverages an object-centric transformer design for more scalable and efficient video generation

An efficient and scalable video generation architecture that first learns a VQ-VAE to compress video data, and then learns a GPT-style transformer to model discrete latent codes

Using contrastive methods to learn plannable representations for manipulating deformable objects such as rope and cloth

Performing image interpolation and semantic manipulation (e.g. hair color, smiling, gender) with a PixelCN on facial images using Fisher scores as an embedding space

Sim-to-Real transfer to learning a cloth-spreading policy on a real robot with any human demonstrations