StableMotion: Repurposing Diffusion-Based Image Priors for Motion Estimation
Paper • 2505.06668 • Published
How to use ziyiwhat/StableMotion with Diffusers:
pip install -U diffusers transformers accelerate
import torch
from diffusers import DiffusionPipeline
# switch to "mps" for apple devices
pipe = DiffusionPipeline.from_pretrained("ziyiwhat/StableMotion", dtype=torch.bfloat16, device_map="cuda")
prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k"
image = pipe(prompt).images[0]This is the official repo for paper StableMotion: Repurposing Diffusion-Based Image Priors for Motion Estimation
requirements.txt.StableMotion_SIR and StableMotion_RSC respectively.StableMotion_SIRcd StableMotion_SIR && sh sample.sh. You may want to change this file to modify the inference configurations.sh metrics.sh to evaluate the results.StableMotion_SIR/MDM_Flow.cd StableMotion_SIR && sh train.sh. You may want to change this file to modify the training configurations. The default configuration requires approximately 80 GB of VRAM per card.StableMotion_RSCcd StableMotion_RSC && sh sample.sh. You may want to change this file to modify the inference configurations.sh metrics.sh to evaluate the results.Run cd StableMotion_RSC && sh train.sh. You may want to change this file to modify the training configurations. The default configuration requires approximately 40 GB of VRAM per card.
Base model
stabilityai/stable-diffusion-2