Base mode

• https://github.com/lichao-sun/Mora
• https://github.com/G-U-N/AnimateLCM
• https://github.com/ali-vilab/VGen
• https://github.com/TIGER-AI-Lab/ConsistI2V
• https://github.com/showlab/MotionDirector
• https://github.com/pixeli99/SVD_Xtend
• https://github.com/Picsart-AI-Research/StreamingT2V
• https://ecnu-cilab.github.io/ExVideoProjectPage/ (ExVideo is a post-tuning technique aimed at enhancing the capability of video generation models. We have extended Stable Video Diffusion to achieve the generation of long videos up to 128 frames. The)

Miscellaneous Video Generation Models

• https://github.com/mbzuai-oryx/Video-LLaVA

Spatial & Temporal Transformer

• https://research.nvidia.com/labs/toronto-ai/VideoLDM/

https://huggingface.co/hotshotco/Hotshot-XL https://github.com/PKU-YuanGroup/Open-Sora-Plan

ControlNet

• https://github.com/lllyasviel/ControlNet

Distributive training

• https://huggingface.co/docs/accelerate/

https://github.com/AUTOMATIC1111/stable-diffusion-webui https://github.com/comfyanonymous/ComfyUI https://github.com/facebookresearch/xformers https://github.com/Dao-AILab/flash-attention

FiT: Flexible Vision Transformer for Diffusion Model

SOTA Caption Generation for video : https://github.com/snap-research/Panda-70M https://github.com/willisma/SiT

Unet

• https://huggingface.co/docs/diffusers/en/api/loaders/unet
• https://paperswithcode.com/method/u-net

https://github.com/Vchitect/Latte

VAE

• Consistency Distilled Diff VAE : https://github.com/openai/consistencydecoder

AnimateDiff + freeinit

• https://github.com/guoyww/animatediff/
• https://dagshub.com/ByteDance/AnimateDiff-Lightning
• https://github.com/TianxingWu/FreeInit