We optimize the implemention codes of three representative GFVC works, i.e., FOMM, CFTE and FV2V, and further provide the unified codes regarding the encoder and decoder processes. Current extensions to the original code include DAC and HDAC
-
Download the
CFTE-checkpoint.pth.tar
,FOMM-checkpoint.pth.tar
, andFV2V-checkpoint.pth.tar
to the path./GFVC/CFTE/checkpoint/
,./GFVC/FOMM/checkpoint/
, and./GFVC/FV2V/checkpoint/
respectively. The checkpoints for FOMM, CFTE and FV2V are available at this link while DAC and HDAC are available here. -
The overall testing dataset is available at this link.
-
Download HEVC and VVC and place both codecs in
source/anchors/
directory. -
The specific details can be seen in the subfolder
source
.
similar.mp4
Animating.virtual.faces.mp4
cotrol.mp4
✔2024 «🎯Back To Top»
- (DCC 2024) [GFVC_Review] Generative Face Video Coding Techniques and Standardization Efforts: A Review Chen Bolin, Chen Jie, Wang Shiqi, Ye Yan paper
- (DCC 2024) [GFVC_Translator] Enabling Translatability of Generative Face Video Coding: A Unified Face Feature Transcoding Framework Yin Shanzhi, Chen Bolin, Wang Shiqi, Ye Yan
✔2023 «🎯Back To Top»
- (ICIP 2023) [RDAC] Predictive Coding for Animation-Based Video Compression Goluck Konuko, Stéphane Lathuilière, Giuseppe Valenzise paper
- (arXiv 2023) [IFVC] Interactive Face Video Coding: A Generative Compression Framework Chen Bolin, Wang Zhao, Li Binzhe, Wang Shurun, Wang Shiqi, Ye Yan paper
- (TCSVT 2023) [CTTR] Compact Temporal Trajectory Representation for Talking Face Video Compression Chen Bolin, Wang Zhao, Li Binzhe, Wang Shiqi, Ye Yan paper
✔2022 «🎯Back To Top»
- (ICME 2022) [Bi-Net] Generative Compression for Face Video: A Hybrid Scheme Anni Tang, Yan Huang, Jun Ling, Zhiyu Zhang, Yiwei Zhang, Rong Xie, Li Song paper
- (BMVC 2022) [CVC_STR] Compressing Video Calls using Synthetic Talking Heads Madhav Agarwal, Anchit Gupta, Rudrabha Mukhopadhyay, Vinay P. Namboodiri, C V Jawahar paper
- (ICIP 2022) [HDAC] A Hybrid Deep Animation Codec for Low-bitrate Video Conferencing Goluck Konuko, Stéphane Lathuilière, Giuseppe Valenzise paper
- (ICIP 2022) [DMRGP] Dynamic Multi-Reference Generative Prediction for Face Video Compression Wang Zhao, Chen Bolin, Ye Yan, Wang Shiqi paper
- (CVPRW 2022) [MAX-RS] Neural Face Video Compression Using Multiple Views Anna Volokitin, Stefan Brugger, Ali Benlalah, Sebastian Martin, Brian Amberg, Michael Tschannen paper
- (DCC 2022) [C3DFD] Towards Ultra Low Bit-Rate Digital Human Character Communication via Compact 3D Face Descriptors Li Binzhe, Chen Bolin, Wang Zhao, Wang Shiqi, Ye Yan paper
- (DCC 2022) [CFTE] Beyond Keypoint Coding: Temporal Evolution Inference with Compact Feature Representation for Talking Face Video Compression Chen Bolin, Wang Zhao, Li Binzhe, Lin Rongqun, Wang Shiqi, Ye Yan paper
- (DCC 2022) [SNRVC] Semantic Neural Rendering-based Video Coding: Towards Ultra-Low Bitrate Video Conferencing Hu Yujie, Xu Youmin, Chang Jianhui, Zhang Jian paper
✔2021 «🎯Back To Top»
- (CVPRW 2021) [Mob M-SPADE] Low Bandwidth Video-Chat Compression Using Deep Generative Models Maxime Oquab, Pierre Stock, Daniel Haziza, Tao Xu, Peizhao Zhang, Onur Celebi, Yana Hasson, Patrick Labatut, Bobo Bose-Kolanu, Thibault Peyronel, Camille Couprie paper
- (CVPR 2021) [Face_vid2vid] One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing Wang Ting-Chun, Mallya Arun, Liu Ming-Yu paper
- (ICMEW 2021) [VSBNet] A Generative Compression Framework For Low Bandwidth Video Conference Feng Dahu, Huang Yan, Zhang Yiwei, Ling Jun, Tang Anni, Song Li paper
- (ICASSP 2021) [DAC] Ultra-Low Bitrate Video Conferencing Using Deep Image Animation Goluck Konuko, Giuseppe Valenzise, Stéphane Lathuilière paper
✔2019 «🎯Back To Top»
- (NeurIPS 2021) [FOMM] First Order Motion Model for Image Animation Aliaksandr Siarohin, Stéphane Lathuilière, Sergey Tulyakov, Elisa Ricci, Nicu Sebe paper
✔2024 «🎯Back To Top»
- (JVET January 2024) [JVET-AG0042] AHG16: Proposed Common Software Tools and Testing Conditions for Generative Face Video Compression Bolin Chen, Jie Chen, Ru-Ling Liao, Yan Ye, Shiqi Wang Proposal
- (JVET January 2024) [JVET-AG0048] AHG16: Interoperability Study on Parameter Translator of Generative Face Video Coding Shanzhi Yin, Bolin Chen, Jie Chen, Ru-Ling Liao, Yan Ye, Shiqi Wang Proposal
- (JVET January 2024) [JVET-AG0087] AHG9: On the generative face video SEI message M. M. Hannuksela, F. Cricri, H. Zhang Proposal
- (JVET January 2024) [JVET-AG0088] AHG9: Usage of the neural-network post-filter characteristics SEI message to define the generator NN of the generative face video SEI message M. M. Hannuksela, F. Cricri, H. Zhang Proposal
- (JVET January 2024) [JVET-AG0139] AHG16: Depthwise separable convolution for generative face video compression Renjie Zou, Ru-Ling Liao, Bolin Chen, Jie Chen, Yan Ye Proposal
- (JVET January 2024) [JVET-AG0187] AHG16: Study text for common test conditions and evaluation procedures for generative face video coding (draft 1) Sean McCarthy, Peng Yin, Bolin Chen, Yan Ye, Shiqi Wang Proposal
- (JVET January 2024) [JVET-AG0203] AHG9/AHG16: Common text for proposed generative face video SEI message Jie Chen, Bolin Chen, Yan Ye, Shanzhi Yin, Shiqi Wang, Sean McCarthy, Peng Yin, Guan-Ming Su, Anustup Kuma Choudhury, Walt Husak, Gary J. Sullivan Proposal
✔2023 «🎯Back To Top»
- (MPEG October 2023) [m64987] On VVC-assisted Ultra-low Rate Generative Face Video Coding Yan Ye, Sean McCarthy, Han Boon Teo, Zhuoyi Lv, Shiqi Wang, Kai Zhang, Marta Karczewicz, Iole Moccagatta Proposal
- (JVET October 2023) [JVET-AF0234] AHG9: Common text for proposed generative face video SEI message Bolin Chen, Jie Chen, Yan Ye, Shiqi Wang, Sean McCarthy, Peng Yin, Guan-Ming Su, Anustup Kuma Choudhury, Walt Husak, Gary J. Sullivan Proposal
- (JVET October 2023) [JVET-AF0146] AHG9: On Face Motion Information for Generative Face Video Han Boon Teo, Jing Yuan Thong, Karlekar Jayashree, Chong Soon Lim, Kiyofumi Abe Proposal
- (JVET October 2023) [JVET-AF0048] A Study on Decoder Interoperability of Generative Face Video Compression Bolin Chen, Shanzhi Yin, Jie Chen, Yan Ye, Shiqi Wang Proposal
- (JVET July 2023) [JVET-AE0280] AHG9: Common text for proposed generative face video SEI message Bolin Chen, Jie Chen, Yan Ye, Shiqi Wang, Sean McCarthy, Peng Yin, Guan-Ming Su, Anustup Kuma Choudhury, Walt Husak Proposal
- (JVET July 2023) [JVET-AE0088] AHG9: A study on Generative Face Video SEI Message Han Boon Teo, Jing Yuan Thong, Karlekar Jayashree, Chong Soon Lim, Kiyofumi Abe Proposal
- (JVET July 2023) [JVET-AE0083] AHG9: Common SEI Message of Generative Face Video Bolin Chen, Jie Chen, Yan Ye, Shiqi Wang Proposal
- (JVET July 2023) [JVET-AE0080] AHG9: Generative Face Video SEI message Sean McCarthy, Peng Yin, Guan-Ming Su, Anustup Kuma Choudhury, Walt Husak Proposal
- (JVET April 2023) [JVET-AD0051] AHG9: Common SEI Message of Generative Face Video Bolin Chen, Jie Chen, Yan Ye, Shiqi Wang Proposal
- (JVET January 2023) [JVET-AC0088] AHG9: Generative Face Video SEI Message Bolin Chen, Jie Chen, Shurun Wang, Yan Ye, Shiqi Wang Proposal
- The DCC 2021 keynote video regarding Generative Face Video Compression: Promises and Challenges was presented by Dr. Yan Ye.
- The presentation slides from Dr. Yan Ye can be found in DIS talk: Face video compression with generative networks.
We really appreciate all authors for making their codes available to public.