microsoft / nuwa Goto Github PK

Is there any plan to release the PeppaPig dataset? Hope to reproduce some works on this dataset.

Thanks for this project, it's simply amazing.
Any plans to share the pre-trained model(s)? That would be super helpful to compare it against CLIP, DALL-E, VQGAN, and these models ensemble combinations.

Thanks a lot

As mentioned in #2, #4, #5, #6, #7, #9, #14, #15, #16, #19, they aren't going to give you a fuck, as if they care. Especially they are in China, a place disconnected from the world.

You can find the following codes to achieve the corresponding tasks (many of them also achieves superior quality).
And remember to cite them (instead of NUWA of course) to support of their efforts in making their codes publicly available.

• Text to image:
  • https://github.com/CompVis/stable-diffusion
• Text to video:
  • https://github.com/THUDM/CogVideo
  • https://github.com/lucidrains/imagen-pytorch
• Infinite-visual synthesis:
  • https://github.com/google-research/google-research/tree/master/infinite_nature_zero
  • https://github.com/hubert0527/infinityGAN
  • https://github.com/universome/alis

When will the code of [NUWA-XL: Diffusion over Diffusion for eXtremely Long Video Generation] be released?

First off congratulation to this amazing work. I think you managed to find the closing gap to make generative Deep learning relevant for real-world application, besides being just a nice toy as previous work in this area.

However to truly judge the performance of your approach I have to say I was a bit disappointed after reading your paper there was not a single note on execution time for either training or more crucial actually sampling of a single final image.

Would you be able to provide some numbers on how long a sample generation takes for a 4kx1k images with 256^2 patch size and on which setup?

Also if possible could you also shed some light on training times and which setup was used.

Thank you!

Thank you for your excellent work! From the paper, I know that you sample 10 frames from a 2.5 FPS video. I want to know how many frames per video in the dataset you use? Is the 10 frames evenly sampled from the video or the first 10 frames in the video?

I would like to use the software for image development for my university architecture thesis (argentina)

The idea is to animate several images.

The code of this paper will be extremely helpful in evaluating it against other techniques and fine tuning as well.

A very polite question, When will the code and model be released?
I would appreciate it if I could expose the code and the model.

The source code can be public? I want to reproduce your work because the task sound interesting :)

This repository is currently missing a LICENSE.MD file outlining its license. A license helps users understand how to use your project in a compliant manner. You can find the standard MIT license text at the Microsoft repo templates LICENSE file: https://github.com/microsoft/repo-templates/blob/main/shared/LICENSE.
If you would like to learn more about open source licenses, please visit the document at https://aka.ms/license.

As the paper says in appendix：
"For example, for long videos or high-resolution frames with large h, w, s, usually (e^h)(e^w)(e^s)< (h + w + s)"
Is there any situation that (e^h)(e^w)(e*s)< (h + w + s)?

In this paper, we show that simply using 2D VQ-GAN to encode each frame of a video can also generate temporal consistency videos and at the same time benefit from both image and video data.

In the paper, I believe you mean "temporally consistent" here. Subtle change in wording.

Judging by the results, the transformer is taking in a single frame, and would be considered an Image to Video process.
Something like video inpainting or camera FOV extrapolation(like in FGVC) would be input video -> output video.
Am I missing something in the documentation that maybe shows it as some sort of sparse video interpolation where it can input more than a (D1, D2, single frame); or was it called V2V in order to match the I2I label on the inpainting/image completion counterparts?

Additionally, there isn't a direct link to the paper, which documents that the V2V model only takes in a single image.
https://arxiv.org/abs/2111.12417

look forward coming

快点开源

Has the project not been developed yet?

There are important files that Microsoft projects should all have that are not present in this repository. A pull request has been opened to add the missing file(s). When the pr is merged this issue will be closed automatically.

Microsoft teams can learn more about this effort and share feedback within the open source guidance available internally.

Merge this pull request