universome / stylegan-v Goto Github PK

View Code? Open in Web Editor NEW

324.0 324.0 35.0 85 KB

[CVPR 2022] StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2

Home Page: https://universome.github.io/stylegan-v

Python 92.07% Shell 0.04% C++ 2.42% Cuda 5.47%

gans generative-adversarial-networks pytorch stylegan video-generation

stylegan-v's People

Contributors

Stargazers

Watchers

stylegan-v's Issues

Questions about calc_metrics_for_dataset

I encountered an issue of calc_metrics_for_dataset.py that, the dummy_dataset_cfg is set as,

dummy_dataset_cfg = OmegaConf.create({'max_num_frames': 10000, 'sampling': {}})

while constructing the VideoFramesFolderDataset, it computes raw_shape by calling

raw_shape = [len(self._video_idx2frames)] + list(self._load_raw_frames(0)[0][0].shape)

self._load_raw_frames(0) has frame_idx as None, therefore requires to sample frames

frames_idx = sample_frames(self.sampling_dict, total_video_len=min(total_len, self.max_num_frames)) + offset

This sampling_dict is an empty dict, as is set in dummy_dataset_cfg, and causes an error that type key is not found in the dict when sample_frames function executes

if cfg['type'] == 'random':

Seemingly I should not arbitrarily set a type for the sampling_dict as both sampling methods require more arguments in sampling_dict, in which there is nothing available.

May I know how to modify the video dataset construction logic for metric evaluation?

FileNotFoundError: [Errno 2] No such file or directory: 'src\\experiment_config.yaml'

I am trying to train the model on my own dataset, but I need 'experiment_config.yaml'. Can you please provide it?

When will the pretrained checkpoints be provided?

Problem regarding true data statics computation for FVD

I ran into trouble trying to reproduce your reported FVD in sky time-lapse dataset. I'll list my problems in the following list

If we take the official dataset from - https://github.com/weixiong-ur/mdgan, the frames are of size 360X640. As far as I can tell reading the code of your data loader, you take a center crop of 360X360 and then resize it down to 256X256. Is that correct?
While evaluating FVD which set do you use to compute real statistics?
1. In the paper in appendix C you write you choose a single clip per video, if we do that with clip length of 128 frames we get about 1K clips from the train set. Bur for FVD we obviously need at least 2048 videos. Therefore, closest we can get is to use maximum 4 clips per video, and we then get 2069 clips. Let's call this is FVD_comp_set.
2. Should we use this set to compute FVD_16f as well? If so do you take first 16 consecutive frames per clip?
3. If we compute FVD against this set (FVD_comp_set) of the given checkpoint we get fvd_2048_16f: 49.94 and fvd_2048_128f:108.82, whereas the mentioned FVD in the paper for the same metrics are 79.52 and 197.0 respectively.
4. Next, I downloaded the sky time-lapse dataset you provided in the checkpoint link. Then reloaded the network checkpoint and attempted to fine-tune it. This evaluates the metric and that reports fvd_2048_16f to be 46.46. Therefore, random train samples as true stats computation of FVD seems not to be the case either.

My request is can you detail the way you computed FVD fro the paper so I can compare our method to yours?

MOCOGAN: training isn't runing

The generator seems to be barebone STG2 generator. And then it only generates a batch of images, which when fed to the video discriminator fails because of dimensionality issues. I don't see where you generate frames auto regressively.

RainbowJelly dataset

Hello,
I am trying to reproduce the result for the RainbowJelly dataset, however, the provided youtube link says that the video is not available anymore.
Is there any other way for me to download the dataset?
Thanks in advance.

Rainbow jelly isn't available

https://www.youtube.com/watch?v=P8Bit37hlsQ

This no longer exists.

Questions about dataset

Hi! Congratulations to your great job!
I am trying to run your code with the FFS dataset, but I met some problems.
I firstly download the FFS videos from the official website, and divide the videos into frames and extract face images to form a dataset as the same format as yours.
Then, how to let the training script know where the dataset is ？ I tried to directly modiy the "path: data/${dataset.name}" configuration in /stylegan-v/configs/training/base.yaml to "path: my_dataset_path", but it will raise error.
Should I repalce all the related dataset configurations with my own dataset path?
e.g. /stylegan-v/configs/env/base.yaml/datasets_dir: ~
/stylegan-v/configs/dataset/base.yaml/path: data/${dataset.name}

FVD Metrics

@universome Hello, I wonder if it is necessary to normalise the data so that it is distributed between [-1,1] when calculating the FVD metrics, I see that there is a normalisation operation in the original repository but I don't find it in your code.
Looking forward to your reply, thanks!

Training diverges

I am trying to reproduce the FaceForensics results. I train with the default settings and after 1kimgs the training diverges while still being at FVD16 of 130. I am using all default settings. What could be the reason? I tried different seeds, same problem.

Minimum System Requirements

It would be really nice to know what are the minimum system requirements to train a custom dataset.

Thanks!

About FaceForensics Dataset

Thanks for your excellent work！
I am new to this field and I'm curious about how do you extract mask for FaceForensics dataset?
I found the this dataset does not provide mask for youtube video.

Low FVD scores and generating inverted samples?

For comparisons, I am training StyleGan-V on a relatively smaller dataset of faces (faces from the how2sign dataset). In particular, I am training StyleGan-V on 10,000 videos and each video has exactly 25 frames. After training for a sufficient amount of time (when I started noticing really good perceptual results), I generated the inference, and the inference looks as follows. The first thing I observe is that the video generated is inverted. Now the orientation of the intermediate predicted/generated video keeps changing during training.
Secondly, I measured the fvd2048_16f scores using this pretrained checkpoint against the dataset on which the model was trained, and I am getting a relatively very high fvd score of ~1100. Is this expected since the model is trained on a fewer number of samples, or if there is something wrong as the inferred videos are inverted? For training on the rest of the datasets, I am able to get videos in the correct orientation (ucf, skytimelapse, rainbow jelly). Also attached below is one frame extracted from the generated video (and the perceptually quality looks good to me).

Error encountered while running multi-gpu training

Hi, I am running stylegan_v on a custom dataset and everything seems to work fine until an error related to "leaked semaphores" is encountered.

The data is correctly loaded, the model is initialized, and it even starts printing out the intermediate losses and accuracy. After a few mins of training, I get this error -- "Exception: process 0 terminated with signal SIGKILL"

Please find the full stack trace below --
Setting up augmentation...
Distributing across 2 GPUs...
Setting up training phases...
Exporting sample images...
Initializing logs...
Training for 25000 kimg...

tick 0 kimg 0.0 time 52s sec/tick 1.5 sec/kimg 62.98 maintenance 51.0 cpumem 2.63 gpumem 9.00 augment 0.000
Traceback (most recent call last):
File "/ssd_scratch/cvit/aditya1/baselines/stylegan-v/experiments/how2sign_faces_styleganv_resized_stylegan-v_random3_max32_how2sign_exp_styleganv_resized-aaf99e5/src/train.py", line 451, in
main() # pylint: disable=no-value-for-parameter
File "/ssd_scratch/cvit/aditya1/baselines/stylegan-v/experiments/how2sign_faces_styleganv_resized_stylegan-v_random3_max32_how2sign_exp_styleganv_resized-aaf99e5/src/train.py", line 446, in main
torch.multiprocessing.spawn(fn=subprocess_fn, args=(args, temp_dir), nprocs=args.num_gpus)
File "/ssd_scratch/cvit/aditya1/baselines/stylegan-v/env/lib/python3.9/site-packages/torch/multiprocessing/spawn.py", line 199, in spawn
return start_processes(fn, args, nprocs, join, daemon, start_method='spawn')
File "/ssd_scratch/cvit/aditya1/baselines/stylegan-v/env/lib/python3.9/site-packages/torch/multiprocessing/spawn.py", line 157, in start_processes
while not context.join():
File "/ssd_scratch/cvit/aditya1/baselines/stylegan-v/env/lib/python3.9/site-packages/torch/multiprocessing/spawn.py", line 105, in join
raise Exception(
Exception: process 0 terminated with signal SIGKILL

/<MY_ENV_PATH>/stylegan-v/env/lib/python3.9/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 34 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '

I tried to generate fake video samples with your pretrained weights but....

I got an error like this... It seems like not my docker problem but in your code...So I want you to check it!
I guess there has been no your pre-trained checkpoint so there no issue like this has been....

Face Forensics dataset preprocessing

Hi! I have a question about the face forensics dataset preprocessing!

Do you guys have some particular strategy for the preprocessing about the ratio that face takes from whole frame, or kind of strategy to choose videos which mostly have frontal faces only?
Because I have processed my face forensic datas but still doesn't fit the data condition of yours!

Thank you!!

Projection of real video with multiple frames

Hello,

I have some questions about the inversion of StyleGAN-V generator in FaceForensics dataset.

In case of image (a single frame), projection works well.
However, when I tried to project the video (multiple frames at once),
I found that the projected video contains almost identical frames in the entire time step.

Is this a normal phenomenon?

For projecting the video (16 frames in my case), I change some codes in "src/scripts/project.py" as below:

adjust times step (0 to 16 frames)
In line 59, from
ts = torch.zeros(num_videos, 1, device=device)
to
ts = torch.arange(num_videos, 16, device=device)
make motion code trainable (comment out the line 110 and uncomment line 109)
extract target_features of real videos per frame, and change the distance as being measured between videos, not frames.
For example, in line 140,
dist = (target_features - synth_features).square().sum()
In batch dimension of target_features and synth_features, they have 16 frames of a single video, not different images as original code does.

Thanks,

SLURM Unrecognized Option

Hi,

We are trying to use the SLURM to submit the job by setting slurm=true but we are getting an unrecognized option error.

sbatch: unrecognized option '--python_bin=~/mc/stylegan-v/env/bin/python'

With the print only option the full command is:

sbatch --constraint=v100 --time=1-0 --gres=gpu:2 --cpus-per-task=5 --mem-per-gpu=32G --cpus-per-gpu=5 --comment=mc_stylegan-v_random3_max32_mc --output ~/mc/stylegan-v/experiments/mc_stylegan-v_random3_max32_mc-None/slurm_0.log  --export=ALL,--project_dir=~/mc/stylegan-v/experiments/mc_stylegan-v_random3_max32_mc-None --python_bin=~/mc/stylegan-v/env/bin/python --python_script=~/mc/stylegan-v/experiments/mc_stylegan-v_random3_max32_mc-None/src/infra/slurm_job.py  src/infra/slurm_job_proxy.sh

I think the error comes from the --export=ALL option which does not have commas between the python bin and project dir. How can we fix this?

The frame sequence is out of order when evaluating fvd

Hi @universome, thanks for your great work!

I followed your instructions to evaluate fvd2048_16f based on calc_metrics_for_dataset.py, however, I found a potential bug.

Specifically, I prepared the ground truth (gt) and predicted datasets according to the following dataset structure:

dataset/
    video1/
        - frame1.jpg
        - frame2.jpg
        - ...
    video2/
        - frame1.jpg
        - frame2.jpg
        - ...
    ...

However, the code in https://github.com/universome/stylegan-v/blob/master/src/training/dataset.py#L319 will sort the frames in lexicographic order of file names (frame1, frame10, frame11, frame12...), instead of numerical order (frame1, frame2, frame3, frame4...):

I wonder if this out-of-order frame sequences lead to inaccurate evaluation results because I3D features are out of order?

'No videos found in the specified archive' error in Evaluating metrics

I get the following IOError -- 'No videos found in the specified archive' when the metric evaluation step is called. The 0th tick runs just fine, after which the error is thrown.

It's coming at result_dict = metric_main.calc_metric( line in training_loop.py. From what I have seen, it is failing to initialize the VideoFramesFolderDataset properly, which is throwing the error - 'No videos found in the specified archive'.

Here is the stack trace --

Evaluating metrics for how2sign_faces_styleganv_resized_stylegan-v_random3_max32_how2sign_exp_styleganv_resized-e2f9580 ...
{"results": {"fvd2048_16f": 5903.660982385764}, "metric": "fvd2048_16f", "total_time": 51.93783378601074, "total_time_str": "52s", "num_gpus": 4, "snapshot_pkl": "network-snapshot-000000.pkl", "timestamp": 1652628736.9207375}
Traceback (most recent call last):
File "/ssd_scratch/cvit/ravi/stylegan-v/experiments/how2sign_faces_styleganv_resized_stylegan-v_random3_max32_how2sign_exp_styleganv_resized-e2f9580/src/train.py", line 451, in
main() # pylint: disable=no-value-for-parameter
File "/ssd_scratch/cvit/ravi/stylegan-v/experiments/how2sign_faces_styleganv_resized_stylegan-v_random3_max32_how2sign_exp_styleganv_resized-e2f9580/src/train.py", line 446, in main
torch.multiprocessing.spawn(fn=subprocess_fn, args=(args, temp_dir), nprocs=args.num_gpus)
File "/ssd_scratch/cvit/ravi/stylegan-v/env/lib/python3.9/site-packages/torch/multiprocessing/spawn.py", line 199, in spawn
return start_processes(fn, args, nprocs, join, daemon, start_method='spawn')
File "/ssd_scratch/cvit/ravi/stylegan-v/env/lib/python3.9/site-packages/torch/multiprocessing/spawn.py", line 157, in start_processes
while not context.join():
File "/ssd_scratch/cvit/ravi/stylegan-v/env/lib/python3.9/site-packages/torch/multiprocessing/spawn.py", line 118, in join
raise Exception(msg)
Exception:

-- Process 2 terminated with the following error:
Traceback (most recent call last):
File "/ssd_scratch/cvit/ravi/stylegan-v/env/lib/python3.9/site-packages/torch/multiprocessing/spawn.py", line 19, in _wrap
fn(i, *args)
File "/ssd_scratch/cvit/ravi/stylegan-v/experiments/how2sign_faces_styleganv_resized_stylegan-v_random3_max32_how2sign_exp_styleganv_resized-e2f9580/src/train.py", line 375, in subprocess_fn
training_loop.training_loop(rank=rank, **args)
File "/ssd_scratch/cvit/ravi/stylegan-v/experiments/how2sign_faces_styleganv_resized_stylegan-v_random3_max32_how2sign_exp_styleganv_resized-e2f9580/src/training/training_loop.py", line 508, in training_loop
result_dict = metric_main.calc_metric(
File "/home2/ravi_mishra/nps/stylegan-v/src/metrics/metric_main.py", line 49, in calc_metric
all_runs_results = [_metric_dictmetric for _ in range(num_runs)]
File "/home2/ravi_mishra/nps/stylegan-v/src/metrics/metric_main.py", line 49, in
all_runs_results = [_metric_dictmetric for _ in range(num_runs)]
File "/home2/ravi_mishra/nps/stylegan-v/src/metrics/metric_main.py", line 123, in fvd2048_128f
fvd = frechet_video_distance.compute_fvd(opts, max_real=2048, num_gen=2048, num_frames=128)
File "/home2/ravi_mishra/nps/stylegan-v/src/metrics/frechet_video_distance.py", line 31, in compute_fvd
mu_real, sigma_real = metric_utils.compute_feature_stats_for_dataset(
File "/ssd_scratch/cvit/ravi/stylegan-v/env/lib/python3.9/site-packages/torch/autograd/grad_mode.py", line 26, in decorate_context
return func(*args, **kwargs)
File "/home2/ravi_mishra/nps/stylegan-v/src/metrics/metric_utils.py", line 195, in compute_feature_stats_for_dataset
dataset = dnnlib.util.construct_class_by_name(**dataset_kwargs)
File "/home2/ravi_mishra/nps/stylegan-v/src/dnnlib/util.py", line 292, in construct_class_by_name
return call_func_by_name(*args, func_name=class_name, **kwargs)
File "/home2/ravi_mishra/nps/stylegan-v/src/dnnlib/util.py", line 287, in call_func_by_name
return func_obj(*args, **kwargs)
File "/ssd_scratch/cvit/ravi/stylegan-v/experiments/how2sign_faces_styleganv_resized_stylegan-v_random3_max32_how2sign_exp_styleganv_resized-e2f9580/src/training/dataset.py", line 329, in init
raise IOError('No videos found in the specified archive')
OSError: No videos found in the specified archive

For now, I have commented the offending lines in training_loop.py and it has started to train. I commented these lines --

# if (snapshot_data is not None) and (len(metrics) > 0):
#     if rank == 0:
#         print(f'Evaluating metrics for {experiment_name} ...')
#     for metric in metrics:
#         result_dict = metric_main.calc_metric(
#             metric=metric,
#             G=snapshot_data['G_ema'],
#             dataset_kwargs=training_set_kwargs,
#             num_gpus=num_gpus,
#             rank=rank,
#             device=device)
#         if rank == 0:
#             metric_main.report_metric(result_dict, run_dir=run_dir, snapshot_pkl=snapshot_pkl)
#         stats_metrics.update(result_dict.results)
# del snapshot_data # conserve memory

Is this expected or am I doing something wrong?

Question about the release of Pre-trained checkpoints

Thank you for your excellent work! I am wondering if you could please inform me when you intend to release the pretrained checkpoints. Your response would be highly appreciated. Thank you very much!

Looking forward for releasing the implementation!

Hello,
I really appreciate your brilliant work on video generation!
Could I know your plan to release the implementation? It will definitely help my future research 😃

Best,
Subin

FVD calculation is not deterministic

I was looking for the implementation of FVD for PyTorch and I came across your work. I really appreciate your effort in standardizing FVD calculation.

I tried to run the code calc_metrics_for_dataset.py on my own datasets and I realized the results are not deterministic, i.e. the FVD scores are slightly different for different runs. It is probably due to frame sampling.

What is the recommended way to run your script? Shall I fix the random seed and run only once or run it multiple times and take the average?

Training Results in Videos with Spring Animation

Hi,

We have a dataset where a liquid flowing in the water from right to left. We are trying to generate similar videos using StyleGAN-V. But the produced videos are have a spring like animation, ie. at first video moves from right to left then left to right. For example the video starts with a nice motion from right to left but after some time it begins to go from left to right:

Trained Videos Merged:
https://drive.google.com/file/d/1tk7y0Tb_nnpTN0NjWbTkVPia-a-MOINN/view?usp=sharing
Example Generated Video
https://user-images.githubusercontent.com/55738066/171813197-cefedf40-9b7c-4802-a3d9-531dfb35d722.mp4
Our Config File: (GitHub did not accept yaml so I had to convert to txt :( )
experiment_config.txt

Will more training solve the issue or is there any optimization that we can do?

Thanks!

Resume training causes output folder File exists error

Hi, I am trying to resume training a model from a pretrained experiment using the command python src/infra/launch.py hydra.run.dir=. exp_suffix=my_experiment_name env=local dataset=ffs dataset.resolution=256 num_gpus=4 training.resume=latest.

The model understands that it needs to resume training and prints the same - "We are going to resume the training and the experiment already exists. That's why the provided config/training_cmd are discarded and the project dir is not created". However, it attempts to recreate the output folder where all the intermediate checkpoints, inferred images, and videos are stored. And if I delete the output directory, it creates a new output directory but starts training from scratch which is weird.

Can you help me understand what I am doing wrong? Below is the full stack trace.

<=== TRAINING COMMAND START ===>
TORCH_EXTENSIONS_DIR=/tmp/torch_extensions cd /ssd_scratch/cvit/aditya1/stylegan-v/experiments/how2sign_faces_styleganv_resized_stylegan-v_random3_max32_50_how2sign-7870d59 && /ssd_scratch/cvit/aditya1/stylegan-v/env/bin/python src/train.py hydra.run.dir=. hydra.output_subdir=null hydra/job_logging=disabled hydra/hydra_logging=disabled
<=== TRAINING COMMAND END ===>
We are going to resume the training and the experiment already exists. That's why the provided config/training_cmd are discarded and the project dir is not created.

Training config is located in `experiment_config.yaml`

Output directory:   /ssd_scratch/cvit/aditya1/stylegan-v/experiments/how2sign_faces_styleganv_resized_stylegan-v_random3_max32_50_how2sign-7870d59/output
Training data:      data/how2sign_faces_styleganv_resized
Training duration:  25000 kimg
Number of GPUs:     2
Number of videos:   10000
Image resolution:   256
Conditional model:  False
Dataset x-flips:    True

Creating output directory...
Traceback (most recent call last):
  File "/ssd_scratch/cvit/aditya1/stylegan-v/experiments/how2sign_faces_styleganv_resized_stylegan-v_random3_max32_50_how2sign-7870d59/src/train.py", line 451, in <module>
    main() # pylint: disable=no-value-for-parameter
  File "/ssd_scratch/cvit/aditya1/stylegan-v/experiments/how2sign_faces_styleganv_resized_stylegan-v_random3_max32_50_how2sign-7870d59/src/train.py", line 437, in main
    os.makedirs(args.run_dir, exist_ok=args.resume_whole_state)
  File "/ssd_scratch/cvit/aditya1/stylegan-v/env/lib/python3.9/os.py", line 225, in makedirs
    mkdir(name, mode)
FileExistsError: [Errno 17] File exists: '/ssd_scratch/cvit/aditya1/stylegan-v/experiments/how2sign_faces_styleganv_resized_stylegan-v_random3_max32_50_how2sign-7870d59/output'

A question on the discriminator detail

Hello,

I have question on the time distance information embedding in Section 3.2. Your paper says that

we encode them with positional encoding [60, 66], preprocess with a 2-layer MLP into p(δ_1), ..., p(δ_{k−1}) ∈ R^d and concatenate into a single vector p_δ ∈ R^(k−1)d

May I ask that what is your choice of the parameter d? And how is the positional encoding done in detail?

Notebook release

Hi!

Thank you for you excellent work! I was just wondering if there was any update on the notebook release?

sky timelapse preprocessing

Hi, thank you for your great jobs and code release!

I have a trouble in training models on sky timelapse.
Among sky timelapse videos, there are very short videos which have less then 256 frames,
so the command you recommended does not work.

In detail, I downloaded dataset from this link.
And then I preprocessed the dataset with crop_video_dataset.py along with satisfying suggested dataset structure.
I run "python src/infra/launch.py hydra.run.dir=. exp_suffix=my_experiment_name env=local dataset=sky_timelapse dataset.resolution=256 num_gpus=4 model.generator.time_enc.min_period_len=256".
I encountered "IndexError: list index out of range" in layers.py", line 396, in random_frame_sampling.
(time_diff: int = random.choice(time_diff_range))

How did you handle very short videos in sky timelapse?

Thank you!

Low IS on UCF101

Hi, I find the IS score of UCF101 real videos in Table 5 is much higher than that I obtained, but I fail to find where is wrong.

I extract frames from UCF101 videos with 32 FPS, then center crop and resize each frame to 256x256 resolution.
Corresponding cmd and results is:

python src/scripts/calc_metrics_for_dataset.py \
--fake_data_path datasets/UCF101/frames/trainval_32fps \
--mirror 1 --gpus 1 --resolution 256 --metrics is50k --verbose 1 --use_cache 0

python src/scripts/calc_metrics_for_dataset.py --fake_data_path datasets/UCF101/frames/trainval_32fps --mirror 1 --gpus 1 --resolution 256 --metrics is50k --verbose 1 --use_cache 0
Real data options:                                              
{'class_name': 'training.dataset.VideoFramesFolderDataset', 'path': None, 'cfg': {'max_num_frames': 10000}, 'xflip': True, 'resolution': 256, 'use_labels':
False}                                                          
Fake data options:                                              
{'class_name': 'training.dataset.VideoFramesFolderDataset', 'path': 'datasets/UCF101/f
rames/trainval_32fps', 'cfg': {'max_num_frames': 10000}, 'xflip': False, 'resolution': 256, 'use_labels': False}
Launching processes...                                          
Calculating is50k...                                            
Downloading https://nvlabs-fi-cdn.nvidia.com/stylegan2-ada-pytorch/pretrained/metrics/inception-2015-12-05.pt ... done
dataset features    items 1024    time 1m 04s       ms/item 62.22
dataset features    items 2048    time 1m 11s       ms/item 6.88
dataset features    items 3072    time 1m 16s       ms/item 5.50
dataset features    items 4096    time 1m 22s       ms/item 5.49
dataset features    items 5120    time 1m 27s       ms/item 5.26
dataset features    items 6144    time 1m 32s       ms/item 4.61
dataset features    items 7168    time 1m 37s       ms/item 4.46
dataset features    items 8192    time 1m 43s       ms/item 5.67
dataset features    items 9216    time 1m 48s       ms/item 5.49
dataset features    items 10240   time 1m 53s       ms/item 4.38
dataset features    items 11264   time 1m 59s       ms/item 6.42                                                                                            
dataset features    items 12288   time 2m 05s       ms/item 6.00          
dataset features    items 13312   time 2m 10s       ms/item 5.05
dataset features    items 14336   time 2m 14s       ms/item 3.76
dataset features    items 15360   time 2m 19s       ms/item 4.52
dataset features    items 16384   time 2m 24s       ms/item 4.45
dataset features    items 17408   time 2m 29s       ms/item 4.91
dataset features    items 18432   time 2m 33s       ms/item 4.37
dataset features    items 19456   time 2m 38s       ms/item 5.06
dataset features    items 20480   time 2m 43s       ms/item 5.12
dataset features    items 21504   time 2m 48s       ms/item 4.27
dataset features    items 22528   time 2m 52s       ms/item 4.02
dataset features    items 23552   time 2m 57s       ms/item 4.87
dataset features    items 24576   time 3m 02s       ms/item 4.82
dataset features    items 25600   time 3m 07s       ms/item 5.42
dataset features    items 26624   time 3m 12s       ms/item 4.83
dataset features    items 27648   time 3m 18s       ms/item 5.08
dataset features    items 28672   time 3m 23s       ms/item 5.35
dataset features    items 29696   time 3m 28s       ms/item 4.51
dataset features    items 30720   time 3m 33s       ms/item 5.29
dataset features    items 31744   time 3m 38s       ms/item 4.55
dataset features    items 32768   time 3m 42s       ms/item 4.34
dataset features    items 33792   time 3m 46s       ms/item 4.09
dataset features    items 34816   time 3m 51s       ms/item 4.48
dataset features    items 35840   time 3m 55s       ms/item 4.07
dataset features    items 36864   time 4m 00s       ms/item 4.86
dataset features    items 37888   time 4m 05s       ms/item 4.76
dataset features    items 38912   time 4m 10s       ms/item 5.04
dataset features    items 39936   time 4m 15s       ms/item 4.49
dataset features    items 40960   time 4m 19s       ms/item 4.47
dataset features    items 41984   time 4m 23s       ms/item 3.90
dataset features    items 43008   time 4m 28s       ms/item 4.29
dataset features    items 44032   time 4m 32s       ms/item 4.49
dataset features    items 45056   time 4m 37s       ms/item 4.25
dataset features    items 46080   time 4m 42s       ms/item 5.23
dataset features    items 47104   time 4m 46s       ms/item 4.28
dataset features    items 48128   time 4m 51s       ms/item 4.41
dataset features    items 49152   time 4m 55s       ms/item 3.67
dataset features    items 50000   time 4m 59s       ms/item 5.52
{"results": {"is50k_mean": 2.733875036239624, "is50k_std": 0.36327630281448364}, "metric": "is50k", "total_time": 414.2170798778534, "total_time_str": "6m 5
4s", "num_gpus": 1, "snapshot_pkl": null, "timestamp": 1665990497.9141998}

TypeError: Descriptors cannot be created directly

Hi team,

I would love to use your model to train a custom dataset, so I cloned the repo, setup the env with conda and added my dataset according to the description in a data folder.
When I run the command I get a strange error, mentionning library versions issues.

Here is the log from the prompt:

$ python src/infra/launch.py hydra.run.dir=. exp_suffix=test_stylegan env=local dataset=strikes dataset.resolution=720 num_gpus=1
<=== TRAINING COMMAND START ===>
TORCH_EXTENSIONS_DIR=/tmp/torch_extensions cd /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69 && /home/lweingart/git/stylegan-v/env/bin/python src/train.py hydra.run.dir=. hydra.output_subdir=null hydra/job_logging=disabled hydra/hydra_logging=disabled
<=== TRAINING COMMAND END ===>
There are uncommited changes. Continue? [y/N]: y
[2023-05-14 17:00:00,711][root][INFO] - creating /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src
[2023-05-14 17:00:00,711][root][INFO] - copying /home/lweingart/git/stylegan-v/src/legacy.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src
[2023-05-14 17:00:00,711][root][INFO] - creating /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/dnnlib
[2023-05-14 17:00:00,711][root][INFO] - copying /home/lweingart/git/stylegan-v/src/dnnlib/util.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/dnnlib
[2023-05-14 17:00:00,712][root][INFO] - copying /home/lweingart/git/stylegan-v/src/dnnlib/__init__.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/dnnlib
[2023-05-14 17:00:00,712][root][INFO] - creating /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/deps
[2023-05-14 17:00:00,712][root][INFO] - creating /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/deps/facial_recognition
[2023-05-14 17:00:00,712][root][INFO] - copying /home/lweingart/git/stylegan-v/src/deps/facial_recognition/helpers.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/deps/facial_recognition
[2023-05-14 17:00:00,712][root][INFO] - copying /home/lweingart/git/stylegan-v/src/deps/facial_recognition/model_irse.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/deps/facial_recognition
[2023-05-14 17:00:00,712][root][INFO] - copying /home/lweingart/git/stylegan-v/src/deps/facial_recognition/__init__.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/deps/facial_recognition
[2023-05-14 17:00:00,712][root][INFO] - copying /home/lweingart/git/stylegan-v/src/deps/__init__.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/deps
[2023-05-14 17:00:00,712][root][INFO] - copying /home/lweingart/git/stylegan-v/src/train.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src
[2023-05-14 17:00:00,713][root][INFO] - creating /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/scripts
[2023-05-14 17:00:00,713][root][INFO] - copying /home/lweingart/git/stylegan-v/src/scripts/convert_videos_to_frames.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/scripts
[2023-05-14 17:00:00,713][root][INFO] - copying /home/lweingart/git/stylegan-v/src/scripts/calc_metrics_for_dataset.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/scripts
[2023-05-14 17:00:00,713][root][INFO] - copying /home/lweingart/git/stylegan-v/src/scripts/profile_model.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/scripts
[2023-05-14 17:00:00,713][root][INFO] - copying /home/lweingart/git/stylegan-v/src/scripts/calc_metrics.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/scripts
[2023-05-14 17:00:00,713][root][INFO] - copying /home/lweingart/git/stylegan-v/src/scripts/__init__.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/scripts
[2023-05-14 17:00:00,713][root][INFO] - copying /home/lweingart/git/stylegan-v/src/scripts/crop_video_dataset.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/scripts
[2023-05-14 17:00:00,713][root][INFO] - copying /home/lweingart/git/stylegan-v/src/scripts/project.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/scripts
[2023-05-14 17:00:00,714][root][INFO] - copying /home/lweingart/git/stylegan-v/src/scripts/clip_edit.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/scripts
[2023-05-14 17:00:00,714][root][INFO] - copying /home/lweingart/git/stylegan-v/src/scripts/construct_static_videos_dataset.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/scripts
[2023-05-14 17:00:00,714][root][INFO] - copying /home/lweingart/git/stylegan-v/src/scripts/convert_video_to_dataset.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/scripts
[2023-05-14 17:00:00,714][root][INFO] - copying /home/lweingart/git/stylegan-v/src/scripts/preprocess_ffs.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/scripts
[2023-05-14 17:00:00,714][root][INFO] - copying /home/lweingart/git/stylegan-v/src/scripts/frames_to_video_grid.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/scripts
[2023-05-14 17:00:00,714][root][INFO] - copying /home/lweingart/git/stylegan-v/src/scripts/generate.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/scripts
[2023-05-14 17:00:00,714][root][INFO] - creating /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/training
[2023-05-14 17:00:00,714][root][INFO] - copying /home/lweingart/git/stylegan-v/src/training/training_loop.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/training
[2023-05-14 17:00:00,714][root][INFO] - copying /home/lweingart/git/stylegan-v/src/training/augment.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/training
[2023-05-14 17:00:00,715][root][INFO] - copying /home/lweingart/git/stylegan-v/src/training/dataset.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/training
[2023-05-14 17:00:00,715][root][INFO] - copying /home/lweingart/git/stylegan-v/src/training/mocogan.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/training
[2023-05-14 17:00:00,715][root][INFO] - copying /home/lweingart/git/stylegan-v/src/training/networks.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/training
[2023-05-14 17:00:00,715][root][INFO] - copying /home/lweingart/git/stylegan-v/src/training/loss.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/training
[2023-05-14 17:00:00,715][root][INFO] - copying /home/lweingart/git/stylegan-v/src/training/__init__.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/training
[2023-05-14 17:00:00,715][root][INFO] - copying /home/lweingart/git/stylegan-v/src/training/layers.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/training
[2023-05-14 17:00:00,715][root][INFO] - copying /home/lweingart/git/stylegan-v/src/training/motion.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/training
[2023-05-14 17:00:00,715][root][INFO] - copying /home/lweingart/git/stylegan-v/src/training/logging.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/training
[2023-05-14 17:00:00,715][root][INFO] - creating /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/__pycache__
[2023-05-14 17:00:00,715][root][INFO] - copying /home/lweingart/git/stylegan-v/src/__pycache__/__init__.cpython-39.pyc -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/__pycache__
[2023-05-14 17:00:00,716][root][INFO] - copying /home/lweingart/git/stylegan-v/src/__init__.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src
[2023-05-14 17:00:00,716][root][INFO] - creating /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/torch_utils
[2023-05-14 17:00:00,716][root][INFO] - copying /home/lweingart/git/stylegan-v/src/torch_utils/custom_ops.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/torch_utils
[2023-05-14 17:00:00,716][root][INFO] - creating /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/torch_utils/ops
[2023-05-14 17:00:00,716][root][INFO] - copying /home/lweingart/git/stylegan-v/src/torch_utils/ops/fma.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/torch_utils/ops
[2023-05-14 17:00:00,716][root][INFO] - copying /home/lweingart/git/stylegan-v/src/torch_utils/ops/grid_sample_gradfix.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/torch_utils/ops
[2023-05-14 17:00:00,716][root][INFO] - copying /home/lweingart/git/stylegan-v/src/torch_utils/ops/upfirdn2d.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/torch_utils/ops
[2023-05-14 17:00:00,716][root][INFO] - copying /home/lweingart/git/stylegan-v/src/torch_utils/ops/bias_act.cu -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/torch_utils/ops
[2023-05-14 17:00:00,716][root][INFO] - copying /home/lweingart/git/stylegan-v/src/torch_utils/ops/conv2d_gradfix.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/torch_utils/ops
[2023-05-14 17:00:00,716][root][INFO] - copying /home/lweingart/git/stylegan-v/src/torch_utils/ops/__init__.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/torch_utils/ops
[2023-05-14 17:00:00,716][root][INFO] - copying /home/lweingart/git/stylegan-v/src/torch_utils/ops/bias_act.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/torch_utils/ops
[2023-05-14 17:00:00,716][root][INFO] - copying /home/lweingart/git/stylegan-v/src/torch_utils/ops/conv2d_resample.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/torch_utils/ops
[2023-05-14 17:00:00,716][root][INFO] - copying /home/lweingart/git/stylegan-v/src/torch_utils/ops/bias_act.h -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/torch_utils/ops
[2023-05-14 17:00:00,716][root][INFO] - copying /home/lweingart/git/stylegan-v/src/torch_utils/ops/upfirdn2d.cu -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/torch_utils/ops
[2023-05-14 17:00:00,716][root][INFO] - copying /home/lweingart/git/stylegan-v/src/torch_utils/ops/upfirdn2d.cpp -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/torch_utils/ops
[2023-05-14 17:00:00,716][root][INFO] - copying /home/lweingart/git/stylegan-v/src/torch_utils/ops/bias_act.cpp -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/torch_utils/ops
[2023-05-14 17:00:00,716][root][INFO] - copying /home/lweingart/git/stylegan-v/src/torch_utils/ops/upfirdn2d.h -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/torch_utils/ops
[2023-05-14 17:00:00,716][root][INFO] - copying /home/lweingart/git/stylegan-v/src/torch_utils/misc.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/torch_utils
[2023-05-14 17:00:00,716][root][INFO] - copying /home/lweingart/git/stylegan-v/src/torch_utils/__init__.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/torch_utils
[2023-05-14 17:00:00,716][root][INFO] - copying /home/lweingart/git/stylegan-v/src/torch_utils/persistence.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/torch_utils
[2023-05-14 17:00:00,717][root][INFO] - copying /home/lweingart/git/stylegan-v/src/torch_utils/training_stats.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/torch_utils
[2023-05-14 17:00:00,717][root][INFO] - creating /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/infra
[2023-05-14 17:00:00,717][root][INFO] - copying /home/lweingart/git/stylegan-v/src/infra/slurm_batch_launch.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/infra
[2023-05-14 17:00:00,717][root][INFO] - copying /home/lweingart/git/stylegan-v/src/infra/utils.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/infra
[2023-05-14 17:00:00,717][root][INFO] - creating /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/infra/__pycache__
[2023-05-14 17:00:00,717][root][INFO] - copying /home/lweingart/git/stylegan-v/src/infra/__pycache__/__init__.cpython-39.pyc -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/infra/__pycache__
[2023-05-14 17:00:00,717][root][INFO] - copying /home/lweingart/git/stylegan-v/src/infra/__pycache__/utils.cpython-39.pyc -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/infra/__pycache__
[2023-05-14 17:00:00,717][root][INFO] - copying /home/lweingart/git/stylegan-v/src/infra/slurm_job_proxy.sh -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/infra
[2023-05-14 17:00:00,717][root][INFO] - copying /home/lweingart/git/stylegan-v/src/infra/__init__.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/infra
[2023-05-14 17:00:00,717][root][INFO] - copying /home/lweingart/git/stylegan-v/src/infra/launch.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/infra
[2023-05-14 17:00:00,717][root][INFO] - copying /home/lweingart/git/stylegan-v/src/infra/experiments.yaml -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/infra
[2023-05-14 17:00:00,717][root][INFO] - copying /home/lweingart/git/stylegan-v/src/infra/slurm_job.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/infra
[2023-05-14 17:00:00,717][root][INFO] - creating /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/metrics
[2023-05-14 17:00:00,717][root][INFO] - copying /home/lweingart/git/stylegan-v/src/metrics/inception_score.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/metrics
[2023-05-14 17:00:00,717][root][INFO] - copying /home/lweingart/git/stylegan-v/src/metrics/video_inception_score.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/metrics
[2023-05-14 17:00:00,717][root][INFO] - copying /home/lweingart/git/stylegan-v/src/metrics/frechet_video_distance.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/metrics
[2023-05-14 17:00:00,717][root][INFO] - copying /home/lweingart/git/stylegan-v/src/metrics/frechet_inception_distance.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/metrics
[2023-05-14 17:00:00,717][root][INFO] - copying /home/lweingart/git/stylegan-v/src/metrics/__init__.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/metrics
[2023-05-14 17:00:00,717][root][INFO] - copying /home/lweingart/git/stylegan-v/src/metrics/kernel_inception_distance.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/metrics
[2023-05-14 17:00:00,717][root][INFO] - copying /home/lweingart/git/stylegan-v/src/metrics/metric_utils.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/metrics
[2023-05-14 17:00:00,718][root][INFO] - copying /home/lweingart/git/stylegan-v/src/metrics/metric_main.py -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/metrics
[2023-05-14 17:00:00,718][root][INFO] - creating /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/configs
[2023-05-14 17:00:00,718][root][INFO] - copying /home/lweingart/git/stylegan-v/configs/infra.yaml -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/configs
[2023-05-14 17:00:00,718][root][INFO] - creating /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/configs/dataset
[2023-05-14 17:00:00,718][root][INFO] - copying /home/lweingart/git/stylegan-v/configs/dataset/mead.yaml -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/configs/dataset
[2023-05-14 17:00:00,718][root][INFO] - copying /home/lweingart/git/stylegan-v/configs/dataset/base.yaml -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/configs/dataset
[2023-05-14 17:00:00,718][root][INFO] - copying /home/lweingart/git/stylegan-v/configs/dataset/ucf101.yaml -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/configs/dataset
[2023-05-14 17:00:00,718][root][INFO] - copying /home/lweingart/git/stylegan-v/configs/dataset/strikes.yaml -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/configs/dataset
[2023-05-14 17:00:00,718][root][INFO] - copying /home/lweingart/git/stylegan-v/configs/dataset/sky_timelapse.yaml -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/configs/dataset
[2023-05-14 17:00:00,718][root][INFO] - copying /home/lweingart/git/stylegan-v/configs/dataset/ffs.yaml -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/configs/dataset
[2023-05-14 17:00:00,718][root][INFO] - copying /home/lweingart/git/stylegan-v/configs/dataset/rainbow_jelly.yaml -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/configs/dataset
[2023-05-14 17:00:00,718][root][INFO] - creating /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/configs/training
[2023-05-14 17:00:00,718][root][INFO] - copying /home/lweingart/git/stylegan-v/configs/training/base.yaml -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/configs/training
[2023-05-14 17:00:00,718][root][INFO] - copying /home/lweingart/git/stylegan-v/configs/config.yaml -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/configs
[2023-05-14 17:00:00,718][root][INFO] - creating /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/configs/model
[2023-05-14 17:00:00,718][root][INFO] - copying /home/lweingart/git/stylegan-v/configs/model/stylegan-v.yaml -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/configs/model
[2023-05-14 17:00:00,719][root][INFO] - copying /home/lweingart/git/stylegan-v/configs/model/mocogan.yaml -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/configs/model
[2023-05-14 17:00:00,719][root][INFO] - copying /home/lweingart/git/stylegan-v/configs/model/base.yaml -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/configs/model
[2023-05-14 17:00:00,719][root][INFO] - creating /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/configs/env
[2023-05-14 17:00:00,719][root][INFO] - copying /home/lweingart/git/stylegan-v/configs/env/base.yaml -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/configs/env
[2023-05-14 17:00:00,719][root][INFO] - copying /home/lweingart/git/stylegan-v/configs/env/local.yaml -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/configs/env
[2023-05-14 17:00:00,719][root][INFO] - copying /home/lweingart/git/stylegan-v/configs/env/raven-local.yaml -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/configs/env
[2023-05-14 17:00:00,719][root][INFO] - creating /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/configs/sampling
[2023-05-14 17:00:00,719][root][INFO] - copying /home/lweingart/git/stylegan-v/configs/sampling/uniform.yaml -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/configs/sampling
[2023-05-14 17:00:00,719][root][INFO] - copying /home/lweingart/git/stylegan-v/configs/sampling/random_1.yaml -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/configs/sampling
[2023-05-14 17:00:00,719][root][INFO] - copying /home/lweingart/git/stylegan-v/configs/sampling/base.yaml -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/configs/sampling
[2023-05-14 17:00:00,719][root][INFO] - copying /home/lweingart/git/stylegan-v/configs/sampling/random.yaml -> /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/configs/sampling
Creating a symlink to /home/lweingart/git/stylegan-v/data, so try not to delete it occasionally!
Created a project dir: /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69
Saved training command in /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/training_cmd.sh
Saved config in /home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/experiment_config.yaml
Traceback (most recent call last):
  File "/home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/train.py", line 24, in <module>
    from training import training_loop
  File "/home/lweingart/git/stylegan-v/experiments/strikes_720_unstable_stylegan-v_random3_max32_test_stylegan-3fecd69/src/training/training_loop.py", line 19, in <module>
    from torch.utils.tensorboard import SummaryWriter # Note: importing torchvision BEFORE tensorboard results in SIGSEGV
  File "/home/lweingart/git/stylegan-v/env/lib/python3.9/site-packages/torch/utils/tensorboard/__init__.py", line 8, in <module>
    from .writer import FileWriter, SummaryWriter  # noqa F401
  File "/home/lweingart/git/stylegan-v/env/lib/python3.9/site-packages/torch/utils/tensorboard/writer.py", line 10, in <module>
    from tensorboard.compat.proto.event_pb2 import SessionLog
  File "/home/lweingart/git/stylegan-v/env/lib/python3.9/site-packages/tensorboard/compat/proto/event_pb2.py", line 17, in <module>
    from tensorboard.compat.proto import summary_pb2 as tensorboard_dot_compat_dot_proto_dot_summary__pb2
  File "/home/lweingart/git/stylegan-v/env/lib/python3.9/site-packages/tensorboard/compat/proto/summary_pb2.py", line 17, in <module>
    from tensorboard.compat.proto import tensor_pb2 as tensorboard_dot_compat_dot_proto_dot_tensor__pb2
  File "/home/lweingart/git/stylegan-v/env/lib/python3.9/site-packages/tensorboard/compat/proto/tensor_pb2.py", line 16, in <module>
    from tensorboard.compat.proto import resource_handle_pb2 as tensorboard_dot_compat_dot_proto_dot_resource__handle__pb2
  File "/home/lweingart/git/stylegan-v/env/lib/python3.9/site-packages/tensorboard/compat/proto/resource_handle_pb2.py", line 16, in <module>
    from tensorboard.compat.proto import tensor_shape_pb2 as tensorboard_dot_compat_dot_proto_dot_tensor__shape__pb2
  File "/home/lweingart/git/stylegan-v/env/lib/python3.9/site-packages/tensorboard/compat/proto/tensor_shape_pb2.py", line 36, in <module>
    _descriptor.FieldDescriptor(
  File "/home/lweingart/git/stylegan-v/env/lib/python3.9/site-packages/google/protobuf/descriptor.py", line 561, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

Would you have any suggestion to help me understand the problem by chance ?

Also, you may notice that I set only one GPU. This is because I am first testing the code and configuration on my workstation before putting it on a multiple GPU cloud instance

I also have a subsidiary question: is it easily feasible to modify the conf/code to generate videos that are not squared, and/or train the model with 16/9 videos ?

Thank you very much in advance for your esteemed help

Cheers

How to avoid setting up the environment from environment.yaml

Hi, I am trying to setup stylegan-v on a slurm server. However, I am a little constrained by the space where I can install the environment related packages. I see that to reduce the effort of setting up the packages themselves, you provide these nice commands --

conda env create -f environment.yaml -p env
conda activate ./env

However, I already have all the pip packages setup in a different environment which I activate when running other repos. I have a custom dataset setup in the format specified in this repository, and the corresponding yaml files modified/created.

When I try the running the training code -- python src/infra/launch.py hydra.run.dir=. exp_suffix=my_experiment_name env=local dataset=custom_dataset dataset.resolution=256 num_gpus=4

I get the following error --

<=== TRAINING COMMAND END ===>
There are uncommited changes. Continue? [y/N]: y
Creating a symlink to /ssd_scratch/cvit/aditya1/baselines/stylegan-v/data, so try not to delete it occasionally!
Created a project dir: /ssd_scratch/cvit/aditya1/baselines/stylegan-v/experiments/how2sign_faces_stylegan-v_random3_max32_how2sign_faces_exp-aaf99e5
Saved training command in /ssd_scratch/cvit/aditya1/baselines/stylegan-v/experiments/how2sign_faces_stylegan-v_random3_max32_how2sign_faces_exp-aaf99e5/training_cmd.sh
Saved config in /ssd_scratch/cvit/aditya1/baselines/stylegan-v/experiments/how2sign_faces_stylegan-v_random3_max32_how2sign_faces_exp-aaf99e5/experiment_config.yaml
sh: 1: /ssd_scratch/cvit/aditya1/baselines/stylegan-v/env/bin/python: not found

This is because I think somehow you are trying to activate the environment "env" which would have been setup if the exact steps as outlined in your repo were followed. How do I run given that I already have a conda environment setup with all the necessary packages installed.

I am however get stylegan_v to train if I follow the steps that sets up the "env" environment.

A question on the cropping of FaceForensics dataset

Hello!

I have a question about the cropping of FaceForensics dataset. I realize that the cropping strategy is different from FFHQ and ArcFace. I want to ask that is your implementation in preprocess_ffs.py equivalent to that of MoCoGAN-HD and TGAN-v2?

Do you use the wide-crop or not?

Thanks!

MEAD results

Hi,
you report MEAD results in the paper in the text and compare to MoCoGANHD but I can not find their numbers in the paper. Can you share the full results for MEAD? Did you compare DIGAN on MEAD as well?

Also I want to use your new evaluation protocol and compare to your model, DIGAN and MoCoGAN-HD and want to use your evaluation scripts. Do you mind to share the metrics scripts / changed dataset file you used for the DIGAN and MoCoGAN-HD repos?

Linearly spaced periods stated in Appendix B.3 is not correct

Your equation in Appendix B.3 states that the sigma vector is linearly spaced between omega_min and omega_max.

However, your code in src/training/motion.py here implements frequencies as log-linearly spaced:

def construct_linspaced_frequencies(num_freqs: int, min_period_len: int, max_period_len: int) -> torch.Tensor:
    freqs = 2 * np.pi / (2 ** np.linspace(np.log2(min_period_len), np.log2(max_period_len), num_freqs)) # [num_freqs]
    freqs = torch.from_numpy(freqs[::-1].copy().astype(np.float32)).unsqueeze(0) # [1, num_freqs]

    return freqs

as the generated sequence satisfies that sigma_{i+1}/sigma_i = C

From my perspective, your code implementation makes more sense, so perhaps you should refine the paper?

Large GPU memory consumption at the beginning of training

Hi, thanks for the great work!

I run the code using 8 A100 GPU cards and find that the gpu memory consumption is extremely large at the first several ticks!
Here is the output log:

tick 0     kimg 0.2      time 1m 31s       sec/tick 21.8    sec/kimg 113.36  maintenance 69.7   cpumem 4.70   gpumem 67.21  augment 0.000
Evaluating metrics for 3sky_timelapse_256_stylegan-v_random3_max32_3-4468dd1 ...
{"results": {"fvd2048_16f": 992.2131880075198}, "metric": "fvd2048_16f", "total_time": 80.59011363983154, "total_time_str": "1m 21s", "num_gpus": 8, "snapshot_pkl": "network-snapshot-000000.pkl", "timestamp": 1655261186.965401}
{"results": {"fvd2048_128f": 1764.0538105755193}, "metric": "fvd2048_128f", "total_time": 230.15506172180176, "total_time_str": "3m 50s", "num_gpus": 8, "snapshot_pkl": "network-snapshot-000000.pkl", "timestamp": 1655261417.1899228}
{"results": {"fvd2048_128f_subsample8f": 1241.4737946211158}, "metric": "fvd2048_128f_subsample8f", "total_time": 54.82384514808655, "total_time_str": "55s", "num_gpus": 8, "snapshot_pkl": "network-snapshot-000000.pkl", "timestamp": 1655261472.0662923}
{"results": {"fid50k_full": 381.6109859044359}, "metric": "fid50k_full", "total_time": 83.23335003852844, "total_time_str": "1m 23s", "num_gpus": 8, "snapshot_pkl": "network-snapshot-000000.pkl", "timestamp": 1655261555.3598747}
tick 1     kimg 5.4      time 11m 28s      sec/tick 23.0    sec/kimg 4.43    maintenance 573.9  cpumem 12.42  gpumem 69.14  augment 0.000
tick 2     kimg 10.6     time 11m 50s      sec/tick 21.4    sec/kimg 4.13    maintenance 0.0    cpumem 12.42  gpumem 10.26  augment 0.000
tick 3     kimg 15.7     time 12m 11s      sec/tick 21.6    sec/kimg 4.17    maintenance 0.0    cpumem 12.42  gpumem 10.26  augment 0.000
tick 4     kimg 20.9     time 12m 33s      sec/tick 21.4    sec/kimg 4.12    maintenance 0.0    cpumem 12.42  gpumem 10.26  augment 0.003
tick 5     kimg 26.1     time 12m 55s      sec/tick 21.7    sec/kimg 4.18    maintenance 0.0    cpumem 12.42  gpumem 10.29  augment 0.010
tick 6     kimg 31.3     time 13m 16s      sec/tick 21.9    sec/kimg 4.22    maintenance 0.0    cpumem 12.42  gpumem 10.29  augment 0.026
tick 7     kimg 36.5     time 13m 39s      sec/tick 22.4    sec/kimg 4.32    maintenance 0.0    cpumem 12.42  gpumem 10.33  augment 0.038
tick 8     kimg 41.7     time 14m 00s      sec/tick 21.6    sec/kimg 4.16    maintenance 0.1    cpumem 12.42  gpumem 10.33  augment 0.036
tick 9     kimg 46.8     time 14m 23s      sec/tick 22.3    sec/kimg 4.30    maintenance 0.1    cpumem 12.42  gpumem 10.32  augment 0.038
tick 10    kimg 52.0     time 14m 44s      sec/tick 21.4    sec/kimg 4.13    maintenance 0.0    cpumem 12.42  gpumem 10.33  augment 0.028

As you can see, gpumem in the first two ticks is abnormal. Do you have any idea about this problem?

FaceForensics download

FaceForensics can not download, anyone could give me a way such as drive?

ImportError: cannot import name 'CoordFuser' from 'src.training.layers'

Hi. Thanks for your grate wor.

I just run your code with inference (generate.py)

But I got an error which is

" ImportError: cannot import name 'CoordFuser' from 'src.training.layers' "

And I checked the src.training.layers. but there is not CoordFuser.
Also no results on searching google CoordFuser.

What is it? and how to fix it?

Thanks.

Run stylegan-v Conditionally

I added a json like this:

{
"labels":
[
["folder1/1.jpg", 0], ["folder1/2.jpg", 0], ["folder1/3.jpg", 0],
["folder2/4.jpg", 1], ["folder2/5.jpg", 1], ["folder2/6.jpg", 1],
["folder3/7.jpg", 2], ["folder3/8.jpg", 2], ["folder3/9.jpg", 2],
]
}

containing labels of each frame in the zip folder but still, when I try to run the code conditionally I see some error.

What is the correct method to train it conditionally?

Error: batch_size should be a positive integer value, but got batch_size=0

Hello guys,

Thank you very much for your work.
I'm trying to train my model based on my own dataset, but when running the training command it always fails with the following error:

File "/home/ubuntu/git/stylegan-v/env-ampere/lib/python3.9/site-packages/torch/utils/data/sampler.py", line 215, in __init__
    raise ValueError("batch_size should be a positive integer value, "
ValueError: batch_size should be a positive integer value, but got batch_size=0

Even if I try to specify the batch size in the command by adding training.batch_size=8 for instance, it keeps failing with the same error message.

The command I use is:

python src/infra/launch.py hydra.run.dir=test_rundir exp_suffix=test_exp env=raven-local dataset=ffs dataset.resolution=512 num_gpus=2 training.batch_size=8

Would someone have any idea what I should do to get around this please?
Thank you very much for your help in advance.

Cheers

Questions about ModDiscrBlock

Hi, Great Work!

I have a question about the discriminator. The current discriminator is conditioned on δ_{i}^x only in DiscrEpilogue, but not the DiscriminatorBlock. Please kindly enlighten me if I misunderstand anything.

Questions about details of hyperparameters in AlignedTimeEncoder

Hi, thank you for sharing this amazing works!!

Can I ask you about the hyperparameters in AlignedTimeEncoder?

When we train the model on SkyTimelapse dataset,
what is the values of the member variables (time_enc_dim, min_period_len, max_period_len) in AlignedTimeEncoder?

I think each values are 256, 256, 1024. Is it correct?

(Current I'm on limited (HW and Internet) situation, It's hard to run it.)

Question about FVD calculation

Thanks you for sharing this great codebase!

I have some question about the FVD caculation in this repository.

As noted in paper, you use (train + test) split for UCF-101 dataset and (train) split for other datasets.

Then, when you calculating the real statistics of FVD,
did you also sample real data from (train + test) split for UCF-101 dataset and (train) split for other datasets?

Thanks,

universome / stylegan-v Goto Github PK

stylegan-v's People

Contributors

Stargazers

Watchers

Forkers

stylegan-v's Issues

Recommend Projects

Recommend Topics

Recommend Org