akanimax / bmsg-gan Goto Github PK

Hi,

Thank you for sharing your work. It seems that the resolution of Flower dataset is not 256x256. How do you process this dataset, resize of crop? Thanks.

Hi， as I know this trick start with StyleGAN， but I not train GAN successful with it，I see， you also recommend it， So， I want to ask， how importance about eql with your feeling

Nice work. It helps me a lot. Can you please provide any pretrained model for celeba-hq or ffhq (the new released dataset in stylegan paper)

hi, It is me again.

I think many kind of method for computing IS and FID , so would you release your code about how to compute IS and FID for cifar10 , celeba_HQ, dataset?

Could you please share the 128*128 pretrained models on celebA?

Hi, @markstrefford @owang @sridharmahadevan @akanimax @huangzh13

I trained successfully anomalyGAN with your MSG-GAN using my only normal images.

And now , I want to just test the discriminator of this GAN using my abnormal images...

How can I extract the discriminator from your MSG-GAN ?

Thanks in advance.

Best,
@bemoregt.

Elapsed [0:05:20.807550] batch: 252 d_loss: 0.000000 g_loss: 9.195433
Elapsed [0:05:32.021947] batch: 261 d_loss: 0.000000 g_loss: 9.544848
Elapsed [0:05:43.313830] batch: 270 d_loss: 0.000000 g_loss: 8.533874
Elapsed [0:05:54.605321] batch: 279 d_loss: 0.000000 g_loss: 8.899170
Elapsed [0:06:06.190011] batch: 288 d_loss: 0.000000 g_loss: 18.478725
Elapsed [0:06:17.612306] batch: 297 d_loss: 0.000000 g_loss: 7.378197

What is the config parameter to generate 512 or 1024px images? Default training setup generates up to 256*256 only

Hi, @markstrefford @owang @sridharmahadevan @akanimax @huangzh13

I met this error when run python3 demo.py

tezro@sgi320:~/deepZ/BMSG-GAN/sourcecode$ python3 demo.py --generator_file ./GAN_GEN_60.pth
Traceback (most recent call last):
File "demo.py", line 9, in
from generate_multi_scale_samples import progressive_upscaling
File "/home/tezro/deepZ/BMSG-GAN/sourcecode/generate_multi_scale_samples.py", line 9, in
from torchvision.utils import make_grid
File "/usr/local/lib/python3.7/dist-packages/torchvision/init.py", line 1, in
from torchvision import models
File "/usr/local/lib/python3.7/dist-packages/torchvision/models/init.py", line 12, in
from . import detection
File "/usr/local/lib/python3.7/dist-packages/torchvision/models/detection/init.py", line 1, in
from .faster_rcnn import *
File "/usr/local/lib/python3.7/dist-packages/torchvision/models/detection/faster_rcnn.py", line 13, in
from .rpn import AnchorGenerator, RPNHead, RegionProposalNetwork
File "/usr/local/lib/python3.7/dist-packages/torchvision/models/detection/rpn.py", line 8, in
from . import _utils as det_utils
File "/usr/local/lib/python3.7/dist-packages/torchvision/models/detection/_utils.py", line 74, in
@torch.jit.script
File "/usr/local/lib/python3.7/dist-packages/torch/jit/init.py", line 364, in script
graph = _script_graph(fn, _frames_up=_frames_up + 1)
File "/usr/local/lib/python3.7/dist-packages/torch/jit/init.py", line 359, in _script_graph
ast = get_jit_ast(fn)
File "/usr/local/lib/python3.7/dist-packages/torch/jit/frontend.py", line 132, in get_jit_ast
return build_def(SourceRangeFactory(source), py_ast.body[0])
File "/usr/local/lib/python3.7/dist-packages/torch/jit/frontend.py", line 151, in build_def
build_stmts(ctx, body))
File "/usr/local/lib/python3.7/dist-packages/torch/jit/frontend.py", line 123, in build_stmts
stmts = [build_stmt(ctx, s) for s in stmts]
File "/usr/local/lib/python3.7/dist-packages/torch/jit/frontend.py", line 123, in
stmts = [build_stmt(ctx, s) for s in stmts]
File "/usr/local/lib/python3.7/dist-packages/torch/jit/frontend.py", line 140, in call
return method(ctx, node)
File "/usr/local/lib/python3.7/dist-packages/torch/jit/frontend.py", line 205, in build_Assign
rhs = build_expr(ctx, stmt.value)
File "/usr/local/lib/python3.7/dist-packages/torch/jit/frontend.py", line 140, in call
return method(ctx, node)
File "/usr/local/lib/python3.7/dist-packages/torch/jit/frontend.py", line 314, in build_Call
func = build_expr(ctx, expr.func)
File "/usr/local/lib/python3.7/dist-packages/torch/jit/frontend.py", line 140, in call
return method(ctx, node)
File "/usr/local/lib/python3.7/dist-packages/torch/jit/frontend.py", line 300, in build_Attribute
value = build_expr(ctx, expr.value)
File "/usr/local/lib/python3.7/dist-packages/torch/jit/frontend.py", line 140, in call
return method(ctx, node)
File "/usr/local/lib/python3.7/dist-packages/torch/jit/frontend.py", line 422, in build_Subscript
raise NotSupportedError(base.range(), "slicing multiple dimensions at the same time isn't supported yet")
torch.jit.frontend.NotSupportedError: slicing multiple dimensions at the same time isn't supported yet
proposals (Tensor): boxes to be encoded

What's wrong to me ??

Best,
@bemoregt.

Hi, Would supply the flower dataset link ??

Hi!
I like your paper.

Here is my version of your algorithm trained to generate dresses:
https://github.com/TropComplique/multi-scale-gradient-gan.

But I made some modifications:

1. I use self modulation (stylegan-like generator),
2. I use R1 penalty for discriminator,
3. I use PacGAN discriminator.

When I use wgan-gp as a loss function, the training fails. Any explanation?

I encountered this error when I was training in the following configuration：

python train.py --images_dir='data/bird' --folder_distributed=True --sample_dir='samples/exp_1' --model_dir='models/exp_1'

I changed default options in train.py where
"default='os.environ['SM_MODEL_DIR']," to “samples/exp_1',” "default=os.environ['SM_CHANNEL_TRAINING'],“ to "default='data/bird',",
"default='os.environ['SM_MODEL_DIR']," to “samples/exp_1',”

The dataset is loaded correctly.
but it failed in training with this error :

Traceback (most recent call last):
File "train.py", line 267, in
main(parse_arguments())
File "train.py", line 261, in main
start=args.start
File "/data/user/BMSG-GAN/sourcecode/MSG_GAN/GAN.py", line 417, in train
for (i, batch) in enumerate(data, 1):
File "/data/user/.local/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 582, in next
return self._process_next_batch(batch)
File "/data/user/.local/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 608, in _process_next_batch
raise batch.exc_type(batch.exc_msg)
RuntimeError: Traceback (most recent call last):
File "/data/user/.local/lib/python3.5/site-packages/torch/utils/data/_utils/worker.py", line 99, in worker_loop
samples = collate_fn([dataset[i] for i in batch_indices])
File "/data/user/.local/lib/python3.5/site-packages/torch/utils/data/utils/worker.py", line 99, in
samples = collate_fn([dataset[i] for i in batch_indices])
File "/data/user/BMSG-GAN/sourcecode/data_processing/DataLoader.py", line 132, in getitem
img = self.transform(img)
File "/data/user/.local/lib/python3.5/site-packages/torchvision/transforms/transforms.py", line 60, in call
img = t(img)
File "/data/user/.local/lib/python3.5/site-packages/torchvision/transforms/transforms.py", line 163, in call
return F.normalize(tensor, self.mean, self.std, self.inplace)
File "/data/user/.local/lib/python3.5/site-packages/torchvision/transforms/functional.py", line 208, in normalize
tensor.sub(mean[:, None, None]).div(std[:, None, None])
RuntimeError: output with shape [1, 128, 128] doesn't match the broadcast shape [3, 128, 128]

I think this error has nothing to do with my modification.
Could you please tell me what caused this error?

How much GPU memory to use for 1024*1024?

Hello @akanimax , awesome results.

I have my own use case which is generate realistic images of vehicles.
I am using a publicly available dataset for testing, I think stanford cars dataset.

I am wondering how important alignment is. For example if I have cars in different orientation (facing left, right etc) should I make them all aligned for the GAN training? If I don't, will this hinder the ability to generate images?

Thanks for any insight,
Mason

Hi, I've added config to allow this to run on AWS SageMaker here:
https://github.com/markstrefford/Sagemaker-BMSG-GAN/
Do you want a PR for this?

Hi， I have test different dataset by using BMSG-GAN， and it work well，so can I ask when you will update this repo to the latest one ??

Seeing very interesting results so far with the code in this repo. I'm wondering if @akanimax or anyone else has had any luck speeding up training with this repo by using half-precision or mixed-precision?

Hi,

Thanks for a cool implementation. Your notebook ends at fitting - now I am a little bit confused on how to actually generate samples on the fit sagemaker model?

Can you just deploy and predict?

Thanks!

Hi, you published BBMSG-GAN. And In order to match the performance of ProGANs, you increased the size of the Discriminator (the one here in BMSG-GAN is technically half that of ProGAN) and use tricks like Batch Spoofing. So, do you mean that the performance of Pro-GANs is better than BMSG-GAN? But BBMSG-GAN is better than Pro-GANs? Of course, the number of parameters is bigger?

i ran !python train.py --depth=7 --latent_size=512 --images_dir='../data/train' --num_epochs=100 --num_samples=8 --sample_dir=samples/exp_1 --model_dir=models/exp_1
and got this error:

Traceback (most recent call last):
  File "train.py", line 267, in <module>
    main(parse_arguments())
  File "train.py", line 57, in parse_arguments
    default=os.environ['SM_CHANNEL_TRAINING'],
  File "/usr/lib/python3.6/os.py", line 669, in __getitem__
    raise KeyError(key) from None
KeyError: 'SM_CHANNEL_TRAINING'

i'm using google colab on pytorch 1.7 version
dataset im using is "Linnaeus 5 256X256" from here
data folder has 2 folders inside: train and test. inside them folders containing jpg images

what i tried and didnt work:
used a different path
pytorch 1.0.0 version
on my local machine
removing some & all parameters
using a different dataset

i hope somebody help me, i spent hours trying to find a solution and didn't succeed.

Thank you.

There are 8k+ 256*256 images in my datasets, I set batchsize=32 and trained with 4 12GB GPUs, but it's not as good as I set batchsize = 4 and trained with one GPU,neither the training speed nor the image quality. Why a smaller batch size results better results? Is there a best batchsize?
Beside,I set batchsize=4, when epoch>70, g-loss went up obviously,and qualities of generated images went worse.Why did this happen? (Training process was interrupted when epoch = 54, I reloaded weight files and optimer states from epoch 53 )
Thanks!

I think input feature x , the shape of x is (BS, C, H, W)
and the shape of y is (BS, 1, H, W)
so the output of concate them should be (BS, C+1, H, W), but (BS, C, H, W) in code ?

Hi, @akanimax ,thanks for sharing your code. I want to train my own data (colour fundus photos) to data augmentation. the number and resolution of my data set are 235 and 1024*1024. I trained more than 260 epochs, but generated images are still bad and happened the error "No space left on disk" , it is that the saved models and samples used too much space of my disk(170G). How can I solve the problems? Thanks a lot.

Hi, I read your newest version paper (v3) in arxiv, and i saw that add MSG_styleGAN information, I'm glad to see it , but I have some question about paper :

1. I think you paper said the result of showing in paper is trained MSG_ProGAN by using WGAN-GP, and trained MSG_styleGAN by using Non-saturating GAN loss with 1-sided gp. This repo is for training MSG_ProGAN but default GAN loss is relativistic-hinge and then I changed it to wgan-gp it failed ,because of not support for mult-scale Discriminator , Can you update this function code. (B.T.W) I implemented by myself but I don't right or false)

2. why default using relativistic-hinge not WGAN-gp

Why are all the images generated by me using the trained network exactly the same?
I observed the training process, and the images generated using different noises are still somewhat different, but gradually they become the same image. What is the reason? Is my training data too small?

After a whole training with

$ python3 sourcecode/train.py --depth=7 --latent_size=256 --images_dir=DATASET --sample_dir=sampledir/exp2 --model_dir=modeldir/exp2

when launching

$ python3 sourcecode/generate_samples.py --generator_file=modeldir/exp2/GAN_GEN_100.pth --depth=7 --out_dir=outputdir

I got:

''
Creating generator object ...
Loading the generator weights from: modeldir/exp2/GAN_GEN_100.pth
Traceback (most recent call last):
File "sourcecode/generate_samples.py", line 134, in
main(parse_arguments())
File "sourcecode/generate_samples.py", line 105, in main
gen.load_state_dict(
File "~/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for DataParallel:
Missing key(s) in state_dict: "module.layers.7.conv_1.weight", "module.layers.7.conv_1.bias", "module.layers.7.conv_2.weight", "module.layers.7.conv_2.bias", "module.layers.8.conv_1.weight", "module.layers.8.conv_1.bias", "module.layers.8.conv_2.weight", "module.layers.8.conv_2.bias", "module.rgb_converters.7.weight", "module.rgb_converters.7.bias", "module.rgb_converters.8.weight", "module.rgb_converters.8.bias".
''

Both training and generation are executed on the same machine with multiple GPUs enabled and ready.

Sorry @akanimax to open a new issue but this is a question I would love some help with. Basically I want to modify your architecture slightly to add the functionality of conditioning on image labels. For example if training on the flowers set, rather than just load image data to the model, load both image data + accompanying image label data like image of sunflower and label like tall yellow flower, black seeds in middle etc.

Can you give me an idea of how to start with this with your code?
My plan to generate labels is use subfolders in image set where each subfolder name contains images of that name.

Thanks again,
Mason

I try to implement msg way to cycleGAN for two weeks.
The generator : Unet-withSkipConnection and downsample(k=4, s=2 conv) upsample(Deconv) 8x.

But the D_loss, both D_A and D_B always decreased near to zero and G_loss(lsgan) always stuck in 0.01 ...

What I want to know is whether this combination will work? msg + cycleGAN?
Anybody tried before?

Hello!

I am using msggan, but I get worse results than in the paper.

I tried celebHQ whole dataset (100k samples), with resolution 64x64 and 128x,128.
For 64x64 I have done ~250 epochs
and for 128x128 ~ 47epochs.

Results are not bad (much better than DCGAN), but they have worse texture quality than in msggan paper, and only 30% samples looks realistic. 70% samples look like monster (weird faces, weird artifacts etc).

I am using 3060ti
batch size 12
Rest of hyperparameters are default like in repo (LR = 0.003, latent_size = 512, loss_function = relativistic-hinge, flip-augment=True)

Any ideas? Can anyone share results?

I am attaching my 64x64 results (every epoch is 100k samples, so 251 epoch is 25mln real samples)

and 47 epoch for 256x256

I am a bit lame with GitHub pull requests so here is a note that a call is deprecated:

generate_samples.py:126: DeprecationWarning: imsave is deprecated!
imsave is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.
Use imageio.imwrite instead.
ss_image.squeeze(0).permute(1, 2, 0).cpu())

Hi Akanimax, thanks for your great work!

Could you please also release the training scripts for CIFAR10 generation?

Hi, @owang @sridharmahadevan @akanimax @huangzh13

I have met this error when run train.py ...
What's wrong to me?

oem@sgi:~/BMSG-GAN/sourcecode$ python3 train.py --depth=7 --latent_size=128 --images_dir='../data/celebJapan/train' --sample_dir=samples/exp_2 --model_dir=models/exp_2
Total number of images in the dataset: 6604

error message -
Starting the training process ...

Epoch: 1
Elapsed [0:00:04.581270] batch: 1 d_loss: 4.346926 g_loss: 6.674685
Traceback (most recent call last):
File "train.py", line 254, in
main(parse_arguments())
File "train.py", line 248, in main
start=args.start
File "/home/oem/BMSG-GAN/sourcecode/MSG_GAN/GAN.py", line 482, in train
gen_img_files)
File "/home/oem/BMSG-GAN/sourcecode/MSG_GAN/GAN.py", line 345, in create_grid
samples = [Generator.adjust_dynamic_range(sample) for sample in samples]
File "/home/oem/BMSG-GAN/sourcecode/MSG_GAN/GAN.py", line 345, in
samples = [Generator.adjust_dynamic_range(sample) for sample in samples]
File "/home/oem/BMSG-GAN/sourcecode/MSG_GAN/GAN.py", line 96, in adjust_dynamic_range
data = data * scale + bias
TypeError: mul() received an invalid combination of arguments - got (numpy.float32), but expected one of:

• (Tensor other)
  didn't match because some of the arguments have invalid types: (numpy.float32)
• (Number other)
  didn't match because some of the arguments have invalid types: (numpy.float32)

Thanks in advance ~

Hi, I have train the MSG-GAN model with flower-102 dataset, and I have trained model weight.
So, how can I quick test this model to generate same flower from noise ??

Will I write load network and generate code ?? or you have this code but Not show then in ReadMe

Would you update ReadMe about how to test ?? this is better for use, thank you

Is there a way we can project back the images to get the latent vectors just like it is available in StyleGan2 repo?

Hi @akanimax

First of all many thanks for this amazing contribution.

In the paper, it is stated we modified the gradient penalty to be the average of the penalties over each input.
I didn't see any of that in the original MSG-GAN tensorflow implementation, neither in the BMSG-GAN implementation: it seems rather like only the gradient penalty of the higher resolution image is used.

Did I miss something, or is it a slight variation between the paper and the implementation?

Cheers

edit: related to #35

USE_SAGEMAKER is checked at the beginning of train.py but not in the arg code which includes a few sagemaker variables. Easy enough to comment out but would be nice just to pull the USE_SAGEMAKER through completely in the code for all variables.

@akanimax.Hi,
Nice work,I've implemented your code.It helped me a lot. But,I want to change the number of gpus.I own two Tesla graphics CARDS.Would you please tell me how to modify it! Thanks!

Hi, @markstrefford @owang @sridharmahadevan @akanimax @huangzh13

I want to stop and resume the training continuously with saved model.

My resuming commad : "$ python3 train.py --start 300" # I stop at 350 epoch

But, My Training sample is started from the scratch. --;

What's wrong to me ??

Thanks.
Best,
@bemoregt.

I trained the model with:
python train.py --depth=7 --latent_size=512 --images_dir=../data/flowers --sample_dir=samples/exp_1 --model_dir=models/exp_1

and I test with:
python demo.py --generator_file=models/exp_1/GAN_DIS_3.pth --depth=7 --latent_size=512

but some error:

Missing key(s) in state_dict: "module.layers.0.conv_1.weight", "module.layers.0.conv_1.bias", "module.layers.0.conv_2.weight", "module.layers.0.conv_2.bias", "module.layers.1.conv_1.weight", "module.layers.1.conv_1.bias", "module.layers.1.conv_2.weight", "module.layers.1.conv_2.bias", "module.layers.2.conv_1.weight", "module.layers.2.conv_1.bias", "module.layers.2.conv_2.weight", "module.layers.2.conv_2.bias", "module.layers.3.conv_1.weight", "module.layers.3.conv_1.bias", "module.layers.3.conv_2.weight", "module.layers.3.conv_2.bias", "module.layers.4.conv_1.weight", "module.layers.4.conv_1.bias", "module.layers.4.conv_2.weight", "module.layers.4.conv_2.bias", "module.layers.5.conv_1.weight", "module.layers.5.conv_1.bias", "module.layers.5.conv_2.weight", "module.layers.5.conv_2.bias", "module.layers.6.conv_1.weight", "module.layers.6.conv_1.bias", "module.layers.6.conv_2.weight", "module.layers.6.conv_2.bias", "module.rgb_converters.0.weight", "module.rgb_converters.0.bias", "module.rgb_converters.1.weight", "module.rgb_converters.1.bias", "module.rgb_converters.2.weight", "module.rgb_converters.2.bias", "module.rgb_converters.3.weight", "module.rgb_converters.3.bias", "module.rgb_converters.4.weight", "module.rgb_converters.4.bias", "module.rgb_converters.5.weight", "module.rgb_converters.5.bias", "module.rgb_converters.6.weight", "module.rgb_converters.6.bias".
Unexpected key(s) in state_dict: "rgb_to_features.0.weight", "rgb_to_features.0.bias", "rgb_to_features.1.weight", "rgb_to_features.1.bias", "rgb_to_features.2.weight", "rgb_to_features.2.bias", "rgb_to_features.3.weight", "rgb_to_features.3.bias", "rgb_to_features.4.weight", "rgb_to_features.4.bias", "rgb_to_features.5.weight", "rgb_to_features.5.bias", "final_converter.weight", "final_converter.bias", "layers.0.conv_1.weight", "layers.0.conv_1.bias", "layers.0.conv_2.weight", "layers.0.conv_2.bias", "layers.1.conv_1.weight", "layers.1.conv_1.bias", "layers.1.conv_2.weight", "layers.1.conv_2.bias", "layers.2.conv_1.weight", "layers.2.conv_1.bias", "layers.2.conv_2.weight", "layers.2.conv_2.bias", "layers.3.conv_1.weight", "layers.3.conv_1.bias", "layers.3.conv_2.weight", "layers.3.conv_2.bias", "layers.4.conv_1.weight", "layers.4.conv_1.bias", "layers.4.conv_2.weight", "layers.4.conv_2.bias", "layers.5.conv_1.weight", "layers.5.conv_1.bias", "layers.5.conv_2.weight", "layers.5.conv_2.bias", "final_block.conv_1.weight", "final_block.conv_1.bias", "final_block.conv_2.weight", "final_block.conv_2.bias", "final_block.conv_3.weight", "final_block.conv_3.bias".

how to solve it?