xuezhemax / wolf Goto Github PK
View Code? Open in Web Editor NEWInvertible Generative Flows
License: Apache License 2.0
Invertible Generative Flows
License: Apache License 2.0
Hi @XuezheMax ,
Thanks for your nice works,
for the "MaCow" paper, would you please share checkpoints for "CelebHQ dataset"?
to have "Celeb-HQ" images, I have used "CelebAMask-HQ" dataset which has 2993 validation and 2824 test samples. is that correct?
Hi,
I am trying to run the cifar10 example in the README file. The command line arguments there specify 15000 as the number of epochs. How important is it to train the model for that many epochs? In other words, what is the minimum number of epochs to train for and still get reasonable results? Based on the speed I am seeing so far, it would take my system (with a single GPU) at least 7 weeks to finish 15000 epochs.
Thanks!
Thanks for great repository. I got an error while testing your code. Why it happens?
The versions of pytorch and cuda are 1.9.1 and 11.1, respectively.
Rank -1: random seed=65537
Rank -1: Namespace(amsgrad=False, batch_size=512, batch_steps=2, beta1=0.9, beta2=0.999, category=None, checkpoint_name='test_cifar10/checkpoint', config='configs/cifar10/macow/macow-base-var.json', cuda=True, data_path='../../../data/cifar-10', dataset='cifar10', epochs=15000, eps=1e-08, eval_batch_size=1000, grad_clip=0.0, image_size=32, init_batch_size=2048, local_rank=0, log=<io.TextIOWrapper name='test_cifar10/log.txt' mode='w' encoding='UTF-8'>, log_interval=10, lr=0.001, lr_decay=0.999997, model_path='test_cifar10', n_bins=256.0, n_bits=8, nx=3072, rank=-1, recover=-1, result_path='test_cifar10/images', seed=65537, test_k=5, train_k=1, valid_epochs=10, warmup_steps=50, weight_decay=1e-06, workers=4, world_size=1)
Files already downloaded and verified
Data size: training: 50000, val: 10000
of Parameters: 51046555
Rank -1, init model: 2048 instances
Epoch: 1 (lr=0.000000, betas=(0.9, 0.999), eps=1.0e-08, amsgrad=False, lr decay=0.999997, clip=0.0, l2=1.0e-06, train_k=1)
Traceback (most recent call last):
File "train.py", line 506, in
main(args)
File "train.py", line 500, in main
train(args, train_loader, train_index, train_sampler, val_loader, val_data, val_index, wolf)
File "train.py", line 334, in train
loss_gen, loss_kl, loss_dequant = wolf.loss(data, y=y, n_bits=n_bits, nsamples=train_k)
File "/root/share/NF/wolf/wolf/wolf.py", line 242, in loss
return core(data, y=y, n_bits=n_bits, nsamples=nsamples)
File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1051, in call_impl
return forward_call(*input, **kwargs)
File "/root/share/NF/wolf/wolf/wolf.py", line 78, in forward
u, log_probs_dequant = self.dequantizer.dequantize(x, nsamples=nsamples)
File "/root/share/NF/wolf/wolf/modules/dequantization/dequantizer.py", line 93, in dequantize
u, logdet = self.flow.fwdpass(epsilon, h)
File "/root/share/NF/wolf/wolf/flows/flow.py", line 73, in fwdpass
return self.forward(x, *h, **kwargs)
File "/root/share/NF/wolf/wolf/modules/dequantization/dequantizer.py", line 134, in forward
out, logdet_accum = self.core.forward(input, h=h)
File "/root/share/NF/wolf/wolf/flows/multiscale_architecture.py", line 301, in forward
out, logdet = block.forward(out, h=h)
File "/root/share/NF/wolf/wolf/flows/multiscale_architecture.py", line 102, in forward
out, logdet = step.forward(out, h=h)
File "/root/share/NF/wolf/wolf/flows/macow.py", line 152, in forward
out, logdet = unit.forward(out, h=h)
File "/root/share/NF/wolf/wolf/flows/macow.py", line 42, in forward
out, logdet_accum = self.conv1.forward(input, h=h)
File "/root/share/NF/wolf/wolf/flows/couplings/coupling.py", line 485, in forward
params = self.transform.calc_params(self.calc_params(input, h=h))
File "/root/share/NF/wolf/wolf/flows/couplings/transform.py", line 58, in calc_params
scale = log_scale.mul(0.5).tanh().mul(self.alpha).add(1.0)
RuntimeError: Output 1 of SplitBackward is a view and is being modified inplace. This view is an output of a function that returns multiple views. Inplace operators on such views is forbidden. You should replace the inplace operation by an out-of-place one.
Hello!
I am getting an error TypeError: ActNorm1dFlow.forward: kwargs
is not present.
This is still occurs regardless of config file or the dataset being used.
Still happens after clean installs of the packages and also when trying the previous MaCow repository.
How can I resolve this?
Thank you for the help!!!
Terminal:
(wolf) (base) james@LXMint:/media/james/LinuxStorage/Bayes/wolf/experiments$ python -u train.py \
> --config configs/cifar10/macow/macow-base-var.json \
> --epochs 15000 --valid_epochs 10 \
> --batch_size 512 --batch_steps 2 --eval_batch_size 1000 --init_batch_size 2048 \
> --lr 0.001 --beta1 0.9 --beta2 0.999 --eps 1e-8 --warmup_steps 50 --weight_decay 1e-6 --grad_clip 0 \
> --image_size 32 --n_bits 8 \
> --data_path data/cifar10 --model_path models
Traceback (most recent call last):
File "train.py", line 23, in <module>
from wolf.data import load_datasets, get_batch, preprocess, postprocess
File "/media/james/LinuxStorage/Bayes/wolf/wolf/__init__.py", line 3, in <module>
from wolf.wolf import WolfModel
File "/media/james/LinuxStorage/Bayes/wolf/wolf/wolf.py", line 12, in <module>
from wolf.modules import DeQuantizer
File "/media/james/LinuxStorage/Bayes/wolf/wolf/modules/__init__.py", line 3, in <module>
from wolf.modules.dequantization import *
File "/media/james/LinuxStorage/Bayes/wolf/wolf/modules/dequantization/__init__.py", line 3, in <module>
from wolf.modules.dequantization.dequantizer import DeQuantizer, UniformDeQuantizer, FlowDeQuantizer
File "/media/james/LinuxStorage/Bayes/wolf/wolf/modules/dequantization/dequantizer.py", line 9, in <module>
from wolf.flows.flow import Flow
File "/media/james/LinuxStorage/Bayes/wolf/wolf/flows/__init__.py", line 4, in <module>
from wolf.flows.normalization import ActNorm1dFlow, ActNorm2dFlow
File "/media/james/LinuxStorage/Bayes/wolf/wolf/flows/normalization.py", line 13, in <module>
class ActNorm1dFlow(Flow):
File "/media/james/LinuxStorage/Bayes/wolf/wolf/flows/normalization.py", line 26, in ActNorm1dFlow
def forward(self, input: torch.Tensor, mask: Union[torch.Tensor, None] = None) -> Tuple[torch.Tensor, torch.Tensor]:
File "/media/james/LinuxStorage/Bayes/wolf/lib/python3.7/site-packages/overrides/overrides.py", line 88, in overrides
return _overrides(method, check_signature, check_at_runtime)
File "/media/james/LinuxStorage/Bayes/wolf/lib/python3.7/site-packages/overrides/overrides.py", line 114, in _overrides
_validate_method(method, super_class, check_signature)
File "/media/james/LinuxStorage/Bayes/wolf/lib/python3.7/site-packages/overrides/overrides.py", line 135, in _validate_method
ensure_signature_is_compatible(super_method, method, is_static)
File "/media/james/LinuxStorage/Bayes/wolf/lib/python3.7/site-packages/overrides/signature.py", line 95, in ensure_signature_is_compatible
super_sig, sub_sig, super_type_hints, sub_type_hints, is_static, method_name
File "/media/james/LinuxStorage/Bayes/wolf/lib/python3.7/site-packages/overrides/signature.py", line 136, in ensure_all_kwargs_defined_in_sub
raise TypeError(f"{method_name}: `{name}` is not present.")
TypeError: ActNorm1dFlow.forward: `kwargs` is not present.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.