jantic / deoldify Goto Github PK

Hi all!

I'm getting this error:
IndexError: index 0 is out of bounds for axis 0 with size 0

For some reason the data couldn't be loaded properly in _load_model_data()
Maybe someone had similar issue?

Thanks,
Or

Thanks for making this! I had no problems getting it working on Windows 10, and /colou?ri[zs]ing/ some images on my 980Ti.

The image I want to use it for (two boys on a row-boat) doesn't come out great, so I'd love to be able to fine-tune on a bunch of similar images so I can get a good result. You've kindly uploaded the generator weights, but I'd like the discriminator weights too (DCCritic / GANTrainer.netD).

No way I can train this all the way myself on my 980Ti 😅 but maybe just a few more examples is possible...

using render factor =42 in colab but looks like this two result is not same

this is my config
Collecting environment information...
PyTorch version: 0.3.1.post2
Is debug build: No
CUDA used to build PyTorch: 9.0
OS: Microsoft Windows 10 Pro
GCC version: Could not collect
CMake version: Could not collect
Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: 9.0.176
GPU models and configuration: GPU 0: GeForce GTX 1070
Nvidia driver version: 417.35
cuDNN version: Probably one of the following:
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0\bin\cudnn64_7.dll
Versions of relevant libraries:
[pip] Could not collect
[conda] blas 1.0 mkl
[conda] mkl 2019.1 144
[conda] mkl_fft 1.0.6 py36h6288b17_0
[conda] mkl_random 1.0.2 py36h343c172_0
[conda] pytorch 0.3.1 py36_cuda90_cudnn7he774522_2 [cuda90] peterjc123
[conda] torchtext 0.3.1
[conda] torchvision 0.2.1

this is the output of collect_env.py
this is the error i get if i try to use colorizeVisualization in jupyter notebook
RuntimeError: cuda runtime error (2) : out of memory at ..\aten\src\THC\THCGeneral.cpp:663

so does this mean my 1070 is running out of memory? any suggestions in this matter please?
what gpu did you people use to run this ?
thanks

flake8 testing of https://github.com/jantic/DeOldify on Python 3.7.1

$ flake8 . --count --select=E901,E999,F821,F822,F823 --show-source --statistics

./fastai/core.py:37:32: E999 SyntaxError: invalid syntax
    if cuda: a = to_gpu(a, async=True)
                               ^
./fastai/models/inceptionresnetv2.py:316:20: F821 undefined name 'pretrained_settings'
        settings = pretrained_settings['inceptionresnetv2'][pretrained]
                   ^
./fastai/models/inceptionresnetv2.py:321:17: F821 undefined name 'InceptionResNetV2'
        model = InceptionResNetV2(num_classes=1001)
                ^
./fastai/models/inceptionresnetv2.py:337:17: F821 undefined name 'InceptionResNetV2'
        model = InceptionResNetV2(num_classes=num_classes)
                ^
./fastai/models/cifar10/main_dxy.py:179:32: E999 SyntaxError: invalid syntax
      target = target.cuda(async=True)
                               ^
./fastai/models/cifar10/utils.py:114:32: F821 undefined name 'random'
  return string + '-{}'.format(random.randint(1, 10000))
                               ^
./fastai/models/cifar10/utils_kuangliu.py:17:18: F821 undefined name 'torch'
    dataloader = torch.utils.data.DataLoader(dataset, batch_size=1, shuffle=True, num_workers=2)
                 ^
./fastai/models/cifar10/utils_kuangliu.py:18:12: F821 undefined name 'torch'
    mean = torch.zeros(3)
           ^
./fastai/models/cifar10/utils_kuangliu.py:19:11: F821 undefined name 'torch'
    std = torch.zeros(3)
          ^
2     E999 SyntaxError: invalid syntax
7     F821 undefined name 'random'
9

As I check codes, I find that following codes in module.py file(https://github.com/jantic/DeOldify/blob/master/fasterai/modules.py)

def forward(self, up_p:int, x_p:int):
up_p = self.tr_conv(up_p)
x_p = self.x_conv(x_p)
x = torch.cat([up_p,x_p], dim=1)
x = self.relu(x)
return self.out(x)
return out

Why there is two return value in this function?

I think you may like to use this for squarifying: https://github.com/Vooban/Smoothly-Blend-Image-Patches

I'll wait a little to make sure I didn't screw up usage of git-lfs and make sure that everybody is set....

Title says it all

Images consume a lot of space in this repo.

Please consider an option to move it outside e.g. to git-lfs.

Though in order to reduce current repo size history cleanup will also be required.

Had another minor issue when running from command line

Traceback (most recent call last):
  File "test.py", line 28, in <module>
    vis.plot_transformed_image("/tmp/Alfred-James-Boyce.jpg")
  File "/mnt/disks/400gb_ssd/DeOldify/fasterai/visualize.py", line 37, in plot_transformed_image
    self._save_result_image(path, result)
  File "/mnt/disks/400gb_ssd/DeOldify/fasterai/visualize.py", line 46, in _save_result_image
    result_path = self.results_dir/source_path.name
AttributeError: 'str' object has no attribute 'name'

Looked like the output path wasn't being handled correctly?

In fasterai/visualize.py plot_transformed_image() accepts path as a string and converts it to a Path object but by the time we get to calling self._save_result_image() it's reverted back to a string. Maybe the casting on line 31 is by reference and not by value - I don't know enough about internals to know if that's the case.

Anyway, I just re-cast in the call to _save_result_image() and we're all hunky dory again
self._save_result_image(Path(path), result)

EDIT:
Ubuntu Xenial 16.,06
Python 3.6.6 :: Anaconda, Inc.

Can you please add a small howto in README.md on how to use DeOldify on own set of images? It's not that obvious :)

Such an interesting, outstanding project. Kudos. Also appreciate the obvious care you and the community are putting into documentation of this, which is no easy task.

Two main comments:

First, for Windows users trying it out locally, I noticed an out-of-memory error on the "Color Visualization" notebook, where memory doesn't seem to be automatically released. I was able to resolve it with explicit memory cleanup before each visualization. Please see this thread for a workaround: #49

Second -- a couple general observations:

1. This is amazing work. Really fun to see photos come to life.

2. In my test trials, medium head shots (i.e., waist up) seem to do much better overall than, say, full shots set on a larger landscape. And my own anecdotal tests are right in line with your observation about a blue clothing bias -- it seems to want to bias toward blue for many articles of clothing.

This got me wondering: given the relative higher accuracy of medium head shots (if my anecdotal observation is actually really true) I was wondering if one optimization for the generator during training might be to "heavily weight flesh tone of a generated medium shot" -- i.e., try a face-detect first, get the largest face in the picture, try to build a "medium shot" of that by cropping then bias heavily toward those weights? I don't know at all if or how this would map to your existing code, just thought I'd throw it out there if it sparks any ideas.

I think the fastai has some compatible issue as well as pytorch, in terms of complicity, less wrapper would be better

Firstly, thank you for your tremendous work on colorization. It gives me a lot of new ideas.

Here is my problem, I have used your training code for my own training process. I have noticed that it generate scaled images like 128*128 or 256*256 before training process and it takes a lot of time. But when I train on my own dataset, the BlackAndWhiteTransform() transformation is not working. The input image is become colorful. So the whole training is become meaningless.

And I did add BlackAndWhiteTransform() to all my progressive GANs.

x_tfms = [BlackAndWhiteTransform()]
scheds.extend(GANTrainSchedule.generate_schedules(szs=[64, 64],
                                                  bss=[64, 64],
                                                  path=IMAGENET,
                                                  x_tfms=x_tfms,
                                                  extra_aug_tfms=extra_aug_tfms,
                                                  keep_pcts=[1.0,1.0],
                                                  save_base_name=proj_id,
                                                  c_lrs=c_lrs,
                                                  g_lrs=g_lrs,
                                                  lrs_unfreeze_factor=lrs_unfreeze_factor,
                                                  gen_freeze_tos=gen_freeze_tos))

I don't know what the problem is, please help me. Thank you very much.

There's gotta be a way... Suggestions welcome.

No one seems to be colorized correctly...

I was shocked when I was able to colorize my own photos today.

I followed the directions in the readme and I'll be damned, it WORKED.

It's my first time using a Jupyter Notebook so there was a lot of room for error.

I wasn't able to figure out where exactly to put the .h5 file however. I didn't see an empty set of folders to drop it into. Perhaps more instructions on how to create that folder structure would be good. I ended up manually creating the folders user c:\user\mike\data....

Thanks for creating and posting a working application. SO many times I've met with dead-ends in these machine learning projects (edges for cats comes to mind).

I try to run the code on linux machine with video - card.
So I started a file with only 4 lines of code

import os
import multiprocessing
from torch import autograd
from fastai.transforms import TfmType

And got an error Segmentation fault
And it happens when I try to import smth from fast.ai.

Does it mean that I have shortage of RAM (which is 4Gb at the moment ) ?
It's quite strange because I just imported the library and that's it

This was a quick hack to get denormalization in place for images when doing visualization. I'm going to get rid of this now because it turns out it's a pain in the ass for others (like doing a Colab notebook).

This will only be tackled once image generation is significantly improved.

May improve generalization across different aspect ratios/sizes.

I'm on a machine without a GUI so I'd prefer to just run everything from the command line like a boss and I can do some general tinkering on a range of images and weights.

I've basically just copied the relevant bits from the notebooks

import multiprocessing
import os
from torch import autograd
from fastai.transforms import TfmType
from fasterai.transforms import *
from fastai.conv_learner import *
from fasterai.images import *
from fasterai.dataset import *
from fasterai.visualize import *
from fasterai.callbacks import *
from fasterai.loss import *
from fasterai.modules import *
from fasterai.training import *
from fasterai.generators import *
from fasterai.filters import *
from fastai.torch_imports import *
torch.backends.cudnn.benchmark=True
torch.cuda.set_device(0)

colorizer_path = 'colorize_gen_192.h5'
render_factor=42
weights_path = "/mnt/disks/400gb_ssd/DeOldify/colorize_gen_192.h5"
results_dir="/mnt/disks/400gb_ssd/DeOldify/result_images"
filters = [Colorizer34(gpu=0, weights_path=weights_path)]
vis = ModelImageVisualizer(filters, render_factor=render_factor, results_dir=results_dir)
vis.plot_transformed_image("test_images/1852GatekeepersWindsor.jpg")

It's unfortunately resulting in the below error. I don't doubt for a second the issue is with me

Traceback (most recent call last):
  File "/mnt/disks/400gb_ssd/DeOldify/fastai/dataset.py", line 234, in open_image
    im = cv2.imread(str(fn), flags).astype(np.float32)/255
AttributeError: 'NoneType' object has no attribute 'astype'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "test.py", line 28, in <module>
    vis.plot_transformed_image("test_images/1852GatekeepersWindsor.jpg")
  File "/mnt/disks/400gb_ssd/DeOldify/fasterai/visualize.py", line 31, in plot_transformed_image
    result = self._get_transformed_image_ndarray(path, render_factor)
  File "/mnt/disks/400gb_ssd/DeOldify/fasterai/visualize.py", line 49, in _get_transformed_image_ndarray
    orig_image = open_image(str(path))
  File "/mnt/disks/400gb_ssd/DeOldify/fastai/dataset.py", line 238, in open_image
    raise OSError('Error handling image at: {}'.format(fn)) from e
OSError: Error handling image at: test_images/1852GatekeepersWindsor.jpg

Which in itself is odd because you'd expect the filecheck at the top of open_image() to have dealt with that. I'm guessing this is not a DeOldify issue per se as it's all inside fastai but wonder if you've any thoughts.

NOTE: You have to update your code to use the weights. The model was altered a bit in terms of dimensions.

CPU rendering is desirable to get around the limitations of memory in GPUs when wanting to render at high quality. Additionally, of course- not everybody has a GPU with a decent amount of memory. This issue will track creating both an install and instructions/best practices for CPU rendering.

Title

These colorizations look amazing! Not sure if you noticed, but the hand in this image on the main page stuck out like a … sore thumb?

Keep up the great work!

Standard Python class/function documentation, basically.

Memory usage is way too high- barely practical. I suspect simple model tweaks will make a big difference.

Would be useful to manually correct some of the colorization glitches, or make the output more historically accurate.

Ideally, a secondary image with user-defined colour hints could be provided to both the generator and critic. Generating training hints that look like user brush-strokes might be a bit difficult, although I'm not sure how much that would matter.

Another, easier option would be to specify only a palette:
http://openaccess.thecvf.com/content_cvpr_2017_workshops/w12/papers/Cho_PaletteNet_Image_Recolorization_CVPR_2017_paper.pdf

Awesome project by the way, can't wait to see how it develops!

Several images in the Colab notebook are now returning 404 errors (doh!). 404 errors are not nice.

Source: MayeulC on HackerNews, thread:

https://news.ycombinator.com/item?id=18363870#18369410

"Now, there seems to be a distinct loss of details in the restored images. The network being resolution-limited, is the black-and-white image displayed at full resolution besides the restored one?

What I would like to see is the output of the network to be treated as chrominance only.

Take the YUV transform of both the input and output images, scale back the UV matrix of the restored one to match the input, and replace the original channels. I'd be really curious to look at the output (and would do it myself if I was not on a smartphone)!"

Hello, I am sorry to bother you again. I have tried your case several times, but it still has problems:
My computer environment is win10, cpu. I have seen the cpu implementation you mentioned in the issue. It mentions the modification of the command. But if I follow 'conda env create -f environment.yml' at the beginning, the following error will occur:

So I tried to download the cpu version of the module such as pytorch. Then tried to use the weight you have given me and run "ColorizeVisualization" directly. But when I load the module, the following problem occurs.

Then I found the file "\fastai\torch_imports.py" along the path and saw this syntax error.

I have problems loading the module. So python can't continue working down.
I don't understand the meaning of the two folders "fastai" and "fasterai". Is this your own module? Why not choose to use conda install/pip install to download it? Could you give me some guiding advice? Thank you very much!!

I try my best to reproduce your case. Because I am shocked by your case and want to promote it to more friends around me. But I have not been successful.
I sincerely hope to get your help.

Can a Docker file and a Docker image with a pretrained model be provided for use with nvidia-docker? The idea is to make it possible to colorize easily.

I was just playing around with some of the example photos given but I noticed as I convert an image, the memory used by CUDA gets used and is never deallocated.

Basically if I convert a couple images my memory gets used up and I get the CUDA out of memory error.

I noticed if I restart the kernal or kill the python process I get the memory back and can continue trying to convert a different image.

Shouldn't the memory be deallocated after the image is converted and saved?

(I was running the project on windows 10 with an NVIDIA GTX970 graphics card. I used the weights linked to in your article:
https://blog.floydhub.com/colorizing-and-restoring-old-images-with-deep-learning/ )

I'm such an amateur for leaving this out lol

Will probably wait until Pytorch 1.0 is in stable release.

Hopefully knocking down the time to train and improving quality of results in the process. I'm pretty sure the current training regime in the notebooks is not ideal.

This project is so awesome! I have tested several pictures, and the results are very good.
But There are some badcase.
First, as shown in the picture.Lei Feng -- a very famous figure in China，I use this picture to test your wonderful project, but the result turns out a little wrong. As i known ,the clothes Lei Feng dress should be green but in the result turns out blue.

Second, as shown in the below picture,red scarf is a symbol of Chinese youth, but the little girl wears a blue scarf. Another point is that their hands are a bit scary（Blue,Dark）.😂

So are there any optimizition advices for these problems?
Looking forword to your reply~
Best wishes.
Thank you

They are all free for Open Source projects like this one.

• https://github.com/marketplace/category/continuous-integration

Good meaning it works reliably.

from the last commit
colored image files are not showing /saving in /content/drive/My Drive/deOldifyImages/results
vis.plot_transformed_image(img_path) works correctly

As far as I got it right, this class is related to CUDA.
But how can I run it on CPU ?
I disabled this line of code.

#torch.cuda.set_device(0)

Title says it all

Hello, I want to ask how to download colorize34_gen_192.h5 and bwdefade3_gen_160.h5? Where could I find the file"data/imagenet/ILSVRC/Data/CLS-LOC/train". Extremely grateful

ModuleNotFoundError Traceback (most recent call last)
in
2 import os
3 from torch import autograd
----> 4 from fastai.transforms import TfmType
5 from fasterai.transforms import *
6 from fastai.conv_learner import *

ModuleNotFoundError: No module named 'fastai.transforms'
`

Hi!
Your work is great meaningful.I wonder whether it is unsupervised and am interested in the reasons why you don't choose wgan or wgan-gp finally?

thanks.