Code Monkey home page Code Monkey logo

manga-colorizer's Introduction

Manga-Colorizer

I like to read manga in my spare time but sometimes the experience is a bit bland since there is no color, hence I decided to put deep learning to good use and train Pix2Pix GAN to colorize manga.

Dataset

For Training data I scraped the images from manga website. I chose one piece since it has huge collection of colorized images. I won't give the scraping scripts nor the dataset. If you want it you can request it to me via mail.

Training

GAN's are absolutely perfect for such tasks and few years ago a nice paper called Image-to-Image Translation with Conditional Adversarial Networks which did Image2Image translation. Our task is similar we want to convert B/W to RGB, hence this seems like a nice approach to use for out problem.

I won't go into much detail how the model works and all. I will reference the blogs which I used during the developement.

The only thing I observed during my experiments was it took time to converge when generator was trained from scratch. So here is what is I did.

I used segmentation-models library to create a Unet architecture with efficientnet-b1 as the backbone. Then I subsampled some of the images from dataset and pretrained the Unet using just L1-Loss.

Finally I used the pretrained Unet along with discriminator and trained both of them adversarial fashion. This gave me good results but I am limited on GPU compute and did not train the model for too long due to these constraints.

For handling most of the boiler plate code, I used TorchFlare because of its easy to use API and customization to callabacks.

Thats it, the only time I spend was training the models everything else was handled by TorchFlare.

Streamlit APP and ONNX conversion

I first converted the pytorch model to onnx format so that we can use it with any framework. I have also created a small streamlit based application which let's you colorize the images.

I have provided the scripts for onnx model conversion and the streamlit app.

Results

Since it was only trained on one-piece manga images it may or may not perform good on other kinds of manga. But for now let's see some one piece images it created. Also the images are of just 256x256 resolution due to limitations on my hardware.

Orginal Colored Version Generated Colored Version

Future Work

  • I plan on developing this in my free time and I am going to work more on dataset curation and model experimentation with new models.
  • If you would like to contribute please open up issue with the idea and we can discuss more.

manga-colorizer's People

Contributors

atharva-phatak avatar imgbotapp avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.