youyuge34 / pi-rec Goto Github PK

View Code? Open in Web Editor NEW

2.0K 34.0 240.0 46.64 MB

:fire: PI-REC: Progressive Image Reconstruction Network With Edge and Color Domain. :fire: 图像翻译，条件GAN，AI绘画

Home Page: https://arxiv.org/abs/1903.10146

License: Other

Python 100.00%

gan image-to-image-translation cgan auto-painting edge-to-image-translation image image-reconstruction ai-painting

pi-rec's Introduction

PI-REC

Progressive Image Reconstruction Network With Edge and Color Domain

Paper on arXiv | Paper Read Online | BibTex

When I was a schoolchild,

I dreamed about becoming a painter.

With PI-REC, we realize our dream.

For you, for everyone.

English | 中文版

🏳️‍🌈 Demo show time 🏳️‍🌈

Draft2Painting

Tool operation

Introduction

We propose a universal image reconstruction method to represent detailed images purely from binary sparse edge and flat color domain. Here is the open source code and the drawing tool. Learn more about related works here --> image-to-image papers collection.

*The codes of training for release are no completed yet, also waiting for release license of lab.

Find more details in our paper: Paper on arXiv

Quick Overview of Paper

What can we do?

Figure (a): Image reconstruction from extreme sparse inputs.
Figure (b): Hand drawn draft translation.
Figure (c): User-defined edge-to-image (E2I) translation.

Model Architecture

We strongly recommend you to understand our model architecture before running our drawing tool. Refer to the paper for more details.

Prerequisites

Python 3+
PyTorch 1.0 (0.4 is not supported)
NVIDIA GPU + CUDA cuDNN

Installation

Clone this repo
Install PyTorch and dependencies from http://pytorch.org
Install python requirements:

pip install -r requirements.txt

Usage

We provide two ways in this project:

Basic command line mode for batch test
Drawing tool GUI mode for man-machine interactive creation

Firstly, follow steps below with patience to prepare pre-trained models:

Download the pre-trained models you want here: Google Drive | Baidu (Extraction Code: 9qn1)
Unzip the .7z and put it under your dir ./models/.
So make sure your path now is: ./models/celeba/<xxxxx.pth>
Complete the above Prerequisites and Installation

Files are ready now! Read the User Manual for firing operations.

中文版介绍 🀄

Demo演示

自己看上面的咯~

简介

我们提出了一种基于GAN的渐进式训练方法 PI-REC，它能从超稀疏二值边缘以及色块中还原重建真实图像。我们的论文重心是在超稀疏信息输入的还原重建上，并非自动绘画。总之，PI-REC论文/项目属于图像重建，图像翻译，条件图像生成，AI自动绘画的前沿交叉领域的最新产出，而非简单的以图搜图等等。阅读论文中的 Related Work部分或 image-to-image论文整合项目以了解更多。
注意：这里包含了论文代码以及交互式绘画工具。此论文demo仅推荐给不会绘画的人试玩（比如我），或给予相关领域科研人员参考。远远未达到民用或辅助专业人士绘图的程度。

*由于训练过程过于复杂，用于训练的发布版代码还未完成

在我们的论文中你可以获得更多信息: Paper on arXiv (推荐) | 机器之心-中文新闻稿 | b站中文视频教程(有福利?)

论文概览

PI-REC能做啥？

Figure (a): 超稀疏输入信息重建原图。
Figure (b): 手绘草图转换。
Figure (c): 用户自定义的 edge-to-image (E2I) 转换.

模型结构

我们强烈建议你先仔细阅读论文熟悉我们的模型结构，这会对运行使用大有裨益。

基础环境

Python 3
PyTorch 1.0 (0.4 会报错)
NVIDIA GPU + CUDA cuDNN （当前版本已可选cpu，请修改config.yml中的DEVICE）

第三方库安装

Clone this repo
安装PyTorch和torchvision --> http://pytorch.org
安装 python requirements:

pip install -r requirements.txt

运行使用

我们提供以下两种方式运行：

基础命令行模式 用来批处理测试整个文件夹的图片
绘画GUI工具模式 用来实现交互式创作

首先，请耐心地按照以下步骤做准备：

在这里下载你想要的预训练模型文件：Google Drive | Baidu (提取码: 9qn1)

更新：2021.4 baidu网盘和谐了我的权重文件，已无法分享，请自行前往Google drive~

解压，放到目录./models下
现在你的目录应该像这样： ./models/celeba/<xxxxx.pth>
完成上面的基础环境和第三方库安装

啦啦啦啦，到这里准备工作就完成啦，接下来需要阅读用户手册来运行程序~

Acknowledgment

Code structure is modified from Anime-InPainting, which is based on Edge-Connect.

BibTex

@article{you2019pirec,
  title={PI-REC: Progressive Image Reconstruction Network With Edge and Color Domain},
  author={You, Sheng and You, Ning and Pan, Minxue},
  journal={arXiv preprint arXiv:1903.10146},
  year={2019}
}

pi-rec's People

Contributors

Stargazers

Watchers

Forkers

jovidalao mx597014232 jacobw92 flandrecirno awesome-archive naturtle kafurtan mrchujiu jiwenfei minamion yakamozduan ryoumiko easychen fengshow12345 yiwenjie007 ebgaaron keyman9848 passerdhr naumrchen hhy5277 dahaoniuniu xu-shuhao socjs wweihua gaeolus asterism123 walogia merrygreek lyuro jdc08161063 time-to-learn-ml dlwbm123 houyujiang wolfcon pldare rockystevejobs w7yuu countzero1 avatarworld gtdong-ustc luowenvirus pinkuburu chengmuni66 jdk6979 oiccc bapuqln georgezc2003 ywang021 magaminanaya takotab 2411148291 leo-xxx loveexword rjt1990 yc7521 xxzbg ushio-aria soinlovelin shameby tsingchan dufufeisdu kiradiso ryusun1891 fear-wall cy-xu waaaaghyesyes li-ch-en zilipeng fcvbg savourylie nmbahhh bupt2019lx hgwu4869 kizunatl watson1101 meoral wangzhiwei-ai cv-ip zxcasdqwezjy122 publiclassperson fakegit brandonweb santhosh-ks kkykkyy one-sixth johnwang1998 jasonnor lucaspedroni william-zhan 1165048017 jaejun-yoo finesure2017 pangfd faruba hawkit youyeyuki smksyj jerrynpc cn-potatochips frankiegu

pi-rec's Issues

cv.imread不支持中文路径

改成英文路径以后能正确读取中文文件夹名读取出来是None但是不报错之后resize时报错

结束了都结束了

《生成式人工智能服务管理办法（征求意见稿）》:http://www.cac.gov.cn/2023-04/11/c_1682854275475410.htm
按照这个规定，你这个仓库是违法的哦

scipy依赖版本

1.0.1版本在pip安装的时候有报错
疑似和numpy存在版本冲突，使用1.1.0后正常

GUI mode run error

I ran the commad python tool_draw.py -p models/getchu-anime -r on my mac, and got this:

(PI-REC) ➜  PI-REC git:(master) ✗ python tool_draw.py -p models/getchu-anime -r

===============================================================================
Interactive Image Drawing tool of PI-REC.

Paper: PI-REC: Progressive Image Reconstruction Network With Edge and Color Domain 2019.3

Command:
    python tool_draw.py --path <your weights directory path> -r

README FIRST:
    Four windows will show up, one for color domain, one for edge, one for output and a pane.
    [Important] Switch your typewriting into ENG first.


Key | Description

Mouse `Left` | Draw
Mouse `Right` | Erase
Key `h` | Show the help message box.
Key `[` | To make the brush thickness smaller
Key `]` | To make the brush thickness larger
Key `g` | To reconstruct the image from edge and color domain
Key `u` | To refine the output only when `-r` is added in command line
Key `Alt` | To absorb mouse pointer color in color domain (the mouse must be moving at the same time)
Key `x` | To save the binary edge
Key `c` | To save the color domain
Key `s` | To save the output
Key `q` | To quit

                #####################################
                #####  Author: youyuge34@github  ####
                #####    License-CC BY·NC 4.0    ####
                #####################################
===============================================================================

/Users/kylo/Documents/python/PI-REC/src/config.py:8: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  self._dict = yaml.load(self._yaml)
load_config_demo----->
DEVICE is: cpu
Loading G_Model generator weights file: models/getchu-anime/G_Model_gen_2228000.pth...
Model configurations:
---------------------------------
MODE: 2             # 1: train, 2: test, 3: refine, (5:drawing tool, 6:refine in drawing tool)
SEED: 10            # random seed
GPU: [0]            # list of gpu ids
DEBUG: 1            # turns on debugging mode
VERBOSE: 0          # turns on verbose mode in the output console

TEST_FLIST: ./datasets/getchu/val       # testing dir path used in test.py
REFINE_FLIST: ./datasets/getchu/refine  # refinement dir path used in refine.py

BATCH_SIZE: 4                 # input batch size for training
INPUT_SIZE: 128               # image will be cropped into [INPUT_SIZE, INPUT_SIZE]
SIGMA: 4                    # standard deviation of the Gaussian filter used in Canny edge detector (0: random, -1: no edge)
KM: 3                         # number of colors in color domain, used in K-Means algorithm

---------------------------------


############
###Drawing model loaded.###
###########

load_config_demo----->
DEVICE is: cpu
Loading R_Model generator weights file: models/getchu-anime/R_Model_gen_2340000.pth...

############
###Refinement model loaded.###
###########

2019-05-17 17:57:09.568 python[59540:13952343] -[NSApplication _setup:]: unrecognized selector sent to instance 0x7f8e2aebb270
2019-05-17 17:57:09.574 python[59540:13952343] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[NSApplication _setup:]: unrecognized selector sent to instance 0x7f8e2aebb270'
*** First throw call stack:
(
	0   CoreFoundation                      0x00007fff4ef23cf9 __exceptionPreprocess + 256
	1   libobjc.A.dylib                     0x00007fff79ab3a17 objc_exception_throw + 48
	2   CoreFoundation                      0x00007fff4ef9db06 -[NSObject(NSObject) __retain_OA] + 0
	3   CoreFoundation                      0x00007fff4eec60ef ___forwarding___ + 1485
	4   CoreFoundation                      0x00007fff4eec5a98 _CF_forwarding_prep_0 + 120
	5   libtk8.6.dylib                      0x00000001043a331d TkpInit + 413
	6   libtk8.6.dylib                      0x00000001042fb17e Initialize + 2622
	7   _tkinter.cpython-37m-darwin.so      0x0000000104123a0f _tkinter_create + 1183
	8   python                              0x0000000103885116 _PyMethodDef_RawFastCallKeywords + 230
	9   python                              0x00000001039c1e42 call_function + 306
	10  python                              0x00000001039bfaec _PyEval_EvalFrameDefault + 46092
	11  python                              0x00000001039b349e _PyEval_EvalCodeWithName + 414
	12  python                              0x0000000103883de7 _PyFunction_FastCallDict + 231
	13  python                              0x0000000103906381 slot_tp_init + 193
	14  python                              0x0000000103910361 type_call + 241
	15  python                              0x0000000103884ae3 _PyObject_FastCallKeywords + 179
	16  python                              0x00000001039c1ed5 call_function + 453
	17  python                              0x00000001039bfaec _PyEval_EvalFrameDefault + 46092
	18  python                              0x00000001038848d5 function_code_fastcall + 117
	19  python                              0x0000000103906381 slot_tp_init + 193
	20  python                              0x0000000103910361 type_call + 241
	21  python                              0x0000000103884ae3 _PyObject_FastCallKeywords + 179
	22  python                              0x00000001039c1ed5 call_function + 453
	23  python                              0x00000001039bfb27 _PyEval_EvalFrameDefault + 46151
	24  python                              0x00000001039b349e _PyEval_EvalCodeWithName + 414
	25  python                              0x0000000103883de7 _PyFunction_FastCallDict + 231
	26  python                              0x0000000103906381 slot_tp_init + 193
	27  python                              0x0000000103910361 type_call + 241
	28  python                              0x0000000103884ae3 _PyObject_FastCallKeywords + 179
	29  python                              0x00000001039c1ed5 call_function + 453
	30  python                              0x00000001039bfbe0 _PyEval_EvalFrameDefault + 46336
	31  python                              0x00000001039b349e _PyEval_EvalCodeWithName + 414
	32  python                              0x0000000103884fe3 _PyFunction_FastCallKeywords + 195
	33  python                              0x00000001039c1dc7 call_function + 183
	34  python                              0x00000001039bfbe0 _PyEval_EvalFrameDefault + 46336
	35  python                              0x00000001039b349e _PyEval_EvalCodeWithName + 414
	36  python                              0x0000000103a169a0 PyRun_FileExFlags + 256
	37  python                              0x0000000103a15e17 PyRun_SimpleFileExFlags + 391
	38  python                              0x0000000103a43d3f pymain_main + 9663
	39  python                              0x000000010385766d main + 125
	40  libdyld.dylib                       0x00007fff7b2e13d5 start + 1
	41  ???                                 0x0000000000000005 0x0 + 5
)
libc++abi.dylib: terminating with uncaught exception of type NSException
[1]    59540 abort      python tool_draw.py -p models/getchu-anime -r

but in three phase:
X_fake3 = G1_3(E, X_fake2)
and computing loss...... How I got X_fake2?

I guess
method 1:
X_fake_2 = G1_2(E, Cgt)
X_fake3 = G1_3(E, X_fake2)
and computing loss......

method 2:
torch.no_grad():
X_fake_2 = G1_2(E, Cgt)
X_fake3 = G1_3(E, X_fake2)
and computing loss......