pkhungurn / talking-head-anime-demo Goto Github PK

Exception in Tkinter callback
Traceback (most recent call last):
File "D:\PYTHON\anaconda\lib\tkinter_init_.py", line 1705, in call
return self.func(*args)
File "D:/OneDrive/Python/aa/talking-head-anime-demo/app/puppeteer.py", line 94, in load_image
self.load_image_from_file(file_name)
File "D:/OneDrive/Python/aa/talking-head-anime-demo/app/puppeteer.py", line 106, in load_image_from_file
self.source_image = extract_pytorch_image_from_filelike(file_name).to(self.torch_device).unsqueeze(dim=0)
File "D:\PYTHON\anaconda\lib\site-packages\torch\cuda_init_.py", line 196, in _lazy_init
check_driver()
File "D:\PYTHON\anaconda\lib\site-packages\torch\cuda_init.py", line 101, in _check_driver
http://www.nvidia.com/Download/index.aspx""")
AssertionError:
Found no NVIDIA driver on your system. Please check that you
have an NVIDIA GPU and installed a driver from
http://www.nvidia.com/Download/index.aspx
[ WARN:0] terminating async callback

I am using conda to create an environements on Windows 7. The example waifu works nice, but while trying to use some cropped/moded PNG files from photoshop, the result become weired.

Is that because something unable to determine from eyes in the file polluted my self-prepared PNG picture?

Thanks for the efforts and the time you spent on the project.

python app\manual_poser.py
Traceback (most recent call last):
  File "app\manual_poser.py", line 13, in <module>
    from poser.morph_rotate_combine_poser import MorphRotateCombinePoser256Param6
ModuleNotFoundError: No module named 'poser.morph_rotate_combine_poser'; 'poser' is not a package

I am using Python 3.6.8 , and I have installed all the dependencies including CUDA and cudnn

Requirement already satisfied: torch in c:\users\gaope\appdata\local\programs\python\python36\lib\site-packages (1.4.0)
Requirement already satisfied: dlib in c:\users\gaope\appdata\local\programs\python\python36\lib\site-packages (19.19.0)
Requirement already satisfied: opencv-python in c:\users\gaope\appdata\local\programs\python\python36\lib\site-packages (4.2.0.32)
Requirement already satisfied: numpy in c:\users\gaope\appdata\local\programs\python\python36\lib\site-packages (1.18.1)
Requirement already satisfied: pillow in c:\users\gaope\appdata\local\programs\python\python36\lib\site-packages (7.0.0)
 

I've run the program on anaconda .It can recognize my face，however,once I uploaded the photo,the python app/puppeteer.py
window stucked. My GPU is RTX3070. Can anybody offer me help？

Hi, Thanks for your work and I have bad infer results using your Colab. Could you please tell me why this happen?

I am sure that I use the correct format image. (256 * 256, .png that have alpha channel)

Hi, I find that the project page can't be opened. https://pkhungurn.github.io/talking-head-anime/
the problem also exist in talking-head-anime-2 and talking-head-anime-3
Could you fix it, thanks a lot

This happened when I try to run "puppeteer.py", my camera worked perfectly and I could see my face was highlighted. However, when I import a image, it stopped working and showed this:
Exception in Tkinter callback
Traceback (most recent call last):
File "D:\Python\Python36\lib\tkinter_init_.py", line 1705, in call
return self.func(*args)
File "D:\Python\Python36\lib\tkinter_init_.py", line 749, in callit
func(*args)
File "app/puppeteer.py", line 142, in update_image
left_eye_normalized_ratio = compute_left_eye_normalized_ratio(face_landmarks, eye_min_ratio, eye_max_ratio)
File "F:\Source\Python\talking-head-anime-demo\puppet\util.py", line 15, in compute_left_eye_normalized_ratio
min_ratio, max_ratio)
File "F:\Source\Python\talking-head-anime-demo\puppet\util.py", line 26, in compute_eye_normalized_ratio
left_eye_horiz_diff = face_landmarks.part(eye_horiz_points[0]) - face_landmarks.part(eye_horiz_points[1])
TypeError: unsupported operand type(s) for -: 'point' and 'point'
[ WARN:0] global C:\projects\opencv-python\opencv\modules\videoio\src\cap_msmf.cpp (674) SourceReaderCB::~SourceReaderCB terminating async callback

Can we run the 'talking-head-anime-demo' on Linux?

I want to use this program but I don't have NVIDIA's GPU.
So I tried to processing by CPU. But I can't because of my low skills.
Is this program work without using CUDA ? (even any processing speed)

1. Do you have any plans on releasing the data?
2. how about the code you've used to create data?
3. And the 3D models you've downloaded?

Thanks

As in title, code you've uploaded didn't work on 10.2 or higher version of CUDA/Pytorch.
(This error may have occurred because of Pytorch.)
BTW, below is my environment

Ryzen 7 3800X
RTX 2080 Super
RAM 24GB

Installed python

Python 3.7.9
Package           Version
----------------- ------------
backcall          0.2.0
cmake             3.18.4.post1
colorama          0.4.4
decorator         4.4.2
dlib              19.21.1
ipython           7.20.0
ipython-genutils  0.2.0
jedi              0.18.0
numpy             1.20.0
opencv-python     4.5.1.48
parso             0.8.1
pickleshare       0.7.5
Pillow            8.1.0
pip               20.1.1
prompt-toolkit    3.0.14
Pygments          2.7.4
setuptools        47.1.0
torch             1.7.1+cu101
torchaudio        0.7.2
torchvision       0.8.2+cu101
traitlets         5.0.5
typing-extensions 3.7.4.3
wcwidth           0.2.5
wheel             0.36.2

Last but not least, thanks for your great job!
I'm very excited on how this project goes on.

hi!When i load the app demo
python app/puppeteer.py
it doesn't work like the other one.It appears maɔv and i can not see my face tracing~
sry my english is bad so i could not offer more description to u . QAQ

i find that the mouth ratio is changing in code but in posed img, the mouth seems never opened, even if i specify the mouth ratio in code.

First, this is an amazing paper and project. Thank you for sharing all of this!

Will the models created as part of the v2 project be added here eventually?

Hi, Great job! Thanks for your sharing.

The method works well in given examples, however, I get a worse result when using the picture generated by waifulabs.com.

The pic is from waifulabs.com, and I convert it to RGBA format with Photoshop. I alignment the face to 128x128 center box and resize the pic to 256x256

the results look like this:

I believe it should be something wrong at the alignment,
so could you please share the alignment code?

Here my code of alignment:

from PIL import Image
import numpy as np
import cv2

def alinment_image_lowhigh(image, high, low, save_path=None):
    '''
    :param image: image path
    :param high: the bigger hight cordinate
    :param low:  the smaller hight cordinate
    :param save_path:
    :return:
    '''
    image = np.array(Image.open(image))
    size = np.array(image.shape[:2])

    # adjust face scale to 128
    h = high - low
    scale = 128 / h
    high, low = high * scale, low * scale
    hc = (high + low) / 2
    image = np.array(Image.fromarray(image).resize((int(size[0] * scale), int(size[1] * scale))))

    # move face to center
    size = np.array(image.shape[:2])
    delta = size[0] / 2 - hc
    M = np.float32([[1, 0, 0], [0, 1, delta]])
    image = cv2.warpAffine(image, M, (size[1], size[0]), borderValue=(255, 255, 255))
    image = Image.fromarray(image)
    image = image.resize(256, 256)
    if save_path is not None: image.save(save_path)
    image.show()

Thanks, your reply will be appreciated.

It happened after run it about 30 seconds.The camera suddenly stuck.Moreover,importing a picture into it,the right side says "nothing yet"

PS G:\talking-head-anime-demo> python app/puppeteer.py
Exception in Tkinter callback
Traceback (most recent call last):
  File "C:\Python39\lib\tkinter\__init__.py", line 1885, in __call__
    return self.func(*args)
  File "C:\Python39\lib\tkinter\__init__.py", line 806, in callit
    func(*args)
  File "G:\talking-head-anime-demo\app\puppeteer.py", line 119, in update_image
    self.draw_face_box(rgb_frame, face_box_points)
  File "G:\talking-head-anime-demo\app\puppeteer.py", line 175, in draw_face_box
    cv2.line(frame, face_box_points[start], face_box_points[end], (255, 0, 0), thickness=2)
cv2.error: OpenCV(4.5.5) :-1: error: (-5:Bad argument) in function 'line'
> Overload resolution failed:
>  - Can't parse 'pt1'. Sequence item with index 0 has a wrong type
>  - Can't parse 'pt1'. Sequence item with index 0 has a wrong type

Hello, it's a very nice project. i'm very interesting about your training data, could you release it?

The torch.cuda.is_available() print out True, and the dlib also use gpu (dlib.DLIIB_USE_CUDA() is True)
However when the script runs it occupies 20% CPU and only 0.5% GPU

And the frame rate only 4 fps
I wonder if it is other step that slow the frames output rate?How can I accelerate it><

Hello, @pkhungurn. First of all, it's a great honor for seeing your excellent project. 😄
BTW, I have a question while seeing your code with your blog post

Here in the figure above, it looks that you adapt the module of Appreance flow (Zhou et al. ECCV 2016).

device = self.zhou_grid_change.weight.device
identity = torch.Tensor([[1, 0, 0], [0, 1, 0]]).to(device).unsqueeze(0).repeat(n, 1, 1)
base_grid = affine_grid(identity, [n, c, h, w])
grid = base_grid + grid_change
resampled = grid_sample(image, grid, mode='bilinear', padding_mode='border')

However in the code tha/two_algo_face_rotator.py, it seems that you use a PyTorch function affine_grid, which builds Spatial Transformer Networks (M Jaderberg, et al. NIPS 2015).

Is the function of affine_grid corresponding with the implementation of appearance flow ? If not, is there any block (or snippet) that implements appearance flow?

Again, thank you for sharing your great project! See ya.

When I an running this project with my GTX 960M, I found out that the GPU resource haven't been used up , but it is still very laggy. How can I run it with full speed?

Hello,I want to put the module on my website.But I found that need a mtn file seems like every action that the model makes.

So is there any ways to save these action？

First of all, Kudos to your amazing work.
For some of the anime character, there is hardly any lip movement when in webcam I am moving my lips. Specially if the character is male. I have generated some character myself. Attaching two examples of avatar image.I have maintained all the criteria you mentioned. Any suggestions would be very helpful.

unsupported operand type(s) for -: 'point' and 'point'

Hye ,@dragonmeteor @sheiun,
First of All thanks for this great work. Its results are really good. I just want to move the shoulder as well . Kindly guide me how i can achieve this thing.

Have opened the program, loaded the sample image, but the right screen keeps saying "Nothing yet!", was there anything i missed?

can i use an alternative customed image as the source image?

Hi, thank you for amazing job!
I've read that there is no plan to release trining data or data generation code.
So, I'm thinking that I follow your instruction or use your tool to create my own dataset.
Is there any problem if I use your work as annotation tool ??

I'm a Japanese student. Sorry for the poor English.
When I tried this program, got error as this issue's title.
I use M1 MacBook Air and GPU is Metal Performance Shaders (MPS).
Can anybody offer me help？

Thanks for this outstanding work!

I tried this demo to generate some gif for my favorite waifu images, most of them works well.
But when character doesn't face to front, the model works not well on eye area generation.

Is there any open source plan for model training? That will be helpful to train model that meets my requirement.

It seems that some new features are created such as iris rotation, is there any plan to share that?

Linux can't be tested without camera and display interface！！could you support MP4 test ?

Traceback (most recent call last):
File "app/manual_poser.py", line 135, in
root = Tk()
File "/home/dd/anaconda3/envs/head_anime/lib/python3.8/tkinter/init.py", line 2261, in init
self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use)
_tkinter.TclError: no display name and no $DISPLAY environment variable
(head_anime) dd@ubuntu150:/150/talking-head-anime-demo$ ls app/
init.py manual_poser.py puppeteer.py
(head_anime) dd@ubuntu150:/150/talking-head-anime-demo$ python app/puppeteer.py
[ WARN:0] global /io/opencv/modules/videoio/src/cap_v4l.cpp (887) open VIDEOIO(V4L2:/dev/video0): can't open camera by index
Traceback (most recent call last):
File "app/puppeteer.py", line 201, in
master = Tk()
File "/home/dd/anaconda3/envs/head_anime/lib/python3.8/tkinter/init.py", line 2261, in init
self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use)
_tkinter.TclError: no display name and no $DISPLAY environment variable

When I have all the dependencies installed, and ran the python app, it occurred the following errors:
Exception in Tkinter callback
Traceback (most recent call last):
File "D:\Users\Administrator\AppData\Local\Programs\Python\Python38\lib\tkinter_init_.py", line 1883, in call
return self.func(*args)
File "D:\Users\Administrator\AppData\Local\Programs\Python\Python38\lib\tkinter_init_.py", line 804, in callit
func(*args)
File "app/manual_poser.py", line 112, in update_image
posed_image = self.poser.pose(self.source_image, self.current_pose).detach().cpu()
File "D:\Downloads\talking-head-anime-demo-master\poser\morph_rotate_combine_poser.py", line 82, in pose
combined_image = combine_module(rotated_images[0], rotated_images[1], rotate_params)
File "D:\Users\Administrator\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\nn\modules\module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "D:\Downloads\talking-head-anime-demo-master\tha\combiner.py", line 42, in forward
y = self.main_body(x)
File "D:\Users\Administrator\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\nn\modules\module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "D:\Downloads\talking-head-anime-demo-master\nn\u_net_module.py", line 61, in forward
y = torch.cat([x, downward_outputs[i]], dim=1)
RuntimeError: error in LoadLibraryA
And the program cannot present any live2d pictures.
Can U take a look at it?

How does it work with existing videos ?

Thank you so much

Hi，I have a question about character images. Is there any ready-made character images, or I can only get character images manually ?

I have already resize these images(some of them came from MakeGirls.Moe @makegirlsmoe) to 256x256, their heads in 128x128, and background is transparent.
Is there any thing I should do?

Hi, I find your demo pictures have alpha channel. However, the pictures I download directly from waifu website have only RGB channel. How do you make the alpha channel of the waifu's pic?

Link : "Youtube" TimePoint: 15:33

I want to use the model in other software like facerig. Is it possible to export the model of Anime Character?

Always stay healthy.

I think this project is very positive, But the graphics card I have now is RX580.
Is there any way the AMD product line can do in this project?
If not, what is the best product to use this project?

Hello. I tried to down batch size because my NVIDIA Driver(GTX750 Ti) is not good to run this demo.
but, I can not find where is batch size value. Could you let me know the place that batch size value is located?

---[Error infomation]---
RuntimeError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 2.00 GiB total capacity; 1.15 GiB already allocated; 0 bytes free; 1.20 GiB reserved in total by PyTorch)