How to use CIHP_PGN for parsing? about hr-viton HOT 17 CLOSED

Hello guys. I just created 'Preprocess.md', the description for our preprocess including human parsing.
Please check it :)

from hr-viton.

The issues in the CIHP_PGN repo discuss this. I found this one particularly useful. Engineering-Course/CIHP_PGN#38

from hr-viton.

It does give me output but some labels are missing compared to the png files found in hr vton training data. I ran cihp pgn on the hr vton images to compare if I got the correct labels

from hr-viton.

Not sure what you mean by labels here. The different segmentation classes? An example could be useful

from hr-viton.

yes, by labels i mean different segmentations classes. For example:

0-Background
1-Hat
2-Hair
3-Glove
4-Sunglasses
5-Upper-clothes
6-Dress
7-Coat
8-Socks
9-Pants
10-tosor-skin
11-Scarf
12-Skirt
13-Face
14-Left-arm
15-Right-arm
16-Left-leg
17-Right-leg
18-Left-shoe
19-Right-shoe

from hr-viton.

Ah yeah. In the cp_dataset files, they map the 20 classes to the 13 that they use:

        labels = {
            0: ['background', [0, 10]],
            1: ['hair', [1, 2]],
            2: ['face', [4, 13]],
            3: ['upper', [5, 6, 7]],
            4: ['bottom', [9, 12]],
            5: ['left_arm', [14]],
            6: ['right_arm', [15]],
            7: ['left_leg', [16]],
            8: ['right_leg', [17]],
            9: ['left_shoe', [18]],
            10: ['right_shoe', [19]],
            11: ['socks', [8]],
            12: ['noise', [3, 11]]
        }

from hr-viton.

yes, but when I run cihp_pgn on hr vton data, it fails to give segmentation labels for the lower part of body. I am using tensorflow version 2.8.0 with gtx 3090

from hr-viton.

Interesting. Maybe best to wait for the authors response then. I've run it on additional data and fed that into the model just fine (though training quality seems significantly worse when I add more data to the dataset... maybe this has something to do with it). Also, I think I had to use tensorflow 1.15 to be able to run the model - it wasn't written for tensorflow 2

from hr-viton.

hr vton model is just bad in generalization. I used the simple human parser as well, changed the labels and still results were very poor

from hr-viton.

@nihirv I am able to get the parsed image from CIHP_PGN, but the format is not ok, somehow. I am getting an error because of the image format, when I load the image given by the authors, it is loaded as mode= "P" in the PIL image, whereas my parse image is loaded as mode = "RGB" image, what is going wrong there?

I managed to get this image

I am getting this error

from hr-viton.

@RAJA-PARIKSHAT "P" means that it is a palletised image. You need to write some code which will take the RGB image created by CIHP_PGN and map it to the palette that is used by HR-VITON. You can open one of the parse images they provide in their dataset (in PIL "P" mode) to see what a palette image looks like - and you can look at the cp_dataset files to see the IDs they use for each of the classifications made by CIHP_PGN.

https://stackoverflow.com/questions/67967366/how-to-write-image-with-palette-information might help you get started

from hr-viton.

Hi guys, sorry for my late response. I couldn't avoid reading your thread. I have been working in this for a month already. My conclusions, don't try to test the model with data that the model was not trained with, specifically human models. Apparently, the model is pixel sensible just for the case of the 'humans dataset' just try changing one pixel from the original dataset for testing and you will see. Anyway, I have a code to parse the human figures with CIHP model. However, you need dedicated GPU, otherwise it will crash with high resolution pictures. Also, I tried downsizing the resolution to reduce the time spent per picture. For my 300 images dataset, it took like 6 hours. I also recommend downsizing because the model is more accurate when parsing the body parts. The resize the outcome to the original dimensions. I will anyway trained the model and see what happens with external data. I will publish my code to parse with CIHP later.

from hr-viton.

@RAJA-PARIKSHAT Also, to get the PIL format, you will need to do this:

from PIL import Image
# assuming you read the image with opencv
im = Image.fromarray(im).convert('L')

from hr-viton.

Hi guys, sorry for my late response. I couldn't avoid reading your thread. I have been working in this for a month already. My conclusions, don't try to test the model with data that the model was not trained with, specifically human models. Apparently, the model is pixel sensible just for the case of the 'humans dataset' just try changing one pixel from the original dataset for testing and you will see. Anyway, I have a code to parse the human figures with CIHP model. However, you need dedicated GPU, otherwise it will crash with high resolution pictures. Also, I tried downsizing the resolution to reduce the time spent per picture. For my 300 images dataset, it took like 6 hours. I also recommend downsizing because the model is more accurate when parsing the body parts. The resize the outcome to the original dimensions. I will anyway trained the model and see what happens with external data. I will publish my code to parse with CIHP later.

Hello! Thank you for the information you provided. Could you please tell me about your results? Did you train the model on your data, which results did you have if so?

from hr-viton.

@yarfomin Yes, at first I was able to train and render some results. However apparently I have some issues with the parse map because I'm not having the desired agnostic representation images with your code. I'm working in a Google Colab notebook developing the whole pre-process pipeline, you just need the images with the human models. However, Google updated Colab 6 days ago and now I can't import tensorflow 1.15. I'm trying to figure out another way now. But almost there.

from hr-viton.

@yarfomin Yes, at first I was able to train and render some results. However apparently I have some issues with the parse map because I'm not having the desired agnostic representation images with your code. I'm working in a Google Colab notebook developing the whole pre-process pipeline, you just need the images with the human models. However, Google updated Colab 6 days ago and now I can't import tensorflow 1.15. I'm trying to figure out another way now. But almost there.

@Atotem I had the same issue in the Colab, you can use sudo update-alternatives --config python3 and choose python 3.6

from hr-viton.

Hi guys, sorry for my late response. I couldn't avoid reading your thread. I have been working in this for a month already. My conclusions, don't try to test the model with data that the model was not trained with, specifically human models. Apparently, the model is pixel sensible just for the case of the 'humans dataset' just try changing one pixel from the original dataset for testing and you will see. Anyway, I have a code to parse the human figures with CIHP model. However, you need dedicated GPU, otherwise it will crash with high resolution pictures. Also, I tried downsizing the resolution to reduce the time spent per picture. For my 300 images dataset, it took like 6 hours. I also recommend downsizing because the model is more accurate when parsing the body parts. The resize the outcome to the original dimensions. I will anyway trained the model and see what happens with external data. I will publish my code to parse with CIHP later.

Dear @Atotem ,
Just wanting to know if you are planning to share your code?
I have my implemented version as well, we can reconcile with each other code for better outcome may be. Thanks

from hr-viton.