A keypoint detection scheme based on OpenPose.

FashionAI是阿里天池组织的一个服饰关键点检测比赛，包括blouse/dress/outwear/skirt/trousers五种类型服饰。具体情况可参见FashionAI.

第一赛季rank 37,第二赛季rank 42，比赛已结束。

[2018-07-24]该比赛已开放成为长期比赛：https://tianchi.aliyun.com/getStart/introduction.htm?spm=5176.11165320.5678.1.68a84226w3Hys5&raceId=231670。

整个过程中在focal loss/dsconv/upsampling/stage 方面实验较多，遗憾的是refine net方面没有来得及完成实验，目前还在尝试。

最终结果为单模型4.51%，如果做refine net或者ensemble model或许会有所提高。

This is an experimental project for FashionAI competetion which is developed based on OpenPose(Cao et al) and tf-openpose.

And for each clothe type (dress / skirt / outwear / trousers / blouse) we trained one model separately taking vgg19 network as initial weights for the first several layers. Basically each model need 2 days to train on a 8-gpu machine.

Some results:

dress	outwear	blouse	skirt	trousers

My result is based on vgg19, then I did following experiments about backbone contrast.

1. Multi-scale feature paramid is not an effective scheme which gives basically no difference on result;

2. As many people mentioned OHEM may be useful, weighted-heatmaps/focal loss/hard retrain are no-use in my experiments;

3. Coarse detection + align detection actually will ignore many hard keypoints, it needs further research;

4. In my 6 CPM stage-outputs, the last two stage do the best, but merging their results will give unstable results;

5. Upsampling and heatmap-resize-in-postprocess really boost the result.

6. Do not try too much before the contrast between backbone has been done!!!![IMPORTANT :(]

7. Batchsize is not so important.

8. ......

tensorflow(1.4.1+)

opencv

scikit-image

numpy

python-prctl(strongly recommended)

...

I wrote a shell bash to facilitate the training procedure. To train a model, taking blouse as an example just set the --tag in ./begin_to_train.sh to 'blouse'.

And then run ./begin_to_train.sh.

This command will start the training progress and the trained checkpoint models are saved under ./models/trained. Next you need to frozen the model to use it do the prediction.

Just run ./begin_to_frozen.sh.

This command will generate frozen graph under ./tmp .

Copy it to the corresponding directory at ./models/trained/clothe_type/graph because the test shell will load model from this directory.

>  cd code

>  ./begin_to_train.sh

>  ./begin_to_frozen.sh

>  cp ./tmp/frozen_graph.pb ./models/trained/blouse/graph

That's all !

Watch this: begine_to_test.sh

Here is another shell for test.

First please make sure thers exits the corresponding frozen_graph.pb under "./models/trained/%s" % clothe_type.

Modify the --tag in begin_to_test.sh and run it.

This command will generate 'submit_%s.csv' % clothe_type under ./submit directory.

After test all 5 clothe_types then python merge.py will merge all results into one .csv file which is in the submit csv format.

>  cd code

>  python run.py --modelpath="./tmp/frozen_graph.pb" --imagepath="../data/test_b" --csv="../data/test_b/test.csv" --tag="skirt" --test="submit" --inputsize='368' --resolution="368x368" --scales="[1.0, (0.5,0.25,1.5), (0.5,0.75,1.5), (0.25,0.5,1.5), (0.75,0.5,1.5), (0.5,0.5,1.5)]"

>  python run.py --modelpath="./tmp/frozen_graph.pb" --imagepath="../data/test_b" --csv="../data/test_b/test.csv" --tag="trousers" --test="submit" --inputsize='368' --resolution="368x368" --scales="[1.0, (0.5,0.25,1.5), (0.5,0.75,1.5), (0.25,0.5,1.5), (0.75,0.5,1.5), (0.5,0.5,1.5)]"

>  python run.py --modelpath="./tmp/frozen_graph.pb" --imagepath="../data/test_b" --csv="../data/test_b/test.csv" --tag="outwear" --test="submit" --inputsize='368' --resolution="368x368" --scales="[1.0, (0.5,0.25,1.5), (0.5,0.75,1.5), (0.25,0.5,1.5), (0.75,0.5,1.5), (0.5,0.5,1.5)]"

>  python run.py --modelpath="./tmp/frozen_graph.pb" --imagepath="../data/test_b" --csv="../data/test_b/test.csv" --tag="blouse" --test="submit" --inputsize='368' --resolution="368x368" --scales="[1.0, (0.5,0.25,1.5), (0.5,0.75,1.5), (0.25,0.5,1.5), (0.75,0.5,1.5), (0.5,0.5,1.5)]"

>  python run.py --modelpath="./tmp/frozen_graph.pb" --imagepath="../data/test_b" --csv="../data/test_b/test.csv" --tag="dress" --test="submit" --inputsize='368' --resolution="368x368" --scales="[1.0, (0.5,0.25,1.5), (0.5,0.75,1.5), (0.25,0.5,1.5), (0.75,0.5,1.5), (0.5,0.5,1.5)]"

>  cd ../submit

>  python merge.py 

That's all!

If you have any questions about this project. then send an email to [email protected] to let me know.