Code Monkey home page Code Monkey logo

pytorch-mask-x-rcnn's Introduction

PyTorch-mask-x-rcnn

PyTorch implementation of the Mask-X-RCNN network proposed in the 'Learning to Segment Everything' paper by Facebook AI Research.

Introduction

The paper is about Instance Segmentation given a huge dataset with only bounding box and a small dataset with both bbox and segmentation ground truths. It follows the semi-supervised learning paradigm. The base architecture is same as that of Mask-RCNN.

Model Architecture

  • The pipeline is as shown in the Figure. For little more explanation checkout this blog post (last section).
  • Backproping both losses will induce a discrepancy in the weights of w_seg as for common classes between COCO and VG there are two losses (bbox and mask) while for rest classes its only one (bbox). There's a fix for this
    • Fix: When back-propping the mask, compute the gradient of predicted mask weights (w_seg) wrt weight transfer function parameters $\theta$ but not bounding box weight $w_{det}^c$ .
    • eqn where tau is the transfer function.

Mask-X-RCNN

Implementation Details

  • The model is based on the Mask-RCNN implementation from here. Thanks to him and original Keras version on which its based on! Integrate it with the pipeline from the repo to train the network!
  • Modules added
    • transfer_function in fpn_classifier_graph
    • cls , box , cls+box choices for the detection weights in fpn_classifier_graph
    • class-agnostic (baseline) and transfer (above diagram) modes for the Mask branch as explained in the paper.
    • Optional MLP fusion (class agnostic MLP) as explained in Section 3.4 of the paper.
    • stop_grad for backpropping mask loss (keeping w_det out of gradient calculation)

Results

  • I'm planning to run it on VOC+COCO soon. Will update once it's done.
  • Note - The official Detectron (Caffe2) models and code are up here

References

Hu, Ronghang, Piotr Dollár, Kaiming He, Trevor Darrell and Ross B. Girshick. “Learning to Segment Every Thing.” *CoRR*abs/1711.10370 (2017): n. pag.

pytorch-mask-x-rcnn's People

Contributors

skrish13 avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.