Code Monkey home page Code Monkey logo

gem's Introduction

GEM: Glass-Segmentor

Jing Hao, Moyun Liu, Jinrong Yang, Kuo Feng Hung.

This repository is the official implementation of the GEM: Boost Simple Network for Glass Surface Segmentation via Vision Foundation Models. Our code is based on MaskDINO.

Todo list
  • Release code and GEM-Tiny checkpoint

  • Release S-GSD dataset

  • Release multiple pre-trained models

  • Release GEM-Base model


Features

  • A simple but accurate segmentation framework, named GEM, for glass surface segmentation.
  • Automatically construct a large-scale synthesized glass surface dataset with precise mask annotation, termed S-GSD.
  • Surpasses the previous state-of-the-art methods by a large margin (IoU +2.1%).

Installation

See Mask DINO.

Getting Started

See Inference Demo with Pre-trained Model

See Results.

See Preparing Datasets for GEM.

See Getting Started.


Results

Model Pre-trained dataset IoU F_β MAE BER FPS download
GEM-Tiny | config S-GSD-1x 0.755 0.852 0.038 8.39 16.09 BaiduDisk
GEM-Tiny | config S-GSD-5x 0.757 0.855 0.035 8.54 16.09 BaiduDisk
GEM-Tiny | config S-GSD-10x 0.764 0.866 0.034 8.62 16.09 BaiduDisk
GEM-Tiny | config S-GSD-20x 0.770 0.865 0.032 8.21 16.09 BaiduDisk
GEM-Base | config S-GSD-1x 0.766 0.873 0.031 9.44 11.55 BaiduDisk
GEM-Base | config S-GSD-5x 0.769 0.858 0.032 8.16 11.55 BaiduDisk
GEM-Base | config S-GSD-10x 0.774 0.868 0.032 8.56 11.55 BaiduDisk
GEM-Base | config S-GSD-20x 0.774 0.865 0.029 8.35 11.55 BaiduDisk

[07/15/2024] All pre-trained models can also download from Huggingface

[07/18/2024] S-GSD-1x dataset can also download from Huggingface

The download link of S-GSD-5x is blow here, it be divided into three parts:

Part1: https://pan.baidu.com/s/1CL0x8s1LXdIIoOjcr5wirw?pwd=2ff4

Part2: https://pan.baidu.com/s/18uCDKFmzy7vSmWm1IF4JFQ?pwd=9jqb

Part3: https://pan.baidu.com/s/1Z8pePl9Ps3QtrZB7WQ7aoQ?pwd=8fku

The S-GSD-10x and S-GSD-20x are not released because of the large disk storage, if you want to get these two large-scale datasets, feel free to contact me via [email protected].



Getting Started

In the above tables, the corresponding model checkpoints can pre-trained models can be downloaded from the link.

If your dataset files are not under this repo, you need to add export DETECTRON2_DATASETS=/path/to/your/data or use Symbolic Link ln -s to link the dataset into this repo before the following command first.

Evalaluate our pretrained models

  • You can download our pretrained models and evaluate them with the following commands.
    python train_net.py --eval-only --num-gpus 8 --config-file config_path MODEL.WEIGHTS /path/to/checkpoint_file

Train GEM to reproduce results

  • Use the above command without eval-only will train the model. For MobileSAM/SAM backbones, you need to download its weight from () and specify the path of the pretrained backbones with MODEL.WEIGHTS /path/to/pretrained_checkpoint
    python train_net.py --num-gpus 8 --config-file config_path MODEL.WEIGHTS /path/to/checkpoint_file

You can also refer to Getting Started with Detectron2 for full usage.

Citing GEM

If you find our work helpful for your research, please consider citing the following BibTeX entry.

@article{hao2024gem,
  title={GEM: Boost Simple Network for Glass Surface Segmentation via Segment Anything Model and Data Synthesis},
  author={Hao, Jing and Liu, Moyun and Hung, Kuo Feng},
  journal={arXiv preprint arXiv:2401.15282},
  year={2024}
}

If you find the code useful, please also consider the following BibTeX entry.

@misc{li2022mask,
      title={Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentation}, 
      author={Feng Li and Hao Zhang and Huaizhe xu and Shilong Liu and Lei Zhang and Lionel M. Ni and Heung-Yeung Shum},
      year={2022},
      eprint={2206.02777},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Acknowledgement

Many thanks to these excellent opensource projects

gem's People

Contributors

isbrycee avatar lmomoy avatar

Stargazers

Akif avatar MillX avatar  avatar Ellery Queen avatar  avatar  avatar Amir avatar  avatar

Watchers

Lucian avatar Kostas Georgiou avatar  avatar

Forkers

lmomoy

gem's Issues

Baidu link not usable in Canada

Hi there,

This is some great work with impressive results. I want to check the results on some custom data.

I'm trying to download the checkpoints but baidu links are unusable if you dont have a baidu account. I don't have a chinese number so i cannot create a baidu account.

I'm not sure if you have access to google drive or one drive, but if you do can you upload the checkpoints there?

Any help will be very much appreciated.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.