lidq92 / compressai-vision Goto Github PK

CompressAI-Vision helps you to develop, test and evaluate compression models with standardized tests in the context of compression methods optimized for machine tasks algorithms such as Neural-Network (NN)-based detectors.

It currently focuses on two types of pipeline:

• Video compression for remote analysis, which corresponds to the MPEG "Video Coding for Machines" (VCM) activity.

• Split inference, which includes an evaluation framework for compressing intermediate features produced in the context of split models. The software supports all thepipelines considered in the related MPEG activity: "Feature Compression for Machines" (FCM).

The figure below shows the split inference pipeline:

• Detectron2 is used for object detection (Faster-RCNN) and instance segmentation (Mask-RCNN)

• JDE is used for Object Tracking

A complete documentation is provided here, including installation, CLI usage, as well as tutorials.

To get started locally and install the development version of CompressAI-Vision, first create a virtual environment with python==3.8:

python3.8 -m venv venv
source ./venv/bin/activate
pip install -U pip

The CompressAI library providing learned compresion modules is available as a submodule. It can be initilized by running:

git submodule update --init --recursive

To install the models relevant for the FCM (feature compression):

First, if you want to manually export CUDA related paths, please source (e.g. for CUDA 11.8):

bash scripts/env_cuda.sh 11.8

Then, run:, please run:

bash scripts/install.sh

For more otions, check:

bash scritps/install.sh --help

For split-inference pipelines, after installing, please use the following entry point:

compressai-vision-eval --help

For example for full pipelines without any compression, run

compressai-vision-eval --config-name=eval_example

Please check other configuration examples provided in ./cfgs as well as examplary scripts in ./scripts

Test data related to the MPEG FCM activity can be found in ./data/mpeg-fcm/

After your dev, you can run (and adapt) test scripts from the scripts/tests directory. Please check scripts/tests/Readme.md for more details

Code is formatted using black and isort. To format code, type:

make code-format

Static checks with those same code formatters can be run manually with:

make static-analysis

Tutorials are produced from notebooks that are in docs/source/tutorials. If you update the notebooks, first you need to run compile.bash therein.

To produce the html documentation, from docs/, run:

make html

To check the pages locally, open docs/_build/html/index.html

CompressAI-Vision is licensed under the BSD 3-Clause Clear License

Fabien Racapé, Hyomin Choi, Eimran Eimon, Sampsa Riikonen, Jacky Yat-Hong Lam

• HEVC HM reference software
• VVC VTM reference software
• Detectron2
• JDE