This repository includes the evaluation scripts for both Semcity challenges:

• Multi-Class Semantic Segmentation
• Instance Segmentation of Buildings

If the directory with the predictions works on the corresponding script, then its zipped version will also work on the evaluation server. In order to try the scripts, we suggest using either the training and validation labels to evaluate the metrics, or use the test data submission directory against itself to check if the evaluation server will run.

To run the Semantic Segmentation evaluation (-v for verbose):

  $ ./evaluate_semantic_segmentation.py  -s /path/to/prediction/dir -l /path/to/label/dir -v

To run the Instance Segmentation evaluation (-v for verbose):

  $ ./evaluate_instance_segmentation.py  -s /path/to/prediction/dir -l /path/to/label/dir -v

In order to install the requirements needed for this script (which are listed in the included "requirements.txt" file):

  $ sudo pip3 install -r requirements.txt