Code Monkey home page Code Monkey logo

openvinotoolkit / anomalib Goto Github PK

View Code? Open in Web Editor NEW
3.4K 38.0 604.0 55.25 MB

An anomaly detection library comprising state-of-the-art algorithms and features such as experiment management, hyper-parameter optimization, and edge inference.

Home Page: https://anomalib.readthedocs.io/en/latest/

License: Apache License 2.0

Python 100.00%
anomaly-localization anomaly-segmentation anomaly-detection unsupervised-learning neural-network-compression openvino

anomalib's Introduction

A library for benchmarking, developing and deploying deep learning anomaly detection algorithms


Key FeaturesDocsNotebooksLicense

python pytorch openvino

Pre-Merge Checks Documentation Status codecov Downloads Discord


👋 Introduction

Anomalib is a deep learning library that aims to collect state-of-the-art anomaly detection algorithms for benchmarking on both public and private datasets. Anomalib provides several ready-to-use implementations of anomaly detection algorithms described in the recent literature, as well as a set of tools that facilitate the development and implementation of custom models. The library has a strong focus on visual anomaly detection, where the goal of the algorithm is to detect and/or localize anomalies within images or videos in a dataset. Anomalib is constantly updated with new algorithms and training/inference extensions, so keep checking!

Key features

  • Simple and modular API and CLI for training, inference, benchmarking, and hyperparameter optimization.
  • The largest public collection of ready-to-use deep learning anomaly detection algorithms and benchmark datasets.
  • Lightning based model implementations to reduce boilerplate code and limit the implementation efforts to the bare essentials.
  • The majority of models can be exported to OpenVINO Intermediate Representation (IR) for accelerated inference on Intel hardware.
  • A set of inference tools for quick and easy deployment of the standard or custom anomaly detection models.

📦 Installation

Anomalib provides two ways to install the library. The first is through PyPI, and the second is through a local installation. PyPI installation is recommended if you want to use the library without making any changes to the source code. If you want to make changes to the library, then a local installation is recommended.

Install from PyPI Installing the library with pip is the easiest way to get started with anomalib.
pip install anomalib

This will install Anomalib CLI using the dependencies in the pyproject.toml file. Anomalib CLI is a command line interface for training, inference, benchmarking, and hyperparameter optimization. If you want to use the library as a Python package, you can install the library with the following command:

# Get help for the installation arguments
anomalib install -h

# Install the full package
anomalib install

# Install with verbose output
anomalib install -v

# Install the core package option only to train and evaluate models via Torch and Lightning
anomalib install --option core

# Install with OpenVINO option only. This is useful for edge deployment as the wheel size is smaller.
anomalib install --option openvino
Install from source To install from source, you need to clone the repository and install the library using pip via editable mode.
# Use of virtual environment is highly recommended
# Using conda
yes | conda create -n anomalib_env python=3.10
conda activate anomalib_env

# Or using your favorite virtual environment
# ...

# Clone the repository and install in editable mode
git clone https://github.com/openvinotoolkit/anomalib.git
cd anomalib
pip install -e .

This will install Anomalib CLI using the dependencies in the pyproject.toml file. Anomalib CLI is a command line interface for training, inference, benchmarking, and hyperparameter optimization. If you want to use the library as a Python package, you can install the library with the following command:

# Get help for the installation arguments
anomalib install -h

# Install the full package
anomalib install

# Install with verbose output
anomalib install -v

# Install the core package option only to train and evaluate models via Torch and Lightning
anomalib install --option core

# Install with OpenVINO option only. This is useful for edge deployment as the wheel size is smaller.
anomalib install --option openvino

🧠 Training

Anomalib supports both API and CLI-based training. The API is more flexible and allows for more customization, while the CLI training utilizes command line interfaces, and might be easier for those who would like to use anomalib off-the-shelf.

Training via API
# Import the required modules
from anomalib.data import MVTec
from anomalib.models import Patchcore
from anomalib.engine import Engine

# Initialize the datamodule, model and engine
datamodule = MVTec()
model = Patchcore()
engine = Engine()

# Train the model
engine.fit(datamodule=datamodule, model=model)
Training via CLI
# Get help about the training arguments, run:
anomalib train -h

# Train by using the default values.
anomalib train --model Patchcore --data anomalib.data.MVTec

# Train by overriding arguments.
anomalib train --model Patchcore --data anomalib.data.MVTec --data.category transistor

# Train by using a config file.
anomalib train --config <path/to/config>

🤖 Inference

Anomalib includes multiple inferencing scripts, including Torch, Lightning, Gradio, and OpenVINO inferencers to perform inference using the trained/exported model. Here we show an inference example using the Lightning inferencer. For other inferencers, please refer to the Inference Documentation.

Inference via API

The following example demonstrates how to perform Lightning inference by loading a model from a checkpoint file.

# Assuming the datamodule, model and engine is initialized from the previous step,
# a prediction via a checkpoint file can be performed as follows:
predictions = engine.predict(
    datamodule=datamodule,
    model=model,
    ckpt_path="path/to/checkpoint.ckpt",
)
Inference via CLI
# To get help about the arguments, run:
anomalib predict -h

# Predict by using the default values.
anomalib predict --model anomalib.models.Patchcore \
                 --data anomalib.data.MVTec \
                 --ckpt_path <path/to/model.ckpt>

# Predict by overriding arguments.
anomalib predict --model anomalib.models.Patchcore \
                 --data anomalib.data.MVTec \
                 --ckpt_path <path/to/model.ckpt>
                 --return_predictions

# Predict by using a config file.
anomalib predict --config <path/to/config> --return_predictions

⚙️ Hyperparameter Optimization

Anomalib supports hyperparameter optimization (HPO) using wandb and comet.ml. For more details refer the HPO Documentation

HPO via API
# To be enabled in v1.1
HPO via CLI

The following example demonstrates how to perform HPO for the Patchcore model.

anomalib hpo --backend WANDB  --sweep_config tools/hpo/configs/wandb.yaml

🧪 Experiment Management

Anomalib is integrated with various libraries for experiment tracking such as Comet, tensorboard, and wandb through pytorch lighting loggers. For more information, refer to the Logging Documentation

Experiment Management via API
# To be enabled in v1.1
Experiment Management via CLI

Below is an example of how to enable logging for hyper-parameters, metrics, model graphs, and predictions on images in the test data-set.

You first need to modify the config.yaml file to enable logging. The following example shows how to enable logging:

# Place the experiment management config here.
# Place the Experiment Management CLI command here.

📊 Benchmarking

Anomalib provides a benchmarking tool to evaluate the performance of the anomaly detection models on a given dataset. The benchmarking tool can be used to evaluate the performance of the models on a given dataset, or to compare the performance of multiple models on a given dataset.

Each model in anomalib is benchmarked on a set of datasets, and the results are available in src/anomalib/models/<type>/<model_name>/README.md. For example, the MVTec AD results for the Patchcore model are available in the corresponding README.md file.

Benchmarking via API
# To be enabled in v1.1
Benchmarking via CLI

To run the benchmarking tool, run the following command:

anomalib benchmark --config tools/benchmarking/benchmark_params.yaml

✍️ Reference

If you use this library and love it, use this to cite it 🤗

@inproceedings{akcay2022anomalib,
  title={Anomalib: A deep learning library for anomaly detection},
  author={Akcay, Samet and Ameln, Dick and Vaidya, Ashwin and Lakshmanan, Barath and Ahuja, Nilesh and Genc, Utku},
  booktitle={2022 IEEE International Conference on Image Processing (ICIP)},
  pages={1706--1710},
  year={2022},
  organization={IEEE}
}

👥 Contributing

For those who would like to contribute to the library, see CONTRIBUTING.md for details.

Thank you to all of the people who have already made a contribution - we appreciate your support!

anomalib's People

Contributors

abc-125 avatar adrianboguszewski avatar alexanderdokuchaev avatar alexriedel1 avatar ashishbdatta avatar ashwinvaidya17 avatar blakshma avatar blaz-r avatar davnn avatar dependabot[bot] avatar djdameln avatar harimkang avatar jpcbertoldo avatar lukasbommes avatar nahuja-intel avatar orippler avatar paularamo avatar phcarval avatar ravindu987 avatar sahusiddharth avatar samet-akcay avatar sherpan avatar stillgreyfox avatar triet1102 avatar turbojonte avatar vdlmv avatar wenjingkangintel avatar willyfh avatar youngquan avatar yunchu avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

anomalib's Issues

Refactor Anomalib CLI

Current anomalib CLI implements before_instantiate_classes method. However, the implementation is not clean and needs refactoring.

Tasks

  • Use console scripts to replace python tools/train.py

Create CONTRIBUTING guidelines.

the repo is missing the contributing guidelines. It should show how contribution to the repo could be made.

It should also note that feature extraction-based methods now need to avoid performing validation before collecting the features from the entire training set.

Always reorder auc inputs after upgrading to pytorch 1.9.x

In the AUROC metric computation, we should ideally always reorder the inputs. Currently the lack of support for stable sorting in pytorch 1.8.1 prevents us from doing so (more detailed explanation here). Stable sorting was added in pytorch 1.9.0, so once we upgrade our requirements to that version, we should be able to always reorder the inputs using the stable=True option of torch.sort().

On Test Model load callback does not load recently created model

Model is loading default model from config file

  • While training model multiple times, new model file version gets created however the old model gets loaded for testing and results are evaluated with the old model. Any change in data or code does not get reflected in the results.

To Reproduce
Steps to reproduce the behavior:

  1. Train a model with subset of images
  2. Retrain with complete dataset
  3. Evaluate performance

Upload the library to PYPI

This is to upload anomalib to pypi so that the library could be installed via pip.

We would have two options.

  1. Basic install that doesn't have openvino. pip install anomalib
  2. Full option with openvino. This would be something like pip install anomalib["full"] or something similar.

Whether to install openvino within the basic version is still a question, and needs further discussion.

Diverged metrics in PatchCore: dropping from 0.99 to 0.44 and 0.03 is rather critical?

diverged metrics: dropping from 0.99 to 0.44 and 0.03 is rather critical?
log images: nice :)
Also padim dropped in performance, but not as crazy.
Here's a patchcore result of "good" parts:
008

this is padim:
DATALOADER:0 TEST RESULTS
{'image_AUROC': 0.7589669823646545,
'image_F1': 0.8787878751754761,
'pixel_AUROC': 0.9781586527824402,
'pixel_F1': 0.22379672527313232}
008padim

Originally posted by @sequoiagrove in #67 (comment)

patchcore torch.pairwise_distance OOM on larger datasets

I get CUDA memory blowing when doing larger datasets

return torch.pairwise_distance(x1, x2, p, eps, keepdim)
RuntimeError: CUDA out of memory. Tried to allocate 724.00 MiB (GPU 0; 11.00 GiB total capacity; 7.62 GiB already allocated; 190.31 MiB free; 8.77 GiB reserved in total by PyTorch)

Is there anything that could be done here? like moving this computation to CPU?

How to train in a custom dataset?

I am having a bit of trouble training on a custom dataset. I reproduced the MVTec structure but I still get some errors like:

IndexError: too many indices for tensor of dimension 0

e

Hey There!

Who can help me to modify : all_in_one_block.py script
to avoid this warning, to happen:

C:\Users\libro\AppData\Local\Programs\Python\Python38\lib\site-packages\FrEIA\modules\all_in_one_block.py:123: UserWarning: Soft permutation will take a very long time to initialize with 2048 feature channels. Consider using hard permutation instead.
warnings.warn(("Soft permutation will take a very long time to initialize "

I would like to switch to hard permutation mode, but i dont know how to do it.

Thank you so much.

ModuleNotFoundError: No module named 'anomalib.utils.callbacks'

Describe the bug
-anomalib package install error, seems the package released do not include all function of utils file

To Reproduce
Steps to reproduce the behavior:

  1. Go to https://pypi.org/project/anomalib/0.2.4/#files download the file anomalib-0.2.4-py3-none-any.whl
  2. pip install anomalib-0.2.4-py3-none-any.whl
  3. cd anomalib
  4. python tools/train.py
  5. See error ModuleNotFoundError: No module named 'anomalib.utils.callbacks'

Expected behavior
-seems the package released on pypi.org do not include all function of the git file anomalib,such as utils file.

Refactor benchmarking script

  • Add grid search instead of nested for loops
  • Run gpu and cpu training in parallel as they don't share resources.

Get rid of task_type parameter

The task_type parameter currently dictates which types of ground truth data is passed to the model. When the task_type parameter is set to segmentation, the ground truth pixel masks are passed in addition to the image-level labels. This might no longer be needed. Instead we can just pass whatever ground truth information is available, and let the PL module handle the evaluation based on what type of ground truth data it receives with the batches.

Inference Bug - Too many variables to unpack

Hello, thank you for the great repo!

Describe the bug
I cannot run inference.

To Reproduce
Steps to reproduce the behavior:
conda install openvino-ie4py-ubuntu20 -c intel
conda install pytorch==1.8.0 torchvision==0.9.0 torchaudio==0.8.0 cudatoolkit=11.1 -c pytorch -c conda-forge

python tools/inference.py \
    --model_config_path anomalib/models/padim/config.yaml \
    --weight_path results/padim/mvtec/leather/weights/model.ckpt \
    --image_path datasets/MVTec/leather/test/color/000.png
  • If applicable, add screenshots to help explain your problem.
Traceback (most recent call last):
  File "tools/inference.py", line 90, in <module>
    infer()
  File "tools/inference.py", line 78, in infer
    output = inference.predict(image=args.image_path, superimpose=True)
  File "anomalib/anomalib/core/model/inference.py", line 90, in predict
    anomaly_map, pred_score = self.post_process(predictions, meta_data=meta_data)
ValueError: too many values to unpack (expected 2)

Hardware and Software Configuration

  • OS: [Ubuntu, 20]
  • NVIDIA Driver Version [470.57.02]
  • CUDA Version [e.g. 11.4]
  • CUDNN Version [e.g. v11.4.120]
  • OpenVINO Version [Optional e.g. v2021.4.2]

Additional context
I also noticed that when I do inference from a ckpt file it still imports openvino- i expected this not to since it should run purely in pytorch.

Update Pre-commit Documentation

  • Pre-commit documentation refers to the old requirements_dev.txt format. This needs to be updated to reflect requirements/dev.txt
  • Also, command for running it on all files should be added for reference.

Refactor DFKDE NormalityModel to contain the feature extractor

Refactor according to comments on PR #122

This would probably work, but I feel like the model would still need another refactoring. Ideally DfkdeModel should be the normality model that contains feature_extractor, pca and kde modules. I guess this would be scope of another PR, but something to keep in mind.
Alternatively, there could be a better naming than NormalityModel if we want to keep NormalityModel. For instance, in DFM, there SingleClassGaussian. Again, in another PR

Reference to comment: #122 (comment)

Inference with OpenCV

Hello,
Thank you for providing such a fantastic repo!

I was curious if you could show a tutorial on how to use a custom trained model on a live stream webcam with OpenCV. Perhaps with torch and with openvino?

import cv2

cap = cv2.VideoCapture(0)

# Check if the webcam is opened correctly
if not cap.isOpened():
    raise IOError("Cannot open webcam")

while True:
    ret, frame = cap.read()
    
    # infer with custom model
    # show results on output_frame
    
    cv2.imshow('output', output_frame)

    k = cv2.waitKey(1)
    if k == 27:
        break

cap.release()
cv2.destroyAllWindows()

Thank you in advance 😃

Update README Badges

Due to refactor in workflows, the coverage badge in README is broken. Add badges for nightly and pre-merge

Code attribution

Hi,

Your implementation of PatchCore coreset selection:

def create_coreset(

is VERY similar to my implementation, which I have designed and coded from scratch:
https://github.com/rvorias/ind_knn_ad/blob/57de628b334f9dd62316ac7770db6e963b71ee8e/indad/utils.py#L36

Please elaborate on this.

More details:
this repo:

        select_idx = 0
        last_item = sample_set[select_idx : select_idx + 1]
        coreset_idx = [torch.tensor(select_idx).to(embedding.device)]  # pylint: disable=not-callable
        min_distances = torch.linalg.norm(sample_set - last_item, dim=1, keepdims=True)

        for _ in range(sample_count - 1):
            distances = torch.linalg.norm(sample_set - last_item, dim=1, keepdims=True)  # broadcast
            min_distances = torch.minimum(distances, min_distances)  # iterate
            select_idx = torch.argmax(min_distances)  # select

            last_item = sample_set[select_idx : select_idx + 1]
            min_distances[select_idx] = 0
            coreset_idx.append(select_idx)

        coreset_idx = torch.stack(coreset_idx)
        self.memory_bank = embedding[coreset_idx]

my repo:

    select_idx = 0
    last_item = z_lib[select_idx:select_idx+1]
    coreset_idx = [torch.tensor(select_idx)]
    min_distances = torch.linalg.norm(z_lib-last_item, dim=1, keepdims=True)
    # The line below is not faster than linalg.norm, although i'm keeping it in for
    # future reference.
    # min_distances = torch.sum(torch.pow(z_lib-last_item, 2), dim=1, keepdims=True)
    
    ...
    
    for _ in tqdm(range(n-1), **TQDM_PARAMS):
        distances = torch.linalg.norm(z_lib-last_item, dim=1, keepdims=True) # broadcasting step
        # distances = torch.sum(torch.pow(z_lib-last_item, 2), dim=1, keepdims=True) # broadcasting step
        min_distances = torch.minimum(distances, min_distances) # iterative step
        select_idx = torch.argmax(min_distances) # selection step

        # bookkeeping
        last_item = z_lib[select_idx:select_idx+1]
        min_distances[select_idx] = 0
        coreset_idx.append(select_idx.to("cpu"))

    return torch.stack(coreset_idx)

Fix NNCF Issues.

OpenvinoCallback and NNCFCallback implementations are quite similar. They only differ in terms of type of the quantization.

Tasks

  • Refactor Compression and NNCF callbacks could be merged into a single callback
  • Design an error handling mechanism to pick up the errors caused by nncf 2.1.0 vs 2.2.0. Warn the user to manually upgrade the nncf to 2.2.0
  • configure openvino and nncf callbacks from the config.yaml file.
  • change the requirements in openvino.txt - nncf>=2.1.0
  • Modify the nightly tests to handle nncf tests.
  • Refactor NNCF config.

Use console scripts to replace `python tools/train.py`

To run training or evaluation, we call a python script such as python tools/train.py --model padim. With console scripts, it would be possible to run the same code via anomalib fit --model padim. To achieve this we need to modify setup.py. For more information, see this

🧪 Refactor nightly thresholds.

Current Behaviour

Nightly job checks whether the tests produce results above a particular threshold. This might not be the best way to handle the tests.

Proposed Behaviour

Run the tests multiple times and report mean and standard deviation. These values should be compared for the nightly job.

A few questions

Hi,there are some codes which make me confused, hope to get your answer.

  1. In padim/config.yaml, what does nncf(belongs to optimization) mean? and what does function update_nncf_config mean?
  2. If i want to use multiple gpus to train, how should i set? and there are many parameters in trainer section which i wanna figure out each is used for and be clear which can be tuned or not.

Support python 3.7

Is your feature request related to a problem? Please describe.
I mainly use anomalib on Google Colab but it only supports python=3.7 at the moment. There is an error when running the training command:
python -m tools.train --model_config_path path/to/config.yaml

The error was:

Traceback (most recent call last):
  File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/content/anomalib/tools/train.py", line 66, in <module>
    train()
  File "/content/anomalib/tools/train.py", line 55, in train
    model = get_model(config)
  File "/content/anomalib/anomalib/models/__init__.py", line 61, in get_model
    module = import_module(f"anomalib.models.{config.model.name}.model")
  File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/content/anomalib/anomalib/models/stfpm/__init__.py", line 17, in <module>
    from .model import (  # noqa # pylint: disable=unused-import
  File "/content/anomalib/anomalib/models/stfpm/model.py", line 31, in <module>
    from anomalib.data.tiler import Tiler
  File "/content/anomalib/anomalib/data/tiler.py", line 19, in <module>
    from typing import Optional, Sequence, SupportsIndex, Tuple, Union
ImportError: cannot import name 'SupportsIndex' from 'typing' (/usr/lib/python3.7/typing.py)

Suggestion
AFAIK, SupportsIndex is used for type-hint of classes that have __item__ method, but the default value doesn't seem right and int is better.

After making the below modification:

diff --git a/anomalib/data/tiler.py b/anomalib/data/tiler.py
index 5a274c2..d061337 100644
--- a/anomalib/data/tiler.py
+++ b/anomalib/data/tiler.py
@@ -16,7 +16,7 @@
 
 from itertools import product
 from math import ceil
-from typing import Optional, Sequence, SupportsIndex, Tuple, Union
+from typing import Optional, Sequence, Tuple, Union
 
 import torch
 import torchvision.transforms as T
@@ -161,7 +161,7 @@ class Tiler:
         stride: Union[int, Sequence],
         remove_border_count: int = 0,
         mode: str = "padding",
-        tile_count: SupportsIndex = 4,
+        tile_count: int = 4,
     ) -> None:
 
         self.tile_size_h, self.tile_size_w = self.__validate_size_type(tile_size)

The code works well.

I did not have further investigation, e.g. checking dependencies, other models, etc; but I think supporting python 3.7 is possible with minimal changes. I ran with the STFMP model.

CUDA out of Memory when training `STFPM`

Got this error when training STFPM on pill category.
Occurred when training from the benchmarking script.

Error

output = self.trainer.call_hook('validation_step_end', *args, **kwargs)                                                                        
  File "/home/ashwin/miniconda3/envs/anomalib/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1230, in call_hook           
    output = hook_fx(*args, **kwargs)                                                                                                              
  File "/home/ashwin/anomalib/anomalib/core/model/anomaly_module.py", line 105, in validation_step_end                                             
    self.pixel_metrics(val_step_outputs["anomaly_maps"].flatten(), val_step_outputs["mask"].flatten().int())
  File "/home/ashwin/miniconda3/envs/anomalib/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/ashwin/miniconda3/envs/anomalib/lib/python3.8/site-packages/torchmetrics/collections.py", line 110, in forward
    return {k: m(*args, **m._filter_kwargs(**kwargs)) for k, m in self.items()}
  File "/home/ashwin/miniconda3/envs/anomalib/lib/python3.8/site-packages/torchmetrics/collections.py", line 110, in <dictcomp>
    return {k: m(*args, **m._filter_kwargs(**kwargs)) for k, m in self.items()}
  File "/home/ashwin/miniconda3/envs/anomalib/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/ashwin/miniconda3/envs/anomalib/lib/python3.8/site-packages/torchmetrics/metric.py", line 205, in forward
    self._forward_cache = self.compute()
  File "/home/ashwin/miniconda3/envs/anomalib/lib/python3.8/site-packages/torchmetrics/metric.py", line 367, in wrapped_func
    self._computed = compute(*args, **kwargs)
  File "/home/ashwin/anomalib/anomalib/core/metrics/optimal_f1.py", line 38, in compute
    precision, recall, thresholds = self.precision_recall_curve.compute() 
File "/home/ashwin/miniconda3/envs/anomalib/lib/python3.8/site-packages/torchmetrics/metric.py", line 367, in wrapped_func
    self._computed = compute(*args, **kwargs)
  File "/home/ashwin/miniconda3/envs/anomalib/lib/python3.8/site-packages/torchmetrics/classification/precision_recall_curve.py", line 148, in comp
ute
    return _precision_recall_curve_compute(preds, target, self.num_classes, self.pos_label)
  File "/home/ashwin/miniconda3/envs/anomalib/lib/python3.8/site-packages/torchmetrics/functional/classification/precision_recall_curve.py", line 2
60, in _precision_recall_curve_compute
    return _precision_recall_curve_compute_single_class(preds, target, pos_label, sample_weights)
  File "/home/ashwin/miniconda3/envs/anomalib/lib/python3.8/site-packages/torchmetrics/functional/classification/precision_recall_curve.py", line 1
40, in _precision_recall_curve_compute_single_class
    fps, tps, thresholds = _binary_clf_curve(
  File "/home/ashwin/miniconda3/envs/anomalib/lib/python3.8/site-packages/torchmetrics/functional/classification/precision_recall_curve.py", line 3
6, in _binary_clf_curve
    desc_score_indices = torch.argsort(preds, descending=True)
RuntimeError: CUDA out of memory. Tried to allocate 5.82 GiB (GPU 0; 23.70 GiB total capacity; 9.28 GiB already allocated; 2.61 GiB free; 19.41 GiB
 reserved in total by PyTorch)

Relevant Config

  • Image size = 256

1 epoch

Is it meant to be only one epoch in training?
Your config files state 1 epoch, is that just as quick example?
I tried to train PADIM for 10 on mvtec leather and wood and the metrics stay the same anyway, so it seems nothing as gained by training more.
Lightning module also warn that there is no optimizer so I guess train only finds the correct thresholds and that takes 1 epoch.

IndexError: tensors used as indices must be long, byte or bool tensors

Describe the bug
When training PatchCore, I'm getting the following error:
IndexError: tensors used as indices must be long, byte or bool tensors
caused by the line

To Reproduce
Steps to reproduce the behavior:
Run the PatchCore algorithm on CPU (maybe on GPU too)

The problem can be solved by casting to c_idx to c_idx.long()

Resize bug in inference

  • When input the width and height of the image are not equal, sizes of input arguments that do not match will occur

Line 152 in ./anomalib/deploy/inferencers/torch.py should be anomaly_map = cv2.resize(anomaly_map, (meta_data["image_shape"][1], meta_data["image_shape"][0]))

meta_data["image_shape"] is (image_width, image_height)
-# cv2.resize(img, (image_height,image_width ),interpolation=cv2.INTER_CUBIC)

Verify that best (not latest) model is used for testing after early stopping

In train.py, we call trainer.test right after the model has been trained with trainer.train. It is unclear if this leads to correct behaviour when early stopping is applied. Since we do not explicitly load the best model before calling trainer.test, it could be the case that we simply use the latest model. This should be investigated.

Add proper logging and remove `print` statements with `logger` via `logger = logging.getLogger("pytorch_lightning")`

Describe the bug

  • Currently print statements are used to log to console. Instead, logger should be used.
  • Polish out pl loggings.

Expected behavior

  • Modules should get access to the pytorch lightning logger via logger = logging.getLogger("pytorch_lightning") and log to the console.

Hardware and Software Configuration

  • OS: [Ubuntu, OD] - any
  • NVIDIA Driver Version [470.57.02] - any
  • CUDA Version [e.g. 11.4] - any
  • CUDNN Version [e.g. v11.4.120] - any
  • OpenVINO Version [Optional e.g. v2021.4.2] - any

Additional context

  • N/A

Heavy Augmentations + More Epochs

Hello!
I'm wondering if anyone has experimented with heavy augmentations and therefore increasing the number of training epochs.
If so, did you get any performance gains and/or information about conflicting augmentations?

Thanks!

PatchCore: Training only on good images and then inference on good and bad images

First of all, great work! I found this work very interesting! I have a question related to PatchCore implementation.

When there is both training, and testing dataset (images and ground truth) available, then the PatchCore works good and I think during the validation step, it is using the testing data (GT and images) to calculate coreset subsampling.

But imagine a realworld scenario when only good images are available to teach the anomaly detection algorithm, and no defective images are available (GT and images), then how to train the model? I've tried limit_val_batches and limit_test_batches to zero, so during training, no validation and testing are performed. But with this setup, during inference time, the model is producing this error

raceback (most recent call last):
  File "tools/inference.py", line 118, in <module>
    infer()
  File "tools/inference.py", line 101, in infer
    output = inference.predict(image=args.image_path, superimpose=True)
  File "/notebooks/anomalib/anomalib/deploy/inferencers/base.py", line 90, in predict
    predictions = self.forward(processed_image)
  File "/notebooks/anomalib/anomalib/deploy/inferencers/torch.py", line 118, in forward
    return self.model(image)
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/notebooks/anomalib/anomalib/models/components/base/anomaly_module.py", line 75, in forward
    return self.model(batch)
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/notebooks/anomalib/anomalib/models/patchcore/model.py", line 170, in forward
    patch_scores = self.nearest_neighbors(embedding=embedding, n_neighbors=9)
  File "/notebooks/anomalib/anomalib/models/patchcore/model.py", line 238, in nearest_neighbors
    distances = torch.cdist(embedding, self.memory_bank, p=2.0)  # euclidean norm
  File "/opt/conda/lib/python3.8/site-packages/torch/functional.py", line 1119, in cdist
    return _VF.cdist(x1, x2, p, None)  # type: ignore
RuntimeError: cdist only supports at least 2D tensors, X2 got: 1D

Any guidance will be highly appreciated!

Switch to CSV Logger

The repo implements csv logging as a custom callback. Since Pytorch Lightning already has a logger for this, the callback should be replaced with the logger.

Rename normalization to cdf

When we created this normalisation, we thought this would be the only method we would use. However, we implemented more methods such as min-max, due to which we need to explicitly rename this method to cdf normalisation

Add GANomaly metrics to READMEs

PR #70 does not add GANomaly metrics to the readmes.
It will be more convenient to add it after PR #17 is merged.

The READMEs should follow a similar structure as DFM and DFKDE

Tasks

  • GANomaly readme
  • Repository readme

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.