Code Monkey home page Code Monkey logo

pyronear / pyro-vision Goto Github PK

View Code? Open in Web Editor NEW
51.0 10.0 24.0 7.84 MB

Computer vision library for wildfire detection 🌲 Deep learning models in PyTorch & ONNX for inference on edge devices (e.g. Raspberry Pi)

Home Page: https://pyronear.org/pyro-vision/

License: Apache License 2.0

Python 96.16% Dockerfile 1.56% Makefile 2.28%
wildfire python deep-learning pytorch image-classification object-detection keypoint-detection computer-vision onnx

pyro-vision's Introduction

PyroNear Logo

CI Status Documentation Status Test coverage percentage black

PyPi Status Anaconda Docker Image Version pyversions license

Pyrovision: wildfire early detection

The increasing adoption of mobile phones have significantly shortened the time required for firefighting agents to be alerted of a starting wildfire. In less dense areas, limiting and minimizing this duration remains critical to preserve forest areas.

Pyrovision aims at providing the means to create a wildfire early detection system with state-of-the-art performances at minimal deployment costs.

Quick Tour

Automatic wildfire detection in PyTorch

You can use the library like any other python package to detect wildfires as follows:

from pyrovision.models import rexnet1_0x
from torchvision import transforms
import torch
from PIL import Image


# Init
normalize = transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])

tf = transforms.Compose([transforms.Resize(size=(448)), transforms.CenterCrop(size=448),
                         transforms.ToTensor(), normalize])

model = rexnet1_0x(pretrained=True).eval()

# Predict
im = tf(Image.open("path/to/your/image.jpg").convert('RGB'))

with torch.no_grad():
    pred = model(im.unsqueeze(0))
    is_wildfire = torch.sigmoid(pred).item() >= 0.5

Setup

Python 3.6 (or higher) and pip/conda are required to install PyroVision.

Stable release

You can install the last stable release of the package using pypi as follows:

pip install pyrovision

or using conda:

conda install -c pyronear pyrovision

Developer installation

Alternatively, if you wish to use the latest features of the project that haven't made their way to a release yet, you can install the package from source:

git clone https://github.com/pyronear/pyro-vision.git
pip install -e pyro-vision/.

What else

Documentation

The full package documentation is available here for detailed specifications.

Demo app

The project includes a minimal demo app using Gradio

demo_app

You can check the live demo, hosted on πŸ€— HuggingFace Spaces πŸ€— over here πŸ‘‡ Hugging Face Spaces

Docker container

If you wish to deploy containerized environments, a Dockerfile is provided for you build a docker image:

docker build . -t <YOUR_IMAGE_TAG>

Minimal API template

Looking for a boilerplate to deploy a model from PyroVision with a REST API? Thanks to the wonderful FastAPI framework, you can do this easily. Follow the instructions in ./api to get your own API running!

Reference scripts

If you wish to train models on your own, we provide training scripts for multiple tasks! Please refer to the ./references folder if that's the case.

Citation

If you wish to cite this project, feel free to use this BibTeX reference:

@misc{pyrovision2019,
    title={Pyrovision: wildfire early detection},
    author={Pyronear contributors},
    year={2019},
    month={October},
    publisher = {GitHub},
    howpublished = {\url{https://github.com/pyronear/pyro-vision}}
}

Contributing

Please refer to CONTRIBUTING to help grow this project!

License

Distributed under the Apache 2 License. See LICENSE for more information.

pyro-vision's People

Contributors

akilditu avatar blenzi avatar dependabot[bot] avatar fe51 avatar frgfm avatar mateolostanlen avatar powerexploit avatar skupr-anaconda avatar tekayanidham avatar x0s avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pyro-vision's Issues

[models] Module design suggestion

Here is a suggestion for the pyrovision.models module's organization:

  • utils.py: shared utility functions
  • XYZ.py: classification architecture definion with pretrained weights
  • detection: folder with the definition of object detection models with pretrained weights
  • segmentation: folder with the definition of segmentation models with pretrained weights

All performances of pretrained versions should be reported either in the documentation, the readme or a readme in the references folder.

[test] Clean out fixture content files

There are multiple content files used as fixtures for this project. In the current use cases, this isn't useful:

  • for csv-like fixture, identify the minimal structure required and hardcode it in python in the test
  • for image-like fixtures, find a publicly available image URL, and DL it as a tmp file to run the unittests

[datasets] Allow parallel processing of downloads

πŸš€ Feature

Use parallel processing for dataset downloads when a list of URLs is involved.

Motivation

pyronear.datasets.OpenFire downloading process is taking several minutes while allowing multi-processing would shorten this by a lot.

Pitch

Add a multi-processing feature for URL downloads and let the users pick the number of workers. Potential libraries to check: Ray, python multiprocessing
A good implementation for URL downloading is already present in fastai (but it is a large dependency).

[ci] Unpickling error for pretrained parameter loading

πŸ› Bug

The pretrained=True option on models cannot be tested by CI, since passing it produces a pickle error that cannot be reproduced locally.

To Reproduce

Steps to reproduce the behavior:

  1. Running the pkg-test CI job on this commit

which produces:

Traceback (most recent call last):
  File "/home/circleci/repo/test/test_models.py", line 22, in test_resnet
    model = models.resnet(depth=18, pretrained=True, bin_classif=True).eval()
  File "/home/circleci/repo/pyronear/models/resnet.py", line 49, in resnet
    progress=progress)
  File "/home/circleci/repo/venv/lib/python3.6/site-packages/torch/hub.py", line 499, in load_state_dict_from_url
    return torch.load(cached_file, map_location=map_location)
  File "/home/circleci/repo/venv/lib/python3.6/site-packages/torch/serialization.py", line 426, in load
    return _load(f, map_location, pickle_module, **pickle_load_args)
  File "/home/circleci/repo/venv/lib/python3.6/site-packages/torch/serialization.py", line 603, in _load
    magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, 'Y'.

Expected behavior

State dictionary loading should not produce errors.

Environment

Please copy and paste the output from our
environment collection script
(or fill out the checklist below manually).

You can get the script and run it with:

wget https://raw.githubusercontent.com/frgfm/PyroNear/master/pyronear/utils/collect_env.py
# For security purposes, please check the contents of collect_env.py before running it.
python collect_env.py
  • PyTorch Version: 1.3.0
  • Torchvision Version: 0.4.1a0+d94043a
  • OS: Ubuntu 18.04
  • How you installed PyTorch: conda
  • Python version: 3.7
  • CUDA/cuDNN version: CUDA 10.1.168 / cuDNN 7.6.2
  • GPU models and configuration: GeForce GTX 1050 (driver: 430.50)

Issue downloading OpenFire

πŸ› Bug

This test do not pass anymore :

We have a problem with OpenFire, the links to the images work less and less.

Now this test doesn't work anymore:

self.assertAlmostEqual(len(train_set) + len(test_set), len(extract), delta=20)

In the past we have already increased the threshold from 15 to 20 to get around this problem.

I propose to do it again with a big margin, let's take 30.

On the other hand we have to absolutely store this dataset somewhere soon and change the download mode.

pytorch and torchvision compiled for ARM processor

πŸš€ Feature

Add a version of pytorch and torchvision packages compiled for ARM processor

Motivation

To date, the pyronear prototypes are made of raspberry pi, composed of ARM processors.
torch and torchvision, required to use pyro-vision package are not compiled for ARM processors.

Pitch

It would be great to have those available from pyro-vision (versions of packages available according to pyro-vision release and versions used).

Also, it would be great to have a tutorial to get to the compiled versions of these packages.

Thanks a lot and happy to discuss this issue !

[datasets] Documentation error within WildfireDataset class where path_to_frame should be a path type

πŸ› Bug

path_to_frame should be a path type and not an str as mentioned below -->

https://github.com/frgfm/PyroNear/blob/61d5d527187a6f6b0696da0bc199c1a80a2ee2c0/pyronear/datasets/wildfire/wildfire.py#L31-L32

str is not supported by PyTorch when iterating trough DataLoader

To Reproduce

Steps to reproduce the behavior:

  1. str path

path_to_frames = "/Users/sebastienelmaleh/Google Drive/Deep Learning/Pyronear/frames_raw_wildfire/"

  1. Iterating through DataLoader
for epoch in range(num_epochs):
    for i, (images, labels) in enumerate(wildfire_loader_train):

which produces :


TypeError Traceback (most recent call last)
in
4 iter = 0
5 for epoch in range(num_epochs):
----> 6 for i, (images, labels) in enumerate(wildfire_loader_train):
7
8 labels = labels.squeeze()

/usr/local/lib/python3.7/site-packages/torch/utils/data/dataloader.py in next(self)
344 def next(self):
345 index = self._next_index() # may raise StopIteration
--> 346 data = self._dataset_fetcher.fetch(index) # may raise StopIteration
347 if self._pin_memory:
348 data = _utils.pin_memory.pin_memory(data)

/usr/local/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py in fetch(self, possibly_batched_index)
42 def fetch(self, possibly_batched_index):
43 if self.auto_collation:
---> 44 data = [self.dataset[idx] for idx in possibly_batched_index]
45 else:
46 data = self.dataset[possibly_batched_index]

/usr/local/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py in (.0)
42 def fetch(self, possibly_batched_index):
43 if self.auto_collation:
---> 44 data = [self.dataset[idx] for idx in possibly_batched_index]
45 else:
46 data = self.dataset[possibly_batched_index]

~/seb_PyroNear/PyroNear/pyronear/datasets/wildfire/wildfire.py in getitem(self, index)
59 - x,y (float, float) and
60 - Exploitable(True/False)"""
---> 61 path_to_frame = self.path_to_frames / self.metadata['imgFile'].iloc[index]
62 observation = self.load_image(path_to_frame)
63

TypeError: unsupported operand type(s) for /: 'str' and 'str'

Expected behavior

Importing pathlib and converting str to path type corrected the error

[license] Choose an appropriate license for the project

This has been discussed several times, but no final conclusion has been made. After our last discussion, it seemed that everyone agreed on a single license, but now there is the choice between:

  • Apache 2.0
  • GPLv3
  • aGPLv3
  • CECILL

The double constraint being:

  • avoiding disloyal usage of our work for financial profit (without sharing it back)
  • make it simple enough so that it does not discourage contributors (we don't want a situation like 50 pages of User conditions)

Let's discuss this here and act on it πŸ™

SSL CERTIFICATE_VERIFY_FAILED: cannot download files in dataset, branch frgfm/dataset_draft

πŸ› Bug

Running locally, SSL CERTIFICATE_VERIFY_FAILED: cannot download files in dataset

To Reproduce

Steps to reproduce the behavior:

  1. Checkout locally branch frgfm/dataset_draft
  2. Create OpenFire object
git remote add upstream [email protected]:frgfm/PyroNear.git
git checkout -t upstream/dataset_draft
python -c "from pyronear.datasets import OpenFire; dataset = OpenFire(root='./data', download=True)"

Expected behavior

urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1045)>

Environment

Please copy and paste the output from our
environment collection script
(or fill out the checklist below manually).

You can get the script and run it with:

wget https://raw.githubusercontent.com/frgfm/PyroNear/master/pyronear/utils/collect_env.py
# For security purposes, please check the contents of collect_env.py before running it.
python collect_env.py

PyTorch version: 1.2.0
Is debug build: No
CUDA used to build PyTorch: None
Torchvision version: 0.4.0

OS: Mac OSX 10.13.4
GCC version: Could not collect
CMake version: version 3.6.2

Python version: 3.7
Is CUDA available: No
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA

Versions of relevant libraries:
[pip3] numpy==1.17.2
[pip3] torch==1.2.0
[pip3] torchvision==0.4.0
[conda] Could not collect

Additional context

[models] Improve the robustness of classification models againt fog occurrences

Our baseline classification model is not robust in case of atmospheric fog (from high altitude view point looking down in the valley) or general fog.

Here are a few examples:

IMG_6613
IMG_6616

Some suggestions about how to handle this:

  • use a depth estimation model and manually correct the brightness/contrast to avoid sending confusing images to the bare model
  • label a dataset with cases of fog, different from wildfires and train the model with this

[datasets] Add multi-label classification annotations to OpenFire

πŸš€ Feature

Add multi-classification targets to OpenFire dataset.

Motivation

Currently, OpenFire only contains binary classification target, which is not enough to reach production-ready performances. Extending the target to multi-label will help reaching better model performances.

Pitch

The JSON file with annotations will have to be updated with including targets for multiple task:

  • binary classification: single boolean value per sample (True notifying the presence of a fire)
  • multi-label classification: a few options
  1. list of present class per sample (eg. ['class1', 'class3'])
  2. list of present class indices per samples (eg. [0, 2])
  3. one-hot encoded vector of present classes (eg. [True, False, True, ...])

[models] Add keypoint detection models

Currently we perform keypoint detection by:

  • using a robust image classifier
  • use its class activation to extrapolate the keypoint

We should explore alternative ways with light backbones to perform actual keypoint detection and train it properly.

[references] Disable codecarbon logger

Running the classification training script with any options gets the codecarbon logger to start. But this becomes a bit too much as it pops information in the middle of training cycles:

Namespace(amp=False, arch='rexnet1_0x', batch_size=32, data_path='/home/fg/Downloads/', dataset='openfire', device=None, epochs=20, find_lr=False, freeze_until=None, img_size=224, lr=0.001, name=None, norm_wd=None, opt='adamp', output_file='./model.pth', pretrained=False, resume='', sched='onecycle', show_samples=True, test_only=False, wb=False, weight_decay=0, workers=16)
Loading data
  3%|β–ˆβ–ˆβ–                                                                           | 225/7880 [00:13<05:50, 21.84it/s][codecarbon INFO @ 23:26:55] Energy consumed for RAM : 0.000024 kWh. RAM Power : 5.744381904602051 W
[codecarbon INFO @ 23:26:55] Energy consumed for all CPUs : 0.000000 kWh. All CPUs Power : 0.0 W
[codecarbon INFO @ 23:26:55] 0.000024 kWh of electricity used since the begining.
  6%|β–ˆβ–ˆβ–ˆβ–ˆβ–Œ                                                                         | 457/7880 [00:28<07:48, 15.84it/s][codecarbon INFO @ 23:27:10] Energy consumed for RAM : 0.000048 kWh. RAM Power : 5.744381904602051 W
[codecarbon INFO @ 23:27:10] Energy consumed for all CPUs : 0.000000 kWh. All CPUs Power : 0.0 W
[codecarbon INFO @ 23:27:10] 0.000048 kWh of electricity used since the begining.
  8%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‰                                                                        | 598/7880 [00:42<08:30, 14.26it/s][codecarbon INFO @ 23:27:25] Energy consumed for RAM : 0.000072 kWh. RAM Power : 5.744381904602051 W
[codecarbon INFO @ 23:27:25] Energy consumed for all CPUs : 0.000000 kWh. All CPUs Power : 0.0 W
[codecarbon INFO @ 23:27:25] 0.000072 kWh of electricity used since the begining.
 10%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–Œ                                                                      | 762/7880 [00:57<09:24, 12.60it/s][codecarbon INFO @ 23:27:40] Energy consumed for RAM : 0.000096 kWh. RAM Power : 5.744381904602051 W
[codecarbon INFO @ 23:27:40] Energy consumed for all CPUs : 0.000000 kWh. All CPUs Power : 0.0 W
[codecarbon INFO @ 23:27:40] 0.000096 kWh of electricity used since the begining.
 11%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‰                                                                     | 899/7880 [01:13<08:53, 13.09it/s][codecarbon INFO @ 23:27:55] Energy consumed for RAM : 0.000120 kWh. RAM Power : 5.744381904602051 W
[codecarbon INFO @ 23:27:55] Energy consumed for all CPUs : 0.000000 kWh. All CPUs Power : 0.0 W
[codecarbon INFO @ 23:27:55] 0.000120 kWh of electricity used since the begining.
 14%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–

I think it's better to either set a logger level that will prevent this or disable the logger completely

[ci] Optimize cache for datasets downloading

πŸš€ Feature

Save downloaded datasets in CI cache.

Motivation

With the addition of datasets.OpenFire, the test require the download of the entire dataset. Considering that the samples are scattered in multiple URLs, the download is taking at least several minutes.

Pitch

  • find a unique identifier to checksum all dataset URLs
  • save cache once the job is completed to avoid downloading again if the file is the same

[models] Create specific model's heads for binary classification

πŸš€ Feature

Create a models.utils submodule with a function to easily create model heads depending on the type of task, namely binary classification here.

Motivation

Current training scripts in references assume that the target is categorical (multi-label) rather than binary. A proper task setting would allow the model to better optimize during training.

Pitch

The function would take as arguments:

  • model
  • task type
  • options for head design (dropout, bn, concatenated pooling, etc.)

and return a torch.nn.Sequential object.

[datasets] some OpenFire images have numeric extensions

πŸ› Bug

Some downloaded images from OpenFire have numeric extensions .760

To Reproduce

Steps to reproduce the behavior:

python -c "from pyronear.datasets import OpenFire; dataset = OpenFire('./data', download=True)
find ./data -name "*.760"

Expected behavior

All files should have image type extensions

Environment

  • PyTorch Version: 1.2.0
  • Torchvision Version: 0.4.0a0
  • OS: Ubuntu
  • How you installed PyTorch: conda
  • Python version: 3.7
  • CUDA/cuDNN version: 10.1.168 / 7.6.2
  • GPU models and configuration: GeForce GTX 1050 (driver: 430.50)

Merge csv files for testing

πŸš€ Feature

Complete csv file for testing

Motivation

We have two csv's to test the wildfire dataset, we should put them together in one. This new csv must be as complete as possible to allow future tests.

[package setup] pip install on remote git fails

πŸ› Bug

When using pip install from git directly, modules are not picked correctly by the installation script.

ModuleNotFoundError: No module named 'pyronear'

To Reproduce

Steps to reproduce the behavior:

  1. Install the package from GitHub
pip install git+https://github.com/frgfm/PyroNear
  1. Import the library
python -c 'import pyronear'

Expected behavior

Expected import to run correctly

Environment

  • PyTorch Version: 1.2.0
  • Torchvision Version: 0.4.0
  • OS (e.g., Linux): Linux
  • How you installed PyTorch (conda, pip, source): conda
  • Python version: 3.7

[models] Add the possibility to use custom activation

πŸš€ Feature

Offer the possibility to change default activation used in models

Motivation

Recent new activation functions such as Swish or Mish proved to be very useful in vision-related deep learning models. If training from scratch, any user may want to use custom activation such as the ones mentioned above.

Pitch

Add activation_fn as an argument for model

Alternatives

Create an alternate version of each model with the best activation option for each architecture

Bug: "No module named virtualenvwrapper"

πŸ› Bug

At Project Setup, when sourcing the bashrc to create a virtual environment, this error arouse

/usr/bin/python: No module named virtualenvwrapper
virtualenvwrapper.sh: There was a problem running the initialization hooks.

If Python could not import the module virtualenvwrapper.hook_loader,
check that virtualenvwrapper has been installed for
VIRTUALENVWRAPPER_PYTHON=/usr/bin/python and that PATH is
set properly.

To Reproduce

Steps to reproduce the behavior:

pip install virtualenvwrapper

# add at the end of your .bashrc
export WORKON_HOME=$HOME/.virtualenvs
export PROJECT_HOME=$HOME/Devel
source /usr/local/bin/virtualenvwrapper.sh

Expected behavior

Install properly virtual env, no error shown.

Environment

  • OS: Ubuntu 20.04 LTS
  • Python version: 3.8.2

[datasets] Split creation of path structure from actual download of images

πŸš€ Feature

Split creation of path structure and files that define train/test splitting from actual download of images

Motivation

In order to use an external folder with images and preserve the test/train split defined in OpenFire without having to manually create the path structure.

Pitch

Alternatives

Additional context

[models] Add an object detection model

The latest torchvision releases include highly optimized models for object detection and semantic segmentation. Using our keypoint annotations, we could extrapolate a small bounding box around the keypoint and train a FasterRCNN with a MobileNetV3 backbone.

Release tracker - v0.2.0

This issue is to be used to track the roadmap of pyrovision for release v0.2.0, and collect feedback from users & contributors.

  • Datasets
  • Models
    • Deprecates inhouse models & use external sources for architecture definition (#138)
    • #150
    • #123
  • NN
    • Deprecates the entire module (#138)
  • References
    • Updates training script with modern tools (#138)
    • #140
    • Adds W&B logging (#138)
  • Tests
    • Switch to pytest (#138)
  • API
    • Adds a minimal FastAPI template to deploy pyrovision (#152)
  • Demo
    • Adds a minimal Streamlit/Gradio demo (#151)
  • Other

[models] Pretrained param loading design

There is currently no clear design for model checkpoint loading. Here is a proposal:

  • Data integrity: add the first 8 character of the SHA256 hash at the end of file name. Use this to check integrity upon download
  • Hosting: all pretrained models should be hosted on Github
  • File naming: each file should follow the naming convention architecture_imgsize-hash.pth (example: resnet50_224-64cb5b95.pth)
  • Architecture module: small suggestion https://github.com/frgfm/Holocron/blob/master/holocron/models/repvgg.py since this library is already in the dependencies

[models] Add binary classification model with pretrained weights

πŸš€ Feature

Create a models module in the package that allows users to pick a binary classification model with pretrained weights.

Motivation

While users are able to train on the dataset, for ease of use, a stable pretrained model should be accessible for people who only wish to perform inference.

Pitch

The module will implement model similarly to https://github.com/pytorch/vision/tree/master/torchvision/models where a pretrained argument can be set to True to load in the pretrained parameters.

[datasets] OpenFire sample downloading fails

πŸ› Bug

OpenFire dataset relies on URLs of publicly available images. If some images cannot be accessed, the behaviour of instantiating the dataset should be clear.

To Reproduce

Steps to reproduce the behavior:

from pyronear.datasets import OpenFire
dataset = OpenFire(root='./data', download=True)

which throws a warning showing that there were some HTTP Errors:

UserWarning: 2/1009 samples could not be downloaded. Please retry later.

Expected behavior

Either raise an Error as the dataset is incomplete or attempts downloading the files that threw an error earlier.

Environment

  • PyTorch Version: 1.2.0
  • Torchvision Version: 0.4.0a0
  • OS: Ubuntu
  • How you installed PyTorch: conda
  • Python version: 3.7
  • CUDA/cuDNN version: 10.1.168 / 7.6.2
  • GPU models and configuration: GeForce GTX 1050 (driver: 430.50)

Release tracker - v0.3.0

This issue is to be used to track the roadmap of pyrovision for release v0.3.0, and collect feedback from users & contributors.

  • Datasets

    • Add an object detection dataset
  • Models

    • [] Add object detection models
  • References

    • [] Adds object detection training script
  • Tests

  • API

  • Demo

    • Add an object detection model as a demo
  • Other

[references] Add training script for binary classification

πŸš€ Feature

Add training script for binary classification model

Motivation

In order to have comparable evaluation runs of training, a training script can serve as a template for comparison and reproducibility.

Pitch

The training script could be called with optional arguments that would tweak the different parameters according to the user's preferences.

[DataSet] Allow test ratio to zero in WildFireSplitter

πŸš€ Feature

Motivation

At the moment the input ratios of WildFireSplitter cannot be equal to zero. I would like to add this option. Indeed we are going to do a lot of tests with different extraction methods (number of frames, spacing between frames ...). As the frame extraction is long I think we will save a lot of time if we don't have to do it every time for the test set.

I will try to propose a PR to add this option today

Wrong batch images format when iterating through DataLoader

πŸ› Bug

https://github.com/frgfm/PyroNear/blob/61d5d527187a6f6b0696da0bc199c1a80a2ee2c0/pyronear/datasets/utils.py#L19

Within VisionMixin class, 'load_image' method's 'to_tensor' parameter should be set to 'True' by default.

To Reproduce

Steps to reproduce the behavior:

  1. Iterate through dataLoader
for epoch in range(num_epochs):
    for i, (images, labels) in enumerate(wildfire_loader_train):

which produces :

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-12-4d31344fa390> in <module>
      4 iter = 0
      5 for epoch in range(num_epochs):
----> 6     for i, (images, labels) in enumerate(wildfire_loader_train):
      7 
      8         labels = labels.squeeze()

/usr/local/lib/python3.7/site-packages/torch/utils/data/dataloader.py in __next__(self)
    344     def __next__(self):
    345         index = self._next_index()  # may raise StopIteration
--> 346         data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
    347         if self._pin_memory:
    348             data = _utils.pin_memory.pin_memory(data)

/usr/local/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py in fetch(self, possibly_batched_index)
     45         else:
     46             data = self.dataset[possibly_batched_index]
---> 47         return self.collate_fn(data)

/usr/local/lib/python3.7/site-packages/torch/utils/data/_utils/collate.py in default_collate(batch)
     77     elif isinstance(elem, container_abcs.Sequence):
     78         transposed = zip(*batch)
---> 79         return [default_collate(samples) for samples in transposed]
     80 
     81     raise TypeError(default_collate_err_msg_format.format(elem_type))

/usr/local/lib/python3.7/site-packages/torch/utils/data/_utils/collate.py in <listcomp>(.0)
     77     elif isinstance(elem, container_abcs.Sequence):
     78         transposed = zip(*batch)
---> 79         return [default_collate(samples) for samples in transposed]
     80 
     81     raise TypeError(default_collate_err_msg_format.format(elem_type))

/usr/local/lib/python3.7/site-packages/torch/utils/data/_utils/collate.py in default_collate(batch)
     79         return [default_collate(samples) for samples in transposed]
     80 
---> 81     raise TypeError(default_collate_err_msg_format.format(elem_type))

TypeError: default_collate: batch must contain tensors, numpy arrays, numbers, dicts or lists; found <class 'PIL.Image.Image'>

Expected behavior

If set to True, image is transformed into a pytorch tensor as expected.

Change demo image

πŸš€ Feature

This may be a detail but I think it would be a good idea to change the demo image to one that is closer to our use case. An image like this one for example

Motivation & pitch

not much to add here

Alternatives

No response

Additional context

No response

Consider changing the image color normalization

Bug description

I wrote a short python script to try out the pyro-vision model. It can be found at https://github.com/KevinJMcEntee/WildfireDetector. It contains this image color normalization transform:

normalize = transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])

These values were copied from the pyro-vision/README.md file. Further internet searches showed me these values are the mean and standard deviation values of the ImageNet image database.

Here is an image I ran through my script. It shows a wildfire I photographed with my iPhone a few summers ago.
Santa5FireResized

Here is that same image, transformed using the ImageNet values shown above. Notice the transform has left the smoke with a color closely matching the surrounding sky. The pyro-vision model predicted this image is negative for wildfire.
Santa5Fire_withImageNetNormalization

My manual visual inspection led to this insight. So, just for a baseline, I changed the transform to:
normalize = transforms.Normalize(mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5])
This resulted in this normalized image:
Santa5Fire_withPoint5Normalization

Using this last image transform, pyro-vision predicted this image is positive for wildfire.

This investigation leads me to recommend developing a color transform that is specific to the wildfire smoke training set rather than using the general purpose ImageNet as the source of those values.

Code snippet to reproduce the bug

No Errors

Error traceback

No Errors

Environment

python3 on Macbook Air running MacOS 12.6. Using values from pyro-vision as of November 14, 2022. Script found at https://github.com/KevinJMcEntee/WildfireDetector/blob/main/cliWildFireDetect.py

Add demo

πŸš€ Feature

I think we should add a demo script that loads a pre-training model and applies it to a video sequence.

Motivation

The purpose of this demo is to show an example of how to use the latest pyronear model, allowing anyone to use it on the video of their choice.

[models] Standardize cnn model creation

πŸš€ Feature

All CNN models should have a clear similar high-level architecture.

Motivation

References script can currently handle only resnet models since the modifications of last layers require knowing their name (since their position varies). Having a similar indicator (positional or name) would allow for standard manipulation, saving and loading of models.

Pitch

Taking fastai as an inspiration, a good option might be to have high-level model as a torch.nn.Sequential with two subcomponents: the first one being the convolutional layers, and the second being the classifier.

[test] Add a linter test to CI

πŸš€ Feature

Add a linter verification to CI

Motivation

For better readability, and to apply by industry standards, linter verification would allow users to amend their PR for readability.

Pitch

Include flake8 linter test into CI jobs.

[datasets] Add a classification dataset

πŸš€ Feature

Add a classification dataset to the datasetsmodule

Motivation

Since the project needs to provide models for wildfire detection and allow external contributors to train their own, making datasets available is a priority.

Pitch

  • similarly to torchvision datasets module, each dataset needs to be a class inheriting from torchvision.datasets.vision.VisionDataset or at least torch.utils.data.Dataset.
  • she download URL needs to be accessible by the class.

[docs] Add a simple documentation

πŸš€ Feature

Add package documentation built with Sphinx for easier user usage.

Motivation

Non-developer users will most likely install the package and will not read the code or docstrings to use the package. Hence the need for documentation

No matching distribution found for pyrovision

πŸ› Bug

Context: Installing requirements

To Reproduce

Steps to reproduce the behavior:

pip install pyrovision

output:

  Collecting pyrovision
  Could not find a version that satisfies the requirement pyrovision (from versions: )
  No matching distribution found for pyrovision
  • Tested with pip 19.0 and 20.3.3

Environment

Please copy and paste the output from our
environment collection script
(or fill out the checklist below manually).

  • PyTorch Version (e.g., 1.2): 1.7.1
  • Torchvision Version (e.g., 0.4): 0.8.2
  • OS (e.g., Linux): Ubuntu 20.04
  • How you installed PyTorch (conda, pip, source):pip
  • Python version:3.6

Additional context

Veille

πŸš€ Feature

Hello l'Γ©quipe πŸ‘‹ pas une feature request pas comment peut on contribuer autrement que par dΓ©tecter les feux ?
par exemple je voulais vous envoyer cet article Γ©crit par une amie si vous ne l'aviez pas vu : https://humeco.fr/feux-de-foret-france/

Motivation & pitch

voir plus haut

Alternatives

No response

Additional context

Ancienne de Bayes ;)

Prediction issues with pyronear/rexnet1_3x

Bug description

It seems that there is a problem with the new pyronear/rexnet1_3x model, the predictions are not very good. It predicts a score of 14% for the mock_wildfire_image used in pyro-engie tests, which makes the test that expects an image with fire/smoke to fail.

FAILED tests/test_engine_core.py::test_engine_offline - assert 0 == 1

Code snippet to reproduce the bug

hub_repo = "pyronear/rexnet1_3x"
_path =  hf_hub_download(hub_repo, filename="config.json")
with open(_path, "rb") as f:
    cfg = json.load(f)

_path = hf_hub_download(hub_repo, filename="model.onnx")
ort_session = onnxruntime.InferenceSession(_path)

# Load image

url = "https://github.com/pyronear/pyro-vision/releases/download/v0.1.2/fire_sample_image.jpg"
mock_wildfire_stream = requests.get(url).content
img = Image.open(BytesIO(mock_wildfire_stream))
img = img.resize(cfg["input_shape"][-2:], Image.BILINEAR)
print("img size", img.size)
# (H, W, C) --> (C, H, W)
img = np.asarray(img).transpose((2, 1, 0)).astype(np.float32) / 255
# Normalization
img -= np.array(cfg["mean"])[:, None, None]
img /= np.array(cfg["std"])[:, None, None]
ort_input = {ort_session.get_inputs()[0].name: img[None, ...]}
ort_out = ort_session.run(None, ort_input)
# Sigmoid
print(1 / (1 + np.exp(-ort_out[0][0])))

Error traceback

FAILED tests/test_engine_core.py::test_engine_offline - assert 0 == 1

Environment

Collecting environment information...
Pyrovision version: 0.1.2.dev0+ce4db67
PyTorch version: 1.12.1+cu102
Torchvision version: 0.13.1+cu102

OS: Ubuntu 20.04.4 LTS

Python version: 3.8
Is CUDA available: Yes
CUDA runtime version: Could not collect
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3050 Ti Laptop GPU
Nvidia driver version: 512.72
cuDNN version: Could not collect

[datasets] Module design suggestion

Here is a suggestion for the pyrovision.datasets module's organization:

  • utils.py: shared utility functions
  • XYZ.py: a VisionDataset inherited class definion with functions for dataset XYZ

This suggestion is based on the organization of https://github.com/pytorch/vision/tree/master/torchvision/datasets which is quite effective. The documentation should also include detailed explanation on how to use it.

If one dataset was made available by Pyronear, it should be accessible via a link with a hash for integrity check.

  • Refactors into a flat module (#97)
  • Assess which functions/classes should be kept in the repo (#136, #138)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.