Code Monkey home page Code Monkey logo

opencv_transforms's Introduction

opencv_transforms

This repository is intended as a faster drop-in replacement for Pytorch's Torchvision augmentations. This repo uses OpenCV for fast image augmentation for PyTorch computer vision pipelines. I wrote this code because the Pillow-based Torchvision transforms was starving my GPU due to slow image augmentation.

Requirements

  • A working installation of OpenCV. Tested with OpenCV version 3.4.1, 4.1.0
  • Tested on Windows 10 and Ubuntu 18.04. There is evidence that OpenCV doesn't work well with multithreading on Linux / MacOS, for example num_workers >0 in a pytorch DataLoader. I haven't run into this issue yet.

Installation

opencv_transforms is now a pip package! Simply use

  • pip install opencv_transforms

Usage

Breaking change! Please note the import syntax!

  • from opencv_transforms import transforms
  • From here, almost everything should work exactly as the original transforms.

Example: Image resizing

import numpy as np
image = np.random.randint(low=0, high=255, size=(1024, 2048, 3))
resize = transforms.Resize(size=(256,256))
image = resize(image)

Should be 1.5 to 10 times faster than PIL. See benchmarks

Performance

  • Most transformations are between 1.5X and ~4X faster in OpenCV. Large image resizes are up to 10 times faster in OpenCV.
  • To reproduce the following benchmarks, download the Cityscapes dataset.
  • An example benchmarking file can be found in the notebook bencharming_v2.ipynb I wrapped the Cityscapes default directories with a HDF5 file for even faster reading.

resize random crop change brightness change brightness and contrast change contrast only random horizontal flips

The changes start to add up when you compose multiple transformations together. composed transformations

TODO

  • Initial commit with all currently implemented torchvision transforms
  • Cityscapes benchmarks
  • Make the resample flag on RandomRotation, RandomAffine actually do something
  • Speed up augmentation in saturation and hue. Currently, fastest way is to convert to a PIL image, perform same augmentation as Torchvision, then convert back to np.ndarray

opencv_transforms's People

Contributors

jbohnslav avatar norwegianrockcat avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

opencv_transforms's Issues

bug fixed but haven't released yet

Hi, I noticed you fixed a bug of Resize, commit id: 2f6a07d. However, the latest version on pip doesn't include this change. Could you please release a new version? Thx.

Bug in functional.five_crop

In five_crop()

  1. get h,w from img shape wrongly.
  2. crop the image in a wrong way.
    It should be as belowed
def five_crop(img, size):
    if isinstance(size, numbers.Number):
        size = (int(size), int(size))
    else:
        assert len(size) == 2, "Please provide only two dimensions (h, w) for size."
    h,w = img.shape[0:2]
    crop_h, crop_w = size
    if crop_w > w or crop_h > h:
        raise ValueError("Requested crop size {} is bigger than input size {}".format(size,
                                                                                      (h, w)))
    tl = crop(img, 0, 0, crop_h, crop_w)
    tr = crop(img, 0, w-crop_w, crop_h, crop_w)
    bl = crop(img, h - crop_h,0, crop_h, crop_w)
    br = crop(img, h - crop_h, w- crop_w, crop_h, crop_w)
    center = center_crop(img, (crop_h, crop_w))
    return [tl, tr, bl, br, center]

pytorch tranformer output is different form the opencv_transorms

first of all. thanks for work. it is what i want.
the opencv _transform snippet output

def custom_transform(path):
    img = cv2.cvtColor(cv2.imread(path), cv2.COLOR_BGR2RGB)
    img_resize = transforms.Resize(size=(224,224))(img)
    img_tensor = transforms.ToTensor()(img_resize)
    img_norm = transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])(img_tensor)
    out = np.expand_dims(img_norm, 0)
    print(out[0][0][0][0])

the pytorch transform output

def torch_transform(path):
    transform_list = torch_transforms.Compose([
            torch_transforms.Resize(224),
            torch_transforms.ToTensor(),
            torch_transforms.Normalize(mean=[0.485, 0.456, 0.406],
                                     std=[0.229, 0.224, 0.225]),
        ])
    img = Image.open(path).convert('RGB')
    out = transform_list(img).unsqueeze_(0)
    print(out[0][0][0][0])

the result :
tensor(0.2453)
0.1939379

ColorJitter can't handle float32 images?

Hi there, using cvtransforms.ColorJitter(brightness=(0, 0.1), contrast=(0, 0.1)) gives me error:
File "/home/ma/anaconda3/envs/ml/lib/python3.6/site-packages/opencv_transforms/transforms.py", line 710, in <lambda> transforms.append(Lambda(lambda img: F.adjust_brightness(img, brightness_factor)))' File "/home/ma/anaconda3/envs/ml/lib/python3.6/site-packages/opencv_transforms/functional.py", line 334, in adjust_brightness return cv2.LUT(img, table)[:,:,np.newaxis] cv2.error: OpenCV(4.2.0) /io/opencv/modules/core/src/lut.cpp:368: error: (-215:Assertion failed) (lutcn == cn || lutcn == 1) && _lut.total() == 256 && _lut.isContinuous() && (depth == CV_8U || depth == CV_8S) in function 'LUT'

Looking at these god awful opencv error messages, it might crash because the image is float32?

Gray image error

If the input image is grayscale, I think you should use len(image.shape) == 2 instead of image.shape[2] == 1

OpenCV implementations of adjust_hue and adjust_saturation

Great repo.
But it seems that adjust_hue and adjust_saturation functions are exactly the PIL implementations (PyTorch official implementations). I wonder are there OpenCV implementations for these two functions, since the adjust_hue is rather slow, and adjust_saturation is a little slow when compared with adjust_contrast and adjust_brightness functions.

In my testing cases, given one 224x224x3 input, adjust_contrast takes about 188 µs, adjust_saturation takes about 2.33 ms, and adjust_hue takes about 6.99 ms.

ImportError: cannot import name 'PILLOW_VERSION' from 'PIL'

when I try to use the package:
from opencv_transforms import transforms
I get an error:

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-71-d4430fa4d55e> in <module>
----> 1 from opencv_transforms import transforms

~/.local/lib/python3.8/site-packages/opencv_transforms/transforms.py in <module>
     16 # import opencv_functional as F
     17 import cv2
---> 18 from . import functional as F
     19 
     20 __all__ = [

~/.local/lib/python3.8/site-packages/opencv_transforms/functional.py in <module>
      2 import math
      3 import random
----> 4 from PIL import Image, ImageOps, ImageEnhance, PILLOW_VERSION
      5 try:
      6     import accimage

ImportError: cannot import name 'PILLOW_VERSION' from 'PIL' (/usr/lib64/python3.8/site-packages/PIL/__init__.py)

I've installed the package by:
pip install --user opencv_transforms
I have OpenCV package installed, as well as PIL.

According to Pillow documentation PILLOW_VERSION has been removed, and one is supposed to use __version__ now.
https://pillow.readthedocs.io/en/stable/releasenotes/7.0.0.html#pillow-version-constant

Normalize giving an error

RESIZE = (128, 171)
MEAN = [0.43216, 0.394666, 0.37645]
STD = [0.22803, 0.22145, 0.216989]
CROP_SIZE = 112

transform = Compose(
                [
                    # Resize(RESIZE),
                    # CenterCrop(CROP_SIZE),
                    # ToTensor(),
                    Normalize(MEAN, STD)
                ]
            )

On trying the above code (even without commenting) on google collab I get the following error:
image
Does anyone know what the issue is or how to fix this? Attaching list of pip packages in colab for reference if needed packages.txt

Thanks

Repo

This is not an issue, just to let you know I've commited an update to my fork of your repo, that I combined with https://github.com/YU-Zhiyang/opencv_transforms_torchvision and additional augmentations using OpenCV from one of my repos.

I tried to keep the best from all sources, use only OpenCV or np functions and extend functionality where possible, like the resample flag on RandomRotation.

In case you are interested, here's the repo: https://github.com/victorca25/opencv_transforms should be able to easily pull from it, since I kept it as a fork.

Performance comparison with Pillow-simd

Hi, I really appreciate your effort to share some valuable code.

My only question is whether using OpenCV is faster than Pillow-SIMD. I have read articles from the web that Pillow is surely slower than OpenCV, but referring to pages like https://python-pillow.org/pillow-perf/, there may be some cases when Pillow-SIMD is faster than OpenCV.

Have you tried using Pillow-SIMD instead of plain Pillow? If so, have you done any comparison between OpenCV and Pillow-SIMD on video datasets like Kinetics?

Thanks again.

transforms.Resize Seems to Not Work With 'int'

Not sure if I'm doing something wrong, but this gives an error: "TypeError: 'int' object is not iterable"

transformations = transforms.Compose([
    transforms.Resize(255),
    # some other transforms
])

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.