Code Monkey home page Code Monkey logo

debayer's Introduction

Debayer

Debayer is a commandline tool to convert camera raw images into scene-linear exr.

Dependencies

The only required dependency is oiiotool. However other "debayer engines" are also supported.

  • OpenImageIO - oiiotool is used for converting debayered tif images to exr.
  • Debayer Engines
    • RawTherapee - Powerful raw development software used to decode raw images. High quality, good selection of debayer algorithms, and more advanced raw processing like chromatic aberration removal.
    • LibRaw - dcraw_emu commandline utility included with LibRaw. Optional alternative for debayer. Simple, fast and effective.
    • Darktable - Uses darktable-cli plus an xmp config to process.
    • vkdt - uses vkdt-cli to debayer. Pretty experimental still. Uses Vulkan for image processing. Stupidly fast. Pretty limited.

Configuration

You need to configure a few things in the debayer python file. At the top are some variables you need to customize. Described below are the important ones and what they do.

First you need to choose which debayer engine you will use. DEBAYER_ENGINES lists all possibilities. DEBAYER_ENGINE is where you set which debayer software will be used.

# Debayer engine options
DEBAYER_ENGINES = ['rt', 'art', 'dc', 'dcrcd', 'dt', 'vkdt', 'oiio'] 
# Debayer engine to use
DEBAYER_ENGINE = 'rt'

This configuration will use RawTherapee as the debayer engine.

IMPORTANT: If you are using RawTherapee 5.8 or lower, you need to install an output icc profile.

Next you need to set the paths to the binary executable files. oiiotool is required. You only need to specify paths for the debayer engine you will use.

# Binary executable locations
OIIO_BIN    = '/usr/bin/oiiotool'
RT_BIN      = '/usr/bin/rawtherapee-cli'
ART_BIN     = '/usr/bin/ART-cli'
DCRAW_BIN   = '/usr/bin/dcraw_emu'
DT_BIN      = '/usr/bin/darktable-cli'
VKDT_BIN    = '/opt/vkdt/vkdt/vkdt-cli'

You also need to specify a temp directory for debayer to use as a location for intermediate files (don't worry they are cleaned up automatically). It speeds things up if this is on a fast local drive.

CACHE_DIR = '/var/tmp/debayer'

You can set other options as well: THREADS = 2 - Number of simultaneous jobs (Useful if using a single-threaded debayer engine like dcraw).

EXR_COMPRESSION = 'dwaa:15' - Type of compression to use for output exr files.

DEFAULT_EXPOSURE = '4.0' - Default multiply value to get from raw linear to scene-linear.

RT_DEFAULT_PROFILE = '' - Path to the default RawTherapee pp3 preset to use. There is one included and set by default, but you can override this path if desired. There are similar variables for the other debayer engines that need config files as well.

Debayer Engine Configuration

Here is some more information about how to configure custom debayer settings for each of the debayer engines. I've included quite a bit of extra information here about the decisions I've made and why.

Usage

Using debayer is very simple if you are familiar with commandline utilities. In the simplest possible form, you could write

debayer rawfile.cr2

This will process the raw file into the directory you are currently in.

If you want to specify a custom directory you could do

debayer rawfile.cr2 -o /path/to/output_dir

You can also process an entire source directory recursively. For example say you have this source directory structure:

/media/footage/20211012
└── dng
    ├── M22-1558
    │   ├── M22-1558.000000.dng
    │   ├── M22-1558.000001.dng
    │   ├── M22-1558.000002.dng
    │   ├── M22-1558.000003.dng
    │   ├── M22-1558.000004.dng
    └── M22-1600
        ├── M22-1600_000000.dng
        ├── M22-1600_000001.dng
        ├── M22-1600_000002.dng
        ├── M22-1600_000003.dng
        └── M22-1600_000004.dng

Say you want to recursively process all raw files inside /media/footage/20211012/dng, and output the results, including subfolders, into /media/footage/20211012/exr. You could run

debayer /media/footage/20211012/dng -o /media/footage/20211012/exr

# or alternatively, you can use relative paths
cd /media/footage/20211012
debayer dng -o exr

Commandline Options

$ debayer -h
usage: debayer [-h] [-o OUTPUT] [-w] [-p PROFILE] [-en ENGINE] [-ca]
               [-r RESIZE] [-e EXPOSURE] [-j JOBS] [-f FILTER]
               input_paths [input_paths ...]

Debayer is a commandline tool to convert camera raw images into scene-linear
exr.

positional arguments:
  input_paths           Source(s) to process. Can be one or more images or
                        directories containing images.

optional arguments:
  -h, --help            show this help message and exit
  -o OUTPUT, --output OUTPUT
                        Output directory. If not specified, the current
                        directory will be used.
  -w, --overwrite       Overwrite existing output files.
  -p PROFILE, --profile PROFILE
                        Optional override to specify a custom profile to use.
                        If none, the default config file will be used. Could
                        be a pp3 file for rawtherapee, xmp for darktable, or
                        cfg for vkdt.
  -en ENGINE, --engine ENGINE
                        Debayer engine to use: If none, the DEBAYER_ENGINE
                        variable in will be used. one of: 'rt', 'art', 'dc',
                        'dcrcd', 'dt', 'vkdt', 'oiio'
  -ca, --aberration     Remove chromatic aberration (rawtherapee or ART only)
  -r RESIZE, --resize RESIZE
                        Apply a resize to the output image. Aspect ratio is
                        not preserved if both width and height are specified.
                        <width>x<height> Or preserve aspect ratio if w or h =
                        0: e.g. "1920x0" Can also be a percentage like
                        "50%"(For Magic Lantern 1x3 anamorphic) 1920x2340 ->
                        5760x2340 -> 2880x1170 (ML 1x3 2.35) - 1808x2300 ->
                        (unsqueeze) 5424x2300 -> (0.5) 2712x1150 Or 1280x2160
                        (1x3) -> 3840x2160 -> 2560x1440 -> 1920x1080
  -e EXPOSURE, --exposure EXPOSURE
                        Raw to scene-linear exposure adjustment. Default is
                        4.0
  -j JOBS, --jobs JOBS  Number of simultaneous jobs.
  -f FILTER, --filter FILTER
                        Include only files that match regex. Can be comma
                        separated list.

Exposure

One important parameter when debayering to scene-linear is how much to expose up with the -e parameter. Camera raw files are usually stored in an integer data format, which means the data is contained in a 0-1 container. How many bits of precision that data has within that container varies with the camera and sensor.

To get this 0-1 data range into sensible scene-linear data, we need to expose up the raw data so that an 18% grey diffuse reflector will live at a scene-linear value of around 0.18. How much we need to expose up will of course depend on how the image was exposed in the camera. This also determines how much headroom you will have in highlights. A default value of 4.0 is provided as a starting point, but you will probably want to customize this depending on your source raw images.

Custom Profile

You can specify a custom config file for your debayer engine with -p path/to/file. This is useful if you need to customize whitebalance or sharpening settings for example.

A default config is included in the repo for most debayer engines is used by default there is a -p switch enabled.

Debayer Algorithms

If you are curious about the different available debayer algorithms in open software I have put together a big set of comparison images in jpg and exr available at this mega.nz link.

The images are processed with dcraw_emu for AHD, AAHD, DHT and VNG, and with rawtherapee-cli for RCD, DCB and AMaZE. The test images are a selection of 256x128 pixel crops of interesting image regions for judging debayer quality. My subjective ranking would be

RCD > DCB > AMaZE > DHT > AAHD > AHD > VNG

If you're curious, I wrote a more in-depth discussion.

Installation

OpenImageIO

oiiotool can be installed with common pacakge managers on all platforms. See this documentation for help.

Rawtherapee

Packages are provided for all platforms. However if you are on linux, the appimage won't give you access to the rawtherapee-cli commandline utility. You will need to install or compile it (or extract the contents of the appimage with ./Rawtherapee-5.8.Appimage --appimage-extract ). Note if you are on Centos 7, the Appimage will not work because it was compiled with a newer version of GLIBC. There is however a copr for many fedora and rhel distros including el7 here.

IMPORTANT: For Rawtherapee 5.8 and below, you need to install a custom icc output profile in order to work around a posterization bug.

LibRaw

LibRaw provides packages for Windows and Mac, and for Linux it is easy to compile with minimal dependencies. I would recommend compiling from master to get the latest raw formats like Canon CR3.

debayer's People

Contributors

jedypod avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

debayer's Issues

请问使用debayer需要安装哪些

请问使用debayer需要安装什么,比如设置DEBAYER_ENGINE = 'dc',需要安装哪些?如何安装?
libraw安装编译之后并没有起作用,oiio安装不了

why is `mtx_XYZ_to_ACEScg` not same as ACES offical transform matrix?

I am confused about the color transform matrix.
In debayer file line179
mtx_XYZ_to_ACEScg = '1.65885437,-0.31185693,-0.24315603,-0.66228336,1.61220002,0.01585913,0.01148057,-0.00923633,0.91668653'

This has slight differences with the aces offical transform matrix:
R_out = 1.6410233797 * R_in + -0.3248032942 * G_in + -0.2364246952 * B_in;
G_out = -0.6636628587 * R_in + 1.6153315917 * G_in + 0.0167563477 * B_in;
B_out = 0.0117218943 * R_in + -0.0082844420 * G_in + 0.9883948585 * B_in;

In comments, it says "D65-aligned XYZ to ACEScg with CAT02 CAT. Works with dcraw"
What is the meaning of D65-aligned XYZ? Is the XYZ output of dcraw not standard?

I am trying to read raw image into ACES Ap0 color space. I find in libraw color matrix, there is a comment:
Coffin's values (commented out) are not adapted from ACES "D60-like" WP to D65
I did some tests with the libraw color matrix:

  1. srgb->xyz(use LibRaw_constants::xyz_rgb)->aces(use offical aces xyz2Ap0)
  2. srgb->aces(use LibRaw_constants::aces_rgb)
  3. srgb->aces(use Coffin's values which are commented out)
    the results of 1)and 3)are same, which are different from 2).

So, is the result of libraw xyz standard? At least I found the LibRaw_constants::xyz_rgb matrix is standard matrix for sRGB->XYZ.

Thanks for your reply!

MacOS

Ok so first a few "fixes"for the readme.md
on MacOS BigSur I have to install pip using get-pip.py it does not work via homebrew:

brew install openimageio dcraw python exiftool

curl -O https://bootstrap.pypa.io/pip/2.7/get-pip.py

python get-pip.py

pip install PyYAML

so far so good if I launch ./debayer I get the help printout, but when I run ./debayer input_paths /Volumes/raid/HDri/testJEDY

I end up with this:

Traceback (most recent call last):
  File "./debayer", line 709, in <module>
    debayer = Debayer()
  File "./debayer", line 61, in __init__
    self.success = self.validate()
  File "./debayer", line 374, in validate
    if not os.path.exists(self.ocioconfig):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/genericpath.py", line 26, in exists
    os.stat(path)
TypeError: coercing to Unicode: need string or buffer, NoneType found

Any Idea?

Debayer command not found

Hi,

I really want to try this tool. I finally figured out how to get everything installed on my Mac using brew and pip. I get Debayer command not found when I try to run it in terminal. Not sure where to go from here. Any help is appreciated.

License

Hello,
could you please add the license ?
I am a supporter of free and opens source software :)

thank you !

Andrea

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.