Code Monkey home page Code Monkey logo

dnglab's Introduction

DNGLab - A camera RAW to DNG file format converter

CI Matrix

Command line tool to convert camera RAW files to Digital Negative Format (DNG).

It is currently in alpha state, so don't expect a polished and bugfree application. Please report bugs in our issue tracker.

Rawler crate is now published to crates.io, but please notice that the API is not yet stable and thus rawler is not following SemVer.

Installation

There are pre-built binary packages for each release which can be downloaded from the asset section under latest release. The .deb packages are for Debian based systems (amd64 and arm64), which can be installed with dpkg -i dnglab_x.x.x_amd64.deb. For non-Debian systems, you can use the single-binary file, for example ./dnglab_linux_x64 convert IMG_1234.CR2 IMG_1234.dng.

Windows is not officially supported, but the release assets contains a dnglab-win-x64_vx.x.x.zip file with pre-built Windows binary. Please be aware that this build is untested.

Build from source

Dnglab is written in Rust, so you can compile it by your own on your target machine. You need the Rust toolchain installed on your machine, see https://rustup.rs/ for that. Once the toolchain is installed, you can simply compile Dnglab with:

git clone https://github.com/dnglab/dnglab.git
cd dnglab
cargo build --release

The dnglab binary is found at ./target/release/dnglab.

Examples

Convert a single file:

dnglab convert IMG_1234.CR3 IMG_1234.DNG

Convert whole directory:

dnglab convert ~/DCIM/100EOS ~/filmrolls/photos-france

Supported cameras and file formats

For a list of supported cameras please see SUPPORTED_CAMERAS.md.

Supported raw file formats

Manufacturer Format Supported Remarks
ARRI ARI ✅ Yes
Canon CR3 ✅ Yes CR3_STATE.md
Canon CR2 ✅ Yes
Canon CRW ✅ Yes
Epson ERF ✅ Yes
Fujifilm RAF ✅ Yes
Hasselblad 3FR ✅ Yes
Kodak KDC ✅ Yes
Kodak DCS ✅ Yes
Kodak DCR ✅ Yes
Leaf IIQ ✅ Yes
Leaf MOS ✅ Yes
Mamiya MEF ✅ Yes
Minolta MRW ✅ Yes
Nikon NEF ✅ Yes
Nikon NRW ✅ Yes
Olympus ORF ✅ Yes
Panasonic/Leica RW2 ✅ Yes
Pentax/Ricoh PEF ✅ Yes
Phase One IIQ ✅ Yes
Samsung SRW ✅ Yes
Sony ARW ✅ Yes
Sony SRF ✅ Yes
Sony SR2 ✅ Yes

Supported DNG features

  • DNG lossless compression (LJPEG-92)

Command line help

convert subcommand

dnglab-convert
Convert raw image(s) into dng format

USAGE:
    dnglab convert [OPTIONS] <INPUT> <OUTPUT>

ARGS:
    <INPUT>     Input file or directory
    <OUTPUT>    Output file or existing directory

OPTIONS:
        --artist <artist>
            Set the artist tag

    -c, --compression <compression>
            Compression for raw image [default: lossless] [possible values: lossless, none]

        --crop <crop>
            DNG default crop [default: best] [possible values: best, activearea, none]

    -d
            turns on debugging mode

        --dng-preview <preview>
            DNG include preview image [default: true]

        --dng-thumbnail <thumbnail>
            DNG include thumbnail image [default: true]

        --embed-raw <embedded>
            Embed the raw file into DNG [default: true]

    -f, --override
            Override existing files

    -h, --help
            Print help information

        --image-index <index>
            Select a specific image index (or 'all') if file is a image container [default: 0]

        --ljpeg92-predictor <predictor>
            LJPEG-92 predictor [default: 1] [possible values: 1, 2, 3, 4, 5, 6, 7]

    -r, --recursive
            Process input directory recursive

    -v
            Print more messages

analyze subcommand

dnglab-analyze
Analyze raw image

USAGE:
    dnglab analyze [OPTIONS] <FILE>

ARGS:
    <FILE>    Input file

OPTIONS:
    -d                          turns on debugging mode
        --full-pixel            Write uncompressed full pixel data to STDOUT
    -h, --help                  Print help information
        --json                  Format metadata as JSON
        --meta                  Write metadata to STDOUT
        --preview-checksum      Write MD5 checksum of preview pixels to STDOUT
        --preview-pixel         Write uncompressed preview pixel data to STDOUT
        --raw-checksum          Write MD5 checksum of raw pixels to STDOUT
        --raw-pixel
        --srgb                  Write sRGB 16-bit TIFF to STDOUT
        --structure             Write file structure to STDOUT
        --summary               Write summary information for file to STDOUT
        --thumbnail-checksum    Write MD5 checksum of thumbnail pixels to STDOUT
        --thumbnail-pixel       Write uncompressed preview pixel data to STDOUT
    -v                          Print more messages
        --yaml                  Format metadata as YAML

With analyze, you can get a full dump of the internal file structure as YAML or JSON. With JSON output, it's possible to filter and transform the data with jq. For example, to get the cfa_layout from the CMP1 box for CR3 files, you can write:

find /cr3samples/ -type f -name "*.CR3" -exec dnglab analyze --structure '{}' --json \; | \
  jq ". | { file: .file.fileName, cfa_layout: .format.cr3.moov.trak[1].mdia.minf.stbl.stsd.craw.cmp1.cfa_layout}"

The output is:

{
  "file": "Canon EOS 90D_CRAW_ISO_250_nocrop_nodual.CR3",
  "cfa_layout": 1
}
{
  "file": "Canon EOS 90D_CRAW_ISO_100_nocrop_nodual.CR3",
  "cfa_layout": 1
}

extract subcommand

dnglab-extract
Extract embedded original Raw from DNG

USAGE:
    dnglab extract [OPTIONS] <FILE> <INPUT> <OUTPUT>

ARGS:
    <FILE>      Input file
    <INPUT>     Input file or directory
    <OUTPUT>    Output file or existing directory

OPTIONS:
    -d                  turns on debugging mode
    -f, --override      Override existing files
    -h, --help          Print help information
    -r, --recursive     Process input directory recursive
        --skipchecks    Skip integrity checks
    -v                  Print more messages

makedng subcommand

Lowlevel command to make a DNG file

Usage: dnglab makedng [OPTIONS] --input <INPUT>...

Options:
  -d...
          turns on debugging mode

  -o, --output <OUTPUT>
          Output DNG file path

  -i, --input <INPUT>...
          Input files to merge into a single DNG file. Usually only a single input file is used.
          If multiple input files are given, --map should be used to specifiy how to interpret each intput file.

  -v
          Print more messages

      --map <map>...
          When multiple input files given, each input file should be mapped to a specific type of data.
          First input file starts with index 0. Possible types are 'raw', 'preview', 'thumbnail', 'exif', 'xmp'.
          Input files which are not mapped are ignored.

          [default: 0:raw 0:preview 0:thumbnail 0:exif 0:xmp]

      --dng-backward-version <version>
          DNG specification version

          [default: 1.4]
          [possible values: 1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.6]

      --colorimetric-reference <reference>
          Reference for XYZ values

          [default: scene]
          [possible values: scene, output]

      --unique-camera-model <id>
          Unique camera model

      --artist <artist>
          Set the Artist tag

      --make <make>
          Set the Make tag

      --model <model>
          Set the Model tag

      --matrix1 <MATRIX>
          Matrix 1

          [possible values: XYZ_sRGB_D50, XYZ_sRGB_D65, XYZ_AdobeRGB_D50, XYZ_AdobeRGB_D65, "custom 3x3 matrix (comma seperated)"]

      --matrix2 <MATRIX>
          Matrix 2

          [possible values: XYZ_sRGB_D50, XYZ_sRGB_D65, XYZ_AdobeRGB_D50, XYZ_AdobeRGB_D65, "custom 3x3 matrix (comma seperated)"]

      --matrix3 <MATRIX>
          Matrix 3

          [possible values: XYZ_sRGB_D50, XYZ_sRGB_D65, XYZ_AdobeRGB_D50, XYZ_AdobeRGB_D65, "custom 3x3 matrix (comma seperated)"]

      --illuminant1 <ILLUMINANT>
          Illuminant 1

          [possible values: Unknown, A, B, C, D50, D55, D65, D75]

      --illuminant2 <ILLUMINANT>
          Illuminant 2

          [possible values: Unknown, A, B, C, D50, D55, D65, D75]

      --illuminant3 <ILLUMINANT>
          Illuminant 3

          [possible values: Unknown, A, B, C, D50, D55, D65, D75]

      --linearization <TABLE>
          Linearization table

          [possible values: 8bit_sRGB, 8bit_sRGB_invert, 16bit_sRGB, 16bit_sRGB_invert, 8bit_gamma1.8, 8bit_gamma1.8_invert, 8bit_gamma2.0, 8bit_gamma2.0_invert, 8bit_gamma2.2, 8bit_gamma2.2_invert, 8bit_gamma2.4, 8bit_gamma2.4_invert, 16bit_gamma1.8, 16bit_gamma1.8_invert, 16bit_gamma2.0, 16bit_gamma2.0_invert, 16bit_gamma2.2, 16bit_gamma2.2_invert, 16bit_gamma2.4, 16bit_gamma2.4_invert, "custom table (comma seperated)"]

      --wb <R,G,B>
          Whitebalance as-shot

      --white-xy <x,y>
          Whitebalance as-shot encoded as xy chromaticity coordinates

          [possible values: D50, D65, "custom x,y value (comma seperated)"]

  -f, --override
          Override existing files

  -h, --help
          Print help (see a summary with '-h')

Contribute samples

Please see our guide: CONTRIBUTE_SAMPLES.md.

FAQ

Why a DNG tool if there is already something from Adobe?

The DNG converter from Adobe is free (at cost), but not free in terms of free software. Nobody can add or fix camera support except of Adobe. And it has no support for Linux. That's why I've started writing my own little DNG swiss army knife.

Why should I use DNG instead of RAW?

Never ask. If you need DNG you will know.

Will camera/format (...) be added?

Well, depends on developer resources.

Is a GUI in planning?

Yes, DNGLab should get a GUI in near future.

How can I donate to this project?

I don't have any sponsoring or donation account like Patreon or Paypal. If you want to surprise me, please have a look at my Amazon wish list.

Credits

Special thanks goes to:

Without the support and engagement from these people the development of dnglab would not have been possible.

dnglab's People

Contributors

agoode avatar cytrinox avatar dependabot[bot] avatar eyeplum avatar johannesvollmer avatar kdm9 avatar luckydye avatar manouchehri avatar matt-maguire avatar maximumstock avatar michalskalski avatar paolobarbolini avatar pedrocr avatar pfjason avatar pick2510 avatar powturns avatar runfalk avatar schoenenberg avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

dnglab's Issues

Use specular white level instead of normal white level for CR3

For Canon CR2 and CR3 files, there exists two white levels in metadata:

Per Channel Black Level         : 2048 2048 2048 2048
Normal White Level              : 13535
Specular White Level            : 14888

dnglab uses the Normal White Level while Adobe DNG Converter uses Specular White Level. This should be changed.

The White and Black level only depends on the choosen ISO value.

Performance improvements

I assume most of the math done here can be speed up, for example with SIMD or better multi threading.

CR3 GPS information is invalid and must be fixed in DNG

dnglab copies the CMT4 box (TIFF structure with EXIF GPS info) into DNG. But CR3 has invalid values for

  • Exif.GPSInfo.GPSLatitude
  • Exif.GPSInfo.GPSLongitude
  • Exif.GPSInfo.GPSTimeStamp

These should be arrays of 3 rationals, but each tag only contains two rationals. The data structure is correct, but programs like darktable can't decode these tags.

This should be fixed by checking the array length and extend the value to 3 rationals if needed.

Add support for Canon PowerShot SX70 HS

Hello! Thank you so much for this tool! I was really excited to see a way I might be able to bring these CR3 files into Darktable on GNU/Linux, but it seems that dnglab doesn't quite support the camera which created the files. What's involved in adding support for different versions of the PowerShot cameras, and more specifically the SX70 HS?

I'd be happy to help, if you can provide some guidance.

Worth noting, I've verified that https://raw.pixls.us/ has images for this camera. If more are needed, I can provide them.

Output from running

~/projects/third-party/dnglab on main at 18:08:04 
❯ ./target/release/dnglab convert ~/sharebox/IMG_0694.CR3 test.dng
thread 'tokio-runtime-worker' panicked at 'called `Result::unwrap()` on an `Err` value: DecoderFail("File is unsupported: Couldn't find camera \"Canon\" \"Canon PowerShot SX70 HS\" mode \"\".\nPlease submit samples at https://raw.pixls.us/")', bin/dnglab/src/dnggen.rs:96:78
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: JoinError::Panic(...)', bin/dnglab/src/dnggen.rs:100:4

Request for arm64 package release

Could you please include an arm64 build in your releases?! Sheer lack of compiling knowledge is the reason I'm asking ;) BTW I'm thrilled by the project and hope to see it grow! Thanks!

Add missing lenses

Missing Lenses:
Sigma 18-300mm F3.5-6.3 DC MACRO OS HSM | Contemporary 014
Canon EF-S24mm f/2.8 STM

I don't know enough rust to find an answer, but what is the purpose of the toml table full of lenses? Why not just copy the information from existing exif data? Are the other parameters used somewhere I haven't seen in the code?

Unknown lens model: 'EF50mm f/1.8 STM'.

Hi, I'm using the Canon 90D with mainly two lens: the canon EF 50mm F1.8 and the canon EF 85mm F1.8.
I got the error message "Unknown lens model: 'EF50mm f/1.8 STM'. Please open an issue at https://github.com/dnglab/dnglab/issues and provide the RAW file".
I will provide as well shot with the EF 85mm F1.8 if I get the same issue.
Thanks a lot for your work, program is great and hugely usefull while waiting for native CR3 support in darktable.

I got a github error saying that the CR3 format is not supported, how can I give you the .CR3 files ?
I try to ZIP the CR3 file but got the error "is not included in the list".

Best regard,

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.