pedrocr / rawloader Goto Github PK
View Code? Open in Web Editor NEWrust library to extract the raw data and some metadata from digital camera images
License: GNU Lesser General Public License v2.1
rust library to extract the raw data and some metadata from digital camera images
License: GNU Lesser General Public License v2.1
Hi I would like to make a pull request for iphone pro raw DNG support (which is 3-component ljpeg, using predictor 7).
I have already implemented the decoding, very simple, but the design is not finalised. I see a few ways to proceed:
decompressors.rs
, just for decoding 3-component predictor 7 data.
decode_ljpeg_(1|2|3|4)components
decode_ljpeg_(1|2|3|4)components
, this will be slower.decode_ljpeg_(1|2|3|4)components
in to one generic function - like fn decode_ljpeg<const N: usize>(...)
, where component number is a compile time parameter. and then have separate functions for each predictor:fn decode_ljpeg_predictor_1<const N: usize>(...)
fn decode_ljpeg_predictor_7<const N: usize>(...)
Please let me know what would be best as a pull request!
Also, the data seems to be BGR not RGB, but I guess that can be considered later Nevermind, that was an issue with how I was saving test files. The data is indeed RGB.
I've found this project have not updating for years, so will this project keep going?
in the example of the README.md file i had to change line
let pixlow = (pix&0x0f) as u8;
into
let pixlow = (pix&0xff) as u8;
else i got dramatic posterization. i don't know whether this is a typo or has to do with the bit depth of my camera (14-Bit, arw format, Sony IlCE-7SM2).
Forgive a noob question, I'm fairly new to rust. Looking at the source, I don't think it is, but figured I would ask. Is it possible to access raw image data without full loading the image into ram? (for example to load the image in tiles or by scanline or something, at least for .nef, .cr2, and .awr). I'm stacking raw files in different ways, and I can only load part of them in ram at once. (due to limited ram) Any help would be appreciated. If it's not possible, is it something that could be done with an easy modification? (I could probably fund some development if its something thats not a huge change) Thanks!
Thanks for the library!
To learn Rust I'm trying to re-implement the magiclantern cr2hdr tool. One of the methods attempts to calculate the standard deviation of the black noise in the image. It would be useful to be able to access the values of blackareah and blackareav in a loaded raw image for this purpose (perhaps similar to the crops
property).
Any advice on how to do this, or why this is a bad idea are appreciated. I can also send you a pull request that will expose it too!
Hi,
So glad you made this library, thank you!
Can you please add more examples, besides ppm? Maybe also a link to a good learning resource.
Some image formats contain orientation information in the Makernote
tag. Here's one example where the normal Orientation
tag doesn't match the one in the Makernote
:
https://raw.pixls.us/getfile.php/346/nice/Sony%20-%20SLT-A99%20-%2014bit%20(3:2).ARW
https://raw.pixls.us/getfile.php/346/exif/RAW_SONY_SLT-A99V.ARW.exif.txt
Exiftool outputs the following data where 270 CW is correct:
0x0112 Orientation : Rotate 180
0x0028 Camera Orientation : Rotate 270 CW
The CameraOrientation
tag for this particular image is within Tag9400a (0x9400)
in the Makernote. According to http://www.sno.phy.queensu.ca/~phil/exiftool/TagNames/Sony.html there are many different places where orientation can be stored.
Hi Pedro,
I would like to fix some clippy
-warnings in the code, if this is ok for you.
Best regards,
Maximilian
RawLoaderError { msg: "Couldn't find tag Tag::Make" }
Happens during rawloader::decode_file(path).unwrap();
, with dng
file.
I noticed while checking a project with cargo-deny
that the spdx license expression used by this crate in Cargo.toml
is deprecated.
See https://spdx.org/licenses/LGPL-2.1.html
LGPL-2.1-only, or LGPL-2.1-or-later should be used instead.
I usualy idle freenode and thought it would be a good idea to have an IRC channel for implementation discussion so I registered #rawloader
. Some of the issues with my pull request (#2) might have been avoided if I discussed them with you beforehand.
When an image file contains an alias of the camera model, rather than the official camera model, there is no way to relate back to the official model.
For example, with a T3i, I would expect to get back the official model of Canon EOS 600D
(from data/cameras/canon/600d.toml
) when calling camera.model
. However, instead I get back the localized model Canon EOS REBEL T3i
.
This can introduce difficulties when you want to branch behaviour based on the camera model as each localized case must be handled, rather than a single official model case.
Proposal would be to modify RawImage.{model, clean_model}
to return the official model name, and add two new properties RawImage.{localized_model, clean_localized_model}
that return the localized model name.
There's a ljpeg decoder here, but it's not exposed and not used in any way yet
https://github.com/pedrocr/rawloader/blob/master/src/decoders/mod.rs#L73
Is it possible to make it public? This decoder is unique because at the moment it's the only library in pure rust able to decode lossless jpeg. As a result, it's usable in webassembly context.
In addition, there's a decode_ljpeg_2components
, but no decode_ljpeg_1components
. I made a simple version of decode_ljpeg_1components
function, would you mind reviewing it if I submit a pr?
The rebel line of canon cameras (T*i) have duplicate camera definition files - one for the official model and another for the localized model name.
I don't think the localized model name file is necessary as the camera is populated from the alias section of the official model file.
Eg: a T2i camera has its properties defined in T2i.toml
and 550d.toml
Hello! I tried the sample usage on in the readme with this file from raw.pixls.us and got a panic on decode. Here's the backtrace from rawloader which looks suspiciously like an underflow:
thread 'main' panicked at 'index out of bounds: the len is 4096 but the index is 65535', /home/genbyte/src/rawloader/src/decoders/basics.rs:127:30
stack backtrace:
0: backtrace::backtrace::libunwind::trace
at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.46/src/backtrace/libunwind.rs:86
1: backtrace::backtrace::trace_unsynchronized
at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.46/src/backtrace/mod.rs:66
2: std::sys_common::backtrace::_print_fmt
at src/libstd/sys_common/backtrace.rs:78
3: <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt
at src/libstd/sys_common/backtrace.rs:59
4: core::fmt::write
at src/libcore/fmt/mod.rs:1076
5: std::io::Write::write_fmt
at src/libstd/io/mod.rs:1537
6: std::sys_common::backtrace::_print
at src/libstd/sys_common/backtrace.rs:62
7: std::sys_common::backtrace::print
at src/libstd/sys_common/backtrace.rs:49
8: std::panicking::default_hook::{{closure}}
at src/libstd/panicking.rs:198
9: std::panicking::default_hook
at src/libstd/panicking.rs:218
10: std::panicking::rust_panic_with_hook
at src/libstd/panicking.rs:486
11: rust_begin_unwind
at src/libstd/panicking.rs:388
12: core::panicking::panic_fmt
at src/libcore/panicking.rs:101
13: core::panicking::panic_bounds_check
at src/libcore/panicking.rs:73
14: <usize as core::slice::SliceIndex<[T]>>::index
at /home/genbyte/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libcore/slice/mod.rs:2872
15: core::slice::<impl core::ops::index::Index<I> for [T]>::index
at /home/genbyte/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libcore/slice/mod.rs:2732
16: <alloc::vec::Vec<T> as core::ops::index::Index<I>>::index
at /home/genbyte/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/liballoc/vec.rs:1942
17: rawloader::decoders::basics::LookupTable::dither
at /home/genbyte/src/rawloader/src/decoders/basics.rs:127
18: rawloader::decoders::nef::NefDecoder::do_decode
at /home/genbyte/src/rawloader/src/decoders/nef.rs:311
19: rawloader::decoders::nef::NefDecoder::decode_compressed
at /home/genbyte/src/rawloader/src/decoders/nef.rs:237
20: <rawloader::decoders::nef::NefDecoder as rawloader::decoders::Decoder>::image
at /home/genbyte/src/rawloader/src/decoders/nef.rs:140
21: rawloader::decoders::RawLoader::decode_unsafe
at /home/genbyte/src/rawloader/src/decoders/mod.rs:522
22: rawloader::decoders::RawLoader::decode::{{closure}}
at /home/genbyte/src/rawloader/src/decoders/mod.rs:530
23: std::panicking::try::do_call
at /home/genbyte/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/panicking.rs:297
24: __rust_try
25: std::panicking::try
at /home/genbyte/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/panicking.rs:274
26: std::panic::catch_unwind
at /home/genbyte/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/panic.rs:394
27: rawloader::decoders::RawLoader::decode
at /home/genbyte/src/rawloader/src/decoders/mod.rs:529
28: rawloader::decoders::RawLoader::decode_file
at /home/genbyte/src/rawloader/src/decoders/mod.rs:544
29: rawloader::decode_file
at /home/genbyte/src/rawloader/src/lib.rs:104
30: easyraw::main
at src/main.rs:14
31: std::rt::lang_start::{{closure}}
at /home/genbyte/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/rt.rs:67
32: std::rt::lang_start_internal::{{closure}}
at src/libstd/rt.rs:52
33: std::panicking::try::do_call
at src/libstd/panicking.rs:297
34: std::panicking::try
at src/libstd/panicking.rs:274
35: std::panic::catch_unwind
at src/libstd/panic.rs:394
36: std::rt::lang_start_internal
at src/libstd/rt.rs:51
37: std::rt::lang_start
at /home/genbyte/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/rt.rs:67
38: main
39: __libc_start_main
40: _start
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
Hi, I mostly wanted to thank you for having done this raw decoder in Rust! (don't hesitate to close the issue)
In case you have a little more time I'd love to know how you managed to do the NEF decoder. In particular, I've been trying to understand how to decompress the strips without much success. I've detailed my issue in this post where someone redirected me here.
I've found the source code of dcraw very intimidating and not easy to understand for me (I'm not coding much in C also). Did you have other sources of information to help you? I haven't found much on the Web, but maybe just as with this library I didn't search for the right things.
Regarding the licensing of this code, are you set on LGPL?
Best regards
heya,
i'm using rawloader
through a thin c-bindings layer (which you can find here).
when running this code on a sequence of dng files, i get random tile decoding issues which manifest themselves like so:
or so:
jumping around the images. looks like a threading issue to me (?). is this a bug? am i calling it wrong? do i have to wait before reading out the buffer after calling rawloader::decode_file()
?
Maybe I'm missing something but I do not understand yet how I can determine the final/real brightness.
For example, if a camera only a bit-depth of 12 then 0x0fff is max brightness. If a camera uses 14 bit then 0x3ff is the max brightness of a pixel. How can I find out the final/real brightness with what's given right now by the library?
First of all - awesome library, thanks for putting it out there!
I'm using rawloader to uniquely identify raw files, even when some of the attributes have been written to the file (e.g. an jpg
preview has been updated). I do so by reading the raw image buffer and creating a sha256 hash out of it.
Works perfectly fine for my CR2
files, but it looks like CR3
is not supported yet.
You can find sample files on https://raw.pixls.us by searching for "EOS RP" or "EOS M50".
Let me know if there's anything I can do to assist you here!
Edit: A description of the file format can be found at https://github.com/lclevy/canon_cr3.
Hi! Is there any hope that shutter speed, ISO and aperture will appear in RawImage
struct? How difficult is it to implement?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.