cykooz / fast_image_resize Goto Github PK

Is there an equivalent to OpenCV's INTER_AREA algorithm in fast_image_resize?

See comparison here: https://gist.github.com/georgeblck/e3e0274d725c858ba98b1c36c14e2835

GDB Backtrace:

(gdb) bt
#0  0x0000737952d91d4a in do_cos (dx=6.123233995736766e-17, x=<optimized out>) at ../sysdeps/ieee754/dbl-64/s_sin.c:107
#1  __sin_fma (x=1.3089969389957472) at ../sysdeps/ieee754/dbl-64/s_sin.c:229
#2  0x000064776f8ab601 in core::ops::function::Fn::call ()
#3  0x000064776f8bd552 in fast_image_resize::convolution::precompute_coefficients ()
#4  0x000064776f8af97b in fast_image_resize::resizer::resample_convolution ()
#5  0x000064776f8adc0c in fast_image_resize::resizer::Resizer::resize ()
#6  0x000064776f5176e2 in blackbox_core::capture::preview::resize_frame ()
#7  0x000064776f514b01 in blackbox_core::capture::preview::preview_manager_thread ()
#8  0x000064776f54c225 in std::sys_common::backtrace::__rust_begin_short_backtrace ()
#9  0x000064776f6b0fc9 in core::ops::function::FnOnce::call_once{{vtable.shim}} ()
#10 0x000064776fedecb5 in alloc::boxed::{impl#47}::call_once<(), dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global> () at library/alloc/src/boxed.rs:2007
#11 alloc::boxed::{impl#47}::call_once<(), alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global>, alloc::alloc::Global> () at library/alloc/src/boxed.rs:2007
#12 std::sys::unix::thread::{impl#2}::new::thread_start () at library/std/src/sys/unix/thread.rs:108
#13 0x0000737951294ac3 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#14 0x0000737951326850 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

This issue is incredibly rare; my application does this roughly 40 times per second and it takes roughly an hour for it to happen, but it does happen unconditionally given enough time. Let me know if there's any information that would be useful.

After
Before(Original image)

First I'd like to say this crate definitely fills a need for Rust. Thank you so much.

Is there any intention of adding wasm32 support?

Hi,
I know you have already implemented a resize_image_with_cropping method but do you have a good way to resize_with_pad? I.e. resize but maintain input image proportions by applying a symmetric padding to the output image (aka. letterbox).

Thanks

I am using this library to resize video frames from Gstreamer. Gstreamer sometimes creates frame buffers which are oversized.

When calling ImageView::from_buffer before resizing such frames, fast_image_resize returns error ImageBufferError::InvalidBufferSize. I expected that an oversize buffer would not cause an error.

For comparison, the image crate's "equivalent" type ImageBuffer::<Luma, &[u8]> does accept buffers which are oversized.

(It is easy to slice the buffer down to the right size before passing it into this crate, but it would be nice if this was not necessary)

(P.S. This crate is very fast :) -- it is nice to be able to resize frames in rust with competetive speed to resizing inside the Gstreamer pipeline)

cool crate 👍 . I was trying it out, but couldn't get it to work.

I believe latest version was just released about an hour ago, I tried a prev version as well, kept getting the same error. am I missing some dependency?

My entire code is just

fast_image_resize = "3.0.2"

in Cargo.toml and importing fr nothing else

I can't find anything in the documentation (or the source) that indicates that there is any support for conversion of data into a linear color space during resizing.

This is required to get correct results when downscaling images that are not in a linear color space. I.e. any 24/32bit PNG, 24bit JPG etc. one would find in the wild.

While this linearization could be done outside the resizing code (and thus this crate) for the supported i32 and f32 types, for the u8 case this is not possible. If the crate user linearizes the input data from some color space u8 to linear u8 there is too much loss of information – banding/posterization ensues. See example images in above linked article.
For this reason the linearization before the filtering and de-linearization after has to be part of the resizing process itself for u8 grayscale or [u8; 3] RGB.

Would it be possible to add support for this? Basically u8 data needs to be linearized to at least u16, better u32 or f32, resized, de-linearized and stored back into u8. And the color space can't be baked into the crate. I.e. you can not just assume sRGB or gamma 2.2 if you want to do this properly.

Possibly through feature-flag gated support for e.g.:

• user-specified closures for transforming into/from linear color space during filtering
• a crate like colstodian

In case of closures the closure should have access to all components of a color so it can convert to/from other spaces, e.g. RGB to Lab, if the user wishes the resize to happen in that space.

Several of the functions in simd_utils.rs are doing UB in unsafe blocks, eg

pub unsafe fn ptr_i16_to_256set1_epi32(buf: &[i16], index: usize) -> __m256i {
    _mm256_set1_epi32(*(buf.get_unchecked(index..).as_ptr() as *const i32))
}

is taking a slice of [i16] which is only going to be 2-byte aligned and casting it to an *i32 which has to be 4-byte aligned. rust-lang/rust@22a7a19 Was recently added to rustc to catch this kind of UB at runtime, and it's firing in the test suite:

---- resize_to_same_width_after_cropping stdout ----
thread 'resize_to_same_width_after_cropping' panicked at 'misaligned pointer dereference: address must be a multiple of 0x4 but is 0x7fca6400346a', src/convolution/vertical_u8/native.rs:49:21
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- upscale_u8 stdout ----
thread 'upscale_u8' panicked at 'misaligned pointer dereference: address must be a multiple of 0x4 but is 0x7fca4402ec7e', src/simd_utils.rs:61:20

---- downscale_u8x3 stdout ----
thread 'downscale_u8x3' panicked at 'misaligned pointer dereference: address must be a multiple of 0x4 but is 0x7fca680aeb4d', src/convolution/vertical_u8/native.rs:49:21

---- downscale_u8 stdout ----
thread 'downscale_u8' panicked at 'misaligned pointer dereference: address must be a multiple of 0x4 but is 0x7fca74ffbd7f', src/convolution/vertical_u8/native.rs:49:21

---- downscale_u8x2 stdout ----
thread 'downscale_u8x2' panicked at 'misaligned pointer dereference: address must be a multiple of 0x4 but is 0x7fca6df64d9e', src/convolution/vertical_u8/native.rs:49:21

---- upscale_u8x2 stdout ----
thread 'upscale_u8x2' panicked at 'misaligned pointer dereference: address must be a multiple of 0x8 but is 0x7fca480a404e', src/simd_utils.rs:71:21

---- downscale_u8x4 stdout ----
thread 'downscale_u8x4' panicked at 'misaligned pointer dereference: address must be a multiple of 0x4 but is 0x7fca7001c82a', src/simd_utils.rs:61:20

---- upscale_u8x3 stdout ----
thread 'upscale_u8x3' panicked at 'misaligned pointer dereference: address must be a multiple of 0x4 but is 0x7fca380a100e', src/simd_utils.rs:61:20

---- upscale_u8x4 stdout ----
thread 'upscale_u8x4' panicked at 'misaligned pointer dereference: address must be a multiple of 0x4 but is 0x7fca400a404e', src/simd_utils.rs:61:20

easy to reproduce with cargo +nightly test.

Thanks for making this library! I'd like to try to use it for a Vec<f32> source. I'm fairly new to rust and can seem to create the appropriate ImageView or Image structs. Is there an example of the resizing for F32? Thanks in advance!

impl Matrix<f32> {
    pub(crate) fn to_image_view(&self) -> ?? {
        let (height, width) = self.shape();
        let img = fr::ImageView::new(
            fr::NonZeroU32::new(width as u32).unwrap(),
            fr::NonZeroU32::new(height as u32).unwrap(),
            self.data(), // Vec<f32>
        );
        img
    }
}

PixelType::U8x3 - RGB/BGR
PixelType::U8 - grayscale

Does this crate rely too much on the pixels being 32 bits for this addition to not be possible?

Hi, I see that this project is able to resize a lot of different values but U8x2 seems to be missing.

What is the recommended strategy of working with LumaA pixels?

Code here

use std::io::BufWriter;
use std::num::NonZeroU32;

use image::codecs::png::PngEncoder;
use image::io::Reader as ImageReader;
use image::{ColorType, ImageEncoder};

use fast_image_resize as fir;

fn main() {
    not_work(); // output: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]

    work(); // output: [255, 196, 181, 255, 196, 181, 255, 198, 181, 255]
}

fn work() {
    let img = ImageReader::open("/dev/shm/0/1.png")
        .unwrap()
        .decode()
        .unwrap();
    let width = NonZeroU32::new(img.width()).unwrap();
    let height = NonZeroU32::new(img.height()).unwrap();

    let src_image = fir::Image::from_vec_u8(
        width,
        height,
        img.to_rgb8().into_raw(),
        fir::PixelType::U8x3,
    )
    .unwrap();

    let fix_width = NonZeroU32::new(img.width() * 2).unwrap();
    let fix_height = NonZeroU32::new(img.height() * 2).unwrap();
    let dst_width = fix_width; // fix here
    let dst_height = fix_height;

    let mut dst_image = fir::Image::new(dst_width, dst_height, src_image.pixel_type());
    let mut dst_view = dst_image.view_mut();
    let mut resizer = fir::Resizer::new(fir::ResizeAlg::Convolution(fir::FilterType::Box));

    resizer.resize(&src_image.view(), &mut dst_view).unwrap();

    eprintln!("{:?}", &dst_image.buffer()[0..10]);
}

fn not_work() {
    let img = ImageReader::open("/dev/shm/0/1.png")
        .unwrap()
        .decode()
        .unwrap();
    let width = NonZeroU32::new(img.width()).unwrap();
    let height = NonZeroU32::new(img.height()).unwrap();

    let src_image = fir::Image::from_vec_u8(
        width,
        height,
        img.to_rgb8().into_raw(),
        fir::PixelType::U8x3,
    )
    .unwrap();
    let dst_width = width; // bug here
    let dst_height = height;

    let mut dst_image = fir::Image::new(dst_width, dst_height, src_image.pixel_type());
    let mut dst_view = dst_image.view_mut();
    let mut resizer = fir::Resizer::new(fir::ResizeAlg::Convolution(fir::FilterType::Box));

    resizer.resize(&src_image.view(), &mut dst_view).unwrap();

    eprintln!("{:?}", &dst_image.buffer()[0..10]);
}

unexpected but here no standard gaussian kernel as opencv has https://docs.opencv.org/4.x/d4/d1f/tutorial_pyramids.html

how does the speed compare with opencv?

I have been trying to do a crop of a 640x480 image to a 640x640 image (which is really about padding using a CropBox { left: 0, top: 80, width: 640, height: 480 })

I believe there is an error in this line:
https://github.com/Cykooz/fast_image_resize/blob/main/src/image_view.rs#L402

I think this inclusive end range is incorrect (and differs from your original example #13 (comment)):

let row_range = (crop_box.left as usize)..=(right as usize);

and should be:

let row_range = (crop_box.left as usize)..(right as usize);

While debugging an (unrelated) memory leak in my own code using valgrind's memcheck tool, the following error was reported in this library.

Unfortunately I'm not quite in a state to be able to share some code to reproduce this error, but hopefully the output from memcheck will help

==1102663== Use of uninitialised value of size 8
==1102663== at 0x3D3A64: fast_image_resize::convolution::u8x1::::horiz_convolution (optimisations.rs:113)
==1102663== by 0x3E72A3: fast_image_resize::resizer::resample_convolution (resizer.rs:285)
==1102663== by 0x3E3B95: fast_image_resize::resizer::Resizer::resize (resizer.rs:0)
==1102663== by 0x3B2C9F: vid_dup_finder_common::video_frames_gray::crop_resize_frame (video_frames_gray.rs:382)

Currently, this method takes a mutable slice, but it does not need to if the Image is used as the source.

For my use case, I do not have mutable ownership of the source image because I am using the same source image across multiple threads. I am therefore having to unsafely cast the immutable slice to a mutable slice to adhere to the interface:

let mut_ptr = image.bytes.as_ptr() as *mut u8;
let mut_slice = unsafe { std::slice::from_raw_parts_mut(mut_ptr, image.bytes.len()) };

let width = NonZeroU32::new(image.bytes_per_row / 4).unwrap();
let height = NonZeroU32::new(image.height).unwrap();
let source = fr::Image::from_slice_u8(width, height, mut_slice, fr::PixelType::U8x4).unwrap();

Please would you be able to change the interface so that from_slice_u8 can be called with an immutable slice, or provide an alternative method for this? Thank you for maintaining a very useful crate.

Hello, I want to prepare a large number of images in a loop using your library. I have an application as below for performance optimization, but I am encountering errors. Which path should we follow?

pub struct Variation {
    pub id: String,
    pub size: (u32, u32),
}
pub struct VariationResult {
    pub id: String,
    pub buffer: Vec<u8>,
}

fn bulk_process( src_image: &fr::Image, variations: &Vec<Variation> ) -> Result<Vec<VariationResult>, Box<dyn Error>> 
{
    let mut image_variations: Vec<VariationResult> = Vec::new();
    
    let mut result_buffer: Vec<u8> = Vec::new();
    let mut buf_writer = BufWriter::new( &mut result_buffer );
    let mut encoder = JpegEncoder::new_with_quality( &mut buf_writer, 100 );
    
    let mut resizer = fr::Resizer::new( fr::ResizeAlg::Nearest );

    for variation in variations {
        
        let dst_width = NonZeroU32::new( variation.size.0 ).unwrap();
        let dst_height = NonZeroU32::new( variation.size.1 ).unwrap();
        let mut dst_image = fr::Image::new(
            dst_width,
            dst_height,
            src_image.pixel_type(),
        );

        resizer.resize( &src_image.view(), &mut dst_image.view_mut() ).unwrap();

        encoder.encode( 
            dst_image.buffer(), 
            dst_width.get(), 
            dst_height.get(), 
            ColorType::Rgb8 
        ).unwrap();

        let image = VariationResult { 
            id: variation.id.to_string(), 
            buffer: result_buffer, // ??
        };

        image_variations.push( image );
    }

    Ok( image_variations )

}