Error when executing Mixed operator decoders__Image when sending image binary to dali in triton about dali_backend HOT 9 OPEN

@proevgenii

Got it. In that case, you're good with open. I believe, that the TYPE_STRING is the real problem. Please use TYPE_UINT8 combined with .astype(np.uint8) and everything should work well.

You can use a snippet from one of our examples:

def load_image(img_path: str):
    """
    Loads image as an encoded array of bytes.
    This is a typical approach you want to use in DALI backend
    """
    with open(img_path, "rb") as f:
        img = f.read()
        return np.array(list(img)).astype(np.uint8)

This function will create a byte stream, that should be passed to set_data_from_numpy:

input = grpc.InferInput(input_name, input_shape, "UINT8")
input.set_data_from_numpy([load_image("path_to_my_image")])

You can refer to the ensemble_client for an example, which reflects quite well what you want to do. Especially functions: load_image, load_images, array_from_list.

from dali_backend.

Then the pipeline you've pasted at the top is a good starting point:

@dali.pipeline_def(batch_size=64, num_threads=4, device_id=0)
def pipe():
    images = dali.fn.external_source(device="cpu", name="DALI_INPUT_0")
    images = dali.fn.decoders.image(images, device="mixed", output_type=types.RGB)
    images = dali.fn.resize(images, resize_x=224, resize_y=224, device='gpu')
    return dali.fn.crop_mirror_normalize(images,
                                           dtype=types.FLOAT16,
                                           output_layout="CHW",
                                           device='gpu',
                                           mean=[0.485 * 255, 0.456 * 255, 0.406 * 255],
                                           std=[0.229 * 255, 0.224 * 255, 0.225 * 255])

When working with images and requiring only resize and normalize, the best approach is to use fn.resize and fn.crop_mirror_normalize.

from dali_backend.

Hello @proevgenii
This is orders of magnitude faster approach (I think it's actually zero-copy).

np.frombuffer(img, dtype=np.uint8)

where img is your bytes object.

from dali_backend.

Hi @proevgenii !

Your code looks more-or-less OK. I believe there might be few reasons, that the image format is not recognized properly:

1. We advice to load the data using np.fromfile function instead of open(..., 'rb') (I know, many our examples show open(...) approach). The former is expected to be 5 times faster than the latter:

img_data = np.fromfile('test_img.png', dtype=np.uint8)

2. In your configuration file, please use TYPE_UINT8 as the input type. I believe TYPE_STRING in Triton has a special meaning and won't work correctly as input type for DALI.

input [
{
    name: "DALI_INPUT_0"
    data_type: TYPE_UINT8
    dims: [ -1 ]
}
]

3. Please verify transformed_img shape. It should be something like: (1, 28172930) (i.e. (batch_size, number_of_bytes_in_encoded_img))
4. I'm not sure you need binary_data=True argument in set_data_from_numpy function

If none of these points help, please let us know, we'd try to figure something out.

from dali_backend.

Hello, @szalpal!
Thanks for such a quick reply!

About 1. and 2.

I have already tried approach with np.fromfile and using TYPE_UINT8 in configuration, and this works
But I'm using this triton server in production system where images already in byte format
And images looks like this string:
b'\xff\xd8\xff\xe0\x00\x10JFIF\x00\x01\x01\x01\x00H\...
And when I'm using open(..., 'rb') I get similar string and so I use this method in the example above
I can save my byte images to .png file and then use np.fromfile but this will dramatically degrade system performance
So is there are any way to send image in byte string format to dali_backend?

3.

img_bytes = open('test_img.png', "rb").read() ### len(img_bytes) = 915829
img_data = np.array([img_bytes], dtype=bytes) ### img_data.shape = (1,)
transformed_img = np.stack([img_data], axis=0) ### transformed_img.shape = (1,1)

4.

Removing binary_data=True doesn't change anything, I'm still the same getting error

Error when executing Mixed operator decoders__Image encountered:
Error in thread 1: [/opt/dali/dali/operators/decoder/nvjpeg/nvjpeg_decoder_decoupled_api.h:615] [/opt/dali/dali/image/image_factory.cc:102] Unrecognized image format. Supported formats are: JPEG, PNG, BMP, TIFF, PNM, JPEG2000 and WebP.

from dali_backend.

Thanks again, @szalpal !
It works perfect!
And if it's possible I have one more question about the image preprocessing pipeline
What is the most time-efficient pipeline, I only need two operations - resize and normalize)

from dali_backend.

Hello here again! 🖖🖖
I still need to send data in the form of byte strings.
Because the np.array(list(img)).astype(np.uint8) operation is too time-consuming
Were there any updates? I do everything as written in this issue

But I get the same error

Error when executing Mixed operator decoders__Image encountered:
Error in thread 0: [/opt/dali/dali/operators/decoder/nvjpeg/nvjpeg_decoder_decoupled_api.h:616] [/opt/dali/dali/image/image_factory.cc:100] Unrecognized image format. Supported formats are: JPEG, PNG, BMP, TIFF, PNM, JPEG2000 and WebP.

from dali_backend.

Hi @mzient
Yes it works, and it much faster than my previous method, thank you 😊

But is there any way to send binary string to dali? Or dali can't perform decoding of byte strings?

from dali_backend.

@mzient @szalpal
Any updates?)

from dali_backend.