The doc seems complicated, could you give us a simple guide as how to use triton to inference the model with pre and post data processing with python code? about dali_backend HOT 12 CLOSED

We have a quick start guide in the Python backend repository. Have you tried that example?

https://github.com/triton-inference-server/python_backend/tree/r21.05#quick-start

from dali_backend.

You can also check out the image_client.py

from dali_backend.

In addition to the above you can use DALI along with Triton ensembling to perform pre/post processing. For example: https://github.com/triton-inference-server/dali_backend/tree/main/docs/examples/inception_ensemble

It is not clear from your question where you are trying to use Python and where you want to perform the pre/post processing.

from dali_backend.

In addition to the above you can use DALI along with Triton ensembling to perform pre/post processing. For example: https://github.com/triton-inference-server/dali_backend/tree/main/docs/examples/inception_ensemble

It is not clear from your question where you are trying to use Python and where you want to perform the pre/post processing.

@deadeyegoodwin
So, how to run the examples?
https://github.com/triton-inference-server/dali_backend/tree/main/docs/examples/inception_ensemble
There is no clear guide to run the model, but only generate the model.dali file. Could you tell me how to get the ensemble model, and how to use grpc to inference it?

from dali_backend.

The readme points to a script that generates the model repository for you. Did you follow the quickstart guide to learn how to run triton once you have a model repository? https://github.com/triton-inference-server/server/blob/main/docs/quickstart.md.

Here is another example that has some more details: https://github.com/triton-inference-server/dali_backend/tree/main/docs/examples/resnet50_trt

from dali_backend.

hello @deadeyegoodwin can i use batching for python-backend. Can you provide any examples for batching in python_backend?
ex: batching for add_sub example? i only use "for loop" for processing batch in each request. :( i known it is bad idea

from dali_backend.

@HoangTienDuc it would be better if you didn't ask unrelated questions in this issue. Your python model is receiving the batch of requests... how you can efficiently process those requests is going to depend on what you are trying to do.

from dali_backend.

In addition to the above you can use DALI along with Triton ensembling to perform pre/post processing. For example: https://github.com/triton-inference-server/dali_backend/tree/main/docs/examples/inception_ensemble
It is not clear from your question where you are trying to use Python and where you want to perform the pre/post processing.

@deadeyegoodwin
So, how to run the examples?
https://github.com/triton-inference-server/dali_backend/tree/main/docs/examples/inception_ensemble
There is no clear guide to run the model, but only generate the model.dali file. Could you tell me how to get the ensemble model, and how to use grpc to inference it?

Can we use TEXT data in DALI for pre/post processing? It looks like it supports only image, audio and video type of data. I want to perform pre/post processing within the scope of TritonServer.

from dali_backend.

@szalpal Is seems like the instructions for the DALI inception ensemble can be improved. I'm going to move this issue to the dali_backend repo issues.

from dali_backend.

@HoangTienDuc , if you have more questions about Python Backend, please create an issue in python_backend repo

from dali_backend.

@Edwardmark ,

as @deadeyegoodwin mentioned, good place to start would be to get accustomed to the quickstart guide. If you have any specific question, don't hesitate do create an issue in dali_backend repository

from dali_backend.

@chandrameenamohan ,

Unfortunately, DALI is not the best tool for raw text processing. The meat and potatos of DALI is to transfer data processing operations to the GPU. That's why we focus on image, audio and video: offloading these kinds of data bring the most benefit in terms of processing time.

I believe, that the best shot for you would be to use Python Backend for Triton. This way you still preprocess within the scope of TritonServer, while coding in python.

Should you have any more questions for DALI, don't hesitate to open an issue here, or in DALI repository

from dali_backend.