requirements.txt not honored in 1.2.0 about sagemaker-pytorch-inference-toolkit HOT 4 CLOSED

Sorry for the delay in response.

Unfortunately, requirements.txt support is available only for PyTorch 1.3.1+ with the serving images. Is there a reason you need 1.2.0 specifically?

from sagemaker-pytorch-inference-toolkit.

It is also supported in 1.1.0. I need 1.2.0 because pytorch 1.3.1 gives a higher latency for my model. Can you point me to the code section where this part is implemented? I will look into incorporating this.

from sagemaker-pytorch-inference-toolkit.

you will need to rebuild the Docker image with the updated source in https://github.com/aws/sagemaker-inference-toolkit, which is a dependency of this repository.

alternatively, you could try something like

import sys

sys.check_call([sys.executable, '-m', 'pip', 'install', '-r', 'requirements.txt'])

in your inference script.

I've also reached out to the team responsible for building the PyTorch images to see if they would be able to update the 1.2.0 images.

from sagemaker-pytorch-inference-toolkit.

Hi @alohia, we recently release the PyTorch 1.4.0 containers. Does that work with your inference workflow?

from sagemaker-pytorch-inference-toolkit.