This repository contains the necessary configuration and code to deploy a Hugging Face model to Azure using SkyPilot.

Before you begin, ensure you have met the following requirements:

• Azure CLI: Installed the Azure Command Line Interface (CLI). Installation guide available here.

• SkyPilot: Install SkyPilot following the instructions on their official installation page. SkyPilot is essential for deploying and managing your model on Azure.

• You have a .env file in your repository root with your Azure credentials and other necessary environment variables.

• You have installed the necessary Python packages. You can do this by running pip install -r requirements.txt in your repository root.

py available-resources.py

To deploy the model to Azure, run the following command:

sky launch -c vllm-serve -s serve.yaml

This deploys your specified model to a cluster named vllm-serve

More info about how SkyPilot works with vLLM here.