HTTP Proxy provides unified API to different chat completion and embedding models, assistants and applications. Written in Java 17 and built on top of Eclipse Vert.x.

Build the project with Gradle and Java 17:

./gradlew build

Run the project with Gradle:

./gradlew run

Or run com.epam.aidial.core.AIDial class from your favorite IDE.

Static settings are used on startup and cannot be changed while application is running. Priority order:

• Environment variables with extra "aidial." prefix. E.g. "aidial.server.port", "aidial.config.files".
• File specified in "AIDIAL_SETTINGS" environment variable.
• Default resource file: src/main/resources/aidial.settings.json.

There are two types of credential providers supported:

• User credentials. You can create a service account and authenticate using its private key obtained from Developer console
• Temporary credentials. Application default credentials (ADC)

You should set storage.credential to a path to the private key JSON file and storage.identity must be unset. See example below:

{
  "type": "service_account",
  "project_id": "<your_project_id>",
  "private_key_id": "<your_project_key_id>",
  "private_key": "-----BEGIN PRIVATE KEY-----\n<your_private_key>\n-----END PRIVATE KEY-----\n",
  "client_email": "gcp-dial-core@<your_project_id>.iam.gserviceaccount.com",
  "client_id": "<client_id>",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/gcp-dial-core.iam.gserviceaccount.com",
  "universe_domain": "googleapis.com"
}

Otherwise storage.credential is a private key in PEM format and storage.identity is a client email address.

You should follow instructions to setup your pod in GKE. storage.credential and storage.identity must be unset. JClouds property jclouds.oauth.credential-type should be set bearerTokenCredentials, e.g.

{
  "storage": {
    "overrides": {
      "jclouds.oauth.credential-type": "bearerTokenCredentials"
    }
  }
}

There are two types of credential providers supported:

• User credentials. You can create a service principle and authenticate using its secret from Azure console
• Temporary credentials with Azure AD Workload Identity

You should set storage.credential to service principle secret and storage.identity - service principle ID.

You should follow instructions to setup your pod in Azure k8s. storage.credential and storage.identity must be unset.

The properties to be overridden are below:

{
  "storage": {
    "endpoint": "https://<Azure Blob storage account>.blob.core.windows.net"
    "overrides": {
      "jclouds.azureblob.auth": "azureAd",
      "jclouds.oauth.credential-type": "bearerTokenCredentials"
    }
  }
}

The Redis can be used as a cache with volatile-* eviction policies:

maxmemory 4G
maxmemory-policy volatile-lfu

Note: Redis will be strictly required in the upcoming releases 0.8+.

Dynamic settings are stored in JSON files, specified via "config.files" static setting, and reloaded at interval, specified via "config.reload" static setting. Dynamic settings include:

• Models
• Assistant
• Applications
• Access Keys
• Access Permissions
• Rate Limits

Copyright (C) 2023 EPAM Systems

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.