[SETUP-BUG] endpoint deployment 7000, get_deploy_client() return endpoint_url about mlflow HOT 4 OPEN

@LazyCatLee Can you try a smaller temperature like 1?

from mlflow.

@harupy

yeah, I have set the temperature 1 ,but ERROR like this:

data = dict({
"temperature": 1,
"n": 1,
"max_tokens": 500,
"stream": True,
"messages": [
{
"role": "user",
"content": "Which harry potter house am I most likely to belong to?"
}
]
})

Running Result:

File "/home/master/mlflow/mlflow/utils/rest_utils.py", line 141, in http_request
raise MlflowException(f"API request to {url} failed with exception {e}")
mlflow.exceptions.MlflowException: API request to http://127.0.0.1:7000/endpoints/chat/invocations failed with exception Response ended prematurely

MlflowException: API request to http://127.0.0.1:7000/endpoints/chat/invocations
but endpoint_url='http://127.0.0.1:7000/gateway/chat/invocations'
this why

from mlflow.

@LazyCatLee did you get the same error or a different one?

from mlflow.

@mlflow/mlflow-team Please assign a maintainer and start triaging this issue.

from mlflow.