opencsgs / llm-inference Goto Github PK
View Code? Open in Web Editor NEWllm-inference is a platform for publishing and managing llm inference, providing a wide range of out-of-the-box features for model deployment, such as UI, RESTful API, auto-scaling, computing resource management, monitoring, and more.
License: Apache License 2.0