💡 idea: Soften the Enterprise requirement for multiple LLM backends (Add AI Service) about mattermost-plugin-ai HOT 2 OPEN

Hey @AnnoyingTechnology, thanks for the feedback. Can you help us understand your use case for multi LLM support? Also curious in general how your team is using Mattermost and how large your instance is in terms of user count?

We've heard the overwhelming majority of small teams are using just one LLM model, while larger instances and enterprises may need multi-LLM support to run various custom and tuned models. Also, please check out the design preview of some work we're doing to enhance multi-LLM support even further: #69 (comment). We'd look forward to hearing your feedback on that thread regarding the designs and functionality.

from mattermost-plugin-ai.

Hi,

Just to answer on the use case :

• We have roughly ten people who would use this feature, but only 3 on a regular basis.
• We would use gpt-4-turbo and Mixtral 8x7B on a local 3090.

Most requests

• we prefer not having the data leave the network
• we enjoy the low latency ultra high token/s of a local LLM
• no need for another language than English

Some requests

• we actually need GPT4
• and/or we need French language support
• and/or we need document upload support (currently using ChatGPT for that, but I guess it will come to mattermost)
• and we are ready to accept that our is data sent to OpenAI

Hence the two LLM endpoints.

That multi-bot feature would be ideal.

from mattermost-plugin-ai.