Comments (10)
crspeller
commented on July 20, 2024
@johnsmzr Could you post the section in the Mattermost config.json related to the AI plugin. Search for mattermost-ai it should have a "config" section directly underneath it (not the one that just says enabled)
What do you see in Mattermost? no response at all or an error response?
from mattermost-plugin-ai.
johnsmzr
commented on July 20, 2024
@crspeller Thank you for the reply!
I do not find the config.json, do you mean plugin.json?
What do you see in Mattermost? no response at all or an error response?
I can see nothing in mattermost ai-chat. Yes, no response at all.
from mattermost-plugin-ai.
johnsmzr
commented on July 20, 2024
update:
I think this is the config you want:
"Plugins": {
"mattermost-ai": {
"config": {
"allowPrivateChannels": true,
"allowedTeamIds": "",
"enableLLMTrace": true,
"enableUserRestrictions": false,
"imageGeneratorBackend": "gpt",
"llmBackend": "gpt",
"onlyUsersOnTeam": "",
"services": [
{
"apiKey": "",
"defaultModel": "",
"id": "8hoeqet22qd",
"name": "gpt",
"password": "",
"serviceName": "openaicompatible",
"tokenLimit": 0,
"url": "http://host.docker.internal:8080/v1",
"username": ""
}
],
"transcriptBackend": "gpt"
}
},
from mattermost-plugin-ai.
TheMasterFX
commented on July 20, 2024
I had the same issue with Ollama. Then I entered a valid default model and it works
I think in LocalAI they only return the you requested but not load it.
from mattermost-plugin-ai.
johnsmzr
commented on July 20, 2024
@TheMasterFX
Thanks for the help. I tried it ( i.e. entering a valid default model ), it does not work.
The LocalAI backend loaded the model and generated the complete answer.
(The response I posted before is just a part of the whole response, because it's too long.)
Call Localai API:
curl http://{LOCALAI}/v1/chat/completions -H "Content-Type: application/json" -d '{
"messages": [{"role": "user", "content": "hello"}],
"temperature": 0.9,
"stream": true
}'
whole response:
data: {"created":1709113748,"object":"chat.completion.chunk","id":"ac63af57-ba76-4c3d-a455-0269249f5e04","choices":[{"index":0,"finish_reason":"","delta":{"role":"assistant","content":""}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
data: {"created":1709113748,"object":"chat.completion.chunk","id":"ac63af57-ba76-4c3d-a455-0269249f5e04","choices":[{"index":0,"finish_reason":"","delta":{"content":"H"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
data: {"created":1709113748,"object":"chat.completion.chunk","id":"ac63af57-ba76-4c3d-a455-0269249f5e04","choices":[{"index":0,"finish_reason":"","delta":{"content":"e"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
data: {"created":1709113748,"object":"chat.completion.chunk","id":"ac63af57-ba76-4c3d-a455-0269249f5e04","choices":[{"index":0,"finish_reason":"","delta":{"content":"l"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
data: {"created":1709113748,"object":"chat.completion.chunk","id":"ac63af57-ba76-4c3d-a455-0269249f5e04","choices":[{"index":0,"finish_reason":"","delta":{"content":"l"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
data: {"created":1709113748,"object":"chat.completion.chunk","id":"ac63af57-ba76-4c3d-a455-0269249f5e04","choices":[{"index":0,"finish_reason":"","delta":{"content":"o"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
data: {"created":1709113748,"object":"chat.completion.chunk","id":"ac63af57-ba76-4c3d-a455-0269249f5e04","choices":[{"index":0,"finish_reason":"","delta":{"content":","}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
data: {"created":1709113748,"object":"chat.completion.chunk","id":"ac63af57-ba76-4c3d-a455-0269249f5e04","choices":[{"index":0,"finish_reason":"","delta":{"content":" "}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
data: {"created":1709113748,"object":"chat.completion.chunk","id":"ac63af57-ba76-4c3d-a455-0269249f5e04","choices":[{"index":0,"finish_reason":"","delta":{"content":"h"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
data: {"created":1709113748,"object":"chat.completion.chunk","id":"ac63af57-ba76-4c3d-a455-0269249f5e04","choices":[{"index":0,"finish_reason":"","delta":{"content":"o"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
data: {"created":1709113748,"object":"chat.completion.chunk","id":"ac63af57-ba76-4c3d-a455-0269249f5e04","choices":[{"index":0,"finish_reason":"","delta":{"content":"w"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
data: {"created":1709113748,"object":"chat.completion.chunk","id":"ac63af57-ba76-4c3d-a455-0269249f5e04","choices":[{"index":0,"finish_reason":"","delta":{"content":" "}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
data: {"created":1709113748,"object":"chat.completion.chunk","id":"ac63af57-ba76-4c3d-a455-0269249f5e04","choices":[{"index":0,"finish_reason":"","delta":{"content":"m"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
data: {"created":1709113748,"object":"chat.completion.chunk","id":"ac63af57-ba76-4c3d-a455-0269249f5e04","choices":[{"index":0,"finish_reason":"","delta":{"content":"a"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
data: {"created":1709113748,"object":"chat.completion.chunk","id":"ac63af57-ba76-4c3d-a455-0269249f5e04","choices":[{"index":0,"finish_reason":"","delta":{"content":"y"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
data: {"created":1709113748,"object":"chat.completion.chunk","id":"ac63af57-ba76-4c3d-a455-0269249f5e04","choices":[{"index":0,"finish_reason":"","delta":{"content":" "}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
data: {"created":1709113748,"object":"chat.completion.chunk","id":"ac63af57-ba76-4c3d-a455-0269249f5e04","choices":[{"index":0,"finish_reason":"","delta":{"content":"I"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
data: {"created":1709113748,"object":"chat.completion.chunk","id":"ac63af57-ba76-4c3d-a455-0269249f5e04","choices":[{"index":0,"finish_reason":"","delta":{"content":" "}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
data: {"created":1709113748,"object":"chat.completion.chunk","id":"ac63af57-ba76-4c3d-a455-0269249f5e04","choices":[{"index":0,"finish_reason":"","delta":{"content":"a"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
data: {"created":1709113748,"object":"chat.completion.chunk","id":"ac63af57-ba76-4c3d-a455-0269249f5e04","choices":[{"index":0,"finish_reason":"","delta":{"content":"s"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
data: {"created":1709113748,"object":"chat.completion.chunk","id":"ac63af57-ba76-4c3d-a455-0269249f5e04","choices":[{"index":0,"finish_reason":"","delta":{"content":"s"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
data: {"created":1709113748,"object":"chat.completion.chunk","id":"ac63af57-ba76-4c3d-a455-0269249f5e04","choices":[{"index":0,"finish_reason":"","delta":{"content":"i"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
data: {"created":1709113748,"object":"chat.completion.chunk","id":"ac63af57-ba76-4c3d-a455-0269249f5e04","choices":[{"index":0,"finish_reason":"","delta":{"content":"s"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
data: {"created":1709113748,"object":"chat.completion.chunk","id":"ac63af57-ba76-4c3d-a455-0269249f5e04","choices":[{"index":0,"finish_reason":"","delta":{"content":"t"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
data: {"created":1709113748,"object":"chat.completion.chunk","id":"ac63af57-ba76-4c3d-a455-0269249f5e04","choices":[{"index":0,"finish_reason":"","delta":{"content":" "}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
data: {"created":1709113748,"object":"chat.completion.chunk","id":"ac63af57-ba76-4c3d-a455-0269249f5e04","choices":[{"index":0,"finish_reason":"","delta":{"content":"y"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
data: {"created":1709113748,"object":"chat.completion.chunk","id":"ac63af57-ba76-4c3d-a455-0269249f5e04","choices":[{"index":0,"finish_reason":"","delta":{"content":"o"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
data: {"created":1709113748,"object":"chat.completion.chunk","id":"ac63af57-ba76-4c3d-a455-0269249f5e04","choices":[{"index":0,"finish_reason":"","delta":{"content":"u"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
data: {"created":1709113748,"object":"chat.completion.chunk","id":"ac63af57-ba76-4c3d-a455-0269249f5e04","choices":[{"index":0,"finish_reason":"","delta":{"content":"?"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
data: {"created":1709113748,"object":"chat.completion.chunk","id":"ac63af57-ba76-4c3d-a455-0269249f5e04","choices":[{"index":0,"finish_reason":"stop","delta":{"content":""}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
data: [DONE]
from mattermost-plugin-ai.
Bazze
commented on July 20, 2024
Iām seeing the exact same issue with LocalAI in my setup. Same versions as mentioned above.
from mattermost-plugin-ai.
crspeller
commented on July 20, 2024
@johnsmzr Not seeing the same behavior when I try it. I don't see anything wrong with your configuration.
I have merged a few PRs to add some more resiliance and errors to this code path. If you would be willing to try master again and tell me what errors you see that might be helpful.
It also might be useful to know exactly how you are using LocalAI. Though docker? What model? etc.
from mattermost-plugin-ai.
johnsmzr
commented on July 20, 2024
@crspeller
Thanks for the reply!
How I use LocalAI:
I build the LocalAI as binary and run it locally on Macbook Pro M3.
What model?
- name: ggml-gpt4all-j
- source: https://gpt4all.io/models/ggml-gpt4all-j.bin
I updated the mattermost-ai-plugin to 0.6.2. which contains major fixes and tested again.
Now there are two types of errors:
part of the LocalAI debug info:
4:25PM DBG Prompt (after templating): The prompt below is a question to answer, a task to complete, or a conversation to respond to; decide which and write an appropriate response.
### Prompt:
Write a short title for the following request. Include only the title and nothing else, no quotations. Request:
how are you?
### Response:
[127.0.0.1]:62612 200 - POST /v1/chat/completions
4:25PM DBG Sending chunk: {"created":1710429813,"object":"chat.completion.chunk","id":"86948288-d2b3-427e-8f34-ab3d012126e5","model":"ggml-gpt4all-j","choices":[{"index":0,"finish_reason":"","delta":{"role":"assistant","content":""}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
4:25PM DBG Model already loaded in memory: ggml-gpt4all-j
4:25PM DBG Model 'ggml-gpt4all-j' already loaded
4:25PM DBG Function return: I am a virtual assistant named "AI Copiplot". I am a copy of human assistants that respond automatically to users' requests on the Mattermost chat server. map[]
4:25PM DBG Sending chunk: {"created":1710429813,"object":"chat.completion.chunk","id":"86948288-d2b3-427e-8f34-ab3d012126e5","model":"ggml-gpt4all-j","choices":[{"index":0,"finish_reason":"","delta":{"content":"I"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
4:25PM DBG Sending chunk: {"created":1710429813,"object":"chat.completion.chunk","id":"86948288-d2b3-427e-8f34-ab3d012126e5","model":"ggml-gpt4all-j","choices":[{"index":0,"finish_reason":"","delta":{"content":"'"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
4:25PM DBG Sending chunk: {"created":1710429813,"object":"chat.completion.chunk","id":"86948288-d2b3-427e-8f34-ab3d012126e5","model":"ggml-gpt4all-j","choices":[{"index":0,"finish_reason":"","delta":{"content":"m"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
4:25PM DBG Sending chunk: {"created":1710429813,"object":"chat.completion.chunk","id":"86948288-d2b3-427e-8f34-ab3d012126e5","model":"ggml-gpt4all-j","choices":[{"index":0,"finish_reason":"","delta":{"content":" "}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
4:25PM DBG Sending chunk: {"created":1710429813,"object":"chat.completion.chunk","id":"86948288-d2b3-427e-8f34-ab3d012126e5","model":"ggml-gpt4all-j","choices":[{"index":0,"finish_reason":"","delta":{"content":"d"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
4:25PM DBG Sending chunk: {"created":1710429813,"object":"chat.completion.chunk","id":"86948288-d2b3-427e-8f34-ab3d012126e5","model":"ggml-gpt4all-j","choices":[{"index":0,"finish_reason":"","delta":{"content":"o"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
4:25PM DBG Sending chunk: {"created":1710429813,"object":"chat.completion.chunk","id":"86948288-d2b3-427e-8f34-ab3d012126e5","model":"ggml-gpt4all-j","choices":[{"index":0,"finish_reason":"","delta":{"content":"i"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
4:25PM DBG Sending chunk: {"created":1710429813,"object":"chat.completion.chunk","id":"86948288-d2b3-427e-8f34-ab3d012126e5","model":"ggml-gpt4all-j","choices":[{"index":0,"finish_reason":"","delta":{"content":"n"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
4:25PM DBG Sending chunk: {"created":1710429813,"object":"chat.completion.chunk","id":"86948288-d2b3-427e-8f34-ab3d012126e5","model":"ggml-gpt4all-j","choices":[{"index":0,"finish_reason":"","delta":{"content":"g"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
4:25PM DBG Sending chunk: {"created":1710429813,"object":"chat.completion.chunk","id":"86948288-d2b3-427e-8f34-ab3d012126e5","model":"ggml-gpt4all-j","choices":[{"index":0,"finish_reason":"","delta":{"content":" "}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
4:25PM DBG Sending chunk: {"created":1710429813,"object":"chat.completion.chunk","id":"86948288-d2b3-427e-8f34-ab3d012126e5","model":"ggml-gpt4all-j","choices":[{"index":0,"finish_reason":"","delta":{"content":"w"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
4:25PM DBG Sending chunk: {"created":1710429813,"object":"chat.completion.chunk","id":"86948288-d2b3-427e-8f34-ab3d012126e5","model":"ggml-gpt4all-j","choices":[{"index":0,"finish_reason":"","delta":{"content":"e"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
4:25PM DBG Sending chunk: {"created":1710429813,"object":"chat.completion.chunk","id":"86948288-d2b3-427e-8f34-ab3d012126e5","model":"ggml-gpt4all-j","choices":[{"index":0,"finish_reason":"","delta":{"content":"l"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
4:25PM DBG Sending chunk: {"created":1710429813,"object":"chat.completion.chunk","id":"86948288-d2b3-427e-8f34-ab3d012126e5","model":"ggml-gpt4all-j","choices":[{"index":0,"finish_reason":"","delta":{"content":"l"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
4:25PM DBG Sending chunk: {"created":1710429813,"object":"chat.completion.chunk","id":"86948288-d2b3-427e-8f34-ab3d012126e5","model":"ggml-gpt4all-j","choices":[{"index":0,"finish_reason":"","delta":{"content":","}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
4:25PM DBG Sending chunk failed: connection closed
Error rpc error: code = Canceled desc = context canceled
error from mattermost server log:
{
"caller": "app/plugin_api.go:976",
"level": "error",
"msg": "LLM closed stream with no result",
"plugin_id": "mattermost-ai",
"timestamp": "2024-03-14 15:30:11.571 Z"
}
I hope these information could be helpful.
from mattermost-plugin-ai.
Related Issues (20)
- š docs: Update README and issues ahead of v1.0 release
- Webapp bad export of AdvancedCreateComment
- š bug: Wrong file added to the 0.5.0 pre release HOT 1
- š” idea: Ollama support HOT 7
- š bug: Not able to active in Mattermost V6&V7 HOT 3
- š docs: adjust v1.0 feature documentation
- š” idea: Add Organization Input Option to Open AI Engine Configuration Form
- š” idea: Is it possible to make it work properly without setting up the github plugin? HOT 1
- š bug: Anthropic model integration Issue: Header requirement error and failed responses HOT 5
- š bug: deleting suggested prompt prevents me from using another suggested prompt
- š” idea: add supported Mattermost server versions to README
- š bug: When replying in a thread, the response from OpenAI is not updated on the main screen in the stream.
- Why Does My AI Reply to My Threads Instead of Posting Directly in the Channel? HOT 5
- Feature Idea: Image Generation HOT 2
- Enhancement: Provide image as input to the model - gpt-4-vision-preview HOT 2
- Bug: RPC call MessageHasBeenPosted to plugin failed. HOT 3
- Feature Request: Gracefully handle when conversation contexts become too long HOT 3
- Feature Idea: Add in a "copy to clipboard" function onto AI messages HOT 3
- Bug: Plugin crashes when API request faces network failure
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
š Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ššš
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ā¤ļø Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mattermost-plugin-ai.