Comments (7)
@theasp Ok, perfect. Thanks for trying! I will open a PR here when I got the event source and OpenAI working as well.
from llm.
@r0man can you take a look at this?
from llm.
Hi @theasp,
which proxy are you using? I tried to re-produce this with tinyproxy but my request seems to work.
Could you please run this:
(with-current-buffer (plz 'post "https://xxxxx:443/api/chat"
:as 'buffer
:body "{\"stream\":false,\"model\":\"DEFAULT/llama3-8b:latest\",\"messages\":[{\"role\":\"user\",\"content\":\"Describe elisp in one word?\"}]}"
:decode t
:headers '(("Content-Type" . "application/json")))
(widen)
(write-region (point-min) (point-max) "/tmp/response.txt"))
and send me /tmp/response.txt
?
from llm.
Hi @r0man, it's an old version of Squid. I'm not the one that maintains it sadly.
Based on the output below, I suspect that HTTP/1.1 200
came from the proxy and HTTP/2 200
came from Traefik which is in front of Ollama on the remote side.
This is response.txt when using the proxy:
HTTP/1.1 200 Connection established
HTTP/2 200
content-type: application/json; charset=utf-8
date: Thu, 06 Jun 2024 18:40:19 GMT
content-length: 318
{"model":"DEFAULT/llama3-8b:latest","created_at":"2024-06-06T18:40:19.165000711Z","message":{"role":"assistant","content":"**Esoteric**"},"done_reason":"stop","done":true,"total_duration":350939608,"load_duration":2292974,"prompt_eval_count":15,"prompt_eval_duration":133575000,"eval_count":5,"eval_duration":81834000}
This is without the proxy:
HTTP/2 200
content-type: application/json; charset=utf-8
date: Thu, 06 Jun 2024 18:44:10 GMT
content-length: 294
{"model":"DEFAULT/llama3-8b:latest","created_at":"2024-06-06T18:44:10.531876786Z","message":{"role":"assistant","content":"**Esoteric**"},"done_reason":"stop","done":true,"total_duration":295274440,"load_duration":1335630,"prompt_eval_duration":74681000,"eval_count":5,"eval_duration":87733000}
from llm.
@theasp The support for proxies was missing. I added this here: r0man/plz-media-type#8
But now I'm running into another issue:
Debugger entered--Lisp error: (wrong-type-argument number-or-marker-p nil)
<=(200 nil 299)
(let* ((status val)) (<= 200 status 299))
(if (let* ((status val)) (<= 200 status 299)) (let ((status val)) (ignore status) (ignore status) (funcall (process-get process :plz-then))) (let nil (let ((err (make-plz-error :response (plz--response)))) (let* ((val (process-get process :plz-else))) (cond ((null val) (let nil (process-put process :plz-result err))) ((functionp val) (let (...) (funcall fn err))) (t (let (...) (error "No clause matching `%S'" x46))))))))
(let* ((val (plz--http-status))) (if (let* ((status val)) (<= 200 status 299)) (let ((status val)) (ignore status) (ignore status) (funcall (process-get process :plz-then))) (let nil (let ((err (make-plz-error :response (plz--response)))) (let* ((val (process-get process :plz-else))) (cond ((null val) (let nil ...)) ((functionp val) (let ... ...)) (t (let ... ...))))))))
(save-current-buffer (set-buffer buffer) (goto-char (point-min)) (plz--skip-proxy-headers) (while (plz--skip-redirect-headers)) (let* ((val (plz--http-status))) (if (let* ((status val)) (<= 200 status 299)) (let ((status val)) (ignore status) (ignore status) (funcall (process-get process :plz-then))) (let nil (let ((err (make-plz-error :response ...))) (let* ((val ...)) (cond (... ...) (... ...) (t ...))))))))
(let nil (save-current-buffer (set-buffer buffer) (goto-char (point-min)) (plz--skip-proxy-headers) (while (plz--skip-redirect-headers)) (let* ((val (plz--http-status))) (if (let* ((status val)) (<= 200 status 299)) (let ((status val)) (ignore status) (ignore status) (funcall (process-get process :plz-then))) (let nil (let ((err ...)) (let* (...) (cond ... ... ...))))))))
(cond ((eql val 0) (let nil (save-current-buffer (set-buffer buffer) (goto-char (point-min)) (plz--skip-proxy-headers) (while (plz--skip-redirect-headers)) (let* ((val (plz--http-status))) (if (let* (...) (<= 200 status 299)) (let (...) (ignore status) (ignore status) (funcall ...)) (let nil (let ... ...))))))) ((let* ((code val)) (<= 1 code 90)) (let ((code val)) (ignore code) (let* ((curl-exit-code (cond (... ...) (... code))) (curl-error-message (alist-get curl-exit-code plz-curl-errors)) (err (record 'plz-error (cons curl-exit-code curl-error-message) nil nil))) (let* ((val (process-get process :plz-else))) (cond ((null val) (let nil ...)) ((functionp val) (let ... ...)) (t (let ... ...))))))) ((let* ((code val)) (not (<= 1 code 90))) (let ((code val)) (ignore code) (let* ((message (cond (... ...) (... ...) (t ...))) (err (record 'plz-error nil nil message))) (let* ((val (process-get process :plz-else))) (cond ((null val) (let nil ...)) ((functionp val) (let ... ...)) (t (let ... ...))))))) (t (let ((code val)) (let* ((message (format "Unexpected curl process status:%S code:%S. Please..." status code)) (err (record 'plz-error nil nil message))) (let* ((val (process-get process :plz-else))) (cond ((null val) (let nil ...)) ((functionp val) (let ... ...)) (t (let ... ...))))))))
(let* ((val (process-exit-status process))) (cond ((eql val 0) (let nil (save-current-buffer (set-buffer buffer) (goto-char (point-min)) (plz--skip-proxy-headers) (while (plz--skip-redirect-headers)) (let* ((val ...)) (if (let* ... ...) (let ... ... ... ...) (let nil ...)))))) ((let* ((code val)) (<= 1 code 90)) (let ((code val)) (ignore code) (let* ((curl-exit-code (cond ... ...)) (curl-error-message (alist-get curl-exit-code plz-curl-errors)) (err (record ... ... nil nil))) (let* ((val ...)) (cond (... ...) (... ...) (t ...)))))) ((let* ((code val)) (not (<= 1 code 90))) (let ((code val)) (ignore code) (let* ((message (cond ... ... ...)) (err (record ... nil nil message))) (let* ((val ...)) (cond (... ...) (... ...) (t ...)))))) (t (let ((code val)) (let* ((message (format "Unexpected curl process status:%S code:%S. Please..." status code)) (err (record ... nil nil message))) (let* ((val ...)) (cond (... ...) (... ...) (t ...))))))))
(unwind-protect (let* ((val (process-exit-status process))) (cond ((eql val 0) (let nil (save-current-buffer (set-buffer buffer) (goto-char (point-min)) (plz--skip-proxy-headers) (while (plz--skip-redirect-headers)) (let* (...) (if ... ... ...))))) ((let* ((code val)) (<= 1 code 90)) (let ((code val)) (ignore code) (let* ((curl-exit-code ...) (curl-error-message ...) (err ...)) (let* (...) (cond ... ... ...))))) ((let* ((code val)) (not (<= 1 code 90))) (let ((code val)) (ignore code) (let* ((message ...) (err ...)) (let* (...) (cond ... ... ...))))) (t (let ((code val)) (let* ((message ...) (err ...)) (let* (...) (cond ... ... ...))))))) (let* ((finally (and t (process-get process :plz-finally)))) (if finally (funcall finally) nil)) (if (or (process-get process :plz-sync) (eq 'buffer (process-get process :plz-as))) nil (kill-buffer buffer)))
plz--respond(#<process plz-request-curl> #<buffer *plz-request-curl*-661119> "finished\n")
apply(plz--respond (#<process plz-request-curl> #<buffer *plz-request-curl*-661119> "finished\n"))
timer-event-handler([t 26210 58253 269908 nil plz--respond (#<process plz-request-curl> #<buffer *plz-request-curl*-661119> "finished\n") nil 635000 nil])
I will see what I can do ....
from llm.
@theasp Well, I think the remaining issue is with the event source and OpenAI. But since you use Ollama, could you try evaluating this file:
https://github.com/r0man/plz-media-type/blob/154a71ee4f0d5549f355714a58d8fc5ca8201e13/plz-media-type.el
And see if it is working for you?
I don't have Ollama setup via an HTTPS proxy. But I'm using this now with Vertex and it seems to be working.
from llm.
@r0man that seems to be working fine for me, thanks!
from llm.
Related Issues (20)
- Use tiktoken.el for token counting of openai's models HOT 6
- llama2 embedding vectors from Python and llm.el don't seem to match HOT 4
- LLM request timed out for Gemini
- Unparseable buffer saved to *llm-vertex-unparseable* HOT 6
- llm-gemini.el is using the non-streaming URL for llm-chat-streaming HOT 1
- [feature] Anthropic / Claude2 Support HOT 4
- Using `make-llm-openai-compatible` with Mistral AI fails parsing the partial responses HOT 1
- llm-embedding ignores host with ollama provider
- Using make-llm-openai-compatible with Azure OpenAI service fails to connect to endpoint HOT 7
- Error callback not called if url request failed HOT 1
- Feature request/idea: Use generic functions to extract provider errors HOT 10
- Context does not get sent if list of interactions ≥2 HOT 27
- [Q/FR] "Legacy" Completion HOT 33
- [FR] Support JSON mode HOT 3
- Open WebUI compatibility HOT 1
- Support function calling with ollama HOT 3
- Error using open ai HOT 19
- Errors when function calling with Claude HOT 13
- Streaming multiple function calls gives an error
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from llm.