Code Monkey home page Code Monkey logo

Comments (7)

r0man avatar r0man commented on August 23, 2024 1

@theasp Ok, perfect. Thanks for trying! I will open a PR here when I got the event source and OpenAI working as well.

from llm.

ahyatt avatar ahyatt commented on August 23, 2024

@r0man can you take a look at this?

from llm.

r0man avatar r0man commented on August 23, 2024

Hi @theasp,

which proxy are you using? I tried to re-produce this with tinyproxy but my request seems to work.

Could you please run this:

(with-current-buffer (plz 'post "https://xxxxx:443/api/chat"
                       :as 'buffer
                       :body "{\"stream\":false,\"model\":\"DEFAULT/llama3-8b:latest\",\"messages\":[{\"role\":\"user\",\"content\":\"Describe elisp in one word?\"}]}"
                       :decode t
                       :headers '(("Content-Type" . "application/json")))
  (widen)
  (write-region (point-min) (point-max) "/tmp/response.txt"))

and send me /tmp/response.txt?

from llm.

theasp avatar theasp commented on August 23, 2024

Hi @r0man, it's an old version of Squid. I'm not the one that maintains it sadly.

Based on the output below, I suspect that HTTP/1.1 200 came from the proxy and HTTP/2 200 came from Traefik which is in front of Ollama on the remote side.

This is response.txt when using the proxy:

HTTP/1.1 200 Connection established

HTTP/2 200
content-type: application/json; charset=utf-8
date: Thu, 06 Jun 2024 18:40:19 GMT
content-length: 318

{"model":"DEFAULT/llama3-8b:latest","created_at":"2024-06-06T18:40:19.165000711Z","message":{"role":"assistant","content":"**Esoteric**"},"done_reason":"stop","done":true,"total_duration":350939608,"load_duration":2292974,"prompt_eval_count":15,"prompt_eval_duration":133575000,"eval_count":5,"eval_duration":81834000}

This is without the proxy:

HTTP/2 200
content-type: application/json; charset=utf-8
date: Thu, 06 Jun 2024 18:44:10 GMT
content-length: 294

{"model":"DEFAULT/llama3-8b:latest","created_at":"2024-06-06T18:44:10.531876786Z","message":{"role":"assistant","content":"**Esoteric**"},"done_reason":"stop","done":true,"total_duration":295274440,"load_duration":1335630,"prompt_eval_duration":74681000,"eval_count":5,"eval_duration":87733000}

from llm.

r0man avatar r0man commented on August 23, 2024

@theasp The support for proxies was missing. I added this here: r0man/plz-media-type#8

But now I'm running into another issue:

Debugger entered--Lisp error: (wrong-type-argument number-or-marker-p nil)
  <=(200 nil 299)
  (let* ((status val)) (<= 200 status 299))
  (if (let* ((status val)) (<= 200 status 299)) (let ((status val)) (ignore status) (ignore status) (funcall (process-get process :plz-then))) (let nil (let ((err (make-plz-error :response (plz--response)))) (let* ((val (process-get process :plz-else))) (cond ((null val) (let nil (process-put process :plz-result err))) ((functionp val) (let (...) (funcall fn err))) (t (let (...) (error "No clause matching `%S'" x46))))))))
  (let* ((val (plz--http-status))) (if (let* ((status val)) (<= 200 status 299)) (let ((status val)) (ignore status) (ignore status) (funcall (process-get process :plz-then))) (let nil (let ((err (make-plz-error :response (plz--response)))) (let* ((val (process-get process :plz-else))) (cond ((null val) (let nil ...)) ((functionp val) (let ... ...)) (t (let ... ...))))))))
  (save-current-buffer (set-buffer buffer) (goto-char (point-min)) (plz--skip-proxy-headers) (while (plz--skip-redirect-headers)) (let* ((val (plz--http-status))) (if (let* ((status val)) (<= 200 status 299)) (let ((status val)) (ignore status) (ignore status) (funcall (process-get process :plz-then))) (let nil (let ((err (make-plz-error :response ...))) (let* ((val ...)) (cond (... ...) (... ...) (t ...))))))))
  (let nil (save-current-buffer (set-buffer buffer) (goto-char (point-min)) (plz--skip-proxy-headers) (while (plz--skip-redirect-headers)) (let* ((val (plz--http-status))) (if (let* ((status val)) (<= 200 status 299)) (let ((status val)) (ignore status) (ignore status) (funcall (process-get process :plz-then))) (let nil (let ((err ...)) (let* (...) (cond ... ... ...))))))))
  (cond ((eql val 0) (let nil (save-current-buffer (set-buffer buffer) (goto-char (point-min)) (plz--skip-proxy-headers) (while (plz--skip-redirect-headers)) (let* ((val (plz--http-status))) (if (let* (...) (<= 200 status 299)) (let (...) (ignore status) (ignore status) (funcall ...)) (let nil (let ... ...))))))) ((let* ((code val)) (<= 1 code 90)) (let ((code val)) (ignore code) (let* ((curl-exit-code (cond (... ...) (... code))) (curl-error-message (alist-get curl-exit-code plz-curl-errors)) (err (record 'plz-error (cons curl-exit-code curl-error-message) nil nil))) (let* ((val (process-get process :plz-else))) (cond ((null val) (let nil ...)) ((functionp val) (let ... ...)) (t (let ... ...))))))) ((let* ((code val)) (not (<= 1 code 90))) (let ((code val)) (ignore code) (let* ((message (cond (... ...) (... ...) (t ...))) (err (record 'plz-error nil nil message))) (let* ((val (process-get process :plz-else))) (cond ((null val) (let nil ...)) ((functionp val) (let ... ...)) (t (let ... ...))))))) (t (let ((code val)) (let* ((message (format "Unexpected curl process status:%S code:%S.  Please..." status code)) (err (record 'plz-error nil nil message))) (let* ((val (process-get process :plz-else))) (cond ((null val) (let nil ...)) ((functionp val) (let ... ...)) (t (let ... ...))))))))
  (let* ((val (process-exit-status process))) (cond ((eql val 0) (let nil (save-current-buffer (set-buffer buffer) (goto-char (point-min)) (plz--skip-proxy-headers) (while (plz--skip-redirect-headers)) (let* ((val ...)) (if (let* ... ...) (let ... ... ... ...) (let nil ...)))))) ((let* ((code val)) (<= 1 code 90)) (let ((code val)) (ignore code) (let* ((curl-exit-code (cond ... ...)) (curl-error-message (alist-get curl-exit-code plz-curl-errors)) (err (record ... ... nil nil))) (let* ((val ...)) (cond (... ...) (... ...) (t ...)))))) ((let* ((code val)) (not (<= 1 code 90))) (let ((code val)) (ignore code) (let* ((message (cond ... ... ...)) (err (record ... nil nil message))) (let* ((val ...)) (cond (... ...) (... ...) (t ...)))))) (t (let ((code val)) (let* ((message (format "Unexpected curl process status:%S code:%S.  Please..." status code)) (err (record ... nil nil message))) (let* ((val ...)) (cond (... ...) (... ...) (t ...))))))))
  (unwind-protect (let* ((val (process-exit-status process))) (cond ((eql val 0) (let nil (save-current-buffer (set-buffer buffer) (goto-char (point-min)) (plz--skip-proxy-headers) (while (plz--skip-redirect-headers)) (let* (...) (if ... ... ...))))) ((let* ((code val)) (<= 1 code 90)) (let ((code val)) (ignore code) (let* ((curl-exit-code ...) (curl-error-message ...) (err ...)) (let* (...) (cond ... ... ...))))) ((let* ((code val)) (not (<= 1 code 90))) (let ((code val)) (ignore code) (let* ((message ...) (err ...)) (let* (...) (cond ... ... ...))))) (t (let ((code val)) (let* ((message ...) (err ...)) (let* (...) (cond ... ... ...))))))) (let* ((finally (and t (process-get process :plz-finally)))) (if finally (funcall finally) nil)) (if (or (process-get process :plz-sync) (eq 'buffer (process-get process :plz-as))) nil (kill-buffer buffer)))
  plz--respond(#<process plz-request-curl> #<buffer  *plz-request-curl*-661119> "finished\n")
  apply(plz--respond (#<process plz-request-curl> #<buffer  *plz-request-curl*-661119> "finished\n"))
  timer-event-handler([t 26210 58253 269908 nil plz--respond (#<process plz-request-curl> #<buffer  *plz-request-curl*-661119> "finished\n") nil 635000 nil])

I will see what I can do ....

from llm.

r0man avatar r0man commented on August 23, 2024

@theasp Well, I think the remaining issue is with the event source and OpenAI. But since you use Ollama, could you try evaluating this file:
https://github.com/r0man/plz-media-type/blob/154a71ee4f0d5549f355714a58d8fc5ca8201e13/plz-media-type.el

And see if it is working for you?

I don't have Ollama setup via an HTTPS proxy. But I'm using this now with Vertex and it seems to be working.

from llm.

theasp avatar theasp commented on August 23, 2024

@r0man that seems to be working fine for me, thanks!

from llm.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.