Comments (6)
xpilasneo4j
commented on August 12, 2024
Checking the file pull_model.Dockerfile, I see the below
(process/shell {:env {"OLLAMA_HOST" url} :out :inherit :err :inherit} (format "./bin/ollama pull %s" llm))
I don't believe that will work on windows or it has to follow the same path with a bin/ directory
I changed the ./bin into my windows path to Ollama server and it worked
Then it's a question of making sure the path to call Ollama is recognized by Windows
from genai-stack.
jexp
commented on August 12, 2024
@mchiang0610 either we need to figure out a way to generalize this or at least make a clear call-out in the readme?
from genai-stack.
prillcode
commented on August 12, 2024
So does Ollama run on Windows in the WSL linux subsystem that Docker runs in?
from genai-stack.
xpilasneo4j
commented on August 12, 2024
Ollama can run on both:
- you can run it on WSL
- you can download the code and build it on Windows to run natively
from genai-stack.
icejean
commented on August 12, 2024
Checking the file pull_model.Dockerfile, I see the below
(process/shell {:env {"OLLAMA_HOST" url} :out :inherit :err :inherit} (format "./bin/ollama pull %s" llm))I don't believe that will work on windows or it has to follow the same path with a bin/ directory I changed the./bininto my windows path to Ollama server and it workedThen it's a question of making sure the path to call Ollama is recognized by Windows
How to fix the issue with replaceing the ./bin into my windows path? Does it need to replace all ./bin in file pull_model.Dockerfile? For example, my windows path is C:\Users\Jean\AppData\Local\Programs\Ollama.
#syntax = docker/dockerfile:1.4
FROM ollama/ollama:latest AS ollama
FROM babashka/babashka:latest
# just using as a client - never as a server
COPY --from=ollama /bin/ollama ./bin/ollama
COPY <<EOF pull_model.clj
(ns pull-model
(:require [babashka.process :as process]
[clojure.core.async :as async]))
(try
(let [llm (get (System/getenv) "LLM")
url (get (System/getenv) "OLLAMA_BASE_URL")]
(println (format "pulling ollama model %s using %s" llm url))
(if (and llm url (not (#{"gpt-4" "gpt-3.5" "claudev2"} llm)))
;; ----------------------------------------------------------------------
;; just call `ollama pull` here - create OLLAMA_HOST from OLLAMA_BASE_URL
;; ----------------------------------------------------------------------
;; TODO - this still doesn't show progress properly when run from docker compose
(let [done (async/chan)]
(async/go-loop [n 0]
(let [[v _] (async/alts! [done (async/timeout 5000)])]
(if (= :stop v) :stopped (do (println (format "... pulling model (%ss) - will take several minutes" (* n 10))) (recur (inc n))))))
(process/shell {:env {"OLLAMA_HOST" url} :out :inherit :err :inherit} (format "bash -c './bin/ollama show %s --modelfile > /dev/null || ./bin/ollama pull %s'" llm llm))
(async/>!! done :stop))
(println "OLLAMA model only pulled if both LLM and OLLAMA_BASE_URL are set and the LLM model is not gpt")))
(catch Throwable _ (System/exit 1)))
EOF
ENTRYPOINT ["bb", "-f", "pull_model.clj"]
from genai-stack.
icejean
commented on August 12, 2024
Well, addressed already. All ./bin need to be replaced, and should use OLLAMA_BASE_URL=http://host.docker.internal:11434 to reference to llama2/3 model running on local Windows machine.
pull_model.Dockerfile:
#syntax = docker/dockerfile:1.4
FROM ollama/ollama:latest AS ollama
FROM babashka/babashka:latest
# just using as a client - never as a server
#COPY --from=ollama /bin/ollama ./bin/ollama
COPY --from=ollama /bin/ollama C:/Users/Jean/AppData/Local/Programs/Ollama/ollama
COPY <<EOF pull_model.clj
(ns pull-model
(:require [babashka.process :as process]
[clojure.core.async :as async]))
(try
(let [llm (get (System/getenv) "LLM")
url (get (System/getenv) "OLLAMA_BASE_URL")]
(println (format "pulling ollama model %s using %s" llm url))
(if (and llm url (not (#{"gpt-4" "gpt-3.5" "claudev2"} llm)))
;; ----------------------------------------------------------------------
;; just call `ollama pull` here - create OLLAMA_HOST from OLLAMA_BASE_URL
;; ----------------------------------------------------------------------
;; TODO - this still doesn't show progress properly when run from docker compose
(let [done (async/chan)]
(async/go-loop [n 0]
(let [[v _] (async/alts! [done (async/timeout 5000)])]
(if (= :stop v) :stopped (do (println (format "... pulling model (%ss) - will take several minutes" (* n 10))) (recur (inc n))))))
#(process/shell {:env {"OLLAMA_HOST" url} :out :inherit :err :inherit} (format "bash -c './bin/ollama show %s --modelfile > /dev/null || ./bin/ollama pull %s'" llm llm))
(process/shell {:env {"OLLAMA_HOST" url} :out :inherit :err :inherit} (format "bash -c 'C:/Users/Jean/AppData/Local/Programs/Ollama/ollama show %s --modelfile > /dev/null || C:/Users/Jean/AppData/Local/Programs/Ollama/ollama pull %s'" llm llm))
(async/>!! done :stop))
(println "OLLAMA model only pulled if both LLM and OLLAMA_BASE_URL are set and the LLM model is not gpt")))
(catch Throwable _ (System/exit 1)))
EOF
ENTRYPOINT ["bb", "-f", "pull_model.clj"]
.env:
#*****************************************************************
# LLM and Embedding Model
#*****************************************************************
LLM=llama3 #or any Ollama model tag,llama2, gpt-4, gpt-3.5, or claudev2
EMBEDDING_MODEL=sentence_transformer # sentence_transformer or google-genai-embedding-001 openai, ollama, or aws
#*****************************************************************
# Ollama
#*****************************************************************
#MAC & Windows, Access services on local machine through host.docker.internal
OLLAMA_BASE_URL=http://host.docker.internal:11434
#Linux
# OLLAMA_BASE_URL=http://llm:11434
from genai-stack.
Related Issues (20)
- genai-stack-pull-model-1 | Error: one of '--license', '--modelfile', '--parameters', '--system', or '--template' must be specified genai-stack-pull-model-1 exited with code 1 HOT 3
- How to load my own data into Neo4j instead of stackOverflow? HOT 1
- Pull Model 1 exited with code 0 HOT 6
- This software in its current form is useless HOT 5
- /data folder permission issue when running on Linux HOT 5
- ollama failed to load model gemma:2b HOT 1
- How can I use my own Neo4j Graph to do RAG?
- continue integration
- build issue, stopped building at bot[5/8]
- dependency failed to start: container genai-stack-api-1 is unhealthy HOT 8
- App 1 - Support Agent Bot not working HOT 5
- Steamlit Text Box Covered by "Select RAG mode" HOT 4
- I'm unable to use the pdf_bot using AWS claudev2
- llama3: HOT 3
- Error: pull model manifest: ssh: no key found - Cannot pull model from within containerized Ollama instance HOT 1
- Error: Head "http://host.docker.internal:11434/": dial tcp 192.168.178.36:11434: i/o timeout -> pull-model-1 exited with code 1 HOT 1
- Version Update needed: ValueError: alternative_import must be a fully qualified module path
- pull-model Warning
- service "pull-model" didn't complete successfully: exit 1
- ERROR [genai-stack-api internal] load metadata for docker.io/langchain/langchain:latest
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from genai-stack.