๐งโ๐ณ
maknee / minigpt4.cpp Goto Github PK
View Code? Open in Web Editor NEWPort of MiniGPT4 in C++ (4bit, 5bit, 6bit, 8bit, 16bit CPU inference with GGML)
License: MIT License
Port of MiniGPT4 in C++ (4bit, 5bit, 6bit, 8bit, 16bit CPU inference with GGML)
License: MIT License
๐งโ๐ณ
Hi,
I enabled the cublas compilation option.
The problem is that not charge o process all in GRAM memory?
What is the best line command to construct and execute in a CUDA 3090 with 24GB GRAM in the more fast posibility for each model?
Hi ๏ผ
I want to test the token speed of minigpt4, but tokenizer failed
AutoTokenizer.from_pretrained('maknee/ggml-vicuna-v0-quantized/13B') or
huggingface_hub.utils._validators.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': 'maknee/ggml-vicuna-v0-quantized/13B'. Use repo_type
argument if needed.
AutoTokenizer.from_pretrained('maknee/ggml-vicuna-v0-quantized') both failed.
Repository Not Found for url: https://huggingface.co/maknee/ggml-vicuna-v0-quantized/resolve/main/tokenizer_config.json.
Please make sure you specified the correct repo_id
and repo_type
.
what is correct command for tokennizer? thanks
I tried setting option(MINIGPT4_CUBLAS "minigpt4: use cuBLAS" ON) in the CMakeLists.txt.
But when i run cmake --build . --config Release,
i get this error below unfortunately : -
Any advice to deal with is highly appreciated
Originally posted by @deadpipe in #16 (comment)
In the "Convert the model to ggml" section of the README.md, what is the "" parameter referring to? Following the instructions in MiniGPT4, I ended up with a folder containing the model weights. What specific file should I point to with this parameter?
Following https://github.com/OpenTalker/SadTalker/issues/693#issuecomment-1789530148 I uninstalled gradio 4.7.1 and installed version 3.41.2.
python3 -m pip uninstall gradio
python3 -m pip install gradio=3.41.2
Posting to gauge/express interested in MiniGPT-v2 support being added.
Trying to build following the instructions of the README. I'm on Ubuntu 20.04.6 LTS.
I first cloned the repo:
git clone --recursive https://github.com/Maknee/minigpt4.cpp
cd minigpt4.cpp
I then installed cmake
version 3.16.3-1ubuntu1.20.04.1 with
sudo apt install cmake
.
Then I ran cmake .
, but it fails:
cmake .
-- The C compiler identification is GNU 9.4.0
-- The CXX compiler identification is GNU 9.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- x86 detected
-- Working on fmt
-- Module support is disabled.
-- Version: 9.1.0
-- Build type:
-- CXX_STANDARD: 23
-- Required features: cxx_variadic_templates
-- Working on unordered_dense
-- Working on stb
Header only
-- Working on spdlog
-- Build spdlog: 1.11.0
-- Build type: Release
-- Working on nlohmann_json
-- Using the multi-header code from /home/marnix/ART/CAMERA/minigpt4.cpp/_deps/nlohmann_json-src/include/
-- Working on tl_expected
-- Working on llama_cpp
CMake Warning (dev) at _deps/llama_cpp-src/CMakeLists.txt:40 (option):
Policy CMP0077 is not set: option() honors normal variables. Run "cmake
--help-policy CMP0077" for policy details. Use the cmake_policy command to
set the policy and suppress this warning.
For compatibility with older versions of CMake, option is clearing the
normal variable 'LLAMA_STATIC'.
This warning is for project developers. Use -Wno-dev to suppress it.
CMake Warning (dev) at _deps/llama_cpp-src/CMakeLists.txt:41 (option):
Policy CMP0077 is not set: option() honors normal variables. Run "cmake
--help-policy CMP0077" for policy details. Use the cmake_policy command to
set the policy and suppress this warning.
For compatibility with older versions of CMake, option is clearing the
normal variable 'LLAMA_NATIVE'.
This warning is for project developers. Use -Wno-dev to suppress it.
CMake Warning (dev) at _deps/llama_cpp-src/CMakeLists.txt:42 (option):
Policy CMP0077 is not set: option() honors normal variables. Run "cmake
--help-policy CMP0077" for policy details. Use the cmake_policy command to
set the policy and suppress this warning.
For compatibility with older versions of CMake, option is clearing the
normal variable 'LLAMA_LTO'.
This warning is for project developers. Use -Wno-dev to suppress it.
CMake Warning (dev) at _deps/llama_cpp-src/CMakeLists.txt:55 (option):
Policy CMP0077 is not set: option() honors normal variables. Run "cmake
--help-policy CMP0077" for policy details. Use the cmake_policy command to
set the policy and suppress this warning.
For compatibility with older versions of CMake, option is clearing the
normal variable 'LLAMA_AVX'.
This warning is for project developers. Use -Wno-dev to suppress it.
CMake Warning (dev) at _deps/llama_cpp-src/CMakeLists.txt:56 (option):
Policy CMP0077 is not set: option() honors normal variables. Run "cmake
--help-policy CMP0077" for policy details. Use the cmake_policy command to
set the policy and suppress this warning.
For compatibility with older versions of CMake, option is clearing the
normal variable 'LLAMA_AVX2'.
This warning is for project developers. Use -Wno-dev to suppress it.
CMake Warning (dev) at _deps/llama_cpp-src/CMakeLists.txt:57 (option):
Policy CMP0077 is not set: option() honors normal variables. Run "cmake
--help-policy CMP0077" for policy details. Use the cmake_policy command to
set the policy and suppress this warning.
For compatibility with older versions of CMake, option is clearing the
normal variable 'LLAMA_AVX512'.
This warning is for project developers. Use -Wno-dev to suppress it.
CMake Warning (dev) at _deps/llama_cpp-src/CMakeLists.txt:60 (option):
Policy CMP0077 is not set: option() honors normal variables. Run "cmake
--help-policy CMP0077" for policy details. Use the cmake_policy command to
set the policy and suppress this warning.
For compatibility with older versions of CMake, option is clearing the
normal variable 'LLAMA_FMA'.
This warning is for project developers. Use -Wno-dev to suppress it.
CMake Warning (dev) at _deps/llama_cpp-src/CMakeLists.txt:67 (option):
Policy CMP0077 is not set: option() honors normal variables. Run "cmake
--help-policy CMP0077" for policy details. Use the cmake_policy command to
set the policy and suppress this warning.
For compatibility with older versions of CMake, option is clearing the
normal variable 'LLAMA_ACCELERATE'.
This warning is for project developers. Use -Wno-dev to suppress it.
-- Found Git: /usr/bin/git (found version "2.25.1")
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- x86 detected
-- Working on magic_enum
-- Configuring done
CMake Error in CMakeLists.txt:
The CXX_STANDARD property on target "minigpt4" contained an invalid value:
"23".
CMake Generate step failed. Build files cannot be regenerated correctly.
Any idea what is going on?
Is building from source necessary to get optimal performance for my system's architecture?
I noticed that the files uploaded to maknee/minigpt4-7b-ggml are actually the same that were uploaded to maknee/minigpt4-13b-ggml.
Also confirmed by running the model. It identifies minigpt4-7b-ggml
with Model type: Vicuna13B
.
Hello!
thank you for this.
is there any chance getting a gui without needing to install python like koboldcpp in the future for its portability?
also, can we use this with other models like wizardlm too?
kind regards
Is it possible to offload some of the computation (on the LLM) side to the GPU? As with llama.cpp?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.