rupeshs / alpaca.cpp Goto Github PK
View Code? Open in Web Editor NEWThis project forked from antimatter15/alpaca.cpp
Locally run an Instruction-Tuned Chat-Style LLM (Android/Linux/Windows/Mac)
License: MIT License
This project forked from antimatter15/alpaca.cpp
Locally run an Instruction-Tuned Chat-Style LLM (Android/Linux/Windows/Mac)
License: MIT License
I followed the instructions in the documentation on my Android phone, and I got the following error, it seems that the first step of mkdir is not in the right place, I hope someone will guide me how to solve it
CMake Error: The source directory "/data/data/com.termux/files/home" does not appear to contain CMakeLists.txt.
Specify --help for usage, or press the help button on the CMake GUI.
Hi! Thank you for the repo!
Would it be possible to run a 2 bit version?
I am building for Android on Manjaro. The command line ./chat works. It builds for Linux and runs. When I try to build for Android following the steps in the repo I get this error when I run the cmake command in bash:
CMake Error at /usr/share/cmake/Modules/CMakeTestCCompiler.cmake:70 (message):
The C compiler
"/home/simonl/bin/build/android-ndk-r25c/toolchains/llvm/prebuilt/linux-x86_64/bin/clang"
is not able to compile a simple test program.
It fails with the following output:
Change Dir: /home/simonl/bin/build/alpaca.cpp/build-android/CMakeFiles/CMakeScratch/TryCompile-AE6Oa8
Run Build Command(s):/usr/bin/make -f Makefile cmTC_3abbb/fast && /usr/bin/make -f CMakeFiles/cmTC_3abbb.dir/build.make CMakeFiles/cmTC_3abbb.dir/build
make[1]: Entering directory '/home/simonl/bin/build/alpaca.cpp/build-android/CMakeFiles/CMakeScratch/TryCompile-AE6Oa8'
Building C object CMakeFiles/cmTC_3abbb.dir/testCCompiler.c.o
/home/simonl/bin/build/android-ndk-r25c/toolchains/llvm/prebuilt/linux-x86_64/bin/clang -g -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -march=armv8.4a+dotprod -o CMakeFiles/cmTC_3abbb.dir/testCCompiler.c.o -c /home/simonl/bin/build/alpaca.cpp/build-android/CMakeFiles/CMakeScratch/TryCompile-AE6Oa8/testCCompiler.c
/home/simonl/bin/build/android-ndk-r25c/toolchains/llvm/prebuilt/linux-x86_64/bin/clang: line 1: clang-14: command not found
make[1]: *** [CMakeFiles/cmTC_3abbb.dir/build.make:78: CMakeFiles/cmTC_3abbb.dir/testCCompiler.c.o] Error 127
make[1]: Leaving directory '/home/simonl/bin/build/alpaca.cpp/build-android/CMakeFiles/CMakeScratch/TryCompile-AE6Oa8'
make: *** [Makefile:127: cmTC_3abbb/fast] Error 2
CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
CMakeLists.txt:2 (project)
Hey there, I have followed every instruction you have given and built the binaries. However when I try to run them from termux home I get this Exec format error. I have given them appropriate executable permissions. How do I solve this problem?
I made a simple web page using flask in Python but I'm having trouble sending and receiving to/from the interactive shell. I'm calling chat.exe with subprocess.run() but the output is always only the first 3 lines of chat.exe's output, or just an unknown error. I'm using -p to submit the prompt, but I can't tell if it works or not because I don't get the full output.
Any help would be very much appreciated!
Python script for reference(simplified the command to just --help, still doesn't work):
from flask import Flask, request, jsonify
import subprocess
import time
import logging
app = Flask(__name__)
# Set up logging
logging.basicConfig(level=logging.DEBUG, format='%(asctime)s - %(levelname)s - %(message)s')
html_code = '''
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Interactive Shell</title>
<style>
#output {
background-color: blue;
padding: 10px;
color: white;
}
</style>
</head>
<body>
<textarea id="commandInput" placeholder="Enter command"></textarea>
<button onclick="executeCommand()">Execute</button>
<div id="output"></div>
<script>
function executeCommand() {
const command = document.getElementById('commandInput').value;
fetch('/execute-command', {
method: 'POST',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify({ command: command })
})
.then(response => response.json())
.then(data => {
// Check for the 'execution_complete' signal
if (data.execution_complete) {
document.getElementById('output').innerText = data.output || data.error || "Error executing command!";
} else {
// If execution is not complete, recursively call executeCommand after a short delay
setTimeout(executeCommand, 500);
}
});
}
</script>
</body>
</html>
'''
@app.route('/')
def index():
return html_code
@app.route('/execute-command', methods=['POST'])
def execute_command():
try:
command = ['C:\\Users\\Intel NUC\\Desktop\\alpaca\\alpaca.cpp\\build\\Release\\chat.exe', '--help']
result = subprocess.run(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, check=False)
output = result.stdout if result.returncode == 0 else result.stderr
response_data = {'output': output, 'execution_complete': True}
# Log any errors to the console
if result.returncode != 0:
logging.error(f"Error executing command: {output}")
return jsonify(response_data)
except Exception as e:
# Log any exceptions to the console
logging.exception(f"An exception occurred: {str(e)}")
return jsonify({'error': str(e), 'execution_complete': True})
if __name__ == '__main__':
app.run(debug=True)
Edit: I noticed the -i, --interactive flag exists, but I'm a bit confused; isn't an interactive shell the default? I want the opposite of that so I can pipe the output somewhere.
I followed all the steps, copied the "build-android" folder with the "chat" and the other 5 files to my internal storage. The model is also in the same folder. However, when I navigate to the "build-android" folder using the "cd" command in Termux and try to execute "./chat", I receive the error message "permission denied".
Sorry if this is not the right place to ask, but I'm trying to follow the guide to build for android.
I'm on Windows 10 so I have used SET instead of export for the path variable, and %NKD% instead of $NKS.
I have also tried to just paste the direct path instead of using the variable.
I have never used cmake before. Got other llama/alpaca projects to work.
I would apreceate any help, thank you!
C:\Users\AE\Downloads\alpaca.cpp-master\alpaca.cpp-master\build-android>cmake -DCMAKE_TOOLCHAIN_FILE=%NDK%\build\cmake\android.toolchain.cmake -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-23 -DCMAKE_C_FLAGS=-march=armv8.4a+dotprod ..
CMake Error at CMakeLists.txt:2 (project):
Failed to run MSBuild command:
C:/Program Files/Microsoft Visual Studio/2022/Community/MSBuild/Current/Bin/amd64/MSBuild.exe
to get the value of VCTargetsPath:
MSBuild version 17.5.1+f6fdcf537 for .NET Framework
Der Buildvorgang wurde am 23.03.2023 19:41:40 gestartet.
Included response file: C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Current\Bin\amd64\MSBuild.rsp
Projekt "C:\Users\AE\Downloads\alpaca.cpp-master\alpaca.cpp-master\build-android\CMakeFiles\3.26.1\VCTargetsPath.vcxproj" auf Knoten "1" (Standardziele).
C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Current\Bin\amd64\Microsoft.Common.CurrentVersion.targets(832,5): error : The BaseOutputPath/OutputPath property is not set for project 'VCTargetsPath.vcxproj'. Please check to make sure that you have specified a valid combination of Configuration and Platform for this project. Configuration='Debug' Platform='x64'. You may be seeing this message because you are trying to build a project without a solution file, and have specified a non-default Configuration or Platform that doesn't exist for this project. [C:\Users\AE\Downloads\alpaca.cpp-master\alpaca.cpp-master\build-android\CMakeFiles\3.26.1\VCTargetsPath.vcxproj]
Die Erstellung des Projekts "C:\Users\AE\Downloads\alpaca.cpp-master\alpaca.cpp-master\build-android\CMakeFiles\3.26.1\VCTargetsPath.vcxproj" ist abgeschlossen (Standardziele) -- FEHLER.
Fehler beim Buildvorgang. == ERROR DURING BUILDING PROCESS
"C:\Users\AE\Downloads\alpaca.cpp-master\alpaca.cpp-master\build-android\CMakeFiles\3.26.1\VCTargetsPath.vcxproj" (Standardziel) (1) ->
(_CheckForInvalidConfigurationAndPlatform Ziel) ->
C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Current\Bin\amd64\Microsoft.Common.CurrentVersion.targets(832,5): error : The BaseOutputPath/OutputPath property is not set for project 'VCTargetsPath.vcxproj'. Please check to make sure that you have specified a valid combination of Configuration and Platform for this project. Configuration='Debug' Platform='x64'. You may be seeing this message because you are trying to build a project without a solution file, and have specified a non-default Configuration or Platform that doesn't exist for this project. [C:\Users\AE\Downloads\alpaca.cpp-master\alpaca.cpp-master\build-android\CMakeFiles\3.26.1\VCTargetsPath.vcxproj]
0 Warnung(en) ==WARNINGS
1 Fehler == ERRORS
Verstrichene Zeit 00:00:01.31
Exit code: 1
-- Configuring incomplete, errors occurred!
(I know the path alpaca.cpp-master\alpaca.cpp-master\ is source for many headaches, but as I made multiple sub folders to get everything to work, I stuck with it. Sorry!)
Hello, I would like to change the "ggml-alpaca-7b-q4.bin" model to a Polish-only model, what other model can I use?
Hi
I installed it on Windows.
Now how do I launch the chat?
See antimatter15#133 for the Dockerfile
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.