rupeshs / alpaca.cpp Goto Github PK

View Code? Open in Web Editor NEW

This project forked from antimatter15/alpaca.cpp

259.0 259.0 24.0 1.48 MB

Locally run an Instruction-Tuned Chat-Style LLM (Android/Linux/Windows/Mac)

License: MIT License

Shell 0.07% C++ 16.92% Python 1.23% C 79.60% Makefile 1.16% CMake 1.01%

alpaca.cpp's People

Contributors

Stargazers

Watchers

alpaca.cpp's Issues

change the "ggml-alpaca-7b-q4.bin" model

Hello, I would like to change the "ggml-alpaca-7b-q4.bin" model to a Polish-only model, what other model can I use?

cmake error on Android

I followed the instructions in the documentation on my Android phone, and I got the following error, it seems that the first step of mkdir is not in the right place, I hope someone will guide me how to solve it

CMake Error: The source directory "/data/data/com.termux/files/home" does not appear to contain CMakeLists.txt.
Specify --help for usage, or press the help button on the CMake GUI.

Build for Android copmiler error

I am building for Android on Manjaro. The command line ./chat works. It builds for Linux and runs. When I try to build for Android following the steps in the repo I get this error when I run the cmake command in bash:

CMake Error at /usr/share/cmake/Modules/CMakeTestCCompiler.cmake:70 (message):
  The C compiler

    "/home/simonl/bin/build/android-ndk-r25c/toolchains/llvm/prebuilt/linux-x86_64/bin/clang"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: /home/simonl/bin/build/alpaca.cpp/build-android/CMakeFiles/CMakeScratch/TryCompile-AE6Oa8
    
    Run Build Command(s):/usr/bin/make -f Makefile cmTC_3abbb/fast && /usr/bin/make  -f CMakeFiles/cmTC_3abbb.dir/build.make CMakeFiles/cmTC_3abbb.dir/build
    make[1]: Entering directory '/home/simonl/bin/build/alpaca.cpp/build-android/CMakeFiles/CMakeScratch/TryCompile-AE6Oa8'
    Building C object CMakeFiles/cmTC_3abbb.dir/testCCompiler.c.o
    /home/simonl/bin/build/android-ndk-r25c/toolchains/llvm/prebuilt/linux-x86_64/bin/clang   -g -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -march=armv8.4a+dotprod  -o CMakeFiles/cmTC_3abbb.dir/testCCompiler.c.o -c /home/simonl/bin/build/alpaca.cpp/build-android/CMakeFiles/CMakeScratch/TryCompile-AE6Oa8/testCCompiler.c
    /home/simonl/bin/build/android-ndk-r25c/toolchains/llvm/prebuilt/linux-x86_64/bin/clang: line 1: clang-14: command not found
    make[1]: *** [CMakeFiles/cmTC_3abbb.dir/build.make:78: CMakeFiles/cmTC_3abbb.dir/testCCompiler.c.o] Error 127
    make[1]: Leaving directory '/home/simonl/bin/build/alpaca.cpp/build-android/CMakeFiles/CMakeScratch/TryCompile-AE6Oa8'
    make: *** [Makefile:127: cmTC_3abbb/fast] Error 2
    
    

  

  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:2 (project)

How to Interact with the Shell Outside of the Console

I made a simple web page using flask in Python but I'm having trouble sending and receiving to/from the interactive shell. I'm calling chat.exe with subprocess.run() but the output is always only the first 3 lines of chat.exe's output, or just an unknown error. I'm using -p to submit the prompt, but I can't tell if it works or not because I don't get the full output.

Any help would be very much appreciated!

Python script for reference(simplified the command to just --help, still doesn't work):

from flask import Flask, request, jsonify
import subprocess
import time
import logging

app = Flask(__name__)

# Set up logging
logging.basicConfig(level=logging.DEBUG, format='%(asctime)s - %(levelname)s - %(message)s')

html_code = '''
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Interactive Shell</title>
    <style>
        #output {
            background-color: blue;
            padding: 10px;
            color: white;
        }
    </style>
</head>
<body>
    <textarea id="commandInput" placeholder="Enter command"></textarea>
    <button onclick="executeCommand()">Execute</button>
    <div id="output"></div>
    <script>
        function executeCommand() {
            const command = document.getElementById('commandInput').value;

            fetch('/execute-command', {
                method: 'POST',
                headers: {
                    'Content-Type': 'application/json'
                },
                body: JSON.stringify({ command: command })
            })
            .then(response => response.json())
            .then(data => {
                // Check for the 'execution_complete' signal
                if (data.execution_complete) {
                    document.getElementById('output').innerText = data.output || data.error || "Error executing command!";
                } else {
                    // If execution is not complete, recursively call executeCommand after a short delay
                    setTimeout(executeCommand, 500);
                }
            });
        }
    </script>
</body>
</html>
'''

@app.route('/')
def index():
    return html_code

@app.route('/execute-command', methods=['POST'])
def execute_command():
    try:
        command = ['C:\\Users\\Intel NUC\\Desktop\\alpaca\\alpaca.cpp\\build\\Release\\chat.exe', '--help']
        result = subprocess.run(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, check=False)

        output = result.stdout if result.returncode == 0 else result.stderr

        response_data = {'output': output, 'execution_complete': True}

        # Log any errors to the console
        if result.returncode != 0:
            logging.error(f"Error executing command: {output}")

        return jsonify(response_data)
    except Exception as e:
        # Log any exceptions to the console
        logging.exception(f"An exception occurred: {str(e)}")
        return jsonify({'error': str(e), 'execution_complete': True})

if __name__ == '__main__':
    app.run(debug=True)

Edit: I noticed the -i, --interactive flag exists, but I'm a bit confused; isn't an interactive shell the default? I want the opposite of that so I can pipe the output somewhere.

Dockerfile provided

See antimatter15#133 for the Dockerfile

how launch the chat?

Hi
I installed it on Windows.
Now how do I launch the chat?

2 bits weights

Hi! Thank you for the repo!

Would it be possible to run a 2 bit version?

Advice to build for Android under Windows

Sorry if this is not the right place to ask, but I'm trying to follow the guide to build for android.
I'm on Windows 10 so I have used SET instead of export for the path variable, and %NKD% instead of $NKS.
I have also tried to just paste the direct path instead of using the variable.

I have never used cmake before. Got other llama/alpaca projects to work.

I would apreceate any help, thank you!

C:\Users\AE\Downloads\alpaca.cpp-master\alpaca.cpp-master\build-android>cmake -DCMAKE_TOOLCHAIN_FILE=%NDK%\build\cmake\android.toolchain.cmake -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-23 -DCMAKE_C_FLAGS=-march=armv8.4a+dotprod ..
CMake Error at CMakeLists.txt:2 (project):
Failed to run MSBuild command:

C:/Program Files/Microsoft Visual Studio/2022/Community/MSBuild/Current/Bin/amd64/MSBuild.exe

to get the value of VCTargetsPath:

MSBuild version 17.5.1+f6fdcf537 for .NET Framework
Der Buildvorgang wurde am 23.03.2023 19:41:40 gestartet.
Included response file: C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Current\Bin\amd64\MSBuild.rsp

Projekt "C:\Users\AE\Downloads\alpaca.cpp-master\alpaca.cpp-master\build-android\CMakeFiles\3.26.1\VCTargetsPath.vcxproj" auf Knoten "1" (Standardziele).
C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Current\Bin\amd64\Microsoft.Common.CurrentVersion.targets(832,5): error : The BaseOutputPath/OutputPath property is not set for project 'VCTargetsPath.vcxproj'.  Please check to make sure that you have specified a valid combination of Configuration and Platform for this project.  Configuration='Debug'  Platform='x64'.  You may be seeing this message because you are trying to build a project without a solution file, and have specified a non-default Configuration or Platform that doesn't exist for this project. [C:\Users\AE\Downloads\alpaca.cpp-master\alpaca.cpp-master\build-android\CMakeFiles\3.26.1\VCTargetsPath.vcxproj]
Die Erstellung des Projekts "C:\Users\AE\Downloads\alpaca.cpp-master\alpaca.cpp-master\build-android\CMakeFiles\3.26.1\VCTargetsPath.vcxproj" ist abgeschlossen (Standardziele) -- FEHLER.

Fehler beim Buildvorgang. == ERROR DURING BUILDING PROCESS

"C:\Users\AE\Downloads\alpaca.cpp-master\alpaca.cpp-master\build-android\CMakeFiles\3.26.1\VCTargetsPath.vcxproj" (Standardziel) (1) ->
(_CheckForInvalidConfigurationAndPlatform Ziel) ->
  C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Current\Bin\amd64\Microsoft.Common.CurrentVersion.targets(832,5): error : The BaseOutputPath/OutputPath property is not set for project 'VCTargetsPath.vcxproj'.  Please check to make sure that you have specified a valid combination of Configuration and Platform for this project.  Configuration='Debug'  Platform='x64'.  You may be seeing this message because you are trying to build a project without a solution file, and have specified a non-default Configuration or Platform that doesn't exist for this project. [C:\Users\AE\Downloads\alpaca.cpp-master\alpaca.cpp-master\build-android\CMakeFiles\3.26.1\VCTargetsPath.vcxproj]

    0 Warnung(en) ==WARNINGS
    1 Fehler  == ERRORS

Verstrichene Zeit 00:00:01.31

Exit code: 1

-- Configuring incomplete, errors occurred!

(I know the path alpaca.cpp-master\alpaca.cpp-master\ is source for many headaches, but as I made multiple sub folders to get everything to work, I stuck with it. Sorry!)

cant run with ./chat

I followed all the steps, copied the "build-android" folder with the "chat" and the other 5 files to my internal storage. The model is also in the same folder. However, when I navigate to the "build-android" folder using the "cd" command in Termux and try to execute "./chat", I receive the error message "permission denied".

Android execution in termux results in Exec format error

Hey there, I have followed every instruction you have given and built the binaries. However when I try to run them from termux home I get this Exec format error. I have given them appropriate executable permissions. How do I solve this problem?

rupeshs / alpaca.cpp Goto Github PK

alpaca.cpp's People

Contributors

Stargazers

Watchers

Forkers

alpaca.cpp's Issues

change the "ggml-alpaca-7b-q4.bin" model

cmake error on Android

Build for Android copmiler error

How to Interact with the Shell Outside of the Console

Dockerfile provided

how launch the chat?

2 bits weights

Advice to build for Android under Windows

cant run with ./chat

Android execution in termux results in Exec format error

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent