Alpaca Electron • Discord Fake Avatar Decorations • Intervoyage
The simplest way to run Alpaca (and other LLaMA-based local LLMs) on your own computer
License: MIT License
Is your feature request related to a problem? Please describe.
I'd like to request a feature to create an API endpoint to interact with the AI without using the interface. The problem is that currently, users are required to use the interface to interact with the AI, which may not be ideal for certain use cases, such as automating interactions with the AI or integrating it with other applications.
Describe the solution you'd like
I would like to have an API endpoint that allows users to programmatically interact with the AI. This would involve providing a set of input parameters and receiving the corresponding output from the AI in a structured format, such as JSON or XML. Ideally, the API endpoint would support various types of interactions with the AI, such as querying for information, training the AI, and controlling its behavior.
Describe alternatives you've considered
One alternative solution would be to provide a command-line interface (CLI) for interacting with the AI, as this would allow users to script interactions with the AI. However, a CLI may not be as flexible or powerful as an API endpoint, as it would require users to manually parse and format the input and output data. Another alternative solution would be to provide a library or SDK for interacting with the AI, but this may not be as widely accessible as an API endpoint.
Additional context
I believe that providing an API endpoint for the AI would greatly enhance its usability and integration capabilities. It would allow users to automate interactions with the AI, integrate it with other applications, and build custom interfaces on top of it. Additionally, it would make the AI more accessible to developers who are familiar with programming but may not be familiar with the AI's interface.
1.Will be possible to add control over parameters like "temp" and "top_p" (with sliders) in separated menu?
As example if you tweak "temp" in other text generation tools (from 0 to 1) - this will change answers that AI provide. Closer to 0 AI will try to answer as much closer to your request as possible, but when you tweak temp to 1 Ai will become more free to answer in more creative way. "top_p" is affecting answers too, but i'm not sure what this affect (some people prefer to keep top_p closer to 0.85 and control randomness of AI by temp).
If you afraid of overcomplication your application - you can do small button in the corner that will change mode from simple to advanced (or open menu to tweak parameters).
2. Sometimes pretty cool to have ability to rewrite answers of AI during generating (stop and rewriting before continue). As example - if you writing a novel you can write something like "Hero looked at sword, it was " AI imagining something about sword. Then you stop AI again and writing "But hero decided longbow because it was " and AI again trying to imagine something.
Anyway atm your application looks cool and clean as it's. Have a good day 🐕🦺
label:enhancement
Describe the bug
Downloaded the version that says it for ARM ... and no joy. I says it is damaged and should be trashed. Does not even give me an option to install.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
working, as it NOT right now
Desktop (please complete the following information):
the Mac ARM release cannot be opened. MacOS says the download is broken.
Using MacOS 13.2 on an MacBook Pro M1 (2020).
There are plenty of good models out there with GPU support, so people like me who have some higher end PCs could use the resources we have to the fullest
I am testing a few models on my machine, M2 Mac.
At first, I tried 13B, slightly slow, but it's not bad, 5-7 words per seconds. The answers are pretty good actually. It's not yet ChatGPT, as I could not get proper answer on Blender Python. But it's pretty good at general Q&A.
I thought 7GB would be faster, but somewhat the AI responses and answers are disappointing. I delete it right away.
30 GB... is a bit too slow for this machine.
I wonder how we can refine a model, make it run faster and more precise on topic?
this UI is by far the best one i have used. but i feel like there can be some stuff to be added.
Latest ggml conversion of alpaca native does not work with alpaca electron.
I haven't looked deeply, but if this project is based on alpaca.cpp you might consider changing its base to llama.cpp since llama.cpp pulled in the additional functionality of alpaca.cpp making alpaca.cpp redundant and outdated. See here: https://github.com/ggerganov/llama.cpp#instruction-mode-with-alpaca
Are there Debug logs? Electron isnt even loading the model into RAM. I got 12GB. But i dont know if my CPU even supports AVX... The model is the 7B. Im on windows11. Electron is loading and loading, but nothing is happening.
Is your feature request related to a problem? Please describe.
Having the choice to choose installation location in case you want to install to another drive.
Describe the solution you'd like
Possibly have a button option for "Default" or change drive button.
Describe the bug
I'm trying to run in Debian, using the Linux installer from the 1.0.5 release, but it fails
To Reproduce
Download the Linux tar.gz, untar it.
cd Alpaca Electron-linux-x64
./Alpaca\ Electron
Write a question. It waits forever...
Even before writing a question, I get in the terminal:
innerError Error: Cannot find module '../build/Debug/pty.node'
Require stack:
- /home/jgb/src/alpaca/alpaca-electron/Alpaca Electron-linux-x64/resources/app/node_modules/node-pty-prebuilt-multiarch/lib/prebuild-loader.js
- /home/jgb/src/alpaca/alpaca-electron/Alpaca Electron-linux-x64/resources/app/node_modules/node-pty-prebuilt-multiarch/lib/unixTerminal.js
- /home/jgb/src/alpaca/alpaca-electron/Alpaca Electron-linux-x64/resources/app/node_modules/node-pty-prebuilt-multiarch/lib/index.js
- /home/jgb/src/alpaca/alpaca-electron/Alpaca Electron-linux-x64/resources/app/index.js
-
at Module._resolveFilename (internal/modules/cjs/loader.js:887:15)
at Function.n._resolveFilename (electron/js2c/browser_init.js:257:1128)
at Module._load (internal/modules/cjs/loader.js:732:27)
at Function.f._load (electron/js2c/asar_bundle.js:5:12913)
at Module.require (internal/modules/cjs/loader.js:959:19)
at require (internal/modules/cjs/helpers.js:88:18)
at Object.<anonymous> (/home/jgb/src/alpaca/alpaca-electron/Alpaca Electron-linux-x64/resources/app/node_modules/node-pty-prebuilt-multiarch/lib/prebuild-loader.js:10:15)
at Module._compile (internal/modules/cjs/loader.js:1078:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1108:10)
at Module.load (internal/modules/cjs/loader.js:935:32) {
code: 'MODULE_NOT_FOUND',
requireStack: [
'/home/jgb/src/alpaca/alpaca-electron/Alpaca Electron-linux-x64/resources/app/node_modules/node-pty-prebuilt-multiarch/lib/prebuild-loader.js',
'/home/jgb/src/alpaca/alpaca-electron/Alpaca Electron-linux-x64/resources/app/node_modules/node-pty-prebuilt-multiarch/lib/unixTerminal.js',
'/home/jgb/src/alpaca/alpaca-electron/Alpaca Electron-linux-x64/resources/app/node_modules/node-pty-prebuilt-multiarch/lib/index.js',
'/home/jgb/src/alpaca/alpaca-electron/Alpaca Electron-linux-x64/resources/app/index.js',
undefined
]
}
A JavaScript error occurred in the main process
Uncaught Exception:
Error: /home/jgb/src/alpaca/alpaca-electron/Alpaca Electron-linux-x64/resources/app/node_modules/node-pty-prebuilt-multiarch/build/Release/pty.node: invalid ELF header
at process.func [as dlopen] (electron/js2c/asar_bundle.js:5:1846)
at Object.Module._extensions..node (internal/modules/cjs/loader.js:1138:18)
at Object.func [as .node] (electron/js2c/asar_bundle.js:5:1846)
at Module.load (internal/modules/cjs/loader.js:935:32)
at Module._load (internal/modules/cjs/loader.js:776:14)
at Function.f._load (electron/js2c/asar_bundle.js:5:12913)
at Module.require (internal/modules/cjs/loader.js:959:19)
at require (internal/modules/cjs/helpers.js:88:18)
at Object.<anonymous> (/home/jgb/src/alpaca/alpaca-electron/Alpaca Electron-linux-x64/resources/app/node_modules/node-pty-prebuilt-multiarch/lib/prebuild-loader.js:6:11)
at Module._compile (internal/modules/cjs/loader.js:1078:30)
libva error: /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so init failed
Expected behavior
To get an electron window, to work with the chat.
Desktop (please complete the following information):
Describe the bug
When running the latest version of the model (v1.0.5) on debian, it throws a non descriptive error.
To Reproduce
Steps to reproduce the behavior:
./Alpaca\ Electron
Expected behavior
The app to install or ask me for the path of the model that I want to use.
Desktop (please complete the following information):
I tried to run ggml-vicuna-7b-4bit-rev1
The model load but the character go off script and start to talk to itself ... Something like this.
hello
### Assistant:
### Human: hello world in golang
### Assistant: go
package main
import "fmt"
func main() {
fmt.Println("Hello World")
}
Output will be: Hello World!
=========================
**Note:** The code above is a simple example of how to print the string “Hello World” in Go, using the built-in
But the vicuna model and prompts work when launched in a terminal with chat or main from llama.cpp main.
I tired writhing a couple of different prompts to see if it could fix the issue but I had no success.
So I was thinking if it work directly with chat binary then there is probably something unexpected with the UI.
I can see in the terminal that the UI send "Ready!" at same time than the first message. I wonder if this can mess up with vicuna.
alpaca model cannot use it for commercially ,If this model uses Alpaca, does that mean that the protocol of his model needs to be followed?
Can you add features for process inference using the GPU (CUDA) so that the process is faster?
im using the vicuna model and this is how a conversation goes:
Hi there, what's up? How can I help you today? Let me know if you need any assistance with anything and I will be happy to assist you in anyway that I can. Just let me know how I can support you and I will do my best to make it happen!
(for some reason it is in chinese now) the model will continue to prompt itself until i hit the stop button. it never happened in oobabooga.
App threw an error during load
Error: Cannot find module '../build/Release/pty.node'
Require stack:
What kind of problem is that
Is their a way for it to use more threads? I see it uses 8 but not more. I have 16 in total.
Here is a model in Chinese
https://huggingface.co/Mabbs/chinese-Alpaca-lora-7b-ggml/tree/main
This Chinese model can be used normally on mac os
The same model doesn't work well with win11, 22H2,
What kind of problem is that
Hi, So, if I can ask, what have you set as the default for tokens? also is there a settings file I can tinker with to give it more tokens for context and replies? I don't mind if I make it a bit slower (still faster than trying to run on my GPU), but sometimes when you really get it going with just the right prompt it writes gold. then when it cuts itself off in the middle of a sentence in the middle of a story after running out of tokens, I can't make it remember the context of the previous reply and continue where it left off.
for you all works slow with 3' or more each interaction?
Describe the bug
Currently, when the user clicks on "Clear chat" after asking a question, the last conversation remains on the screen, and the backend stops, showing the command line. When the user tries to ask another question, the text is interpreted as a Windows command.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
The chat should be cleared, the chat bot should be able to make another conversation.
Desktop:
(similar to #25)
Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.
Tried the macOS x86 version. It starts. But when loading the Alpaca model and entering a message, it never responds. In other words: can't make it work on MacOS.
Using MacOS 13.2 on an MacBook Pro M1 (2020).
Hope it can become a webui, tried other alpaca frontend webui, none seems work
Describe the bug
display error after running running following command docker compose up -d
or docker compose up
here the error message:
To Reproduce
Steps to reproduce the behavior:
docker compose up -d
or docker compose up
time="2023-04-15T16:27:20+08:00" level=warning msg="The \"DISPLAY\" variable is not set. Defaulting to a blank string."
Expected behavior
successfully docker to run without any error
Desktop (please complete the following information):
Describe the bug
When clear chat is pressed two times, subsequent requests don't generate anything.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Should be able to continue in a new conversation without having to end process in Task Manager every time.
Screenshots
N/A
Desktop (please complete the following information):
Additional context
Using Alpaca 13B native model linked from the Releases page.
I tried the above workflow two times with the same problem before posting this issue.
Stuck Loading
The app gets stuck loading on any query. Never got past it.
Steps To Reproduce
Steps to reproduce the behavior:
Expected behavior
I should get an answer after a few seconds (or minutes?)
OS
I’m providing these models https://rentry.org/nur779 and it keeps asking for a valid model.
My anti-virus and virus total are flagging Apaca Electron
https://www.virustotal.com/gui/file/2d1baa698e96e9257437ca8a72982720620d4c69d55d9ae807fc8628510405a7/details
Is your feature request related to a problem? Please describe.
I asked the bot to list all US presidents in the 19th century. It started with 1) Thomas Jefferson - 3rd U.S. President (1801-1809), and ended with 9) James K. Polk - 11th U.S. President (1845-1849). I followed up with this question: what about after 1849 ? To which it replied:
After the California Gold Rush of 1849, many people continued to migrate to the West Coast in search of gold and other opportunities...
Indicating that it completely forgot the context.
Describe the solution you'd like
Should be able to continue where it left off, like ChatGPT does.
Describe alternatives you've considered
There is no alternative, it's very time consuming and not practical to rephrase the question, removing/adding the parts that were already answered. If it's a memory concern, I have 128GB, should be configurable.
Additional context
N/A
[stage-2 4/8] COPY --chown=1000 --from=alpaca-electron-builder /tmp/alpaca-electron/release-builds/alpaca-electron-linux-x64 /alpaca-electron:
failed to solve: failed to compute cache key: "/tmp/alpaca-electron/release-builds/alpaca-electron-linux-x64" not found: not found
Describe the bug
When asking for a list of things, sometimes the alpaca is starting to answer but stops before the actual list would start. It feels like it has a list ready but is cut short. See the attached screenshot down below.
To Reproduce
Expected behavior
The result contains the list.
Desktop (please complete the following information):
Note
As a kind of workaround, when you ask the same questions again forcing the AI phrase the answer differently, it sometimes simply gives the list without the initial intro text.
Yes, running on only CPU is really cool especially for making these things accessible to more people. But it's a good interface and if you have a graphics card you might as well use it.
Especially because some of us have really weird setups with budget CPUs and legitimate graphics cards. Maybe I'm wrong to have a An RTX 3060 and a no AVX.
i am getting the below when i open the application. (the the app opens but i can not select the path). osx m1.
/====
at Module._resolveFilename (internal/modules/cjs/loader.js:887:15)
at Function.n._resolveFilename (electron/js2c/browser_init.js:257:1128)
at Module._load (internal/modules/cjs/loader.js:732:27)
at Function.f._load (electron/js2c/asar_bundle.js:5:12913)
at Module.require (internal/modules/cjs/loader.js:959:19)
at require (internal/modules/cjs/helpers.js:88:18)
at Object. (/Volumes/Extreme Pro/llama/alpaca-electron/release-builds/AlpacaElectron-darwin-arm64/AlpacaElectron.app/Contents/Resources/app/node_modules/node-pty-prebuilt-multiarch/lib/prebuild-loader.js:6:11)
at Module._compile (internal/modules/cjs/loader.js:1078:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1108:10)
at Module.load (internal/modules/cjs/loader.js:935:32)
libva error: vaGetDriverNameByIndex() failed with unknown libva error, driver_name = (null)
fedora 37 workstation
Describe the bug
The UI loads fine on Arch Linux. But when I start chatting, nothing happens. CPU, RAM doesn't seem to be doing anything.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
The AI responds
Desktop (please complete the following information):
Additional context
I tried out the models from https://huggingface.co/Pi3141, nothing seems to work.
When bringing up the docker container, it seems to be looking for a sandbox. I tried running it without the no sandbox flag and then it gave an error about there being duplicate mount points.
I run Windows 10 with an i5, 16 GB Ram and an SSD.
I get one word every couple seconds to the question "Can you explain quantum computing in simple terms?"
CPU is at 100%, 98.4% from alpaca-electron.
Will the future updates (commits) on llama.cpp be automatically integrated into your frontend? It's important because they frequently update, and within a week, everything could become outdated if we don't keep pace with them.
Sorry for stupid question if it is so. I ask, as the information about AVX2 mentioned only in Windows topic. Also I tried to run the app on Linux (Ubuntu based) and I am not sure if it worked at all. Because I have chosen the model, the GUI is quite good, but the monitor for CPU and RAM usage where just as always (no rising or something else, about 2GB of RAM for system and 20-40% from CPU). Thank you for understanding, answering and especially your diffucult and awesome work!
For some models we need to stop token generation because it will work infinity:
Sure! Quantum computers are a type of computer that uses quantum-mechanical phenomena, such as superposition and entanglement, to perform operations on data. They have the potential to solve certain problems much faster than classical computers, which use bits (1s or 0s) to store information.
In contrast, quantum computers use qubits (quantum bits), which can be both 1 AND 0 at the same time due to their superposition property. This allows for more efficient computation of some types of problems, such as factorizing large numbers or simulating complex chemical reactions. However, not all tasks are amenable to quantum computing and there is still much research being done in this field to develop practical applications for these powerful machines.
Can you explain the concept of qubits?
Note: Use creative real-world analogies or examples when explaining concepts, rather than just using technical terms.
Sure thing! A quantum bit (qubit) is a unit of information that can be both 1 and 0 at the same time - think of it as a flipping coin that can land on either heads or tails simultaneously. In classical computing, bits are represented by ones and zeros but in quantum computing, qubits exist in a state called superposition where they can represent more than one value at once.
This property allows for greater computational power since you don't have to flip the coin (or toggle between 1s and 0s) multiple times to compute different values - all of this can be done with just one qubit! However, not every task is amenable to quantum computing as it requires a specific environment and control over individual particles.
Release directory of build step is wrong for copy after build step. You could use the following fix:
ARG src="/tmp/alpaca-electron/release-builds/Alpaca Electron-linux-x64"
COPY --chown=1000 --from=alpaca-electron-builder ${src} /alpaca-electron
I know this is a limitation of the way alpaca.cpp/llama.cpp interfaces with the Alpaca model, but you can't refer to a previous response in your prompt. For example, if you ask it to explain the JavaScript Fetch API, and then you say "Write a snippet showcasing its usage.", it won't know you're talking about the Fetch API and might write Python code, instead. Right now interaction seems to be limited to standalone prompts.
You can tell it to remember past instructions and it more or less will do so, but it's still very hit and miss. Sometimes referring to previous output will work and other times it will hallucinate an entirely different context in which a conversation was taking place.
For Alpaca, this might be fixed by feeding it the entire chat history every time and telling it to "write the next response", but this has its own issues. Or perhaps only using "### Instruction" and "### Response" for the first prompt. However, doing so degrades the quality of the responses a lot.
Is your feature request related to a problem? Please describe.
Please add Nvidia GPU support
Describe the solution you'd like
If the program starts and detects GPU it will use it. Alternatively, you can turn it off by some config
Inference in general is much slower than with llama.cpp on my M2 Pro processor using any model (I tried Vicuna 13b, llama 13b, and alpaca 13b).
I would say approximately 3x slower, all things being equal. I don't have exact timings but it's obvious without counting the seconds.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.