Code Monkey home page Code Monkey logo

oobabot's Introduction

oobabot

oobabot is a Discord bot which talks to a Large Language Model AIs (like LLaMA, llama.cpp, etc...), running on oobabooga's text-generation-webui.

python lint and test with poetry

Installation

  pip install oobabot

requires python 3.8+

Usage

$ oobabot --wakewords rosie cat --ai-name Rosie --persona "you are a cat named Rosie"

2023-05-04 00:24:10,968 DEBUG Oobabooga base URL: ws://localhost:5005
2023-05-04 00:24:11,133 INFO Connected to Oobabooga!
2023-05-04 00:24:11,133 DEBUG Connecting to Discord...
2023-05-04 00:24:13,807 INFO Connected to discord as RosieAI#0000 (ID: 1100100011101010101)
2023-05-04 00:24:13,807 DEBUG monitoring DMs, plus 24 channels across 1 server(s)
2023-05-04 00:24:13,807 DEBUG AI name: Rosie
2023-05-04 00:24:13,807 DEBUG AI persona: you are a cat named Rosie
2023-05-04 00:24:13,807 DEBUG wakewords: rosie, cat

See below for more details on the command line options.

Motivation

oobabot in action!

Text-generative UIs are cool to run at home, and Discord is fun to mess with your friends. Why not combine the two and have something awesome!

Real motivation: I wanted a chatbot in my discord that would act like my cat. A "catbot" if you will.

Features

oobabot how that's awesome
user-supplied persona you supply the persona on how would like the bot to behave
multiple conversations can track multiple conversational threads, and reply to each in a contextually appropriate way
watchwords can monitor all channels in a server for one or more wakewords or @-mentions
private conversations can chat with you 1:1 in a DM
good Discord hygiene splits messages into independent sentences, pings the author in the first one
low-latency streams the reply live, sentence by sentence. Provides lower latency, especially on longer responses.
stats track token generation speed, latency, failures and usage
easy networking connects to discord from your machine using websockets, so no need to expose a server to the internet
Stable Diffusion new in v0.1.4! Optional image generation with AUTOMATIC1111
Slash Commands coming in v0.1.6... did your bot get confused? /lobotomize it!

Getting Started with oobabot

See the Installation Guide for step-by-step instructions

Installation tl;dr

  1. Install oobabooga's text-generation-webui and enable its API module
  2. Create a Discord bot account, invite it to your server, and note its authentication token.
  3. Install oobabot (see INSTALL.md)
~: pip install oobabot

~: export DISCORD_TOKEN = __your_bots_discord_token__

~: oobabot --base-url ws://oobabooga-hostname:5005/ --ai-name YourBotsName --persona "You are a cat named YourBotsName"

You should now be able to run oobabot from wherever pip installed it.

usage: oobabot [-h] [-c CONFIG] [--generate-config] [--invite-url] [--ai-name AI_NAME]
               [--persona PERSONA] [--wakewords [WAKEWORDS ...]]
               [--discord-token DISCORD_TOKEN] [--dont-split-responses]
               [--history-lines HISTORY_LINES] [--ignore-dms] [--reply-in-thread]
               [--stream-responses] [--base-url BASE_URL] [--log-all-the-things]
               [--message-regex MESSAGE_REGEX] [--image-words [IMAGE_WORDS ...]]
               [--stable-diffusion-url STABLE_DIFFUSION_URL]
               [--extra-prompt-text EXTRA_PROMPT_TEXT]

oobabot v0.2.2: Discord bot for oobabooga's text-generation-webui

General Settings:

  -h, --help
  -c CONFIG, --config CONFIG
                        Path to a config file to read settings from. Command line
                        settings will override settings in this file. (default:
                        config.yml)
  --generate-config     If set, oobabot will print its configuration as a .yml file, then
                        exit. Any command-line settings also passed will be reflected in
                        this file. (default: False)
  --invite-url          Print a URL which can be used to invite the bot to a Discord
                        server. Requires that the Discord token is set. (default: False)

Persona:

  --ai-name AI_NAME     Name the AI will use to refer to itself (default: oobabot)
  --persona PERSONA     This prefix will be added in front of every user-supplied
                        request. This is useful for setting up a 'character' for the bot
                        to play. Alternatively, this can be set with the OOBABOT_PERSONA
                        environment variable. (default: )
  --wakewords [WAKEWORDS ...]
                        One or more words that the bot will listen for. The bot will
                        listen in all discord channels it can access for one of these
                        words to be mentioned, then reply to any messages it sees with a
                        matching word. The bot will always reply to @-mentions and direct
                        messages, even if no wakewords are supplied. (default:
                        ['oobabot'])

Discord:

  --discord-token DISCORD_TOKEN
                        Token to log into Discord with. For security purposes it's
                        strongly recommended that you set this via the DISCORD_TOKEN
                        environment variable instead, if possible. (default: )
  --dont-split-responses
                        Post the entire response as a single message, rather than
                        splitting it into separate messages by sentence. (default: False)
  --history-lines HISTORY_LINES
                        Number of lines of chat history the AI will see when generating a
                        response. (default: 7)
  --ignore-dms          If set, the bot will not respond to direct messages. (default:
                        False)
  --reply-in-thread     If set, the bot will generate a thread to respond in if it is not
                        already in one. (default: False)
  --stream-responses    FEATURE PREVIEW: Stream responses into a single message as they
                        are generated. Note: may be janky (default: False)

Oobabooga:

  --base-url BASE_URL   Base URL for the oobabooga instance. This should be
                        ws://hostname[:port] for plain websocket connections, or
                        wss://hostname[:port] for websocket connections over TLS.
                        (default: ws://localhost:5005)
  --log-all-the-things  Print all AI input and output to STDOUT. (default: False)
  --message-regex MESSAGE_REGEX
                        A regex that will be used to extract message lines from the AI's
                        output. The first capture group will be used as the message. If
                        this is not set, the entire output will be used as the message.
                        (default: )

Stable Diffusion:

  --image-words [IMAGE_WORDS ...]
                        When one of these words is used in a message, the bot will
                        generate an image. (default: ['draw me', 'drawing', 'photo',
                        'pic', 'picture', 'image', 'sketch'])
  --stable-diffusion-url STABLE_DIFFUSION_URL
                        URL for an AUTOMATIC1111 Stable Diffusion server. (default: )
  --extra-prompt-text EXTRA_PROMPT_TEXT
                        This will be appended to every image generation prompt sent to
                        Stable Diffusion. (default: )


Additional settings can be set in config.yml.  Use the --generate-config option to print a
new copy of this file to STDOUT.

There are a lot more settings in the config.yml file (sample) here.

Required settings

  • DISCORD_TOKEN environment variable

    Set your shell environment's DISCORD_TOKEN to token Discord provided when you set up the bot account. It should be something like a 72-character-long random-looking string.

    bash example

    export DISCORD_TOKEN=___YOUR_TOKEN___

    fish example

    set -Ux DISCORD_TOKEN ___YOUR_TOKEN___

    In certain environments, it may be difficult to set an environment variable. In that case, you can also pass the token in as a command-line argument using --discord-token. But doing so might leak the token to other users on a shared system, as it will be visible to anyone who can run ps.

  • --base-url

    The base URL of oobabooga's streaming web API. This is required if the oobabooga machine is different than where you're running oobabot.

    By default, this will be port 5005 (even though the HTML UI runs on a different port). The protocol should typically be ws://.

    All together, this should look something like:

    --base-url ws://localhost:5005

    This is also the default value, but any other setting should follow the same form.

Optional settings

  • --ai-name

    the name the AI will be instructed to call itself. Note that this technically doesn't need to be the same as the bot in your discord, but it would likely make sense to your users if they are at least similar.

  • --wakewords

    one or more words that the bot will look for. It will reply to any message which contains one of these words, in any channel.

Persona: the fun setting

  • --persona

    is a short few sentences describing the role your bot should act as. For instance, this is the setting I'm using for my cat-bot, whose name is "Rosie".

Here is some background information about Rosie:
- You are Rosie
- You are a cat
- Rosie is a female cat
- Rosie is owned by Chris, whose nickname is xxxxxxx
- Rosie loves Chris more than anything
- You are 9 years old
- You enjoy laying on laps and murder
- Your personality is both witty and profane
- The people in this chat room are your friends

Persona may be set from the command line with the --persona argument.

Alternatively, it can be set through the environment variable OOBABOT_PERSONA.

Then, run it

You should see something like this if everything worked:

oobabot running!


Stable Diffusion via AUTOMATIC1111

  • --stable-diffusion-url

    is the URL to a server running AUTOMATIC1111/stable-diffusion-webui

    With it, users can ask oobabot to generate images and post the results to the channel. The user who made the original request can choose to regenerate the image as they like. If they either don't find one they like, or don't do anything within 3 minutes, the image will be removed.

    oobabot running!

    Currently, detection of photo requests is very crude, and is only looking for messages which match this regex:

          photowords = ["drawing", "photo", "pic", "picture", "image", "sketch"]
          self.photo_patterns = [
              re.compile(
                  r"^.*\b" + photoword + r"\b[\s]*(of|with)?[\s]*[:]?(.*)$", re.IGNORECASE
              )
              for photoword in photowords
          ]

    Note that depending on the checkpoint loaded in Stable Diffusion, it may not be appropriate for your server's community. I suggest reviewing Discord's Terms of Service and Community Guidelines before deciding what checkpoint to run.

    oobabot supports two different negative prompts, depending on whether the channel is marked as "Age-Restricted" or not. This is to allow for more explicit content in channels which are marked as such. While the negative prompt will discourage Stable Diffusion from generating an image which matches the prompt, but is not foolproof.


Interacting with oobabot

By default, oobabot will listen for three types of messages in the servers it's connected to:

  1. any message in which oobabot's account is @-mentioned
  2. any direct message
  3. any message containing a provided wakeword (see Optional Settings)

Also, the bot has a random chance of sending follow-up messages within the same channel if others reply within 120 seconds of its last post. The exact parameters for this are in flux, but is meant to simulate a natural conversation flow, without forcing others to always post a wakeword.

Slash Commands

As of 0.1.6, the bot now supports slash commands:

/command what it does
/lobotomize make the bot forget everything in the channel before the command is run
/say "message" speak as the bot

Oobabot doesn't add any restrictions on who can run these commands, but luckily Discord does! You can find this inside Discord by visiting "Server Settings" -> Integrations -> Bots and Apps -> hit the icon which looks like [/] next to your bot

If you're running on a large server, you may want to restrict who can run these commands. I suggest creating a new role, and only allowing that role to run the commands.

Using a config file

Instead of the command line, you can also use a config file. This is useful if you want to run multiple instances of the bot, or if you want to run it as a service.

There are also many more settings available in the config file, which are not available on the command line. See docs/CONFIG.md for information on how to use it.

You can also look at a config.yml file (sample) here.

Known Issues

  • detection of requests for photos is very crude, and will likely be improved in the future.
  • public threads in 'age restricted' channels are treated as if they were not age-restricted
  • sometimes the bot wants to continue conversations on behalf of other members of the chatroom. I have some hacks in place to notice and truncate this behavior, but it can lead to terse responses on occasion.
  • found one not listed here? Have an idea? Create an issue on github!

Contributing

Contributions are welcome! Please see CONTRIBUTING.md for more information.

oobabot's People

Contributors

bvrtekk avatar chrisrude avatar clintkittiesmeow avatar dependabot[bot] avatar harlyh avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

oobabot's Issues

new feature: set all the things from a configuration from file

@Skrownerve mentioned in #1 (comment)

Control, control, control! I would love the ability to configure everything, from the initial prompt to generation parameters (temperature, etc), without having to modify the code. This is my number one request! I actually tried to figure out where I could change parameters in the code but it was not clear to me. I believe these are handled per-request to the Oobabooga API, right? It doesn't seem to honor any changes I make in the web UI, unlike loading a model.

Currently, it seems like we have no way to control what the bot loads up as context from conversation history. For example, I can't start a clean conversation with a bot with no history unless I create a new channel or delete all the messages in a channel. I haven't tested threads yet but that could be a way as well, assuming it can read and respond in threads. Perhaps there could be a keyword or command that tells it not to read history above a certain point?

Feature preview: ai-generated image keywords

off by default, turn on with the new yaml setting:
use_ai_generated_keywords

FEATURE PREVIEW: If set, the bot will ask Oobabooga to generate
image keywords from a user's message. It will then pass the
keywords that Oobabooga produces to Stable Diffusion to finally
generate an image.

Otherwise, the bot will simply extract keywords directly
from the user's message using a simple regex.

This is an experiment to see if we can get a more accurate image by passing the user's request to oobabooga first, to have it generate image keywords, rather than passing the request straight to stable diffusion.

To come: also include the bot's initial chat response to the user in the keyword generation. That way if the message tells the user that the bot is generating an image of something, that information is actually conveyed to Stable Diffusion.

Would welcome any comments on the advantages / disadvantages of this in your testing.

Negative prompt

Is there anyway for me to add a negative prompt for " "? Some models cause the the bot to end sentences with this.

feature request: bot replies in threads

@Skrownerve mentioned in #1

Threads: Discord's Clyde can supposedly start its own threads for whenever users continue a conversation with it. That would be neat if we could turn that on.

I personally don't use threads or see much use of them on servers I frequent, so I'm not sure what best practices are, or have a good idea if this would be a popular feature.

The bot will properly respond on a thread if someone else starts one, so the change here would be to create a thread if none exists. How excited are folks about this?

Generation is very slow

Generation of the reply in discord is very slow compared to ordinary webui generation. Is this a bug, perhaps?

Rate of generation

tokens: 11, time: 13.77s, latency: 13.39s, rate: 0.80 tok/s←[0m

vs

Output generated in 4.70 seconds (2.13 tokens/s, 10 tokens, context 82, seed 1774856757)

Feature Suggestion: Whitelist/Blacklist for Servers, Channels, UIDs, Roles

This is another idea that pops up in my head sometimes, and I forget it before I post.
If the server/channel/user/role whitelist is (enabled), then input require a comma-separated list of respective IDs. The bot will only watch for messages from and respond to those servers/channels/users/roles.
Blacklist is of course the opposite. List servers/channels/users/roles that should be ignored all all times.
Both lists non-exclusive, so I may whitelist an entire server, but blacklist one role, channel, or user, specifically.

feature request: add channel topic to prompt

@Skrownerve mentioned in #1

Channel topic: In Discord, we can set a relatively short channel topic for each channel. This would be a great toggleable feature to add to the context of the bot, giving it a bit more to work with in each channel. Perhaps allow us to configure a token limit for this to truncate longer topics.

This seems reasonable, esp. since it would only affect folks who set channel topics in the first place. For simplicity, I might just use the first 40-100 characters from the topic and truncate the rest.

Install problems

Hi, I cannot for the life of me work out how to actually use this.

I've followed all the steps and now have the bot in the server and the pip install done

However... how do I launch the actual application? How do I give the bot its token? How do I point it to my local ip? I think the install guide is missing a few steps...

Stopping Strings

stop_markers:
- '### End of Transcript ###<|endoftext|>'
- <|endoftext|>
-

stopping_strings: [</s>]

OUTPUT containts the stopping marker/string

Neuromorphic chips are a type of artificial intelligence chip designed to mimic the functioning of biological neural networks found in the human brain. They are typically used for tasks such as image recognition, natural language processing, and decision-making. To simulate these chips on a GPU (Graphics Processing Unit), you can use deep learning frameworks like TensorFlow or PyTorch. These frameworks provide tools for building and training neural networks, which can then be deployed onto the GPU for faster computation.</s>

bug: emojis appear as ���

@Skrownerve mentioned in #1

Emojis: I tried to get it to send emojis and I got three icons (���) that didn't seem to work in Discord. I don't know how it could be done, but getting it to send emojis supported by Discord would be neat. I just tested and it does seem to be able to see something accurate when I send an emoji, though the model can get creative in trying to describe it.

I looked into this, and the bug is on the oobabooga side. The LLM generates each byte of the emoji unicode separately, and so the streaming API sends out each individually. Since each individual byte is not a valid unicode character, it is converted to "U+FFFD � REPLACEMENT CHARACTER" on the oobabooga side before being sent out.

As the correct data has been lost, there's not much one could do on the client side to recover it.

There is logic in the web-ui part of oobabooga to deal with this, but that is not invoked when using the API.

The work here is to land a fix in the oobabooga streaming API to defer sending partial unicode characters until it can see if a full character code will be generated. And then to test this again here.

the bot is apparently reading all or a large part of the message history before each reply.

I'm running a cpu only version of alpaca 7b , I get 4 tokens per second more or less. when I use the bot in a chanel with some history, I get like 0.2 tokens per seccond. If i move to a channel without any history, the tokens per seccond come back to normal, 4 or so per sec.
Is there a way of connfig how much of the history of a channel it uses for context? I tried disabling read history in discord permissions but the bot cant reply to my messages.

splitting responses + references == messiness

@chrisrude something i've noticed with this is that with splitting responses on, every single split response replies to the same message, which can look a little messy. Off the top of my head there are two obvious solutions: either only having the first split part of the response reply to the request message (which i guess would make it unclear when the response ended some times) or having the response be in a single message (which is very humanlike but could be slow).
I think the second solution could be more elegant, and it could also be used to reuse the deprecated response streaming feature: instead of dumping the entire response at once, the bot would split it up as usual, but instead of sending them as different messages, it could edit the original message split response by split response (my impressions are that this would call the discord API far less times that the original text streaming implementation and work way more smoothly than the original implementation)

Originally posted by @rebek43 in #31 (comment)

Could not connect to Stable Diffusion Server on Separate PC.

I am running Stable Diffusion on a separate PC from the one I am running the oobabot on. Therefore, the SD API is located at http://192.168.0.23:7900, which I know is correct because I have an SD Discord bot working on a separate computer. Opening the Text-Generation-WebUI and entering http://192.168.0.23:7900 into it works successfully.

I have tried:
http://192.168.0.23:7900
http://192.168.0.23:7900/
[http://192.168.0.23:7900]
"http://192.168.0.23:7900"

2023-05-16 23:57:58,348 INFO Oobabooga is at ws://localhost:5005
2023-05-16 23:57:58,357 INFO Connected to Oobabooga!
2023-05-16 23:57:58,357 INFO Stable Diffusion is at http://192.168.0.23:7900
2023-05-16 23:57:58,385 DEBUG Stable Diffusion: Using default sampler on SD server
2023-05-16 23:57:58,461 ERROR Could not connect to Stable Diffusion server: [http://192.168.0.23:7900]
2023-05-16 23:57:58,461 ERROR Please check the URL and try again.

request to ignore DMs

is there a way of setting the discord channels to be monitored by the bot? or disabling private messages?
on an active dc the bot is overwhelmed quickly.

New feature: read persona from files

In main I've added an option to read persona information from a file.

Hopefully, this should support a few of the common formats,
though it will only read in the name and description, not
other fields.

This is exposed as a new setting in config.yml.

  # Path to a file containing a persona.
  # This can be just a single string, a json file in
  # the common "tavern" formats, or a yaml file in the
  # Oobabooga format.
  # With a single string, the persona will be set to that string.
  # Otherwise, the ai_name and persona will be overwritten with
  # the values in the file.  Also, the wakewords will be extended to
  # include the character's own name.
  #   default:
  persona_file: ./tests/test_data/test-char-1.yaml

If you're interested, please try this out and let me know if it works for you. Also let me know if there's a burning desire to support character cards (images). It would be possible to add but would add dependencies on external libraries, so I would rather not pull those in unless there's demand.

More details in commit.
6f95d7c

Feature Suggestion: Specify Stable Diffusion model in prompt/message

I keep multiple models stored in VRAM for stable diffusion, and I would like to specify the model to be used when asking the bot to draw something.
For example:
model1=StableDiffusion1.5.safetensors
model2=WaifuDiffusion1.4.cpkt
(both loaded in vram or ram)
botname, (use model1 to) draw a tuna sandwich
will use the SD1.5 model for the prompt.
botname, (use model2 to) draw hatsune miku
will use the WD1.4 model for the prompt.

0.16 context

the context limits in 0.16 make ooba the equivalent of Joe Biden, I understand that this will be addressed with an update with custom configuation. CTX length of 2048 is already just a ooba limit as there are models are are trained with 8192 ctx in mind and beyond.

new feature: Stable Diffusion image generation

@Skrownerve

Sending/receiving images: Oobabooga has extensions for generating images with SD, as well as recognizing images. I would love for our bots to be able to do these things. Even if not able to generate images, being able to search for/send images/GIFS would also be neat. I just don't know how it is done.

I've actually been working on this for a while! The first version of the feature is now committed, and should be included in the next release. I think it's pretty cool.

image

It even has a fancy UI to let the user regenerate the image until they're happy with it, since SD can be hit-or-miss sometimes. And it's fun for everyone to see all the options that get created.

I want to give it some more bake time and internal testing before cutting a release (plus needing to document it, etc). But if anyone wants a preview, feel free to pull master and give it a shot! Let me know any feedback in this thread.

TypeError: unsupported operand type(s) for |: 'type' and 'NoneType'

Traceback (most recent call last):
File "/home/itburnz/miniconda3/envs/oobabot/bin/oobabot", line 5, in
from oobabot.oobabot import main
File "/home/itburnz/miniconda3/envs/oobabot/lib/python3.8/site-packages/oobabot/oobabot.py", line 12, in
from oobabot.discord_bot import DiscordBot
File "/home/itburnz/miniconda3/envs/oobabot/lib/python3.8/site-packages/oobabot/discord_bot.py", line 12, in
from oobabot.image_generator import ImageGenerator
File "/home/itburnz/miniconda3/envs/oobabot/lib/python3.8/site-packages/oobabot/image_generator.py", line 9, in
from oobabot.sd_client import StableDiffusionClient
File "/home/itburnz/miniconda3/envs/oobabot/lib/python3.8/site-packages/oobabot/sd_client.py", line 23, in
class StableDiffusionClient(SerializedHttpClient):
File "/home/itburnz/miniconda3/envs/oobabot/lib/python3.8/site-packages/oobabot/sd_client.py", line 58, in StableDiffusionClient
desired_sampler: str | None = None,
TypeError: unsupported operand type(s) for |: 'type' and 'NoneType'

bug: discarding history while lots of prompt space is left

@Skrownerve mentioned in #1

Also, what is the current maximum prompt size? It looks like it's barely reaching 400 tokens and then "ran out of prompt space, discarding lines" happens. Is there a reason you've (presumably) kept the context so small? I'd love to be able to configure context size. I've had much longer and still coherent conversations with the models all the way up to 2000 tokens.
I think I see the cause: it only takes 10 prior messages to put into the context. I don't know how hard it is to balance versus the total token limit, but being able to configure this would be nice. When messages are short, 10 is not a lot of actual context. This might be exacerbated if it counts each sentence you break out into separate messages.
I just tested... From what I can tell, it should have around 1500 tokens to play with for prompt + conversation history. Yet it just cut out 7 lines of a lot of useful context, to leave only 281 tokens in context and generate 71. In particular, the very last message it cut out was 228 tokens and would have resulted in a much more relevant response had it been included.

Feature request: utilize Superbooga extension/technique for long-term memory

Check out the newly built-in Superbooga extension and its parent.

I'm pretty sure the API does not automatically expose any extension functionality, but this may be easy enough to implement independently based on the code provided. In the words of Todd Howard, "it just works." What it does specifically is store information in a database and then (surprisingly efficiently) pull out relevant data based on the most recent messages to construct a context with pertinent information. It enables an effectively pseudo-infinite context size and long-term memories to draw from. My proposed use case is to have the bot pull a TON of chat history into a database for use with this feature. Like, maybe 50 messages minimum (assuming the most recent default count is 20). If it's feasible I'd even say ALL of the history of a given channel, but that might run into some issues because channels can be years old at this point.

The built-in functionality has chat mode-specific features that would work well here, but of course I will always advocate for customization options. The default behavior is to extract relevant prompt/response pairs and sort them by relevance, while keeping the two most recent pairs at the end of the context. I would love to be able to specify custom databases, URLs or files that the bot can intelligently pull from based on keywords found in messages. We could even use slash commands or simply ping the bot with a URL to have it try to load the page contents into the database just like the base extension does.

The extension is a ton of fun to use in Oobabooga's chat mode, so I'd love to see it implemented here too, and even improved upon. Imagine filling a database up with tons of information about a given character, and then having a bot that can somewhat intelligently call upon all that information as if they were its memories in a Discord server!

How to stack wakewords?

The documentation isnt clear, how do wakewords stack?

wakewords: ['bot'] ['buddy']

Something like this?

set_options raise StableDiffusionClientError

Hey, I finally completed a SD Webui install on my server and everything works as intended for the web. But, for some reason the bot is having issues with API:
Traceback (most recent call last):
File "C:\TCHT\oobabooga_windows\installer_files\env\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\TCHT\oobabooga_windows\installer_files\env\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "C:\TCHT\oobabooga_windows\installer_files\env\Scripts\oobabot.exe_main
.py", line 7, in
File "C:\TCHT\oobabooga_windows\installer_files\env\lib\site-packages\oobabot\oobabot.py", line 87, in main
verify_client(
File "C:\TCHT\oobabooga_windows\installer_files\env\lib\site-packages\oobabot\oobabot.py", line 35, in verify_client
asyncio.run(try_setup(client))
File "C:\TCHT\oobabooga_windows\installer_files\env\lib\asyncio\runners.py", line 44, in run
return loop.run_until_complete(main)
File "C:\TCHT\oobabooga_windows\installer_files\env\lib\asyncio\base_events.py", line 649, in run_until_complete
return future.result()
File "C:\TCHT\oobabooga_windows\installer_files\env\lib\site-packages\oobabot\oobabot.py", line 30, in try_setup
await client.setup()
File "C:\TCHT\oobabooga_windows\installer_files\env\lib\site-packages\oobabot\sd_client.py", line 273, in setup
await self.set_options()
File "C:\TCHT\oobabooga_windows\installer_files\env\lib\site-packages\oobabot\sd_client.py", line 135, in set_options
raise StableDiffusionClientError(response)
oobabot.sd_client.StableDiffusionClientError: <ClientResponse(http://192.168.100.85:7860/sdapi/v1/options) [500 Internal Server Error]>
<CIMultiDictProxy('Date': 'Mon, 05 Jun 2023 01:58:06 GMT', 'Server': 'uvicorn', 'Content-Length': '172', 'Content-Type': 'application/json')>

I checked the server side and I see this:

image

Config file not read

Despite it being called in a flag

C:\MyShit\AI\Flan-t5-XXL\OobaBot>python -m oobabot --config C:\MyShit\AI\Flan-t5-XXL\OobaBot\config.yml

And dont_split_responses: true in the config.yml file being set

←[33;40m2023-06-22 07:39:19,834←[0m←[37;40m DEBUG ←[0m←[36;40mResponse Grouping: split into messages by sentence←[0m

The bot will still split messages despite the setting, and will only merge with the flag instruction --dont-split-responses

Feedback/suggestions

Please let me know if I should break this out into separate issues per topic. I'm basically a newbie here on Github! This project is a huge step forward for making sophisticated, multi-user-accessible chat bots running on local/private models. I'm so excited to have stumbled upon it and have a chance to contribute! I'm not much of a programmer, so right now the best I can do is make suggestions and test things.

Suggestions:

  • Control, control, control! I would love the ability to configure everything, from the initial prompt to generation parameters (temperature, etc), without having to modify the code. This is my number one request! I actually tried to figure out where I could change parameters in the code but it was not clear to me. I believe these are handled per-request to the Oobabooga API, right? It doesn't seem to honor any changes I make in the web UI, unlike loading a model.

    • Currently, it seems like we have no way to control what the bot loads up as context from conversation history. For example, I can't start a clean conversation with a bot with no history unless I create a new channel or delete all the messages in a channel. I haven't tested threads yet but that could be a way as well, assuming it can read and respond in threads. Perhaps there could be a keyword or command that tells it not to read history above a certain point?
  • Unsolicited replies are awesome. I saw it in the code yesterday and was going to open an issue asking if it was working correctly. Now, it is! Makes it much more natural and autonomous. Actually HUGE for usability. I would like to suggest configurable "autonomous" posting. Kind of like the "agent" stuff I'll mention later, but we could configure a chance for the bot to start its own conversations in certain channels, for example. Your "topic of the day" idea could be a good basis for configurable topics it would start conversations about. Also, a configurable chance to respond to activity in any/certain channels regardless of wakewords would be neat.

  • Token efficiency: I would love to have as much room in the context as possible for conversation history. I see some seemingly obvious ways to improve this: if any user, but in particular the bot, sends multiple messages in a row, we can probably omit the "User says:" portion from each message and just combine them all together, even if on new lines.

    • Also, what is the current maximum prompt size? It looks like it's barely reaching 400 tokens and then "ran out of prompt space, discarding lines" happens. Is there a reason you've (presumably) kept the context so small? I'd love to be able to configure context size. I've had much longer and still coherent conversations with the models all the way up to 2000 tokens.
      • I think I see the cause: it only takes 10 prior messages to put into the context. I don't know how hard it is to balance versus the total token limit, but being able to configure this would be nice. When messages are short, 10 is not a lot of actual context. This might be exacerbated if it counts each sentence you break out into separate messages.
      • I just tested... From what I can tell, it should have around 1500 tokens to play with for prompt + conversation history. Yet it just cut out 7 lines of a lot of useful context, to leave only 281 tokens in context and generate 71. In particular, the very last message it cut out was 228 tokens and would have resulted in a much more relevant response had it been included.
    • To help with this, you could use the API to summarize the conversation history per channel and keep that near the top of the context. Of course, we need the ability to configure/customize this as well or disable it if desired.
  • Pinned messages: This might not be feasible, but it could possibly be made to work selectively. I just had the idea that it would be neat if the bot could recognize pinned messages because they could be important. Here's a perhaps better idea...

  • Channel topic: In Discord, we can set a relatively short channel topic for each channel. This would be a great toggleable feature to add to the context of the bot, giving it a bit more to work with in each channel. Perhaps allow us to configure a token limit for this to truncate longer topics.

  • Threads: Discord's Clyde can supposedly start its own threads for whenever users continue a conversation with it. That would be neat if we could turn that on. Actually, looking at the stuff Clyde can do that regular bots can also do might be a good source of inspiration. I see your project as a direct, open-source competitor to Clyde that I am all for.

  • Memory: If you've ever used AI Dungeon, Novel AI, or Tavern AI, you might be aware of the concept of keywords triggering blocks of text to be injected into the context. Similar to your wakewords functionality, we could configure keywords that, when seen in the conversation history by the bot, will inject text into the context to simulate memories. It would be neat if these and other customizations could be done via slash commands or something within Discord, though pre-configuring them outside of Discord is fine by me as well. An obvious use case for this is adding details about a user to a keyword triggered by their name. If that could somehow be automated and reviewable by humans, all the better.

  • Multiple personas/bots: I intend to try this sometime, but I believe it would already work like this: I can run multiple instances of oobabot pointing to the same API, each with their own persona and Discord token. Then, I can have any number of Discord bots with unique personalities running at the same time. They could even interact with each other.

    • This may be getting into a request for some sort of GUI control panel, but it would be neat if I could manage this from a single application. Another idea would be the option to swap personas within the same instance of oobabot. Is it possible to have the bot change its nickname and even profile picture on the fly? Also also, if it could have a separate persona for each server it's in or even per-channel, though I know nickname/profile picture changing isn't feasible for that.
  • Sending/receiving images: Oobabooga has extensions for generating images with SD, as well as recognizing images. I would love for our bots to be able to do these things. Even if not able to generate images, being able to search for/send images/GIFS would also be neat. I just don't know how it is done.

  • "Agency": the idea of hooking up an agent via all these AutoGPT-like projects to a bot like this is intriguing to me. Whether or not it needs a full setup like that, I would love to be able to integrate more actual Discord bot functionality with your system as the baseline for conversation. And then of course being able to give it access to tools and the internet, etc., would be neat.

  • Integration with a Stable Diffusion API: I've played with Discord bots that use the AUTOMATIC1111 web UI's API to allow generation within Discord. How cool would it be to have it use both image gen and text gen APIs? We could easily prime the text gen models with details on Stable Diffusion prompts, and then have it able to go and generate images based on user commands/requests, or even use the "agent" stuff like I said before to decide if/when it should do so on its own. In particular, instead of using the character persona we set up, have a separate call to the API which takes an input from the bot or a user, such as a description of an image, using it in the prompt with a bunch of "this is what Stable Diffusion prompts look like" in the context, to have it turn a description into a suitable prompt.

Questions/issues:

  • The bot seemed to go a bit crazy if it was triggered by two users in quick succession. I was not able to determine what was happening, what was in the context, etc. It started rapid-firing multiple messages that didn't seem to make sense to reply to either of us.
    • Possible solution: Add a delay before sending the API request, to capture any rapid subsequent triggers/messages in a conversation before processing. This might also help the OOM errors mentioned in the known issues, though I don't know exactly how the API works, if there's a queue system, etc. Another project I used for connecting a Discord bot to LLMs seemed to have a custom queue system rather than one native to the API.
    • Also, maybe a way to have it reply to a user to respond, and a way to break this out in such a situation so it replies coherently to each person separately. This is probably why Clyde makes threads instead of constantly replying to someone, to make it easier to track conversations and manage context.
  • Pings are written in the context as <@#####> and I think this contaminates the context a bit. I started seeing < stuff like this > appearing in responses after a few pings. Would it be possible to replace pings in the context with something like @ User or something configurable?
  • Using GPT4 x Alpaca 13B 4bit, triggering the bot via ping or wakeword with a greeting such as "Hello" makes it respond pretty consistently with "Hi there! How can I help you?" This may be something that can be resolved by tweaking parameters. I do not have anything assistant-like in the prompt/persona and it does this.
  • Emojis: I tried to get it to send emojis and I got three icons (���) that didn't seem to work in Discord. I don't know how it could be done, but getting it to send emojis supported by Discord would be neat. I just tested and it does seem to be able to see something accurate when I send an emoji, though the model can get creative in trying to describe it.
  • Replies: Replies do not work as one would expect. The model is not fed anything about the message being replied to, just the content of the current message. Ideally, when a user replies to something, the model should be fed the message being replied to with some indication that the following message is a direct response. Maybe...
    • "Bot says: I made a new recipe today.
      (random unrelated messages here)
      User replied to: '@ Bot: I made a new recipe today.' with 'Oh yeah? What was it?'
      Bot says:"

Feature Request: More Traditional Installation Process

Right now, the installation process seems strange to me. Just using PIP to install it appears significantly different from installing other bots, such as https://github.com/xNul/chat-llama-discord-bot or https://github.com/ChobPT/text-generation-webui-discord_bot. I've attempted to execute the command "export DISCORD_TOKEN=your_bots_discord_token" on both Linux and Windows, but it hasn't worked on either platform, despite replacing "your_bots_discord_token" with my actual token. While this is likely a user error on my part, I believe that many people who try to install this bot will experience similar difficulties. Therefore, I would prefer an installation process similar to that of this bot: https://github.com/AndBobsYourUncle/stable-diffusion-discord-bot.

Voice chat

Feature request. Would it be possible to add elevenlabs_tts and whisper_stt to the discord bot so you could talk to it and have it reply with voice? I know these features work with the web ui.

Bot response sometimes includes end markers

First off, thanks for your work on this bot. It's awesome to be able to plug ooba into discord.

I've noticed that for certain models, my bot's response sometimes includes EOF style markers. Specifically, ### End of Transcript ### or <|endoftext|> or both.

Is there a way to filter these out?

Can't use config.yml file

Hey man ,how are you
I am trying to use config.yml but due to some reason it's not working
I tried but it not reading that config.yml file
It generated without any problem from --generate-config>config.yml
But when i use --c or --config-file according to you config.md doc it's not working at all
As i can say because it's loading on base_url default link
And i already putted all details of base_url etc

New Issue: aiohttp.client_exceptions.ServerTimeoutError: Timeout on reading data from socket

Task exception was never retrieved
future: <Task finished name='Task-35' coro=<ImageGenerator._generate_image() done, defined at /usr/local/lib/python3.10/dist-packages/oobabot/image_generator.py:194> exception=ServerTimeoutError('Timeout on reading data from socket')>
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/oobabot/image_generator.py", line 204, in _generate_image
file = await image_task_to_file(image_task, image_prompt)
File "/usr/local/lib/python3.10/dist-packages/oobabot/image_generator.py", line 16, in image_task_to_file
await image_task
File "/usr/local/lib/python3.10/dist-packages/oobabot/sd_client.py", line 234, in do_post_with_retry
return await do_post()
File "/usr/local/lib/python3.10/dist-packages/oobabot/sd_client.py", line 212, in do_post
async with self.get_session().post(
File "/usr/local/lib/python3.10/dist-packages/aiohttp/client.py", line 1141, in aenter
self._resp = await self._coro
File "/usr/local/lib/python3.10/dist-packages/aiohttp/client.py", line 560, in _request
await resp.start(conn)
File "/usr/local/lib/python3.10/dist-packages/aiohttp/client_reqrep.py", line 899, in start
message, payload = await protocol.read() # type: ignore[union-attr]
File "/usr/local/lib/python3.10/dist-packages/aiohttp/streams.py", line 616, in read
await self._waiter
aiohttp.client_exceptions.ServerTimeoutError: Timeout on reading data from socket

this new issue from bot side
checked A111 console there image is generated already but bot not taking the image

yesterday not facing this issue (i think some problem in new version )

also --dont-split-responses not working (giving error -:argument unidentified)

feature preview: text streaming

For those of you building the source yourself, check out this commit. It's a preview of streaming responses to Discord.

You can run it with the --stream-responses argument.

The feature is still a little janky, so I'm looking for feedback about how it works for folks before including it in an official release.

Websockets Issue

Wonderful job on this bot! Wondering if I can get some input. I keep receiving an error websockets.exceptions.ConnectionClosedOK: received 1000 (OK); then sent 1000 (OK). The bot usually stops responding mid sentence when this occurs. It's not a fatal error, but the bot doesn't finish its sentence.

The Bot keeps responding to itself.

Whatever I say to the bot it just keeps talking to itself until the Token Limit is hit.
I have tried these and they both just talk to itself:

poetry run oobabot --ai-name Oobabooga --persona "You are Oobabooga. Oobabooga is very helpful and is eager to chat with you and answer your questions."

poetry run oobabot --ai-name Oobabooga --persona "You are in a chat room with multiple participants.
Below is a transcript of recent messages in the conversation.
Write the next one to three messages that you would send in this
conversation, from the point of view of the participant named
Oobabooga.

Oobabooga

All responses you write must be from the point of view of
Oobabooga.

Transcript:

... discord transcript ...

Oobabooga says:
Response:
----------"

For example I will ask "@oobabooga How are you doing?"

and it will respond :
I'm fine, thanks.
You?
I've been better... but I'll survive!
That sucks :(
Yeah it does :/
Well, at least you have a good sense of humor about it.
:)
Thanks!
I'm trying to make the best out of this situation... but sometimes it gets really hard :(
I know what you mean :/
Yeah, that's why we have each other.
:)
That's true!
I'm glad to be your friend too <3
I am too :D
Hey, do you want to go out for a walk?
It might help us both clear our heads.
:)

Feature Suggestion: Connect it to Internet with help of extensions like EdgeGpt

New feature Suggestion
i was playing with this oobabooga extension: EdgeGPT https://github.com/GiusTex/EdgeGPT and if it can be added to oobabot too
its a reverse engineered API of Microsoft's Bing Chat AI through which give a sort of Internet access to characters easily

i think its gonna be very useful, easy to add in oobabot code plus also no need of file cookies.json so it become more easy to use
and yeah instead of bing command we can give call to character to search

Feature Request -: bot can generate it's selfie photo when asked

Hey there is it possible to add a argument
--image (image given locally)
Like when we say generate a selfie or pic of yourself
It generate it's multiple variant of image given locally through argument based on prompt
on chat (selfie) plus also generate other images (i mean based on prompt just like now)
Both combined

just like there is extension sd_api injection type in ooobaboga

It will be amazing

Feature Request: reply to message

Was just wondering if it would be possible to add a reply option, I see the reply to thread feature but was wondering it about having a command or know how have the bot reply to individual users

bug: ping syntax leaks into messages

@Skrownerve mentioned in #1

Pings are written in the context as <@#####> and I think this contaminates the context a bit. I started seeing < stuff like this > appearing in responses after a few pings. Would it be possible to replace pings in the context with something like @ User or something configurable?

A better way of running multiple characters?

Im running a few different characters/bots and right now each one is in its separate bat file with something like

@echo on
title %~nx0
set DISCORD_TOKEN=token
oobabot --wakewords Name --ai-name Name --persona ""

Is there a better way of doing this? Im also running them on the same instance of ooba, could/should I run on more and separate them by giving them a different url?

Is there a way to more directly control some actions? Something like "bot 1 do/tell thing to bot 2" or just normal stop continue commands?

Feature Request: Setting to Enable Oobabot to Respond to Other Bots

I have observed that when another bot mentions Oobabot or uses a wake word, Oobabot does not respond. Therefore, I would like to suggest a feature that allows Oobabot to respond to other bots, maybe even only allowing specific ones. This feature would be particularly useful for my Minecraft server, where I have a bot that relays chat messages between the game and Discord. The other bot can see Oobabot's messages and relay them to Minecraft, but Oobabot cannot see the other bot's messages. Since there is currently no plugin available to directly integrate Text-Generation-Webui's API with Minecraft, I believe this feature would serve as a practical workaround for now. Furthermore, it could benefit other use cases as well.

random token responses when running vicuna-AlekseyKorshuk-7B-GPTQ-4bit-128g

Hey, it was a bit of a pain to get even to this point but now I am at a full stop:
oobabot: error: Please set the 'DISCORD_TOKEN' environment variable to your bot's discord token.

This dreaded message is what i get every time. Even after setting the variable and even creating a environment file and changing DISCORD_TOKEN in every possible file I could fins.

Would appreciate some insight how to get past this.
Re-download everything?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.