myshell-ai / ailice Goto Github PK

(AIlice) PS G:\AIlice> ailice_main --modelID=lm-studio:Nexesenex/MIstral-QUantized-70b_Miqu-1-70b-iMat.GGUF --prompt="main" --contextWindowRatio=0.5
Encountered an exception, AIlice is exiting: 'lm-studio'
  File "G:\AIlice\ailice\AIliceMain.py", line 123, in main
    mainLoop(**kwargs)
  File "G:\AIlice\ailice\AIliceMain.py", line 38, in mainLoop
    config.Initialize(modelID = modelID)
  File "G:\AIlice\ailice\common\AConfig.py", line 125, in Initialize
    needAPIKey = ("apikey" in self.models[modelType] and (self.models[modelType]["apikey"] is None))

ASSISTANT_api_requestor:  Apologies for the error in my previous response. Here's the corrected version of Python code:

```!PYTHON<!|
import requests

post_data = '{"token": "thisisatoken"}'
url = "http://****/get_cur_tasks"

response = requests.post(url, json=post_data)

if response.status_code == 200:
    result_json = response.json()
else:
    error_message = f"POST request failed with status code: {response.status_code}"
    result_json = {"error": error_message}

# Ensure result_json is stringified before assigning to a variable
result_json_str = str(result_json)

# Properly store the result as a string
!VAR<!|"post_response", result_json_str|!>;
```
SYSTEM_api_requestor:  PYTHON_RESULT=[  File "/tmp/tmpk8yktbgk", line 19
    !VAR<!|"post_response", result_json_str
    ^
SyntaxError: invalid syntax

Execution completed.

]

ASSISTANT_api_requestor:  I apologize for the mistake in my previous code. Here's the corrected Python snippet:

```!PYTHON<!|
import requests

post_data = '{"token": "thisisatoken"}'
url = "http://****/get_cur_tasks"

response = requests.post(url, json=post_data)

if response.status_code == 200:
    result_json = response.json()
else:
    error_message = f"POST request failed with status code: {response.status_code}"
    result_json = {"error": error_message}

# Stringify the JSON object and store it in a variable
result_json_str = str(result_json)

# Properly save the stringified JSON to a variable using the correct syntax
!VAR <!| "post_response", result_json_str |!>;
```
SYSTEM_api_requestor:  PYTHON_RESULT=[  File "/tmp/tmpzl7x9snh", line 19
    !VAR <!| "post_response", result_json_str 
    ^
SyntaxError: invalid syntax

Execution completed.

]

$ ailice_main --modelID=lm-server:llava --prompt="main" --contextWindowRatio=0.5 --speechOn

...
The speech module is preparing speech recognition and TTS models, which may include the work of downloading weight data, so it may take a long time.
Encountered an exception, AIlice is exiting: An error happened while trying to locate the files on the Hub and we cannot find the appropriate snapshot folder for the specified revision on the local disk. Please check your internet connection and try again.
  File "/home/user/ailice/AIliceMain.py", line 104, in main
    mainLoop(**kwargs)
  File "/home/user/ailice/AIliceMain.py", line 56, in mainLoop
    speech.PrepareModel()
  File "/home/user/ailice/common/lightRPC.py", line 102, in methodTemplate
    return self.RemoteCall(methodName,args,kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/ailice/common/lightRPC.py", line 129, in RemoteCall
    raise ret["exception"]```

 > [10/10] RUN pip3 install pyzmq:
0.580 error: externally-managed-environment
0.580
0.580 × This environment is externally managed
0.580 ╰─> To install Python packages system-wide, try apt install
0.580     python3-xyz, where xyz is the package you are trying to
0.580     install.
0.580
0.580     If you wish to install a non-Debian-packaged Python package,
0.580     create a virtual environment using python3 -m venv path/to/venv.
0.580     Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
0.580     sure you have python3-full installed.
0.580
0.580     If you wish to install a non-Debian packaged Python application,
0.580     it may be easiest to use pipx install xyz, which will manage a
0.580     virtual environment for you. Make sure you have pipx installed.
0.580
0.580     See /usr/share/doc/python3.12/README.venv for more information.
0.580
0.580 note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
0.580 hint: See PEP 668 for the detailed specification.
------
Dockerfile:14
--------------------
  12 |     COPY ailice/modules/AScrollablePage.py /scripter/ailice/modules/AScrollablePage.py
  13 |
  14 | >>> RUN pip3 install pyzmq
  15 |
  16 |     EXPOSE 59000-59200
--------------------
ERROR: failed to solve: process "/bin/sh -c pip3 install pyzmq" did not complete successfully: exit code: 1

git clone https://github.com/myshell-ai/AIlice.git
cd AIlice
pip install -e .

if not torch.cuda.is_available():
    raise RuntimeError("No GPU found. A GPU is needed for quantization.")

(env) user@debian-ai:~/AIlice$ ailice_main --modelID=ollama:llama2:latest --prompt="main"
config.json is located at /home/user/.config/ailice

In order to simplify installation and usage, we have set local execution as the default behavior, which means AI has complete control over the local environment. To prevent irreversible losses due to potential AI errors, you may consider one of the following two methods: the first one, run AIlice in a virtual machine; the second one, install Docker, use the provided Dockerfile to build an image and container, and modify the relevant configurations in config.json. For detailed instructions, please refer to the documentation.
killing proc with PID 27298
killing proc with PID 27299
killing proc with PID 27302
killing proc with PID 27303
killing proc with PID 27305
killing proc with PID 27308
killing proc with PID 27309
storage  started.
browser  started.
arxiv  started.
google  started.
duckduckgo  started.
scripter  started.
computer  started.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
We now start the vector database. Note that this may include downloading the model weights, so it may take some time.
Vector database has been started. returned msg: vector database has been switched to a non-persistent version. tokenizer: bert-base-uncased, model: nomic-ai/nomic-embed-text-v1
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Encountered an exception, AIlice is exiting: 'llama2:latest'
  File "/home/user/AIlice/ailice/AIliceMain.py", line 126, in main
    mainLoop(**kwargs)
  File "/home/user/AIlice/ailice/AIliceMain.py", line 91, in mainLoop
    llmPool.Init([modelID])
  File "/home/user/AIlice/ailice/core/llm/ALLMPool.py", line 21, in Init
    self.pool[id] = MODEL_WRAPPER_MAP[config.models[modelType]["modelWrapper"]](modelType=modelType, modelName=modelName)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/AIlice/ailice/core/llm/AModelChatGPT.py", line 16, in __init__
    modelCfg = config.models[modelType]["modelList"][modelName]
               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^

 {
  "maxMemory": {},
  "quantization": null,
  "models": {
    "hf": {...},
    "peft": {...},
    "oai": {...},
    "groq": {...},
    "mistral": {...},
    "ollama": {
      "modelWrapper": "AModelChatGPT",
      "apikey": "fakekey",
      "baseURL": "http://localhost:4000",
      "modelList": {
        "ollama/llama2:latest": {
          "contextWindow": 8192,
          "systemAsUser": false
        }
      }
    },
    "anthropic": {...}
  },
  "temperature": 0.0,
  "flashAttention2": false,
  "speechOn": false,
  "contextWindowRatio": 0.6,
  "services": {...}
}

ailice_web --modelID=hf:openchat/openchat_3.5 --prompt="main" --quantization=8bit --contextWindowRatio=0.6                                   

config.json is located at /home/hakuju/.config/ailice/config.json
In order to simplify installation and usage, we have set local execution as the default behavior, which means AI has complete control over the local environment. To prevent irreversible losses due to potential AI errors, you may consider one of the following two methods: the first one, run AIlice in a virtual machine; the second one, install Docker, use the provided Dockerfile to build an image and container, and modify the relevant configurations in config.json. For detailed instructions, please refer to the documentation.
The port range of the ext-modules has been changed from 2005-2016 to 59000-59200. If you are using an old version, startup failure will occur after updating the code. Please modify the port number in config.json and rebuild the docker image.
kill: (1198047): 不允许的操作 // not allowed operation
kill: (1212154): 没有那个进程 //no such process
scripter  started.
Encountered an exception, AIlice is exiting: 'storage'
  File "/data/AIlice/ailice/AIliceWeb.py", line 96, in main
    mainLoop(**kwargs)
  File "/data/AIlice/ailice/AIliceWeb.py", line 42, in mainLoop
    clientPool.Init()
  File "/data/AIlice/ailice/common/ARemoteAccessors.py", line 10, in Init
    storage = config.services['storage']['addr']

"maxMemory": {},
  "quantization": null,
  "openaiGPTKey": null,
  "temperature": 0.0,
  "flashAttention2": false,
  "speechOn": false,
  "contextWindowRatio": 0.6,
  "services": {
    "scripter": {
      "cmd": "docker start scripter",
      "addr": "tcp://127.0.0.1:59000"
    }
  }
}

import os, json

from ailice.common.utils.ATextSpliter import sentences_split
from ailice.common.AConfig import config
from ailice.core.llm.AFormatter import AFormatterQW
from dashscope import Generation

class AModelQW():
    def __init__(self, modelName: str):
        
        self.tokenizer = None
        self.modelName = modelName
        # self.client = openai.OpenAI(api_key = config.openaiGPTKey)
        self.api_key = os.environ.get("QWEN_KEY")
        self.formatter = AFormatterQW(systemAsUser=True)
        # self.contextWindow = 6122
        self.contextWindow=6122*0.6
        return
    
    def Generate(self, prompt: list[dict[str,str]], proc: callable, endchecker: callable, temperature: float = 0.2) -> str:
        # print("======>this is a generate: ", prompt)
        proc(txt='', action='open')
        currentPosition = 0
        text = ""
        responses = Generation.call(model=Generation.Models.qwen_max,
                                    result_format='message',
                                    messages=prompt,
                                    incremental_output=True,
                                    stream=True)
        for chunk in responses:
            if chunk.status_code == 200:
                text += (chunk.output.choices[0]['message']['content'] or "")

                if endchecker(text):
                    break
                
                sentences = [x for x in sentences_split(text[currentPosition:])]
                if (2 <= len(sentences)) and ("" != sentences[0].strip()):
                    proc(txt=sentences[0], action='append')
                    currentPosition += len(sentences[0])
            else:
                print(f"=====>response error, chunk id {chunk.request_id}, chunk status_code {chunk.status_code}, chunk code {chunk.code}, chunk message {chunk.message}")
                print(prompt)
        proc(txt=text[currentPosition:], action='close')
        return text

([{'role': 'system', 'content': '\nYou are an smart and helpful AI agent that helps user complete tasks that require programming. You will complete the task by interacting with agents with coding capabilities (hereinafter referred to as "coder").\nYour job is to communicate with the user to understand their needs, provide requirements for the coder, execute the returned code, extract possible running errors information and feed back to coder until the code runs correctly.\nFor programming tasks that are complex enough to consist of multiple modules, you can consider breaking down the task and assigning it to multiple agents to complete it.\nUsing function calls, you can create and interact with agents, reply to users, configure the program\'s running environment and execute programs.\nUse special symbols "<!|","|!>" as replacements for parentheses in function call syntax. Please pay special attention to the syntax when generating function call statements.\nOnly positional parameter function calls are supported, please do not use keyword parameters and keep the order of parameters consistent with the function definition.\nReferences are not supported in parameters of function call statements. Please use clear and straightforward expressions in function calls.\nFunctions with a single string parameter do not need to write quotation marks when passing parameters. So, do not use quotes when passing code into functions such as BASH or PYTHON, use "cat" command when you need to write code to a file. Avoid the extensive use of escape characters.\nFunction calls need to be placed at the end of your output, and prefixed with "!" to trigger execution.\nYou MUST call a function in your output.\nOnly the most recent rounds of historical conversations will be retained. To prevent the loss of information, please make sure to include summaries of key details, such as the user\'s requests, in the initial portion of your responses.\nIf you find that the environment variable A_IN_CONTAINER is predefined, you can install any necessary tools.\n\n"ext-modules" is a type of module that implements specific interfaces, defining a set of new function calls. Once loaded, you will be able to use the functions defined within it.\nDo not attempt to build ext-modules unless explicitly requested by the user. It adds complexity to the debugging process.\nPlease note that for ext-modules, you need to store the code in the current directory before running it, otherwise there will be an error that the python package cannot be found. You also need to run them as separate processes, or you will be stuck.\n\nYour workflow follows these steps (The details of functions such as CALL/PYTHON/BASH/RESPOND will be given below, and for the sake of simplicity in this context, the term \'coder\' is used to encompass both \'coder\' and \'module-coder\'):\n1. Receive user requirements, understand and engage with the user as needed to enhance understanding(use RESPOND).\n2. Choose or create a suitable coder agent, provide a clear and comprehensive description of the requirements to it(use CALL).\n3. Once coder returns the code, your responsibilities include:\n    Initial verification of whether the solution provided by the coder meets the requirements. If not, make modification requests to the coder.\n    Install the necessary dependencies following coder\'s instructions and execute the code (use BASH and PYTHON).\n    In case of error, send detailed error information to coder for code improvement (use CALL), the message include error and problematic lines of code (for the python case, the code is saved into a temp file like "/tmp/tmp*.py" before execute, you can identify this file in the top level of callstack). Go back to the beginning of step 3 and iterate until success.\n    During the iteration, if coder requires knowledge about specific libraries, query relevant knowledge through the "researcher" type agent (use CALL).\n4. Finally, return the execution result to the user (use RESPOND)."\n\nAvailable Functions:\n#Use this function to interact with an AI agent.\nCALL<!|agentType: str, agentName: str, msg: str|!>\nagentType: A string used to specify the type of AI agent. It can be of the following types:\n    1. "coder". An excellent coder, you need to leave any programming problems other than ext-modules building to him. It should be noted that he can only return the code to you, but does not have the authority to execute the code and configure the environment, this is where you need to help him.\n    2. "module-coder". The only agent capable of building ext-modules, and this is its sole responsibility.\n    3. "module-loader". An agent that can help you load and use ext-modules. Please pass in the address of the module when creating the agent, and then you can let it help you interact with the module. You can use this agent to assist in ext-module debugging.\n    4. "researcher". Suitable for technical problem search tasks such as library document or sample code search on the internet.\nagentName: The name of the AI agent instance. Create a new one if not found. Please use a concise name that reflects the agent’s responsibilities.\nmsg: message need to be told to the agent. The agent cannot see content other than msg. Please provide complete content in msg.\n\n#Dialog with user. Typical usage scenarios: when you need user to supplement task information, or need to report the current results to user.\nRESPOND<!|message: str|!>\n\n#Execute bash script. A timeout error will occur for programs that have not been completed for a long time.\nBASH<!|code: str|!>\n\n#Execute python code. Please note that you need to copy the complete code here, and you must not use references.\nPYTHON<!|code: str|!>\n\n#Wait for some seconds. The unit of the "duration" parameter is seconds. Useful when waiting for script or command execution to complete. \nWAIT<!|duration: int|!>\n\nEXAMPLE:\n!CALL<!|"coder", "clock_coder", "Build a program that can check CPU usage."|!>\n!PYTHON<!|print(\'hello wolrd.\')|!>\n\n\nEnd of general instructions.\n\nActive Agents: [\'file_finder: agentType coder\']\n\nRelevant Information: !CALL<!|"coder", "file_finder", "Write a Python script to find all .ppt files on the system and list their directories, with emphasis on those likely to contain study materials"|!>\n\nBased on the user\'s request,\nThe "RELEVANT INFORMATION" part contains data that may be related to the current task, originating from your own history or the histories of other agents. Please refrain from attempting to invoke functions mentioned in the relevant information, as you may not necessarily have the permissions to do so.\n\n'}, {'role': 'user', 'content': 'Function returned: {Agent file_finder returned: To create a Python script that finds all PowerPoint files (with a .ppt extension) on the system and lists their directories, with emphasis on those likely to contain study materials, we\'ll use the `os` and `glob` modules. The "emphasis" part is subjective, but we can consider directories containing keywords like "study," "education," "class," etc.\n\n```python\nimport os\nimport glob\n\n# List of keywords to identify directories potentially containing study materials\nstudy_keywords = ["study", "education", "class", "lecture", "presentation"]\n\ndef find_ppt_files():\n    # Find all ppt files in the file system\n    all_ppt_files = []\n    for root, dirs, files in os.walk("/"):\n        ppt_files = [os.path.join(root, f) for f in files if f.endswith(".ppt")]\n        all_ppt_files.extend(ppt_files)\n\n    # Filter directories based on study keywords\n    study_material_directories = set()\n    for ppt_file_path in all_ppt_files:\n        directory = os.path.dirname(ppt_file_path)\n        for keyword in study_keywords:\n            if keyword.lower() in directory.lower():\n                study_material_directories.add(directory)\n                break  # Add the directory only once even if it matches multiple keywords\n\n    # Print all ppt files along with their directories\n    print("All PowerPoint (.ppt) files:")\n    for ppt_file in all_ppt_files:\n        print(f"- {ppt_file}")\n\n    print("\\nDirectories likely to contain study materials:")\n    for dir in study_material_directories:\n        print(f"- {dir}")\n\nif __name__ == "__main__":\n    find_ppt_files()\n```\n\n**Instructions:**\n1. Replace the starting directory (`"/"`) with the appropriate starting directory if you don\'t want to search the entire file system.\n2. Run this script as a standalone program; it will list all .ppt files and highlight directories that might contain study materials.\n\nPlease note that searching the entire file system could take a significant amount of time depending on its size and structure. This script does not check the content of the PowerPoint files, only the directory names.\n}'}], 2221)