Code Monkey home page Code Monkey logo

abbot-py's Introduction

Abbot Python Skill Runner

This is the Python skill runner for Abbot hosted as an Azure Function (abbot-skills-python).

Local Development

Start with script/bootstrap.

To run the Abbot Python runner locally:

script/server.

Testing

Add tests by adding files in src/tests that match the pattern test_*.py. We use the unittest testing framework. Tests can viewed, run, and debugged in VS Code.

Run all tests from the command line with:

script/test

Related Repositories

Known Issues

Packages with C dependencies won't get compiled in Azure. We have to deploy binary versions (i.e. psycopg2-binary).

GitHub Actions doesn't seem to be deploying binaries, so publish the app directly from the src directory when adding those dependencies:

func azure functionapp publish abbot-skills-python --python

abbot-py's People

Contributors

analogrelay avatar pmn avatar haacked avatar shana avatar dependabot[bot] avatar dahlbyk avatar

Watchers

 avatar  avatar  avatar

abbot-py's Issues

Reading missing key from the brain fails

Calling bot.brain.get("foo") fails if foo does not exist. It should return None instead.

Example error: https://app.ab.bot/skills/read-empty-key-repro/activity/2c453d65-7e1a-4601-868c-ef421f3069c2

Python Runner exception:

Runtime Error
Traceback (most recent call last):
File "/home/site/wwwroot/SkillRunner/bot/apiclient.py", line 91, in send
result.raise_for_status()
File "/home/site/wwwroot/requests/models.py", line 943, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://app.ab.bot/api/skills/2384/brain?key=some_key

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/site/wwwroot/SkillRunner/bot/bot.py", line 166, in run_user_script
policy.exec(self.code, script_locals)
File "/home/site/wwwroot/SkillRunner/bot/policy.py", line 328, in exec
exec(byte_code, globals)
File "<skill code>", line 1, in <module>
File "/home/site/wwwroot/SkillRunner/bot/storage.py", line 27, in get
output = self._api_client.get(path)
File "/home/site/wwwroot/SkillRunner/bot/apiclient.py", line 56, in get
return self.send(path, 'GET')
File "/home/site/wwwroot/SkillRunner/bot/apiclient.py", line 96, in send
self.logger.exception(f"There was an error {method}ing to {url}", e)
File "/usr/local/lib/python3.9/logging/__init__.py", line 1481, in exception
self.error(msg, *args, exc_info=exc_info, **kwargs)
File "/usr/local/lib/python3.9/logging/__init__.py", line 1475, in error
self._log(ERROR, msg, args, **kwargs)
File "/usr/local/lib/python3.9/logging/__init__.py", line 1589, in _log
self.handle(record)
File "/usr/local/lib/python3.9/logging/__init__.py", line 1599, in handle
self.callHandlers(record)
File "/usr/local/lib/python3.9/logging/__init__.py", line 1661, in callHandlers
hdlr.handle(record)
File "/usr/local/lib/python3.9/logging/__init__.py", line 952, in handle
self.emit(record)
File "/azure-functions-host/workers/python/3.9/LINUX/X64/azure_functions_worker/dispatcher.py", line 713, in emit
msg = self.format(record)
File "/usr/local/lib/python3.9/logging/__init__.py", line 927, in format
return fmt.format(record)
File "/usr/local/lib/python3.9/logging/__init__.py", line 663, in format
record.message = record.getMessage()
File "/usr/local/lib/python3.9/logging/__init__.py", line 367, in getMessage
msg = msg % self.args
TypeError: not all arguments converted during string formatting

Load a full python object for incoming JSON objects

When loading the properties of bot, we do stuff like this:

self.from_user = skillInfo.get('From')

That works fine for simple properties, but in the case of bot.from_user it leads to surprising results. For example, to get the user name, you can't do the following:

  1. bot.from_user.name NOPE!
  2. bot.from_user.Name NOPE!

Instead you have to do this: bot.from_user.get('Name'). I don't think any python skill author would expect that. @pmn thinks there's a built-in that might help with this.

Triggers are raising 500 errors

Reported by user netsplit in Discord:

When sending a request to a trigger, I'm getting HTTP 500 responses back with the following message. How would I setup my function to ensure HTTP 200 is returned?

[
  {
    "ErrorId": "AttributeError",
    "Description": "'TriggerResponse' object has no attribute '_raw_content'",
    "LineStart": 0,
    "LineEnd": 0,
    "SpanStart": 0,
    "SpanEnd": 0,
    "StackTrace": null
  }
]

DEPRECATION warning for Python 2.7

When I run PIP I get a warning:

DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.

What Python should we be on and is this something we can easily set up in bootstrap?

API endpoint configurations should fall back to localhost if not set

The production API endpoints are the fallback if the runner's environment does not specify an endpoint. This was done when we were running into issues maintaining env clearly. Since that problem has since been resolved, the fallback endpoints should point to https://localhost:4979 to make local development simpler (and to break correctly if the runner environment isn't set).

The variables that need to be updated are AbbotReplyApiUrl and SkillApiBaseUriFormatString.

Binary dependencies are not being installed when new versions are deployed via GitHub Actions

Noticed that someone was getting an error trying to run the sql skill:

libpq.so.5: cannot open shared object file: No such file or directory

This means that the psycopg2 binary was not loaded (the postgres drivers). Something is happening when we deploy abbot-py via Actions versus local deployment that is causing binaries to not load correctly.

The short term workaround is to deploy from local machine with the following, but the Actions deploy should always Do The Right Thing:tm:

Workaround command (run from the src directory):

func azure functionapp publish abbot-skills-python --verbose --python

Open Source this repository

It's a lot easier to write Abbot Python skills if developers can see how their code is run. Plus, there are a lot of smart Python developers out there who might have great improvements to how we run their code.

Before we open source anything, we need to:

  • Review the source code to ensure there are no secrets.
  • Review the commit history to ensure there are no... inappropriate commit messages.
  • Get a ๐Ÿ‘ from @haacked, who is the permission-giver for these sorts of things.

Direct db access with psycopg2 is causing timeouts

Originally reported by a user in Discord who was modifying their own version of a SQL command and noticed that even though the command completed successfully, Abbot was reporting an exception after some period of time.

On review of the audit logs, it looked like there was some kind of a timeout. My suspicion is that opening a connection pool with psycopg2 is keeping the runner working until it gets killed.

Here's the code they're using:

import psycopg2
import pandas


def run_query(query):
  connstr = bot.secrets.read("connstring")

  if not connstr:
    return "There's no connection string set up. Please add one before running this skill."
  
  with psycopg2.connect(connstr) as conn:
    conn.autocommit = True
    df = pandas.read_sql(query, conn)
    return f"```{df.to_markdown()}```"
   
QUERIES = {
    "test": """ redacted""",
    "astatus": """ redacted """,
}

# Check to see if the user has asked for a predefined query.
if bot.arguments in QUERIES:
  result = run_query(QUERIES[bot.arguments])
else:
  result = "I dont' recognize that query. Available queries are: "
  for k, _ in QUERIES.items():
    result += f"\n * {k}"

bot.reply(result)                    

Blank responses in `abbot-cli` and the console when using a custom image

When using a custom image, Abbot's console and abbot-cli return blank responses even if the skill ran correctly.

To replicate, set up a custom Python runner. The instructions for setting up a runner on Fly are good for this.

Once the custom runner is set up, create a new Python skill. The default skill code is sufficient for testing. Try running the skill in the console, and then run in the Slack workspace. Currently, the console will return a blank response and running in Slack will return correctly:

console:
Screenshot 2023-06-08 at 11 51 27 AM

Slack:
Screenshot 2023-06-08 at 11 51 56 AM

The audit log is reporting all runs successful:
Screenshot 2023-06-08 at 11 52 51 AM

ImportError: cannot import name 'cygrpc' from 'grpc._cython'

Brand new MacBook M1. When I run script/server for this repository (I already ran script/bootstrap, I get an error.

ImportError: cannot import name 'cygrpc' from 'grpc._cython' (/usr/local/Cellar/azure-functions-core-tools@3/3.0.3904/workers/python/3.8/OSX/X64/grpc/_cython/__init__.py)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.