Code Monkey home page Code Monkey logo

kaiju-scheduler's Introduction

pypi docs codecov tests mypy code style: black

python

kaiju-scheduler is a simple asynchronous tasks scheduler / executor for asyncio functions. It adds a bit of extra such as retries, timeouts, execution policies etc.

Installation

With pip and python 3.8+:

pip3 install kaiju-scheduler

How to use

See the user guide for more info.

Initialize a scheduler and schedule your procedure for periodic execution. Then start the scheduler.

from kaiju_scheduler import Scheduler

async def call_async_procedure(*args, **kws):
    ...

async def main():
    scheduler = Scheduler()
    scheduler.schedule_task(call_async_procedure, interval_s=10, args=(1, 2), kws={'value': True})
    await scheduler.start()
    ...
    await scheduler.stop()

Alternatively you can use the scheduler contextually.

async def main():
    async with Scheduler() as scheduler:
        scheduler.schedule_task(call_async_procedure, interval_s=10, args=(1, 2), kws={'value': True})

Scheduler.schedule_task returns a task object which you can enable / disable or supress the task execution in your code temporarily using task.suspend context. You can also access the previous call results from task.result attribute.

class Cache:

    def __init__(self, scheduler: Scheduler):
        self._scheduler = scheduler
        self._cache_task = self._scheduler.schedule_task(
            self.cache_all, interval_s=600, policy=scheduler.ExecPolicy.WAIT)

    async def cache_all(self):
        ...

    async def reconfigure_cache(self):
        async with self._cache_task.suspend():
            "Do something while the caching is suspended"

You can specify retries for common types of errors such as IOError or ConnectionError using retries parameter. The scheduler will try to retry the call on such type of error.

scheduler.schedule_task(call_async_procedure, interval_s=300, retries=3, retry_interval_s=1)

There are various policies considering task execution. See the reference for more info on that.

Server

There's also a simple 'server' for handling asyncio tasks inside Python. It extends the standard loop functionality with retries, timeouts and impose some rate limit and prevent the loop from growing infinitely.

The server returns an asyncio.Task object which can be awaited independently. The idea is that any error is not raised but instead returned inside of the result. This allows for more convenient handling of errors while using this in streams, queues and server applications.

See the reference for more info on server functions.

from kaiju_scheduler import Server


async def call_something(arg1: int, arg2: int):
    return arg1 + arg2


async def main():
    async with Server() as server:
        task = await server.call(call_something, [1, 2])
        await task

kaiju-scheduler's People

Contributors

violet-black avatar

Stargazers

 avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.