powergo / django-celery-fulldbresult Goto Github PK

Make sure that scheduled tasks without a schedule id are never collected by backend.cleanup.

Make sure that the task is saved before it is published so we can track tasks that never complete.

Create a list filter in the admin of PeriodicTask and TaskResultMeta that enables an admin to filter by task name.

Move our own customization to django-celery-fulldbresult. Django-celery has very limited default and a questionable list_display order (a boolean flag as the entry link).

We must force the serialization method, otherwise kombu does not serialize strings and an error occurs when deserializing.

• Fix the bug
• Add a test case if possible
• Create a migration command, fix_json_result, that loops through all results and add missing quotes. Could even be used for legacy db.

When running migration, with Django 2.0 got this traceback

venv/lib64/python3.6/site-packages/MySQLdb/connections.py", line 292, in query
mysql.connection.query(self, query)
django.db.utils.OperationalError: (1071, 'Specified key was too long; max key length is 767 bytes')
Applying django_celery_fulldbresult.0001_initial...

Django==2.0
celery==3.1.25
django-celery==3.2.2
django-celery-fulldbresult==0.5.3

Due to celery/celery#2218 being closed, it seems we will have to create an alternative strategy to schedule the execution of a a task.

requirements
circle-ci
readme
setup.py
private key for bart (pypi)

The task name filter used for PeriodicTask in #13 is supposed to be the task name, not the periodic task name.

Executing this code in celery task

t = TaskResultMeta.objects.filter(Q(task__startswith='myapp.tasks.sm')).filter(some_query)
t.delete()

Using Django 1.9.2 with Mysql.

Here's a problem I've noticed only with the latest version of django-celery-fulldbresult.

Description of problem

After setting up the fulldbresult backend with an existing Celery and django-celery installation, I saw this error from the Celery beat worker:

SchedulingError: Couldn't apply scheduled task Send queued mail: unbound method apply_async() must be called with Task instance as first argument (got TaskType instance instead)

The particular task had been run previously as set up as a djcelery PeriodicTask in the admin.

Environment

The worker is executed with the ./manage.py celery beat command.

Package versions:

django: 1.9.7
celery: 3.1.23
django-celery: 3.1.17
django-celery-fulldbresult: 0.5.0

_Everything works fine if I drop down to the previous version of django-celery-fulldbersult, 0.4.1.

My analysis

I added IPython.embed into the new_apply_async function used to monkeypatch Task.apply_async (as the first line in the function) and discovered that self is a celery.app.TaskType instance, not a celery.app.Task instance.

Here's the session output (with a few names updated):

In [1]: type(self)
Out[1]: [2016-08-29 17:21:20,401: WARNING/MainProcess] celery.app.task.TaskType
In [2]: self??
[2016-08-29 17:21:22,875: WARNING/MainProcess] Init signature: self(name, *args, **kwargs)
Source:
@task(name="run_management_command")
def run_management_command(name, *args, **kwargs):
    call_command(name, *args, **kwargs)
File:           ~/Projects/myproject/app/tasks.py
Type:           TaskType

That's the full task definition but Python sees a TaskType not a Task instance. I can't quite put my finger on why this is happening, but I do wonder if it has to do with how the monkeypatch is applied. Of course if no one else is experiencing this then not sure how that'd be the case.