powergo / django-celery-fulldbresult Goto Github PK
View Code? Open in Web Editor NEWDjango Celery DB Backend that keeps enough info to retry a task.
License: Other
Django Celery DB Backend that keeps enough info to retry a task.
License: Other
requirements
circle-ci
readme
setup.py
private key for bart (pypi)
Make sure that the task is saved before it is published so we can track tasks that never complete.
When running migration, with Django 2.0 got this traceback
venv/lib64/python3.6/site-packages/MySQLdb/connections.py", line 292, in query
mysql.connection.query(self, query)
django.db.utils.OperationalError: (1071, 'Specified key was too long; max key length is 767 bytes')
Applying django_celery_fulldbresult.0001_initial...
Django==2.0
celery==3.1.25
django-celery==3.2.2
django-celery-fulldbresult==0.5.3
Here's a problem I've noticed only with the latest version of django-celery-fulldbresult.
Description of problem
After setting up the fulldbresult backend with an existing Celery and django-celery installation, I saw this error from the Celery beat worker:
SchedulingError: Couldn't apply scheduled task Send queued mail: unbound method apply_async() must be called with Task instance as first argument (got TaskType instance instead)
The particular task had been run previously as set up as a djcelery PeriodicTask in the admin.
Environment
The worker is executed with the ./manage.py celery beat
command.
Package versions:
django
: 1.9.7
celery
: 3.1.23
django-celery
: 3.1.17
django-celery-fulldbresult
: 0.5.0
_Everything works fine if I drop down to the previous version of django-celery-fulldbersult
, 0.4.1.
My analysis
I added IPython.embed
into the new_apply_async
function used to monkeypatch Task.apply_async
(as the first line in the function) and discovered that self
is a celery.app.TaskType
instance, not a celery.app.Task
instance.
Here's the session output (with a few names updated):
In [1]: type(self)
Out[1]: [2016-08-29 17:21:20,401: WARNING/MainProcess] celery.app.task.TaskType
In [2]: self??
[2016-08-29 17:21:22,875: WARNING/MainProcess] Init signature: self(name, *args, **kwargs)
Source:
@task(name="run_management_command")
def run_management_command(name, *args, **kwargs):
call_command(name, *args, **kwargs)
File: ~/Projects/myproject/app/tasks.py
Type: TaskType
That's the full task definition but Python sees a TaskType
not a Task
instance. I can't quite put my finger on why this is happening, but I do wonder if it has to do with how the monkeypatch is applied. Of course if no one else is experiencing this then not sure how that'd be the case.
Make sure that scheduled tasks without a schedule id are never collected by backend.cleanup.
Executing this code in celery task
t = TaskResultMeta.objects.filter(Q(task__startswith='myapp.tasks.sm')).filter(some_query)
t.delete()
Using Django 1.9.2 with Mysql.
Due to celery/celery#2218 being closed, it seems we will have to create an alternative strategy to schedule the execution of a a task.
We must force the serialization method, otherwise kombu does not serialize strings and an error occurs when deserializing.
Create a list filter in the admin of PeriodicTask and TaskResultMeta that enables an admin to filter by task name.
Move our own customization to django-celery-fulldbresult. Django-celery has very limited default and a questionable list_display order (a boolean flag as the entry link).
The task name filter used for PeriodicTask in #13 is supposed to be the task name, not the periodic task name.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.