Timeout for waiting Self.Chanel about django-health-check HOT 15 CLOSED

If you guys are running this inside a docker container, I've experienced this as well, and fixed it by reloading the celery app within Health Check

like so:

            app = Celery('HealthCheckApp')
            app.config_from_object('django.conf:settings', namespace='CELERY')
            installed_apps = [app_config.name for app_config in apps.get_app_configs()]
            app.autodiscover_tasks(lambda: installed_apps, force=True)

Apparently there is an issue when you run celery inside a thread on docker that it fails to get the config correctly...

from django-health-check.

Hi @thanhcg
Thanks for reaching out. I took the liberty to add some code highlighting.
Would be be so good to tell me, which version of django-health-check you are using. Can please also provide me with a full stack trace of your error? That would help me a lot.

Thanks
-Joe

from django-health-check.

@henryfentanes I was trying a few more ways to see if I could figure out myself - before asking you the question. So I deleted the question.
One thing I didn't realize was that I would be getting such a quick reply! I really appreciate that! Thanks!
I'll try with your suggestion to see if it works! 👍

from django-health-check.

You're welcome @mhoonjeon. I really hope it works 😬 👍, please let us know...

from django-health-check.

Hi @codingjoe,
I am using the latest version of the django-health-check (2.3.0).
I cannot get the error details because the code is stopped in the messaging.py file line 187:
channel = self.channel

I cannot catch the exception. Please help me to fix it.
Thanks,
Thanh

from django-health-check.

@thanhcg you said you are getting a timeout exception. That is the one I am curious about. Can you please provide me with one? Otherwise I don't know how to help you. Thanks -Joe

from django-health-check.

Just a note that (I think) I experienced this issue as well. I enabled the health_check.contrib.celery app and none of my tasks would execute. They would just hang. It appears that the health check is blocking tasks from being registered by the celery worker. I am using version 3.10.2. I can try to relay more information if necessary. I have disabled it for now. All other facets of your plugin work really well, so thank you for those!

from django-health-check.

I will test this! Thank you!

from django-health-check.

@peterflat any update? Did it work?

from django-health-check.

Hi there, has there been any progress here? I'm hitting this issue too...

Thanks for this module btw :)

from django-health-check.

For anyone coming here, encountering this issue in a cloud environment - I've added #209 as a custom health check to my application and disabled this contrib one. Things seems to work as expected now ;)

@codingjoe might be worth getting #209 merged as some point?

from django-health-check.

@bynare yes, that is a good point. Would since you have a working copy, maybe you want to take over from @witold-gren and finish the patch?

from django-health-check.

If you guys are running this inside a docker container, I've experienced this as well, and fixed it by reloading the celery app within Health Check
like so:

            app = Celery('HealthCheckApp')
            app.config_from_object('django.conf:settings', namespace='CELERY')
            installed_apps = [app_config.name for app_config in apps.get_app_configs()]
            app.autodiscover_tasks(lambda: installed_apps, force=True)

Apparently there is an issue when you run celery inside a thread on docker that it fails to get the config correctly...

@henryfentanes where exactly did you put the code in? Did you override the default CeleryPingHealthCheck? I don't understand exactly what you mean by within Health Check to reload the celery app.

Sorry if it wasn't clear, but yes, in my case I used CeleryHealthCheck (https://github.com/KristianOellegaard/django-health-check/blob/master/health_check/contrib/celery/backends.py).

e.g.:
You can create your own health check like this, based off an existing one ...

class CeleryCheck(BaseHealthCheckBackend):

    def check_status(self):
        timeout = getattr(settings, 'HEALTHCHECK_CELERY_TIMEOUT', 3)

        try:
            app = Celery('HealthCheckApp')
            app.config_from_object('django.conf:settings', namespace='CELERY')
            installed_apps = [app_config.name for app_config in apps.get_app_configs()]
            app.autodiscover_tasks(lambda: installed_apps, force=True)

            result = app.send_task(
                'app.tasks.add',
                args=[4, 4],
                expires=timeout,
                queue=self.queue
            )
            result.get(timeout=timeout)

            if result.result != 8:
                self.add_error(ServiceReturnedUnexpectedResult("Celery returned wrong result"))

        except IOError as e:
            self.add_error(ServiceUnavailable("IOError"), e)
        except NotImplementedError as e:
            self.add_error(ServiceUnavailable("NotImplementedError: Make sure CELERY_RESULT_BACKEND is set"), e)
        except BaseException as e:
            self.add_error(ServiceUnavailable("Unknown error"), e)

from django-health-check.

@henryfentanes Works for me. Thanks! You saved my days :)

from django-health-check.

It seems I missed a lot and @henryfentanes solved the issue. Thanks!

from django-health-check.