Code Monkey home page Code Monkey logo

pylogctx's Introduction

GitHub homepage BSD Licensed CI Status Version on PyPI

Python Logging Context

pylogctx is a library for enriching each logs records from a context. Typical usage is for adding some request_id to all logs in order to make troubleshooting more comfortable. This context is shared by all piece of code using logging, transparently.

import logging.config

from pylogctx import context as log_context


logging.config.dictConfig({
    'formatters': {'json': {
        '()': 'pythonjsonlogger.jsonlogger.JsonFormatter',
        'format': "%(asctime)s %(name)s %(levelname)s %(message)s",
    }},
    'filters': {'context': {
        '()': 'pylogctx.AddContextFilter',
    }},
    'handlers': {'console': {
        'class': 'logging.StreamHandler',
        'filters': ['context'],
        'formatter': 'json',
    }},
    'root': {
        'level': 'INFO',
        'handlers': ['console'],
    },
})


logger = logging.getLogger(__name__)


def mycode(request, ticket_id):
    # push new fields
    log_context.update(requestId=uuid.uuid4())
    myticket = get_object_or_404(models.Ticket, pk=ticket_id)

    # push objects, they will be adapted to log fields
    log_context.update(myticket):

    # Log as usual
    logger.info("Working on %r", myticket)

    for comment in myticket.comments:
        # A context manager allow to push and pop fields
        with log_context(comment):
            logger.info("Working on comment %r", comment)
            # code, use external libs, etc.

    # Don't forget to clear the context for the next request. Use the
    # middleware to have it clean.
    log_context.clear()

The output looks like:

{'loggerName': 'package.module', 'levelname': 'INFO', 'message': 'Working on <Ticket #1>', 'ticketId': 1, 'requestId': 'c5521138-031a-4da6-b9db-c9eda3e090f1'}
{'loggerName': 'package.module', 'levelname': 'INFO', 'message': 'Working on comment <Comment #4>', 'ticketId': 1, 'ticketCommentId': 4, 'requestId': 'c5521138-031a-4da6-b9db-c9eda3e090f1'}
{'loggerName': 'package.module', 'levelname': 'INFO', 'message': 'Working on comment <Comment #5>', 'ticketId': 1, 'ticketCommentId': 5, 'requestId': 'c5521138-031a-4da6-b9db-c9eda3e090f1'}
{'loggerName': 'package.module', 'levelname': 'INFO', 'message': 'Working on comment <Comment #78>', 'ticketId': 1, 'ticketCommentId': 78, 'requestId': 'c5521138-031a-4da6-b9db-c9eda3e090f1'}
{'loggerName': 'package.module', 'levelname': 'INFO', 'message': 'Working on comment <Comment #9>', 'ticketId': 1, 'ticketCommentId': 9, 'requestId': 'c5521138-031a-4da6-b9db-c9eda3e090f1'}
{'loggerName': 'package.module', 'levelname': 'INFO', 'message': 'Working on <Ticket #890>', 'ticketId': 890, 'requestId': 'c64aaae7-049b-4a02-929b-2d0ac9141f5c'}
{'loggerName': 'package.module', 'levelname': 'INFO', 'message': 'Working on comment <Comment #80>', 'ticketId': 890, 'ticketCommentId': 80, 'requestId': 'c64aaae7-049b-4a02-929b-2d0ac9141f5c'}

Install it with your favorite python package installer:

$ pip install pylogctx

There is a few helpers for Celery and Django projects. See USAGE for details!

Contributors

Join us to make log rocking better! Read HACKING and ask maintainers:

pylogctx's People

Contributors

bersace avatar dsteinberger avatar hsmett avatar lorehov avatar moulde avatar nnachit avatar vrialland avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pylogctx's Issues

Is this library maintained?

I've noticed builds are failing and that there hasn't been a release since 2018. I also opened this PR #47 (without discussion on the feature though). But, is this library still maintained?

Nested dict

J'avais un souci, c'est que pour les logs à plusieurs niveau, je ne pouvais pas ajouter un log dans un enfant, lors de l'appel du log (un peu comme un ajout d'un extra)

Alors j'ai créé un filtre qui permet de passer des clés a nester, un exemple ça sera plus simple que d'expliquer :

en clair je permet d'update dans le context de log ce genre de param : peopledoc__domain__id=4 pour qu'il soit transformé en (split sur le séparator __):

{
   "peopledoc":{
         "domain":{
                "id": 4 }}}

Et si on souhaite ajouter un peu plus tard peopledoc__domain__name=foo

{
   "peopledoc":{
         "domain":{
                "id": 4,
                "name": "foo" }}}

Mais je ne sais pas si je l'incorpore ici ? ou si je le garde que dans RH2 ? ça fonctionne aussi dans les adapter to class, donc ça me parait une feature intéressante, qui pourrait être utile, mais peut être trop spécifique, j'ai du mal à dire pour une lib publique ?

Tu en penses quoi @bersace ?

Python3 Deprecation warning

Here are some deprecation warnings to fix :)

python3.6/site-packages/pylogctx/celery.py:32: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec()
    args_repr = inspect.getargspec(self.before_call)
python3.6/site-packages/pylogctx/celery.py:50: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec()
    args_repr = inspect.getargspec(self.after_call)

Pull request coming soon :)

celery: add args/kwargs to `before_update()`

It would be great to have access to args/kwargs in before_update().

The fix is easy but it will break compatibility with previous versions.
Should I change this method ? add a new one ?

Adding support for contextvars

Currently this package uses thread locals to maintain the logging context. However, that doesn't work for async code, especially ASGI servers (which may concurrently handle multiple requests with a single thread).

The way things like this are typically handled in the async world is through the use of contextvars. (They were added in 3.7, but there is a backport package for 3.6.)

Would there be any possibility of accepting a PR adding optional support for the use of contextvars to manage the logging context rather than thread locals?

I'm imagining something along the lines of adding a top level use_contextvars() call that would attempt to import contextvars (or the 3.6 backport), and if successful would replace the pylogctx.core.context threading.Local instance with a contextvars.ContextVar or similar.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.