Code Monkey home page Code Monkey logo

python-django's Introduction

Django Opentracing

image

image

image

image

This package enables distributed tracing in Django projects via The OpenTracing Project. Once a production system contends with real concurrency or splits into many services, crucial (and formerly easy) tasks become difficult: user-facing latency optimization, root-cause analysis of backend errors, communication about distinct pieces of a now-distributed system, etc. Distributed tracing follows a request on its journey from inception to completion from mobile/browser all the way to the microservices.

As core services and libraries adopt OpenTracing, the application builder is no longer burdened with the task of adding basic tracing instrumentation to their own code. In this way, developers can build their applications with the tools they prefer and benefit from built-in tracing instrumentation. OpenTracing implementations exist for major distributed tracing systems and can be bound or swapped with a one-line configuration change.

If you want to learn more about the underlying python API, visit the python source code.

If you are migrating from the 0.x series, you may want to read the list of breaking changes.

Installation

Run the following command:

$ pip install django_opentracing

Setting up Tracing

In order to implement tracing in your system, add the following lines of code to your site's settings.py file:

import django_opentracing

# OpenTracing settings

# if not included, defaults to True.
# has to come before OPENTRACING_TRACING setting because python...
OPENTRACING_TRACE_ALL = True

# defaults to []
# only valid if OPENTRACING_TRACE_ALL == True
OPENTRACING_TRACED_ATTRIBUTES = ['arg1', 'arg2']

# Callable that returns an `opentracing.Tracer` implementation.
OPENTRACING_TRACER_CALLABLE = 'opentracing.Tracer'

# Parameters for the callable (Depending on the tracer implementation chosen)
OPENTRACING_TRACER_PARAMETERS = {
    'example-parameter-host': 'collector',
}

If you want to directly override the DjangoTracing used, you can use the following. This may cause import loops (See #10)

# some_opentracing_tracer can be any valid OpenTracing tracer implementation
OPENTRACING_TRACING = django_opentracing.DjangoTracing(some_opentracing_tracer)

Note: Valid request attributes to trace are listed here. When you trace an attribute, this means that created spans will have tags with the attribute name and the request's value.

Tracing All Requests

In order to trace all requests, OPENTRACING_TRACE_ALL needs to be set to True (the default). If you want to trace any attributes for all requests, then add them to OPENTRACING_TRACED_ATTRIBUTES. For example, if you wanted to trace the path and method, then set OPENTRACING_TRACED_ATTRIBUTES = ['path', 'method'].

Tracing all requests uses the middleware django_opentracing.OpenTracingMiddleware, so add this to your settings.py file's MIDDLEWARE_CLASSES at the top of the stack.

MIDDLEWARE_CLASSES = [
    'django_opentracing.OpenTracingMiddleware',
    ... # other middleware classes
]

Tracing Individual Requests

If you don't want to trace all requests to your site, set OPENTRACING_TRACE_ALL to False. Then you can use function decorators to trace individual view functions. This can be done by adding the following lines of code to views.py (or any other file that has url handler functions):

from django.conf import settings

tracing = settings.OPENTRACING_TRACING

@tracing.trace(optional_args)
def some_view_func(request):
    ... # do some stuff

This tracing method doesn't use middleware, so there's no need to add it to your settings.py file.

The optional arguments allow for tracing of request attributes. For example, if you want to trace metadata, you could pass in @tracing.trace('META') and request.META would be set as a tag on all spans for this view function.

Note: If OPENTRACING_TRACE_ALL is set to True, this decorator will be ignored, including any traced request attributes.

Accessing Spans Manually

In order to access the span for a request, we've provided an method DjangoTracing.get_span(request) that returns the span for the request, if it is exists and is not finished. This can be used to log important events to the span, set tags, or create child spans to trace non-RPC events.

Tracing an RPC

If you want to make an RPC and continue an existing trace, you can inject the current span into the RPC. For example, if making an http request, the following code will continue your trace across the wire:

@tracing.trace()
def some_view_func(request):
    new_request = some_http_request
    current_span = tracing.get_span(request)
    text_carrier = {}
    opentracing_tracer.inject(span, opentracing.Format.TEXT_MAP, text_carrier)
    for k, v in text_carrier.items():
        request.add_header(k,v)
    ... # make request

Example

Here is an example of a Django application that acts as both a client and server, with integrated OpenTracing tracers.

Breaking changes from 0.x

Starting with the 1.0 version, a few changes have taken place from previous versions:

  • DjangoTracer has been renamed to DjangoTracing, although DjangoTracer can be used still as a deprecated name. Likewise for OPENTRACING_TRACER being renamed to OPENTRACING_TRACING.
  • When using the middleware layer, OPENTRACING_TRACE_ALL defaults to True.
  • When no opentracing.Tracer is provided, DjangoTracing will rely on the global tracer.

Further Information

If you’re interested in learning more about the OpenTracing standard, please visit opentracing.io or join the mailing list. If you would like to implement OpenTracing in your project and need help, feel free to send us a note at [email protected].

python-django's People

Contributors

alanjds avatar asherf avatar bensigelman avatar bhs avatar carlosalberto avatar kcamenzind avatar lylescott avatar maguowei avatar mikebryant avatar nothingelsematters7 avatar omprakash95 avatar stuart-warren avatar tux-o-matic avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

python-django's Issues

Chunks with extreme offset cause ValueError

I guess we could further stricten our sanity checks to disallow chunks having greater than (1<<63)-1 offsets.
The offending chunks comes from the yaffs handler:

Traceback (most recent call last):
  File "/home/vlaci/devel/git/github.com/IoT-Inspector/unblob/unblob/finder.py", line 74, in search_chunks_by_priority
    chunk = handler.calculate_chunk(limited_reader, real_offset)
  File "/home/vlaci/devel/git/github.com/IoT-Inspector/unblob/unblob/handlers/filesystem/yaffs.py", line 148, in calculate_chunk
    files, offset = self.get_files(file, start_offset, config)
  File "/home/vlaci/devel/git/github.com/IoT-Inspector/unblob/unblob/handlers/filesystem/yaffs.py", line 105, in get_files
    file.seek(current_offset, io.SEEK_SET)
  File "/home/vlaci/devel/git/github.com/IoT-Inspector/unblob/unblob/file_utils.py", line 215, in seek
    new_pos = self._file.seek(offset, whence)
ValueError: cannot fit 'int' into an offset-sized integer

Setup instructions in conjunction with jaeger-client-python leads to pitfall

See jaegertracing/jaeger-client-python#60. Doing the obvious setup, according to the readme of these two projects, leads to a deadlock which is quite tricky to sort out.

I've not yet found an elegant workaround, though I'm not an expert in Jaeger, or Django. In the linked issue, it's suggested the Jaeger tracer not be initialized until after the WSGI server (gunicorn in my case) forks. That seems like a good idea, even if it didn't deadlock.

The closest solution would seem to be putting the initialization of the Jaeger tracer in gunicorns post_fork() hook (or similar for other WSGI servers). Trouble is by that time I'd think OpenTracingMiddleware has already saved a reference to the tracer. So I'm looking at doing something ugly.

Error: object has no attribute 'META'

Hi! I got the following error 'GetExampleView' object has no attribute 'META' when i implement @tracing.trace() inside a class that inherits from any kind of APiView related model.

class GetExampleView(APIView):
    @tracing.trace()
    def get(self, request):
        pass

I don't know if the fact that it was used inside a class and the attribute self (which is the GetExampleView) was sent before request caused this error.

There is any kind of support when this is implemented inside classes? i noticed that all examples where used directly in functions that received request as first param or maybe i'm doing something wrong with the implementation.

django project with this lib starting block

image
i start my app as above ,and it just get stuck with no Exception traceback,so where is the problem,
my code modify as your README.md,like this .

import django_opentracing
from jaeger_client import Config

config = Config(
        config={ # usually read from some yaml config
            'sampler': {
                'type': 'const',
                'param': 1,
            },
            'logging': True,
        },
        service_name='OpsManange')
# OpenTracing settings

# if not included, defaults to False
# has to come before OPENTRACING_TRACER setting because python...
OPENTRACING_TRACE_ALL = False,

# defaults to []
# only valid if OPENTRACING_TRACE_ALL == True
OPENTRACING_TRACED_ATTRIBUTES = ['arg1', 'arg2'],
open_tracer = config.initialize_tracer()
OPENTRACING_TRACER = django_opentracing.DjangoTracer(open_tracer)

Setup instructions result in an import loop

OPENTRACING_TRACER = django_opentracing.DjangoTracer(some_opentracing_tracer), in settings.py means:

  • we've had to import django_opentracing
  • which has imported django_opentracing.tracer
  • which imports django.conf
  • which should not be done during settings initialisation.

Settings should probably follow Django convention, which is not importing stuff in settings.py, but using stringified imports.

about RPC in django

if i want to get rpc in django, should i must use
@tracing.trace()
def some_view_func(request):
new_request = some_http_request
current_span = tracing.get_span(request)
text_carrier = {}
opentracing_tracer.inject(span, opentracing.Format.TEXT_MAP, text_carrier)
for k, v in text_carrier.items():
request.add_header(k,v)
... # make request
this function to get it?but i had a old project,i want to make its all api to get rpc, what should i do easily

Need for a lock when setting the global `Tracer`

Hey @mikebryant

Currently we have a lock used when setting opentracing.Tracer (and also, indirectly, when creating a Tracer through the OPENTRACING_TRACER_CALLABLE value), and I'm wondering whether this is needed.

Before Django 1.10, the middleware is supposed to be initialized only once when the first request takes place (https://docs.djangoproject.com/en/1.9/topics/http/middleware/#init)

As of Django 1.10, the middleware will be initialized only once at server start (https://docs.djangoproject.com/en/1.10/topics/http/middleware/#init-get-response)

Is there any advanced need to take a lock? I remember you mentioning forking, but I have forgotten the details now ;)

Trying to add opentracing support to an app without specifying a tracer fails

Desire: We can add tracing support to an app without being required to configure a tracer.

Currently: It breaks the app.

Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
Internal Server Error: /
Traceback (most recent call last):
  File "/home/m/.virtualenvs/quartz/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 204, in get_response
    response = middleware_method(request, response)
  File "/home/m/.virtualenvs/quartz/local/lib/python2.7/site-packages/django_opentracing/middleware.py", line 40, in process_response
    self._tracer._finish_tracing(request)
AttributeError: 'Tracer' object has no attribute '_finish_tracing'
quartz[(process)d]: [django.request][ERROR] Internal Server Error: /
Traceback (most recent call last):
  File "/home/m/.virtualenvs/quartz/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 204, in get_response
    response = middleware_method(request, response)
  File "/home/m/.virtualenvs/quartz/local/lib/python2.7/site-packages/django_opentracing/middleware.py", line 40, in process_response
    self._tracer._finish_tracing(request)
AttributeError: 'Tracer' object has no attribute '_finish_tracing'

PR to follow

Feature request: add support for ignored attributes

Would it be a good idea to have an option for OPENTRACING_IGNORED_ATTRIBUTES in addition to OPENTRACING_TRACED_ATTRIBUTES?

This would allow easy prevention of logging sensitive information such as auth headers etc.

OPENTRACING_TRACED_ATTRIBUTES passed as a tuple of a list

Right now I have
OPENTRACING_TRACED_ATTRIBUTES = ['path', 'method']

Which fails with the error:

Traceback (most recent call last):
  File "/home/parallels/hv4/hv/venv/local/lib/python2.7/site-packages/django/core/handlers/exception.py", line 41, in inner
    response = get_response(request)
  File "/home/parallels/hv4/hv/venv/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 178, in _get_response
    response = middleware_method(request, callback, callback_args, callback_kwargs)
  File "/home/parallels/hv4/hv/venv/local/lib/python2.7/site-packages/django_opentracing/middleware.py", line 36, in process_view
    self._tracer._apply_tracing(request, view_func, traced_attributes)
  File "/home/parallels/hv4/hv/venv/local/lib/python2.7/site-packages/django_opentracing/tracer.py", line 89, in _apply_tracing
    if hasattr(request, attr):
TypeError: hasattr(): attribute name must be string

It appears that OPENTRACING_TRACED_ATTRIBUTES is being passed as a tuple with the list inside of it.

So when django_opentracing/tracer.py:88 calls for attr in attributes: attr will be a list instead of a string. Here is a screen cap from my debugger.
screen shot 2018-07-20 at 1 51 22 pm

Usage with Tracer client: 'Settings' object has no attribute 'OPENTRACING_TRACING'

Scratching my head over how to use this library with a Tracer client (jaeger) inside my views

Do i need to explicitly set OPENTRACING_TRACING in settings.py?

Getting the following error on startup

AttributeError: 'Settings' object has no attribute 'OPENTRACING_TRACING'

settings.py

# default tracer is opentracing.Tracer(), which does nothing
OPENTRACING_TRACER_CALLABLE = __name__ + '.tracer'

def tracer():
    from jaeger_client import Config
    config = Config(
        config={
            'sampler': {
                'type': 'const',
                'param': 1,
            },
            'local_agent': {
                'reporting_host': os.environ.get('JAEGER_HOST', "localhost"),
                'reporting_port': os.environ.get('JAEGER_PORT', 6831),
            },
            'logging': True,
        },
        service_name='-'.join([os.environ.get('JAEGER_SERVICE_NAME', ''), 'api']))

    return config.initialize_tracer()


# default is False
OPENTRACING_TRACE_ALL = True

# default is []
OPENTRACING_TRACED_ATTRIBUTES = ['META']



views.py

from django.conf import settings

tracing = settings.OPENTRACING_TRACING

@tracing.trace(optional_args)
def some_view_func(request):
    ... # do some stuff

Feature Request: custom operation name in middleware

The current implementation of the DjangoTracing class forces the use of view_func.__name__ as the operation name for the span. I have a use case with class based views where the view func ends up being the same, but they are distinct operations. I would like the ability to customize this without overriding the entire _apply_tracing method.

Perhaps accept a kwarg for operation name so subclasses could handle it.

Initialization of the global tracer

Hey @mikebryant

I was checking the initialization of the global tracer, and I'm not sure this is the right approach. Specifically this is the part I'm talking about:

        if hasattr(settings, 'OPENTRACING_TRACER'):
            # Backwards compatibility with the old way of defining the tracer
            opentracing.tracer = settings.OPENTRACING_TRACER._tracer

Usually, they way we have done things for other instrumentation in Java, is to do it the other way around: if the user doesn't specify explicitly a tracer (through OPENTRACING_TRACER or OPENTRACING_TRACER_CALLABLE in this case), then fallback to opentracing.tracer).

Setting the opentracing.tracer instance right from python-django kinda breaks the contract, and I'm afraid that the user won't be able to set the global tracer himself if we do this - and also, I'm afraid other instrumentation libraries may also want to set opentracing.tracer each, and we end up in an override cascade.

Is there any need from your part to have this code running? Let me know, and maybe we can find a solution in the middle ;)

Django 4.1 all tests fail with 'OpenTracingMiddleware' object has no attribute '_is_coroutine'

[   37s] ERROR: test_middleware_untraced_no_trace_all (test_site.test_middleware.TestDjangoOpenTracingMiddleware)
[   37s] ----------------------------------------------------------------------
[   37s] Traceback (most recent call last):
[   37s]   File "/usr/lib/python3.8/site-packages/django/test/utils.py", line 460, in inner
[   37s]     return func(*args, **kwargs)
[   37s]   File "/home/abuild/rpmbuild/BUILD/python-django-1.1.0/tests/test_site/test_middleware.py", line 37, in test_middleware_untraced_no_trace_all
[   37s]     response = client.get('/untraced/')
[   37s]   File "/usr/lib/python3.8/site-packages/django/test/client.py", line 836, in get
[   37s]     response = super().get(path, data=data, secure=secure, **extra)
[   37s]   File "/usr/lib/python3.8/site-packages/django/test/client.py", line 424, in get
[   37s]     return self.generic(
[   37s]   File "/usr/lib/python3.8/site-packages/django/test/client.py", line 541, in generic
[   37s]     return self.request(**r)
[   37s]   File "/usr/lib/python3.8/site-packages/django/test/client.py", line 810, in request
[   37s]     self.check_exception(response)
[   37s]   File "/usr/lib/python3.8/site-packages/django/test/client.py", line 663, in check_exception
[   37s]     raise exc_value
[   37s]   File "/usr/lib/python3.8/site-packages/django/core/handlers/exception.py", line 55, in inner
[   37s]     response = get_response(request)
[   37s]   File "/usr/lib/python3.8/site-packages/django/utils/deprecation.py", line 131, in __call__
[   37s]     if self._is_coroutine:
[   37s] AttributeError: 'OpenTracingMiddleware' object has no attribute '_is_coroutine'
[   37s] 

python3 errors on iteritems()

With recent changes landed in jaeger-python-client python3 is now almost supported for django_opentracing.

I only had to modify one line (and remove erroneous commas from the Readme settings.py example) to get it to work.

My environment is
GCE kubernetes engine or docker-compose
pip install django_opentracing jaeger-client
jaeger.version=Python-3.7.1
django_opentracing/_version.py
{
"full-revisionid": "21b48cd0998ffdc3f1284341beb2a9606fd8f30e",
"version": "0.1.18"
}

For docker-compose environment I add this snippet to settings.py
import django_opentracing
from jaeger_client import Config
import logging

log_level = logging.DEBUG
logging.getLogger('').handlers = []
logging.basicConfig(format='%(asctime)s %(message)s', level=log_level)

config = Config(
    config={ # usually read from some yaml config
        'sampler': {
            'type': 'const',
            'param': 1,
        },
        'logging': True,
        # docker compose containers see jaeger all-in-one as <parent_folder>_jaeger_1
        'local_agent': {'reporting_host': 'core_jaeger_1'},
    },  
    service_name='web-app',
)

tracer = config.initialize_tracer()

# OpenTracing settings

# if not included, defaults to False
# has to come before OPENTRACING_TRACER setting because python...
OPENTRACING_TRACE_ALL = True

# defaults to []
# only valid if OPENTRACING_TRACE_ALL == True
OPENTRACING_TRACED_ATTRIBUTES = ['arg1', 'arg2',]

# Callable that returns an `opentracing.Tracer` implementation.
OPENTRACING_TRACER_CALLABLE = 'opentracing.Tracer'

# Parameters for the callable (Depending on the tracer implementation chosen)
OPENTRACING_TRACER_PARAMETERS = {
'example-parameter-host': 'collector',
}

OPENTRACING_TRACER = django_opentracing.DjangoTracer(tracer)`

I get the following error:

web-app_1 | File "/opt/conda/lib/python3.6/site-packages/django_opentracing/middleware.py", line 36, in process_view web-app_1 | self._tracer._apply_tracing(request, view_func, traced_attributes) web-app_1 | File "/opt/conda/lib/python3.6/site-packages/django_opentracing/tracer.py", line 67, in _apply_tracing web-app_1 | for k,v in request.META.iteritems(): web-app_1 | AttributeError: 'dict' object has no attribute 'iteritems'
exec into the django (web_app_1) container. Modify "/opt/conda/lib/python3.6/site-packages/django_opentracing/tracer.py", line 67 iteritems() -> items()`
Ctrl + C, docker-compose up to restart
Then I get

web-app_1 | File "/opt/conda/lib/python3.6/site-packages/django_opentracing/tracer.py", line 89, in _apply_tracing web-app_1 | if hasattr(request, attr): web-app_1 | TypeError: hasattr(): attribute name must be string

This is due to following the Readme.md too closely. The readme snippet for django settings.py has trailing commas for several items which causes the hasattr() errors. Remove the commas and everything worked.

I'm guessing my config settings (maybe OPENTRACING_TRACE_ALL = True) caused the iteritems() error to be exposed.

I will submit a pr with the fix

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.