Code Monkey home page Code Monkey logo

djproxy's Introduction

djproxy

Build Status Coverage Status Latest Version PyPI - Python Version PyPI - Downloads

Why?

If an application depends on a proxy (to get around Same Origin Policy issues in JavaScript, perhaps), djproxy can be used to provide that functionality in a web server agnostic way. This allows developers to keep local development environments for proxy dependent applications fully functional without needing to run anything other than the django development server.

djproxy is also suitable for use in production environments and has been proven to be performant in large scale deployments. However, a web server's proxy capabilities will be more performant in many cases. If one needs to use this in production, it should be fine as long as upstream responses aren't large. Performance can be further increased by aggressively caching upstream responses.

Note that djproxy doesn't currently support websockets.

Installation

pip install djproxy

djproxy requires requests >= 1.0.0, django >= 1.11 and python >= 2.7. The goal is to maintain compatibility with all versions of Django that are still officially supported. However, djproxy may still work with older versions.

If you encounter issues using djproxy with a supported version of django, please report it.

Usage

Start by defining a new proxy:

from djproxy.views import HttpProxy

class LocalProxy(HttpProxy):
    base_url = 'https://google.com/'

Add a url pattern that points at the proxy view. The url kwarg will be urljoined with base_url:

urlpatterns = [
    url(r'^local_proxy/(?P<url>.*)$', LocalProxy.as_view(), name='proxy')
]

/local_proxy/some/content will now proxy https://google.com/some/content/.

Additional examples can be found here: views, urls.

HttpProxy configuration:

HttpProxy view's behavior can be further customized by overriding the following class attributes.

  • base_url: The proxy url is formed by urlparse.urljoin(base_url, url_kwarg)
  • ignored_upstream_headers: A list of headers that shouldn't be forwarded to the browser from the proxied endpoint.
  • ignored_request_headers: A list of headers that shouldn't be forwarded to the proxied endpoint from the browser.
  • proxy_middleware: A list of proxy middleware to apply to request and response data.
  • pass_query_string: A boolean indicating whether the query string should be sent to the proxied endpoint.
  • reverse_urls: An iterable of location header replacements to be made on the constructed response (similar to Apache's ProxyPassReverse directive).
  • verify_ssl*: This attribute corresponds to requests' verify parameter. It may be either a boolean, which toggles SSL certificate verification on or off, or the path to a CA_BUNDLE file for private certificates.
  • cert*: This attribute corresponds to requests' cert parameter. If a string is specified, it will be treated as a path to an ssl client cert file (.pem). If a tuple is specified, it will be treated as a ('cert', 'key') pair.
  • timeout*: This attribute corresponds to requests' timeout parameter. It is used to specify how long to wait for the upstream server to send data before giving up. The value must be either a float representing the total timeout time in seconds, or a (connect timeout float, read timeout float) tuple.

* The behavior changes that result from configuring verify_ssl, cert, and timeout will ultimately be dependent on the specific version of requests that's installed. For example, in older versions of requests, tuple values are not supported for the cert and timeout properties.

Adjusting location headers (ProxyPassReverse)

Apache has a directive called ProxyPassReverse that makes replacements to three location headers: URI, Location, and Content-Location. Without this functionality, proxying an endpoint that returns a redirect with a Location header of http://foo.bar/go/cats/ would cause a downstream requestor to be redirected away from the proxy. djproxy has a similar mechanism which is exposed via the reverse_urls class variable. The following proxies are equivalent:

Djproxy:

class ReverseProxy(HttpProxy):
    base_url = 'https://google.com/'
    reverse_urls = [
        ('/google/', 'https://google.com/')
    ]

urlpatterns = patterns[
    url(r'^google/(?P<url>.*)$', ReverseProxy.as_view(), name='gproxy')
]

Apache:

<Proxy https://google.com/*>
    Order deny,allow
    Allow from all
</Proxy>
ProxyPass /google/ https://google.com/
ProxyPassReverse /google/ https://google.com/

HttpProxy dynamic configuration and route generation helper:

To specify the configuration for a set of proxies, without having to maintain specific classes and url routes, one can use djproxy.helpers.generate_routes as follows:

In urls.py, pass generate_routes a configuration dict to configure a set of proxies:

from djproxy.urls import generate_routes

configuration = {
    'test_proxy': {
        'base_url': 'https://google.com/',
        'prefix': '/test_prefix/',
    },
    'service_name': {
        'base_url': 'https://service.com/',
        'prefix': '/service_prefix/',
        'verify_ssl': False,
        'append_middlware': ['myapp.proxy_middleware.add_headers']
    }
}

urlpatterns += generate_routes(configuration)

Using the snippet above will enable a Django app to proxy https://google.com/X at /test_prefix/X and https://service.com/Y at /service_prefix/Y.

These correspond to the following production Apache proxy configuration:

<Proxy https://google.com/*>
    Order deny,allow
    Allow from all
</Proxy>
ProxyPass /test_prefix/ https://google.com/
ProxyPassReverse /test_prefix/ https://google.com/


<Proxy https://service.com/*>
    Order deny,allow
    Allow from all
</Proxy>
ProxyPass /service_prefix/ http://service.com/
ProxyPassReverse /service_prefix/ http://service.com/

Required configuration keys:

  • base_url
  • prefix

Optional configuration keys:

  • verify_ssl: defaults to True.
  • csrf_exempt: defaults to True.
  • cert: defaults to None.
  • timeout: defaults to None.
  • middleware: Defaults to None. Specifying None causes djproxy to use the default middleware set. If a list is passed, the default middleware list specified by the HttpProxy definition will be replaced with the provided list.
  • append_middleware: Defaults to None. None results in no changes to the default middleware set. If a list is specified, the list will be appended to the default middleware list specified in the HttpProxy definition or, if provided, the middleware key specified in the config dict.

Proxy middleware

HttpProxys support custom middleware for preprocessing data from downstream to be sent to upstream endpoints and for preprocessing response data before it is sent back downstream. X-Forwarded-Host, X-Forwarded-For, X-Forwarded-Proto and the ProxyPassRevere functionality area all implemented as middleware.

HttProxy views are configured to execute particular middleware by setting their proxy_middleware attribute. The following HttpProxy would attach XFF and XFH headers, but not preform the ProxyPassReverse header translation or attach an XFP header:

class ReverseProxy(HttpProxy):
    base_url = 'https://google.com/'
    reverse_urls = [
        ('/google/', 'https://google.com/')
    ]
    proxy_middleware = [
        'djproxy.proxy_middleware.AddXFF',
        'djproxy.proxy_middleware.AddXFH'
    ]

If a custom middleware is needed to modify content, headers, cookies, etc before the content is sent upstream of if one needs to make similar modifications before the content is sent back downstream, a custom middleware can be written and proxy views can be configured to use it. djproxy contains a middleware template to make this process easier.

Terminology

It is important to understand the meaning of these terms in the context of this project:

upstream: The destination that is being proxied.

downstream: The agent that initiated the request to djproxy.

Contributing

To run the tests, first install development dependencies:

pip install -r requirements.txt

To test this against a version of Django other than the latest supported on the test environment's Python version, wipe out the requirements.txt installation by pip installing the desired version.

Run nosetests to execute the test suite.

To automatically run the test suite, flake8, and pep257 checks whenever python files change use testtube by executing stir in the top level djproxy directory.

To run a Django dev server that proxies itself, execute the following:

django-admin.py runserver --settings=tests.test_settings --pythonpath="./"

Similarly, to run a configure Django shell, execute the following:

django-admin.py shell --settings=tests.test_settings --pythonpath="./"

See tests/test_settings.py and tests/test_urls.py for configuration information.

djproxy's People

Contributors

asherf avatar bitdeli-chef avatar blaix avatar ds10x avatar michaeljoseph avatar thomasw avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

djproxy's Issues

base_url same site?

I want to set up a reverse proxy that will proxy a Django app URL to another part of the same website that is outside of Django.

(for example):

www.mysite.com/MyDjangoApp/foo/* -> www.mysite.com/bar/*

And of course, I need this to work for whatever domain/server the app is installed on.

The example in the docs only shows proxying to a static upstream domain (base_url = 'https://google.com/'). How can I set "base_url" in order to use whatever server the original request was made to, instead of always going to a specific domain?

It this isn't specifically configurable, I suppose I could look at request.META['HTTP_HOST'] to find out what server to use, but is there a method that I can hook into where I will have access to the request, and also be able to set base_url at runtime?

Passing CSRF tokens

Hi

I am trying to proxy requests to an internal Nginx server which in turn proxy_passes again. At the end, there is an IPython Notebook.

The problem is, that it complaints about missing CSRF cookie. Is there something special I need to set for that to work? Using the Nginx proxy directly works.

BTW: Is is also possible to proxy the websocket protocol?

Thanks!

request.user is anonymous user

i want to check user permissions and then give him access to proxy.
but when i want to check request.user and its permissions, it give me Anonymous user however it is not anonymous!!!

    def process_request(self, proxy, request, **kwargs):
        if request.method != 'OPTIONS':
            _u = User.objects.get(username=request.user)
            if not _u.has_perm('dummy.cm'):
                return HttpResponse({"status": -1, "msg": "you don't have permission to be here"}, content_type='application/json', status=HTTP_403_FORBIDDEN)
            else:
                return response
        else:
            return response

websockets/channels

Hi thomas,
great proxy you have developed there, thanks in first place ๐Ÿ‘
Just a question if you took a look already at supporting websockets via the django-channels project? I was said to be included in django 1.10 but it wasn't but i wanted to ask if you would consider supporting it anyway??
Thanks!

Handle multiple set-cookie in http response

In case the server returns more than one set-cookie header, djproxy returns a single set-cookie header that is incomprehensible to the browser.

This bug is known and reported in the operation of request. It is appropriate to process this usecase to use the headers returned by urllib3 instead of the one from request.

For now, I have implemented this processing in a ForwardSetCookie middleware.

I have the impression that it should be part of the core of the library, not to be deported in a middleware. It is the role of the proxy to correctly manage headers, cookies, ... I can redo my PR if it's the case.

class ForwardSetCookie(object):
    """
    Handles the transmission of multiple cookies returned by a server as multiple set-cookie headers.

    Request merges the set-cookie headers into one. The default behavior is ok
    if the server returns only one cookie per http response.

    The browser will receive only one cookie.

    {
        Set-Cookie: hello=world; Expires=Wed, 21 Oct 2015 07:28:00 GMT, world=hello
    }

    instead

    {
        Set-Cookie: hello=world; Expires=Wed, 21 Oct 2015 07:28:00 GMT
        Set-Cookie: world=hello
    }
    pass

Using the proxy with a prefix causes an issue with src/href links that start with '/'

URL links in proxied documents that begin with '/' always request the document at the url of the website. For example, if my website is www.example.com, and the proxy for an external site www.external.com is hosted at '/prefix/', if the proxied HTML document that is served as a response contains any links (href or src) starting with '/', for example, '/image/puppy.png', the link resolves to 'www.example.com/image/puppy.png' which is incorrect, since it ignores the prefix we have set up for the proxy.

Question. Reverse Proxy.

Hello Team,

Thank you for your efforts with this library.
I am confused about something. When configuring the middleware to ProxyPassReverse in the view, should I expect all links in the response content to the user (for example images inside the Html page) are going to be changed to the hostname the one is doing the proxy? I could not make this work. Am I doing something wrong?

Thank you, I appreciate it.
M

Development broken in python < 2.7

The requests file this ships with installs a version of Django not compatible with Python 2.6. If you need to test things with that version of Python, you'll need to install a version of Django < 1.7.

url() is deprecated in Django 3.1+

For anyone else looking for how to make this work in Django 3.1+, you'll find that url() is deprecated, so where you see this in the documentation:

url(r'^local_proxy/(?P<url>.*)$', LocalProxy.as_view(), name='proxy')

...change it to this:

path('local_proxy/<path:url>', views.LocalProxy.as_view(), name='proxy'),

(you could also use re_path(), but there's really no need)

I hope this helps someone out :-)

Headers not passed from upstream to request

Forgive me if it's a noob question. I usually code in other programming language. I tried using this library to proxy request but i was getting garbage value in prod which connects to a https url. After struggling i figured out that the headers is not passed from upstream to the proxy request. so i wrote a middle ware and added it to make it work. Is that intentional or bug ?

The middleware looks like this

class AddHeaders(object):

    def process_response(self, proxy, request, upstream_response, response):
        return HttpResponse(response.content, headers=upstream_response.headers)

bug in HttpProxy.__init__

Take look at a call of the base class' constructor here.

class HttpProxy(View):
    ...
    def __init__(self, *args, **kwargs):
        return super(View, self).__init__(*args, **kwargs)

It must be return super(HttpProxy, self).__init__(*args, **kwargs).

reverse_urls help

Trying to use ReverseProxy on Django 2.1 but is does not add the location headers.
Later I see that it is only tested on Django 1 version.
Will this project get into a Version 2 release ?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.