Code Monkey home page Code Monkey logo

dj-database-url's Introduction

DJ-Database-URL

Jazzband https://codecov.io/gh/jazzband/dj-database-url/branch/master/graph/badge.svg?token=7srBUpszOa

This simple Django utility allows you to utilize the 12factor inspired DATABASE_URL environment variable to configure your Django application.

The dj_database_url.config method returns a Django database connection dictionary, populated with all the data specified in your URL. There is also a conn_max_age argument to easily enable Django's connection pool.

If you'd rather not use an environment variable, you can pass a URL in directly instead to dj_database_url.parse.

Supported Databases

Support currently exists for PostgreSQL, PostGIS, MySQL, MySQL (GIS), Oracle, Oracle (GIS), Redshift, CockroachDB, Timescale, Timescale (GIS) and SQLite.

Installation

Installation is simple:

$ pip install dj-database-url

Usage

  1. If DATABASES is already defined:
  • Configure your database in settings.py from DATABASE_URL:

    import dj_database_url
    
    DATABASES['default'] = dj_database_url.config(
        conn_max_age=600,
        conn_health_checks=True,
    )
  • Provide a default:

    DATABASES['default'] = dj_database_url.config(
        default='postgres://...',
        conn_max_age=600,
        conn_health_checks=True,
    )
  • Parse an arbitrary Database URL:

    DATABASES['default'] = dj_database_url.parse(
        'postgres://...',
        conn_max_age=600,
        conn_health_checks=True,
    )
  1. If DATABASES is not defined:
  • Configure your database in settings.py from DATABASE_URL:

    import dj_database_url
    
    DATABASES = {
        'default': dj_database_url.config(
            conn_max_age=600,
            conn_health_checks=True,
        ),
    }
  • You can provide a default, used if the DATABASE_URL setting is not defined:

    DATABASES = {
        'default': dj_database_url.config(
            default='postgres://...',
            conn_max_age=600,
            conn_health_checks=True,
        )
    }
  • Parse an arbitrary Database URL:

    DATABASES = {
        'default': dj_database_url.parse(
            'postgres://...',
            conn_max_age=600,
            conn_health_checks=True,
        )
    }

conn_max_age sets the CONN_MAX_AGE setting, which tells Django to persist database connections between requests, up to the given lifetime in seconds. If you do not provide a value, it will follow Django’s default of 0. Setting it is recommended for performance.

conn_health_checks sets the CONN_HEALTH_CHECKS setting (new in Django 4.1), which tells Django to check a persisted connection still works at the start of each request. If you do not provide a value, it will follow Django’s default of False. Enabling it is recommended if you set a non-zero conn_max_age.

Strings passed to dj_database_url must be valid URLs; in particular, special characters must be url-encoded. The following url will raise a ValueError:

postgres://user:p#ssword!@localhost/foobar

and should instead be passed as:

postgres://user:p%23ssword!@localhost/foobar

TEST settings can be configured using the test_options attribute:

DATABASES['default'] = dj_database_url.config(default='postgres://...', test_options={'NAME': 'mytestdatabase'})

URL schema

Engine Django Backend URL
PostgreSQL django.db.backends.postgresql [1] postgres://USER:PASSWORD@HOST:PORT/NAME [2] postgresql://USER:PASSWORD@HOST:PORT/NAME
PostGIS django.contrib.gis.db.backends.postgis postgis://USER:PASSWORD@HOST:PORT/NAME
MSSQL sql_server.pyodbc mssql://USER:PASSWORD@HOST:PORT/NAME
MSSQL [5] mssql mssqlms://USER:PASSWORD@HOST:PORT/NAME
MySQL django.db.backends.mysql mysql://USER:PASSWORD@HOST:PORT/NAME [2]
MySQL (GIS) django.contrib.gis.db.backends.mysql mysqlgis://USER:PASSWORD@HOST:PORT/NAME
SQLite django.db.backends.sqlite3 sqlite:///PATH [3]
SpatiaLite django.contrib.gis.db.backends.spatialite spatialite:///PATH [3]
Oracle django.db.backends.oracle oracle://USER:PASSWORD@HOST:PORT/NAME [4]
Oracle (GIS) django.contrib.gis.db.backends.oracle oraclegis://USER:PASSWORD@HOST:PORT/NAME
Redshift django_redshift_backend redshift://USER:PASSWORD@HOST:PORT/NAME
CockroachDB django_cockroachdb cockroach://USER:PASSWORD@HOST:PORT/NAME
Timescale [6] timescale.db.backends.postgresql timescale://USER:PASSWORD@HOST:PORT/NAME
Timescale (GIS) [6] timescale.db.backend.postgis timescalegis://USER:PASSWORD@HOST:PORT/NAME
[1]The django.db.backends.postgresql backend is named django.db.backends.postgresql_psycopg2 in older releases. For backwards compatibility, the old name still works in newer versions. (The new name does not work in older versions).
[2](1, 2) With PostgreSQL or CloudSQL, you can also use unix domain socket paths with percent encoding: postgres://%2Fvar%2Flib%2Fpostgresql/dbname mysql://uf07k1i6d8ia0v@%2fcloudsql%2fproject%3alocation%3ainstance/dbname
[3](1, 2) SQLite connects to file based databases. The same URL format is used, omitting the hostname, and using the "file" portion as the filename of the database. This has the effect of four slashes being present for an absolute file path: sqlite:////full/path/to/your/database/file.sqlite.
[4]Note that when connecting to Oracle the URL isn't in the form you may know from using other Oracle tools (like SQLPlus) i.e. user and password are separated by : not by /. Also you can omit HOST and PORT and provide a full DSN string or TNS name in NAME part.
[5]Microsoft official mssql-django adapter.
[6](1, 2) Using the django-timescaledb Package which must be installed.

Contributing

We welcome contributions to this project. Projects can take two forms:

  1. Raising issues or helping others through the github issue tracker.
  2. Contributing code.

Raising Issues or helping others:

When submitting an issue or helping other remember you are talking to humans who have feelings, jobs and lives of their own. Be nice, be kind, be polite. Remember english may not be someone first language, if you do not understand or something is not clear be polite and re-ask/ re-word.

Contributing code:

  • Before writing code be sure to check existing PR's and issues in the tracker.
  • Write code to the pylint spec.
  • Large or wide sweeping changes will take longer, and may face more scrutiny than smaller confined changes.
  • Code should be pass black and flake8 validation.

dj-database-url's People

Contributors

adamchainz avatar carlio avatar dbrgn avatar dctalbot avatar doismellburning avatar dresdn avatar eisensheng avatar flashingpumpkin avatar gabrielgrant avatar j4mie avatar jacobian avatar jairhenrique avatar jeffpaine avatar jezdez avatar johntellsall avatar jtdoepke avatar kennethreitz avatar lino avatar loisaidasam avatar mattseymour avatar mgorny avatar mjtamlyn avatar mwarkentin avatar palfrey avatar parths007 avatar pre-commit-ci[bot] avatar sigmavirus24 avatar sobolevn avatar timgraham avatar tomkins avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

dj-database-url's Issues

Add the URL that was used to the returned config?

Would it be possible to return the URL that was used in the returned config?

dj-database-url currently looks for an environment variable and if not present it uses a default URL. I'd like to know the URL that was parsed by dj-database-url.

This allows the routine to both parse for Django DATABASE components, as well as a being a general routine for getting a URL from the environment or falling back to a default if it doesn't exist - MONGOLAB_URI, etc.

The code change seems minor. Something like...

def parse(url):
    """Parses a database URL."""

    #--->Initialise config with URL key instead of {}

    config = {
        'URL': url
    }

    url = urlparse.urlparse(url)

    # Remove query strings.
    path = url.path[1:]
    path = path.split('?', 2)[0]

    # Update with environment configuration.
    config.update({
        'NAME': path,
        'USER': url.username,
        'PASSWORD': url.password,
        'HOST': url.hostname,
        'PORT': url.port,
    })

    if url.scheme in SCHEMES:
        config['ENGINE'] = SCHEMES[url.scheme]

    return config

Apologies for not creating a pull request but I'm not completely au fait with github - not a 'real' programmer you see.

Thanks for your time.

django 1.10 warning

python manager migrate

System check identified some issues:

WARNINGS:
?: (mysql.W002) MySQL Strict Mode is not set for database connection 'default'
HINT: MySQL's Strict Mode fixes many data integrity problems in MySQL, such as data truncation upon insertion, by escalating warnings into errors. It is strongly recommended you activate it. See: https://docs.djangoproject.com/en/1.10/ref/databases/#mysql-sql-mode
Operations to perform:
Apply all migrations: admin, auth, contenttypes, mysite, sessions
Running migrations:
Rendering model states... DONE
Applying auth.0008_alter_user_username_max_length... OK

Handle special characters in database URL

If the password contains special characters such as ? or /, the following exception is raised (here I tried to use "foo?bar" as a password):

Traceback (most recent call last):
  File "./manage.py", line 19, in <module>
    execute_from_command_line(sys.argv)
  File "/var/www/myproject/dev/env/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 399, in execute_from_command_line
    utility.execute()
  File "/var/www/myproject/dev/env/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 376, in execute
    sys.stdout.write(self.main_help_text() + '\n')
  File "/var/www/myproject/dev/env/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 240, in main_help_text
    for name, app in six.iteritems(get_commands()):
  File "/var/www/myproject/dev/env/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 107, in get_commands
    apps = settings.INSTALLED_APPS
  File "/var/www/myproject/dev/env/local/lib/python2.7/site-packages/django/conf/__init__.py", line 54, in __getattr__
    self._setup(name)
  File "/var/www/myproject/dev/env/local/lib/python2.7/site-packages/django/conf/__init__.py", line 49, in _setup
    self._wrapped = Settings(settings_module)
  File "/var/www/myproject/dev/env/local/lib/python2.7/site-packages/django/conf/__init__.py", line 128, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/var/www/myproject/dev/env/local/lib/python2.7/site-packages/django/utils/importlib.py", line 40, in import_module
    __import__(name)
  File "/var/www/myproject/dev/myproject/myproject/settings/base.py", line 175, in <module>
    'default': dj_database_url.parse(get_env_variable('DATABASE_URL'))
  File "/var/www/myproject/dev/env/local/lib/python2.7/site-packages/dj_database_url.py", line 84, in parse
    'PORT': url.port or '',
  File "/usr/lib/python2.7/urlparse.py", line 110, in port
    port = int(port, 10)
ValueError: invalid literal for int() with base 10: 'foo'

I tried to escape the special characters with backslashes but then it got interpreted as "foo". I also tried to url-encode them, ie. replacing "?" by %63 but then the password returned by dj-database-url was "foo%63bar".

The issue can be easily reproduced with the following URL: postgres://foo:foo?bar@localhost/foobar.

Maybe I'm just missing something but if that's the case I think it would be worth documenting it somewhere.

django.core.exceptions.ImproperlyConfigured when trying to parse a postgres url

I'm running Django 1.4 and I'm currently trying to connect to a postgres 9.1 server.

However, I'm getting this error when I try to run ./manage.py syncdb.

The url I'm using looks like this

postgres://user:password@localhost:5432/mydb

The error I'm getting is the following:

raise ImproperlyConfigured("settings.DATABASES is improperly configured. "
django.core.exceptions.ImproperlyConfigured: settings.DATABASES is improperly configured. Please supply the ENGINE     value. Check settings documentation for more details.```

And the DATABASES variable in settings.py looks like this:

DATABASES = {'default':dj_database_url.config()}

Reverse functionality

It would be nice if this package could also parse a Django database dictionary into a valid connection URI.

backend?

Can this be a backend, that, when used, looks at ENV to find URL, or gets url from DATABASES config and then creates underlying backend according to that URL.

just thoughts.

OPTIONS entry

I'm using the OPTIONS entry to set the storage engine to INNODB. Is there any way to use this? In MySQL < 5.5 the default storage engine is MyISAM :(

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'live',
        'USER': 'trololo',
        'PASSWORD': 'CHANGE THIS',
        'HOST': 'uop.qwopop.yeye.rds.amazonaws.com',
        'PORT': '',
        'OPTIONS': { 'init_command': 'SET storage_engine=INNODB', }
    }
}

MSSQL port specification issue

Not sure if this applies to all MSSQL adapters, but for "ODBC Driver 13 for SQL Server", using a colon to separate port did not work - using a comma does:

DATABASE_URL=mssql://{{user}}:{{password}}@{{host}},{{port}}/{{database}}?driver=ODBC+Driver+13+for+SQL+Server

Document that user, password, and db name should be urlencoded

I ran into an issue today deploying usage of dj-database-url to production, which had a password that included a non-url-safe character. Since putting usernames / passwords into urls is not something I've done a lot of, I didn't know that quoting was necessary (and I wasn't even sure it would work.)

I ended up finding this web page, which mentioned that quoting was needed: https://support.brightcove.com/special-characters-usernames-and-passwords

The error I got was:

File "/usr/lib/python2.7/urlparse.py", line 214, in urlsplit
    raise ValueError("Invalid IPv6 URL")

It would be great if the documentation for dj-database-url could include a note about this. It doesn't seem like there would be a good way to catch this error and re-raise with more helpful information, unfortunately, but if I'm mistaken then that would also be great.

fail to pick up database NAME

So I checked out a django project which works fine on heroku. But when I'm working with it locally, it gives me this error:

settings.DATABASES is improperly configured. Please supply the NAME value.

Here is my settings.py

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
    }
}
import dj_database_url
# Parse database configuration from $DATABASE_URL
DATABASES['default'] =  dj_database_url.config()

# Enable Connection Pooling (if desired)
DATABASES['default']['ENGINE'] = 'django_postgrespool'

Here is my environment

Request Method: GET
Request URL: http://0.0.0.0:5000/

Django Version: 1.8
Python Version: 2.7.6
Installed Applications:
('django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'django.contrib.humanize',
 'django.contrib.sites',
 'app',
 'storages',
 'imagekit')
Installed Middleware:
('django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware')

and below is the complete traceback

Traceback:
File "/Users/qiaoweiliu/.virtualenvs/heroku/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
  132.                     response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/Users/qiaoweiliu/Google Drive/heroku/luxingnan/views.py" in home
  17.         return render(request,'luxingnan/home.html',{'auth_form':auth_form, 'user_form':user_form,'cars':cars,'next_url': '/',})
File "/Users/qiaoweiliu/.virtualenvs/heroku/lib/python2.7/site-packages/django/shortcuts.py" in render
  67.             template_name, context, request=request, using=using)
File "/Users/qiaoweiliu/.virtualenvs/heroku/lib/python2.7/site-packages/django/template/loader.py" in render_to_string
  99.         return template.render(context, request)
File "/Users/qiaoweiliu/.virtualenvs/heroku/lib/python2.7/site-packages/django/template/backends/django.py" in render
  74.         return self.template.render(context)
File "/Users/qiaoweiliu/.virtualenvs/heroku/lib/python2.7/site-packages/django/template/base.py" in render
  209.                     return self._render(context)
File "/Users/qiaoweiliu/.virtualenvs/heroku/lib/python2.7/site-packages/django/template/base.py" in _render
  201.         return self.nodelist.render(context)
File "/Users/qiaoweiliu/.virtualenvs/heroku/lib/python2.7/site-packages/django/template/base.py" in render
  903.                 bit = self.render_node(node, context)
File "/Users/qiaoweiliu/.virtualenvs/heroku/lib/python2.7/site-packages/django/template/debug.py" in render_node
  79.             return node.render(context)
File "/Users/qiaoweiliu/.virtualenvs/heroku/lib/python2.7/site-packages/django/template/loader_tags.py" in render
  135.         return compiled_parent._render(context)
File "/Users/qiaoweiliu/.virtualenvs/heroku/lib/python2.7/site-packages/django/template/base.py" in _render
  201.         return self.nodelist.render(context)
File "/Users/qiaoweiliu/.virtualenvs/heroku/lib/python2.7/site-packages/django/template/base.py" in render
  903.                 bit = self.render_node(node, context)
File "/Users/qiaoweiliu/.virtualenvs/heroku/lib/python2.7/site-packages/django/template/debug.py" in render_node
  79.             return node.render(context)
File "/Users/qiaoweiliu/.virtualenvs/heroku/lib/python2.7/site-packages/django/template/loader_tags.py" in render
  65.                 result = block.nodelist.render(context)
File "/Users/qiaoweiliu/.virtualenvs/heroku/lib/python2.7/site-packages/django/template/base.py" in render
  903.                 bit = self.render_node(node, context)
File "/Users/qiaoweiliu/.virtualenvs/heroku/lib/python2.7/site-packages/django/template/debug.py" in render_node
  79.             return node.render(context)
File "/Users/qiaoweiliu/.virtualenvs/heroku/lib/python2.7/site-packages/django/template/defaulttags.py" in render
  328.             if match:
File "/Users/qiaoweiliu/.virtualenvs/heroku/lib/python2.7/site-packages/django/db/models/query.py" in __nonzero__
  170.         return type(self).__bool__(self)
File "/Users/qiaoweiliu/.virtualenvs/heroku/lib/python2.7/site-packages/django/db/models/query.py" in __bool__
  166.         self._fetch_all()
File "/Users/qiaoweiliu/.virtualenvs/heroku/lib/python2.7/site-packages/django/db/models/query.py" in _fetch_all
  965.             self._result_cache = list(self.iterator())
File "/Users/qiaoweiliu/.virtualenvs/heroku/lib/python2.7/site-packages/django/db/models/query.py" in iterator
  238.         results = compiler.execute_sql()
File "/Users/qiaoweiliu/.virtualenvs/heroku/lib/python2.7/site-packages/django/db/models/sql/compiler.py" in execute_sql
  827.         cursor = self.connection.cursor()
File "/Users/qiaoweiliu/.virtualenvs/heroku/lib/python2.7/site-packages/django/db/backends/base/base.py" in cursor
  162.             cursor = self.make_debug_cursor(self._cursor())
File "/Users/qiaoweiliu/.virtualenvs/heroku/lib/python2.7/site-packages/django/db/backends/base/base.py" in _cursor
  135.         self.ensure_connection()
File "/Users/qiaoweiliu/.virtualenvs/heroku/lib/python2.7/site-packages/django/db/backends/base/base.py" in ensure_connection
  130.                 self.connect()
File "/Users/qiaoweiliu/.virtualenvs/heroku/lib/python2.7/site-packages/django/db/backends/base/base.py" in connect
  118.         conn_params = self.get_connection_params()
File "/Users/qiaoweiliu/.virtualenvs/heroku/lib/python2.7/site-packages/django/db/backends/postgresql_psycopg2/base.py" in get_connection_params
  154.                 "settings.DATABASES is improperly configured. "

Exception Type: ImproperlyConfigured at /
Exception Value: settings.DATABASES is improperly configured. Please supply the NAME value.

And I checked my heroku config, the DATABASE_URL contains all of the required things including DATABASE NAME.

Thank you in advance^_^

cxOracle(and Django) expects port numbers as strings not integers

With a port number specified in the url cxOracle fails as follows:

File "/home/test/.virtualenvs/test/local/lib/python2.7/site-packages/django/db/backends/oracle/base.py", line 213, in get_new_connection
conn_string = convert_unicode(self._connect_string())
File "/home/test/.virtualenvs/test/local/lib/python2.7/site-packages/django/db/backends/oracle/base.py", line 197, in _connect_string
if settings_dict['PORT'].strip():
AttributeError: 'int' object has no attribute 'strip'

I had a look at the Django docs and it shows database ports as strings.

I need to disable ssl_require=True in local sqlite databases

I can't do DATABASE_URL = 'sqlite:///db.sqlite3', because in this piece of code there is no support for other storage mechanism for sqlite database:

if url == 'sqlite://:memory:':
# this is a special case, because if we pass this URL into
# urlparse, urlparse will choke trying to interpret "memory"
# as a port number
return {
'ENGINE': SCHEMES['sqlite'],
'NAME': ':memory:'
}
# note: no other settings are required for sqlite

I'm going to create a PR to add support to it, I need to put this immediately in my deployment environment in TravisCI.

Way to abstract this for CACHES setting?

This is probably out of scope of this app, but given the (intended) similarity of the CACHES setting, I think it would be interesting to think about adding that to the app (and call it differently, like dj-backends-url or whatever).

question marks in database passwords break in unexpected ways

I am getting the following error because I have a ? after e5 in my DB password:

[Sat Jun 25 06:44:00.793929 2016] [:error] [pid 3529] [remote 127.0.0.1:42186]   File "/opt/python/current/app/foo/settings.py", line 63, in <module>
[Sat Jun 25 06:44:00.793943 2016] [:error] [pid 3529] [remote 127.0.0.1:42186]     'default': dj_database_url.config(default='postgres:///foo'),
[Sat Jun 25 06:44:00.793960 2016] [:error] [pid 3529] [remote 127.0.0.1:42186]   File "/opt/python/run/venv/lib/python3.4/site-packages/dj_database_url.py", line 46, in config
[Sat Jun 25 06:44:00.793964 2016] [:error] [pid 3529] [remote 127.0.0.1:42186]     config = parse(s, engine)
[Sat Jun 25 06:44:00.793979 2016] [:error] [pid 3529] [remote 127.0.0.1:42186]   File "/opt/python/run/venv/lib/python3.4/site-packages/dj_database_url.py", line 84, in parse
[Sat Jun 25 06:44:00.793982 2016] [:error] [pid 3529] [remote 127.0.0.1:42186]     'PORT': url.port or '',
[Sat Jun 25 06:44:00.793999 2016] [:error] [pid 3529] [remote 127.0.0.1:42186]   File "/usr/lib64/python3.4/urllib/parse.py", line 156, in port
[Sat Jun 25 06:44:00.794002 2016] [:error] [pid 3529] [remote 127.0.0.1:42186]     port = int(port, 10)
[Sat Jun 25 06:44:00.794020 2016] [:error] [pid 3529] [remote 127.0.0.1:42186] ValueError: invalid literal for int() with base 10: 'e5'

It would be nice to support question marks in passwords or at least display a more helpful error message when this occurs.

Changelog

Where is the project's changelog kept, please?

I couldn't find an overview of changes from 0.3.0 to 0.4.0.

Local MySQL Database URL

tl;dr; The MySQL URL string does not seem to accept parameters as expected.

In a Django project, this local database configuration:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql', # Add 'postgresql_psycopg2', 'mysql', 'sqlite3' or 'oracle'.
        'NAME': 'example_db',        # Or path to database file if using sqlite3.
        'USER': 'myname',                   # Not used with sqlite3.
        'PASSWORD': 'mypass',            # Not used with sqlite3.
        'HOST': 'localhost',                  # Set to empty string for localhost. Not used with sqlite3.
        'PORT': '3306',                       # Set to empty string for default. Not used with sqlite3.
    }
}

works just fine when I run 'python manage.py syncdb' and creates tables in local mysql db example_db nicely, so the user and permissions are set correctly.

I'd like to use dj_database_url instead:

import dj_database_url
# Uses DATABASE_URL environment variable if available. Otherwise, the default
DATABASES['default'] =  dj_database_url.config(default='mysql://localhost:3306/example_db?username=myname&password=mypass')

I cannot run syncdb anymore then because I get an access denied error. The username and password are NOT being read from the MySQL string because the username Django tries is not 'myname' and it probably isn't reading the password correctly.

"Access denied for user 'some_other_username'@'localhost' (using password: NO)")

Searched around on SO and tried many different ways of writing the default string 'mysql://...' without much luck. I'm guessing the default string is malformed and I could use guidance.

Django 2.0: django.db.backends.postgresql_psycopg2 module is deprecated

https://docs.djangoproject.com/en/2.0/releases/2.0/#id1

The django.db.backends.postgresql_psycopg2 module is deprecated in favor of django.db.backends.postgresql. It’s been an alias since Django 1.9. This only affects code that imports from the module directly. The DATABASES setting can still use 'django.db.backends.postgresql_psycopg2', though you can simplify that by using the 'django.db.backends.postgresql' name added in Django 1.9.

Not something that will break code now but something to keep in mind. Just dropping that here.

https://github.com/kennethreitz/dj-database-url/blob/master/dj_database_url.py#L31

Support for named :memory: urls?

Some lore / background on this: https://www.sqlite.org/inmemorydb.html

django/django@3543fec

https://www.sqlite.org/inmemorydb.html

python 3.6.3

dj-database-url 0.4.2

django 1.11.8

3.19.3 2017-06-27 16:48:08 2b0954060fe10d6de6d479287dd88890f1bef6cc1beca11bc6cdb79f72e2377b

assuming

            'secondary': dj_database_url.config(
                env='SECONDARY_DATABASE',
                default='sqlite://testdb?mode=memory',
            ),
  File "/Users/me/work/python/django-polymorphic/.venv/lib/python2.7/site-packages/django/db/migrations/recorder.py", line 65, in applied_migrations
    self.ensure_schema()
  File "/Users/me/work/python/django-polymorphic/.venv/lib/python2.7/site-packages/django/db/migrations/recorder.py", line 52, in ensure_schema
    if self.Migration._meta.db_table in self.connection.introspection.table_names(self.connection.cursor()):
  File "/Users/me/work/python/django-polymorphic/.venv/lib/python2.7/site-packages/django/db/backends/base/base.py", line 254, in cursor
    return self._cursor()
  File "/Users/me/work/python/django-polymorphic/.venv/lib/python2.7/site-packages/django/db/backends/base/base.py", line 229, in _cursor
    self.ensure_connection()
  File "/Users/me/work/python/django-polymorphic/.venv/lib/python2.7/site-packages/django/db/backends/base/base.py", line 213, in ensure_connection
    self.connect()
  File "/Users/me/work/python/django-polymorphic/.venv/lib/python2.7/site-packages/django/db/backends/base/base.py", line 189, in connect
    self.connection = self.get_new_connection(conn_params)
  File "/Users/me/work/python/django-polymorphic/.venv/lib/python2.7/site-packages/django/db/backends/sqlite3/base.py", line 198, in get_new_connection
    conn = Database.connect(**conn_params)
TypeError: keywords must be strings

Trying to debug an unrelated issue at jazzband/django-polymorphic#318 (comment) where we use dj-database-url in tests and I want to rule out if two :memory: sqlite connections could be causing stuff to fizzle or not.

slow speed with dj_database_url

After i setup my django project and run dj_database_url with this config the speed of migrations and the development server run extrem slow.

#settings.py
DATABASES = {'default': dj_database_url.config()}

.env file
DATABASE_URL=postgres://user:passwd@localhost:5432/my_db

After i changed localhost with 127.0.0.1 everything works fine

Document the 'env' parameter

dj-database-url defaults to reading in the value of DATABASE_URL, but this can be overridden using env, like so:

DATABASES = {
    'default': dj_database_url.config(),
    'read_only': dj_database_url.config(env='DATABASE_URL_RO')
}

It would be good to mention this in the README.
I'll open a PR when I get a chance soon, if someone hasn't beaten me to it :-)

MySQL backend doesn't work on python 3

'django.db.backends.mysql' uses MySQL-python lib, but this lib does not support python 3.

You could use another backend when python 3 is used. The mysql-connector-python lib already comes with a django backend: 'mysql.connector.django'. It is a good alternative.

Want to set `False` value in options.

I want to pass a value of False to Django.
For example, ?use_unicode=False.

DATABASES['default']['OPTIONS']['use_unicode'] = 'False'  #=> True

However, if you do this, it will be set as a character string as above.

Even if ?use_unicode=0, '0' is treated as True in Python, so it does not become the expected value.

Is there a way to set False appropriately?

Doesn't support in-memory sqlite databases

There is no way to use an in-memory sqlite database.

  • sqlite://:memory: chokes because urlparse attempts to split on colons and parse 'memory' as a port number:
File "<path>/urlparse.py", line 106, in port
    return int(port, 10)
ValueError: invalid literal for int() with base 10: 'memory'
  • sqlite:// doesn't set a value for NAME in the database (or rather, sets it to an empty string), causing Django to complain:
django.core.exceptions.ImproperlyConfigured: Please fill out the database NAME in the settings module before using the database.

heroku+django+mysql: sslmode not supported.

Based on docs: there's no 'sslmode' attribute. The following code gives me the exception: 'sslmode' is an invalid keyword argument.
This issue is possibly related to other backends like sqlite as well.

Workaround:
On settings.py after:
django_heroku.settings(locals())
add:
del DATABASES['default']['OPTIONS']['sslmode']

What is special about it?

It's just changed from dict config to invoking function of this package to do connection.

I have read 12factor but I don't still understand.

Anyone can answer?

Allow colons (%3A) in percent-encoded paths

Postgres supports percent-encoded paths to allow for connecting to the DB over a domain socket since this change, and it would be super helpful to allow percent-encoded colons as well. That way, dj-database-url could be used with Google Cloud SQL, because its socket names are in the format /cloudsql/{project-id}:{sql-instance-id}. Without percent-encoding the colon, it gets interpreted as an invalid port. Adding it would support connection strings like this, like Postgres:

>>> dj_database_url.parse('mysql://root@%2fcloudsql%2fproject-id%3asql-id/database')
{'ENGINE': 'django.db.backends.mysql', 'HOST': '/cloudsql/project-id:sql-id', 'NAME': 'database', 'CONN_MAX_AGE': 0, 'PASSWORD': '', 'PORT': '', 'USER': 'root'}

I have an existing patch if that would be helpful.

Database URL not updating database connection even when URL in settings is changed

When trying to switch URLs on the same host e.g. postgresql://user:secret@localhost/my_db_1 to postgresql://user:secret@localhost/my_db_2, Django wasn't reflecting the change when I did this and kept connecting to my_db_1

$ python manage.py shell

>> from django import db
>> db.connection.settings_dict['NAME']
"my_db_1"

I changed back to using the normal Django database settings dict (instead of a URL with dj-database-url) and it worked. What could be causing this?

No wheel for latest version 0.4.1

This project has bdist_wheel configuration in its setup.cfg but no wheel has been published. This implies that the bdist_wheel step was missed in release. A full sdist + bdist_wheel release can be done with python setup.py clean sdist bdist_wheel upload. You can also upload the wheel right now by checking out the source code at the version for the last release and running python setup.py clean bdist_wheel upload. This will speed up everyone's installs 🐇 😄

Should we raise a KeyError if the env var is missing and default is None?

Shouldn't this function raise a KeyError if s is None?

def config(env=DEFAULT_ENV, default=None, engine=None, conn_max_age=0, ssl_require=False):
    """Returns configured DATABASE dictionary from DATABASE_URL."""

    config = {}

    s = os.environ.get(env, default)

    if s:
        config = parse(s, engine, conn_max_age, ssl_require)

    return config

If the user has not properly configured his environment settings (most likely), and not provided a default fallback value either (also likely), the config will return an empty dictionary, implicitly silencing the issue... (speaking of experience here...)
https://devcenter.heroku.com/articles/heroku-postgresql#connecting-in-python

License

Could you please add the license in the source distribution.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.