jimfunk / django-postgresql-netfields Goto Github PK

View Code? Open in Web Editor NEW

153.0 153.0 65.0 256 KB

Proper INET and CIDR fields for Django running on PostgreSQL

License: BSD 3-Clause "New" or "Revised" License

Python 100.00%

django-postgresql-netfields's People

Contributors

Stargazers

Watchers

Forkers

eide key crustymonkey jathanism zapcome ekohl pobk cheekybastard wolever pcraston spideroak jmacul2 mauroasilva overcastsoftware riklaunim arbor smclenithan chorsley justinazoff hgiasac aaron212 tracywebtech jsenecal leifurhauks brandoncazander a-musing-moose wsot davekr hcastilho flokli andir hasegeli sbrandtb dgaus dmippolitov msuszko hylje tarkatronic bytearchive fxfitz thatnerdjosh ivan-shumilov silasray codeout mikelane erikdewildt totebrick etene yuvadm bitcity gforcada samantha-milligan galst1234 sevdog vboiko-gt friendkak tfromme kbirkeland hoefling arpitjain799 nosnilmot a-detiste

django-postgresql-netfields's Issues

Switch to ipaddr

ipaddr is slatted for stdlib inclusion, drop IPy in favor of this.

All travis tests are run against python 3.5

See https://docs.travis-ci.com/user/languages/python for setting up multiple versions.

Unify and update tox/travis tests

Using something like https://github.com/eventlet/eventlet/blob/master/.travis.yml one could more easily keep the travis tests and the tox enviorments in sync ensuring the same tests run in both locations.

Limit fields to a specific family (IPv6, IPv4, or both)

It could be a good idea to add a named parameter to fields in order to restrict them to a specific address family.

Of course, I'm not talking about an address family restriction on a database level but on a model level. In this way a model could include only IPv4 (or IPv6) address fields.

By default, a field could accept both IPv6 and IPv4 addresses (or networks) or if given a named parameter (family for instance), it will validate the value passed to it, checking it it from the right address family.

I can work on a PR if this issue is proposal is legit.

Use IPAddress object for fields that represent a single IP

I am experiencing an issue where my model can have an IP field that can be a network represented by a CIDR address or a single IP.

class MyModel:
    inet = CidrAddressField()

When creating an object with just a single IP (i.e 156.25.3.189), the object is created but the field returns a string:

k = MyModel.objects.create(inet='156.25.3.189')
k.inet
'156.25.3.189'

It would be nice if the conversion could be done to a IP Address object instead. Would this be possible?

Adding a CidrAddressField to a model causes a 'blank' admin page

I have a very simple model:

class Network(BaseModel):
    block = CidrAddressField()
    office = models.ForeignKey(Office, null=True, blank=True)
    description = models.CharField(max_length=255, null=True, blank=True)

...and it is registered in admin.py:

@admin.register(Network)
class NetworkAdmin(BaseAdmin):
    list_display = ['block', 'office', 'description']
    fields = ['office', 'block', 'description']

When I attempt to view it in the admin section, I see the following:

When I change the NetworkAdmin object to remove the 'block' field like so:

@admin.register(Network)
class NetworkAdmin(BaseAdmin):
    list_display = ['block', 'office', 'description']
    fields = ['office', 'description']

I see:

The 'list view' display in admin works fine displaying the 'block' field, it just doesn't show up when I want to edit.

I've tried overriding the form generated by admin to specifically use the AdminTextInputWidget for the 'block' field, and it still shows up as a blank page.

Am I missing something?

Python field type is str instead of ipaddress.ip_address

I have a model something like this that makes use of a InetAddressField. When referencing the field from a model method, i.e. self.field it returns a str instead of an IPv4Address instance. Wouldn't the expected behavior be to return a type of ipaddress.ip_address like the README says? Here's a crude unittest I wrote that exhibits this behavior (causes a AttributeError: 'str' object has no attribute 'is_loopback' error).

Doesn't work with the ArrayField

Hi!

I've got a model where I use netfields in array field:

from django.contrib.postgres.fields import ArrayField

class Service(models.Model):
    name = models.CharField(unique=True, max_length=64)
    address_classes = ArrayField(netfields.CidrAddressField(), default=[], 
            blank=True)

Unfortunately there is PostgreSQL error when I edit this with admin interface:

ERROR:  column "address_classes" is of type inet[] but expression is of type text[] at character 177
HINT:  You will need to rewrite or cast the expression.
STATEMENT:  UPDATE "base_service" SET "name" = 'some_name', "address_classes" = ARRAY['10.0.0.0/24', '10.1.0.0/16', '10.2.0.0/20'] WHERE "base_service"."id" = 1

I tested it with Django 1.9.4.

I believe there's a typo on the front page / readme

On both the Git Repo page and PyPI page, the section that talks about the MACAddressField, I believe has a typo, as the given example references "CidrAddressField", and should probably be "MACAddressField". Excerpt follows:

MACAddressField will store values in PostgreSQL as type MACADDR. In Python, the value will be represented as a netaddr.EUI object. Note that the default text representation of EUI objects is not the same as that of the netaddr module. It is represented in a format that is more commonly used in network utilities and by network administrators (00:11:22:aa:bb:cc).

from netfields import CidrAddressField, NetManager

class Example(models.Model):
inet = CidrAddressField() #

Lookup by host

I've been playing with this project for the past few hours, and have been at a loss due to what seems like a major missing feature (or a completely undocumented feature)...

It should be possible to have a lookup such as __net_host or __net_address for direct IP lookups against an InetAddressField which has a CIDR prefix. This is already supported by PostgreSQL with WHERE HOST(my_inet_field) = '1.2.3.4'

Without this, I would have to store the IP address separately from the subnet (which defeats one of my main reasons for using the INET type) to be able to query them effectively by individual IP, without knowing the CIDR prefix beforehand.

Something which would operate similar to this:

SELECT  id, hostname, external_ip FROM servers;

   id    | hostname | external_ip
---------+----------+-------------
10       |     test | 1.2.3.4/22
11       |    test2 | 1.2.3.44/22

SELECT id, hostname, external_ip FROM servers WHERE HOST(external_ip) = '1.2.3.4';

   id    | hostname | external_ip
---------+----------+-------------
10       |     test | 1.2.3.4/22

It could be used like so:

Server.objects.get(external_ip__net_host='1.2.3.4')

Neither startswith nor endswith work for this, as they risk running into similar IP addresses, e.g. a query for 1.2.3.4 could bring results for 1.2.3.44 (starts with), or 11.2.3.4 (ends with).

Does Not Officially Support Django 2.0

And by "not officially support" Django 2.0, I mean it doesn't have any tests running against Django 2.

Kill doctests

Get rid of doctests.

'strict' parameter for network

By default, some network like 1.2.3.4/24 is not a valid in python ipaddress module unless strict=False is set.

>>> ipaddress.IPv4Network("1.2.3.4/24")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/virusdefender/.pyenv/versions/3.7.0/lib/python3.7/ipaddress.py", line 1536, in __init__
    raise ValueError('%s has host bits set' % self)
ValueError: 1.2.3.4/24 has host bits set

>>> ipaddress.IPv4Network("1.2.3.4/24", strict=False)
IPv4Network('1.2.3.0/24')

However it can be used in pg database.

So there may be some problems due to different validation logic

# cidr = InetAddressField(null=True)
>>> SomeModel.objects.filter(cidr__net_overlaps="1.2.3.4/24")
...
...

  File "/Users/virusdefender/Desktop/workspace/waf/venv/lib/python3.7/site-packages/netfields/lookups.py", line 65, in get_prep_lookup
    return str(ipaddress.ip_network(self.rhs))
  File "/Users/virusdefender/.pyenv/versions/3.7.0/lib/python3.7/ipaddress.py", line 74, in ip_network
    return IPv4Network(address, strict)
  File "/Users/virusdefender/.pyenv/versions/3.7.0/lib/python3.7/ipaddress.py", line 1536, in __init__
    raise ValueError('%s has host bits set' % self)
ValueError: 1.2.3.4/24 has host bits set

Missing tags for 0.7.x releases

It seems that there is no tag for 0.7.1 and 0.7.2 releases? Would you mind adding them so I can use them for a Debian package?

'module' object has no attribute 'util'

Django==1.7.7
IPy==0.81
Twisted==11.1.0
amqp==1.4.6
anyjson==0.3.3
billiard==3.3.0.19
celery==3.1.17
django-celery==3.1.16
django-extensions==1.5.2
django-netfields==0.2.2
django-tagging==0.3.4
exabgp==3.4.9
ipython==3.0.0
kombu==3.0.24
netaddr==0.7.13
psycopg2==2.6
py-radix==0.7.0
pynfdump==0.3dev
python-dateutil==2.4.1
python-memcached==1.54
pytz==2015.2
redis==2.10.3
six==1.9.0
txAMQP==0.6.2
whisper==0.9.13
wsgiref==0.1.2
zope.interface==4.1.2

Pulling from master, I get the following error when the netfields are in my models.py class. When removed from the class, the admin page works as intended:

Environment:


Request Method: GET
Request URL: http://10.30.63.46:8081/admin/core/device/add/

Django Version: 1.7.7
Python Version: 2.7.8
Installed Applications:
('django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'core',
 'flow',
 'bgpd',
 'probe')
Installed Middleware:
('django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware')


Template error:
In template /opt/bgp-optimizer/lib/python2.7/site-packages/django/contrib/admin/templates/admin/includes/fieldset.html, error at line 19
   'module' object has no attribute 'util'
   9 :             {% for field in line %}


   10 :                 <div{% if not line.fields|length_is:'1' %} class="field-box{% if field.field.name %} field-{{ field.field.name }}{% endif %}{% if not field.is_readonly and field.errors %} errors{% endif %}{% if field.field.is_hidden %} hidden{% endif %}"{% elif field.is_checkbox %} class="checkbox-row"{% endif %}>


   11 :                     {% if not line.fields|length_is:'1' and not field.is_readonly %}{{ field.errors }}{% endif %}


   12 :                     {% if field.is_checkbox %}


   13 :                         {{ field.field }}{{ field.label_tag }}


   14 :                     {% else %}


   15 :                         {{ field.label_tag }}


   16 :                         {% if field.is_readonly %}


   17 :                             <p>{{ field.contents }}</p>


   18 :                         {% else %}


   19 :                              {{ field.field }} 


   20 :                         {% endif %}


   21 :                     {% endif %}


   22 :                     {% if field.field.help_text %}


   23 :                         <p class="help">{{ field.field.help_text|safe }}</p>


   24 :                     {% endif %}


   25 :                 </div>


   26 :             {% endfor %}


   27 :         </div>


   28 :     {% endfor %}


   29 : </fieldset>


Traceback:
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
  137.                 response = response.render()
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/template/response.py" in render
  103.             self.content = self.rendered_content
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/template/response.py" in rendered_content
  80.         content = template.render(context)
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/template/base.py" in render
  148.             return self._render(context)
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/template/base.py" in _render
  142.         return self.nodelist.render(context)
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/template/base.py" in render
  844.                 bit = self.render_node(node, context)
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/template/debug.py" in render_node
  80.             return node.render(context)
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/template/loader_tags.py" in render
  126.         return compiled_parent._render(context)
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/template/base.py" in _render
  142.         return self.nodelist.render(context)
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/template/base.py" in render
  844.                 bit = self.render_node(node, context)
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/template/debug.py" in render_node
  80.             return node.render(context)
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/template/loader_tags.py" in render
  126.         return compiled_parent._render(context)
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/template/base.py" in _render
  142.         return self.nodelist.render(context)
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/template/base.py" in render
  844.                 bit = self.render_node(node, context)
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/template/debug.py" in render_node
  80.             return node.render(context)
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/template/loader_tags.py" in render
  65.                 result = block.nodelist.render(context)
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/template/base.py" in render
  844.                 bit = self.render_node(node, context)
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/template/debug.py" in render_node
  80.             return node.render(context)
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/template/loader_tags.py" in render
  65.                 result = block.nodelist.render(context)
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/template/base.py" in render
  844.                 bit = self.render_node(node, context)
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/template/debug.py" in render_node
  80.             return node.render(context)
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/template/defaulttags.py" in render
  201.                             nodelist.append(node.render(context))
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/template/loader_tags.py" in render
  150.                 return template.render(context)
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/template/base.py" in render
  148.             return self._render(context)
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/template/base.py" in _render
  142.         return self.nodelist.render(context)
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/template/base.py" in render
  844.                 bit = self.render_node(node, context)
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/template/debug.py" in render_node
  80.             return node.render(context)
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/template/defaulttags.py" in render
  201.                             nodelist.append(node.render(context))
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/template/defaulttags.py" in render
  201.                             nodelist.append(node.render(context))
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/template/defaulttags.py" in render
  312.                 return nodelist.render(context)
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/template/base.py" in render
  844.                 bit = self.render_node(node, context)
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/template/debug.py" in render_node
  80.             return node.render(context)
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/template/defaulttags.py" in render
  312.                 return nodelist.render(context)
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/template/base.py" in render
  844.                 bit = self.render_node(node, context)
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/template/debug.py" in render_node
  80.             return node.render(context)
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/template/debug.py" in render
  93.             output = force_text(output)
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/utils/encoding.py" in force_text
  85.                 s = six.text_type(s)
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/forms/forms.py" in __str__
  508.         return self.as_widget()
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django/forms/forms.py" in as_widget
  560.         return force_text(widget.render(name, self.value(), attrs=attrs))
File "/opt/bgp-optimizer/lib/python2.7/site-packages/django_netfields-0.2.2-py2.7.egg/netfields/forms.py" in render
  21.         return mark_safe(u'<input%s />' % forms.util.flatatt(final_attrs))

Exception Type: AttributeError at /admin/core/device/add/
Exception Value: 'module' object has no attribute 'util'

please switch from py2-ipaddress to ipaddress

Many Python distributions now have problems with pip's new strict security requirements and so end up following the steps here:

http://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning

Sadly, one of these pulls in this backport as a dependency:

https://pypi.python.org/pypi/ipaddress

Sadly, this is incompatible with the backport you are using:

https://pypi.python.org/pypi/py2-ipaddress

...but both distributions install an ipaddress.py, resulting in heisenbugs when the wrong package ends up being installed last.

I think some changes would be required, but it might make the world a bit of a less crazy place...

Adding 'netfields' to INSTALLED_APPS creates 7 empty tables in the database

The table names are: cidr, inet, mac, nullcidr, nullinet, uniquecidr, uniqueinet.

From the code it looks like they are for testing only.

Maybe putting this models and other testing code into a separate directory would help. This approach is described here: https://docs.djangoproject.com/en/dev/topics/testing/advanced/#using-the-django-test-runner-to-test-reusable-applications

Validation for CIDR is not correct

I've received an error when I tried to save a CIDR like '64.34.117.2/20', and I got a database error, says the CIDR is not correct.
Then I check the source code of this plug-in, and found the piece of code here:

try:
    return IPNetwork(value)
except (AddrFormatError, TypeError), e:
    raise ValidationError(str(e))

I highly doubt that this is doing the right thing, it only check if we can convert the IP value to a valid IPNetwork object, instead of a valid CIDR.

Handle whitespace around input

Eg, ' 0.0.0.0/0' and '0.0.0.0/0 ' will not validate in the CIDRfield. Both area pretty easy sins to commit if you're cutting/pasting CIDRs.

I've been looking through the fields.py and forms.py modules and it looks like a few

if isinstance(value, str):
    value = value.strip()

should cover this. I haven't read enough of the surrounding code yet to know if that'll blow something else up. I'm happy to open a PR if this is a good approach.

Problem with ArrayField and ModelSerializer

There are some demo code

https://github.com/virusdefender/django-postgresql-netfields/commit/cf21f8fdd351f19ada3d10a3d574921055170367

I added three tests, but the last two tests will fail

https://travis-ci.org/virusdefender/django-postgresql-netfields/jobs/470819231

======================================================================
ERROR: test_serialize_inet_array (test.tests.test_rest_framework_fields.FieldsTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/virusdefender/django-postgresql-netfields/test/tests/test_rest_framework_fields.py", line 83, in test_serialize_inet_array
    self.assertEqual(TestSerializer(instance).data, {"field": ["1.2.3.0/24"]})
  File "/home/travis/build/virusdefender/django-postgresql-netfields/.tox/py36-django18/lib/python3.6/site-packages/rest_framework/serializers.py", line 534, in data
    ret = super(Serializer, self).data
  File "/home/travis/build/virusdefender/django-postgresql-netfields/.tox/py36-django18/lib/python3.6/site-packages/rest_framework/serializers.py", line 263, in data
    self._data = self.to_representation(self.instance)
  File "/home/travis/build/virusdefender/django-postgresql-netfields/.tox/py36-django18/lib/python3.6/site-packages/rest_framework/serializers.py", line 501, in to_representation
    ret[field.field_name] = field.to_representation(attribute)
  File "/home/travis/build/virusdefender/django-postgresql-netfields/.tox/py36-django18/lib/python3.6/site-packages/rest_framework/fields.py", line 1574, in to_representation
    return [self.child.to_representation(item) if item is not None else None for item in data]
  File "/home/travis/build/virusdefender/django-postgresql-netfields/.tox/py36-django18/lib/python3.6/site-packages/rest_framework/fields.py", line 1574, in <listcomp>
    return [self.child.to_representation(item) if item is not None else None for item in data]
  File "/home/travis/build/virusdefender/django-postgresql-netfields/.tox/py36-django18/lib/python3.6/site-packages/rest_framework/fields.py", line 1791, in to_representation
    value = value_from_object(self.model_field, obj)
  File "/home/travis/build/virusdefender/django-postgresql-netfields/.tox/py36-django18/lib/python3.6/site-packages/rest_framework/compat.py", line 150, in value_from_object
    return field._get_val_from_obj(obj)
  File "/home/travis/build/virusdefender/django-postgresql-netfields/.tox/py36-django18/lib/python3.6/site-packages/django/db/models/fields/__init__.py", line 843, in _get_val_from_obj
    return getattr(obj, self.attname)
AttributeError: 'str' object has no attribute 'field'
======================================================================

Create a demo app

Create a demo app using contrib.admin to show how code works and test it in use.

postgresql_psycopg2 deprecation warning (django-netfields 1.2.2)

/Users/i1t/.pyenv/versions/3.6.4/lib/python3.6/site-packages/netfields/compat.py:4: RemovedInDjango30Warning: The django.db.backends.postgresql_psycopg2 module is deprecated in favor of django.db.backends.postgresql.
from django.db.backends.postgresql_psycopg2.base import DatabaseWrapper

ValidationError instead of Exceptions?

If I insert incorrect values in the fields (in the django admin) I get exceptions instead of regular validation errors.

It is possible to raise validation errors instead?

I can help out as soon as I start understanding more about it.

Bug in commit #29

Previous commit broke __in functionality when passing in a list with one value.

ie = Model.objects.filter(netfield__in=['127.0.0.1'])

Since the fix changes the list to a string if it only has one element, it passes the string onto django queryset lookups and breaks.

Can someone tell me why this fix was made in the first place. I can't figure out what it was trying to fix.

Incompatible with Django 3

'ENGINE': 'django.db.backends.postgresql_psycopg2', is deprecated and has been removed in Django 3 making this no longer compatible with Django 3.

There's an open PR open to fix the issue:
#96

Trying to save a model with a CidrAddressField from Django Admin failes

I was trying to save a model in Django Admin, and it failed saying

TypeError: unexpected type <type 'list'> for addr arg

Model

class IpRule(models.Model):
    description = models.CharField(max_length=255)
    active = models.BooleanField(default=True)
    port = models.IntegerField(default=22)
    cidr_ip = CidrAddressField()

Stacktrace

Traceback (most recent call last):
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/django/contrib/staticfiles/handlers.py", line 67, in __call__
    return self.application(environ, start_response)
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/ambition_messenger/server/django_runserver.py", line 88, in application
    return _django_app(environ, start_response)
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 206, in __call__
    response = self.get_response(request)
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/django/core/handlers/base.py", line 194, in get_response
    response = self.handle_uncaught_exception(request, resolver, sys.exc_info())
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/django/core/handlers/base.py", line 229, in handle_uncaught_exception
    return debug.technical_500_response(request, *exc_info)
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/django_extensions/management/technical_response.py", line 5, in null_technical_500_response
    six.reraise(exc_type, exc_value, tb)
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/django/core/handlers/base.py", line 112, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/django/contrib/admin/options.py", line 465, in wrapper
    return self.admin_site.admin_view(view)(*args, **kwargs)
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/django/utils/decorators.py", line 99, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/django/views/decorators/cache.py", line 52, in _wrapped_view_func
    response = view_func(request, *args, **kwargs)
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/django/contrib/admin/sites.py", line 198, in inner
    return view(request, *args, **kwargs)
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/django/utils/decorators.py", line 29, in _wrapper
    return bound_func(*args, **kwargs)
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/django/utils/decorators.py", line 99, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/django/utils/decorators.py", line 25, in bound_func
    return func(self, *args2, **kwargs2)
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/django/db/transaction.py", line 371, in inner
    return func(*args, **kwargs)
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/django/contrib/admin/options.py", line 1244, in change_view
    if form.is_valid():
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/django/forms/forms.py", line 129, in is_valid
    return self.is_bound and not bool(self.errors)
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/django/forms/forms.py", line 121, in errors
    self.full_clean()
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/django/forms/forms.py", line 275, in full_clean
    self._post_clean()
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/django/forms/models.py", line 419, in _post_clean
    self.validate_unique()
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/django/forms/models.py", line 428, in validate_unique
    self.instance.validate_unique(exclude=exclude)
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/django/db/models/base.py", line 754, in validate_unique
    errors = self._perform_unique_checks(unique_checks)
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/django/db/models/base.py", line 838, in _perform_unique_checks
    qs = model_class._default_manager.filter(**lookup_kwargs)
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/django/db/models/manager.py", line 163, in filter
    return self.get_queryset().filter(*args, **kwargs)
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/django/db/models/query.py", line 593, in filter
    return self._filter_or_exclude(False, *args, **kwargs)
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/django/db/models/query.py", line 611, in _filter_or_exclude
    clone.query.add_q(Q(*args, **kwargs))
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/django/db/models/sql/query.py", line 1204, in add_q
    clause = self._add_q(where_part, used_aliases)
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/django/db/models/sql/query.py", line 1240, in _add_q
    current_negated=current_negated)
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/django/db/models/sql/query.py", line 1131, in build_filter
    clause.add(constraint, AND)
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/django/utils/tree.py", line 104, in add
    data = self._prepare_data(data)
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/django/db/models/sql/where.py", line 79, in _prepare_data
    value = obj.prepare(lookup_type, value)
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/django/db/models/sql/where.py", line 352, in prepare
    return self.field.get_prep_lookup(lookup_type, value)
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/netfields/fields.py", line 32, in get_prep_lookup
    return self.get_prep_value(value)
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/netfields/fields.py", line 41, in get_prep_value
    return unicode(self.to_python(value))
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/netfields/fields.py", line 21, in to_python
    return self.python_type()(value)
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/netaddr/ip/__init__.py", line 907, in __init__
    implicit_prefix, flags)
  File "/Users/micahhausler/code/secret-project/venv/lib/python2.7/site-packages/netaddr/ip/__init__.py", line 802, in parse_ip_network
    raise TypeError('unexpected type %s for addr arg' % type(addr))
TypeError: unexpected type <type 'list'> for addr arg

Searches using startswith, istartswith, endswith, iendswith fail on lastest code

I've been AFK for a bit but have gotten back into using this library and updated my project to use this and I am getting some issues. Searches with startswith, endswith and their case insensitive counterparts are returning no records. I dug into the code and it appears to not be implemented correctly. Lookups are not extending the respective core Django lookup.

class EndsWith(NetFieldDecoratorMixin, BuiltinLookup):
    lookup_name = 'endswith'


class IEndsWith(NetFieldDecoratorMixin, BuiltinLookup):
    lookup_name = 'iendswith'


class StartsWith(NetFieldDecoratorMixin, BuiltinLookup):
    lookup_name = 'startswith'


class IStartsWith(NetFieldDecoratorMixin, BuiltinLookup):
    lookup_name = 'istartswith'

When I change the code in lookups to what is below, things appear to be running fine.

class EndsWith(NetFieldDecoratorMixin, EndsWith):
    lookup_name = 'endswith'


class IEndsWith(NetFieldDecoratorMixin, EndsWith):
    lookup_name = 'iendswith'


class StartsWith(NetFieldDecoratorMixin, StartsWith):
    lookup_name = 'startswith'


class IStartsWith(NetFieldDecoratorMixin, IStartsWith):
    lookup_name = 'istartswith'

I am unclear why the the previous lookup commit is extending "BuiltinLookup" in these cases. I cant find documentation on Django's website that indicates that this is best practice. https://docs.djangoproject.com/en/1.9/howto/custom-lookups/

It indicated that you should just extend the core class as was done when creating the "Family" lookup.

I verified the generated SQL code and indeed, the SQL is missing the necessary "%" before and after the LIKE operator to generate the proper SQL.

IMO, I think it best to refactor this to just extend the proper classes so the work as intended. Please tell me if I am missing something. Otherwise I'll create a patch. I am quite surprised no one has caught this until now. This change was introduced last April. :(

loaddata failure with natural keys but successful without them

Hello,

On a Django 3.1.7 setup, I've got a models like this:
`class LANManager(models.Manager):
def get_by_natural_key(self, slug_name):
return self.get(slug_name=slug_name)

class LAN(models.Model):
slug_name = models.SlugField(max_length=64, unique=True)
network_address = CidrAddressField()
net_objects = NetManager()
objects = LANManager()

def __str__(self):
    return f'{self.slug_name}'

def natural_key(self):
    return (self.slug_name,)

class DHCPRangeManager(models.Manager):
def get_by_natural_key(self, ip_min, ip_max):
return self.get(ip_min=ip_min, ip_max=ip_max)

class DHCPRange(models.Model):
ip_min = InetAddressField(store_prefix_length=False)
ip_max = InetAddressField(store_prefix_length=False)
lan = models.ForeignKey(LAN, on_delete=models.PROTECT, related_name='dhcp_ranges')
net_objects = NetManager()
objects = DHCPRangeManager()

def __str__(self):
    return f'{str(self.ip_min)}-{str(self.ip_max)}'

def natural_key(self):
    return (self.ip_min, self.ip_max)`

I can positively run
python manage.py dumpdata --format=yaml app.lan app.dhcprange > app.fixtures.foo python manage.py loaddata foo python manage.py dumpdata --format=yaml --natural-primary --natural-foreign app.lan app.dhcprange > app.fixtures.foo2

But the next one fails:
python manage.py loaddata app.fixtures.foo2
with:
["“['toip_lan']” value must be an integer."]

If I remove DHCPRange data (ie the with a foreign key ) from fixture file, it works.
Fixture file contents looked OK.

Is netfield compliant with Django 3.1 ?
Can I rewrite my models to work around this ?
Suggestions ?

Best regards

Deprecation Warnings for Django v3.0

Just saw this in my CI pipeline today. Thought I would let you all know:

/usr/local/lib/python3.6/site-packages/django/db/models/sql/compiler.py:995: RemovedInDjango30Warning: Remove the context parameter from InetAddressField.from_db_value(). Support for it will be removed in Django 3.0.
  RemovedInDjango30Warning,
/usr/local/lib/python3.6/site-packages/django/db/models/sql/compiler.py:995: RemovedInDjango30Warning: Remove the context parameter from InetAddressField.from_db_value(). Support for it will be removed in Django 3.0.
  RemovedInDjango30Warning,
/usr/local/lib/python3.6/site-packages/django/db/models/sql/compiler.py:995: RemovedInDjango30Warning: Remove the context parameter from InetAddressField.from_db_value(). Support for it will be removed in Django 3.0.
  RemovedInDjango30Warning,
/usr/local/lib/python3.6/site-packages/django/db/models/sql/compiler.py:995: RemovedInDjango30Warning: Remove the context parameter from InetAddressField.from_db_value(). Support for it will be removed in Django 3.0.
  RemovedInDjango30Warning,

Update and replace PyPI package

Have been trying the lib with Django 1.6 (a sorely needed feature, text fields don't cut it for IP addresses!). The official version 0.2.1 in PyPI is from May 2013, and doesn't have the Django 1.6 support rolled in yet. Would it be possible to update the version number and update the PyPI package?

netfields is incompatible with Django 1.9: ImportError on "from django.db.models.lookups import default_lookups"

From diff it seems that in Django 1.9, filling 'default_lookups' dictionary was replaced with Field.register_lookup(<lookup_class>).

Full trace:

Traceback (most recent call last):
File "/opt/ep/project/manage.py", line 10, in
execute_from_command_line(sys.argv)
File "/opt/ep/env/3/lib/python3.4/site-packages/django/core/management/init.py", line 350, in execute_from_command_line
utility.execute()
File "/opt/ep/env/3/lib/python3.4/site-packages/django/core/management/init.py", line 324, in execute
django.setup()
File "/opt/ep/env/3/lib/python3.4/site-packages/django/init.py", line 18, in setup
apps.populate(settings.INSTALLED_APPS)
File "/opt/ep/env/3/lib/python3.4/site-packages/django/apps/registry.py", line 85, in populate
app_config = AppConfig.create(entry)
File "/opt/ep/env/3/lib/python3.4/site-packages/django/apps/config.py", line 116, in create
mod = import_module(mod_path)
File "/opt/ep/env/3/lib/python3.4/importlib/init.py", line 109, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 2254, in _gcd_import
File "", line 2237, in _find_and_load
File "", line 2226, in _find_and_load_unlocked
File "", line 1200, in _load_unlocked
File "", line 1129, in _exec
File "", line 1471, in exec_module
File "", line 321, in _call_with_frames_removed
File "/opt/ep/env/3/lib/python3.4/site-packages/netfields/apps.py", line 3, in
from django.db.models.lookups import default_lookups
ImportError: cannot import name 'default_lookups'

Unsupported lookup 'net_contains_or_equals' for CidrAddressField or join in the field not permitted

I have a Network model defined as so:

from netfields import CidrAddressField, NetManager

class Network(BaseModel):
    block = CidrAddressField()
    description = models.CharField(max_length=255, null=True, blank=True) 
    objects = NetManager()

When I call:

Network.objects.filter(block__net_contains_or_equals='10.142.0.0/22')

Django 1.7 throws:

FieldError: Unsupported lookup 'net_contains_or_equals' for CidrAddressField or join on the field not permitted.

I am using the latest commit from git.

Compatibility issues with Django 2.0a1

I've started testing a local application for compatibility with Django 2.0a1, and found that netfields is encountering the following error:

  File ".tox/py36-django20/lib/python3.6/site-packages/netfields/__init__.py", line 1, in <module>
    from netfields.managers import NetManager
  File ".tox/py36-django20/lib/python3.6/site-packages/netfields/managers.py", line 35, in <module>
    class NetQuery(sql.Query):
  File ".tox/py36-django20/lib/python3.6/site-packages/netfields/managers.py", line 36, in NetQuery
    query_terms = sql.Query.query_terms.copy()
AttributeError: type object 'Query' has no attribute 'query_terms'

This is with django-netfields==0.7.2.

Unsupported lookup 'net_overlaps' for CidrAddressField

models.py:

class LAN(models.Model): network_address = CidrAddressField() objects = NetManager()

within shell_plus session:
`

from ipaddress import IPv4Network
LAN.objects.create(network_address=IPv4Network('192.168.2.0/24'))
net2 = IPv4Network('192.168.2.64/26')
LAN.objects.filter(network_address__net_overlaps=net2)
...
django.core.exceptions.FieldError: Unsupported lookup 'net_overlaps' for CidrAddressField or join on the field not permitted.
`
Django is 3.0.7 and django-netfields is 1.2.2.

Thoughts ?

Request to support rhs lookup expressions

Would it be possible to support resolving RHS expressions in the custom lookups?

# query to find devices whose IP is not assigned to an interface
(
  Device.objects
  .prefetch_related('interface')
  .filter(
      ~Q(interface__interface_address__net_contains=F('ip_address'))
  )
  .values('name', 'ip_address')
)

They appear to pass through as a literal.

Col(networks_device, networks.Device.ip_address) does not appear to be an IPv4 or IPv6 interface.

Looking at the django source, they appear to have some boilerplate that may work.

    def get_prep_lookup(self):
        if hasattr(self.rhs, 'resolve_expression'):
            return self.rhs
        if self.prepare_rhs and hasattr(self.lhs.output_field, 'get_prep_value'):
            return self.lhs.output_field.get_prep_value(self.rhs)
        return self.rhs

If I get some time at the end of my sprint, I will see if I can work on this, but I wanted to at least get it documented.

Thank you.

setup.py

Hello,
I've created a trivial setup.py to make this an rpm. Could you add it to your project? The version number and classifiers sections needs to be changed I think. I can provide a suse friendly spec file for rpm users as well. Also please note that I'm not a django ninja so if this request is not appropriate you can simply ignore this request.

diff -Naur django-postgresql-netfields-0.05.org/setup.py django-postgresql-netfields-0.05/setup.py
--- django-postgresql-netfields-0.05.org/setup.py 1969-12-31 19:00:00.000000000 -0500
+++ django-postgresql-netfields-0.05/setup.py 2011-12-12 08:56:17.338927906 -0500
@@ -0,0 +1,20 @@
+from distutils.core import setup
+
+setup(name='django-postgresql-netfields',

```
 version='0.05',
```

 description='Django postgresql netfields implementation',

```
 author='Thomas Admacik',
```
```
 author_email='[email protected]',
```

 url='https://github.com/adamcik/django-postgresql-netfields',

 download_url='https://github.com/adamcik/django-postgresql-netfields/zipball/master',

```
 packages=['netfields'],
```

 classifiers=['Development Status :: 4 - Beta',

              'Environment :: Web Environment',

```
              'Framework :: Django',
```

              'Intended Audience :: Developers',

              'License :: OSI Approved :: BSD License',

              'Operating System :: OS Independent',

              'Programming Language :: Python',

```
              'Topic :: Utilities'],
```
```
 )
```

Check that we support Django 1.3

Check Django 1.3 support, and possibly set up 1.2/1.3 tox config for cross version testing.

RemovedInDjango19Warning: The django.forms.util module has been renamed.

When using latest version 0.3.1 with Django 1.8 I get a deprecation warning:

RemovedInDjango19Warning: The django.forms.util module has been renamed. Use django.forms.utils instead.

The reason lies in this file: netfields/forms.py

from django.forms.util import flatatt

Prefixlen lookups should be handled all in one, rather than three separate lookups

As brought up in #61:

This brought to mind a thought: Instead of having 3 lookups: __prefixlen, __max_prefixlen, and __min_prefixlen, wouldn't it make sense to be able to just have a single __prefixlen and be able to chain other operations onto it? So for example you could have:

MyModel.objects.filter(network__prefixlen=24)

MyModel.objects.filter(network__prefixlen__gte=24)

MyModel.objects.filter(network__prefixlen__lte=24)

I don't recall the exact implementation details for something like this, but what are your thoughts?

Currently, using __gte or __lte produces the following error:

django.core.exceptions.FieldError: Unsupported lookup 'prefixlen' for CidrAddressField or join on the field not permitted.

Consider int based storage for other DBs

Using inet_aton etc one can convert the ip addresses to their binary equivalent, this will allow for our extended look up semantics by using binary operators in the queries.

Setup email list for announcements, questions etc.

There should probably be a list for this project so there is a way to announce releases and reach whatever users this really has.

extra lookups don`t work

Django 3.2.7
Python 3.9.5

class Network(models.Model):
    cidr = CidrAddressField(db_index=True, verbose_name=_('cidr'), help_text=_('network cidr'))
    objects = NetManager()

python3 manage.py shell

>>> from netw.models import Network
>>> Network.objects.filter(cidr__startswith='10.58')
<QuerySet [<Network: Loopback*, 10.58.0.1/32>, <Network: Loopback*, 10.58.0.2/32>, <Network: Loopback*, 10.58.0.3/32>, <Network: Managment*, 10.58.8.0/22>, <Network: SCUD*, 10.58.12.0/22>, <Network: Printers*, 10.58.16.0/22>, <Network: Phones*, 10.58.20.0/22>, <Network: VideoCam*, 10.58.24.0/22>, <Network: Terminals*, 10.58.28.0/22>, <Network: Workers*, 10.58.32.0/22>, <Network: Workers2*, 10.58.36.0/22>, <Network: Install*, 10.58.40.0/22>, <Network: OpenPoint*, 10.58.44.0/22>]>
>>> Network.objects.filter(cidr__net_contained='10.58.0.0/16')
Traceback (most recent call last):
  File "/usr/lib/python3.9/code.py", line 90, in runcode
    exec(code, self.locals)
  File "<console>", line 1, in <module>
  File "/home/maxy/.local/lib/python3.9/site-packages/netfields/managers.py", line 19, in filter
    return super(NetManager, self).filter(*args, **kwargs)
  File "/home/maxy/.local/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/home/maxy/.local/lib/python3.9/site-packages/django/db/models/query.py", line 941, in filter
    return self._filter_or_exclude(False, args, kwargs)
  File "/home/maxy/.local/lib/python3.9/site-packages/django/db/models/query.py", line 961, in _filter_or_exclude
    clone._filter_or_exclude_inplace(negate, args, kwargs)
  File "/home/maxy/.local/lib/python3.9/site-packages/django/db/models/query.py", line 968, in _filter_or_exclude_inplace
    self._query.add_q(Q(*args, **kwargs))
  File "/home/maxy/.local/lib/python3.9/site-packages/django/db/models/sql/query.py", line 1393, in add_q
    clause, _ = self._add_q(q_object, self.used_aliases)
  File "/home/maxy/.local/lib/python3.9/site-packages/django/db/models/sql/query.py", line 1412, in _add_q
    child_clause, needed_inner = self.build_filter(
  File "/home/maxy/.local/lib/python3.9/site-packages/django/db/models/sql/query.py", line 1347, in build_filter
    condition = self.build_lookup(lookups, col, value)
  File "/home/maxy/.local/lib/python3.9/site-packages/django/db/models/sql/query.py", line 1187, in build_lookup
    lhs = self.try_transform(lhs, lookup_name)
  File "/home/maxy/.local/lib/python3.9/site-packages/django/db/models/sql/query.py", line 1226, in try_transform
    raise FieldError(
django.core.exceptions.FieldError: Unsupported lookup 'net_contained' for CidrAddressField or join on the field not permitted, perhaps you meant contains or icontains?

Can't search on CidrAddressField fields in the admin (Unsupported lookup 'icontains')

When trying to search on a CidrAddressField in the admin, it raises an exception:

FieldError at /admin/resources/ipsubnet/
Unsupported lookup 'icontains' for CidrAddressField or join on the field not permitted.

IPSubnet basically has a CidrAddressField and a few other fields:

class IPSubnet(models.Model):
    inet = CidrAddressField()
    objects = NetManager()

The admin looks like this:

class IPSubnetAdmin(admin.ModelAdmin):
    search_fields = ('inet__icontains',)

The full traceback is here: http://paste.aliens-lyon.fr/Fz4

The same exception happens for many values of search_fields (inet__contains, inet__iexact, inet__net_contains, etc). However, it works fine when searching just on inet, which seems to default to icontains.

Note that everything works fine in the Django shell:

>>> IPSubnet.objects.filter(inet__icontains='100')
[<IPSubnet: 2a00:5881:4000:100::/56>]
>>> IPSubnet.objects.filter(inet__iexact='2a00:5881:4000:100:0000:0000:0000:0000/56')
[<IPSubnet: 2a00:5881:4000:100::/56>]

I'm using Django 1.7 with Python 2.7.

How to use Django-Netfields with Django packages requiring a custom manager

Hello,

Some Django packages like Django-Polymorphic make use of custom managers.

As Django-Netfields also requires use of NetManager custom manager, one way to satisfy multiple requirements
is to have several managers and use appropriate one, depending on circumstances.

`from netfields import NetManager, InetAddressField
from Foo import FooManager, FooModel

class PersonManager(FooManager):
def get_by_natural_key(self, first_name, last_name):
return self.get(first_name=first_name, last_name=last_name)

class Person(FooModel):
first_name = models.CharField(max_length=100)
last_name = models.CharField(max_length=100)
inet = InetAddressField()

objects = PersonManager()
objects_netfields = NetManager()

n1 = IPv4Network('192.168.1.0/24')
o1 = Person.objects.get(last_name='Doe')
o2 = Person.objects_netfields.get(inet__net_contained_or_equal=n1)`

Is there a way to avoid having several managers ?
I was thinking of something like bellow but any alternative would be welcome.

`from netfields import NetManager, InetAddressField, Whatever
from Foo import FooManager, FooModel

class PersonManager(FooManager, Whatever):
def get_by_natural_key(self, first_name, last_name):
return self.get(first_name=first_name, last_name=last_name)

class Person(FooModel):
first_name = models.CharField(max_length=100)
last_name = models.CharField(max_length=100)
inet = InetAddressField()

objects = PersonManager()

n1 = IPv4Network('192.168.1.0/24')
o1 = Person.objects.get(last_name='Doe')
o2 = Person.objects.get(inet__net_contained_or_equal=n1)`

Thoughts ? Suggestions ?

Best regards

It would be nice to have functional equivalents in SQLite

I see now that the package name and the project name are different here, but for local testing of netfields, it would be nice to have functional parity using other database engines particularly SQLite.

I see there are some unmaintained extensions there that maybe fill in some of the gaps but it seems like a bigger project in general to tackle this.

https://github.com/mobigroup/sqlite3-inet

u'foobar' does not appear to be an IPv4 or IPv6 interface

The error should not show the 'u' prefix and the error message should make more sense to users.