jazzband / django-fernet-encrypted-fields Goto Github PK

I am using this issue as a discussion board sort of.

Since the change in #15 updated the required python version to 3.8 and updated the minimum Django version to 3.2, I would like to add typings to the entire codebase.

As we all know, there are many advantages to typing, most importantly is that it eliminates an entire class of errors.

Let me know if anyone has objections to this. Else, I'd like to get started on this.

As a follow up, we could add some type checkers such as Mypy to the codebase. I of course would be willing to do this as well.

This issue tracks the implementation of the Jazzband guidelines for the project django-fernet-encrypted-fields

It was initiated by @frgmt who was automatically assigned in addition to the Jazzband roadies.

See the TODO list below for the generally required tasks, but feel free to update it in case the project requires it.

Feel free to ping a Jazzband roadie if you have any question.

TODOs

• Fix all links in the docs (and README file etc) from old to new repo
• Add the Jazzband badge to the README file
• Add the Jazzband contributing guideline to the CONTRIBUTING.md or CONTRIBUTING.rst file
• Check if continuous testing works with GitHub Actions
• Check if test coverage is tracked with Codecov
• Add jazzband account to PyPI project as maintainer role (e.g. URL: https://pypi.org/manage/project/django-fernet-encrypted-fields/collaboration/)
• Add jazzband-bot as maintainer to the Read the Docs project (e.g. URL: https://readthedocs.org/dashboard/django-fernet-encrypted-fields/users/)
• Add incoming GitHub webhook integration to Read the Docs project (e.g. URL: https://readthedocs.org/dashboard/django-fernet-encrypted-fields/integrations/)
• Fix project URL in GitHub project description
• Review project if other services are used and port them to Jazzband
• Decide who is project lead for the project (if at all)
• Set up CI for Jazzband project releases if needed and open ticket if yes

Project details

It would be nice to be able to rotate the salt from time to time for increased security (longer salts). I've just added a PR #1 for this based on how django-fernet-fields.

My idea is to make the SALT_KEY a list where you can add new keys to the front of the list. It if's not a list then it's convert into one within the EncryptedFieldMixin for backward compatibility. Then the class properties that are currently there are moved to cached_property methods so that multiple keys could be built and then the Fernet call is replaced by MultiFernet if more than one key is to be checked.

This allows older encrypted records to be read whilst new entries will be saved using the new key. This is useful as it allows people to load an object then save it in order to re-hash and encrypt the plain-text value.

This could be extended in the future to add new settings.py entry to allow the kdf to be changed from PBKDF3HMAC to something else if required or, for now, just by extending the class and overloading the new keys method.

This is all based on https://github.com/orcasgit/django-fernet-fields which is a popular encryption field package but it no longer maintained

Hi!

Great package!

Is there any reason for not leaving SECRET_KEY usable as default encryption key?

Thanks!

Hi,

I stumbled accross your repository while looking for a way to encrypt a model field. I was glad to find one that is maintained, which seems to be a huge problem with model encryption django apps.

As encryption apps are crucial for some applications and cannot be easily replaced, it would be nice to have one model encryption app joining Django Jazzband [https://jazzband.co/] to make sure it will also be well maintained in the future.

I'm getting this error when saving data using django admin:

Traceback (most recent call last):
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/django/contrib/admin/options.py", line 622, in wrapper
    return self.admin_site.admin_view(view)(*args, **kwargs)
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/django/utils/decorators.py", line 130, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/django/views/decorators/cache.py", line 56, in _wrapped_view_func
    response = view_func(request, *args, **kwargs)
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/django/contrib/admin/sites.py", line 236, in inner
    return view(request, *args, **kwargs)
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/django/contrib/admin/options.py", line 1670, in add_view
    return self.changeform_view(request, None, form_url, extra_context)
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/django/utils/decorators.py", line 43, in _wrapper
    return bound_method(*args, **kwargs)
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/django/utils/decorators.py", line 130, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/django/contrib/admin/options.py", line 1549, in changeform_view
    return self._changeform_view(request, object_id, form_url, extra_context)
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/django/contrib/admin/options.py", line 1593, in _changeform_view
    form_validated = form.is_valid()
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/django/forms/forms.py", line 190, in is_valid
    return self.is_bound and not self.errors
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/django/forms/forms.py", line 185, in errors
    self.full_clean()
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/django/forms/forms.py", line 406, in full_clean
    self._post_clean()
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/django/forms/models.py", line 411, in _post_clean
    self.instance.full_clean(exclude=exclude, validate_unique=False)
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/django/db/models/base.py", line 1233, in full_clean
    self.clean_fields(exclude=exclude)
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/django/db/models/base.py", line 1275, in clean_fields
    setattr(self, f.attname, f.clean(raw_value, self))
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/django/db/models/fields/__init__.py", line 670, in clean
    value = self.to_python(value)
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/encrypted_fields/fields.py", line 45, in to_python
    value = self.f.decrypt(bytes(value, 'utf-8')).decode('utf-8')
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/cryptography/fernet.py", line 76, in decrypt
    timestamp, data = Fernet._get_unverified_token_data(token)
  File "/home/victorm/git/oystrprototipo/venv/lib/python3.9/site-packages/cryptography/fernet.py", line 108, in _get_unverified_token_data
    raise InvalidToken
cryptography.fernet.InvalidToken

Model:

from django.db import models
from encrypted_fields.fields import EncryptedCharField


class Setting(models.Model):
    key = models.CharField(max_length=255, verbose_name='Chave', unique=True)
    value = EncryptedCharField(max_length=2000, verbose_name='Valor')

Admin:

from django.contrib import admin
from settings.models.setting import Setting


@admin.register(Setting)
class SettingAdmin(admin.ModelAdmin):
    pass

Django version: 4.0
Database: sqlite

I've just tested rotating Django's SECRET_KEY and my field's couldn't be encrypted anymore even though a SALT_KEY is set:

How to replicate:

1. Create a new django project
2. Create a new app with an EncryptedTextField
3. Start Django shell
4. Create an instance with some example text and save
5. Stop Django shell
6. Update secret key
7. Load earlier data from db and print content of the text field

# Settings
# orignal key
SECRET_KEY = 'django-insecure-i40*4wu4erqc24em97k&fy#bj^rg%25)mtf2_cj)=2u7eg(pht'

# rotated key
# SECRET_KEY = 'django-insecure-i40*4wu4erqc24em97k&fy#bj^rg%25)mtf2_cj)=2u7eg(ph3'

SALT_KEY = '0123456789abcdefghijklmnopqrstuvwxyz'

Model

from encrypted_fields.fields import EncryptedTextField

from django.db import models


class MyModel(models.Model):
    text_field = EncryptedTextField()

What I did.

Console output
In [3]: MyModel.objects.create(text_field="test1")
Out[3]: <MyModel: MyModel object (1)>

In [4]: m = MyModel.objects.last()

In [5]: m.text_field
Out[5]: 'test1'

###########
# Rotate key #
###########

In [2]: from test_app.models import MyModel

In [3]: m = MyModel.objects.last()

In [4]: m.text_field
Out[4]: 'gAAAAABj8is-Hi0WTGpbcC5Xg1J8CYO8Zh5ErsWW631UZ2m3LimZkRrCZrII83VJAy831W9ISWCja1SQecwAA7eZ0panQuNY4g==

I think, it should be changed that it does not depend on the secret key, as this will easily break applications. Developers should only have one setting that affects the encryption.

I'm reasonably new to Django so this may be a fundamental misunderstanding on my part, but I'm unable to do query lookups against the encrypted fields I've defined.

The encrypted fields work as expected when saving or retrieving data from the database but not when I try to do a lookup, e.g. FieldClass.objects.filter(attribute='value')

I assume that this is because the lookup likely just does a low-level SQL query and doesn't pre-fetch/decrypt the data before the query is processed, but I don't know this for fact and haven't had much luck determining that with the Django documentation yet.

I have another project that uses encrypted fields with SQLAlchemy and the queries work as expected even with encrypted fields, so I think it should be able to be done, I'm just not sure if so within Django.

My environment is:
Python 3.9.13 (virtual env)
Django 4.0.6
sqlite3
django-fernet-encrypted-files 0.1.2

macOS 12.5

Please let me know if you need any further information.
Any and all help will be greatly appreciated.

Hi,

I was developing a reusable-applications to use in my projects.
I decided to encrypt some IntegerFields by using EncryptedIntegerField from django-fernet-encrypted-fields

So then when I tried to test my app by tox I got this error:

".tox/py310-4.0/lib/python3.10/site-packages/django/db/backends/base/operations.py", line 673, in integer_field_range return self.integer_field_ranges[internal_type]

KeyError: 'TextField'

As you can see in the error message I'm using python 3.10 and django 4.0.4

I searched for this error and I found a similar problem in django-cryptography project in the issue #16

The suggested solution there was using a DecimalField with decimal_place=0 instead of an IntegerField

I hope there is a possible solution to use IntegerField as I'm trying to keep integer data in my model.