soynatan / django-easy-audit Goto Github PK

View Code? Open in Web Editor NEW

695.0 695.0 176.0 347 KB

Yet another Django audit log app, hopefully the simplest one.

License: GNU General Public License v3.0

Python 98.13% HTML 1.87%

audit audit-logs auditing crud django django-application

django-easy-audit's People

Contributors

Stargazers

Watchers

Forkers

jheld thecthulhukid aidapaul knmkr bondstreet rossettistone flesser morlandi martiuripariu dakotablair jdkizer9 apsl novarac23 guoyu07 dimasciput manishgupta24 worsht lorenzomorandini doriancuentas marexv rmagee robmagee vish216 vcostas shanerich5 simage abartyshealth elliottcf ariera devsar funcho graceshaw sysols marquicus pauloduarte43 htayanloo dopatraman xiaobosheng mugumedavid umsenhorqualquer arefmousakhani theodo-uk arceo-labs leehosung navpreetsamra danielldt builtbykrit jameskirsop dgrant zagrebelin horosin cybeready felixcheruiyot dkgitdev steverecio abishek sparingsoftware gghildyal asifkhan69 lffsantos atakanarikan ringus1 etenil rogersdepelle ionpath red-lilith henribru chrisbrent jsoa pryelluw patrikrazem avallbona zyperco gkachru yusihuu yeexy proceduretech gurkankisaoglu gokberk-sahin sumit4613 jirkav weijie199148 tomasgarzon bugraaydin urfu-online openpolis glauciobb amrlotfy77 tian-yi andreparames jkarstens cspollar freemindcore whiteeyehansel clarityvalue trauty-is-me martinmanzo kirmalyshev xshapira homescriptone

django-easy-audit's Issues

Middleware isn't getting the username in Django < 1.10

There's no __call__ in Django 1.9 Middleware. Check Django documentation: https://docs.djangoproject.com/en/1.11/topics/http/middleware/#upgrading-pre-django-1-10-style-middleware

User not being recorded for CRUD events

This is my first time using easy audit so I may be missing something obvious here, but the list of CRUD events is not recording which user performed the action.

I have set up users in the usual way and easy audit it recording login events for users and the user is being recorded in request events, but not for CRUD events. Does anyone know why this might be?

Thanks.

New model not being audited

I added easy-audit to my app and the one existing model (Survey) was successfully being audited.

I then added another model (Result). That model is not being audited. What do I need to do so that easy-audit audits the Result model (and any other models I add) ?

Thanks
Dave

Error Deleting a model on django 1.11.3

Traceback:

File "/opt//env//lib64/python3.5/site-packages/django/core/handlers/exception.py" in inner
41. response = get_response(request)

File "/opt//env//lib64/python3.5/site-packages/django/core/handlers/base.py" in _legacy_get_response
249. response = self._get_response(request)

File "/opt//env//lib64/python3.5/site-packages/django/core/handlers/base.py" in _get_response
187. response = self.process_exception_by_middleware(e, request)

File "/opt//env//lib64/python3.5/site-packages/django/core/handlers/base.py" in _get_response
185. response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "/opt//env//lib64/python3.5/site-packages/django/views/decorators/csrf.py" in wrapped_view
58. return view_func(*args, **kwargs)

File "/opt//env//lib64/python3.5/site-packages/rest_framework/viewsets.py" in view
86. return self.dispatch(request, *args, **kwargs)

File "/opt//env//lib64/python3.5/site-packages/rest_framework/views.py" in dispatch
489. response = self.handle_exception(exc)

File "/opt//env//lib64/python3.5/site-packages/rest_framework/views.py" in handle_exception
449. self.raise_uncaught_exception(exc)

File "/opt//env//lib64/python3.5/site-packages/rest_framework/views.py" in dispatch
486. response = handler(request, *args, **kwargs)

File "/opt//env//lib64/python3.5/site-packages/rest_framework/mixins.py" in destroy
93. self.perform_destroy(instance)

File "/opt//env//lib64/python3.5/site-packages/rest_framework/mixins.py" in perform_destroy
97. instance.delete()

File "/opt//env//lib64/python3.5/site-packages/django/db/models/base.py" in delete
973. return collector.delete()

File "/opt//env//lib64/python3.5/site-packages/django/db/models/deletion.py" in delete
302. count = query.delete_batch(pk_list, self.using)

File "/opt//env//lib64/python3.5/site-packages/django/db/models/sql/subqueries.py" in delete_batch
45. num_deleted += self.do_query(self.get_meta().db_table, self.where, using=using)

File "/opt//env//lib64/python3.5/site-packages/django/db/models/sql/subqueries.py" in do_query
28. cursor = self.get_compiler(using).execute_sql(CURSOR)

File "/opt//env//lib64/python3.5/site-packages/django/db/models/sql/compiler.py" in execute_sql
886. raise original_exception

File "/opt//env//lib64/python3.5/site-packages/django/db/models/sql/compiler.py" in execute_sql
876. cursor.execute(sql, params)

File "/opt//env//lib64/python3.5/site-packages/django/db/backends/utils.py" in execute
80. return super(CursorDebugWrapper, self).execute(sql, params)

File "/opt//env//lib64/python3.5/site-packages/django/db/backends/utils.py" in execute
60. self.db.validate_no_broken_transaction()

File "/opt//env//lib64/python3.5/site-packages/django/db/backends/base/base.py" in validate_no_broken_transaction
448. "An error occurred in the current transaction. You can't "

Exception Type: TransactionManagementError at /api/v1/media/c807c0dd-d312-4d4f-b151-8d7f0d45b529/
Exception Value: An error occurred in the current transaction. You can't execute queries until the end of the 'atomic' block.

Project settings import

Hey, thanks for the library ;)

I have an issue with settings. It seems django-easy-audit didn't see setting from https://github.com/soynatan/django-easy-audit/wiki/Settings.

I have multiple settings files (in config/settings/ dir), but I think it isn't a problem because it works with other 3rd party libraries.

Here is my config:

# Django Easy Audit
# ------------------------------------------------------------------------------
# https://github.com/soynatan/django-easy-audit/wiki/Settings
DJANGO_EASY_AUDIT_WATCH_MODEL_EVENTS = True
DJANGO_EASY_AUDIT_WATCH_AUTH_EVENTS = False
DJANGO_EASY_AUDIT_WATCH_REQUEST_EVENTS = False

`user` field in CRUDEvent should be SET_NULL again

Turns out my change to allow the CRUDEvent to DO_NOTHING on a user deletion was a mistake (in specific circumstances). It means we can't delete the user, due to a referential integrity.

I propose I put it back to SET_NULL, and I add a new field to the model, maybe user_id_string, which will then hold onto the user pk/id even after deletion. This is needed because my project needs a full audit history of user information for operations on models, but also needs the ability in certain circumstances to delete said user, while not losing the audit history.

I can write this up pretty fast, but wanted your input, first.

KeyError: '_auth_user_id'

"/usr/local/lib/python3.5/dist-packages/easyaudit/signals/request_signals.py", line 40, in request_started_handler
user_id = session.get_decoded()['_auth_user_id']
KeyError: '_auth_user_id'

Please merge the code.
Fix in fork 'https://github.com/vcostas/django-easy-audit'

Exception when creating users in v1.1 (does not happen in v1.0)

Since upgrading to v1.1, when trying to create a User instance we encounter an exception:

> User.objects.create_user(username='test1')

2019-05-20 12:15:24,074 [ERROR] MainThread model_signals easy audit had a pre-save exception.
Traceback (most recent call last):
  File "/home/itais/workspace/infinidat/flux/eggs/django_easy_audit-1.1-py3.7.egg/easyaudit/signals/model_signals.py", line 56, in pre_save
    object_json_repr = serializers.serialize("json", [instance])
  File "/home/itais/workspace/infinidat/flux/eggs/Django-2.0.2-py3.7.egg/django/core/serializers/__init__.py", line 128, in serialize
    s.serialize(queryset, **options)
  File "/home/itais/workspace/infinidat/flux/eggs/Django-2.0.2-py3.7.egg/django/core/serializers/base.py", line 96, in serialize
    self.handle_m2m_field(obj, field)
  File "/home/itais/workspace/infinidat/flux/eggs/Django-2.0.2-py3.7.egg/django/core/serializers/python.py", line 72, in handle_m2m_field
    m2m_value(related) for related in getattr(obj, field.name).iterator()
  File "/home/itais/workspace/infinidat/flux/eggs/Django-2.0.2-py3.7.egg/django/db/models/fields/related_descriptors.py", line 498, in __get__
    return self.related_manager_cls(instance)
  File "/home/itais/workspace/infinidat/flux/eggs/Django-2.0.2-py3.7.egg/django/db/models/fields/related_descriptors.py", line 795, in __init__
    (instance, self.pk_field_names[self.source_field_name]))
ValueError: "<User: >" needs to have a value for field "id" before this many-to-many relationship can be used.

Our settings:

DJANGO_EASY_AUDIT_WATCH_REQUEST_EVENTS = False
DJANGO_EASY_AUDIT_ADMIN_SHOW_REQUEST_EVENTS = False
DJANGO_EASY_AUDIT_REGISTERED_CLASSES = [
    'auth.User',
    'users.UserInfo',
]

Release new version to PyPI

Hi, I was just opening an issue about cookie when I just found that the master branch already fixed that. So I would like to ask if it is possible to release a new version to PyPI so we get it from there instead of manually get a zip out of Github.

Make the app thread safe

From u/boxidea in r/django:

I don't think that storing the request on the EasyAuditMiddleware class is a good idea.

Middleware classes are shared across requests, so anything you store on the class is not going to be thread safe.

https://www.reddit.com/r/django/comments/5ux246/hello_everyone_i_created_a_simple_django_app_to/ddxpw8n/

Compare Old to New Datapoint

Hello! Fantastic library, thanks so much for your work on this! Quick question for you...

Is there a way to see what the old values were on the UPDATE events? So basically when I go into the admin and click on django easy audit CRUD, it shows me the updated value, but doesn't show me what the updated value was before it was updated. Is there a way for me to add that? THanks so much!!

Dave

Update that Changed URL fields from CharField to TextField leads to issue on MS SQLServer

Running migration 0009_auto_20180314_2225 causes the following error:

Running migrations:
  Applying easyaudit.0009_auto_20180314_2225...Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python3.6/site-packages/sql_server/pyodbc/base.py", line 520, in execute
    return self.cursor.execute(sql, params)
pyodbc.ProgrammingError: ('42000', "[42000] [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Column 'url' in table 'easyaudit_requestevent' is of a type that is invalid for use as a key column in an index. (1919) (SQLExecDirectW)")

TransactionManagementError

easy_audit has been working flawlessly for over a year but lately, I am having a similar issue to this one:

encode/django-rest-framework#5859

I couldn't find the root cause exactly but I can confirm that removing easy_audit from the INSTALLED_APPS solved this issue.

I wonder if there are instructions on how to uninstall easy-audit and all related tables cleanly?

Not logging all models

Followed simple setup steps.

pip install django-easy-audit
then add easyaudit in installed apps in settings
then add easyaudit.middleware.easyaudit.EasyAuditMiddleware to middlewares in settings

Works fine for few models but not creating logs for all models. Any method to debug or solve the issue ?

update models and migrations to use unicode explicitly

My project is looking to transition to py3 explicitly (not django 2.0 yet), and easyaudit will have a dumb migration to enable a unicode change, but the migration itself won't be part of the source code, so I'm not really interested in it yet.

It looks like the following may work:

update models.py to mark all string as u (or import unicode_literals)
then change all previous migrations that had b'..' to be u'..'.

I'm willing to test it out locally first and let you know what happens. The idea here is that (aside from a source change) there shouldn't need to be any dumb unicode migrations just because the interpreter changed.

Literal string 'null' stored in changed_field

I'm seeing a literal string of 'null' being stored in the database against some Update events.

If there aren't any changed fields one of the following should be occurring instead:

The event isn't logged as an 'Update' at all - as nothing has changed on the object
The changed_fields column should be left empty - null - instead of 'null'.

This bug makes it tedious to .filter() for actual changes to a given object.

Can I set RequestEventid as foreign key to CRUDEvent?

Can I set RequestEventid as foreign key to CRUDEvent?
i am not able to figure out how can i get RequestEventid while saving CRUDEvent in database.
help..

Add support for custom serializers

Some of my models that I would like to audit contain sensitive information that'd prefer not be saved. Ideally, I would have the option of fields to ignore on a per model basis when performing json serialization. I think that this could be accomplished with something as simple as a callback that post serialization that would allow for modifying the json. Additionally, for changes to sensitive fields, it would be nice to see that a change to that field was made without showing the info itself.

Save READ actions?

It would be awesome if we could also include R in the Crud actions log, for those who need complete auditing setup ;).

CRUDEvents added during migration, leading to IntegrityError in tests

The post_save signal handler uses the should_audit method to determine whether the model being changed should be tracked.

During migration, if model instances are created, should_audit eventually gets passed an instance of __fake__.ContentType instead of ContentType, does not recognize it as an unregistered class, and returns True.

This results in two issues:

Incorrect CRUDEvent entries for ContentType objects
If migration happens during tests, during test tear-down the ContentType cache will be inconsistent with the actual table, leading to errors like

django.db.utils.IntegrityError: insert or update on table "easyaudit_crudevent" violates foreign key constraint "easyaudit_crudevent_content_type_id_618ed0c6_fk_django_co"

DETAIL: Key (content_type_id)=(1) is not present in table "django_content_type".

I think this is really a Django issue, but it's exposed by this library. It might be the cause of #46, too.

Would it make sense to not audit entries during migration (perhaps by checking if type(instance).__module__ == '__fake__'?)

For now, the workaround I'm using is to call ContentType.objects.clear_cache() in my test tear-down.

ValidationError on query_string when running full_clean

I was experimenting with a signal that runs full_clean before saving all model instances, and ran into this error for the easyaudit.models.RequestEvent model:

django.core.exceptions.ValidationError: {'query_string': ['This field cannot be blank.']}

I believe query_string should be flagged as blank=True:

django-easy-audit/easyaudit/models.py

Line 76 in 03e05bc

query_string = models.TextField(null=True)

I am hosting easyaudit in my repo because I had to work around another issue, so this was a quick fix in my case, but I do think this should be addressed in master.

Thanks! Great plugin, by the way. :)

Django 2.0 Compatibility: reverse method moved

The reverse method has moved from django.core.urlresolvers (<2.0) to django.urls (2.0)

IntegrityErrors in tests

I'm creating a user in my test setup like so:

from django.contrib.auth.models import User
from rest_framework.test import APITestCase, APIClient

class ModelViewsTestCase(APITestCase):

    def setUp(self):
        self.user = User.objects.create_user(
            email='[email protected]',
            first_name='Test',
            last_name='User',
            password='password',
            username='[email protected]'
        )
        self.client.force_authenticate(user=self.user)

and it appears to be causing an IntegrityError in the teardown process (specifically in the _post_teardown method on line 925 of django/test/testcases.py. Am I creating a User the wrong way in my setup method?

Traceback (most recent call last):
  File "app/lib/python3.4/site-packages/django/db/backends/utils.py", line 63, in execute
    return self.cursor.execute(sql)
psycopg2.IntegrityError: insert or update on table "easyaudit_crudevent" violates foreign key constraint "easyaudit_crudevent_user_id_09177b54_fk_auth_user_id"
DETAIL:  Key (user_id)=(17) is not present in table "auth_user".


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "app/lib/python3.4/site-packages/django/test/testcases.py", line 216, in __call__
    self._post_teardown()
  File "app/lib/python3.4/site-packages/django/test/testcases.py", line 925, in _post_teardown
    self._fixture_teardown()
  File "app/lib/python3.4/site-packages/django/test/testcases.py", line 1081, in _fixture_teardown
    connections[db_name].check_constraints()
  File "app/lib/python3.4/site-packages/django/db/backends/postgresql/base.py", line 243, in check_constraints
    self.cursor().execute('SET CONSTRAINTS ALL IMMEDIATE')
  File "app/lib/python3.4/site-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
  File "app/lib/python3.4/site-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "app/lib/python3.4/site-packages/django/utils/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "app/lib/python3.4/site-packages/django/db/backends/utils.py", line 63, in execute
    return self.cursor.execute(sql)
django.db.utils.IntegrityError: insert or update on table "easyaudit_crudevent" violates foreign key constraint "easyaudit_crudevent_user_id_09177b54_fk_auth_user_id"
DETAIL:  Key (user_id)=(17) is not present in table "auth_user".

Use request.user to log request signal

In the request signals currently, the user is populated using the COOKIES which wouldn't work for REST APIs based on Django. Can we instead use the request.user object and make sure that the middleware is included after the Authentication middleware?

django-easy-audit/easyaudit/signals/request_signals.py

Line 29 in 3518a7b

if environ.get('HTTP_COOKIE'):

Cannot get user in request events

First of all, thank you so much for the wonderful work. It is the best app for audit I have seen.

For the request events, currently it uses request_started_handler as the trigger and it can only get the auth information from cookies. But for restful application , often other authentication is used and no user can be recorded.

I wonder if it is easier to implement the trigger using the middleware way? As long as the middleware is placed after the authentication middleware, all the information needed should be presented.

Potential security issue

I may have found a security issue. I'm not comfortable posting it here yet as I don't want an unpatched vuln out there. I reached out to @soynatan directly via email a few days ago, but haven't heard back yet. Is there anyone else I can reach out to?

TransactionError

I literally have no time to chase this down ... moving to another "model audit" package. Sorry.

But, I added easyaudit, config'd settings to True for auth and model events. It never worked when I specified the list of models (e..g, ['app.model1', 'app.model2'])
So, I left the list empty -- audit everything.

Quick test shows that easyaudit is picking up CREATIONS and DELETIONS. So, I alter an object in the admin interface to test CHANGED.

... boom: Cannot save the edited object due to TransactionError blah blah blah Atomic.
So, I turned OFF easyaudit in the settings.
... boom: Can save the altered object.

Sorry. I need to move on. I can't keep debugging this. I have to roll something simpler (shadow fields) or something. Deadline looming.

request_started_handler() missing 1 required positional argument: 'environ'

When I include easy_audit via the steps in the readme (add it to INSTALLED_APPS and MIDDLEWARE), I get this error on app startup:

request_started_handler() missing 1 required positional argument: 'environ'

As far as I can tell, this argument is supposed to be passed to the signal receiver automatically. I'm not sure what's causing this issue.

OperationalError when migrating : (1170, "BLOB/TEXT column 'url' used in key specification without a key length")

i encountered followed error when migrating

easyaudit.RequestEvent.url: (fields.W162) MySQL does not support a database index on longtext columns.
        HINT: An index won't be created. Silence this warning if you don't care about it.
Operations to perform:
  Apply all migrations: easyaudit
Running migrations:
  Applying easyaudit.0001_initial... OK
  Applying easyaudit.0002_auto_20170125_0759... OK
  Applying easyaudit.0003_auto_20170228_1505... OK
  Applying easyaudit.0004_auto_20170620_1354... OK
  Applying easyaudit.0005_auto_20170713_1155... OK
  Applying easyaudit.0006_auto_20171018_1242... OK
  Applying easyaudit.0007_auto_20180105_0838... OK
  Applying easyaudit.0008_auto_20180220_1908... OK
  Applying easyaudit.0009_auto_20180314_2225...Traceback (most recent call last):
  File "/Users/shinseunghoon/repository/toy/sample-backoffice/.venv/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/Users/shinseunghoon/repository/toy/sample-backoffice/.venv/lib/python3.7/site-packages/django/db/backends/mysql/base.py", line 71, in execute
    return self.cursor.execute(query, args)
  File "/Users/shinseunghoon/repository/toy/sample-backoffice/.venv/lib/python3.7/site-packages/MySQLdb/cursors.py", line 206, in execute
    res = self._query(query)
  File "/Users/shinseunghoon/repository/toy/sample-backoffice/.venv/lib/python3.7/site-packages/MySQLdb/cursors.py", line 312, in _query
    db.query(q)
  File "/Users/shinseunghoon/repository/toy/sample-backoffice/.venv/lib/python3.7/site-packages/MySQLdb/connections.py", line 224, in query
    _mysql.connection.query(self, query)
MySQLdb._exceptions.OperationalError: (1170, "BLOB/TEXT column 'url' used in key specification without a key length")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "manage.py", line 21, in <module>
    main()
  File "manage.py", line 17, in main
    execute_from_command_line(sys.argv)
  File "/Users/shinseunghoon/repository/toy/sample-backoffice/.venv/lib/python3.7/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/Users/shinseunghoon/repository/toy/sample-backoffice/.venv/lib/python3.7/site-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/shinseunghoon/repository/toy/sample-backoffice/.venv/lib/python3.7/site-packages/django/core/management/base.py", line 323, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/Users/shinseunghoon/repository/toy/sample-backoffice/.venv/lib/python3.7/site-packages/django/core/management/base.py", line 364, in execute
    output = self.handle(*args, **options)
  File "/Users/shinseunghoon/repository/toy/sample-backoffice/.venv/lib/python3.7/site-packages/django/core/management/base.py", line 83, in wrapped
    res = handle_func(*args, **kwargs)
  File "/Users/shinseunghoon/repository/toy/sample-backoffice/.venv/lib/python3.7/site-packages/django/core/management/commands/migrate.py", line 234, in handle
    fake_initial=fake_initial,
  File "/Users/shinseunghoon/repository/toy/sample-backoffice/.venv/lib/python3.7/site-packages/django/db/migrations/executor.py", line 117, in migrate
    state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "/Users/shinseunghoon/repository/toy/sample-backoffice/.venv/lib/python3.7/site-packages/django/db/migrations/executor.py", line 147, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "/Users/shinseunghoon/repository/toy/sample-backoffice/.venv/lib/python3.7/site-packages/django/db/migrations/executor.py", line 245, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/Users/shinseunghoon/repository/toy/sample-backoffice/.venv/lib/python3.7/site-packages/django/db/migrations/migration.py", line 124, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/Users/shinseunghoon/repository/toy/sample-backoffice/.venv/lib/python3.7/site-packages/django/db/migrations/operations/fields.py", line 249, in database_forwards
    schema_editor.alter_field(from_model, from_field, to_field)
  File "/Users/shinseunghoon/repository/toy/sample-backoffice/.venv/lib/python3.7/site-packages/django/db/backends/base/schema.py", line 535, in alter_field
    old_db_params, new_db_params, strict)
  File "/Users/shinseunghoon/repository/toy/sample-backoffice/.venv/lib/python3.7/site-packages/django/db/backends/base/schema.py", line 685, in _alter_field
    params,
  File "/Users/shinseunghoon/repository/toy/sample-backoffice/.venv/lib/python3.7/site-packages/django/db/backends/base/schema.py", line 137, in execute
    cursor.execute(sql, params)
  File "/Users/shinseunghoon/repository/toy/sample-backoffice/.venv/lib/python3.7/site-packages/django/db/backends/utils.py", line 99, in execute
    return super().execute(sql, params)
  File "/Users/shinseunghoon/repository/toy/sample-backoffice/.venv/lib/python3.7/site-packages/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/Users/shinseunghoon/repository/toy/sample-backoffice/.venv/lib/python3.7/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/Users/shinseunghoon/repository/toy/sample-backoffice/.venv/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/Users/shinseunghoon/repository/toy/sample-backoffice/.venv/lib/python3.7/site-packages/django/db/utils.py", line 89, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/Users/shinseunghoon/repository/toy/sample-backoffice/.venv/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/Users/shinseunghoon/repository/toy/sample-backoffice/.venv/lib/python3.7/site-packages/django/db/backends/mysql/base.py", line 71, in execute
    return self.cursor.execute(query, args)
  File "/Users/shinseunghoon/repository/toy/sample-backoffice/.venv/lib/python3.7/site-packages/MySQLdb/cursors.py", line 206, in execute
    res = self._query(query)
  File "/Users/shinseunghoon/repository/toy/sample-backoffice/.venv/lib/python3.7/site-packages/MySQLdb/cursors.py", line 312, in _query
    db.query(q)
  File "/Users/shinseunghoon/repository/toy/sample-backoffice/.venv/lib/python3.7/site-packages/MySQLdb/connections.py", line 224, in query
    _mysql.connection.query(self, query)
django.db.utils.OperationalError: (1170, "BLOB/TEXT column 'url' used in key specification without a key length")

python: 3.7.1
database: mysql 5.7
django: 2.2

it seems url field in requestevent table need index, but it didn't get index.

unique together index

Would you be interested in offering a unique_together index on (object_id and content_type)? We have a lot of CRUDEvent objects and it would be faster to look those up if we could get it through an index.

I'm open to alternative suggestions, and of course a flat out no is "ok" given that this is a fairly generic library that doesn't intend to be particularly opinionated/obtrusive.

OperationalError when migrating

I get this error when I try to migrate:

django.db.utils.OperationalError: (1825, "Failed to add the foreign key constraint on table 'easyaudit_crudevent'. Incorrect options in FOREIGN KEY constraint 'renew_cloud/easyaudit_crudevent_user_id_09177b54_fk_renew_customuser_id'")

Note that I have set "init_command": "SET foreign_key_checks = 0;", in database options, but still there's an error being thrown here.

Am I missing some dependency?

Why the username and id?

Did you have a specific reason for doing so or was it convenience?

Add ability to hide filters in django admin

The django admin shows filters and some may display a long list of data. For example, the filter by user show a very long list.

So having the ability to disable filters would be very useful.

TransactionManagementError While loading fixture in migration

I am trying to load a fixture in migration like:

fixture_dir = os.path.abspath(
    os.path.join(os.path.dirname(__file__), '../fixtures'))
fixture_filename = os.path.join(fixture_dir, 'fixtures.json')


def load_fixture(apps, schema_editor):
    from django.core.management import call_command
    call_command("loaddata", fixture_filename)


class Migration(migrations.Migration):

    dependencies = [
        ('sd_organizations', '0001_initial'),
    ]

    operations = [
        migrations.RunPython(load_fixture),
    ]

I ran unittest like python manage.py test

This error comes up:

2018-04-05 12:10:05,989 easyaudit.signals.model_signals ERROR    easy audit had a post-save exception.
Traceback (most recent call last):
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
psycopg2.ProgrammingError: relation "easyaudit_crudevent" does not exist
LINE 1: INSERT INTO "easyaudit_crudevent" ("event_type", "object_id"...
                    ^


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/easyaudit/signals/model_signals.py", line 78, in post_save
    user_pk_as_string=str(user.pk) if user else user
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/db/models/query.py", line 394, in create
    obj.save(force_insert=True, using=self.db)
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/db/models/base.py", line 808, in save
    force_update=force_update, update_fields=update_fields)
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/db/models/base.py", line 838, in save_base
    updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/db/models/base.py", line 924, in _save_table
    result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/db/models/base.py", line 963, in _do_insert
    using=using, raw=raw)
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/db/models/query.py", line 1076, in _insert
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1112, in execute_sql
    cursor.execute(sql, params)
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/utils/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: relation "easyaudit_crudevent" does not exist
LINE 1: INSERT INTO "easyaudit_crudevent" ("event_type", "object_id"...
                    ^

Traceback (most recent call last):
  File "manage.py", line 15, in <module>
    execute_from_command_line(sys.argv)
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/core/management/__init__.py", line 356, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/core/management/commands/test.py", line 29, in run_from_argv
    super(Command, self).run_from_argv(argv)
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/core/management/base.py", line 283, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/core/management/base.py", line 330, in execute
    output = self.handle(*args, **options)
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/core/management/commands/test.py", line 62, in handle
    failures = test_runner.run_tests(test_labels)
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/test/runner.py", line 601, in run_tests
    old_config = self.setup_databases()
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/test/runner.py", line 546, in setup_databases
    self.parallel, **kwargs
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/test/utils.py", line 187, in setup_databases
    serialize=connection.settings_dict.get('TEST', {}).get('SERIALIZE', True),
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/db/backends/base/creation.py", line 69, in create_test_db
    run_syncdb=True,
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/core/management/__init__.py", line 131, in call_command
    return command.execute(*args, **defaults)
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/core/management/base.py", line 330, in execute
    output = self.handle(*args, **options)
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/core/management/commands/migrate.py", line 204, in handle
    fake_initial=fake_initial,
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/db/migrations/executor.py", line 115, in migrate
    state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/db/migrations/executor.py", line 145, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/db/migrations/executor.py", line 244, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/db/migrations/migration.py", line 129, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/db/migrations/operations/special.py", line 193, in database_forwards
    self.code(from_state.apps, schema_editor)
  File "/home/manish/django-rest-api/sd_organizations/migrations/0002_load_initial_data.py", line 16, in load_fixture
    call_command("loaddata", fixture_filename)
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/core/management/__init__.py", line 131, in call_command
    return command.execute(*args, **defaults)
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/core/management/base.py", line 330, in execute
    output = self.handle(*args, **options)
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/core/management/commands/loaddata.py", line 69, in handle
    self.loaddata(fixture_labels)
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/core/management/commands/loaddata.py", line 109, in loaddata
    self.load_label(fixture_label)
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/core/management/commands/loaddata.py", line 175, in load_label
    obj.save(using=self.using)
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/core/serializers/base.py", line 205, in save
    models.Model.save_base(self.object, using=using, raw=True, **kwargs)
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/db/models/base.py", line 838, in save_base
    updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/db/models/base.py", line 905, in _save_table
    forced_update)
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/db/models/base.py", line 955, in _do_update
    return filtered._update(values) > 0
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/db/models/query.py", line 664, in _update
    return query.get_compiler(self.db).execute_sql(CURSOR)
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1204, in execute_sql
    cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 899, in execute_sql
    raise original_exception
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 889, in execute_sql
    cursor.execute(sql, params)
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/db/backends/utils.py", line 59, in execute
    self.db.validate_no_broken_transaction()
  File "/home/manish/django-rest-api/env/lib/python3.6/site-packages/django/db/backends/base/base.py", line 448, in validate_no_broken_transaction
    "An error occurred in the current transaction. You can't "
django.db.transaction.TransactionManagementError: Problem installing fixture '/home/manish/django-rest-api/sd_organizations/fixtures/fixtures.json': Could notload sd_organizations.OrganizationType(pk=2): An error occurred in the current transaction. You can't execute queries until the end of the 'atomic' block.

How to load fixture without triggering the audit post_save signals?

master branch foreign key constraint on pre-save

I'm still looking into this, but there is an issue when we try to create an object in our unit tests. It looks like it's failing in the pre-save due to a foreign key constraint.

I will be looking into it further tomorrow.

Arbitrary object primary key field

At this point, this package only supports IntegerField as PK field of audited objects. Is it possible to make the field more general. For instance, in our projects we use UUID fields as PK fields. Thank you for considering.

How do I create new migrations in this project?

Most django projects have a manage.py file that gives access to makemigrations. This project does not have one, yet I see programmatically generated migration files. More out of curiosity than anything else, how did @soynatan create them?

it would be nice to config what entities should be audited to avoid ALL by default

CRUDEvent.object_repr field too small

CRUDEvent.obj_repr is defined as a CharField(max_length=255). There are valid reasons to have reprs longer than 255 chars. I suggest changing this to models.TextField(null=True, blank=True).

removal of db_constraint on User

Hello!

My project was/is looking to separate our easyaudit data to a separate database.

What would your thoughts be on removing the db_constraint on the models that reference a User?

This would mean that if the User doesn't exist that an exception would be raised when trying to access it.

However under normal flow it would be fine for most applications.

I'm not sure if there are other side effects from doing this, but I wanted your thoughts. I'm also not sure that this could be a project-level setting (as in it may have to be hard-coded).

pre-save error after running loaddata (for fixtures)

We're getting a pre-save error after loading our fixtures. PR opened here: #77

Delta and full update JSON on CRUDEvent

My team is considering having a delta field of the changes, as opposed to the full JSON object on post_save.

I haven't started my implementation yet, but I was curious what you thought of the following extension:

In the signal code, let the output of the callback/hook code (as we know, it's optional), return False, or True, or some sort of enum-like value. In this case it could be FULL, DELTA. That could be a little confusing, but maybe not so bad. Or, have it be a different kind of hook -- different from the one currently implemented in the code.
Allow saving both the FULL and DELTA on the CRUDEvent, so the project can determine if they want both or one or the other, always, or programmatically. This is in the same vein as above. It unfortunately would potentially duplicate the JSON in the worst case, if both are used on a given object operation.

I want to keep it as simple as possible, but still flexible for the consumer's needs. Are you interested in such a change?

Data too long for column 'query_string'

Hi, I am using Django 1.10.7 with MySQL 5.7 and one of my test failed with the following error:

DataError: (1406, "Data too long for column 'query_string' at row 1")

The test is sending a GET request to one of my API endpoint with a fairly large query string.
So I checked the RequestEvent model and indeed the query_string attribute has a max_length=255.

After a quick search I found out on this StackOverflow question that the actual maximum length of the query string may be bigger. I wonder if it is appropriate to either use a larger max_length (e.g. 4000) or use a TextField for storing the query string.

request signal cookie py3

Looks like we have a python3 incompatibility in the request signals module.

It does from Cookie mport SimpleCookie.

It looks like the module in py3 is: http.cookies.SimpleCookie

If using six it could be six.moves.http_cookies.SimpleCookie. Or we could just do an import try/except.

Token invalidation

I installed easy audit and it seems to have some rather strange effects on my authentication. It is a REST API with Token authentication. However, only for some users, only if "easyaudit" is in INSTALLED_APPS and the middleware is activated, their tokens are instantly invalidated.

Even weirder, if I comment the easy audit stuff from installed apps and middlewares, the user can then properly log in. And if I put everything back I never have issues with that user again, only with others.

The only message I get in the console is No handlers could be found for logger "easyaudit.signals.model_signals"

I know it's a very Byzantine error, but maybe it has happened to others? I'm looking for a sane fix. Will update if I find it :)

pre_save signal fails serialization for new objects

When the signal tries to serialize the django User model it fails because the object doesn't have a PK yet
for many to many relation

object_json_repr = serializers.serialize("json", [instance])

https://github.com/soynatan/django-easy-audit/blob/master/easyaudit/signals/model_signals.py#L56

def _generate_uuid():
    return uuid.uuid4().hex

class SampleModel(models.Model):
    _id = models.CharField(max_length=32, primary_key=True, editable=False, default=_generate_uuid)

How to Disable these signals for testing?

I want to disable the easyaudit signals for testing. How can I do that?