kingbuzzman / django-squash Goto Github PK

View Code? Open in Web Editor NEW

29.0 29.0 4.0 248 KB

A migration squasher that doesn't care how Humpty Dumpty was put together.

License: MIT License

Python 100.00%

django django-migrations

django-squash's People

Contributors

Stargazers

Watchers

Forkers

conntac martinlehoux stianjensen breid1313

django-squash's Issues

IndexError: list index out of range

Hello guys! Thx for great lib with a very appreciate functionality.

Unfortunately I can't use your lib, due to this exception:

DEBUG 2020-12-21 10:42:42,574 selector_events Using selector: KqueueSelector
Traceback (most recent call last):
  File "manage.py", line 21, in <module>
    main()
  File "manage.py", line 17, in main
    execute_from_command_line(sys.argv)
  File "/Users/f.eremeev/sasuke/backend/saas_core/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
    utility.execute()
  File "/Users/f.eremeev/sasuke/backend/saas_core/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 395, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/f.eremeev/sasuke/backend/saas_core/venv/lib/python3.8/site-packages/django/core/management/base.py", line 330, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/Users/f.eremeev/sasuke/backend/saas_core/venv/lib/python3.8/site-packages/django/core/management/base.py", line 371, in execute
    output = self.handle(*args, **options)
  File "/Users/f.eremeev/sasuke/backend/saas_core/venv/lib/python3.8/site-packages/django/core/management/base.py", line 85, in wrapped
    res = handle_func(*args, **kwargs)
  File "/Users/f.eremeev/sasuke/backend/saas_core/venv/lib/python3.8/site-packages/django_squash/management/commands/squash_migrations.py", line 102, in handle
    squashed_changes = autodetector.squash(
  File "/Users/f.eremeev/sasuke/backend/saas_core/venv/lib/python3.8/site-packages/django_squash/management/commands/lib/autodetector.py", line 241, in squash
    self.convert_migration_references_to_objects(real_loader, graph, changes)
  File "/Users/f.eremeev/sasuke/backend/saas_core/venv/lib/python3.8/site-packages/django_squash/management/commands/lib/autodetector.py", line 202, in convert_migration_references_to_objects
    dependency = tuple(migrations[-1])
IndexError: list index out of range

I'm looking forward for your answer, can't wait to squash my migrations 😄

Hello again, sorry I missed your answer. So I plan to reopen issue, cause I tried to squash my migrations again :)

I get this error again:

Traceback (most recent call last):
  File "manage.py", line 21, in <module>
    main()
  File "manage.py", line 17, in main
    execute_from_command_line(sys.argv)
  File "/Users/f.eremeev/haut_ai_team/sasuke/backend/saas_core/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "/Users/f.eremeev/haut_ai_team/sasuke/backend/saas_core/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 413, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/f.eremeev/haut_ai_team/sasuke/backend/saas_core/venv/lib/python3.8/site-packages/django/core/management/base.py", line 354, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/Users/f.eremeev/haut_ai_team/sasuke/backend/saas_core/venv/lib/python3.8/site-packages/django/core/management/base.py", line 398, in execute
    output = self.handle(*args, **options)
  File "/Users/f.eremeev/haut_ai_team/sasuke/backend/saas_core/venv/lib/python3.8/site-packages/django/core/management/base.py", line 89, in wrapped
    res = handle_func(*args, **kwargs)
  File "/Users/f.eremeev/haut_ai_team/sasuke/backend/saas_core/venv/lib/python3.8/site-packages/django_squash/management/commands/squash_migrations.py", line 102, in handle
    squashed_changes = autodetector.squash(
  File "/Users/f.eremeev/haut_ai_team/sasuke/backend/saas_core/venv/lib/python3.8/site-packages/django_squash/management/commands/lib/autodetector.py", line 241, in squash
    self.convert_migration_references_to_objects(real_loader, graph, changes)
  File "/Users/f.eremeev/haut_ai_team/sasuke/backend/saas_core/venv/lib/python3.8/site-packages/django_squash/management/commands/lib/autodetector.py", line 202, in convert_migration_references_to_objects
    dependency = tuple(migrations[-1])
IndexError: list index out of range

I have 126 migrations.
I think that I don't have self referencing migrations, only merge migrations, referencing others.
I think I didn't have any monkey patches. But maybe you could give one little example, how this could be performed.
My django-named packages:

pip list | grep django
django-extensions                       3.1.3
django-pglocks                          1.0.4
django-polymorphic                      3.0.0
django-prometheus                       2.1.0
django-squash                           0.0.6
djangorestframework                     3.12.4
djangorestframework-csv                 2.1.1
flake8-django                           1.1.2
pytest-django                           4.2.0

I don't know where could I start debugging =/

Originally posted by @eremeevfd in #5 (comment)

Touches migrations inside site-packages

This command works great, and is just what I was looking for to workaround shortcomings in the normal sqlashmigrations command.
I was quite surprised to find it tried to squash migrations inside my virtualenv, though. I have a lot of different packages there with migrations, and I also have a lot of self-authored apps, so both --ignore-app and --only will require a long list of apps (that also needs to be updated over time).
Would it be possible to find some other way to automatically ignore all apps inside site-packages?

Issue with RunPython functions being part of the class

Function is inside the class not the outside.

class Migration(migrations.Migration):

    dependencies = [
        ('app', '0001_initial'),
    ]

    def forwards_func(apps, schema_editor):
        ....

    operations = [
        migrations.RunPython(forwards_func, reverse_code=migrations.RunPython.noop)
    ]

Optional auto adding namespace the functions used in the code and reverse_code arguments of RunPython

Problem

In a medium complexity project, I have several migrations whose function names referenced in code and reverse_code of RunPython are respectively forward and backward.
django-squash manages to create these functions in the new migration file, however it creates them with duplicate names. As there are more than a hundred of them per app, it is not so good to analyze and correct errors manually.

A possible solution

I guess maybe modify django-squash instead of using the same name as the original function, change it to generate the function definition name for something like

app_name + _ + migration_file_name_without_extension + _ + function_name + (apps, schema_editor):  # noqa: E501

example:
app_name is user
migration_file is 0106_populate_foobar_field
the code function is forward

def user_0106_populate_foobar_field_forward(apps, schema_editor):  # noqa: E501
    # ....
    # ...

Thoughts?

6/site-packages/django_squash/apps.py", line 2, in <module> from django.db.migrations.serializer import Serializer

i try use this package but this error is generated

my implementation use multi tenant, we use docker and it occurs as soon as we start after installing as indicated in the documentation

Traceback (most recent call last):
File "manage.py", line 30, in
execute_from_command_line(sys.argv)
File "/usr/local/lib/python3.6/site-packages/django/core/management/init.py", line 371, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python3.6/site-packages/django/core/management/init.py", line 347, in execute
django.setup()
File "/usr/local/lib/python3.6/site-packages/django/init.py", line 24, in setup
apps.populate(settings.INSTALLED_APPS)
File "/usr/local/lib/python3.6/site-packages/django/apps/registry.py", line 89, in populate
app_config = AppConfig.create(entry)
File "/usr/local/lib/python3.6/site-packages/django/apps/config.py", line 116, in create
mod = import_module(mod_path)
File "/usr/local/lib/python3.6/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 994, in _gcd_import
File "", line 971, in _find_and_load
File "", line 955, in _find_and_load_unlocked
File "", line 665, in _load_unlocked
File "", line 678, in exec_module
File "", line 219, in _call_with_frames_removed
File "/usr/local/lib/python3.6/site-packages/django_squash/apps.py", line 2, in
from django.db.migrations.serializer import Serializer
ImportError: cannot import name 'Serializer'

Installed database extensions are not kept in the squashed migrations

Hi!

This little library is absolutely amazing, it just changed my life 😅 thank you so much for open-sourcing it!

There has been one little hiccup in my workflow: database extension don't seem to be kept in the resulting squashed migrations. These are non-elidable operations that are not automatically detected by django as we just add them arbitrarily when we need them.

Example:

from django.contrib.postgres.operations import BtreeGistExtension

...

class Migration(migrations.Migration):
...
    operations = [
        BtreeGistExtension(),
        migrations.AlterField(...),
        ...
   ]

If I squash this migration, BtreeGistExtension() will have disappeared. This is fairly risky as it could get overlooked some day if we use the tool regularly!

Any ideas on how to fix this?

Thanks a lot again!

`rename_migrations` can fail if migration name does not start with 4 ints

we have a migration named like 02_some_name. probably bad form, but this package should be resilient to that. I can submit a PR to address this shortly.

NameError: name 'RunPython' is not defined

Hi! I installed the package and squashed my migrations using python manage.py squash_migrations.
I'm using Django 3.2.9 and Python 3.9.7
I faced two errors
One was an unexpected indent of @staticmethod function below and the other being RunPython is not defined.

This is the file : /env/lib/python3.9/site-packages/django/contrib/contenttypes/migrations/0003_squashed.py

import django.contrib.contenttypes.models
from django.db import migrations, models


@staticmethod
def noop(apps, schema_editor):
    return None


def add_legacy_name(apps, schema_editor):
    ContentType = apps.get_model("contenttypes", "ContentType")
    for ct in ContentType.objects.all():
        try:
            ct.name = apps.get_model(ct.app_label, ct.model)._meta.object_name
        except LookupError:
            ct.name = ct.model
        ct.save()


class Migration(migrations.Migration):

    replaces = [("contenttypes", "0001_initial"), ("contenttypes", "0002_remove_content_type_name")]

    initial = True

    dependencies = []

    operations = [
        migrations.CreateModel(
            name="ContentType",
            fields=[
                ("id", models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")),
                ("app_label", models.CharField(max_length=100)),
                ("model", models.CharField(max_length=100, verbose_name="python model class name")),
            ],
            options={
                "verbose_name": "content type",
                "verbose_name_plural": "content types",
                "db_table": "django_content_type",
                "unique_together": {("app_label", "model")},
            },
            managers=[
                ("objects", django.contrib.contenttypes.models.ContentTypeManager()),
            ],
        ),
        migrations.RunPython(
            code=RunPython.noop,
            reverse_code=add_legacy_name,
            hints={"model_name": "contenttype"},
            elidable=False,
        ),
    ]

Import issues

When using only from django.db import migrations and migrations.RunPython.noop, squasher names it RunPython.noop causing issues.