Efficient offline generation for multiple sites about django-compressor HOT 5 CLOSED

Hmm, interesting, I'm open for suggestions, even if this is an edge case I wouldn't want to increase those 4 seconds.

from django-compressor.

Having thought about this a bit, I think it'd be best to just write a shell script / fabfile that calls the compress management command with the appropriate --settings value in a loop over all your settings.

from django-compressor.

Well, the issue is the overhead of starting 100s of processes of a
large django codebase; that is largely CPU-bound, so parallelizing
doesn't help much.

On Sun, Apr 10, 2011 at 12:01 PM, jezdez
[email protected]
wrote:

Having thought about this a bit, I think it'd be best to just write a shell script / fabfile that calls the compress management command with the appropriate --settings value in a loop over all your settings.

Reply to this email directly or view it on GitHub:
#15 (comment)

from django-compressor.

Hm, maby using django.core.management.setup_environ instead for each of the settings? What causes the actual slow down?

from django-compressor.

On Sun, Apr 10, 2011 at 2:16 PM, jezdez
[email protected]
wrote:

Hm, maby using django.core.management.setup_environ instead for each of the settings? What causes the actual slow down?

I agree this is an edge case... The codebase in question has 200K LOC
of python and uses several apps which have an autodiscover mechanism
so that accessing much of Django causes it all to be imported. On top
of that, there are some module-level initializations which are
relatively costly.

I don't think I fully understand what setup_environ does -- that is,
I'm not sure it's safe to run more than once in a process?

from django-compressor.