pyinstrument is very slow on django tests with docker about pyinstrument HOT 8 OPEN

Hi @patroqueeet, how are you invoking pyinstrument?

from pyinstrument.

sry for the slight delay :/

in docker I do docker-compose exec app pyinstrument ./manage.py test --keepdb services.rest.tests.test_views:AddCompanyTestCase.test_add_company -v 3./manage.py test --keepdb services -e functional -v 2 --failed

Generally spoken: intention is to identify what slows down my tests... including setUp() etc.

from pyinstrument.

Not sure what's going on here tbh... my tests in Docker seem to work just as normal. Are you able to make a minimal recreation?

from pyinstrument.

@joerick sure. will go and make a docker image for you to reproduce...

from pyinstrument.

Hey, tried with a simple model test on docker and found all working as expected. But the problems come on a large Py3 project.

Hence, closing this until I have some further reproducible truth.

Still: for the records https://stackoverflow.com/questions/62220875/profiling-python-in-a-docker-container-is-very-slow-cprofile-and-pyinstrument more people having this issue...

from pyinstrument.

Thanks for taking another look! If the problem occurs in cProfile too, it's probably at a lower level than we can fix.

A wild guess at the cause would have me thinking about the gettimeofday syscall in pyinstrument_cext - perhaps this is slower in some Docker containers. This article might add some weight to the idea...

from pyinstrument.

I had a thought on this - pyinstrument_cext could add a feature where, rather than calling gettimeofday on every trace event, it checks an internal variable that's being incremented by a thread who is running in a tight loop doing usleep. Perhaps this would be worse performance for systems with vDSO, but it would definitely improve things otherwise...

from pyinstrument.

hey, to be honest, I cannot judge if your wild guess is going in the right direction. if you provide me a branch/PR I can check it out and give you some feedback.

from pyinstrument.