Experiencing a lot of database overhead with forked jobs about resque HOT 3 CLOSED

Hi @SimantovYousoufov, yup, I have seen this too. There can be a fair bit of overhead creating new connections, etc. on every fork. To get around this we started using resque-multi-job-forks which lets us not have to fork for every job. Once enabled, we saw a massive performance increase with the job execution time and also a noticeable load decrease on our db.

Have you tried using that plugin?

from resque.

Yeah Rails does automatically reconnect after forking now. I don't think Resque can actually do anything to solve this other than forking less often. You can potentially try to configure Rails to not do this, but I'm not sure if that's an out of the box configuration option or if that's something you need to roll yourself. Ideally you'd only do that on resque jobs and not with your web server.

from resque.

Ya, and for good reason. There can a lot of oddness with trying to re-use connections across new forks.

Agreed with @PatrickTulskie, not sure there's much you can really do other than either not fork, or if you still want some of the benefits of forking, use the plugin I mentioned above.

from resque.