Code Monkey home page Code Monkey logo

capistrano3-delayed-job's Introduction

Capistrano::DelayedJob Gem Version

Delayed Job support for Capistrano 3.x

Installation

Add this line to your application's Gemfile:

gem 'capistrano3-delayed-job', '~> 1.0'
gem 'capistrano'

And then execute:

$ bundle

Or install it yourself as:

$ gem install capistrano3-delayed-job

Usage

Require in Capfile to use the default task:

require 'capistrano/delayed_job'

You will get the following tasks

cap delayed_job:restart  # Restart the delayed_job process
cap delayed_job:start    # Start the delayed_job process
cap delayed_job:status   # Status of the delayed_job process
cap delayed_job:stop     # Stop the delayed_job process

Configurable options (copy into deploy.rb), shown here with examples:

# Number of delayed_job workers
# default value: 1
set :delayed_job_workers, 2

# String to be prefixed to worker process names
# This feature allows a prefix name to be placed in front of the process.
# For example:  reports/delayed_job.0  instead of just delayed_job.0
set :delayed_job_prefix, 'reports'

# Delayed_job queue or queues
# Set the --queue or --queues option to work from a particular queue.
# default value: nil
set :delayed_job_queues, ['mailer','tracking']

# Specify different pools
# You can use this option multiple times to start different numbers of workers
# for different queues.
# NOTE: When using delayed_job_pools, the settings for delayed_job_workers and
# delayed_job_queues are ignored.
# default value: nil
#
# Single pool of 3 workers looking at all queues: (when alone, '*' is a
# special case meaning any queue)
# set :delayed_job_pools, { '*' => 3 }
# set :delayed_job_pools, { '' => 3 }
# set :delayed_job_pools, { nil => 3 }
#
# Several queues, some with their own dedicated pools: (symbol keys will be
# converted to strings)
# set :delayed_job_pools, {
#     :mailer => 2,    # 2 workers looking only at the 'mailer' queue
#     :tracking => 1,  # 1 worker exclusively for the 'tracking' queue
#     :* => 2          # 2 on any queue (including 'mailer' and 'tracking')
# }
#
# Several workers each handling one or more queues:
# set :delayed_job_pools, {
#     'high_priority' => 1,                # one just for the important stuff
#     'high_priority,*' => 1,              # never blocked by low_priority jobs
#     'high_priority,*,low_priority' => 1, # works on whatever is available
#     '*,low_priority' => 1,  # high_priority doesn't starve the little guys
#   }

# If you have several servers handling Delayed Jobs and you want to configure
# different pools per server, you can define delayed_job_pools_per_server:
#
# set :delayed_job_pools_per_server, {
#   'server11-prod' => {
#     'default,emails' => 3,
#     'loud_notifications' => 1,
#     'silent_notifications' => 1,
#   },
#   'server12-prod' => {
#     'default' => 2
#   }
# }

# Server names (server11-prod, server12-prod) in :delayed_job_pools_per_server
# must match the hostnames on Delayed Job servers. You can verify it by running
# `hostname` on your servers.

# If you use :delayed_job_pools_per_server, :delayed_job_pools will be ignored.

# Identification is assigned in order 0..3.
# Note that the '*' in this case is actually a queue with that name and does
# not mean any queue as it is not used alone, but alongside other queues.

# Set the roles where the delayed_job process should be started
# default value: :app
set :delayed_job_roles, [:app, :background]

# Set the location of the delayed_job executable
# Can be relative to the release_path or absolute
# default value: 'bin'
# set :delayed_job_bin_path, 'script' # for rails 3.x

# To pass the `-m` option to the delayed_job executable which will cause each
# worker to be monitored when daemonized.
# set :delayed_job_monitor, true

### Set the location of the delayed_job.log logfile
# default value: "#{Rails.root}/log" or "#{Dir.pwd}/log"
# set :delayed_log_dir, 'path_to_log_dir'

### Set the location of the delayed_job pid file(s)
# default value: "#{Rails.root}/tmp/pids" or "#{Dir.pwd}/tmp/pids"
# set :delayed_job_pid_dir, 'path_to_pid_dir'

### Set the options to be passed along to daemons when starting/stopping/restarting delayed_job workers
# options supported by daemons can be found at https://github.com/thuehlinger/daemons/blob/master/lib/daemons/cmdline.rb
# set :delayed_job_daemon_opts, ["no_wait", "shush"]

It also conditionally adds the following hook:

  if Rake::Task.task_defined?('deploy:published')
    after 'deploy:published', 'delayed_job:default'
  end

Where the delayed_job:default is defined as:

  task :default do
    invoke 'delayed_job:restart' if fetch(:delayed_job_default_hooks, true)
  end

Thus you can disable default hook by setting delayed_job_default_hooks to false. (It still gets defined, but will not do anything when called.)

set :delayed_job_default_hooks, false

Following setting is recommended to avoid stop/restart problem. See Issue #16 or PR #22 for more detail.

set :linked_dirs, %w(tmp/pids)

# or

set :delayed_job_pid_dir, '/tmp'

Contributing

  1. Fork it
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create new Pull Request

Credits

Thank you contributors!

capistrano3-delayed-job is maintained Rob Biedenharn and Logo of Agile Consulting LLC

Originally developed by Platanus.

License

capistrano3-delayed-job is © 2016 Platanus SpA. It is free software and may be redistributed under the terms specified in the LICENSE file.

capistrano3-delayed-job's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

capistrano3-delayed-job's Issues

Pool worker configuration example is wrong

The deploy.rb example shown in the README which refers to workers handling one or more queues is incorrect, and causes a misconfiguration which may result in some queues not being handled.

Specifically this part:

# Several workers each handling one or more queues:
# set :delayed_job_pools, {
#     'high_priority' => 1,                # one just for the important stuff
#     'high_priority,*' => 1,              # never blocked by low_priority jobs
#     'high_priority,*,low_priority' => 1, # works on whatever is available
#     '*,low_priority' => 1,  # high_priority doesn't starve the little guys
#   }

Delayed::Command treats the * as no queues (an empty array) when it's passed alone, but treats it as a queue named * when passed with other named queues.

This is obviously not what's intended by the documentation, and if the user follows this example they will end up with only the queues high_priority, low_priority and * being handled, but not anything else.

The solution would be to add a lonely * key to the example and remove the wildcard from the other keys. Haven't made a PR because I don't really know what example would fit here.

Delayed Job failed: "cannot load translations" from capistrano fetching old files fetching from old releases

I am using delayed-job gem and capistrano3-delayed-job to manage my jobs while deploying code via capistrano for my Rails 4 application.

The delayed jobs are failing and i'm getting "can not load translations from /home/bot/getsavvy-api/releases/20170419152135/config/locales/devise.en.yml: #<Errno::ENOENT: No such file or directory @ rb_sysopen" error on production after 2-3 days of deployment.

It looks like its fetching files from old releases which are not present on server anymore.

I have following in my deploy.rb file

set :delayed_job_workers, 4 set :linked_dirs, fetch(:linked_dirs, []).push('log', 'tmp/pids', 'tmp/cache', 'tmp/sockets', 'public/system')

Any idea how to resolve this issue?

Starting delayed job in production environment not working

I'm using capistrano3 and this gem for delayed job in production. When I deployed to production environment using cap production deploy here's the screenshot of the capistrano log

screenshot 2018-01-10 16 57 01

it should restart the delayed job worker but it's not running. The workaround I did was to run

RAILS_ENV=production bin/delayed_job restart

in the root directory of my current application.

Any solutions with this?

Restart workers in parallel?

Hi, we’re using DelayedJob with over 100 workers running on 2 servers. This deploy script works well. Thanks for that.

At the end of the deploy this Capistrano task restarts each worker one by one and waits up to 20 seconds for one worker to finish. With a lot of workers this can take a very long time. This is the command we’re running on one of the servers:

/usr/bin/env bundle exec bin/delayed_job -n 1 --pid-dir=/tmp --pool='default:60' restart
delayed_job.0: trying to stop process with pid 804659 sending TERM and waiting 20s ...
delayed_job.0: trying to stop process with pid 804659 forcefully :( sending KILL and waiting 20s ...
delayed_job.0: process with pid 804659 successfully stopped.
delayed_job.0: process with pid 807701 started.
delayed_job.1: trying to stop process with pid 804881 sending TERM and waiting 20s ...
delayed_job.1: process with pid 804881 successfully stopped.
delayed_job.1: process with pid 807939 started.
delayed_job.2: trying to stop process with pid 805389 sending TERM and waiting 20s ...
delayed_job.2: process with pid 805389 successfully stopped.
delayed_job.2: process with pid 808187 started.
delayed_job.3: trying to stop process with pid 807020 sending TERM and waiting 20s ...
delayed_job.3: process with pid 807020 successfully stopped.
delayed_job.3: process with pid 808206 started.
delayed_job.4: trying to stop process with pid 807168 sending TERM and waiting 20s ...

I was wondering if this could be run in parallel to speed things up? When running many workers this takes a very long time to complete (INFO [addc3a05] Finished in 394.670 seconds with exit status 0 (successful).) and it doesn't make too much sense to wait for a worker to have successfully restarted in order to execute the restart order for the next one.

In my opinion restart should be done as fast as possible because old running workers are executing an older version of the code where incompatible changes can have been made (migrations for instance).

Problem with permissions on the binary

I taked alot of time to get rid of it. On deploy capistrano was giving this error:

SSHKit::Command::Failed: bundle exit status: 126
bundle stdout: bundler: not executable: bin/delayed_job
bundle stderr: Nothing written

I'm really a newbie and this put me down in deploying.
The solution was to add a chmod to the deploy:

task :chmod_delayed_jobs do on roles(:all) do execute "chmod +x #{current_path}/bin/delayed_job " end end after "passenger:restart", "chmod_delayed_jobs"

image

Then it started working. I think that this gem needs to get around this problem. Could you help to resolve this issue inside the gem, without needing to change the permissions on a task?
I'm using DigitalOcean Ubuntu 16.04 and following the GoRails tutorial.

Sorry, I dont know to style the ruby code to work here.

You need to add gem 'daemons' to your Gemfile

After setting up as written in README and running cap deploy I got the following error:

You need to add gem 'daemons' to your Gemfile if you wish to use it

Naturally, adding it solved the error.

It would be great if this gem is listed as a dependency.

Deploy errors with "Aborted" on RHEL/CentOS

Hope you don't mind me adding this here (can be closed immediately). I spent a fair bit of time trying to find what was causing this so putting this here as it's probably one of the first places people will look...

Cap deploys fail at when restarting delayed job - you'll get something like:

-> RAILS_ENV="production" bundle exec bin/delayed_job -n 1 --pool='mailer:1' --pool='*:1' restart
delayed_job.0: trying to stop process with pid 12584...
delayed_job.0: process with pid 12584 successfully stopped.
delayed_job.0: process with pid 12883 started.
delayed_job.1: trying to stop process with pid 12590...
delayed_job.1: process with pid 12590 successfully stopped.
delayed_job.1: process with pid 12900 started.
Aborted

This is due to an issue with ffi bundle update ffi so you have version >= 1.9.25 will fix it - related link: ffi/ffi#621

Error during deploy

Greetings!

Not many issues here, but I'm hoping someone might be able to help out.

I'm getting this error after installing the gem:


/usr/bin/env sudo service delayed_job_(appname)_production restart

 DEBUG [ffd7835a] 	sudo

 DEBUG [ffd7835a] 	: 

 DEBUG [ffd7835a] 	no tty present and no askpass program specified

 DEBUG [ffd7835a] 	

Wondering if anyone has any clue why that might happen? I'm not using PTY, I've got my deploy user set up to work through an SSH key.

dj worker configuration lost after reboot

Hi,

I'm using the pool configuration options

set :delayed_job_pools, {
    'high_priority' => 1,      # one just for the important stuff
    '*' => 3	#three workers working any queue
}

this works great - but it is lost if I reboot the server.

I'm currently using https://github.com/javan/whenever to manage my crontab, and I can set

every :reboot do
  envcommand "bin/delayed_job  -n 1 --pool='high_priority:1' --pool='*:3' restart"
end

but that's duplication of the deploy config.

it would be great if capistrano could either add something to the crontab directly, or write a script that I could then call.

every :reboot do
  envcommand "command_to_run_whatever_capistrano_set_up_for_DJ"
end

alternatively - if there is already support for this kind of thing - can you add documentation?

many thanks!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.