Hey there,

I use beanstalk in rails and within an external daemon which processes jobs in a queue. I use

job.stats["tube"] 

to determine which tube a job came from. Is this a valid operation?

When used with 'bundler/setup', memory usage slowly grows and isn't released.

Running the code below:

• without accessing job.stats["tube"] and excluding 'bundler/setup': 15.7 MB
• including accessing job.stats["tube"] and excluding 'bundler/setup': 18.8 MB
• without accessing job.stats["tube"] and including 'bundler/setup': 23.9 MB
• including accessing job.stats["tube"] and including 'bundler/setup': 567MB

require "bundler/setup"
require 'beanstalk-client'                                                                                                                

beanstalk = Beanstalk::Pool.new(["localhost:11300"])

beanstalk.use "ping"

# Load up beanstalkd

50000.times {|i| beanstalk.put("PING! - #{i}")}

# Grab out entries and delete them.

beanstalk.watch("ping")
job = nil

loop do
  begin
    job = beanstalk.reserve(1)
  rescue Beanstalk::TimedOut
    puts "So bored."
    retry
  end

  # The following access to job.stats["tube"] causes the issue - but only when used with bundler.
  puts job.stats["tube"]
  puts "Deleting job : #{job.body}"
  job.delete
end

Anyone else seen this?

Currently, if a beanstalkd dies, the client eagerly tries to reconnect upon the next request. If the server is offline, this will cause unnecessary delays while these connection attempts timeout. In this case it would be preferable to remove the failed server from the pool until an administrator can attend to the matter.

I've implemented something like this in my branch 2fbf670

Let me know what you think. If you'd like I can prepare a pull request

Things like "宮島".size give the wrong number in ruby 1.9. We want the number of bytes, not the number of unicode codepoints.

Use "宮島".bytesize instead.

Client code assumes that job body is a string with no explicit checks:

def put(body, pri=65536, delay=0, ttr=120)
  interact("put #{pri} #{delay} #{ttr} #{body.size}\r\n#{body}\r\n",
           %w(INSERTED BURIED))[0].to_i
end

So, when you try to put Numeric job:
> beanstalk.put(1)

you get lockup because "#{1}" == '1' and 1.size == 8. Server waits for 7 more characters, while client thinks everything is sent.

Probably, better to raise exception when body is not a String, or to do body.to_s before putting.

I run into a strange issue trying to delete a buried job using Beanstalk::Job instance. See the snippet below...

require 'beanstalk-client'

beanstalk = Beanstalk::Pool.new(['localhost:11300'], 'tube')

# add message
beanstalk.put('0')

# bury message
job = beanstalk.reserve
job.bury

p job = beanstalk.peek_buried
job.delete
p beanstalk.peek_buried

Job is NOT deleted!

(job server=localhost:11300 id=20397 size=1)
(job server=localhost:11300 id=20397 size=1)

When you use delete method in Connection class it magically deletes the job.

beanstalk.last_conn.delete(job.id)

Any idea why it is happening?

Hi,
i'm using beanstalkd in one of my rails projects, i'm having a problem with threads.

I've many threads which connect to a tube in order to empty the queue and only one process filling the queue. Some times i receive this error: Attempt to unlock a mutex which is locked by another thread.

This is the exception backtrace:

/mnt/apps/pulsarplatform/shared/bundle/ruby/1.9.1/gems/beanstalk-client-1.1.1/lib/beanstalk-client/connection.rb:199:in `unlock'
/mnt/apps/pulsarplatform/shared/bundle/ruby/1.9.1/gems/beanstalk-client-1.1.1/lib/beanstalk-client/connection.rb:199:in `ensure in interact'
/mnt/apps/pulsarplatform/shared/bundle/ruby/1.9.1/gems/beanstalk-client-1.1.1/lib/beanstalk-client/connection.rb:199:in `interact'
/mnt/apps/pulsarplatform/shared/bundle/ruby/1.9.1/gems/beanstalk-client-1.1.1/lib/beanstalk-client/connection.rb:173:in `stats_tube'
/mnt/apps/pulsarplatform/shared/bundle/ruby/1.9.1/gems/beanstalk-client-1.1.1/lib/beanstalk-client/connection.rb:394:in `call_wrap'
/mnt/apps/pulsarplatform/shared/bundle/ruby/1.9.1/gems/beanstalk-client-1.1.1/lib/beanstalk-client/connection.rb:424:in `block (2 levels) in send_to_all_conns'
/mnt/apps/pulsarplatform/shared/bundle/ruby/1.9.1/gems/beanstalk-client-1.1.1/lib/beanstalk-client/connection.rb:436:in `block in map_hash'
/mnt/apps/pulsarplatform/shared/bundle/ruby/1.9.1/gems/beanstalk-client-1.1.1/lib/beanstalk-client/connection.rb:436:in `each'
/mnt/apps/pulsarplatform/shared/bundle/ruby/1.9.1/gems/beanstalk-client-1.1.1/lib/beanstalk-client/connection.rb:436:in `map'
/mnt/apps/pulsarplatform/shared/bundle/ruby/1.9.1/gems/beanstalk-client-1.1.1/lib/beanstalk-client/connection.rb:436:in `map_hash'
/mnt/apps/pulsarplatform/shared/bundle/ruby/1.9.1/gems/beanstalk-client-1.1.1/lib/beanstalk-client/connection.rb:424:in `block in send_to_all_conns'
/mnt/apps/pulsarplatform/shared/bundle/ruby/1.9.1/gems/beanstalk-client-1.1.1/lib/beanstalk-client/connection.rb:403:in `retry_wrap'
/mnt/apps/pulsarplatform/shared/bundle/ruby/1.9.1/gems/beanstalk-client-1.1.1/lib/beanstalk-client/connection.rb:424:in `send_to_all_conns'
/mnt/apps/pulsarplatform/shared/bundle/ruby/1.9.1/gems/beanstalk-client-1.1.1/lib/beanstalk-client/connection.rb:339:in `raw_stats_tube'
/mnt/apps/pulsarplatform/shared/bundle/ruby/1.9.1/gems/beanstalk-client-1.1.1/lib/beanstalk-client/connection.rb:343:in `stats_tube'

Have some one of you ever had some similar issue?

Tnx

Duccio

I'm running into a problem that occurs when using beanstalk-client in combination with threads on Ruby 1.9.1. It doesn't occur with Ruby 1.8.7. The script below has the problem, but the problem doesn't occur every time the script runs. It seems to occur about 50% of the time. It occurs with beanstalkd 1.3 and 1.4.4.

The problem is that the script gets stuck and never hits the "End join" line.

require 'rubygems'
require 'beanstalk-client'

$beanstalk = Beanstalk::Pool.new(['localhost:11300'])
$thread_num = 100

def enque(n)
#puts "before enque " + n.to_s
$beanstalk.put('hello world' + n.to_s)
#puts "after enque " + n.to_s
end

threads = []

for i in 1 .. $thread_num
threads << Thread::start {enque i}
end

puts "Start join"

for i in 1 .. $thread_num
threads[i-1].join
end

puts "End join"

Hi,
The github repository includes the following very useful fix for Ruby 1.9 unicode:
"Count number of bytes, not unicode codepoints."
e80c5da

Can you bump this gem's version to 1.1.1 so it will include this fix?
Thanks!