S3 dependency conflicts with AWS/S3 in Paperclips about backup HOT 21 CLOSED

Here's the actual error I'm getting:

http://pastie.org/private/nltfyoacujk1febmdtgviq

from backup.

Hmm. That sucks. I believe someone who forked the project is currently working on implementing another s3 variant. It isn't aws-s3 since we moved from aws-s3 a while ago.

There is a gem that handles a lot of cloud based platforms which supports streaming transfers rather than just hog up memory and push the whole thing from there. So until he has implemented the new gem (along with a MongoDB adapter) you're probably going to be stuck on using S3. However, you could also of course use the UNIX utility the Backup gem provides, which is completely separate from the Rails Environment.

I personally always use the unix utility just so i can manage all backups on my server from one location.

from backup.

Thanks for the heads up. What's the UNIX utility you're talking about? I'm not familiar with this?

from backup.

http://wiki.github.com/meskyanichi/backup/getting-started-unix

from backup.

Hey, we just merged the Mongo branch into Master, this includes the S3 Storage through Fog, rather than the S3 gem. You might want to check it out.

If you're using Bundler in your app, try adding:

gem 'backup', git => 'git://github.com/meskyanichi/backup.git'

And see if that works. : )

from backup.

Just tried the head version but got this:

ruby-1.9.2-p0@swom [Steve@SiMac3] backup $ rake backup:run trigger='mysql-selective-s3' --trace
(in /Users/Steve/work/backup)
** Invoke backup:run (first_time)
** Invoke backup:boot (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute backup:boot
** Execute backup:run
Running: mysql-selective-s3.
rake aborted!
undefined method before_backup_block' for #<Backup::Configuration::Base:0x00000102a595a8> /Users/Steve/.rvm/gems/ruby-1.9.2-p0@swom/bundler/gems/backup-9e5d128eeae0/lib/backup/adapters/base.rb:85:inhandle_before_backup'
/Users/Steve/.rvm/gems/ruby-1.9.2-p0@swom/bundler/gems/backup-9e5d128eeae0/lib/backup/adapters/base.rb:51:in initialize' /Users/Steve/.rvm/gems/ruby-1.9.2-p0@swom/bundler/gems/backup-9e5d128eeae0/lib/backup.rb:100:innew'
/Users/Steve/.rvm/gems/ruby-1.9.2-p0@swom/bundler/gems/backup-9e5d128eeae0/lib/backup.rb:100:in initialize_adapter' /Users/Steve/work/backup/lib/tasks/backup.rake:10:inblock (2 levels) in <top (required)>'
/Users/Steve/.rvm/gems/ruby-1.9.2-p0@swom/gems/rake-0.8.7/lib/rake.rb:636:in call' /Users/Steve/.rvm/gems/ruby-1.9.2-p0@swom/gems/rake-0.8.7/lib/rake.rb:636:inblock in execute'
/Users/Steve/.rvm/gems/ruby-1.9.2-p0@swom/gems/rake-0.8.7/lib/rake.rb:631:in each' /Users/Steve/.rvm/gems/ruby-1.9.2-p0@swom/gems/rake-0.8.7/lib/rake.rb:631:inexecute'
/Users/Steve/.rvm/gems/ruby-1.9.2-p0@swom/gems/rake-0.8.7/lib/rake.rb:597:in block in invoke_with_call_chain' /Users/Steve/.rvm/rubies/ruby-1.9.2-p0/lib/ruby/1.9.1/monitor.rb:201:inmon_synchronize'
/Users/Steve/.rvm/gems/ruby-1.9.2-p0@swom/gems/rake-0.8.7/lib/rake.rb:590:in invoke_with_call_chain' /Users/Steve/.rvm/gems/ruby-1.9.2-p0@swom/gems/rake-0.8.7/lib/rake.rb:583:ininvoke'
/Users/Steve/.rvm/gems/ruby-1.9.2-p0@swom/gems/rake-0.8.7/lib/rake.rb:2051:in invoke_task' /Users/Steve/.rvm/gems/ruby-1.9.2-p0@swom/gems/rake-0.8.7/lib/rake.rb:2029:inblock (2 levels) in top_level'
/Users/Steve/.rvm/gems/ruby-1.9.2-p0@swom/gems/rake-0.8.7/lib/rake.rb:2029:in each' /Users/Steve/.rvm/gems/ruby-1.9.2-p0@swom/gems/rake-0.8.7/lib/rake.rb:2029:inblock in top_level'
/Users/Steve/.rvm/gems/ruby-1.9.2-p0@swom/gems/rake-0.8.7/lib/rake.rb:2068:in standard_exception_handling' /Users/Steve/.rvm/gems/ruby-1.9.2-p0@swom/gems/rake-0.8.7/lib/rake.rb:2023:intop_level'
/Users/Steve/.rvm/gems/ruby-1.9.2-p0@swom/gems/rake-0.8.7/lib/rake.rb:2001:in block in run' /Users/Steve/.rvm/gems/ruby-1.9.2-p0@swom/gems/rake-0.8.7/lib/rake.rb:2068:instandard_exception_handling'
/Users/Steve/.rvm/gems/ruby-1.9.2-p0@swom/gems/rake-0.8.7/lib/rake.rb:1998:in run' /Users/Steve/.rvm/gems/ruby-1.9.2-p0@swom/gems/rake-0.8.7/bin/rake:31:in<top (required)>'
/Users/Steve/.rvm/gems/ruby-1.9.2-p0@swom/bin/rake:19:in load' /Users/Steve/.rvm/gems/ruby-1.9.2-p0@swom/bin/rake:19:in

from backup.

Just to follow up I created a trivial new Rails 3 app with only backup gem (at HEAD) and it's broken out of the box :(

http://pastie.org/private/dz5lzyljbzmyerkoxjlya

Any ideas what the undefined method `before_backup_block' means?

Lots seems to have changed since I last looked at the codebase...

Steve

from backup.

Hi.

Sorry m8 I've been extremely busy lately, and still am.
This is indeed a huge issue at the moment as it breaks the tool apparently.

I also already noticed it was cause of this

before_backup_block

But I haven't had the time to investigate. We've had a lot of contributors hence why a lot has changed. I've also implemented Rails 3 compatibility, however, that was not what broke it.

Once I find time I'll go check this out asap. Sorry for the trouble :x

from backup.

I actually just did some investigation and it got added in this commit:

73994cf5971e9cf89a6fd3f3349a5e8a287836f4

Also now it's a bit more clear as to what it exactly does and I'm trying to see why this dies when invoking a rake task.

from backup.

Well. this is very interesting. Half of that commit does not even exist in HEAD.

The first file has been merged, but if you look at the second file, and my clone of head:

http://cl.ly/59383531582140e26c00

The Configuration isn't there..? Strange.

from backup.

OK I have applied a patch and with it when I run my rake task, it I don't get the undefined method exception anymore.

3be0795

Try to clone from HEAD again in your Gemfile and see if it helps! : )

from backup.

Thanks will do! As always you're doing a great job!

from backup.

Thanks m8! Hope it works!

from backup.

Yup HEAD works without the error now. That's great. Any ideas when this will be incorporated into the next gem?

from backup.

Since we moved S3 to FOG, and since FOG supports CloudFiles as well. We're considering dropping CloudFiles dependency in favor of the already implemented Fog.

There are still some commits left from another branch I think I want merged in first. Also the current HEAD needs to be tested a bit to see if stuff is still functioning properly before I release a gem.

However, since it now also works for you that's great and means it's probably working with everything again because that appeared to be the only issue.

Ill see if i can push a 0.0.1+ out fairly soon.

from backup.

Cool. Until then I'll lock down to the current commit in my Gemfile. Look forward to it!

p.s. I use backup gem for backing up selective tables, but I use LVM snapshots to backup the entire mysql db from a slave. This is because my db is 100GB, and mysqldump would take way too long. It uses a fairly simple ruby script which issues shell commands. I wonder if there is any case of incorporating this into the backup gem?

Steve

from backup.

Hi,

LVM = Linux Virtual Machine?

How does this take snapshots? I assume you can't be on the same VM you're taking the snapshot from? How does this works? :)

from backup.

LVM = Logical Volume Manager
It is a linux filesystem that is widely used and enables you to take a snapshot of a hard drive while it is still running. I happen to use this with Xen but this is not required.

There's lots of info if you google LVM snapshot backup.

It lets you backup entire volumes while the system is still running.

Steve

from backup.

I basically use Xen together with LVM to allow you to remotely backup one server onto another server. You can backup huge mysql dbs this way - it wouldn't be practical to backup 100GB of mysql db using mysqldump :)

from backup.

I just pushed out 2.4.1 which of course contains the undefined method issue patch. This also adds Dropbox Storage support. : )

Thanks for the info on LVM, I will have to take a look at that some time.

from backup.

Closed by dd27459

from backup.