rsync issues about cloudfuse HOT 9 OPEN

As mentioned in a similar-looking issue, I've seen Transport endpoint errors myself, and it turned out it was due to attempting to transfer >5gb files.

See here for how to segment >5gb files for proper storage in cloud files: http://www.rackspace.com/knowledge_center/index.php/Does_Cloud_Files_support_large_file_transfer

from cloudfuse.

Due to cloudfuse instability and lack of support for large files, I've
switched to a tool called "st" for transferring.
Uploading via snet goes extremely fast, I believe it was around 40MB/s
(megabyte). So, regarding cloud to cloud file transfer, no complaints here.

Jonathan Langevin

On Wed, Aug 17, 2011 at 2:02 PM, danielb2 <
[email protected]>wrote:

what are your guys transfer speeds on this? I'm using rsync on a local
rackspace machine to transfer files to a cloudfuse mount. It took me 1.5
minutes to transfer 200K of files. That makes it unusable to me. I didn't
see any errors, so this is a bit OT, but I'm curious of what you've seen.

Reply to this email directly or view it on GitHub:
#18 (comment)

from cloudfuse.

I've just started looking a this so I'm unfamiliar with snet. Do you have a
link to this "st" tool?

thanks! :)

-d

from cloudfuse.

I'll second that. I have had better luck with duplicity, but still the instability is maddening.

One of the shortcomings of Rackspace IMHO is how server size is tied to disk space. For the project I am working on at the moment, only about 5-10% of the data needs to be on block storage and the other 90% (ideally) would be stashed on cloud files.

I haven't heard of 'st' either... link?

from cloudfuse.

looks like S3 is cheaper anyway, so I'm looking at that now.

from cloudfuse.

Not so sure about that and I am no Amazon fan (after years of beating my head against their wall of unecessary complications and hideous documentation) ;-)

In my case, the bandwidth cost between my cloud servers and AWS would be prohibitive as the servers will definitely be staying with Rackspace.

BTW. The term "snet" refers to rackspace's "service net". In other words, the 10.x.x.x network between servers and cloud files. The beauty is, they don't charge for bandwidth on the service net, but there is a speed cap based on the size of the servers involved.

from cloudfuse.

On 1gb instance, snet was faster for me than pub.

Driving now, I'll send info on st tool shortly.

• Jon Langevin -- sent from my Android phone
  On Aug 23, 2011 6:35 PM, "elescondite" <
  [email protected]>
  wrote:

  Not so sure about that and I am no Amazon fan (after years of beating my
  head against their wall of unecessary complications and hideous
  documentation) ;-)

  In my case, the bandwidth cost between my cloud servers and AWS would be
  prohibitive as the servers will definitely be staying with Rackspace.

  BTW. The term "snet" refers to rackspace's "service net". In other words,
  the 10.x.x.x network between servers and cloud files. The beauty is, they
  don't charge for bandwidth on the service net, but there is a speed cap
  based on the size of the servers involved.

  --
  Reply to this email directly or view it on GitHub:
  #18 (comment)

from cloudfuse.

Sorry, meant to post about the 'st' tool.

How-to is here:
http://www.rackspace.com/knowledge_center/index.php/Does_Cloud_Files_support_large_file_transfer

Read carefully, as they have links within the text (like "download here") for you to fetch the st tool, which is actually a script from a larger library.

Works great for me.

from cloudfuse.

I used cloud fuse to mount the cloud and I have issues with rsync. Rsync doesnot work at all, while a plain cp works very well. Any thoughts on that ?

rsync just hangs and errors out at last ..

from cloudfuse.