How to copy an 8TiB snapshot using Start-AzStorageBlobCopy? about azure-powershell HOT 5 OPEN

@martin-merdes

The failure is from storage server, so the size limitation is on server instead of Powershell.
You might can raise a help ticket on Azurite portal to get help to server team: https://learn.microsoft.com/en-us/azure/azure-portal/supportability/how-to-create-azure-support-request#go-to-help--support-from-the-global-header.
(Besides the above request trace, you might can also share the blob type.)

You can also try following workaround:

1. For transfer of big data , we would recommend using AzCopy, it will be more stable and fast in transfer big data, you can call AzCopy command in your PSH script: https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-v10
2. If you must use Powershell cmdlet, you can try "Copy-AzStorageBlob" cmdlet, this is Sync copy cmdlet and will copy data chunk by chunk. It will be very slow for 8TB blob. (Start-AzStorageBlobCopy is async copy, it will just send copy request, but won't wait for copy finish).

from azure-powershell.

@blueww
Thanks for the clarification that there is a size limitation. If this is by design then Start-AzStorageBlobCopy should return a clear error message rather than throwing an exception. Furthermore, this should be documented. I was searching for a limitation using AI before opening this issue and I did not find any documentation regarding a 4TB blob size limit.

from azure-powershell.

@martin-merdes

Thanks for the suggestion!
However, Powershell doesn't do size limitation check (and most of other parameter check), we depends on server to check that and will report the error from server. This can mostly accurate show the real reason of the failure, and avoid incorrect (out-of-date) limitation in Powershell code blocks customer from using the feature which server supported.

Currently Powershell report the error to customer and the error says "The source request body is too large and exceeds the maximum permissible limit.", which is the real reason for the error. Actually, this is also what Powershell can get from the error. Powershell can't guess the reason, or hide/parse information for this kind of error, since it will make the error not so accurate and might misleading customer.

For documentation of the size limitation, this should be updated in server API doc, instead of Powershell doc. You might can raise a help ticket on Azurite portal to get help from related team: https://learn.microsoft.com/en-us/azure/azure-portal/supportability/how-to-create-azure-support-request#go-to-help--support-from-the-global-header.

from azure-powershell.

Thanks for the clarification that this does not work.

from azure-powershell.

Hi @martin-merdes,

Thanks for the reply!
However, not sure what do you mean by "this does not work." Do you mean server team not responds to your help ticket, or your incident?

If so, this might be a little out of Powershell scope. However, if there's an incident on this, you might can share it to me offline, and I can help to see who should follow up this.

from azure-powershell.