@tus/s3-store: add option to disable object tags about tus-node-server HOT 10 CLOSED

Nothing will break if I don't use extensions for expiration and do this patch?

diff --git a/dist/index.js b/dist/index.js
index cd35efbf3298386fa77a75bae10f43ac92e8b4c8..8e72d9e1369fdac7d3616824a1ee1a816f9296d9 100644
--- a/dist/index.js
+++ b/dist/index.js
@@ -106,7 +106,6 @@ class S3Store extends server_1.DataStore {
             Bucket: this.bucket,
             Key: this.infoKey(upload.id),
             Body: JSON.stringify(upload),
-            Tagging: `Tus-Completed=false`,
             Metadata: {
                 'upload-id': uploadId,
                 'tus-version': server_2.TUS_RESUMABLE,
@@ -120,7 +119,6 @@ class S3Store extends server_1.DataStore {
             Bucket: this.bucket,
             Key: this.infoKey(upload.id),
             Body: JSON.stringify(upload),
-            Tagging: `Tus-Completed=true`,
             Metadata: {
                 'upload-id': uploadId,
                 'tus-version': server_2.TUS_RESUMABLE,
@@ -184,7 +182,6 @@ class S3Store extends server_1.DataStore {
             Bucket: this.bucket,
             Key: this.partKey(id, true),
             Body: readStream,
-            Tagging: 'Tus-Completed=false',
         });
         log(`[${id}] finished uploading incomplete part`);
         return data.ETag;

from tus-node-server.

@rougsig no, nothing will break, since tagging is only a useful feature for cleaning up assets using lifecycle rules

Will raise a PR sometime soon to address this

from tus-node-server.

can you try using the onResponseError function on the Server to inspect the error you are getting, that might help know the cause of this error

const tusServer = new Server({
  path: "/files",
  datastore: s3Store,
  onResponseError: (req, res, err) => {
      console.error(err)
  }
});

from tus-node-server.

This is the error log:

onResponseError
InvalidArgument: Unsupported header 'x-amz-tagging' received for this API call.
at throwDefaultError (/app/node_modules/@smithy/smithy-client/dist-cjs/default-error-handler.js:8:22)
at /app/node_modules/@smithy/smithy-client/dist-cjs/default-error-handler.js:18:39
at de_PutObjectCommandError (/app/node_modules/@aws-sdk/client-s3/dist-cjs/protocols/Aws_restXml.js:4841:12)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at async /app/node_modules/@smithy/middleware-serde/dist-cjs/deserializerMiddleware.js:7:24
at async /app/node_modules/@aws-sdk/middleware-signing/dist-cjs/awsAuthMiddleware.js:30:20
at async /app/node_modules/@smithy/middleware-retry/dist-cjs/retryMiddleware.js:31:46
at async /app/node_modules/@aws-sdk/middleware-flexible-checksums/dist-cjs/flexibleChecksumsMiddleware.js:63:20
at async /app/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/region-redirect-endpoint-middleware.js:14:24
at async /app/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/region-redirect-middleware.js:9:20 {
'$fault': 'client',
'$metadata': {
httpStatusCode: 400,
requestId: '1289c689d96015bc',
extendedRequestId: 'adcNu/WuvbnZvUHdPbpw=',
cfId: undefined,
attempts: 1,
totalRetryDelay: 0
},
Code: 'InvalidArgument'
}

I did a little googling and it seems that backblaze b2 (which is my storage provider) has issues with the x-amz-tagging header: https://www.backblaze.com/docs/cloud-storage-s3-compatible-api#get-object-tagging

Any way to remove that header from the COPY and PUT object calls?

from tus-node-server.

mmmh, I see - it seems that this issue is caused by the expiration extension which uses tags.
I'm wondering if we should disable tagging if the expiration extension is not used

https://github.com/tus/tus-node-server/blob/main/packages/s3-store/index.ts#L107

We don't provide a way to turn off tagging, currently, we tested this on AWS S3 / Minio which both supports tagging hence we didn't realise that tagging is not always implemented in all stores

Probably is something we should provide an option in the s3 store

something like:

const s3Store = new S3Store({
  useTags: false, // <-- new option
  s3ClientConfig: {
    ...
  },
});

alternatively we might to do the otherway around where we have to specify a tag name for opting-in on object tags:

const s3Store = new S3Store({
  tagName: 'Tus-Completed', // <-- new option
  s3ClientConfig: {
    ...
  },
});

Thoughts @Murderlon

from tus-node-server.

In the meantime I migrated to using wasabi which just ignores the x-amz-tagging header. Out of curiosity I was also considering using DigitalOcean Spaces (S3 compatible) and they also ignore the x-amz-tagging header. So it seems only AWS S3 and perhaps a few others actually make use of that tag.

I believe your suggested approach where the tag can be disabled or opt-in would be ideal.

from tus-node-server.

if they ignore it and do not throw an error is fine, i was surprised that backblaze b2 errors out

from tus-node-server.

Indeed it's a bit odd that Backblaze throws an error. But an option to disable tags makes sense. The boolean options seems more logical for this use case 👍

from tus-node-server.

Wanted too.

from tus-node-server.

PR is up 👆

from tus-node-server.