procedures to optimize problematic filenames about uppy HOT 10 CLOSED

I think we should be encouraging people to do this on the server rather than on the client.

from uppy.

this point is also important because of security, see #86 (comment)

3.All the control characters and Unicode ones should be removed from the filenames and their extensions without any exception. Also, the special characters such as “;”, “:”, “>”, “<”, “/” ,”\”, additional “.”, “*”, “%”, “$”, and so on should be discarded as well. If it is applicable and there is no need to have Unicode characters, it is highly recommended to only accept Alpha-Numeric characters and only 1 dot as an input for the file name and the extension; in which the file name and also the extension should not be empty at all (regular expression: [a-zA-Z0-9]{1,200}.[a-zA-Z0-9]{1,10}).
4.Limit the filename length. For instance, the maximum length of the name of a file plus its extension should be less than 255 characters (without any directory) in an NTFS partition.

from uppy.

@goto-bus-stop’s suggestion:

onBeforeFileAdded (file) {
  file.name = cleanup(file.name)
  return Promise.resolve()
}

I think outsourcing cleanup procedure to a developer using Uppy rather than trying to invent something generic ourselves is a good idea. I wonder if we should return the modified file from the promise in this case?

from uppy.

@arturi Maybe Uppy should offer a cleanup function or an option that allows developer to run the standard procedure (mentioned above) instead of re-inventing it, or they can choose to do a custom one if they like?

from uppy.

Added to backlog, closing for now. Thank you!

from uppy.

Having this problem currently with Uppy. User put the character "#" in the filename. @goto-bus-stop suggests this is a server sanitation issue, however '#' is not valid in filenames on the web and are being sent to the server with everything after the # stripped. (e.g. "file#name.jpg" gets sent to server as "file" as # is being confused as an anchor tag) I think there is a valid use-case for having a default sanitation of filenames, as well as an override, on the client side.

from uppy.

@sarah-sterchele I think that sounds like a bug to me … IMO validation and correction should happen server-side, but to do that, of course the full file name does need to arrive, including # and other characters. We might be missing an encodeURIComponent() call somewhere. Could you share the plugins + options you are using (ideally just a code snippet)?

from uppy.

@goto-bus-stop Using Tus plugin. We have dynamic upload routes so the route is built on each upload. encodeUri() does not escape '#' as it is a reserved character for a uri. So the onus would be on the developer to replace, remove, or manually escape it. (same as the character '?')

Sample code snipped in the onBeforeUpload callback, where we modify the path:

onBeforeUpload: (files) => {
        const updatedFiles = Object.assign({}, files) as any;
        Object.keys(updatedFiles).forEach((fileId) => {
          const filename = updatedFiles[fileId].name;
          const path = HttpHelper.concatPath(this.endpoint, "file", filename);

          const uploadInfo = {
            endpoint: encodeURI(path),
            headers: setHeaders(),
            chunkSize: Infinity,
          };
          
          if (this.plugins.includes(UppyPlugin.Tus)) {         
            updatedFiles[fileId].tus = uploadInfo;
          } 
        });

        return updatedFiles;

options simply being

  autoProceed: false,
  restrictions: {
    maxFileSize: UppyClient.MAX_FILE_SIZE,
    maxNumberOfFiles: null,
    minNumberOfFiles: null,
  },

I don't think it's a bug, and since we are overriding the path, in our instance we can remove any unwanted characters. But I was thinking it would be convenient if the filename could be transformed without having to go into a callback such as onBeforeUpload. I am just seconding the idea that the original poster made that problematic filenames can become problematic before they hit the serverside.

from uppy.

encodeURI() does not encode #, but encodeURIComponent does. It's true that the filename has to be escaped appropriately before being put into a URL, just like any user variable.

from uppy.

This issue should be open. Even with a server side validation there is a bug with # character.

from uppy.