mayth / go-simple-upload-server Goto Github PK

Hello,

Thanks for your work, it's a pleasure to play around with.

I wanted to flag an issue with CORS support.

When the browser sends a POST request to a server, it first sends an OPTIONS request to verify that the servers returns a valid Access-Control-Allow-Origin header.

(Issue reproduced from both Chrome and Firefox)

From Stack Overflow:

OPTIONS requests are what we call pre-flight requests in Cross-origin resource sharing (CORS).

They are necessary when you're making requests across different origins in specific situations.

This pre-flight request is made by some browsers as a safety measure to ensure that the request being done is trusted by the server. Meaning the server understands that the method, origin and headers being sent on the request are safe to act upon.

Your server should not ignore but handle these requests whenever you're attempting to do cross origin requests.

To fully support CORS, the post suggests the following:

A way to handle these to get comfortable is to ensure that for any path with OPTIONS method the server sends a response with this header: Access-Control-Allow-Origin: *

hi,

thanks for that project.

FYI:
I made a simple RPM with a corresponding systemd-Job.
See: https://github.com/FalcoSuessgott/go-upload-server-rpm

If you interested in a PR. I could submit one.

Maybe use /files/ as prefix for path is too much.
Could you please remove /files/ prefix of path by default?

Is there one method to delete files?

Would you be interested in baking only the resulting Go binary into a Alpine Image to reduce the Docker image size?

Does your image support chunked upload?

I upload a file. I got the message by postman.
{"ok":false,"error":"invalid URL escape "%\x1d(""}

At the moment there is no upload to Subdirectories possible, uploaded files will be always stored in the root directory of /upload. If a curl with /dir/file is executed you will always get "curl: (22) The requested URL returned error: 404 "

See following examples:
curl -X PUT -k -f -Ffile=@ /files//?token=
or
curl -X POST -k -f -Ffile=@ /upload/?token=

I am running the server in a kubernetes statefulset.
Generally the upload is working. The problem is that the MaximumFileSize I set doesn´t seem to have any affect.
I also understand that this error should be Logged which it isn´t.

time="2021-04-08T07:26:13Z" level=info msg="starting up simple-upload-server" time="2021-04-08T07:26:13Z" level=info msg="start listening" cors=false ip=0.0.0.0 port=25478 root=/var/root token=<yes> upload_limit=72118430

The server returns this:
(413) Request Entity Too Large.

the file size is around 7500Kb so the upload size I set should be plenty enough. I have tried several diffrent upload sizes.
I don´t know much about go but as my upload size is logged correctly I figure it probably parses correctly as well.

Btw. it would be great to have support for docker exec :)

Hi !
Im trying to use this api but the CORS Policy is disabled on the server.
Is there a way to enable it easily ?

Thanks !

I planned to use this project as a backup endpoint on my Raspberry Pi but noticed that OOM happens about halfway through an upload of 4.7 GB archive (I've build an image for Arm64v8).
The memory usage peak of this container was 3.8 GB before being killed due to OOM. My Rpi has 4GB RAM.

Due to my lack of knowledge of Go (I'm learning), I cannot say whether is this This seems to be by design (File stored in memory before flushing out to the disk) or there might be a memory leak, but the RAM usage pattern doesn't support that theory well.

EDIT: Found the issue (or at least one part of it): https://github.com/mayth/go-simple-upload-server/blob/master/server.go#L83
This function seems to load the whole file into the memory.

I use this image for a little while.
It is great, but it seems to have nothing about modifying file name automatically.
"Automatically" means that I upload from browser, so I can't change the filename, before upload.
Could you add feature on this?

I'm seeing ./simple_upload_server.go:66:12: undefined: NewServer when I try to use this.

Is there some module it needs?

I am currently trying to upload a html file to a container running your image, by using a "HTTP Request" node in n8n.

The previous step in my workflow writes to "/tmp/index.html", I have no way of providing a path without a directory so far.

Can I POST to your container using something like "@/tmp/index.html" also?

Additional: what about cleaning up? How to remove the file after processing things?

Thanks for any help, Stefan

Could you add a simple HTTPS support please?

I start this server by command ./server tmp. Then upload a file by below command:

curl [email protected] 'http://localhost:25478/upload?token=b30731d81df7fc22338d'

An error will return like this:

{"ok":false,"error":"open tmp/util.go: no such file or directory"}

$ ./simple-upload-server -token foo ./tmp

$ curl [email protected] 'http://localhost:25478/upload?token=foo
{"ok":true,"path":"/files/tmp/test.txt"}

expected: "path":"/files/test.txt"

When running with a config file as

$ go-simple-upload-server -config config.json 

where config.json contains

{
  "addr": "127.0.0.1:8081"
}

I will get the following error (because 8080 is in use for something else on my system)

2024/01/07 12:20:07 Start listening on 127.0.0.1:8080
2024/01/07 12:20:07 server stopped: unable to listen on 127.0.0.1:8080: listen tcp 127.0.0.1:8080: bind: address already in use

But, if I intentionally set addr to an empty string (so bypass triggering of the default flag value), it works:

$ go-simple-upload-server -addr "" -config config.json
2024/01/07 12:21:32 Start listening on 127.0.0.1:8081
2024/01/07 12:21:32 Start serving on 127.0.0.1:8081

Thinking this is related to the merging strategy used to combine flags and the config file, because if I remove the WithOverride strategy the config file works:

diff --git a/app.go b/app.go
index afaf1e9..7b4b279 100644
--- a/app.go
+++ b/app.go
@@ -50,7 +50,7 @@ func main() {
 		if err := json.NewDecoder(f).Decode(&fileConfig); err != nil {
 			log.Fatalf("failed to load config: %v", err)
 		}
-		if err := mergo.Merge(&fileConfig, config, mergo.WithOverride); err != nil {
+		if err := mergo.Merge(&fileConfig, config); err != nil {
 			log.Fatalf("failed to merge config: %v", err)
 		}
 		config = fileConfig

with the consequence that config file options will always take precedence over command line options, which is not entirely intuitive.

Hey there,

i just found your project last week and used it in a test environment for integration-testing.
Unfortunately, your changes last weekend broke my integration as I was using the :latest tag.

It would be awesome if you could publish your release tags also as docker image tags on dockerhub.

Would really appreciate that.
Best regards!

I have this form:

<html>
  <head>
    <title>Test upload</title>
  </head>
  <body>
    <h2>Select files to upload</h2>
    <form name="upload" method="POST" enctype="multipart/form-data" action="https://cdn.masvino.tv/content/upload?token=dca5f3c43c44d4b97">
      <input type="file" name="file1"><br>
      <!--input type="file" name="file2"><br>
      <input type="file" name="file3"><br-->
      <input type="submit" name="submit" value="Upload">
    </form>
  </body>
</html>

After submiting I got this error:

{
ok: false,
error: "http: no such file"
}

Docker container logs:

time="2020-03-06T12:06:03Z" level=error msg="failed to acquire the uploaded content" error="http: no such file"

And Nginx logs:

root@h0579:/var/cache/nginx/client_temp# tail -f /usr/local/nginx/logs/error.log
2020/03/06 09:17:10 [warn] 12376#12376: *41 a client request body is buffered to a temporary file /var/cache/nginx/client_temp/0000000011, client: 168.194.206.109, server: cdn.masvino.tv, request: "POST /content/upload?token=3714ed2fd1dca5f3c43c44d4b972f34ff27242b3 HTTP/1.1", host: "cdn.masvino.tv"
2020/03/06 09:17:10 [warn] 12376#12376: *41 a client request body is buffered to a temporary file /var/cache/nginx/client_temp/0000000011, client: 168.194.206.109, server: cdn.masvino.tv, request: "POST /content/upload?token=3714ed2fd1dca5f3c43c44d4b972f34ff27242b3 HTTP/1.1", host: "cdn.masvino.tv"
2020/03/06 09:18:05 [info] 12376#12376: *40 client closed connection while waiting for request, client: 168.194.206.109, server: 0.0.0.0:443