upload_progress: tracking already registered even if its not-registered and unique about nginx-upload-progress-module HOT 10 OPEN

@khizarsonu we're running in a similar issue. have you solved this somehow?

from nginx-upload-progress-module.

If you can send me a full debug log I can check what is wrong (you need to build nginx with --with-debug). Most of the time the issue is that the same id is used for a subsequent upload.

from nginx-upload-progress-module.

@masterzen i've already turned on debug logging. What i see in the logs is upload_progress: tracking already registered id: upload_536cff9453c7b0.12009139.

I am very sure that the problem is not caused by duplicate progress ids, as we are generating an unique ID for each client. Furthermore, we can reproduce this behavior which is restricted to one specific server (ubuntu 12.04, nginx-extras 1.6.0). The really weird issue is the following:

• If the client is connected via WIFI, the upload works just fine
• If the client is connected via LAN, the upload freezes with the above mentioned warning.

There are no other log messages on the server. Also, we made sure that there's only one single upload happening on the server when we test it (it's our development machine).

Here's what happens on the client:

• We POST a form with a large file (~ 200 MB) via AJAX using a jquery form plugin
• We call the /progress URL via ajax once a second to check the progress and display a progress bar
• After 1 or 2 seconds, the ajax call to nginx_upload_progress keeps sending the same received value for each request
• After 3 to 4 seconds, the ajax call returns error: 500 status, at this time we see the tracking already registered id error in the debug logs.

The original ajax POST request in the browser console at this point is still in state Pending, and will continue to be in this state until you close the window. No further logs on the server.

Here's our nginx.conf

Are there any more debug infos i could provide?

from nginx-upload-progress-module.

@pulse00, please send me privately the full debug.log (you can anonymize it) so that I can have a look. Restart nginx before starting the upload, then stop the experiment at the moment you get the error.

The usual difference between wifi and ethernet is the speed and latency. It might be possible this triggers a bug somewhere.

from nginx-upload-progress-module.

do you need the log at info or debug level?

from nginx-upload-progress-module.

@pulse00 debug please (it might be too large to send it by e-mail, in which case if you can upload this to an http server somewhere that'd be great).

from nginx-upload-progress-module.

@masterzen here's the gzipped log file - it's ~ 230 MB unzipped.

However, after activating debug level the error is not occuring anymore, but it's now showing different behavior: After the upload starts, all calls to /progress result in the received key to be 0. After a while, when the whole file has been transmitted to the server, the upload has succeeded.

Please let me know if there's anything else you need.

from nginx-upload-progress-module.

@pulse00 I thrown an eye on the log file and something struck me:

2014/05/10 00:16:58 [debug] 10393#0: *8 http client request body recv -2
2014/05/10 00:16:58 [debug] 10393#0: *8 http client request body rest 224229458
2014/05/10 00:16:58 [debug] 10393#0: upload-progress: read_event_handler found node: upload_536d5084197908.68183439
2014/05/10 00:16:58 [debug] 10393#0: upload-progress: read_event_handler storing rest 224229458/224229458 for upload_536d5084197908.68183439

As you can see above, nginx http module says there's still 224229458 KiB of upload to be uploaded , and as you can see that's what is stored in the upload-progress.
The issue is that normally when the upload progresses, this number should decrease, in your case it stays like this until the end (I didn't check every occurence).

I think you're seeing the issue #37, which I haven't found the solution for yet.

Can you try to disable TLS/SSL in your experiment and upload again to see if that's the same issue?

from nginx-upload-progress-module.

@pulse00 I've found the issue now. A change in nginx 1.3.9 made that the number of bytes that rest to be uploaded is not updated anymore by nginx unless nginx recycles the buffer where it stores the incoming body.

This buffer is sized with the client_body_buffer_size settings. Yours is at 1GB (which IMHO is way too large unless you have a very powerful server with 64 GB of RAM that could allow at least 64 upload of a 1GB file in parallel (or your uploads are not too heavy)).

If you reduce this to let's say 128k or even 1M and upload a large file, you'll see the upload progress working.

I will fix this issue on the side of the upload progress module later this week-end (I need to find a way to capture the missing information), but I'd appreciate if you could test with a reduced client_body_buffer_size.

from nginx-upload-progress-module.

BTW @pulse00, what you are seeing is issue #36.

from nginx-upload-progress-module.