Code Monkey home page Code Monkey logo

nginx-upload-progress-module's Introduction

Nginx Upload Progress Module
============================

Introduction
============

nginx_uploadprogress_module is an implementation of an upload progress system, that monitors
RFC1867 POST upload as they are transmitted to upstream servers.

It works by tracking the uploads proxied by Nginx to upstream servers without 
analysing the uploaded content and offers a web API to report upload progress in Javscript, Json or any
other format (through the help of templates).

It works because Nginx acts as an accelerator of an upstream server, storing uploaded POST content
on disk, before transmitting it to the upstream server. Each individual POST upload request
should contain a progress unique identifier.

This module is Copyright (c) 2007-2012 Brice Figureau, and is licensed under the BSD license (see LICENSE).
 * rbtree and shm_zone code is based on Igor Sysoev limit_zone Nginx module.
 * expire header code is based on Igor Sysoev header_filter Nginx module.

The JSON idea and the mechanism idea are based on Lighttpd mod_uploadprogress:
http://blog.lighttpd.net/articles/2006/08/01/mod_uploadprogress-is-back


WARNING: 
 * when compiled with --with-debug, this module will produce high number of log messages.

INCOMPATIBLE CHANGES
====================

v0.9.0:

JSONP is now the default output of the progress probes. If you rely on this module serving
the deprecated java output use:
     upload_progress_java_output
in the progress probe location.


Installation
============

nginx_uploadprogress_module has been tested with Nginx 0.6.x, 0.7.x, 0.8.x and 1.0.x.

Download the Nginx sources from http://nginx.net/ and unpack it.

To build Nginx, change to the directory which contains the Nginx
sources, and run the configuration script making sure to add the path
to the nginx_uploadprogress_module sources using the --add-module option: ::

 $ ./configure --add-module=/path/to/nginx_uploadprogress_module/

Now you can build and install the software:

 $ make

and as root:

 $ make install 
 
 
Configuration
=============

Each upload request should be assigned a unique identifier. This unique identifier will be used
to store the request and reference it to report.
This identifier can be transmitted either as a GET argument or as an HTTP header whose name is X-Progress-ID.

upload_progress
+++++++++++++++
    :Syntax: upload_progress <zone_name> <zone_size>
    :Default: none
    :Context: http
    :Description:
    This directive enables the upload progress module and reserve <zone_size> bytes to the <zone_name> which
    will be used to store the per-connection tracking information.
    
track_uploads
+++++++++++++
    :Syntax: track_uploads <zone_name> <timeout>
    :Default: none
    :Context: location
    :Description:
    This directive enables tracking uploads for the current location. Each POST landing in this location will register
    the request in the <zone_name> upload progress tracker.
    Since Nginx doesn't support yet RFC 1867 upload, the location must be a proxy_pass or fastcgi location.
    The POST _must_ have a query parameter called X-Progress-ID (or an HTTP header of the same name) whose value is the
    unique identifier used to get progress information. If the POST has no such information, the upload will not be tracked.
    The tracked connections are kept at most <timeout> seconds after they have been finished to be able to serve 
    useful information to upload progress probes.
    WARNING: this directive must be the last directive of the location. It must be in a proxy_pass or 
    fastcgi_pass location.
    
report_uploads
++++++++++++++
    :Syntax: report_uploads <zone_name>
    :Default: none
    :Context: location
    :Description:
    This directive allows a location to report the upload progress that is tracked by track_uploads for <zone_name>.
    The returned document is a Javascript text with the possible 4 results by default:
      * the upload request hasn't been registered yet or is unknown:             
                new Object({ 'state' : 'starting' })
                
        * the upload request has ended:
                new Object({ 'state' : 'done' })
        
        * the upload request generated an HTTP error
                new Object({ 'state' : 'error', 'status' : <error code> })
            one error code that can be of use to track for the client is 413 (request entity too large).
        
        * the upload request is in progress:
                new Object({ 'state' : 'uploading', 'received' : <size_received>, 'size' : <total_size>})
    
    It is possible to return pure json instead of this javascript (see upload_progress_json_output).
    It is also possible to configure completely the response format with the directive:
            upload_progress_template

    The HTTP request to this location must have a X-Progress-ID parameter or HTTP header containing a valid
    unique identifier of an in progress upload.

upload_progress_content_type
++++++++++++++++++++++++++++
    :Syntax: upload_progress_content_type <content_type>
    :Default: text/javascript
    :Context: location
    :Description:
    This directive allows to change the upload progress probe response content-type.

upload_progress_header
++++++++++++++++++++++
    :Syntax: upload_progress_header <progress-id>
    :Default: X-Progress-ID
    :Context: location
    :Description:
    This directive allows to change the header name of the progress ID.

upload_progress_jsonp_parameter
++++++++++++++++++++++
    :Syntax: upload_progress_jsonp_parameter <callback_parameter>
    :Default: callback
    :Context: location
    :Description:
    This directive allows to change the name of the GET parameter with the jsonp callback name.

upload_progress_java_output
+++++++++++++++++++++++++++
    :Syntax: upload_progress_java_output
    :Default: N/A
    :Context: location
    :Description:
    This directive sets everything to output as eval() javascript compatible code.

upload_progress_json_output
+++++++++++++++++++++++++++
    :Syntax: upload_progress_json_output
    :Default: N/A
    :Context: location
    :Description:
    This directive sets everything to output as pure json.

upload_progress_jsonp_output
++++++++++++++++++++++++++++
    :Syntax: upload_progress_jsonp_output
    :Default: N/A
    :Context: location
    :Description:
    This directive sets everything to output as jsonp (like json output, but with callback).

upload_progress_template
++++++++++++++++++++++++
    :Syntax: upload_progress_template <state> <template>
    :Default: none
    :Context: location
    :Description:
    This directive can be used to install a progress response template.
    The available list of state is:
        * starting
        * uploading
        * error
        * done

    Nginx will replace the value of the following variables with their respective
    value for the upload:
        * $uploadprogress_length: total size of the upload
        * $uploadprogress_received: what the server has received so far
        * $uploadprogress_status: error code in case of HTTP error
        * $uploadprogress_callback: jsonp callback name if provided as a GET query parameter with name 'callback'
    
    For instance to return XML (instead of the default Javascript or json):

    upload_progress_content_type 'text/xml';
    upload_progress_template starting '<upload><state>starting</state></upload>';
    upload_progress_template uploading '<upload><state>uploading</state><size>$uploadprogress_length</size><uploaded>$uploadprogress_received</uploaded></upload>';
    upload_progress_template done '<upload><state>done</state></upload>';
    upload_progress_template error '<upload><state>error</state><code>$uploadprogress_status</code></upload>';

    Example of jsonp response:

    upload_progress_template starting "$uploadprogress_callback({ \"state\" : \"starting\"});";
    upload_progress_template error "$uploadprogress_callback({ \"state\" : \"error\", \"status\" : $uploadprogress_status });";
    upload_progress_template done "$uploadprogress_callback({ \"state\" : \"done\"});";
    upload_progress_template uploading "$uploadprogress_callback({ \"state\" : \"uploading\", \"received\" : $uploadprogress_received, \"size\" : $uploadprogress_length });";

Configuration Example:
+++++++++++++++++++++

http {
    
    # reserve 1MB under the name 'proxied' to track uploads
    upload_progress proxied 1m;

  server {
        listen       127.0.0.1 default;
        server_name  _ *;
        
        root /path/to/root;
        
        location / {
            # proxy to upstream server
            proxy_pass http://127.0.0.1;
            proxy_redirect default;
            
            # track uploads in the 'proxied' zone
            # remember connections for 30s after they finished
            track_uploads proxied 30s;
        }
        
        location ^~ /progress {
            # report uploads tracked in the 'proxied' zone
            report_uploads proxied;
        }
}
    

Usage Example
=============

(based on Lighttd mod_uploadprogress module example):

First we need a upload form:

  <form id="upload" enctype="multipart/form-data" 
    action="/upload.php" method="post" 
    onsubmit="openProgressBar(); return true;">
  <input type="hidden" name="MAX_FILE_SIZE" value="30000000"  />
  <input name="userfile" type="file" label="fileupload" />
  <input type="submit" value="Send File" />
  </form>

And a progress bar to visualize the progress:

  <div>
   <div id="progress" style="width: 400px; border: 1px solid black">
    <div id="progressbar" 
       style="width: 1px; background-color: black; border: 1px solid white">
     &nbsp;
    </div>
   </div>
   <div id="tp">(progress)</div>
  </div>

Then we need to generate the Unique Identifier and launch the upload on submit
action. This also will start the ajax progress report mechanism.

 interval = null;

function openProgressBar() {
 /* generate random progress-id */
 uuid = "";
 for (i = 0; i < 32; i++) {
  uuid += Math.floor(Math.random() * 16).toString(16);
 }
 /* patch the form-action tag to include the progress-id */
 document.getElementById("upload").action="/upload.php?X-Progress-ID=" + uuid;

 /* call the progress-updater every 1000ms */
 interval = window.setInterval(
   function () {
     fetch(uuid);
   },
   1000
 );
}

function fetch(uuid) {
 req = new XMLHttpRequest();
 req.open("GET", "/progress", 1);
 req.setRequestHeader("X-Progress-ID", uuid);
 req.onreadystatechange = function () {
  if (req.readyState == 4) {
   if (req.status == 200) {
    /* poor-man JSON parser */
    var upload = eval(req.responseText);

    document.getElementById('tp').innerHTML = upload.state;

    /* change the width if the inner progress-bar */
    if (upload.state == 'done' || upload.state == 'uploading') {
     bar = document.getElementById('progressbar');
     w = 400 * upload.received / upload.size;
     bar.style.width = w + 'px';
    }
    /* we are done, stop the interval */
    if (upload.state == 'done') {
     window.clearTimeout(interval);
    }
   }
  }
 }
 req.send(null);
}

Companion Software
==================

This software can also work with Valery Kholodkov' Nginx Upload Module:
http://www.grid.net.ru/nginx/upload.en.html

You can also use the following javascript libraries client side:
http://drogomir.com/blog/2008/6/30/upload-progress-script-with-safari-support

Note that when using jQuery AJAX for progress monitoring, such as:
https://github.com/drogus/jquery-upload-progress
you should be sure to set a upload_progress template parameter:
upload_progress_json_output
or
upload_progress_jsonp_output
depending on your jQuery AJAX dataType setting.

nginx-upload-progress-module's People

Contributors

e2dk4r avatar masterzen avatar mayson avatar pyke369 avatar tizoc avatar wbond avatar whissi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

nginx-upload-progress-module's Issues

Nginx doesn't recognize any directives

I have installed nginx via Passenger 3, opting for installation choice #2 for custom nginx and additional compiling options. Everything seems to install fine. You can see a full readout of my customizations, and the resulting output of nginx successfully compiling here:

http://pastie.textmate.org/1433854

Here is a stripped down config:

http://pastie.textmate.org/1433878

Using upload_progress inside the http context throws no errors. The problem occurs when I use any of the directives inside server {}

When I try to boot nginx I get:

unknown directive "upload_store" in /opt/nginx/conf/nginx.conf:69

Is this a bug or am I missing something?

Not working with NginX 0.8.x - 1.0

Following Documentation we got it working on 0.7.x. Upgrading to 0.8.x broke progress.

The returned JSON will always look like:

{ "state" : "starting" }

When TLS is enabled recieved and size are always the same

After I setup TLS for my website the size and the recieved field in the reported progress are always the same (the correct size of the uploaded file). I tested it with large files to make sure the upload doesn't finish to early

The state still changes from uploading to done once the upload finishes.

Using standard unencrypted http everything works as expected.

I configure TLS with the following directives

    ssl_certificate /etc/ssl/private/certificate.pem;
    ssl_certificate_key /etc/ssl/private/certificate.key;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_prefer_server_ciphers on;
    ssl_ciphers "EECDH+ECDSA+AESGCM EECDH+aRSA+AESGCM EECDH+ECDSA+SHA384 EECDH+ECDSA+SHA256 EECDH+aRSA+SHA384 EECDH+aRSA+SHA256 EECDH+aRSA+RC4 EECDH EDH+aRSA RC4 !aNULL !eNULL !LOW !3DES !MD5 !EXP !PSK !SRP !DSS !RC4";
        add_header Strict-Transport-Security "max-age=31536000; includeSubdomains;";
        ssl_session_cache shared:SSL:10m;
        ssl_session_timeout 10m;

A debug session for one file upload is here
https://gist.github.com/Vespasian/d2214d1c25544510f58d

If you need anything feel free to contact me by mail or in this issue. Thanks for your great work on this module

crashed child process for nginx

Hello
my nginx with your module can't serve requests for nginx-upload-progress-module, it crash
i use nginx 0.7.64 and v0.6 you module
[notice] 10742#0: start worker process 11236
2009/11/19 10:54:40 [notice] 10742#0: signal 17 (SIGCHLD) received
2009/11/19 10:54:40 [alert] 10742#0: worker process 11236 exited on signal 11
2009/11/19 10:54:40 [notice] 10742#0: start worker process 11237
2009/11/19 10:54:42 [notice] 10742#0: signal 17 (SIGCHLD) received
2009/11/19 10:54:42 [alert] 10742#0: worker process 11237 exited on signal 11
2009/11/19 10:54:42 [notice] 10742#0: start worker process 11238
2009/11/19 10:54:44 [notice] 10742#0: signal 17 (SIGCHLD) received
2009/11/19 10:54:44 [alert] 10742#0: worker process 11238 exited on signal 11
2009/11/19 10:54:44 [notice] 10742#0: start worker process 11239

Received and size are the same

This is a continuation of the closed issue #36 where the received and size are the same even though the upload is continuing (tested using Firefox on a 3.4Gb file through Charles Proxy throttled to 56kbps). In that thread, others are reporting they are continuing to have this issue.

Built on nginx mainline 1.9.15 (also tried 1.1.10) on Debian jessie (under vagrant) using module v0.9.1.tar.gz.

Build steps used (based off https://serversforhackers.com/compiling-third-party-modules-into-nginx )

Ensure /etc/apt/sources.list.d/nginx.list has an uncommented deb-src :

deb http://nginx.org/packages/mainline/debian/ jessie nginx
deb-src http://nginx.org/packages/mainline/debian/ jessie nginx

sudo apt-get update
sudo apt-get remove nginx-common nginx nginx-extras

# Download module
sudo mkdir /opt/nginxuploadprogressmodule
cd /opt/nginxuploadprogressmodule
sudo wget https://github.com/masterzen/nginx-upload-progress-module/archive/v0.9.1.tar.gz
sudo tar xvf v0.9.1.tar.gz

# Install package creation tools
sudo apt-get install -y dpkg-dev

sudo mkdir /opt/rebuildnginx
cd /opt/rebuildnginx

# Get Nginx source files
sudo apt-get source nginx

# Install the build dependencies
sudo apt-get build-dep nginx

# Edit (sudo) /opt/rebuildnginx/nginx-1.9.15/debian/rules to add at the bottom of the configure flags (ensure \ is included on previous line)
# --add-module=/opt/nginxuploadprogressmodule/nginx-upload-progress-module-0.9.1

# now build
cd /opt/rebuildnginx/nginx-1.9.15
sudo dpkg-buildpackage -b

# once completed, install it
cd /opt/rebuildnginx/
sudo dpkg --install nginx_1.9.15-1~jessie_amd64.deb

# checking the version should show the module installed
sudo nginx -V
nginx version: nginx/1.9.15
built by gcc 4.9.2 (Debian 4.9.2-10)
built with OpenSSL 1.0.1k 8 Jan 2015
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-http_ssl_module --with-http_realip_module --with-http_addition_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_random_index_module --with-http_secure_link_module --with-http_stub_status_module --with-http_auth_request_module --with-http_xslt_module=dynamic --with-http_image_filter_module=dynamic --with-http_geoip_module=dynamic --with-http_perl_module=dynamic --add-dynamic-module=debian/extra/njs-1c50334fbea6/nginx --with-threads --with-stream --with-stream_ssl_module --with-http_slice_module --with-mail --with-mail_ssl_module --with-file-aio --with-ipv6 --with-http_v2_module --with-cc-opt='-g -O2 -fstack-protector-strong -Wformat -Werror=format-security' --with-ld-opt=-Wl,-z,relro --add-module=/opt/nginxuploadprogressmodule/nginx-upload-progress-module-0.9.1

sudo systemctl unmask nginx.service

cat /etc/nginx/nginx.conf

user  vagrant;
worker_processes  1;

error_log  /var/log/nginx/error.log debug;
pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile off;
    #tcp_nopush     on;
    client_max_body_size 4M;
    keepalive_timeout  65;

    #gzip  on;

    include /etc/nginx/conf.d/*.conf;
}

Relevant /etc/nginx/nginx/*.conf entries

upload_progress uploads 1M;
client_body_buffer_size 32k;

server {
    listen 85;
    server_name ""
                192.168.7.7;
    root   /var/www/.../public/;
    index  index.php index.html index.htm;
    access_log /var/www/.../log/access.log;
    error_log /var/www/.../log/error.log debug;

    client_max_body_size 4096M;
    location ^~ /.../progress {
        report_uploads uploads;
    }
    location / {
        include php.conf;
        try_files $uri $uri/ /index.php$is_args$args;
    }
    sendfile off;
    location = /.../upload/file {
        try_files $uri $uri/ /index.php$is_args$args;
        include php.conf;
        track_uploads uploads 60s;
    }

}

Running with debug enabled shows in system log:

2016/05/13 13:08:58 [notice] 1051#1051: using the "epoll" event method
2016/05/13 13:08:58 [notice] 1051#1051: nginx/1.9.15
2016/05/13 13:08:58 [notice] 1051#1051: built by gcc 4.9.2 (Debian 4.9.2-10)
2016/05/13 13:08:58 [notice] 1051#1051: OS: Linux 3.2.0-4-amd64
2016/05/13 13:08:58 [notice] 1051#1051: getrlimit(RLIMIT_NOFILE): 1024:4096
2016/05/13 13:08:58 [notice] 1052#1052: start worker processes
2016/05/13 13:08:58 [notice] 1052#1052: start worker process 1053

and in the site specific log:

2016/05/13 13:09:15 [warn] 1053#1053: *1 a client request body is buffered to a temporary file /var/cache/nginx/client_temp/0000000001, client: 192.168.7.1, server: , request: "POST /.../upload/file?X-Progress-ID=123 HTTP/1.1", host: "192.168.7.7:85", referrer: "http://....php"

This is my first time trying to add a module to nginx (I did try just using debian's nginx-extras package first, but had the same results) so I might be missing something...

allow identifier transmited in POST

Currently as docs says we have two method to pass X-Progress-ID (GET/Headers). The problem with that is all popular ajax fileuploaders only allow to add post paramteres along with file. Yes, we can point to url with GET parameter, but it will be nice to have it in POST also if it possbile

upload_progress: tracking already registered even if its not-registered and unique

hello,

my problem is than when i upload file on nginx.. and getting upload_progress using its module i'm getting error in nginx_error.log
error is "upload_progress: tracking already registered"

i verified that upload_progress id is uniq for every form that are populated on web pages

then why i'm getting this error.

here is my virtual host configuration

server
{
listen 80;
server_name localhost;
index index.html index.htm index.php default.html default.htm default.php;
root /home/wwwroot/mobisfy;

include other.conf;
location ~ .*.(php|php5)?$
{
fastcgi_pass 127.0.0.1:90;
fastcgi_index index.php;
include fcgi.conf;
track_uploads proxied 30s;
}

location ^~ /progress {

report uploads tracked in the 'proxied' zone

report_uploads proxied;
}

location = /favicon.ico {
log_not_found off;
access_log off;
}

location = /robots.txt {
allow all;
log_not_found off;
access_log off;
}

rewrite ^/uploads/files/(.+?)/(.+?)/(.+?)/(.+?)$ /uploads/files/$1?st=$2&e=$3 last;

location ^~ /uploads/files/
{
secure_link $arg_st,$arg_e;
secure_link_md5 sonu$uri$arg_e;

if ($secure_link = 0) {
return 403;
}

if ($secure_link = "")
{
return 500;
}

if ($secure_link != 1)
{
return 500;
}

if everything is ok secure_link is 1

}

location ~ /. {
deny all;
access_log off;
log_not_found off;
}

location ~ /(protected|framework|nbproject|make) {
deny all;
access_log off;
log_not_found off;
}

location ~ /themes/\w+/views {
deny all;
access_log off;
log_not_found off;
}

location /images/r
{
location ~* .jpg$
{
try_files $uri $uri/ /index.php$is_args$args;

return 500;

}
}
location / {
try_files $uri $uri/ /index.php$is_args$args;
}

location ~* .(js|css|png|jpg|jpeg|gif|ico)$ {
expires 24h;
log_not_found off;
}

location ~ .*.(gif|jpg|jpeg|png|bmp|swf)$
{
expires 30d;
}

location ~ .*.(js|css)?$
{
expires 12h;
}

access_log off;
}

Build fails with Nginx 1.1.2

Nginx 1.1.2 no longer defines the NGX_PARSE_LARGE_TIME constant. So the build fails. I was able to get it to build by using this hack:

  #ifndef NGX_PARSE_LARGE_TIME
  #define NGX_PARSE_LARGE_TIME -2
  #endif /* NGX_PARSE_LARGE_TIME */

Adding it to the module file at the top. Of course this is not a fix. A proper fix is to see what changed in the API and update the module according to the new API.

Wrong fix

Hello,

The commit "aa4e8b93558274a45387d215740a6bd4654148f4" has a bug.
If I upload a file, the code enters the first "if()", but not the second "if()", so
"up->rest" is zero, but "up->length" is not.
Therefore the module reports in the beginning 100% uploaded because
up->rest == 0.

Thanks a lot!

    /* Properly handles small files where no read events happen after the */
    /* request is first handled (apparently this can happen on linux with epoll) */
    if (r->headers_in.content_length_n) {
        up->length = r->headers_in.content_length_n;
        if (r->request_body) {
            up->rest = r->request_body->rest;
        }
    }

cannot set tracking id

Hello,

I am using nginx 1.6 - the latest debian extras package with upload progress module. (and php5-fpm)

I cannot get the module to detect and set my tracking id. I do a normal file upload (not ajax call) and however I set the post parameter (I can't set it in header as I can't make an ajax call) the module never recognises it. I get the same lines logged each time

2014/10/27 09:52:26 [debug] 32651#0: *86 upload-progress: get_tracking_id
2014/10/27 09:52:26 [debug] 32651#0: *86 upload-progress: get_tracking_id no header found
2014/10/27 09:52:26 [debug] 32651#0: *86 upload-progress: get_tracking_id no id found
2014/10/27 09:52:26 [debug] 32651#0: *86 trackuploads no id found in POST upload req

The ajax call to progress then gives me a 405 error (not allowed) which I guess is because tracking has not been initialised.

2014/10/27 09:52:27 [debug] 32651#0: *87 upload-progress: get_tracking_id
2014/10/27 09:52:27 [debug] 32651#0: *87 malloc: 0000000001E23BA0:16
2014/10/27 09:52:27 [debug] 32651#0: *87 upload-progress: get_tracking_id found header: 1159563624
2014/10/27 09:52:27 [debug] 32651#0: *87 trackuploads id found: 1159563624
2014/10/27 09:52:27 [debug] 32651#0: *87 trackuploads not tracking in this location for id: 1159563624

and

2014/10/27 09:52:27 [debug] 32651#0: *87 http finalize request: 405, "/progress?" a:1, c:1
2014/10/27 09:52:27 [debug] 32651#0: *87 http special response: 405, "/progress?"
2014/10/27 09:52:27 [debug] 32651#0: *87 http set discard body
2014/10/27 09:52:27 [debug] 32651#0: *87 uploadprogress error-tracker error: 405
2014/10/27 09:52:27 [debug] 32651#0: *87 uploadprogress error-tracker not tracking in this location

Any ideas? I've definitely got the post parameter name correct, and I've debugged the request and can see the correct post parameter on the server - the module just doesn't detect it.

many thanks
Dave

Received and size are the same

upload progress module reports the same values for received and size. Contrary to a bug report #42 this bug is present even when not using http2 connections.

Could something be please done about it?

Best regards,
Andrejs

nginx -V

nginx version: nginx/1.11.4
built with OpenSSL 1.0.2h 3 May 2016
TLS SNI support enabled
configure arguments: --prefix=/usr --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error_log --pid-path=/run/nginx.pid --lock-path=/run/lock/nginx.lock --with-cc-opt=-I/usr/include --with-ld-opt=-L/usr/lib64 --http-log-path=/var/log/nginx/access_log --http-client-body-temp-path=/var/lib/nginx/tmp/client --http-proxy-temp-path=/var/lib/nginx/tmp/proxy --http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi --http-scgi-temp-path=/var/lib/nginx/tmp/scgi --http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi --with-file-aio --with-debug --with-http_v2_module --with-ipv6 --with-pcre --with-threads --without-http_auth_basic_module --without-http_geo_module --without-http_limit_conn_module --without-http_memcached_module --without-http_referer_module --without-http_scgi_module --without-http_split_clients_module --without-http_userid_module --without-http_uwsgi_module --with-http_dav_module --with-http_flv_module --with-http_geoip_module --with-http_mp4_module --with-http_perl_module --with-http_stub_status_module --with-http_realip_module --add-module=external_module/taobao-nginx-http-concat-f3b52a5 --add-module=external_module/nginx-upload-progress-module-0.9.1 --add-module=external_module/headers-more-nginx-module-0.29 --with-http_ssl_module --without-stream_upstream_hash_module --without-stream_upstream_least_conn_module --without-stream_upstream_zone_module --without-stream_upstream_hash_module --without-stream_upstream_least_conn_module --without-stream_upstream_zone_module --without-stream_upstream_hash_module --without-stream_upstream_least_conn_module --without-stream_upstream_zone_module --without-mail_imap_module --without-mail_pop3_module --without-mail_smtp_module --user=nginx --group=nginx

nginx.conf

user nginx www;
worker_processes auto;
worker_cpu_affinity auto;
pcre_jit on;

worker_rlimit_core 500M;
working_directory /var/tmp/nginx;

error_log /var/log/nginx/error_log info;

events {
worker_connections 10000;
use epoll;
multi_accept on;
}

http {
include /etc/nginx/mime.types;
default_type application/octet-stream;

    limit_req_zone $binary_remote_addr zone=zone-requests:10m rate=1r/s;
    limit_req_log_level warn;

    client_header_timeout 10s;
    client_body_timeout 10s;
    send_timeout 10s;

    client_max_body_size 700m;
    connection_pool_size 256;
    client_body_buffer_size 16k;
    client_header_buffer_size 8k;
    large_client_header_buffers 4 16k;
    request_pool_size 4k;
    proxy_buffering on;
    proxy_buffer_size 4k;
    proxy_buffers 4096 4k;

   open_file_cache off;
   open_log_file_cache max=1000 inactive=60s valid=1m;

    geoip_country /usr/share/GeoIP/GeoLiteCountry.dat; # the country IP data
    geoip_city    /usr/share/GeoIP/GeoLiteCity.dat; # the city IP database

    gzip on;
    gzip_min_length 1100;
    gzip_buffers 4 8k;
    gzip_types text/plain text/css application/json application/javascript a

    output_buffers 1 32k;
    postpone_output 1460;

    aio on;
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;

    keepalive_timeout 75s;
    keepalive_requests 100000;
    keepalive_disable msie6;

    ignore_invalid_headers on;
    server_name_in_redirect on;
    merge_slashes on;

    fastcgi_pass_request_headers on;
    fastcgi_intercept_errors off;
    fastcgi_buffer_size 4k;
    fastcgi_buffers 256 4k;
    fastcgi_read_timeout 360;       # xdebug needs larger value for fastcgi_
    fastcgi_send_timeout 180;
    fastcgi_store_access user:rw group:r;
    fastcgi_force_ranges on;
    proxy_intercept_errors off;

    ssl_session_cache   shared:SSL:10m;
    ssl_session_timeout 1h;
    ssl_session_tickets on;
    ssl_session_ticket_key ssl_session_ticket_file.key;
    ssl_dhparam /etc/ssl/certs/dhparam.pem;
    ssl_protocols           TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers             "ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GC
    ssl_prefer_server_ciphers on;
    ssl_stapling on;
    ssl_stapling_verify on;
    ssl_trusted_certificate /etc/ssl/certs/startssl-ca-bundle.crt;
    ssl_certificate         /etc/ssl/certs/1_mydomain.com_bundle.crt;
    ssl_certificate_key     /etc/ssl/private/1_mydomain.com_bundle.key;

    index index.html;
    charset utf-8;

    # common headers

    add_header X-Frame-Options SAMEORIGIN;
    add_header X-Content-Type-Options nosniff;

    # upload progress
    upload_progress uploads 1m;
    upload_progress_content_type "application/json";
    upload_progress_json_output;

    concat on;
    concat_max_files 40;

    resolver 127.0.0.1 valid=300s;
    resolver_timeout 10s;

    # devel.mydomain:443
    server {
            listen 443 ssl;
            server_name devel.mydomain;

            access_log /var/log/nginx/mydomain.access_log main;
            error_log /var/log/nginx/mydomain.error_log info;

            root /opt/www/mydomain;

            add_header Strict-Transport-Security "max-age=63072000;";

            keepalive_timeout       70;

            location / {
              index index.php;
              try_files $uri $uri/ @index;
            }

            location @index {
              include fastcgi_params;
              fastcgi_param PATH_INFO $uri;
              fastcgi_param PATH_TRANSLATED $document_root$uri;
              fastcgi_param SCRIPT_FILENAME $document_root/index.php;
              fastcgi_pass unix:/run/php-fpm/3508-php70.sock;
              track_uploads uploads 30s;
            }

            location = /upload-progress {
              access_log off;
              postpone_output 0;
              report_uploads uploads;
            }
     }

}

Getting 500 Internal Server Error from nginx when upload progress is enabled

I am constantly getting a 500 Internal Server Error from nginx when I try to upload a file to the server.
There is nothing in the error log that corresponds to this error. This happens on a server that is under heavy load. There are several hundreds of users on the website at the same time.

I am using the following configuration:
Nginx 1.1.19 incl. nginx-extras with upload progress module (Debian backports) but I also tried to compile everything myself from the latest sources (0.8.4 and 0.9 from Github).

My nginx.conf:

user www-data;
worker_processes 1;

error_log /var/log/nginx/error.log;
pid /var/run/nginx.pid;

events {
worker_connections 1024;
multi_accept off;
}

http {
include /etc/nginx/mime.types;

access_log  /var/log/nginx/access.log;

sendfile        on;
#tcp_nopush     on;

#keepalive_timeout  0;
keepalive_timeout  65;
tcp_nodelay        on;

gzip  on;
gzip_disable "MSIE [1-6]\.(?!.*SV1)";

upload_progress     uploadtracker 1m;

include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;

}

My site configuration file:

server {
listen 80; ## listen for ipv4
listen [::]:80 default ipv6only=on; ## listen for ipv6
error_log /var/log/nginx/error.site.log;

    server_name  *.domain.tld;

    location /static/ {
            alias /path/to/static/;
    }

    location ~* /upload_progress {
            upload_progress_json_output;
            report_uploads uploadtracker;
    }

    location / {
            access_log      off;
            proxy_pass http://localhost:8080;
            proxy_set_header X-Real-IP  $remote_addr;
            client_max_body_size 1000M;
            track_uploads uploadtracker 6s;
    }

}

Way to reproduce the problem in my case:

  1. upload file (size doesn't matter)
  2. requesting upload progress from nginx. sometimes at the beginning the upload progress modules works for a few seconds and shows the upload progress but suddenly it returns a JSON object containing {state: "error", status: 500}.
  3. The URL where the file was sent to returns a 500 Internal Server Error message from nginx.
  4. After that all requests for upload progress get the JSON object mentioned in step 2.

Maybe it has something to do with memory allocation. I don't know. But it does not seem to work under heavy load.

Any ideas? Maybe how I can make upload progress module errors appear in the nginx error.log?

Compilation fails with Nginx 1.23

Trying to compile as a dynamic module from master branch on Alpine Linux 3.15:

	objs/ngx_http_uploadprogress_module_modules.c
/tmp/ngx_http_uploadprogress_module/ngx_http_uploadprogress_module.c: In function 'ngx_http_reportuploads_handler':
/tmp/ngx_http_uploadprogress_module/ngx_http_uploadprogress_module.c:649:39: error: 'r->headers_out.cache_control' is a pointer; did you mean to use '->'?
  649 |     ccp = r->headers_out.cache_control.elts;
      |                                       ^
      |                                       ->
/tmp/ngx_http_uploadprogress_module/ngx_http_uploadprogress_module.c:652:28: error: passing argument 1 of 'ngx_array_init' from incompatible pointer type [-Werror=incompatible-pointer-types]
  652 |         if (ngx_array_init(&r->headers_out.cache_control, r->pool,
      |                            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |                            |
      |                            ngx_table_elt_t ** {aka struct ngx_table_elt_s **}
In file included from src/core/ngx_core.h:65,
                 from /tmp/ngx_http_uploadprogress_module/ngx_http_uploadprogress_module.c:8:
src/core/ngx_array.h:32:29: note: expected 'ngx_array_t *' but argument is of type 'ngx_table_elt_t **' {aka 'struct ngx_table_elt_s **'}
   32 | ngx_array_init(ngx_array_t *array, ngx_pool_t *pool, ngx_uint_t n, size_t size)
      |                ~~~~~~~~~~~~~^~~~~
/tmp/ngx_http_uploadprogress_module/ngx_http_uploadprogress_module.c:658:30: error: passing argument 1 of 'ngx_array_push' from incompatible pointer type [-Werror=incompatible-pointer-types]
  658 |         ccp = ngx_array_push(&r->headers_out.cache_control);
      |                              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |                              |
      |                              ngx_table_elt_t ** {aka struct ngx_table_elt_s **}
In file included from src/core/ngx_core.h:65,
                 from /tmp/ngx_http_uploadprogress_module/ngx_http_uploadprogress_module.c:8:
src/core/ngx_array.h:27:35: note: expected 'ngx_array_t *' but argument is of type 'ngx_table_elt_t **' {aka 'struct ngx_table_elt_s **'}
   27 | void *ngx_array_push(ngx_array_t *a);
      |                      ~~~~~~~~~~~~~^
/tmp/ngx_http_uploadprogress_module/ngx_http_uploadprogress_module.c:675:53: error: 'r->headers_out.cache_control' is a pointer; did you mean to use '->'?
  675 |         for (i = 1; i < r->headers_out.cache_control.nelts; i++) {
      |                                                     ^
      |                                                     ->
cc1: all warnings being treated as errors
make[1]: *** [objs/Makefile:2047: objs/addon/ngx_http_uploadprogress_module/ngx_http_uploadprogress_module.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make[1]: Leaving directory '/tmp/nginx-1.23.0'
make: *** [Makefile:10: build] Error 2

0 byte file doesn't work

Hello,
First of all thank you for this great plugin.
I have encountered a bug when someone try to upload an empty file. The state of the upload stay at the value 'Starting'.
Thank you.

CORS OPTIONS requests not supported, breaking usage with SPDY

When enabled SPDY support in nginx, Chrome sends an OPTIONS request to e.g. /progress first, before requesting it via GET.

This results in a 405 Method Not Allowed response right now:

Request URL: https://example.com/progress
Request Method: OPTIONS
Status Code: 405 OK

Request Headers:
accept:/
accept-encoding:gzip,deflate,sdch
accept-language:de-DE,de;q=0.8,en-US;q=0.6,en;q=0.4
access-control-request-headers:accept, origin, x-progress-id, x-requested-with, content-type
access-control-request-method:GET
host:example.com
method:OPTIONS
origin:http://example.com
referer:http://example.com/
scheme:https
url:/progress
user-agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.81 Safari/537.36
version:HTTP/1.1

Response Headers
content-length:568
content-type:text/html
date:Mon, 03 Jun 2013 09:04:03 GMT
server:nginx
status:405
version:HTTP/1.1

Instead, something like the following response should be returned by the upload progress module:

Access-Control-Allow-Origin: $request_origin
Access-Control-Allow-Methods: GET, HEAD, OPTIONS
Access-Control-Allow-Headers: $access_control_request_headers
Access-Control-Max-Age: 86400

Possibly somehow related to http://forum.nginx.org/read.php?29,236251,236251.

JSONP doesn't work

The JSONP callback doesn't work. I have tried many combinations of things to try and get it to work (Changing the parameter names, rearranging the parameter order in the URL, compiling with an older changeset, etc.) but none have done the trick.

I have tried nginx 1.0.14 and 0.8.55.

/progress/?X-Progress-ID=12345678901234567890123456789012&callback=jsonp00000000000&_=00000000000

That url, and any combination of parameter ordering, results in the regular JSON output instead of the JSONP output.

Configuration used:

upload_progress prog 10m;

...

        location ^~ /progress {
            report_uploads prog;
        }

...

        location /upload {
                upload_pass   @uploadbackend;
                track_uploads prog 30s;
        }

Using module in cluster architecture (many upload nodes behind LB)

I'm using upload-progress module in cluster architecture, e.g. I have many upload servers behind load balancer (TCP level). And there is a problem, when I make upload request, I know only load balancer address, not upload server address. And there no way to get upload progress of that server, that handle current upload.

Example: 1 LB, 3 Upload Server (A, B, C). When I do upload request, it handled with, for example, server A (choosen by LB). But when I do upload progress request, it handled by server B, next request by server C, then by A (round robin manner). Obviously, I've got 2 error messages (upload ID is not found) and only one correct answer.

So, I want a mechanism to get upload progress correctly. There is native way to do this in nginx: proxy_next_upstream directive, which can pass request to another server when current server answers with bad HTTP code.

There is my suggestion: add some way to return HTTP code from upload-progress module to make it possible to handle answer with proxy_next_upstream directive.

There's an my patch and example configuration: seletskiy/nginx-upload-progress-module@42b6b60d

not work for proxy_pass to other nginx

Here a small case where the module not work ....

nginx1 > nginx2 > uwsgi

nginx1 using proxy_pass to other nginx...

the option report_uploads not report correctly ever show starting ..

Filesize Issue (ref. nginx mailing list)

Referring to the nginx mailing list post at http://forum.nginx.org/read.php?2,53197

We use the upload-progress-module v0.8 and nginx v0.7.64 (compiled with --with-debug and SSL Support, Upload is via normal http though)

The Request to the UploadTracker is triggered exactly like in the example in the README (onsubmit="openprogressbar();" etc. We basically copied the script from the README and adjusted the part where the form is updated with the correct action (in our case: document.getElementById("upload").action="ftp_upload?X-Progress-ID=" + uuid;) Also we adjusted the progress request (req.open("GET", "/upload_progress", 1);)

I will send you two compressed debug logs of two uploads shortly.

Progressbar with "vkholodkov/nginx-upload-module" return completed upload when starting

Hello,

I'm using this module with the "vkholodkov/nginx-upload-module". And I cannot get the actual upload size. The module always return directly the file length in the remaining variable, see the following Nginx debug log:

2016/03/18 16:21:06 [debug] 25572#0: *6 HTTP/1.1 200 OK
Server: nginx/1.8.1
Date: Fri, 18 Mar 2016 15:21:06 GMT
Content-Type: text/javascript
Content-Length: 70
Connection: keep-alive
Expires: Thu, 01 Jan 1970 00:00:01 GMT
Cache-Control: no-cache

2016/03/18 16:21:06 [debug] 25572#0: *6 write new buf t:1 f:0 B9385590, pos B9385590, size: 217 file: 0, size: 0
2016/03/18 16:21:06 [debug] 25572#0: *6 http write filter: l:0 f:0 s:217
2016/03/18 16:21:06 [debug] 25572#0: *6 http output filter "/progress?"
2016/03/18 16:21:06 [debug] 25572#0: *6 http copy filter: "/progress?"
2016/03/18 16:21:06 [debug] 25572#0: *6 image filter
2016/03/18 16:21:06 [debug] 25572#0: *6 xslt filter body
2016/03/18 16:21:06 [debug] 25572#0: *6 http postpone filter "/progress?" BFA65158
2016/03/18 16:21:06 [debug] 25572#0: *6 write old buf t:1 f:0 B9385590, pos B9385590, size: 217 file: 0, size: 0
2016/03/18 16:21:06 [debug] 25572#0: *6 write new buf t:1 f:0 B93854E0, pos B93854E0, size: 70 file: 0, size: 0
2016/03/18 16:21:06 [debug] 25572#0: *6 http write filter: l:1 f:0 s:287
2016/03/18 16:21:06 [debug] 25572#0: *6 http write filter limit 0
2016/03/18 16:21:06 [debug] 25572#0: *6 writev: 287 of 287
2016/03/18 16:21:06 [debug] 25572#0: *6 http write filter 00000000
2016/03/18 16:21:06 [debug] 25572#0: *6 http copy filter: 0 "/progress?"
2016/03/18 16:21:06 [debug] 25572#0: *6 http finalize request: 0, "/progress?" a:1, c:1
2016/03/18 16:21:06 [debug] 25572#0: *6 set http keepalive handler
2016/03/18 16:21:06 [debug] 25572#0: *6 http close request
2016/03/18 16:21:06 [debug] 25572#0: *6 http log handler
2016/03/18 16:21:06 [debug] 25572#0: *6 free: B9384BC0, unused: 1066
2016/03/18 16:21:06 [debug] 25572#0: *6 free: B94239F8
2016/03/18 16:21:06 [debug] 25572#0: *6 hc free: 00000000 0
2016/03/18 16:21:06 [debug] 25572#0: *6 hc busy: 00000000 0
2016/03/18 16:21:06 [debug] 25572#0: *6 reusable connection: 1
2016/03/18 16:21:06 [debug] 25572#0: *6 event timer add: 25: 65000:2320617683
2016/03/18 16:21:06 [debug] 25572#0: *6 post event B943E6F4
2016/03/18 16:21:06 [debug] 25572#0: *8 http keepalive handler
2016/03/18 16:21:06 [debug] 25572#0: *8 malloc: B94239F8:1024
2016/03/18 16:21:06 [debug] 25572#0: *8 recv: fd:27 394 of 1024
2016/03/18 16:21:06 [debug] 25572#0: *8 reusable connection: 0
2016/03/18 16:21:06 [debug] 25572#0: *8 posix_memalign: B9384BC0:4096 @16
2016/03/18 16:21:06 [debug] 25572#0: *8 http header: "Host: progressbar.localhost"
2016/03/18 16:21:06 [debug] 25572#0: *8 http header: "User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:45.0) Gecko/20100101 Firefox/45.0"
2016/03/18 16:21:06 [debug] 25572#0: *8 http header: "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"
2016/03/18 16:21:06 [debug] 25572#0: *8 http header: "Accept-Language: en-US,en;q=0.5"
2016/03/18 16:21:06 [debug] 25572#0: *8 http header: "Accept-Encoding: gzip, deflate"
2016/03/18 16:21:06 [debug] 25572#0: *8 http header: "X-Progress-ID: ddb321ec6c4d04c13a83ff9651cdde74"
2016/03/18 16:21:06 [debug] 25572#0: *8 http header: "Referer: http://progressbar.localhost/"
2016/03/18 16:21:06 [debug] 25572#0: *8 http header: "Connection: keep-alive"
2016/03/18 16:21:06 [debug] 25572#0: *8 http header done
2016/03/18 16:21:06 [debug] 25572#0: *8 generic phase: 0
2016/03/18 16:21:06 [debug] 25572#0: *8 rewrite phase: 1
2016/03/18 16:21:06 [debug] 25572#0: *8 test location: "/"
2016/03/18 16:21:06 [debug] 25572#0: *8 test location: "progress"
2016/03/18 16:21:06 [debug] 25572#0: *8 using configuration "/progress"
2016/03/18 16:21:06 [debug] 25572#0: *8 http cl:-1 max:805306368
2016/03/18 16:21:06 [debug] 25572#0: *8 rewrite phase: 3
2016/03/18 16:21:06 [debug] 25572#0: *8 rewrite phase: 4
2016/03/18 16:21:06 [debug] 25572#0: *8 post rewrite phase: 5
2016/03/18 16:21:06 [debug] 25572#0: *8 generic phase: 6
2016/03/18 16:21:06 [debug] 25572#0: *8 generic phase: 7
2016/03/18 16:21:06 [debug] 25572#0: *8 generic phase: 8
2016/03/18 16:21:06 [debug] 25572#0: *8 access phase: 9
2016/03/18 16:21:06 [debug] 25572#0: *8 access phase: 10
2016/03/18 16:21:06 [debug] 25572#0: *8 access phase: 11
2016/03/18 16:21:06 [debug] 25572#0: *8 access phase: 12
2016/03/18 16:21:06 [debug] 25572#0: *8 post access phase: 13
2016/03/18 16:21:06 [debug] 25572#0: *8 http set discard body
2016/03/18 16:21:06 [debug] 25572#0: *8 upload-progress: get_tracking_id
2016/03/18 16:21:06 [debug] 25572#0: *8 malloc: B9422D20:8
2016/03/18 16:21:06 [debug] 25572#0: *8 upload-progress: get_tracking_id found header: ddb321ec6c4d04c13a83ff9651cdde74
2016/03/18 16:21:06 [debug] 25572#0: *8 reportuploads handler found id: ddb321ec6c4d04c13a83ff9651cdde74
2016/03/18 16:21:06 [debug] 25572#0: *8 upload-progress: find_node ddb321ec6c4d04c13a83ff9651cdde74
2016/03/18 16:21:06 [debug] 25572#0: *8 upload-progress: found node
2016/03/18 16:21:06 [debug] 25572#0: *8 reportuploads found node: ddb321ec6c4d04c13a83ff9651cdde74 (rest: 0, length: 3119977, done: 0, err_status: 0)
2016/03/18 16:21:06 [debug] 25572#0: *8 http script copy: "({ "state" : "uploading", "received" : "
2016/03/18 16:21:06 [debug] 25572#0: *8 http script var: "3119977"
2016/03/18 16:21:06 [debug] 25572#0: *8 http script copy: ", "size" : "
2016/03/18 16:21:06 [debug] 25572#0: *8 http script var: "3119977"
2016/03/18 16:21:06 [debug] 25572#0: *8 http script copy: " });"

nginx version: nginx/1.8.1
built with OpenSSL 1.0.1f 6 Jan 2014
TLS SNI support enabled
configure arguments: --with-debug --with-cc-opt='-g -O2 -fPIE -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2' --with-ld-opt='-Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now' --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-debug --with-pcre-jit --with-ipv6 --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_addition_module --with-http_dav_module --with-http_flv_module --with-http_geoip_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_image_filter_module --with-http_mp4_module --with-http_perl_module --with-http_random_index_module --with-http_secure_link_module --with-http_spdy_module --with-http_sub_module --with-http_xslt_module --with-mail --with-mail_ssl_module --add-module=/opt/rebuildnginx/nginx-1.8.1/debian/modules/headers-more-nginx-module --add-module=/opt/rebuildnginx/nginx-1.8.1/debian/modules/nginx-auth-pam --add-module=/opt/rebuildnginx/nginx-1.8.1/debian/modules/nginx-cache-purge --add-module=/opt/rebuildnginx/nginx-1.8.1/debian/modules/nginx-dav-ext-module --add-module=/opt/rebuildnginx/nginx-1.8.1/debian/modules/nginx-development-kit --add-module=/opt/rebuildnginx/nginx-1.8.1/debian/modules/nginx-echo --add-module=/opt/rebuildnginx/nginx-1.8.1/debian/modules/ngx-fancyindex --add-module=/opt/rebuildnginx/nginx-1.8.1/debian/modules/nginx-http-push --add-module=/opt/rebuildnginx/nginx-1.8.1/debian/modules/nginx-lua --add-module=/opt/rebuildnginx/nginx-1.8.1/debian/modules/nginx-upstream-fair --add-module=/opt/rebuildnginx/nginx-1.8.1/debian/modules/ngx_http_substitutions_filter_module --add-module=/opt/rebuildnginx/nginx-1.8.1/debian/modules/nginx-upload-progress --add-module=/opt/nginx-upload-module-2.2

nginx.conf:

server {
  listen       127.0.0.1 default;
  server_name  _ ;
  root /var/www/progress-bar;
  index index.html;
  client_max_body_size 768M;
  error_page 405 = $uri;
}
server {
  listen 80;
  listen  127.0.0.1:80;
  server_name progressbar.localhost;

  client_max_body_size 768M;

  error_log /var/log/nginx/error_progressbar.log debug;

  location / {
      # # proxy to upstream server
       proxy_pass http://127.0.0.1;
       proxy_redirect default;

      location ~* upload$ {
          # Pass altered request body to this location
          upload_pass   @uploading;

          # Store files to this directory
          # The directory is hashed, subdirectories 0 1 2 3 4 5 6 7 8 9 should exist
          upload_store /tmp/uploads2;

          # Allow uploaded files to be read only by user
          upload_store_access user:rw;

          # Set specified fields in request body
          upload_set_form_field "${upload_field_name}_name" $upload_file_name;
          upload_set_form_field "${upload_field_name}_content_type" $upload_content_type;
          upload_set_form_field "${upload_field_name}_path" $upload_tmp_path;

          # Inform backend about hash and size of a file
          upload_aggregate_form_field "${upload_field_name}_md5" $upload_file_md5;
          upload_aggregate_form_field "${upload_field_name}_size" $upload_file_size;

          upload_pass_form_field "^submit$|^description$";
     }
     # track uploads in the 'proxied' zone
     # remember connections for 30s after they finished
     track_uploads proxied 30s;
  }

  location @uploading {
    proxy_pass http://127.0.0.1;
  }

  location ^~ /progress {
      # report uploads tracked in the 'proxied' zone
      report_uploads proxied;
  }
}

index.html

<html>
<head>  
</head>                          
<body>            
 <form id="upload" enctype="multipart/form-data" 
    action="/index.html" method="post"
    onsubmit="openProgressBar(); return true;">
  <!--<input type="hidden" name="MAX_FILE_SIZE" value="30000000"  />-->
  <input name="userfile" type="file" label="fileupload" />
  <input type="submit" value="Send File" />
  </form>   
<div>                       
   <div id="progress" style="width: 400px; border: 1px solid black">
    <div id="progressbar" 
       style="width: 1px; background-color: black; border: 1px solid white">
     &nbsp;                                           
    </div>                                             
   </div>                                                   
   <div id="tp">(progress)</div>                                          
  </div>                          
<script type="text/javascript">               
interval = null;               

function openProgressBar() {
 /* generate random progress-id */                    
 uuid = "";                        
 for (i = 0; i < 32; i++) {
  uuid += Math.floor(Math.random() * 16).toString(16);
 }          
 /* patch the form-action tag to include the progress-id */
 document.getElementById("upload").action="/upload?X-Progress-ID=" + uuid;

 /* call the progress-updater every 1000ms */
 interval = window.setInterval(
   function () {                                
     fetch(uuid);                                                             
   },            
   1000
 );
}

function fetch(uuid) {                                         
 req = new XMLHttpRequest();                                   
 req.open("GET", "/progress", 1);                              
 req.setRequestHeader("X-Progress-ID", uuid);                  
 req.onreadystatechange = function () {                        
  if (req.readyState == 4) {                                   
   if (req.status == 200) {                                    
    /* poor-man JSON parser */                                 
    var upload = eval(req.responseText);                       

    document.getElementById('tp').innerHTML = upload.state;    

    /* change the width if the inner progress-bar */           
    if (upload.state == 'done' || upload.state == 'uploading') {
     bar = document.getElementById('progressbar');             
     w = 400 * upload.received / upload.size;                  
     bar.style.width = w + 'px';                               
    }                                                          
    /* we are done, stop the interval */                       
    if (upload.state == 'done') {
     window.clearTimeout(interval);                            
    }
   }
  }
 }
 req.send(null);
}
</script>
</body>
</html>

Occasionally fails to track upload

When testing out the module, I noticed that if I repeatedly consecutively attempt to upload large files (~ 100mb) that it will eventually stop tracking the upload.

Looking through the debug logs however it appears to register the upload correctly.

The following log snips highlight the issue:

Is this possibly to do with the size allocated to the zone (currently 1m)?

I am currently writing the Javascript side of the code and am working out all of the likely failure scenarios, and how to handle them. Is this a bug - or should I expect it to fail to track some uploads?

works with PHP-FPM?

I'm trying to configure this module (0.8.3) in a pretty standard nginx (1.1.7) with PHP-FPM (5.3.8) installation, and the problem is that progress always returns:

    { "state" : "starting" }

even when the upload has finished. This is my conf:

    location = /progress
    {
            upload_progress_json_output;
            report_uploads uploads;
    }

    location ^~ /uploads
    {
            fastcgi_pass   127.0.0.1:9000;
            try_files $uri /index.php?ctr=$uri&$args;
            track_uploads uploads 5s;
    }

Tested with an ultra basic HTML form and javascript, and double checked that X-Progress-ID value is sended right.

Problem with 'dynamically loadable module' support.

When I build nginx 1.11.5 (dev) with v0.9.2 nginx -t returns

nginx: [emerg] unknown directive "upload_progress" in /etc/nginx/nginx.conf:86
nginx: configuration file /etc/nginx/nginx.conf test failed

when I switch back to upload progress v0.9.1 it works as expected.

The same happens with nginx 1.10.1 (stable)

37182ce seems to be the only difference.

How can I use this module statically as before?

Can you create download files for each release please?

I'm adding nginx-upload-progress-module to the Macport for nginx as a variant, but not having a fixed download URL is a PITA to deal with...

I'd be forever grateful if you could package the download as nginx_uploadprogress_module-0.9.0.tar.gz as that would make the patchfiles I'm trying to submit to macports easier to produce and manage.

Thanks in advance, and thanks for this nginx module!

Module seems to work correctly with uwsgi_pass

I've tested this module against uwsgi and it works like a charm ;-)

If you think that mine observations are correct, please add to the documentation, information that uwsgi_pass can can be used with this module.

Dynamic module support

Hello,

Is it possible to add dynamic module support for nginx-upload-progress? The module is part of nginx-extras in debian/ubuntu and we are in the process of splitting the modules in separate packages.

Thank you,

Using the same x-progress-id: error or warning ?

Hello,

I'm trying to integrate the upload-progress-module (with upload-module) in your system. In case the client initiate a POST using the same x-progress-id, the module always return a "NGX_HTTP_INTERNAL_SERVER_ERROR".

Is it possible, without changing the code, the bypass the error ?

If there was a problem on the client side (same x-progress-id used twice...) I would like to process the upload without error, even if the progression will not be available ?

What is the best way to do it ? If people upload large data in your case it should be a bad idea at the end to display an error just because of the x-progress-id...

Compilation error with nginx-1.0.10 and c7c663f on Ubuntu Lucid

Hi there,

I've just tried to compile nginx with this module on Ubuntu (10.04.3) and I've received these errors when running a make:

    -o objs/addon/masterzen-nginx-upload-progress-module-c7c663f/ngx_http_uploadprogress_module.o \
    /tmp/nginx-1.0.10/contrib/masterzen-nginx-upload-progress-module-c7c663f//ngx_http_uploadprogress_module.c

/tmp/nginx-1.0.10/contrib/masterzen-nginx-upload-progress-module-c7c663f//ngx_http_uploadprogress_module.c: In function ‘ngx_http_uploadprogress_event_handler’:
/tmp/nginx-1.0.10/contrib/masterzen-nginx-upload-progress-module-c7c663f//ngx_http_uploadprogress_module.c:452:50: error: variable ‘c’ set but not used [-Werror=unused-but-set-variable]
/tmp/nginx-1.0.10/contrib/masterzen-nginx-upload-progress-module-c7c663f//ngx_http_uploadprogress_module.c: In function ‘ngx_http_uploadprogress_cleanup’:
/tmp/nginx-1.0.10/contrib/masterzen-nginx-upload-progress-module-c7c663f//ngx_http_uploadprogress_module.c:1010:38: error: variable ‘ctx’ set but not used [-Werror=unused-but-set-variable]
/tmp/nginx-1.0.10/contrib/masterzen-nginx-upload-progress-module-c7c663f//ngx_http_uploadprogress_module.c: In function ‘ngx_http_uploadprogress_create_loc_conf’:
/tmp/nginx-1.0.10/contrib/masterzen-nginx-upload-progress-module-c7c663f//ngx_http_uploadprogress_module.c:1243:43: error: variable ‘t’ set but not used [-Werror=unused-but-set-variable]
/tmp/nginx-1.0.10/contrib/masterzen-nginx-upload-progress-module-c7c663f//ngx_http_uploadprogress_module.c: In function ‘ngx_http_uploadprogress_init_variables_and_templates’:
/tmp/nginx-1.0.10/contrib/masterzen-nginx-upload-progress-module-c7c663f//ngx_http_uploadprogress_module.c:1317:43: error: variable ‘n’ set but not used [-Werror=unused-but-set-variable]
/tmp/nginx-1.0.10/contrib/masterzen-nginx-upload-progress-module-c7c663f//ngx_http_uploadprogress_module.c:1315:43: error: variable ‘t’ set but not used [-Werror=unused-but-set-variable]
cc1: all warnings being treated as errors

make[1]: *** [objs/addon/masterzen-nginx-upload-progress-module-c7c663f/ngx_http_uploadprogress_module.o] Error 1
make[1]: Leaving directory `/tmp/nginx-1.0.10'
make: *** [build] Error 2

Does anyone have any solutions to this?

Thanks in advance for any help you can provide.

Received and size are the same when using HTTP/2

It might be linked to #44 or some other ticket.

When using HTTP/2 in my server configuration, received and size are always the same. Without HTTP/2, it just works fine.

Result of nginx -V :

nginx version: nginx/1.10.1
built with OpenSSL 1.0.2h 3 May 2016
TLS SNI support enabled
configure arguments: --with-cc-opt='-g -O2 -fPIE -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2' --with-ld-opt='-Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now' --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-debug --with-pcre-jit --with-ipv6 --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_addition_module --with-http_dav_module --with-http_flv_module --with-http_geoip_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_image_filter_module --with-http_mp4_module --with-http_perl_module --with-http_random_index_module --with-http_secure_link_module --with-http_v2_module --with-http_sub_module --with-http_xslt_module --with-mail --with-mail_ssl_module --with-stream --with-stream_ssl_module --with-threads --with-file-aio --add-module=/root/rebuildnginx/nginx-1.10.1/debian/modules/headers-more-nginx-module --add-module=/root/rebuildnginx/nginx-1.10.1/debian/modules/nginx-auth-pam --add-module=/root/rebuildnginx/nginx-1.10.1/debian/modules/nginx-cache-purge --add-module=/root/rebuildnginx/nginx-1.10.1/debian/modules/nginx-dav-ext-module --add-module=/root/rebuildnginx/nginx-1.10.1/debian/modules/nginx-development-kit --add-module=/root/rebuildnginx/nginx-1.10.1/debian/modules/nginx-echo --add-module=/root/rebuildnginx/nginx-1.10.1/debian/modules/ngx-fancyindex --add-module=/root/rebuildnginx/nginx-1.10.1/debian/modules/nginx-http-push --add-module=/root/rebuildnginx/nginx-1.10.1/debian/modules/nginx-lua --add-module=/root/rebuildnginx/nginx-1.10.1/debian/modules/nginx-upload-progress --add-module=/root/rebuildnginx/nginx-1.10.1/debian/modules/nginx-upstream-fair --add-module=/root/rebuildnginx/nginx-1.10.1/debian/modules/ngx_http_substitutions_filter_module --add-module=/root/rebuildnginx/nginx-1.10.1/debian/modules/nginx-rtmp-module --add-module=/root/rebuildnginx/nginx-1.10.1/debian/modules/nginx-vod-module

My Nginx configuration is directly taken from Perusio's Drupal for Nginx (for Drupal 7 in my case) : https://github.com/perusio/drupal-with-nginx, but I'm pretty sure it can be reproduced with a more classic configuration.

In my virtualhost file :
WORKS :

listen 443 ssl;
listen [::]:443 ssl;

DOES NOT WORK :

listen 443 ssl http2;
listen [::]:443 ssl http2;

Ask me if you need more info, but it should be easily reproducible.
Thanks in advance.

Implement HTTP Server-Sent Events

To save on server-load renegotiating several SSL connections it might be interesting to implement server-sent events as means of returning the status. That way, only one connection has to be opened by the client, which reduces load on the server.

More information on Server-Sent Events can be found at the W3C at https://www.w3.org/TR/2009/WD-eventsource-20091029/

Default reporting interval might be 1000ms or 1 MiB, whatever happens first. Optional reporting details via request, e.g. /report?upload_id=foo&interval=750&block=524288 for updates every 750ms or completion of 512KiB.

Nginx master process segfault after second call nginx configuration reload (/etc/init.d/nginx reload)

Greetings.
I use nginx/0.7.64 with nginx-upload-progress-module/0.7 on Ubuntu 9.10.
Just run /etc/init.d/nginx reload (or send -HUP signal) 2 times, and nginx master process will shut down, while children processes still work.

Core dump:
GNU gdb (GDB) 7.0-ubuntu
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "i486-linux-gnu".
For bug reporting instructions, please see:
http://www.gnu.org/software/gdb/bugs/...
Reading symbols from /usr/sbin/nginx...done.
Reading symbols from /lib/tls/i686/cmov/libcrypt.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/tls/i686/cmov/libcrypt.so.1
Reading symbols from /lib/libpcre.so.3...(no debugging symbols found)...done.
Loaded symbols for /lib/libpcre.so.3
Reading symbols from /lib/i686/cmov/libssl.so.0.9.8...(no debugging symbols found)...done.
Loaded symbols for /lib/i686/cmov/libssl.so.0.9.8
Reading symbols from /lib/i686/cmov/libcrypto.so.0.9.8...(no debugging symbols found)...done.
Loaded symbols for /lib/i686/cmov/libcrypto.so.0.9.8
Reading symbols from /lib/tls/i686/cmov/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/tls/i686/cmov/libdl.so.2
Reading symbols from /lib/libz.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libz.so.1
Reading symbols from /lib/tls/i686/cmov/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/tls/i686/cmov/libc.so.6
Reading symbols from /lib/ld-linux.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/ld-linux.so.2
Reading symbols from /lib/tls/i686/cmov/libnss_compat.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/tls/i686/cmov/libnss_compat.so.2
Reading symbols from /lib/tls/i686/cmov/libnsl.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/tls/i686/cmov/libnsl.so.1
Reading symbols from /lib/tls/i686/cmov/libnss_nis.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/tls/i686/cmov/libnss_nis.so.2
Reading symbols from /lib/tls/i686/cmov/libnss_files.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/tls/i686/cmov/libnss_files.so.2
Core was generated by `nginx:'.
Program terminated with signal 11, Segmentation fault.
#0 0x0804d9c2 in ngx_palloc_large (pool=0x9cb6d20, size=) at src/core/ngx_palloc.c:230

230 if (large->alloc == NULL) {
(gdb) bt full
#0 0x0804d9c2 in ngx_palloc_large (pool=0x9cb6d20, size=) at src/core/ngx_palloc.c:230

    p = 0xb6c9b008
    n = 1
    large = 0x8

#1 0x0804daa5 in ngx_palloc (pool=0x8, size=1) at src/core/ngx_palloc.c:141

    m = <value optimized out>
    p = <value optimized out>

#2 0x0804dd2a in ngx_array_push_n (a=0x9cb6cf8, n=8) at src/core/ngx_array.c:131

    elt = <value optimized out>
    new = <value optimized out>
    nalloc = 16
    p = 0x6d002

#3 0x0807bcb7 in ngx_http_script_add_code (codes=0x9cb6cf8, size=8, code=0x0) at src/http/ngx_http_script.c:562

    elts = <value optimized out>
    new = <value optimized out>

#4 0x0807bd14 in ngx_http_script_add_copy_code (sc=0xbfdba0b4, value=0x1, last=446466) at src/http/ngx_http_script.c:589

    len = <value optimized out>
    zero = 134535430
    code = <value optimized out>

#5 0x0807c1ce in ngx_http_script_compile (sc=0xbfdba0b4) at src/http/ngx_http_script.c:367

    ch = <value optimized out>
    name = {len = 38, data = 0x80aa7a4 "new Object({ 'state' : 'starting' })\r\n"}
    i = 38
    bracket = <value optimized out>

#6 0x0809f949 in ngx_http_upload_progress_set_template (cf=, t=, source=0x80b3960)

at /usr/src/nginx-modules/masterzen-nginx-upload-progress-module-ac62a29/ngx_http_uploadprogress_module.c:1486
    sc = {cf = 0xbfdba44c, source = 0x80b3960, flushes = 0x0, lengths = 0x9cb6ce0, values = 0x9cb6cdc, variables = 0, ncaptures = 0, 
      captures_mask = 0, size = 38, main = 0x0, compile_args = 0, complete_lengths = 1, complete_values = 1, zero = 0, 
      conf_prefix = 0, root_prefix = 0, dup_capture = 0, args = 0}

#7 0x0809fbe5 in ngx_http_uploadprogress_init_variables_and_templates (cf=0xbfdba44c)

at /usr/src/nginx-modules/masterzen-nginx-upload-progress-module-ac62a29/ngx_http_uploadprogress_module.c:1308
    var = <value optimized out>
    v = <value optimized out>
    m = 0x80b3880

#8 0x0806a642 in ngx_http_block (cf=0xbfdba44c, cmd=0x80ad3a0, conf=0x9d62c1c) at src/http/ngx_http.c:231

    rv = <value optimized out>
    mi = 36
    m = 40
    s = <value optimized out>
    ports = {elts = 0x9d62c00, nelts = 1163022147, size = 16777216, nalloc = 0, pool = 0x9d62e08}
    module = <value optimized out>
    ctx = 0x9d62e50
    clcf = <value optimized out>
    cmcf = 0x9d62ed4

#9 0x0805a2b7 in ngx_conf_handler (cf=0xbfdba44c, filename=0x9d627b0) at src/core/ngx_conf_file.c:393

    rv = <value optimized out>
    conf = <value optimized out>
    i = 7
    multi = 0
    confp = 0x1
    name = 0x9d62d4c
    cmd = 0x80ad3a0

#10 ngx_conf_parse (cf=0xbfdba44c, filename=0x9d627b0) at src/core/ngx_conf_file.c:243

    rv = <value optimized out>
    fd = 6
    rc = 1
    buf = {
      pos = 0x9d2d489 "\n    include       /etc/nginx/mime.types;\n\n    access_log\t/var/log/nginx/access.log;\n\n    sendfile        on;\n    #tcp_nopush     on;\n\n    #keepalive_timeout  0;\n    keepalive_timeout  65;\n    tcp_nod"..., 
      last = 0x9d2d7d3 "q\206\234\263\313\344\376\031\065\021", file_pos = 579621298477465604, file_last = 708803710033386319, 
      start = 0x9d2d3d8 "user www-data;\nworker_processes  1;\n\nerror_log  /var/log/nginx/error.log;\npid        /var/run/nginx.pid;\n\nevents {\n    worker_connections  1024;\n    # multi_accept on;\n}\n\nhttp {\n    include       /etc"..., end = 0x9d2e3d8 "\016(C_a\a", 
      tag = 0x4000, file = 0x9cd23a4, shadow = 0x9cd23a4, temporary = 1, memory = 0, mmap = 0, recycled = 1, in_file = 1, flush = 1, 
      sync = 0, last_buf = 1, last_in_chain = 1, last_shadow = 1, temp_file = 0, num = 134536150}
    prev = 0x0
    conf_file = {file = {fd = 6, name = {len = 21, data = 0x9d627fc "/etc/nginx/nginx.conf"}, info = {st_dev = 2049, __pad1 = 0, 
          __st_ino = 834362, st_mode = 33188, st_nlink = 1, st_uid = 0, st_gid = 0, st_rdev = 0, __pad2 = 0, st_size = 1019, 
          st_blksize = 4096, st_blocks = 8, st_atim = {tv_sec = 1257134873, tv_nsec = 0}, st_mtim = {tv_sec = 1256334203, 
            tv_nsec = 0}, st_ctim = {tv_sec = 1257135791, tv_nsec = 0}, st_ino = 834362}, offset = 1019, 
        sys_offset = -4621920719170886748, log = 0x9cd23a4, valid_info = 0, directio = 0}, buffer = 0xbfdba36c, line = 12}
    type = parse_file

#11 0x08057ab9 in ngx_init_cycle (old_cycle=0x9cd2398) at src/core/ngx_cycle.c:263

    rv = <value optimized out>
    senv = 0xbfdba87c
    env = <value optimized out>
    i = 45
    n = 45
    log = 0x9cd23a4
    conf = {name = 0x0, args = 0x9d62d38, cycle = 0x9d62708, pool = 0x9d626e0, temp_pool = 0x9db6768, conf_file = 0xbfdba2e0, 
      log = 0x9cd23a4, ctx = 0x9d62e50, module_type = 1163022147, cmd_type = 16777216, handler = 0, handler_conf = 0x0}
    pool = 0x9d626e0
    cycle = 0x9d62708
    old = <value optimized out>
    shm_zone = 0x80000000
    oshm_zone = 0x80000000
    part = <value optimized out>
    opart = 0x0
    file = 0x0
    nls = 0x80000000
    ccf = <value optimized out>
    old_ccf = <value optimized out>
    hostname = "denis", '\000' <repeats 87 times>"\226, \223\071\000\276U\005\bx\245ۿ", '\000' <repeats 148 times>, "|\346%"

#12 0x080651a6 in ngx_master_process_cycle (cycle=0x9cd2398) at src/os/unix/ngx_process_cycle.c:230

    title = 0x80b5700 ""
    p = <value optimized out>
    size = 2
    i = 2
    n = 1
    set = {__val = {0 <repeats 32 times>}}
    itv = {it_interval = {tv_sec = 28, tv_usec = 6809200}, it_value = {tv_sec = 3989036, tv_usec = 340088576}}
    live = 1
    delay = 0
    ccf = 0x9cd294c

#13 0x0804cf8a in main (argc=1, argv=0xbfdba874) at src/core/nginx.c:389

    i = <value optimized out>
    log = 0x80b3b74
    cycle = 0x9cab508
    init_cycle = {conf_ctx = 0x0, pool = 0x9caace0, log = 0x80b3b74, new_log = {log_level = 0, file = 0x0, connection = 0, 
        handler = 0, data = 0x0, action = 0x0}, files = 0x0, free_connections = 0x0, free_connection_n = 0, listening = {elts = 0x0, 
        nelts = 0, size = 0, nalloc = 0, pool = 0x0}, pathes = {elts = 0x0, nelts = 0, size = 0, nalloc = 0, pool = 0x0}, 
      open_files = {last = 0x0, part = {elts = 0x0, nelts = 0, next = 0x0}, size = 0, nalloc = 0, pool = 0x0}, shared_memory = {
        last = 0x0, part = {elts = 0x0, nelts = 0, next = 0x0}, size = 0, nalloc = 0, pool = 0x0}, connection_n = 0, files_n = 0, 
      connections = 0x0, read_events = 0x0, write_events = 0x0, old_cycle = 0x0, conf_file = {len = 21, 
        data = 0x80a039a "/etc/nginx/nginx.conf"}, conf_param = {len = 0, data = 0x0}, conf_prefix = {len = 11, 
        data = 0x80a039a "/etc/nginx/nginx.conf"}, prefix = {len = 17, data = 0x80a0388 "/usr/local/nginx/"}, lock_file = {len = 0, 
        data = 0x0}, hostname = {len = 0, data = 0x0}}
    ccf = <value optimized out>

(gdb) quit

Received bytes begins at complete file size for a few ticks and then functions as normal

I have Chromium network set to throttle at a low speed and here is the output on a 1.0s poll so that I'm not overwhelmed with the console logs. I suppose that I can work around around this by checking if received === size on the uploading state and then setting it to 0 but I'd rather that it just worked.

uploading Object {state: "uploading", received: 304831, size: 304831}
uploading Object {state: "uploading", received: 304831, size: 304831}
uploading Object {state: "uploading", received: 16384, size: 304831}
uploading Object {state: "uploading", received: 16384, size: 304831}
uploading Object {state: "uploading", received: 32768, size: 304831}
uploading Object {state: "uploading", received: 32768, size: 304831}
uploading Object {state: "uploading", received: 32768, size: 304831}
uploading Object {state: "uploading", received: 49152, size: 304831}
uploading Object {state: "uploading", received: 49152, size: 304831}
uploading Object {state: "uploading", received: 49152, size: 304831}
uploading Object {state: "uploading", received: 65536, size: 304831}

From the Ubuntu development PPA.

nginx version: nginx/1.9.11
built with OpenSSL 1.0.1f 6 Jan 2014
TLS SNI support enabled
configure arguments: --with-cc-opt='-g -O2 -fPIE -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2' --with-ld-opt='-Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now' --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-debug --with-pcre-jit --with-ipv6 --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_addition_module --with-http_dav_module --with-http_flv_module --with-http_geoip_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_image_filter_module --with-http_mp4_module --with-http_perl_module --with-http_random_index_module --with-http_secure_link_module --with-http_v2_module --with-http_sub_module --with-http_xslt_module --with-mail --with-mail_ssl_module --with-stream --with-stream_ssl_module --with-threads --add-module=/build/nginx-EGCLOL/nginx-1.9.11/debian/modules/headers-more-nginx-module --add-module=/build/nginx-EGCLOL/nginx-1.9.11/debian/modules/nginx-auth-pam --add-module=/build/nginx-EGCLOL/nginx-1.9.11/debian/modules/nginx-cache-purge --add-module=/build/nginx-EGCLOL/nginx-1.9.11/debian/modules/nginx-dav-ext-module --add-module=/build/nginx-EGCLOL/nginx-1.9.11/debian/modules/nginx-development-kit --add-module=/build/nginx-EGCLOL/nginx-1.9.11/debian/modules/nginx-echo --add-module=/build/nginx-EGCLOL/nginx-1.9.11/debian/modules/ngx-fancyindex --add-module=/build/nginx-EGCLOL/nginx-1.9.11/debian/modules/nginx-http-push --add-module=/build/nginx-EGCLOL/nginx-1.9.11/debian/modules/nginx-lua --add-module=/build/nginx-EGCLOL/nginx-1.9.11/debian/modules/nginx-upload-progress --add-module=/build/nginx-EGCLOL/nginx-1.9.11/debian/modules/nginx-upstream-fair --add-module=/build/nginx-EGCLOL/nginx-1.9.11/debian/modules/ngx_http_substitutions_filter_module

nginx-upload-progress-module appears to be not loaded

I compile nginx with nginx upload progress module but i still can't use any of the configuration variables.I get unknown directive error for all variables.Below is an example

[emerg] unknown directive "track_uploads"

nginx version: nginx/1.11.3
built by gcc 4.8.5 20150623 (Red Hat 4.8.5-4) (GCC)
built with OpenSSL 1.0.2h 3 May 2016
TLS SNI support enabled
configure arguments: --prefix=/usr/share/nginx --sbin-path=/usr/sbin/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --http-client-body-temp-path=/var/lib/nginx/tmp/client_body --http-proxy-temp-path=/var/lib/nginx/tmp/proxy --http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi --http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi --http-scgi-temp-path=/var/lib/nginx/tmp/scgi --pid-path=/var/run/nginx.pid --lock-path=/var/lock/subsys/nginx --user=nginx --group=nginx --with-file-aio --with-ipv6 --with-http_ssl_module --with-http_v2_module --with-openssl=/root/openssl-1.0.2h --with-http_realip_module --with-http_geoip_module --with-http_sub_module --with-http_random_index_module --with-http_gzip_static_module --with-http_stub_status_module --add-module=../nginx-upload-module --add-module=../nginx-upload-progress-module

deb package require

please add this useful module as debian package or guide to rebuild debian package via nginx-upload-progress-module

FTBFS with nginx 1.1.15 on Debian

Hi.

I tried to upgrade the nginx packages using the just released nginx 1.1.15, and the build failed.

Here is the log extract :
/tmp/buildd/nginx-1.1.15/debian/modules/nginx-upload-progress/ngx_http_uploadprogress_module.c: In function 'ngx_http_track_uploads':
/tmp/buildd/nginx-1.1.15/debian/modules/nginx-upload-progress/ngx_http_uploadprogress_module.c:1445:26: error: 'NGX_PARSE_LARGE_TIME' undeclared (first use in this function)
/tmp/buildd/nginx-1.1.15/debian/modules/nginx-upload-progress/ngx_http_uploadprogress_module.c:1445:26: note: each undeclared identifier is reported only once for each function it appears in
make[2]: *** [objs/addon/nginx-upload-progress/ngx_http_uploadprogress_module.o] Error 1

Apparently, they removed the NGX_PARSE_LARGE_TIME variable starting 1.1.15.

Could you take a look at fixing it ?

Thank you very much.

nginx-upload-progress-module fails build on kfreebsd-* + nginx 1.1.4

Hello from Debian maintainer,

I just got build failure of upload progress module on kfreebsd-amd64, kfreebsd-i386. It mostly look gcc issue here.

Version: 0.8.2-0-g8b55a34
Nginx version: 1.1.4

Log: (full log at: https://buildd.debian.org/status/fetch.php?pkg=nginx&arch=kfreebsd-amd64&ver=1.1.4-1&stamp=1316942283&file=log )

gcc -c -pipe -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Wunused-function -Wunused-variable -Wunused-value -Werror -g -DNDK_SET_VAR -I src/core -I src/event -I src/event/modules -I src/os/unix -I /build/buildd-nginx_1.1.4-1-kfreebsd-amd64-vEnU3k/nginx-1.1.4/debian/modules/nginx-development-kit/objs -I objs/addon/ndk -I /usr/include/lua5.1 -I /build/buildd-nginx_1.1.4-1-kfreebsd-amd64-vEnU3k/nginx-1.1.4/debian/modules/nginx-http-push/src -I /usr/include/libxml2 -I objs -I src/http -I src/http/modules -I src/http/modules/perl -I /build/buildd-nginx_1.1.4-1-kfreebsd-amd64-vEnU3k/nginx-1.1.4/debian/modules/nginx-development-kit/src -I src/mail
-o objs/addon/nginx-upload-progress/ngx_http_uploadprogress_module.o
/build/buildd-nginx_1.1.4-1-kfreebsd-amd64-vEnU3k/nginx-1.1.4/debian/modules/nginx-upload-progress/ngx_http_uploadprogress_module.c
/build/buildd-nginx_1.1.4-1-kfreebsd-amd64-vEnU3k/nginx-1.1.4/debian/modules/nginx-upload-progress/ngx_http_uploadprogress_module.c: In function 'ngx_http_uploadprogress_event_handler':
/build/buildd-nginx_1.1.4-1-kfreebsd-amd64-vEnU3k/nginx-1.1.4/debian/modules/nginx-upload-progress/ngx_http_uploadprogress_module.c:452:50: error: variable 'c' set but not used [-Werror=unused-but-set-variable]
/build/buildd-nginx_1.1.4-1-kfreebsd-amd64-vEnU3k/nginx-1.1.4/debian/modules/nginx-upload-progress/ngx_http_uploadprogress_module.c: In function 'ngx_http_uploadprogress_cleanup':
/build/buildd-nginx_1.1.4-1-kfreebsd-amd64-vEnU3k/nginx-1.1.4/debian/modules/nginx-upload-progress/ngx_http_uploadprogress_module.c:1010:38: error: variable 'ctx' set but not used [-Werror=unused-but-set-variable]
/build/buildd-nginx_1.1.4-1-kfreebsd-amd64-vEnU3k/nginx-1.1.4/debian/modules/nginx-upload-progress/ngx_http_uploadprogress_module.c: In function 'ngx_http_uploadprogress_create_loc_conf':
/build/buildd-nginx_1.1.4-1-kfreebsd-amd64-vEnU3k/nginx-1.1.4/debian/modules/nginx-upload-progress/ngx_http_uploadprogress_module.c:1238:43: error: variable 't' set but not used [-Werror=unused-but-set-variable]
/build/buildd-nginx_1.1.4-1-kfreebsd-amd64-vEnU3k/nginx-1.1.4/debian/modules/nginx-upload-progress/ngx_http_uploadprogress_module.c: In function 'ngx_http_uploadprogress_init_variables_and_templates':
/build/buildd-nginx_1.1.4-1-kfreebsd-amd64-vEnU3k/nginx-1.1.4/debian/modules/nginx-upload-progress/ngx_http_uploadprogress_module.c:1312:43: error: variable 'n' set but not used [-Werror=unused-but-set-variable]
/build/buildd-nginx_1.1.4-1-kfreebsd-amd64-vEnU3k/nginx-1.1.4/debian/modules/nginx-upload-progress/ngx_http_uploadprogress_module.c:1310:43: error: variable 't' set but not used [-Werror=unused-but-set-variable]
cc1: all warnings being treated as errors

make[2]: *** [objs/addon/nginx-upload-progress/ngx_http_uploadprogress_module.o] Error 1
make[2]: Leaving directory /build/buildd-nginx_1.1.4-1-kfreebsd-amd64-vEnU3k/nginx-1.1.4/debian/build-extras' make[1]: *** [build] Error 2 make[1]: Leaving directory/build/buildd-nginx_1.1.4-1-kfreebsd-amd64-vEnU3k/nginx-1.1.4/debian/build-extras'
make: *** [build-arch.extras] Error 2

Thanks,
Kartik

Received and size are the same

Contrary to a bug report #42 this bug is present when using http1.1 connections.

Faced the issue on nginx + nginx-extras from

  • 1.6.2 jessy debian package
  • on 1.10.3 16.04.1 LTS ubuntu package
  • on jessy dotdeb #1.12.1 debian package
    { state: "uploading", received: 15703240, size: 15703240 }

http2 is not used
10.211.55.2 - - [01/Oct/2017:19:24:35 +0300] "POST /upload.php?X-Progress-ID=1 HTTP/1.1" 400 0 "http://nem.deb/upload.php" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36"

client_body_buffer_size 32k is not helping

even tried debian wheezy nginx 1.2.1, but there progress will just keep saying {state: "starting"}

js disabled

If for any reason, the frontend (JavaScript on the browser side) is not connected to the location UploadProgressModule, then every second upload finishes the error message (500 error).

nginx v1.0.4 + php5-fpm
progress-module v0.8.2
Debian 6 x64

NginxHttpUploadProgressModule received and size are the same

i have same issue with latest version of nginx using dotdeb.org repository deabian wheezy https://www.ruby-forum.com/topic/204982

new Object({ 'state' : 'starting' })
new Object({ 'state' : 'uploading', 'received' : 4505554, 'size' :
4505554 })
new Object({ 'state' : 'uploading', 'received' : 4505554, 'size' :
4505554 })
new Object({ 'state' : 'uploading', 'received' : 4505554, 'size' :
4505554 })
new Object({ 'state' : 'uploading', 'received' : 4505554, 'size' :
4505554 })
new Object({ 'state' : 'uploading', 'received' : 4505554, 'size' :
4505554 })
new Object({ 'state' : 'done' })

nginx version: nginx/1.4.4
TLS SNI support enabled
configure arguments: --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-log-path=/var/log/nginx/access.log --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --with-pcre-jit --with-debug --with-file-aio --with-http_addition_module --with-http_dav_module --with-http_flv_module --with-http_geoip_module --with-http_gzip_static_module --with-http_gunzip_module --with-http_image_filter_module --with-http_mp4_module --with-http_perl_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_spdy_module --with-http_stub_status_module --with-http_ssl_module --with-http_sub_module --with-http_xslt_module --with-ipv6 --with-mail --with-mail_ssl_module --add-module=/usr/src/nginx/source/nginx-1.4.4/debian/modules/headers-more-nginx-module --add-module=/usr/src/nginx/source/nginx-1.4.4/debian/modules/nginx-auth-pam --add-module=/usr/src/nginx/source/nginx-1.4.4/debian/modules/nginx-cache-purge --add-module=/usr/src/nginx/source/nginx-1.4.4/debian/modules/nginx-dav-ext-module --add-module=/usr/src/nginx/source/nginx-1.4.4/debian/modules/nginx-development-kit --add-module=/usr/src/nginx/source/nginx-1.4.4/debian/modules/nginx-echo --add-module=/usr/src/nginx/source/nginx-1.4.4/debian/modules/ngx-fancyindex --add-module=/usr/src/nginx/source/nginx-1.4.4/debian/modules/nginx-push-stream-module --add-module=/usr/src/nginx/source/nginx-1.4.4/debian/modules/nginx-lua --add-module=/usr/src/nginx/source/nginx-1.4.4/debian/modules/nginx-upload-progress --add-module=/usr/src/nginx/source/nginx-1.4.4/debian/modules/nginx-upstream-fair --add-module=/usr/src/nginx/source/nginx-1.4.4/debian/modules/nginx-syslog --add-module=/usr/src/nginx/source/nginx-1.4.4/debian/modules/ngx_http_pinba_module --add-module=/usr/src/nginx/source/nginx-1.4.4/debian/modules/ngx_http_substitutions_filter_module --add-module=/usr/src/nginx/source/nginx-1.4.4/debian/modules/ngx_pagespeed --add-module=/usr/src/nginx/source/nginx-1.4.4/debian/modules/nginx-x-rid-header --with-ld-opt=-lossp-uuid

PHP-FPM Config refuses to validate& start nginx

For security reasons, the PHP-FPM proxy pass config in a sites conf file should be structures as follows:

location ~ .php$ {
if (-f $request_filename) {
fastcgi_pass 127.0.0.1:9000;
}
fastcgi_split_path_info ^(.+.php)(/.+)$;
#fastcgi_pass unix:/var/run/php-fpm/php-fpm.sock;
include fastcgi_params;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
}

The thing is, adding track_uploads uploads 60s; after the fastcgi_pass doesnt work as it triggers a warning when I try to start nginx (saying you can put it in IF statements).

However for security reasons the If statement out to stay there.... Is there any fix/workaround?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.