Code Monkey home page Code Monkey logo

Comments (4)

carolinebeauchamp avatar carolinebeauchamp commented on July 18, 2024

Progress:

Managed to consistently (ish) reproduce this issue, with the following way:

rebuilt nginx from source with the following:

./configure --add-module=/root/official/nginx-1.5.10/src/modules/nginx-upload-module --with-http_ssl_module

Used the following nginx.conf (basically cut down our production conf while testing to check everything keeps failing:

user vault;
worker_processes  1;
worker_rlimit_core  500M;
worker_rlimit_nofile  20000;
working_directory   /data/nginxcores;
daemon off;

error_log  /var/log/nginx/error.log debug;
pid        /var/run/nginx.pid;

events {
  worker_connections  16384;
}

http {
  include       /etc/nginx/mime.types;
  default_type  application/octet-stream;

  types_hash_max_size 2048;
  types_hash_bucket_size 64;

  access_log    /var/log/nginx/access.log;

  sendfile on;
  tcp_nopush on;
  tcp_nodelay on;

  keepalive_timeout  0;
  client_body_timeout 300;
  client_header_timeout 300;
  send_timeout 300;

  proxy_read_timeout 300;

  gzip  off;

  server_names_hash_bucket_size 64;

     variables_hash_max_size 1024;

server {
       server_name servername servername.example.net  ;
       client_max_body_size 10000m;

        access_log  /var/log/nginx/server.access.log;
        error_log  /var/log/nginx/server.error.log debug;

       listen 10.10.50.17:80;
       listen 127.0.1.1:80;

    location / {
      # an HTTP header important enough to have its own Wikipedia entry:
      #   http://en.wikipedia.org/wiki/X-Forwarded-For
#      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

      # enable this if and only if you use HTTPS, this helps Rack
      # set the proper protocol for doing redirects:
      # proxy_set_header X-Forwarded-Proto https;

      # pass the Host: header from the client right along so redirects
      # can be set properly within the Rack application
#      proxy_set_header Host $http_host;


      # we don't want nginx trying to do something clever with
      # redirects, we set the Host: header above already.
      proxy_redirect off;

      # set "proxy_buffering off" *only* for Rainbows! when doing
      # Comet/long-poll stuff.  It's also safe to set if you're
      # using only serving fast clients with Unicorn + nginx.
      # Otherwise you _want_ nginx to buffer responses to slow
      # clients, really.
      # proxy_buffering off;
root /var/www;
      # Try to serve static files from nginx, no point in making an
      # *application* server like Unicorn/Rainbows! serve static files.

    }

   # Upload form should be submitted to this location
   location = /upload/1 {

        # Pass altered request body to this location
        upload_pass   @test;

        # Store files to this directory
        # The directory is hashed, subdirectories 0 1 2 3 4 5 6 7 8 9 should exist
        upload_store /mnt1/assets/tmp 1;

        # Allow uploaded files to be read only by user
        upload_store_access user:rw group:rw all:rw;
#       upload_store_access 066

        # Set specified fields in request body
        upload_set_form_field $upload_field_name.name "$upload_file_name";
        upload_set_form_field $upload_field_name.content_type "$upload_content_type";
        upload_set_form_field $upload_field_name.path "$upload_tmp_path";

        # Inform backend about hash and size of a file
#        upload_aggregate_form_field "$upload_field_name.sha1" "$upload_file_sha1";
#        upload_aggregate_form_field "$upload_field_name.size" "$upload_file_size";

        upload_pass_form_field "^submit$|^description$|^token$|^qqfile$|^file1$|^chunk$|^chunks$|^chunk_base64_sha_hash$|^storage_path$|^storage_unit$|^server$";

    if ($request_method = 'OPTIONS') {
           add_header Access-Control-Allow-Origin '*';
           add_header Access-Control-Allow-Methods 'POST, OPTIONS';
           add_header Access-Control-Max-Age '1728000';
           add_header Content-Type: 'text/plain; charset=UTF-8';
           add_header Access-Control-Allow-Headers 'X-CSRF-Token';
           return 200;
        }

        # FILE UPLOADS
        if ($request_method = 'POST') {
           add_header Access-Control-Allow-Origin: '*';
        }

        upload_cleanup 400 404 499 500-505;
    }

   # Upload form should be submitted to this location
   location = /upload/2 {

        # Pass altered request body to this location
        upload_pass   @test;

        # Store files to this directory
        # The directory is hashed, subdirectories 0 1 2 3 4 5 6 7 8 9 should exist
        upload_store /mnt2/assets/tmp 1;

        # Allow uploaded files to be read only by user
        upload_store_access user:rw group:rw all:rw;
#       upload_store_access 066

        # Set specified fields in request body
        upload_set_form_field $upload_field_name.name "$upload_file_name";
        upload_set_form_field $upload_field_name.content_type "$upload_content_type";
        upload_set_form_field $upload_field_name.path "$upload_tmp_path";

        # Inform backend about hash and size of a file
#        upload_aggregate_form_field "$upload_field_name.sha1" "$upload_file_sha1";
#        upload_aggregate_form_field "$upload_field_name.size" "$upload_file_size";

        upload_pass_form_field "^submit$|^description$|^token$|^qqfile$|^file1$|^chunk$|^chunks$|^chunk_base64_sha_hash$|^storage_path$|^storage_unit$|^server$";

    if ($request_method = 'OPTIONS') {
           add_header Access-Control-Allow-Origin '*';
           add_header Access-Control-Allow-Methods 'POST, OPTIONS';
           add_header Access-Control-Max-Age '1728000';
           add_header Content-Type: 'text/plain; charset=UTF-8';
           add_header Access-Control-Allow-Headers 'X-CSRF-Token';
           return 200;
        }

        # FILE UPLOADS
        if ($request_method = 'POST') {
           add_header Access-Control-Allow-Origin: '*';
        }

        upload_cleanup 400 404 499 500-505;
    }


    location @test {
        proxy_pass http://127.0.0.1:8090;
    }
}

setup a basic sinatra app to return something valid after a POST and ran it on port 8090

used the following script to exercise nginx:

#!/bin/bash

while true; do
#/etc/init.d/nginx reload
killall -HUP nginx
curl -D - -F [email protected] http://server/upload/1
curl -D - -F [email protected] http://server/upload/2
sleep 3

doesn't take log (ie within 5-10 iterations) before nginx worker will segfault.

from nginx-upload-module.

carolinebeauchamp avatar carolinebeauchamp commented on July 18, 2024

One observed phenomenon was that only having one upload section made it not fail.

Also, with two upload sections, only hitting /1 would also not trigger the segfault, but an immediate hit on /2 would segfault.

Does anyone have any ideas of where to look next?

Thanks

from nginx-upload-module.

carolinebeauchamp avatar carolinebeauchamp commented on July 18, 2024

Following hunches:

Added in a:

upload_state_store

directive into the upload location sections, same format as upload_store, and I'm not getting segfaults in all the tests I'm doing (and am getting segfaults when I don't have that directive in there).

So something up with default values for state_path?

Chris

from nginx-upload-module.

khavishbhundoo avatar khavishbhundoo commented on July 18, 2024

@fdintino This issue can also be closed

from nginx-upload-module.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.