Code Monkey home page Code Monkey logo

nginx-tfs's Introduction

名称

  • nginx-tfs

描述

  • 这个模块实现了TFS的客户端,为TFS提供了RESTful API。TFS的全称是Taobao File System,是淘宝开源的一个分布式文件系统。

编译安装

  1. TFS模块使用了一个开源的JSON库来支持JSON,请先安装yajl-2.0.1。

  2. 下载nginxtengine

  3. ./configure --add-module=/path/to/nginx-tfs

  4. make && make install

配置

http {
    tfs_upstream tfs_rc {
        server 127.0.0.1:6100;
        type rcs;
        rcs_zone name=tfs1 size=128M;
        rcs_interface eth0;
        rcs_heartbeat lock_file=/logs/lk.file interval=10s;
    }
    
    server {
          listen       7500;
          server_name  localhost;

          tfs_keepalive max_cached=100 bucket_count=10;
          tfs_log "pipe:/usr/sbin/cronolog -p 30min /path/to/nginx/logs/cronolog/%Y/%m/%Y-%m-%d-%H-%M-tfs_access.log";

          location / {
              tfs_pass tfs://tfs_rc;
          }
    }
}

指令

server

Syntaxserver address

Defaultnone

Contexttfs_upstream

指定后端TFS服务器的地址,当指令typercs时为RcServer的地址,如果为为ns时为NameServer的地址。此指令必须配置。例如:

server 10.0.0.1:8108;

type

Syntaxtype [ns | rcs]

Defaultns

Contexttfs_upstream

设置server类型,类型只能为ns或者rcs,如果为ns,则指令server指定的地址为NameServer的地址,如果为rcs,则为RcServer的地址。如需使用自定义文件名功能请设置类型为rcs,使用自定义文件名功能需额外配置MetaServer和RootServer。

rcs_zone

Syntaxrcs_zone name=n size=num

Defaultnone

Contexttfs_upstream

配置TFS应用在RcServer的配置信息。若开启RcServer(配置了type rcs),则必须配置此指令。配置此指令会在共享内存中缓存TFS应用在RcServer的配置信息,并可以通过指令rcs_heartbeat来和RcServer进行keepalive以保证应用的配置信息的及时更新。例如:

rcs_zone name=tfs1 size=128M;

rcs_heartbeat

Syntaxrcs_heartbeat lock_file=/path/to/file interval=time

Defaultnone

Contexttfs_upstream

配置TFS应用和RcServer的keepalive,应用可通过此功能来和RcServer定期交互,以及时更新其配置信息。若开启RcServer功能(配置了type rcs),则必须配置此指令。例如:

rcs_heartbeat lock_file=/path/to/nginx/logs/lk.file interval=10s;

rcs_interface

Syntaxrcs_interface interface

Defaultnone

Contexttfs_upstream

配置TFS模块使用的网卡。若开启RcServer功能(配置了type rcs),则必须配置此指令。例如:

rcs_interface eth0;

tfs_upstream

Syntaxtfs_upstream name {...}

Defaultnone

Contexthttp

配置TFS模块的server信息,这个块包括上面几个命令。例如:

tfs_upstream tfs_rc {
    server 127.0.0.1:6100;
    type rcs;
    rcs_zone name=tfs1 size=128M;
    rcs_interface eth0;
    rcs_heartbeat lock_file=/logs/lk.file interval=10s;
}

tfs_pass

Syntaxtfs_pass name

Defaultnone

Contextlocation

是否打开TFS模块功能,此指令为关键指令,决定请求是否由TFS模块处理,必须配置。需要注意,这里不支持直接写ip地址或者域名,这里只支持指令tfs_upstream name {...} 配置的upstream,并且必须以 tfs:// 开头。例如:

tfs_upstream tfs_rc {
};

location / {
	tfs_pass tfs://tfs_rc;
}

tfs_keepalive

Syntaxtfs_keepalive max_cached=num bucket_count=num

Defaultnone

Contexthttp, server

配置TFS模块使用的连接池的大小,TFS模块的连接池会缓存TFS模块和后端TFS服务器的连接。可以把这个连接池看作由多个hash桶构成的hash表,其中bucket_count是桶的个数,max_cached是桶的容量。此指令必须配置。注意,应该根据机器的内存情况来合理配置连接池的大小。例如:

tfs_keepalive max_cached=100 bucket_count=15;

tfs_block_cache_zone

Syntaxtfs_block_cache_zone size=num

Defaultnone

Contexthttp

配置TFS模块的本地BlockCache。配置此指令会在共享内存中缓存TFS中的Block和DataServer的映射关系。注意,应根据机器的内存情况来合理配置BlockCache大小。例如:

tfs_block_cache_zone size=256M;

tfs_log

Syntaxtfs_log path

Defaultnone

Contexthttp, server

是否进行TFS访问记录。配置此指令会以固定格式将访问TFS的请求记录到指定log中,以便进行分析。具体格式参见代码。例如:

tfs_log "pipe:/usr/sbin/cronolog -p 30min /path/to/nginx/logs/cronolog/%Y/%m/%Y-%m-%d-%H-%M-tfs_access.log";

注:cronolog支持依赖于tengine提供的扩展的日志模块。

tfs_body_buffer_size

Syntaxtfs_body_buffer_size size

Default8k|16k

Contexthttp, server, location

配置用于和后端TFS服务器交互时使用的的body_buffer的大小。默认为页大小的2倍。建议设为2m。例如:

tfs_body_buffer_size 2m;

tfs_connect_timeout

Syntaxtfs_connect_timeout time

Default3s

Contexthttp, server, location

配置连接后端TFS服务器的超时时间。

tfs_send_timeout

Syntaxtfs_send_timeout time

Default3s

Contexthttp, server, location

配置向后端TFS服务器发送数据的超时时间。

tfs_read_timeout

Syntaxtfs_read_timeout time

Default3s

Contexthttp, server, location

配置从后端TFS服务器接收数据的超时时间。

其他

能支持上传文件大小决定于client_max_body_size指令配置的大小。

nginx-tfs's People

Contributors

monadbobo avatar zhcn381 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

nginx-tfs's Issues

tfs递归创建文件时怎么第一次操作是创建目录而不是文件?

tfs递归创建文件时怎么第一次操作是创建目录而不是文件?

想创建/cc/cc/cc文件,却创建/cc/cc/cc目录。
[root@tfsdatasvr1 ~]# curl -i -XPOST http://tfs.test.com/v2/tappkey00001/1/1030/file/cc/cc/cc?recursive=1
HTTP/1.1 201 Created
Server: Tengine/2.1.2
Date: Mon, 27 Feb 2017 03:06:59 GMT
Content-Length: 0
Connection: keep-alive

[root@tfsdatasvr1 ~]# curl http://tfs.test.com/v2/tappkey00001/1/1030/dir/cc
[
{
"NAME": "cc",
"PID": 43489476,
"ID": 43489477,
"SIZE": 0,
"IS_FILE": false,
"CREATE_TIME": "Mon, 27 Feb 2017 02:47:12 GMT",
"MODIFY_TIME": "Mon, 27 Feb 2017 02:47:12 GMT",
"VER_NO": 0
}
]

nginx-tfs 不能将大于2M的文件写入到TFS

问题描述:
通过nginx-tfs上传大于2m的文件到TFS失败,nginx-tfs返回500。而使用TFS client可以正常写入大于2m的文件。在dataserver的日志发现, dataserver 解析nginx-tfs传来的数据报,取到数据包长度为负值。会议nginx-tfs存在bug。

nginx-tfs 相关的配置如下:
## TFS
tfs_send_timeout 3s;
tfs_read_timeout 3s;
tfs_connect_timeout 3s;
tfs_body_buffer_size 10M;
tfs_block_cache_zone size=256M;
tfs_keepalive max_cached=128 bucket_count=16;

nginx的debug日志如下:
2013/10/09 16:15:20 [info] 19460#0: 8 meta segment: block_id: 0, fileid: 0, seq_id: 0, suffix: 0, client: 127.0.0.1, server: tfs._.com, request: "POST /v1/tfs HTTP/1.1", host: "tfs..com"
2013/10/09 16:15:20 [info] 19460#0: *8 get block info from ns while connecting server, client: 127.0.0.1, server: tfs.**.com, request: "POST /v1/tfs HTTP/1.1", host: "tfs.**
.com"
2013/10/09 16:15:20 [debug] 19460#0: *8 connecting name server, addr: 10.33.56.125:8108
2013/10/09 16:15:20 [info] 19460#0: *8 http tfs finalize state name server, 0 while reading response header from tfs, client: 127.0.0.1, server: tfs.**.com, request: "POST /v1/tfs HTTP/1.1", host: "tfs.__.com"
2013/10/09 16:15:20 [info] 19460#0: *8 http tfs process next peer is data server, addr: 10.33.56.198:8010 while reading response header from tfs, client: 127.0.0.1, server: tfs._
.com, request: "POST /v1/tfs HTTP/1.1", host: "tfs.**__.com"
2013/10/09 16:15:20 [debug] 19460#0: *8 connecting data server, addr: 10.33.56.198:8010
2013/10/09 16:15:20 [info] 19460#0: *8 http tfs finalize state data server, 0 while reading response header from tfs, client: 127.0.0.1, server: tfs.
_.com, request: "POST /v1/tfs HTTP/1.1", host: "tfs..com"
2013/10/09 16:15:20 [info] 19460#0: *8 http tfs process next peer is data server, addr: 10.33.56.198:8010 while reading response header from tfs, client: 127.0.0.1, server: tfs.**.com, request: "POST /v1/tfs HTTP/1.1", host: "tfs.**
.com"
2013/10/09 16:15:20 [info] 19460#0: 8 write segment index 0, block id: 2404301, file id: 2354, offset: 0, length: 2097152, crc: 2021805896 while connecting server, client: 127.0.0.1, server: tfs.**.com, request: "POST /v1/tfs HTTP/1.1", host: "tfs.__.com"
2013/10/09 16:15:20 [debug] 19460#0: *8 connecting data server, addr: 10.33.56.198:8010
2013/10/09 16:15:20 [error] 19460#0: *8 readv() failed (104: Connection reset by peer) while reading response header from tfs, client: 127.0.0.1, server: tfs._
.com, request: "POST /v1/tfs HTTP/1.1", host: "tfs.**__.com"
2013/10/09 16:15:20 [error] 19460#0: *8 recv chain error while reading response header from tfs, client: 127.0.0.1, server: tfs.
_.com, request: "POST /v1/tfs HTTP/1.1", host: "tfs._
.com"

TFS 使用2.2.13 版本。dataserver的报错日志为:
2013-10-09 18:07:22] ERROR getPacketInfo (base_packet_streamer.cpp:86) [139896236799744] stream error: 69fc7285<>4d534654,4e534654, dataLen: 432408354
[2013-10-09 18:07:22] ERROR reply (base_packet.cpp:221) [139896073172736] post packet failure, server: 10.33.56.23:55566, pcode:1
[2013-10-09 18:07:22] ERROR callback (dataservice.cpp:854) [139896073172736] write data fail. filenumber: 14152088283100828036, blockid: 2399407, fileid: 2915, version: 2914, leaseid: 1198248338, role: master

[2013-10-09 17:28:43] ERROR getPacketInfo (base_packet_streamer.cpp:86) [139896236799744] stream error: ebe940fc<>4d534654,4e534654, dataLen: -1052153523
[2013-10-09 17:28:43] ERROR reply (base_packet.cpp:221) [139896167581440] post packet failure, server: 10.81.102.121:54715, pcode:1
[2013-10-09 17:28:43] ERROR callback (dataservice.cpp:854) [139896167581440] write data fail. filenumber: 14152088283100823092, blockid: 2407850, fileid: 1621, version: 1620, leaseid: 1197348515, role: master

同一个文件有时获取到的大小不一致

日志:

10.81.254.51 - - [18/Aug/2014:17:04:47 +0800] "GET /tfs/v1/tfs/T1Bvh8BTZ_1RCvBVdK HTTP/1.1" 200 909787 "-" "Golang rpc package" "-" 7.765 7.312
10.81.254.51 - - [18/Aug/2014:17:04:48 +0800] "GET /tfs/v1/tfs/T1Bvh8BTZ_1RCvBVdK HTTP/1.1" 200 863255 "-" "Golang rpc package" "-" 1.555 1.342

返回的大小相差不小。

nginx -V:

nginx version: ngx_openresty/1.4.3.9
built by gcc 4.4.7 20120313 (Red Hat 4.4.7-3) (GCC)
TLS SNI support enabled
configure arguments: --prefix=/home/server/openresty/nginx --add-module=../ngx_devel_kit-0.2.19 --add-module=../iconv-nginx-module-0.10 --add-module=../echo-nginx-module-0.50 --add-module=../xss-nginx-module-0.04 --add-module=../ngx_coolkit-0.2rc1 --add-module=../set-misc-nginx-module-0.23 --add-module=../form-input-nginx-module-0.07 --add-module=../encrypted-session-nginx-module-0.03 --add-module=../ngx_postgres-1.0rc3 --add-module=../srcache-nginx-module-0.24 --add-module=../ngx_lua-0.9.3 --add-module=../headers-more-nginx-module-0.24 --add-module=../array-var-nginx-module-0.03 --add-module=../memc-nginx-module-0.14 --add-module=../redis2-nginx-module-0.10 --add-module=../redis-nginx-module-0.3.7 --add-module=../auth-request-nginx-module-0.2 --add-module=../rds-json-nginx-module-0.13 --add-module=../rds-csv-nginx-module-0.05 --with-ld-opt=-Wl,-rpath,/home/server/openresty/luajit/lib --with-http_ssl_module --with-http_stub_status_module --add-module=/home/makerpm/git/nginx-auth-ldap --add-module=/home/makerpm/git/nginx_upstream_hash --add-module=/home/makerpm/git/ngx_http_substitutions_filter_module --add-module=/home/makerpm/git/nginx-tfs --add-module=/home/makerpm/git/nginx-upstream-fair --with-pcre=/home/makerpm/tar/pcre-8.34 --with-pcre-jit --with-openssl=/home/makerpm/tar/openssl-1.0.1h

nginx-tfs 模块是从 tenginx 里面拷贝出来的。

使用 tengine-2.1.0添加nginx-tfs插件出错,无法编译。

Hi,
我正在使用TFS,想使用restapi,发现没办法编译过去。
cc1: warnings being treated as errors
../nginx-tfs-master//ngx_http_tfs_module.c: 在函数‘ngx_http_tfs_tackle_log’中:
../nginx-tfs-master//ngx_http_tfs_module.c:517: 错误:隐式声明函数‘ngx_log_create’
../nginx-tfs-master//ngx_http_tfs_module.c:517: 错误:赋值时将整数赋给指针,未作类型转换
../nginx-tfs-master//ngx_http_tfs_module.c:526: 错误:隐式声明函数‘ngx_log_set_levels’
../nginx-tfs-master//ngx_http_tfs_module.c:526: 错误:返回时将整数赋给指针,未作类型转换
make[1]: *** [objs/addon/nginx-tfs-master/ngx_http_tfs_module.o] 错误 1
make[1]: Leaving directory `/root/tengine-2.1.0'
make: *** [build] 错误 2

去掉MakeFile 的-Werror 选项会出现下面的错误:
objs/addon/nginx-tfs-master/ngx_http_tfs_module.o: In function ngx_http_tfs_tackle_log': /root/tengine-2.1.0/../nginx-tfs-master//ngx_http_tfs_module.c:517: undefined reference to ngx_log_create'
/root/tengine-2.1.0/../nginx-tfs-master//ngx_http_tfs_module.c:526: undefined reference to `ngx_log_set_levels'

请问我该设置什么才能编译过去?

ngx_openresty 的 ngx.location.capture 指令直接调用 nginx-tfs 模块会 blocked 无法返回

类似的问题:

http://code.taobao.org/p/tfs/issue/1312/

相关信息:

系统信息

$ uname -a
Linux tfs004 2.6.32-279.el6.x86_64 #1 SMP Fri Jun 22 12:19:21 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

ngx_openresty 版本

$ ./nginx -V
nginx version: ngx_openresty/1.2.7.1
built by gcc 4.4.6 20120305 (Red Hat 4.4.6-4) (GCC)
TLS SNI support enabled
configure arguments: --prefix=/home/server/openresty-debug/nginx --with-debug --with-cc-opt=-O0 --add-module=../ngx_devel_kit-0.2.18 --add-module=../iconv-nginx-module-0.10rc7 --add-module=../echo-nginx-module-0.42 --add-module=../xss-nginx-module-0.03rc9 --add-module=../ngx_coolkit-0.2rc1 --add-module=../set-misc-nginx-module-0.22rc8 --add-module=../form-input-nginx-module-0.07 --add-module=../encrypted-session-nginx-module-0.02 --add-module=../ngx_postgres-1.0rc2 --add-module=../srcache-nginx-module-0.19 --add-module=../ngx_lua-0.7.16 --add-module=../headers-more-nginx-module-0.19 --add-module=../array-var-nginx-module-0.03rc1 --add-module=../memc-nginx-module-0.13rc3 --add-module=../redis2-nginx-module-0.09 --add-module=../redis-nginx-module-0.3.6 --add-module=../auth-request-nginx-module-0.2 --add-module=../rds-json-nginx-module-0.12rc10 --add-module=../rds-csv-nginx-module-0.05rc2 --with-ld-opt=-Wl,-rpath,/home/server/openresty-debug/luajit/lib --with-http_ssl_module --with-http_stub_status_module --add-module=/home/makerpm/git/nginx-auth-ldap --add-module=/home/makerpm/git/nginx_upstream_hash --add-module=/home/makerpm/git/ngx_http_substitutions_filter_module --add-module=/home/makerpm/git/nginx-tfs

nginx-tfs 模块最新的 commit

$ git log -1
commit d1c8349
Merge: dcb7507 83d9a33
Author: Zheng Cen [email protected]
Date: Thu Jan 17 03:03:57 2013 -0800

Merge pull request #12 from monadbobo/fix_upstream

Fix upstream

重现问题的步骤

(1) 使用下面的 nginx.conf 启动 nginx

nginx.conf: https://gist.github.com/harveyzh/5055847#file-nginx-conf

(2) ngx.location.capture 指令直接调用 nginx-tfs 模块会一直 block

测试命令:
$ curl -i 127.0.0.1/test_block
^C

相关配置片段:

    location /test_block {
        content_by_lua '
            res = ngx.location.capture("/v1/tfs/T17yCTByJT1RCvBVdK")

            if res.status == ngx.HTTP_OK then
                ngx.print(res.body)
            else
                ngx.exit(res.status)
            end
        ';
    }

debug log: https://gist.github.com/harveyzh/5055847#file-error-log-block

block时的 debug log 停在

2013/02/28 18:41:06 [debug] 29836#0: *6 http copy filter: 0 "/v1/tfs/T17yCTByJT1RCvBVdK?"
2013/02/28 18:41:06 [debug] 29836#0: *6 http finalize request: 0, "/v1/tfs/T17yCTByJT1RCvBVdK?" a:0, c:2
2013/02/28 18:41:06 [debug] 29836#0: *6 lua run post subrequest handler, rc:0 c:2
2013/02/28 18:41:06 [debug] 29836#0: *6 lua restoring write event handler
2013/02/28 18:41:06 [debug] 29836#0: *6 http wake parent request: "/test_block?"
2013/02/28 18:41:06 [debug] 29836#0: keepalive dummy handler
2013/02/28 18:41:06 [debug] 29836#0: timer delta: 1
2013/02/28 18:41:06 [debug] 29836#0: posted events 0000000000000000
2013/02/28 18:41:06 [debug] 29836#0: worker cycle
2013/02/28 18:41:06 [debug] 29836#0: epoll timer: -1

(3) ngx.location.capture 指令间接通过 proxy_pass 调用 nginx-tfs 模块则 ok

测试命令:
$ curl -i 127.0.0.1/test_ok
HTTP/1.1 200 OK
Server: ngx_openresty
Date: Thu, 28 Feb 2013 10:59:44 GMT
Content-Type: application/octet-stream
Transfer-Encoding: chunked
Connection: keep-alive

test!

相关配置片段:

    location /tfs/ {
        proxy_pass http://127.0.0.1/v1/tfs/;
    }

    location /test_ok {
        content_by_lua '
            res = ngx.location.capture("/tfs/T17yCTByJT1RCvBVdK")

            if res.status == ngx.HTTP_OK then
                ngx.print(res.body)
            else
                ngx.exit(res.status)
            end
        ';
    }

debug log: https://gist.github.com/harveyzh/5055847#file-error-log-ok

parse uri failed

发现tfs_pass只能配置在 "/" location下面,如果该nginx不光处理tfs,还可能有其他业务的配置,感觉让tfs走”/“ location有点不妥。

how to wirte a file under directory

Hi, I noticed that there are APIs for creating and manipulating directories, but missing docs describing how to store a file under dir, So, could you give me some hints for this? thanks.

编译报错:在函数‘ngx_http_tfs_process_upstream_request’中:

/usr/local/src/nginx-tfs/src/ngx_http_tfs.c: 在函数‘ngx_http_tfs_process_upstream_request’中:
/usr/local/src/nginx-tfs/src/ngx_http_tfs.c:874: 错误:提供给函数‘c->recv_chain’的实参太少
make[1]: *** [objs/addon/src/ngx_http_tfs.o] 错误 1
make[1]: *** 正在等待未完成的任务....
cc1: warnings being treated as errors
/usr/local/src/nginx-tfs/src/ngx_http_tfs_module.c: 在函数‘ngx_http_tfs_log’中:
/usr/local/src/nginx-tfs/src/ngx_http_tfs_module.c:684: 错误:隐式声明函数‘ngx_log_create’
/usr/local/src/nginx-tfs/src/ngx_http_tfs_module.c:684: 错误:赋值时将整数赋给指针,未作类型转换
/usr/local/src/nginx-tfs/src/ngx_http_tfs_module.c:693: 错误:隐式声明函数‘ngx_log_set_levels’
/usr/local/src/nginx-tfs/src/ngx_http_tfs_module.c:693: 错误:返回时将整数赋给指针,未作类型转换
make[1]: *** [objs/addon/src/ngx_http_tfs_module.o] 错误 1
make[1]: Leaving directory `/usr/local/src/nginx-1.10.3'
make: *** [build] 错误 2

编译一直报这个错,这里用的是nginx-1.10.3版本,换了其它版本以及tengine也不行

执行写文件(write_file)接口失败

执行创建文件(create_file) POST /v2/appkey/appid/uid/file/file_name HTTP/1.1接口可以正确返回201状态,但是在执行写文件(write_file) PUT /v2/appkey/appid/uid/file/file_name HTTP/1.1接口时,nginx err.log中出现如下信息:
2017/12/06 17:50:14 [error] 34442#0: *1257 error, cluster id conflict: 1(ns) <> 54(rcs) while reading response header from tfs, client: 192.168.10.57, server: localhost, request: "PUT /v2/tappkey00001/1/21312/file/file1?offset=213213213&size=2436 HTTP/1.1", host: "192.168.10.128:7500" 2017/12/06 17:50:14 [error] 34442#0: *1257 http tfs process name server request failed while reading response header from tfs, client: 192.168.10.57, server: localhost, request: "PUT /v2/tappkey00001/1/21312/file/file1?offset=213213213&size=2436 HTTP/1.1", host: "192.168.10.128:7500"
请大神帮忙解惑!小弟不胜感激!

测试restful API所遇到的一些问题

1.在post数据的时候如果不加suffix=这个参数会提示400错误,然后加了之后就可以提交数据了,然后我多加了个simple_name参数就又报400错误,然后看你api说明的时候这两个不加的时候也应该是可以的,加了两个也可以,现在这个算是bug吗?
2.我用post提交完数据之后返回文件名了,然后用这个文件名想用restful api的get接口搞下来,但始终无效,报404 not found,然后用后台的nameserver查看那个文件名发现是已经提交上去了的,请问这是什么原因?

nginx 每次reload都会有一部分work进程一直在shutting down状态

我找其中一个等待shutdown的进程,跟踪下结果如下,发现有Resource temporarily unavailable不知道什么原因,这个问题只有使用nginx-tfs时才会出现,版本tengine-2.1.0和tengine2.1.2上都会出现,有的进程shutdown状态持续到几个月。
strace -p 31072

Process 31072 attached - interrupt to quit
epoll_wait(27, {}, 512, 7029) = 0
epoll_wait(27, {}, 512, 1) = 0
socket(PF_INET, SOCK_STREAM, IPPROTO_IP) = 2682
ioctl(2682, FIONBIO, [1]) = 0
epoll_ctl(27, EPOLL_CTL_ADD, 2682, {EPOLLIN|EPOLLOUT|EPOLLET|0x2000, {u32=495118000, u64=140699328767664}}) = 0
connect(2682, {sa_family=AF_INET, sin_port=htons(5100), sin_addr=inet_addr("172.16.0.198")}, 16) = -1 EINPROGRESS (Operation now in progress)
epoll_wait(27, {{EPOLLOUT, {u32=495118000, u64=140699328767664}}}, 512, 9999) = 1
getsockopt(2682, SOL_SOCKET, SO_ERROR, [102868521188327424], [4]) = 0
writev(2682, [{"TFSN\r\1\0\0D\0\2\0f\20\r\0\0\0\0\0gi-\3632\0\0\0002-31"..., 293}], 1) = 293
epoll_wait(27, {{EPOLLIN|EPOLLOUT, {u32=495118000, u64=140699328767664}}}, 512, 9997) = 1
readv(2682, [{"TFSN\1\0\0\0E\0\2\0f\20\r\0\0\0\0\0\253\306Hl", 24}, {"\0\311\377\377\27\0\0\0execute message failed\0\2"..., 20971520}], 2) = 25
recvfrom(2682, 0x7fff6b9ed65f, 1, 2, 0, 0) = -1 EAGAIN (Resource temporarily unavailable)
getsockopt(2682, SOL_SOCKET, SO_ERROR, [102868521188327424], [4]) = 0
writev(2682, [{"TFSN\r\1\0\0D\0\2\0g\20\r\0\0\0\0\0\23\310\36232\0\0\0001-31"..., 293}], 1) = 293
readv(2682, 0x7fff6b9ed4c0, 2) = -1 EAGAIN (Resource temporarily unavailable)
epoll_wait(27, {{EPOLLIN|EPOLLOUT, {u32=495118000, u64=140699328767664}}}, 512, 9996) = 1
readv(2682, [{"TFSN\1\0\0\0E\0\2\0g\20\r\0\0\0\0\0\253\306Hl", 24}, {"\0\311\377\377\27\0\0\0execute message failed\0\2"..., 20971520}], 2) = 25
recvfrom(2682, 0x7fff6b9ed60f, 1, 2, 0, 0) = -1 EAGAIN (Resource temporarily unavailable)
close(2682) = 0

nginx-tfs 模块无法和 nginx-1.7.0 编译

objs/addon/src/ngx_http_tfs_module.o: In function `ngx_http_tfs_log':
/home/makerpm/git/nginx-tfs/src/ngx_http_tfs_module.c:686: undefined reference to `ngx_log_create'
/home/makerpm/git/nginx-tfs/src/ngx_http_tfs_module.c:695: undefined reference to `ngx_log_set_levels'
collect2: ld returned 1 exit status
gmake[2]: *** [objs/nginx] Error 1
gmake[2]: Leaving directory `/home/makerpm/tar/ngx_openresty-1.7.0.1/build/nginx-1.7.0'
gmake[1]: *** [build] Error 2
gmake[1]: Leaving directory `/home/makerpm/tar/ngx_openresty-1.7.0.1/build/nginx-1.7.0'
gmake: *** [all] Error 2

ngx_log_create 被不存在了。

请教:在通过http://xxx:7500/v1/tfs 上传文件时,如何加鉴权?

请教:在通过http://xxx:7500/v1/tfs 上传文件时,如何加鉴权?
比如想在客户端http POST时在head中加一个从用户系统得到的token,然后请求TFS时如何鉴权这个token?
目前的想法独立做一个上传文件的WEB应用,在这里做head token鉴权,通过后再转发到TFS上。这样的话就多了一个应用。我们nginx-tfs目前有没有类似的鉴权方法?盼复~~

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.