resp,iorust

infinite loop in Decoder::decode recursion

my custom redis_server impl with resp v1.0.3, but the CPU usage keep 100% when I parse multi large hset requests at same time

top -h

 571600 w         20   0 3545920  85680  39000 R  99.9   0.1  42:01.76 redis_server                                                                                                         
 571785 w         20   0 3545920  85680  39000 R  99.9   0.1  48:01.25 redis_server                                                                                                         
 571786 w         20   0 3545920  85680  39000 R  99.9   0.1  48:01.69 redis_server                                                                                                         
 571787 w         20   0 3545920  85680  39000 R  99.9   0.1  48:00.53 redis_server

sudo pstack -p 571600

Thread 1 (Thread 0x7f380377a6c0 (LWP 571600) "redis_server"):
#0  0x0000555a0b8bd161 in core::ptr::metadata::from_raw_parts_mut<u8> () at /rustc/29e4a9ee0253cd39e552a77f51f11f9a5f1c41e6/library/core/src/slice/iter/macros.rs:144
#1  core::ptr::null_mut<u8> () at /rustc/29e4a9ee0253cd39e552a77f51f11f9a5f1c41e6/library/core/src/ptr/mod.rs:668
#2  core::ptr::mut_ptr::{impl#0}::is_null<core::mem::maybe_uninit::MaybeUninit<u8>> () at /rustc/29e4a9ee0253cd39e552a77f51f11f9a5f1c41e6/library/core/src/ptr/mut_ptr.rs:38
#3  core::slice::iter::{impl#187}::next<core::mem::maybe_uninit::MaybeUninit<u8>> (self=0x7f38037774c0) at /rustc/29e4a9ee0253cd39e552a77f51f11f9a5f1c41e6/library/core/src/slice/iter/macros.rs:144
#4  0x0000555a0a6d89c8 in std::io::readbuf::ReadBuf::initialize_unfilled_to (self=0x7f3803777788, n=8192) at /rustc/29e4a9ee0253cd39e552a77f51f11f9a5f1c41e6/library/std/src/io/readbuf.rs:144
#5  0x0000555a0a6c114e in std::io::readbuf::ReadBuf::initialize_unfilled () at /rustc/29e4a9ee0253cd39e552a77f51f11f9a5f1c41e6/library/std/src/io/mod.rs:468
#6  std::io::default_read_buf<std::io::Read::read_buf::{closure_env#0}<std::net::tcp::TcpStream>> (read=..., buf=0x7f3803777788) at /rustc/29e4a9ee0253cd39e552a77f51f11f9a5f1c41e6/library/std/src/io/mod.rs:468
#7  0x0000555a0a6cae6e in std::io::Read::read_buf<std::net::tcp::TcpStream> (self=0x7f38037785a0, buf=0x7f3803777788) at /rustc/29e4a9ee0253cd39e552a77f51f11f9a5f1c41e6/library/std/src/io/mod.rs:812
#8  0x0000555a0a6cb107 in std::io::impls::{impl#0}::read_buf<std::net::tcp::TcpStream> (self=0x7f3803777780, buf=0x7f3803777788) at /rustc/29e4a9ee0253cd39e552a77f51f11f9a5f1c41e6/library/std/src/io/impls.rs:25
#9  0x0000555a0a6cfd5c in std::io::buffered::bufreader::buffer::Buffer::fill_buf<&mut std::net::tcp::TcpStream> (self=0x7f3803778580, reader=0x7f38037785a0) at /rustc/29e4a9ee0253cd39e552a77f51f11f9a5f1c41e6/library/std/src/io/buffered/bufreader/buffer.rs:98
#10 0x0000555a0a6d1801 in std::io::buffered::bufreader::{impl#4}::fill_buf<std::net::tcp::TcpStream> (self=0x7f3803778580) at /rustc/29e4a9ee0253cd39e552a77f51f11f9a5f1c41e6/library/std/src/io/buffered/bufreader.rs:368
#11 0x0000555a0a6c0dc4 in std::io::read_until<std::io::buffered::bufreader::BufReader<std::net::tcp::TcpStream>> (r=0x7f3803778580, delim=10, buf=0x7f3803777c38) at /rustc/29e4a9ee0253cd39e552a77f51f11f9a5f1c41e6/library/std/src/io/mod.rs:1902
#12 0x0000555a0a6d1245 in std::io::BufRead::read_until<std::io::buffered::bufreader::BufReader<std::net::tcp::TcpStream>> (self=0x7f3803778580, byte=10, buf=0x7f3803777c38) at /rustc/29e4a9ee0253cd39e552a77f51f11f9a5f1c41e6/library/std/src/io/mod.rs:2126
#13 0x0000555a0a6c16fb in resp::serialize::Decoder<std::net::tcp::TcpStream>::decode<std::net::tcp::TcpStream> (self=0x7f3803778580) at /home/w/.cargo/registry/src/rsproxy.cn-8f6827c7555bfaf8/resp-1.0.3/src/serialize.rs:142
#14 0x0000555a0a6c6ccb in redis_server::handle_connection (stream=..., ctx=0x7f3803bfc598) at crates/service/redis_server/src/lib.rs:85
#15 0x0000555a0a6c5bbb in redis_server::main::{closure#0}::{closure#0} () at crates/service/redis_server/src/lib.rs:67

sudo strace -p 571600 -s 4096

recvfrom(19, "", 8192, 0, NULL, NULL)   = 0
recvfrom(19, "", 8192, 0, NULL, NULL)   = 0
recvfrom(19, "", 8192, 0, NULL, NULL)   = 0
recvfrom(19, "", 8192, 0, NULL, NULL)   = 0
recvfrom(19, "", 8192, 0, NULL, NULL)   = 0
recvfrom(19, "", 8192, 0, NULL, NULL)   = 0
recvfrom(19, "", 8192, 0, NULL, NULL)   = 0
recvfrom(19, "", 8192, 0, NULL, NULL)   = 0
recvfrom(19, "", 8192, 0, NULL, NULL)   = 0

proposal: return EOF error kind when Decoder.decode read len is 0

I used bb8-redis on my app as redis client and my custom redis server with resp crate

but I found some error log in my custom redis server: too short: 0

resp/src/serialize.rs

Lines 140 to 147 in 107d804

    
           pub fn decode(&mut self) -> Result<Value> { 
        
               let mut res: Vec<u8> = Vec::new(); 
        
               self.reader.read_until(b'\n', &mut res)?; 
        
               let len = res.len(); 
        
               if len < 3 { 
        
                   return Err(Error::new(ErrorKind::InvalidInput, format!("too short: {}", len))); 
        
               }

I hope resp crate can return std::io::ErrorKind::UnexpectedEof when steam.read return 0 (This means EOF on TCP stream)

then my custom server can match EOF error to clean up some resource

Simple question

Maybe my question is stupid, but i tried to use your crate, and i just want to understand why decoder only take a BufReader,
why not a simple string for example ? Performance reason ?
In my use case i have string like this, that i want to parse "*2\r\n$4\r\nECHO\r\n$3\r\nhey\r\n".

iorust / resp Goto Github PK

resp's People

Contributors

Stargazers

Watchers

Forkers

resp's Issues

infinite loop in Decoder::decode recursion

proposal: return EOF error kind when Decoder.decode read len is 0

Simple question

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent

	pub fn decode(&mut self) -> Result<Value> {
	let mut res: Vec<u8> = Vec::new();
	self.reader.read_until(b'\n', &mut res)?;

	let len = res.len();
	if len < 3 {
	return Err(Error::new(ErrorKind::InvalidInput, format!("too short: {}", len)));
	}