Comments (8)
ashie
commented on May 25, 2024
2
In #4342 we changed the code related to this.
Although we aren't sure yet whether it's the cause of this issue or not, #4342 obviously lacks mutex lock until block.call is called for chunks. We need to fix it.
from fluentd.
daipom
commented on May 25, 2024
1
(Because the staged chunk being handled in Buffer:write can be flushed and purged by the flush thread, although I haven't confirmed it in the actual behavior yet.)
I have reproduced this issue using the following config with v1.16.4.
And, this does not reproduce with v1.16.3 or after #4447.
<source>
@type sample
tag test
size 10000
</source>
<source>
@type sample
tag test
size 50000
</source>
<source>
@type sample
tag test
size 100000
</source>
<match test.**>
@type null
<buffer tag, time>
@type file
path /test/fluentd/buffer/
flush_thread_count 2
timekey 2s
timekey_wait 0
flush_mode interval
flush_interval 3s
chunk_limit_size 1MB
chunk_full_threshold 0.9
overflow_action drop_oldest_chunk
</buffer>
</match>Fluentd logs:
2024-03-27 16:39:43 +0900 [info]: #0 fluentd worker is now running worker=0
2024-03-27 16:39:44 +0900 [warn]: #0 chunk bytes limit exceeds for an emitted event stream: 1350000bytes
2024-03-27 16:39:44 +0900 [warn]: #0 chunk bytes limit exceeds for an emitted event stream: 2700000bytes
2024-03-27 16:39:44 +0900 [warn]: #0 emit transaction failed: error_class=IOError error="not opened for writing" location="/home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer/file_chunk.rb:258:in `write'" tag="test"
2024-03-27 16:39:44 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer/file_chunk.rb:258:in `write'
2024-03-27 16:39:44 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer/file_chunk.rb:258:in `write_metadata'
2024-03-27 16:39:44 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer/file_chunk.rb:69:in `commit'
2024-03-27 16:39:44 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer.rb:377:in `write'
2024-03-27 16:39:44 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:1069:in `block in handle_stream_with_standard_format'
2024-03-27 16:39:44 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:977:in `write_guard'
2024-03-27 16:39:44 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:1068:in `handle_stream_with_standard_format'
2024-03-27 16:39:44 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:971:in `execute_chunking'
2024-03-27 16:39:44 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:897:in `emit_buffered'
2024-03-27 16:39:44 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/event_router.rb:115:in `emit_stream'
2024-03-27 16:39:44 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/event_router.rb:111:in `emit_array'
2024-03-27 16:39:44 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:110:in `block in emit'
2024-03-27 16:39:44 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:109:in `times'
2024-03-27 16:39:44 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:109:in `emit'
2024-03-27 16:39:44 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:98:in `run'
2024-03-27 16:39:44 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin_helper/thread.rb:78:in `block in thread_create'
2024-03-27 16:39:45 +0900 [warn]: #0 chunk bytes limit exceeds for an emitted event stream: 2700000bytes
2024-03-27 16:39:45 +0900 [warn]: #0 chunk bytes limit exceeds for an emitted event stream: 1350000bytes
2024-03-27 16:39:45 +0900 [warn]: #0 emit transaction failed: error_class=IOError error="closed stream" location="/home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer/file_chunk.rb:257:in `seek'" tag="test"
2024-03-27 16:39:45 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer/file_chunk.rb:257:in `seek'
2024-03-27 16:39:45 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer/file_chunk.rb:257:in `write_metadata'
2024-03-27 16:39:45 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer/file_chunk.rb:69:in `commit'
2024-03-27 16:39:45 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer.rb:377:in `write'
2024-03-27 16:39:45 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:1069:in `block in handle_stream_with_standard_format'
2024-03-27 16:39:45 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:977:in `write_guard'
2024-03-27 16:39:45 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:1068:in `handle_stream_with_standard_format'
2024-03-27 16:39:45 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:971:in `execute_chunking'
2024-03-27 16:39:45 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:897:in `emit_buffered'
2024-03-27 16:39:45 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/event_router.rb:115:in `emit_stream'
2024-03-27 16:39:45 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/event_router.rb:111:in `emit_array'
2024-03-27 16:39:45 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:110:in `block in emit'
2024-03-27 16:39:45 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:109:in `times'
2024-03-27 16:39:45 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:109:in `emit'
2024-03-27 16:39:45 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:98:in `run'
2024-03-27 16:39:45 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin_helper/thread.rb:78:in `block in thread_create'
2024-03-27 16:39:46 +0900 [warn]: #0 chunk bytes limit exceeds for an emitted event stream: 1350000bytes
2024-03-27 16:39:46 +0900 [warn]: #0 chunk bytes limit exceeds for an emitted event stream: 2700000bytes
2024-03-27 16:39:46 +0900 [warn]: #0 emit transaction failed: error_class=IOError error="closed stream" location="/home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer/file_chunk.rb:257:in `seek'" tag="test"
2024-03-27 16:39:46 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer/file_chunk.rb:257:in `seek'
2024-03-27 16:39:46 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer/file_chunk.rb:257:in `write_metadata'
2024-03-27 16:39:46 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer/file_chunk.rb:69:in `commit'
2024-03-27 16:39:46 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer.rb:377:in `write'
2024-03-27 16:39:46 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:1069:in `block in handle_stream_with_standard_format'
2024-03-27 16:39:46 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:977:in `write_guard'
2024-03-27 16:39:46 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:1068:in `handle_stream_with_standard_format'
2024-03-27 16:39:46 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:971:in `execute_chunking'
2024-03-27 16:39:46 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:897:in `emit_buffered'
2024-03-27 16:39:46 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/event_router.rb:115:in `emit_stream'
2024-03-27 16:39:46 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/event_router.rb:111:in `emit_array'
2024-03-27 16:39:46 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:110:in `block in emit'
2024-03-27 16:39:46 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:109:in `times'
2024-03-27 16:39:46 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:109:in `emit'
2024-03-27 16:39:46 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:98:in `run'
2024-03-27 16:39:46 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin_helper/thread.rb:78:in `block in thread_create'
2024-03-27 16:39:47 +0900 [warn]: #0 chunk bytes limit exceeds for an emitted event stream: 1350000bytes
2024-03-27 16:39:47 +0900 [warn]: #0 chunk bytes limit exceeds for an emitted event stream: 2700000bytes
2024-03-27 16:39:48 +0900 [warn]: #0 chunk bytes limit exceeds for an emitted event stream: 1350000bytes
2024-03-27 16:39:48 +0900 [warn]: #0 chunk bytes limit exceeds for an emitted event stream: 270000bytes
2024-03-27 16:39:48 +0900 [warn]: #0 emit transaction failed: error_class=IOError error="not opened for writing" location="/home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer/file_chunk.rb:258:in `write'" tag="test"
2024-03-27 16:39:48 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer/file_chunk.rb:258:in `write'
2024-03-27 16:39:48 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer/file_chunk.rb:258:in `write_metadata'
2024-03-27 16:39:48 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer/file_chunk.rb:69:in `commit'
2024-03-27 16:39:48 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer.rb:377:in `write'
2024-03-27 16:39:48 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:1069:in `block in handle_stream_with_standard_format'
2024-03-27 16:39:48 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:977:in `write_guard'
2024-03-27 16:39:48 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:1068:in `handle_stream_with_standard_format'
2024-03-27 16:39:48 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:971:in `execute_chunking'
2024-03-27 16:39:48 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:897:in `emit_buffered'
2024-03-27 16:39:48 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/event_router.rb:115:in `emit_stream'
2024-03-27 16:39:48 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/event_router.rb:111:in `emit_array'
2024-03-27 16:39:48 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:110:in `block in emit'
2024-03-27 16:39:48 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:109:in `times'
2024-03-27 16:39:48 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:109:in `emit'
2024-03-27 16:39:48 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:98:in `run'
2024-03-27 16:39:48 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin_helper/thread.rb:78:in `block in thread_create'
2024-03-27 16:39:48 +0900 [warn]: #0 chunk bytes limit exceeds for an emitted event stream: 2700000bytes
2024-03-27 16:39:49 +0900 [warn]: #0 chunk bytes limit exceeds for an emitted event stream: 1350000bytes
2024-03-27 16:39:49 +0900 [warn]: #0 chunk bytes limit exceeds for an emitted event stream: 2700000bytes
2024-03-27 16:39:49 +0900 [warn]: #0 emit transaction failed: error_class=IOError error="not opened for writing" location="/home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer/file_chunk.rb:258:in `write'" tag="test"
2024-03-27 16:39:49 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer/file_chunk.rb:258:in `write'
2024-03-27 16:39:49 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer/file_chunk.rb:258:in `write_metadata'
2024-03-27 16:39:49 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer/file_chunk.rb:69:in `commit'
2024-03-27 16:39:49 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer.rb:377:in `write'
2024-03-27 16:39:49 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:1069:in `block in handle_stream_with_standard_format'
2024-03-27 16:39:49 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:977:in `write_guard'
2024-03-27 16:39:49 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:1068:in `handle_stream_with_standard_format'
2024-03-27 16:39:49 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:971:in `execute_chunking'
2024-03-27 16:39:49 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:897:in `emit_buffered'
2024-03-27 16:39:49 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/event_router.rb:115:in `emit_stream'
2024-03-27 16:39:49 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/event_router.rb:111:in `emit_array'
2024-03-27 16:39:49 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:110:in `block in emit'
2024-03-27 16:39:49 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:109:in `times'
2024-03-27 16:39:49 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:109:in `emit'
2024-03-27 16:39:49 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:98:in `run'
2024-03-27 16:39:49 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin_helper/thread.rb:78:in `block in thread_create'
2024-03-27 16:39:50 +0900 [warn]: #0 chunk bytes limit exceeds for an emitted event stream: 2700000bytes
2024-03-27 16:39:50 +0900 [warn]: #0 chunk bytes limit exceeds for an emitted event stream: 1350000bytes
2024-03-27 16:39:50 +0900 [warn]: #0 emit transaction failed: error_class=IOError error="not opened for writing" location="/home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer/file_chunk.rb:258:in `write'" tag="test"
2024-03-27 16:39:50 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer/file_chunk.rb:258:in `write'
2024-03-27 16:39:50 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer/file_chunk.rb:258:in `write_metadata'
2024-03-27 16:39:50 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer/file_chunk.rb:69:in `commit'
2024-03-27 16:39:50 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer.rb:377:in `write'
2024-03-27 16:39:50 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:1069:in `block in handle_stream_with_standard_format'
2024-03-27 16:39:50 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:977:in `write_guard'
2024-03-27 16:39:50 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:1068:in `handle_stream_with_standard_format'
2024-03-27 16:39:50 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:971:in `execute_chunking'
2024-03-27 16:39:50 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:897:in `emit_buffered'
2024-03-27 16:39:50 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/event_router.rb:115:in `emit_stream'
2024-03-27 16:39:50 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/event_router.rb:111:in `emit_array'
2024-03-27 16:39:50 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:110:in `block in emit'
2024-03-27 16:39:50 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:109:in `times'
2024-03-27 16:39:50 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:109:in `emit'
2024-03-27 16:39:50 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:98:in `run'
2024-03-27 16:39:50 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin_helper/thread.rb:78:in `block in thread_create'
from fluentd.
ashie
commented on May 25, 2024
1
Great!
from fluentd.
daipom
commented on May 25, 2024
1
@larntz Thanks for your report!
from fluentd.
daipom
commented on May 25, 2024
I cannot deny the possibility of other unexpected causes, but, at least, #4342 's lack of lock can cause race condition and emit error.
(Because the staged chunk being handled in Buffer:write can be flushed and purged by the flush thread, although I haven't confirmed it in the actual behavior yet.)
I can't find any other cause for the emit error since v1.16.4.
So, at this time, I assume this race condition is the cause, and it will be fixed by #4447.
from fluentd.
daipom
commented on May 25, 2024
I confirmed 2 kinds of error
emit transaction failed: error_class=IOError error="not opened for writing"
2024-03-27 16:39:44 +0900 [warn]: #0 emit transaction failed: error_class=IOError error="not opened for writing" location="/home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer/file_chunk.rb:258:in `write'" tag="test"
2024-03-27 16:39:44 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer/file_chunk.rb:258:in `write'
2024-03-27 16:39:44 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer/file_chunk.rb:258:in `write_metadata'
2024-03-27 16:39:44 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer/file_chunk.rb:69:in `commit'
2024-03-27 16:39:44 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer.rb:377:in `write'
2024-03-27 16:39:44 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:1069:in `block in handle_stream_with_standard_format'
2024-03-27 16:39:44 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:977:in `write_guard'
2024-03-27 16:39:44 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:1068:in `handle_stream_with_standard_format'
2024-03-27 16:39:44 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:971:in `execute_chunking'
2024-03-27 16:39:44 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:897:in `emit_buffered'
2024-03-27 16:39:44 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/event_router.rb:115:in `emit_stream'
2024-03-27 16:39:44 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/event_router.rb:111:in `emit_array'
2024-03-27 16:39:44 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:110:in `block in emit'
2024-03-27 16:39:44 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:109:in `times'
2024-03-27 16:39:44 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:109:in `emit'
2024-03-27 16:39:44 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:98:in `run'
2024-03-27 16:39:44 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin_helper/thread.rb:78:in `block in thread_create'
emit transaction failed: error_class=IOError error="closed stream"
2024-03-27 16:39:46 +0900 [warn]: #0 emit transaction failed: error_class=IOError error="closed stream" location="/home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer/file_chunk.rb:257:in `seek'" tag="test"
2024-03-27 16:39:46 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer/file_chunk.rb:257:in `seek'
2024-03-27 16:39:46 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer/file_chunk.rb:257:in `write_metadata'
2024-03-27 16:39:46 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer/file_chunk.rb:69:in `commit'
2024-03-27 16:39:46 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/buffer.rb:377:in `write'
2024-03-27 16:39:46 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:1069:in `block in handle_stream_with_standard_format'
2024-03-27 16:39:46 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:977:in `write_guard'
2024-03-27 16:39:46 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:1068:in `handle_stream_with_standard_format'
2024-03-27 16:39:46 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:971:in `execute_chunking'
2024-03-27 16:39:46 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/output.rb:897:in `emit_buffered'
2024-03-27 16:39:46 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/event_router.rb:115:in `emit_stream'
2024-03-27 16:39:46 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/event_router.rb:111:in `emit_array'
2024-03-27 16:39:46 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:110:in `block in emit'
2024-03-27 16:39:46 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:109:in `times'
2024-03-27 16:39:46 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:109:in `emit'
2024-03-27 16:39:46 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_sample.rb:98:in `run'
2024-03-27 16:39:46 +0900 [warn]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin_helper/thread.rb:78:in `block in thread_create'
from fluentd.
daipom
commented on May 25, 2024
I don't know why only error="closed stream" happens on @larntz's environment.
At least, Buffer on v1.16.4 has race condition issue, and it can cause IOError and emit fail, and it will be fixed by #4447
from fluentd.
daipom
commented on May 25, 2024
We have released v1.16.5 to fix this issue.
from fluentd.
Related Issues (20)
- Windows: td-agent 4.5.2 Too many open files HOT 1
- Buffer files remain after restart of fluentd HOT 5
- stdout P followed my infinite loop of "\\\\\\\\\" in cloudwatch log stream for fluentd
- Compression Support in out_http plugin
- fluent/lib/ruby/3.2.0/win32/registry.rb:910:in `encode': U+767E to ASCII-8BIT in conversion from UTF-16LE to UTF-8 to ASCII-8BIT (Encoding::UndefinedConversionError) HOT 1
- DiskPressure due to 80 GB /home/fluentd/core. files
- Set log_path in system config HOT 2
- Duplicate logs getting created on Opensearch through fluentd HOT 4
- Startup error using docker startup to specify profile location
- Out_file short flush interval with timekey 1h in buffer HOT 5
- in_tail error after upgrading from 3.8.1 to td-agent-4.5.2 HOT 3
- Fluentd logs HOT 1
- Support AWS SigV4 in the http output plugin HOT 3
- Match directive not working HOT 1
- What will be the impact after removal of OpenSSL c_rehash script from td-agent
- in_exec: Can't handle non-ASCII characters output HOT 1
- Syslog TLS: [client_cert_auth false] settings is not applied if [insecure true] is not set. HOT 4
- [YAML] 'log_level' is deprecated parameter name. use '@log_level' instead // parameter '$log_level' is not used HOT 3
- Broken hadoop_snappy compression in some cases HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fluentd.