b/convert sometimes fails to copy the last byte array about byte-streams HOT 11 CLOSED

Is there a way of telling the conversion path that a collection of byte-arrays to a single byte-array would take? It feels like there's something occasionally not being flushed.

from byte-streams.

What version are you using, and is the input a sequence or vector of byte arrays?

from byte-streams.

Hi - version is 0.2.0, I'm using a sequence but could easily be a vector.

from byte-streams.

More info: The problem never occurs when a vector is used. If I use a sequence, however, it fails occasionally/erratically. Here are the results of my test-suite.

FFPPPFFPPPPPPFPPPPPPPPPPF

where F=FAIL, P=PASS.

from byte-streams.

Can you test with the latest 0.2.1 beta? If it still fails, can you provide a reproducing case for me to run?

from byte-streams.

Testing with 0.2.1-alpha1 seems to cause occasionally hangs on my test-suite. I've tried a few different combinations, looks like occasionally the thread is blocking. I'll investigate further the exact nature of these hangs, and try to distill a better failing test.

from byte-streams.

So I've been trying to reproduce this on my end, using some variant of

(bs/to-byte-array (repeatedly 1e3 #(byte-array 1e5)))

I'm not able to, but I did notice that it hung when I ran this inside the byte-streams namespace, since byte-array is shadowed there. An error was logged, but it was otherwise stuck. This obviously shouldn't happen, and I've made a fix such that it logs and actually returns, but maybe this is why it was hanging for you? Let me know.

from byte-streams.

I believe I've reduced the problem down to a simple example here:
https://github.com/malcolmsparks/bs-bug/blob/master/src/bs_bug/core.clj

If you clone the repo

git clone [email protected]:malcolmsparks/bs-bug.git

This might not be the actual issue described above, or might be closely related to it.

If you try the bs_bug/core ns, you'll see the problem I'm having. It could be
that this is normal expected behavior from byte-streams and I'm assuming
this is a bug when it isn't.

The wider context is that I have to use this ->closing-stream function in
my tests that use ring-mock, because the request :body is an input stream.
This causes my tests to hang, but the actual code is working fine with real
browsers. It's a big of a snag, but isn't a huge issue, but would be nice
to understand why it's hanging.

from byte-streams.

This looks like a bug in the stream lifecycle handling. I'll take a closer look. Thanks for the reproducing case.

from byte-streams.

Okay, this was a little stranger than I expected. The problems were two-fold: manifold.stream/reduce doesn't return an exception when given a non-stream input, and the closeable-seq implementation in byte-streams assumed everything it closed over was clojure.lang.IPending. Both of these are fixed, in [byte-streams "0.2.1-alpha2"] and [manifold "0.1.2-alpha1"], respectively. You'll need to wrap get-buffers in s/->source for your failing case to work, though.

from byte-streams.

Hi Zach, I've updated to both library versions and this has been working
well. Thanks, Malcolm

Malcolm Sparks
Director

Email: [email protected]
Web: https://juxt.pro

JUXT LTD.
Software Consulting, Delivery, Training

On 24 November 2015 at 03:23, Zach Tellman [email protected] wrote:

Okay, this was a little stranger than I expected. The problems were
two-fold: manifold.stream/reduce doesn't return an exception when given a
non-stream input, and the closeable-seq implementation in byte-streams
assumed everything it closed over was clojure.lang.IPending. Both of
these are fixed, in [byte-streams "0.2.1-alpha2"] and [manifold
"0.1.2-alpha1"], respectively. You'll need to wrap get-buffers in
s/->source for your failing case to work, though.

—
Reply to this email directly or view it on GitHub
#20 (comment)
.

from byte-streams.