New method: public <T> void Streams.iterate(Stream<T> stream, Consumer<? super T> action); about guava HOT 6 CLOSED

This may not be helpful, but other alternatives solutions are to use Collections.synchronizedList,
Iterators.addAll, or Iterator.forEachRemaining. I personally never minded the classic for each loop idiom or a while loop when dealing with iterators. In Apache Commons' there is IteratorUtils.forEach if you use that library as well.

fwiw, unlike the BaseStream JavaDoc's claim I have observed iterator() eagerly materialize the full I/O stream before traversal, thus resulting in an initialization pause and memory spike. In that case I switched to forEach to lazily materialize the next element. I'd be afraid anyone running into your problem might still be surprised and may find the implementation shifts underneath them as their assumptions are not contractual.

from guava.

I applaud the desire to avoid depending on the current implementation. In this particular case, though, I don't think we should add this API.

Part of that is that I'm not sure the hermeneutics discussion—thanks for the link!—has resolved whether the choice of thread is unspecified. I could see reading "The behavior of this operation is explicitly nondeterministic" as referring only to ordering, with the following sentence's "For parallel stream pipelines" as a qualifier that extends all the way to "in whatever thread the library chooses." That may still be wrong: The Goetz quote speaks of removing the one sentence about parallel pipelines, which would leave us with a clearly unqualified statement that threading is unspecified. Then we could try to contend with passages like this one, in which we're told that a forEach* call like yours is unsafe "if executed in parallel"... which likewise doesn't prove that it otherwise is safe :)

I admit that it's not totally implausible to imagine that a Stream might be backed by, say, an RPC, which might receive responses on multiple threads (though I probably wouldn't recommend it). In that case, I'd still be tempted to say, even if we allow for multiple threads to run the Consumer, that it is defensible to expect for the Stream to set up appropriate happens-before edges so that operations like ArrayList:add are well defined. (Certainly that's true for the specific example of forEachOrdered, which makes an explicit guarantee of that.) Now even that could lead to problems if a caller were depending on ThreadLocal state.

But at some point, this becomes a case in which I wouldn't try to spend our "prescriptiveness budget" in advance of real problems: If we push users to use our API because forEach might someday lead to problems, they're going to be annoyed at the loss of single-statement stream operations, and some of them still aren't going to do what we recommend. So, if someday we see real problems from this, then we're going to have a cleanup to perform no matter what. It will be more efficient for us to do it for all of Google than for us to ask hundreds of developers to do it gradually over the years. (The world outside Google is another story, of course, but we can't force developers to use our method there any more than we can migrate them to it.)

from guava.

Part of that is that I'm not sure the hermeneutics discussion—thanks for the link!—has resolved whether the choice of thread is unspecified...

For the record, there's no ambiguity on this point. The Javadoc for Stream.forEach() says:

For any given element, the action may be performed at whatever time and in whatever thread the library chooses.

And for Stream.forEachOrdered() it says:

...for any given element, the action may be performed in whatever thread the library chooses.

Seems pretty clear to me.

Regarding this:

If we push users to use our API because forEach might someday lead to problems...

It sounds like you are OK with developers using an API incorrectly in such a way that could lead to difficult-to-find bugs and security holes, as long as the way that the API happens to be implemented today by the particular implementation that most people happen to be using today just happens to not trigger these bugs today.

Hmm, OK. I'm sure it will be fine.... what could go wrong? :)

from guava.

Right, my first contention is that it's not clear whether "the action may be performed at whatever time and in whatever thread the library chooses" is qualified by "For parallel stream pipelines" or not. My second contention is that the library provides any necessary happens-before edges to keep your example working correctly (and might also do so in the case of forEach, but that's unclear).

My third contention is that we have almost no power to prevent people from using this API incorrectly. If it starts to bite people in practice and they come crawling back, I am happy to belatedly offer to rescue them :)

from guava.

but isn't this feature request equivalent to s.iterator().forEachRemaining(action)?

from guava.

Ah, true! I'd almost suggested that forEachOrdered would be good enough, but there was the ThreadLocal caveat. But your way solves that. @archiecobbs, what do you think?

from guava.