Comments (6)
The streaming chans are not closed by design. Problem is: we cannot know if/when they're safe to close, where "safe" means "caller has read all lines". The chans are a buffer (of DEFAULT_STREAM_CHAN_SIZE
size). So this sequence of events can happen:
- Caller starts recv on chan
- Caller starts cmd
- Cmd prints "a\n"
- Cmd prints "b\n"
- Cmd prints "c\n"
- Cmd exits
- Caller reads first line ("a\n")
And that's it. If we close the chan after 6, the caller loses some or all of the output.
This is documented:
Since the channel is not closed by the OutputStream, the two indications that all lines have been sent and received are the command finishing and the channel size being zero.
Let me know if I'm missing something, or if there's a safe way to close the chan.
from cmd.
If we close the chan after 6, the caller loses some or all of the output.
Channels don't work like that: a close
on a channel is just another message, so it gets buffered in line like everything else.
For demonstration, refer to this example: https://goplay.space/#B6Kw81F-T1Z
This is also documented in the Go reference (emphasis mine):
After calling close, and after any previously sent values have been received, receive operations will return the zero value for the channel's type without blocking.
from cmd.
Re-reading my last comment, I see I didn't explain it well. My concern was not with the channel but with the program sending to the channel. My original understanding was that Go os/exec does not wait for stdout/stderr. But re-reading cmd.Wait today it's clear:
Wait waits for the command to exit and waits for any copying to stdin or copying from stdout or stderr to complete. ... If any of c.Stdin, c.Stdout or c.Stderr are not an *os.File, Wait also waits for the respective I/O loop copying to or from the process to complete.
These docs changed at some point. Today, the exec.Cmd docs for Stdout io.Writer
and Stderr io.Writer
say,
In this case, Wait does not complete until the goroutine reaches EOF or encounters an error.
but previous versions did not say that. Another example from Cmd.Wait docs:
If any of c.Stdin, c.Stdout or c.Stderr are not an *os.File, Wait also waits for the respective I/O loop copying to or from the process to complete.
vs.
If c.Stdin is not an *os.File, Wait also waits for the I/O loop copying from c.Stdin into the process's standard input to complete.
Former is today, latter is go1.9. No mention of Stdout and Stderr. I wish the docs said if changes are purely doc-related or also code-related because this makes me wonder: did Wait start to wait for stdout/stderr output as of some version?
With go1.11 it's clear: yes, we can close the chan without any data loss. But for go1.9 we should test it.
from cmd.
As of January 2020, can we introduce closing these channels. Go is at 1.13, so I think it's safe to assume we can move forward with the original proposal?
I'd like to move to using waitGroup semantics rather than doing len queries.
from cmd.
Agreed. Will implement and release as v1.2.0. Even though current v1.1.0 doesn't have any additional releases, this change is probably important enough to warrant a new minor level, and to make sure no one using v1.1 is surprised by the channel being closed.
from cmd.
Released as v1.2.0.
from cmd.
Related Issues (20)
- wrong description of Status for receiving a signal of interrupt HOT 2
- not support cmd HOT 1
- special character '>' HOT 1
- How to hide the console window? HOT 2
- [streaming] Line breaks and buffer overflows (broken pipes) HOT 1
- The actual PID is 11 more than the obtained PID
- Part of #11517
- Updated and install
- How can I execute a sentence command directly? HOT 1
- 🔊
- Stdout empty when using Streaming HOT 6
- unable to use with binary stdout HOT 2
- Running a script to start an external application, when the main application terminates, the one started from the script crashes as it no longer has an Stdout & Stderr HOT 1
- make terminating process group optional HOT 1
- python command in streaming mode not streaming the stdout HOT 2
- cant go get HOT 3
- Command is still executed if Stop() is called during BeforeExec hooks HOT 1
- How to restart after Stop()
- the replacement of "+build" to "go:build" needs a change of go version in go mod
- when the process is ready HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cmd.