pdb experience is sub-par about honcho HOT 21 CLOSED

Actually, the experience is only subpar when using start, with run it is fine. The reason I am trying to use start with pdb is that run doesn't support process nicknames from Procfile (#55), so I guess that's my real problem.

from honcho.

IRC

from honcho.

Reopening, since #55 was rejected as wont-fix.

from honcho.

What can we do about this, @nickstenning? :-)

from honcho.

All I am getting is pdb prompt where I cannot enter anything 😟. Oh well, pdb was actually responding to my commands only I was not seeing the output. The output was printed out only after I pressed Ctrl-d.

from honcho.

@zwn I believe if you set PYTHONUNBUFFERED=1 in your environment then you'll at least get the output immediately after you enter a command (though you still won't see the command as you type it in).

from honcho.

@nickstenning This is really a drag. It's disappointing that you've rejected #55, yet aren't proposing an alternate solution over here. :-(

from honcho.

@whit537 How can I put this? Oh, that's right: do fuck off.

This is your periodic reminder that, like almost all open-source maintainers, I fit my work (which I do, by the way, for free) into my not-exactly-copious spare time.

Passive-aggressive issue comments will not get you far.

from honcho.

Or, to put that another way: pull requests welcome.

from honcho.

Passive-aggressive issue comments will not get you far.

You're right. I've actually implemented pdb proxying before (in a test runner I wrote a while back), so I have a pretty good idea what it would take to implement this feature as I think(?) you might want it implemented. I'm sorry for being passive-aggressive. :-(

Or, to put that another way: pull requests welcome.

@nickstenning Well, we did give you a pull request (#57) that both (a) scratches our itch and (b) brings Honcho more into line with the upstream project Honcho advertises itself as being a port of. You rejected it on the grounds that you don't like this particular aspect of Foreman's behavior (#55 (comment)), which feels like a bait-and-switch, because I didn't get the sense from the docs that you consider some of Foreman's features "dumb."

Do you agree that honcho start's current pdb experience is a bug? Have you been bitten by this before? Have you given any thought to how a fix might be implemented? Do you have any pointers to where in the code we'd look if we wanted to start working on this feature? Can you tell us how you'd want this implemented?

from honcho.

I've submitted a doc update (#58) to hopefully prevent future misunderstandings of Honcho's relationship to Foreman.

from honcho.

I'm very happy to have a conversation about this issue and work to fix the problem. I hope you understand that my response above reflects my unwillingness to be guilt-tripped over a piece of software I look after in my spare time, without charge.

I appreciate your desire to fix stuff and I'll take a look at #58 when I'm at a computer. You're probably right that Honcho and Foreman have diverged somewhat, although I feel duty-bound to point out that this is primarily due to a major case of featuritis that I feel Foreman has.

Turning to the issue at hand, I see a couple of possibilities:

1. implement honcho run <task> as you describe
2. make honcho start <task> work for your use case
3. implement yet another command to provide the functionality you require

As I see it, there are problems with all three approaches:

1. As previously mentioned, I feel that this is ill-conceived and hard to explain to users: "arguments to honcho run are interpreted as shell commands, except when they're exactly one word long, in which case... etc." In addition, can you only specify one Procfile process group, and if so that now differs from the semantics of honcho start, and runs into...
2. At the moment, honcho start can take zero or more process names. In order to properly implement support for pdb and similar programs, either this would have to change, such that it took either zero or one process names (and had a different relationship with STDIN/the TTY in each case) or you need to find a way of specifying which process gets hooked up to STDIN. Either option seems ugly to me.
3. I just don't see why the additional complexity is needed. Why add all this stuff to Honcho when you could just write a one-line shell script that contained your command, which you could run by hand (./tools/debugger) when needed, and reference from a Procfile if necessary (debug: ./tools/debugger).

It may well be there are solutions I haven't thought of, and if so I'd be interested hearing them. As it is, it isn't obvious that this behaviour can be "fixed" even in principle without making certain aspects of Honcho either unnecessarily bloated or inconsistent.

from honcho.

I guess my feeling on this one is probably summarised by the last point here. Honcho is fundamentally a process output multiplexer. It's not an input multiplexer because that's something that's not easy to do simply.

What you are asking for is either:

1. A mechanism to select which process Honcho gives STDIN to. This is perfectly possible, but I'm not sure it's desirable. What happens if another process writes some output while you're trying to give input to pdb?
2. A mechanism of invoking a single command and passing over STDIN. This is called a shell script and I don't see why Honcho needs to reimplement it.

from honcho.

Honcho is fundamentally a process output multiplexer.

Hmmm ... well, again, a doc update might save the next person some trouble. Right now the docs say:

Honcho is:

1. A Python port of David Dollar’s Foreman: a command-line application which helps you manage and run Procfile-based applications. It helps you simplify deployment and configuration of your applications in both development and production environments.
2. Secondarily, Honcho is a Python library/API for running multiple external processes and multiplexing their output.

Sounds like now it would be more accurate to say something like:

Honcho is a Python library/API and CLI for running multiple external processes and multiplexing their output. Honcho started life as a port of David Dollar’s Foreman. We still work with Procfiles, though in other ways Honcho has diverged somewhat from Foreman, primarily due to a major case of featuritis that I feel Foreman has.

The reason we ported from Foreman to Honcho is that we wanted a Python port of Foreman. The two goals we're balancing are: keeping our development environment as close to production as possible, and keeping our development environment as simple as possible. We host on Heroku, but our stack is Python-based. Therefore, Honcho seemed to be just what we wanted: a Python port of Foreman.

If Honcho is no longer a port of Foreman, then my options are:

1. Switch back to Foreman.
2. Maintain a Python port of Foreman (starting with a fork of Honcho).
3. Live with Honcho.

There are problems with all three approaches:

1. Switching back to Foreman means we reintroduce a dependency on Ruby, complicating our developer experience.
2. Maintaining a port ourselves increases our workload and dilutes our focus.
3. Living with Honcho means we (a) diverge further from production and (b) introduce idiosyncrasy. Now when onboarding new developers we have to say, "Honcho is kind of like Foreman, but ..."

I'd have to discuss with @zwn and the rest of our team, but I'd probably lean towards (3), on the understanding that the Procfile is the really crucial component here in terms of compatibility with production, and as long as Honcho uses Heroku's Procfile syntax and generally does what we need for development, it's better for us to use Honcho rather than forking or reverting to Foreman.

As to the feature at hand ...

from honcho.

A mechanism of invoking a single command and passing over STDIN. This is called a shell script and I don't see why Honcho needs to reimplement it.

Okay, so here's our Procfile:

web: aspen --network_address :$PORT --www_root www/ --project_root .

I want to launch my web process with the same command in both development and production environments. So it seems like what you're suggesting is that I add another layer of indirection, moving the actual command to a shell script:

#!/bin/sh
exec aspen --network_address :$PORT --www_root www/ --project_root .

And then referencing that in my Procfile:

web: ./web.sh

Yes? Now to run it in development I do:

$ honcho run ./web.sh

And that gives me the pdb experience I want. But now I need to write a shell script wrapper for every Procfile task, don't I? To use the example from the docs, this Procfile:

web: gunicorn -b "0.0.0.0:$PORT" -w 4 myapp:app
worker: python worker.py --priority high,med,low
worker_low: python worker.py --priority med,low

Now becomes:

web: ./web.sh
worker: ./worker.sh
worker_low: ./worker_low.sh

And then I have three shell scripts:

#!/bin/sh
exec gunicorn -b "0.0.0.0:$PORT" -w 4 myapp:app

#!/bin/sh
exec python worker.py --priority high,med,low

#!/bin/sh
exec worker_low: python worker.py --priority med,low

I think that's pretty obviously not awesome. :-)

Two Proposals

The Procfile is a mapping of task names to shell commands. I can, of course, put shell commands in small shell scripts. I can also enter shell commands directly into my shell. But the whole point of Procfiles is to save me the hassle. And surely it's not unreasonable to want to invoke a single one of my shell commands in the foreground of my terminal (such that I could use pdb with it).

I feel that this is ill-conceived and hard to explain to users: "arguments to honcho run are interpreted as shell commands, except when they're exactly one word long, in which case... etc."

I'd actually explain it inversely: "honcho run <task> runs a single Procfile task in the foreground, except when <task> doesn't match a Procfile task, in which case it's interpreted as an arbitrary shell command."

That's my first proposal. If the "except" is too inelegant ...

implement yet another command to provide the functionality you require

How about the other way around? What if there were a honcho sh subcommand to run arbitrary shell commands in the environment? Then you'd have:

• honcho start [<task>*]—Start zero or more tasks and multiplex their output.
• honcho run <task>—Run a single task in the foreground.
• honcho sh <arg>*—Execute an arbitrary shell command.

That'd be a backwards incompatible change and would clearly move Honcho away from Foreman, but that seems to be the direction you're going. :-)

from honcho.

(Don't need to drag our whole team in, but I think @seanlinsley would appreciate being part of this discussion.)

from honcho.

Let me suggest one more approach. When start has just one parameter, there is no question where the stdin/stdout should be attached. If we do not do the wrapping with ProcessManager in that case, pdb would just work. Also there is nothing to explain in the docs. If this would be acceptable solution I am willing to prepare PR.

from honcho.

pdb would just work.

Would all lines be prefixed with 11:29:09 web.1 |? If yes, it'd be a slightly odd pdb experience, but acceptable. If no, it'd make production logging behavior variable, which seems like a bad idea.

from honcho.

I implemented my first proposal in #71.

from honcho.

Curious if the experience is any better with pdbpp, ipdb, etc. or if every debugger is affected by this...?

from honcho.

Closing. Honcho is not going to provide support for connecting STDIN to processes run from Procfiles.

from honcho.