Multiple scripts and script arguments. about mitmproxy HOT 7 CLOSED

Hi there. All of these features would be most welcome (and, in fact, are on my long todo list). Lets take these point by point.

• The fundamental mechanics of adding multiple scripts is pretty simple. Look at the "script" attribute of flow.FlowMaster. This should be a list. Wherever the attribute is used, extend in the obvious way to iterate through the list, and act on the scripts one by one.
• Support this by making it possible to pass the -s argument multiple times.
• Now for a difficult one - the user interface in mitmproxy itself. We will have to show the user a list of scripts that they can toggle/remove one by one. I have a nice, generalised control that can do this - the GridEditor that we use in a bunch of places. If you do the previous two items, I'll take care of this aspect of things (unless you want to have a crack).
• Arguments to scripts. This is actually a tricky design problem. There's no really nice way to pass "sub-arguments" on the command-line, especially when you can have multiple scripts each with its own arguments. There are some obvious (but inelegant) ways to do this. Any suggestions would be welcome.

from mitmproxy.

I'd like to tackle the sub-arguments thing. IMO the best way is to submit them within the -script argument in double quotes:
python mitmproxy -T -s "a.py -sub-arg 1 -sub-argx 2" -s "b.py -sub-arg "space string""
The syntax is pretty clean and you don't get any scope problems. Every script just needs to expose an argparse.ArgumentParser if it takes arguments.
This small snippet should illustrate it:
https://gist.github.com/4273413

from mitmproxy.

I as thinking something in the line of:

mitmproxy -s scriptname.py param1,param2,param3 -s scriptname2.py param1,param2,param3 -s scriptname3.py

And then using the callback function of optparse to consume arguments if present, making something like this:

userscripts = [["scriptname,py", "param1,param2,param3"], ["scriptname2.py", "param1,param2,param3"], ["scriptname3.py", ""]]

Haven't given much thought to how this would avoid screwing with the [filters] of mitmdump, had a quick look at the code and these seem to be uncollected arguments?

from mitmproxy.

Thanks for your input on this.
Using "," for splitting parameters on the command line seems to be an uncommon solution for me though. You can certainly use -script a.py param1 param2 -script b.py param1 but you limitation with this approach is that your parameters must not start with a -. (Assuming that you don't want to hack in the argparse (not optparse!) core). Another thing is that this screws the filters of mitmdump: -script a.py i-am-a-filter-or-an-argument. You could fix this by making the filter a regular argument. However, changing the existing syntax should be planned carefully.

If you put the whole script parameter with arguments into quotes, the handling gets easier I think.
.add_argument('-script', type=shlex.split, action='append')
results with a call of mitmdump -script "a.py -foo -bar" -script b.py in
userscripts = [["a.py","-foo","-bar"],["b.py"]]

While I am not a big fan of the surrounding quotes myself, the syntax gets non-ambiguous and not too verbose. What do you think?

from mitmproxy.

Think you're right, breaking with the old syntax is bound to break something somewhere and doing it your way allows using value names already in use by mitmproxy, -s for example. Good spot.

from mitmproxy.

Max, I think your solution is probably the way to go. I like using argparse for this, and that would also mean that we could extract argument and type specifications programmatically if we ever do want to build a GUI for this (for now, we can just parse the arguments as on the command-line). So, we need to define a standard way for scripts to expose a parser, expand mitmproxy to take multiple -s arguments, and then tackle the GUI issues one by one (and there are quite a few).

from mitmproxy.

Any thoughts on how the scripts should expose a parser?
I'd propose adding script_argv (=["a.py","--foo","42"]) as a second parameter to the start function (breaks compatibility though). If we do this, we could consider moving the .start() logic into the Script class as well. Is there a special reason for the current separation?

On a side note, nosetests crashes for me (SSLCert.from_der(...) => asn1 encoding routines - ASN1_get_object - too long)...

Cheers,
Max

from mitmproxy.