The pwnlib.atexit module should have a version of register which allows the routine to be invoked with a particular context, or a at least particular log_level.

This change will squelch most of the output caused by creating multiple connections / launching multiple processes, and having tube.close invoked via pwnlib.atexit.

The rop module is still lacking documentation on readthedocs. Any way we can cut a new release so the docs are up?

Before I starting to talk, thank you for sharing the brilliant libraries to the public.

As far as I know, this library has been developing therefore sometimes a few functions could be changed without any notification. I think that it is no problem for me at all. (may be someone else too)
However, I think that someone who is not familiar with your fantastic library could try to use "dir" function to list all of methods to understand how to use that.
Therefore, if you don't mind, please consider adding "dir" function as below:

--- a/pwn/rop.py
+++ b/pwn/rop.py
@@ -310,3 +310,6 @@ class ROP:
for f, a in args:
self.call(f, a)
return self
+

• def dir(self):

•    return dir(type(self)) + list(self.**dict**)
  

Thank you.

The change in behavior with regard to use of recvall() to capture all data until no more data is available (but without requiring the connection to terminate) is now broken without replacement.

Old Behavior

Previously, the behavior of various basechatter routines were:

• recv(n)
  • Receive up to N bytes, or '' if a timeout occurs, and returns immediately if any data is available
• recvall(n)
  • In a loop, call recv() until it returns an empty string (no additional data within timeout period or connection closed)
• clean()
  • Effectively the same behavior as recvall(), but with a hard timeout of 100ms, and the data is discarded

New Behavior

The behavior of the new `tube class is:

• recv(n)
  • Same
• recvall()
  • In a loop, call recv() until EOFError is raised (connection closed)
• clean()
  • Same as before, but with half the timeout (50ms instead of 100ms)

Suggested Changes

• Restore recvall functionality or add a function which aggregates all data received until a full timeout period passes with no additional data. EOFError should be eaten by this call, returning ''.
• Increase clean timeout (50ms is lower than network latency over WAN) or allow a timeout= argument to override the 50ms default

Ctrl+D pressed after the two Ctrl+Cs

^CQuit
^CQuit
Exception in thread Thread-4:
Traceback (most recent call last):
File "/home/user/.pyenv/versions/2.7.7/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
File "/home/user/.pyenv/versions/2.7.7/lib/python2.7/threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
File "/home/user/pwntools/pwnlib/tubes/ssh.py", line 124, in recv_thread
    sys.stdout.write(cur)
File "/home/user/pwntools/pwnlib/term/term.py", line 106, in write
    output(s, frozen = True)
File "/home/user/pwntools/pwnlib/term/term.py", line 446, in output
    cell.content = parse(s)
File "/home/user/pwntools/pwnlib/term/term.py", line 288, in parse
    out[-1][1].extend(x[1])
AttributeError: 'str' object has no attribute 'extend'

The current shellcraft mechanism isn't able to handle aliases for the same architecture.

As a specific example, pwntools currently recognizes both 'mips' and 'mipsel' as separate architectures. In order to support both of these for pwnlib.shellcraft, the directories must be duplicated, or there must be a way to alias them.

This also causes issues for arm vs aarch64, which should be able to use the same shellcode in most situations.

Currently, pwnlib.tubes.process.process has no mechanism for creating a controlling pty for stdin/stdout. Some challenges may invoke isatty and behave differently. Additionally, libc writes directly to the controlling TTY for some failures.

It would be awesome if we did not depend on the pre-built binaries in pwnlib/data/binutils.

I built them originally a long time ago and I have a hard time remembering how. I am pretty sure that with the exception of the objcopy/objdump it was a fairly standard thing to do.

I think that for those two I did some magic to figure out every possible target to compile them in. Or rather: Almost every possible target, as some of them made binutils now build.

The .rela.plt section in a statically linked ELF file has sh_link set to zero, causing an attribute error in elf.py

user@ubuntu:~/src$ cat > static.c <<EOF
int main() { return 0; }
EOF
user@ubuntu:~/src$ gcc -o static static.c -static
user@ubuntu:~/src$ python -c "from pwn import *; elf = ELF('./static');"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/pwntools-2.1.3-py2.7.egg/pwnlib/elf.py", line 58, in __init__
    self._populate_got_plt()
  File "/usr/local/lib/python2.7/dist-packages/pwntools-2.1.3-py2.7.egg/pwnlib/elf.py", line 240, in _populate_got_plt
    symbol   = sym_rel_plt.get_symbol(sym_idx)
AttributeError: 'NullSection' object has no attribute 'get_symbol'

We need to agree on a testing framework (and start using it) to avoid breaking functionality when changing stuff in pwntools.

The mov shellcraft utility should be ported over. See #110 for an example of why

Hello, I'm a big fan of this brilliant library.
I'd like to share shellcodes (getdents, sendfile in ARM mode) to you.
If you don't mind, just glances these dumb codes.

Thanks,
-- Alex

getdents.py

--- /home/alex/hack/pwntools/pwn/shellcode/io/getdents.py   2014-04-27 00:39:00.340160965 +0000
+++ /home/alex/hack/pwntools_rev/pwn/shellcode/io/getdents.py   2014-04-26 23:41:02.912063611 +0000
@@ -1,7 +1,33 @@
 from pwn.internal.shellcode_helper import *

-@shellcode_reqs(arch='i386', os='linux')
-def getdents(in_fd = 0, size = 255, allocate_stack = True):
+@shellcode_reqs(arch=['i386', 'arm'], os=['linux'])
+def getdents(in_fd = 0, size = 255, allocate_stack = True, arch = None, os = None):
+
+    in_fd  = arg_fixup(in_fd)
+    if arch == 'i386':
+        if os == 'linux':
+            return _getdents_linux_i386(in_fd, size, allocate_stack)
+
+    elif arch == 'arm':
+        if os == 'linux':
+            return _getdents_linux_arm(in_fd, size, allocate_stack)
+
+    no_support('getdents', os, arch)
+
+
+def _getdents_linux_arm(in_fd = 0, size = 255, allocate_stack = True):
+    out = []
+
+    out += ['mov  r0, #%s' % (str(in_fd)),
+            'adds r6, sp, #268',
+            'mov  r1, r6',
+            'mov  r2, #268',
+            'svc  SYS_getdents',
+            '.align 2']
+
+    return indent_shellcode(out)
+
+def _getdents_linux_i386(in_fd = 0, size = 255, allocate_stack = True):    
     """Args: [in_fd (imm/reg) = STDIN_FILENO] [size = 255] [allocate_stack = True]

     Reads to the stack from a directory.

sendfile.py

--- /home/alex/hack/pwntools/pwn/shellcode/io/sendfile.py   2014-04-27 00:39:22.012161572 +0000
+++ /home/alex/hack/pwntools_rev/pwn/shellcode/io/sendfile.py   2014-04-27 00:00:42.696096640 +0000
@@ -1,7 +1,7 @@
 from pwn.internal.shellcode_helper import *
 from ..misc.pushstr import pushstr

-@shellcode_reqs(arch=['i386', 'amd64'], os=['linux', 'freebsd'])
+@shellcode_reqs(arch=['i386', 'amd64', 'arm'], os=['linux', 'freebsd'])
 def sendfile(in_fd = 0, out_fd = 1, arch = None, os = None):
     """Args: [in_fd (imm/reg) = STDIN_FILENO, [out_fd (imm/reg) = STDOUT_FILENO]

@@ -22,8 +22,25 @@
         elif os == 'freebsd':
             return _sendfile_freebsd_amd64(in_fd, out_fd)

+    elif arch == 'arm':
+        if os == 'linux':
+            return _sendfile_linux_arm(in_fd, out_fd)
+
     no_support('sendfile', os, arch)

+def _sendfile_linux_arm(in_fd, out_fd):
+    out = []
+
+    out += ['mov r0, #%s' % (str(out_fd)),
+            'mov r1, #%s' % (str(in_fd)),
+            'mov r2, #0',
+            'mov r3, #255',
+            'svc SYS_sendfile',
+            '.align 2']
+
+    return indent_shellcode(out)
+
+
 def _sendfile_linux_i386(in_fd, out_fd):
     return """
         """ + pwn.shellcode.mov('ecx', in_fd, raw = True) + """ ; in_fd

...so that for example an attack framework can disable logging or send it to a file.

Some of the default colors cause things like this to happen (using the Solarized theme for xfce4-terminal).

It looks like ssh.interactive is broken when used on ssh.channel objects spawns with ssh.run under certain circumstances. After "switching into interactive mode", the process hangs, requiring me to kill it from another terminal.

Here's an example script which demonstrates the issue (just add r.interactive() at the end).

Consider I am not a pwntools user, but come across a script.

from pwn import *
# ...

If I attempt to install the pwn package, I get something else*.

$ pip install pwn

I now have something different than the latest released version of pwntools. The correct package on pypi is pwntools.

How do we reconcile these?

This doesn't look right.

In [9]: r.call('execvp',['/bin/sh',['/bin/sh',0]])

In [10]: print '\n'.join(r.dump())                                                                                                                            
0x804a038:        0x804b410 (execvp)
0x804a03c:       0xdeadbeef
0x804a040:        0x804a048 (+8)
0x804a044:        0x804a04c (+8)
0x804a048:        '/bin/sh'
0x804a050:        0x804a054 (+4)
0x804a054:              0x0
0x804a058:        '/bin/sh'

The first arg is OK ('/bin/sh\x00' takes up 8 bytes), but the second, nested argument is pretty broken (0x804a044 should point to 0x804a050, 0x804a50 should point to 0x804a58). It looks like something lost four bytes somewhere.

If context.log_level = 'debug', only the outbound side of the conversation is logged when using tube.interactive.

On Debian 7.X, there is no pip2 command.

$ apt-get install python-pip
$ pip2
-bash: pip2: command not found

On the latest Ubuntu LTS (14.04), /usr/bin/pip resolves to the python2.7 version even though the default version of Python is Python3.

Everywhere I've look, pip always points to the Python 2.7 pip (if installed). However, pip2 does not always exist. Is there a reason we use/recommend pip2?

Debian appears to use the pip-2.7 naming scheme, which differs from Ubuntu's pip2 (which I imagine is where the original name came from).

In contrast, python2 is present on all systems that I looked at (which Python 2.7 installed).

Just wanted to point all this out and check to see if there's any objections before I change README.md.

There's no check against elf.address

        #On non position independent executables the base address can be read off the elf.
        if PIE is False and base is None:
            base = filter(lambda x: x['p_type'] == 'PT_LOAD' and (x.header.p_flags & 1), e.segments)[0]['p_vaddr']

        #At this point we should have a base address
        if base is None:
            log.error('Position independent ELF needs a base address')

The context module tries to be helpful in setting sane default values for some settings based on other settings. Unfortunately these are incomplete, and have potentially unwanted side-effects.

Side-Effects

>>> from pwn import *
>>> context.word_size = 1337
>>> context.arch = 'i386'
>>> assert 1337 == context.word_size
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AssertionError

Incomplete

There's no way to set a default endianness. For example, PowerPC is generally big-endian, but setting arch="powerpc" does not set this.

Always meant to ask -- why is the context module implemented so strangely? It implements a ModuleType object and replaces itself with a singleton. Why was this design used rather than a normal object?

The atexit module uses a dictionary to keep track of the routines to call. This means that no routines can be registered more than once, for example with different arguments.

Additionally, this means that pwnlib.atexit does not guarantee order.

Per the Python documentation for atexit:

atexit runs these functions in the reverse order in which they were registered; if you register A, B, and C, at interpreter termination time they will be run in the order C, B, A.

However, multiple registration of routines breaks unregister in its current incarnation. The suggested remedy for this is to return a unique ID for each register call that is passed to unregister, rather than passing in the function to unregister.

If there's no data to read, an EOFError is raised to the top level. It's more desirable for '' to be returned, since this allows printing diagnostic messages in case something went wrong. For example, if I upload a C file to compile, I'll print sh.run('gcc file').recv() in case there's some error output.

Example

c = ssh(...)
c.run('echo hello && cat /dev/null').recv() # OK
c.run('cat /dev/null').recv() # Fail
Traceback (most recent call last):
  File "/home/user/pwntools/pwnlib/tubes/tube.py", line 53, in recv
    data = self.recv_raw(numb)
  File "/home/user/pwntools/pwnlib/tubes/sock.py", line 49, in recv_raw
    raise EOFError

There is enough data in the buffer to serve the request, but the tube still gets polled.

Might be a duplicate of #131, but this affects the main .recv() interface.

class EmptyTube(tube.tube):
    pass

t = EmptyTube()
t.buffer += ['a'*4096]
t.recv()

:( examples # python2 ./ssh_forward.py
[+] Trying to bind to 0.0.0.0 on port 1337: OK
[+] Waiting: Got connection from 127.0.0.1 on port 56711
Traceback (most recent call last):
  File "./ssh_forward.py", line 9, in <module>
    listen(1337).wait_for_connection() <> ssh('root', '127.0.0.1', ' ').connect_remote('google.com', 80)
  File "/root/pwntools/pwnlib/tubes/ssh.py", line 274, in __init__
    h = log.waitfor('Connecting to %s on port %d' % (host, port), log_level = self.log_level)
TypeError: %d format: a number is required, not str

script below

:( examples #
  4 """
  5
  6 from pwn import *
  7
  8 while True:
  9     listen(1337).wait_for_connection() <> ssh('root', '127.0.0.1', password).connect_remote('google.com', 80)
 10
 11 # now point your browser (or curl(1)) to http://localhost:1337

The implementation for pwnlib.tubes.process.connected_raw is backward.

See process.py.

The only reason that the implementation currently works is that internally, process does not consult connected_raw, but checks self.proc.std{in,out}.closed directly.

This is resolved in #158.

Currently the following exception is thrown:

Python 2.7.8 (default, Jul  1 2014, 17:30:21) 
[GCC 4.9.0 20140604 (prerelease)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> >>> >>> >>> >>> Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/pilen/csaw/psifer.py", line 1, in <module>
    from pwn import *
  File "/home/pilen/code/pwntools/pwn/__init__.py", line 3, in <module>
    from .toplevel import *
  File "/home/pilen/code/pwntools/pwn/toplevel.py", line 2, in <module>
    from pwnlib                  import *
  File "/home/pilen/code/pwntools/pwnlib/__init__.py", line 10, in <module>
    from . import \
  File "/home/pilen/code/pwntools/pwnlib/dynelf.py", line 1, in <module>
    from . import elf, log, memleak
  File "/home/pilen/code/pwntools/pwnlib/elf.py", line 4, in <module>
    from .util import misc
  File "/home/pilen/code/pwntools/pwnlib/util/__init__.py", line 7, in <module>
    from . import crc, cyclic, fiddling, hashes, iters
  File "/home/pilen/code/pwntools/pwnlib/util/crc.py", line 15, in <module>
    from . import fiddling, packing
  File "/home/pilen/code/pwntools/pwnlib/util/fiddling.py", line 453, in <module>
    'marker': text.gray if text.has_gray else text.blue,
  File "/home/pilen/code/pwntools/pwnlib/term/text.py", line 98, in __getattr__
    init += self._fg_color(c())
  File "/home/pilen/code/pwntools/pwnlib/term/text.py", line 55, in _fg_color
    return termcap.get('setaf', c) or self._tc.get('setf', c)
  File "/home/pilen/code/pwntools/pwnlib/term/termcap.py", line 27, in get
    return curses.tparm(s, *args)
_curses.error: tparm() returned NULL

Would be neat to be able to auto-attach to each process spawned with a breakpoint at the entry point

I can't find a reference indicating the licence of pwntools in its readme or any COPYING or LICENSE file. A clear statement of it will facilitate the collaboration.

Thanks!

This throws a stack recursion error.

from pwn import *
import grequests # pip2 install grequests

listen(0).wait_for_connection()

recvline behavior changed between 2.1.3 and Gallopsled/pwntools/master.

By default, it now drops the trailing newline.

I imagine this affects other methods, but it's the only one I noticed. Which behavior do we want?

The standard behavior for file.readline is to keep the trailing '\n' by default.

>>> file('foo.c').readline().endswith('\n')
True

All of the tube routines do not time out correctly; instead they repeat the timeout period between each socket receive.

For example, the current implementation of recvrepeat can be summarizes as:

total_data = ''
while True:
  data = self.recv(..., timeout)
  if data:
    total_data += data
  if not data:
    return total_data

This is incorrect, as one would expect the timeout should apply to the total operation.

If I am incorrect on this, please tell me. Don't modify this in the existing code, I've got a PR forthcoming to address this.

If I do this:

from pwn import *
with context.local(log_level = 42):
    def bar():
        print context.log_level
    t = threading.Thread(target = bar)
    t.start()
    t.join()

Then I expect it to print out 42, but it does not.

Binaries that are compiled with full read-only relocations do not contain a .got.plt section, which is used by DynELF.

echo 'void main(){}' > foo.c
gcc -o regular foo.c -z norelro
gcc -o partial foo.c -z relro
gcc -o full foo.c -z relro -z now
python <<EOF
from pwn import *
for f in ['regular','partial','full']:
  e = ELF(f)
  print "%s: %s" % (f, e.get_section_by_name('.got.plt'))
EOF

regular: <elftools.elf.sections.Section object at 0x7faa6f4dea50>
partial: <elftools.elf.sections.Section object at 0x7faa76ba97d0>
full: None

checksec.sh --dir .
RELRO           STACK CANARY      NX            PIE             RPATH      RUNPATH      FILE
Full RELRO      No canary found   NX enabled    No PIE          No RPATH   No RUNPATH   ./full
Partial RELRO   No canary found   NX enabled    No PIE          No RPATH   No RUNPATH   ./partial
No RELRO        No canary found   NX enabled    No PIE          No RPATH   No RUNPATH   ./regular

When writing shellcode in the mako template language it is natural to indent when using mako control structures, however that messes up the indention that shellcraft outputs when using '-f a'.

Maybe shellcraft should prettify the result before outputting.

I get this A LOT:

[*] Found gadgets for './ropasaurusrex-85a84f36f81e11f720b1cf5ea0d1fb0d5a603c0d' in cache '/tmp/pwntoo    ls-rop-cache/ropasaurusrex-85a84f36f81e11f720b1cf5ea0d1fb0d5a603c0d c5bb68949dcc3264cd3a560c05d0b5    66-0x8048000'

If it something you might be interested in I could probably help out :)
Here's some of the IPython shell work I did on another project http://youtu.be/VlxP7OdKQAI

The following hangs indefinitely

from pwn import *
p = process(which('cat'))
p.clean()

Is there any reason for rop.dump() being a list?

I have never used in other ways than "\n".join(rop.dump()).

Maybe we should change it to return a list -- if you really want the individual lines, you can just split it again afterwards.

We already have doctests for testing most stand alone features and the repo Gallopsled/pwntools-write-ups tests most larger features (or will eventually). However none of these would every hit every shellcode or every way a shellcode can be used.

It would be awesome to be able to test the shellcodes in a systematic way.

Ideally we would want both functionality testing (e.g. test that findpeersh actually finds the right socket) and strace/similar based testing.

None of the recvxxxx routines check self.buffer before calling self._recv which skips the fast-return path in self.recv (which does check self.buffer).

Pressing delete in interactive mode immediately breaks.

python -c "from pwn import *; process('/bin/bash').interactive()"                                                                         ⏎
[+] Started program '/bin/bash'
[*] Switching to interactive mode
$ hostname
ubuntu
$ <-- press delete
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/user/pwntools/pwnlib/tubes/tube.py", line 296, in interactive
    data = term.readline.readline(prompt = prompt, float = True)
  File "/home/user/pwntools/pwnlib/term/readline.py", line 372, in readline
    keymap.handle_input()
  File "/home/user/pwntools/pwnlib/term/keymap.py", line 20, in handle_input
    self.send(key.get())
  File "/home/user/pwntools/pwnlib/term/keymap.py", line 46, in send
    cb(self.trace)
  File "/home/user/pwntools/pwnlib/term/readline.py", line 261, in delete_char_forward
    if buffer_right:
UnboundLocalError: local variable 'buffer_right' referenced before assignment

This works as expected:

from pwn import *
from getpass import getuser
user = getuser()
sock1 = ssh(user, 'localhost', keyfile = '~/.ssh/id_rsa')
sock = sock1.connect_remote('localhost', 8080)
sock.send('GET / HTTP/1.0\r\n\r\n')

But this gives an error:

from pwn import *
from getpass import getuser
user = getuser()
sock = ssh(user, 'localhost', keyfile = '~/.ssh/id_rsa').connect_remote('localhost', 8080)
sock.send('GET / HTTP/1.0\r\n\r\n')

Trace:

[+] Connecting to localhost on port 22: OK
[+] Connecting to localhost:8080 via SSH to localhost: OK
Traceback (most recent call last):
  File "bar.py", line 11, in <module>
    sock.send('GET / HTTP/1.0\r\n\r\n')
  File "/home/br0ns/projects/pwntools/pwnlib/tubes/tube.py", line 214, in send
    self.send_raw(data)
  File "/home/br0ns/projects/pwntools/pwnlib/tubes/sock.py", line 63, in send_raw
    raise EOFError
EOFError

One expects that setting context.arch = 'amd64' would implicitly set context.word_size=64.