dm04806 / pipe Goto Github PK

Help on module pipe:

NAME
    pipe - Infix programming toolkit

FILE
    /home/mandark/doc/code/python/pipe/pipe.py

DESCRIPTION
    Module enablig a sh like infix syntax (using pipes).
    
    = Introduction =
    As an exemple, here is the solution for the 2nd Euler Project exercise :
    
    "Find the sum of all the even-valued terms in Fibonacci
     which do not exceed four million."
    
    Given fib a generator of fibonacci numbers :
    
    euler2 = fib() | where(lambda x: x % 2 == 0)
                   | take_while(lambda x: x < 4000000)
                   | add
    
    
    = Vocabulary =
     * a Pipe: a Pipe is a 'pipeable' function, somthing that you can pipe to,
               In the code '[1, 2, 3] | add' add is a Pipe
     * a Pipe function: A standard function returning a Pipe so it can be used like
               a normal Pipe but called like in : [1, 2, 3] | concat("#")
    
    
    = Syntax =
    The basic syntax is to use a Pipe like in a shell :
    >>> [1, 2, 3] | add
    6
    
    A Pipe can be a function call, for exemple the Pipe function 'where' :
    >>> [1, 2, 3] | where(lambda x: x % 2 == 0) #doctest: +ELLIPSIS
    <generator object <genexpr> at ...>
    
    A Pipe as a function is nothing more than a function returning
    a specialized Pipe.
    
    
    = Constructing your own =
    You can construct your pipes using Pipe classe initialized with lambdas like :
    
    stdout = Pipe(lambda x: sys.stdout.write(str(x)))
    select = Pipe(lambda iterable, pred: (pred(x) for x in iterable))
    
    Or using decorators :
    @Pipe
    def stdout(x):
        sys.stdout.write(str(x))
    
    = Existing Pipes in this module =
    
    stdout
        Outputs anything to the standard output
        >>> "42" | stdout
        42
    
    lineout
        Outputs anything to the standard output followed by a line break
        >>> 42 | lineout
        42
    
    as_list
        Outputs an iterable as a list
        >>> (0, 1, 2) | as_list
        [0, 1, 2]
    
    as_tuple
        Outputs an iterable as a tuple
        >>> [1, 2, 3] | as_tuple
        (1, 2, 3)
    
    concat()
        Aggregates strings using given separator, or ", " by default
        >>> [1, 2, 3, 4] | concat
        '1, 2, 3, 4'
        >>> [1, 2, 3, 4] | concat("#")
        '1#2#3#4'
    
    average
        Returns the average of the given iterable
        >>> [1, 2, 3, 4, 5, 6] | average
        3.5
    
    netcat
        Open a socket on the given host and port, and send data to it,
        Yields host reponse as it come.
        netcat apply traverse to its input so it can take a string or
        any iterable.
    
        "GET / HTTP/1.0
    Host: google.fr
    
    "         | netcat('google.fr', 80)                       | concat                                        | stdout
    
    netwrite
        Like netcat but don't read the socket after sending data
    
    count
        Returns the length of the given iterable, counting elements one by one
        >>> [1, 2, 3, 4, 5, 6] | count
        6
    
    add
        Returns the sum of all elements in the preceding iterable
        >>> (1, 2, 3, 4, 5, 6) | add
        21
    
    first
        Returns the first element of the given iterable
        >>> (1, 2, 3, 4, 5, 6) | first
        1
    
    chain
        Unfold preceding Iterable of Iterables
        >>> [[1, 2], [3, 4], [5]] | chain | concat
        '1, 2, 3, 4, 5'
    
        Warning : chain only unfold iterable containing ONLY iterables :
          [1, 2, [3]] | chain
              Gives a TypeError: chain argument #1 must support iteration
              Consider using traverse
    
    traverse
        Recursively unfold iterables
        >>> [[1, 2], [[[3], [[4]]], [5]]] | traverse | concat
        '1, 2, 3, 4, 5'
        >>> squares = (i * i for i in range(3))
        >>> [[0, 1, 2], squares] | traverse | as_list
        [0, 1, 2, 0, 1, 4]
    
    select()
        Apply a conversion expression given as parameter
        to each element of the given iterable
        >>> [1, 2, 3] | select(lambda x: x * x) | concat
        '1, 4, 9'
    
    where()
        Only yields the matching items of the given iterable
        >>> [1, 2, 3] | where(lambda x: x % 2 == 0) | concat
        '2'
    
    take_while()
        Like itertools.takewhile, yields elements of the
        given iterable while the predicat is true
        >>> [1, 2, 3, 4] | take_while(lambda x: x < 3) | concat
        '1, 2'
    
    skip_while()
        Like itertools.dropwhile, skips elements of the given iterable
        while the predicat is true, then yields others
        >>> [1, 2, 3, 4] | skip_while(lambda x: x < 3) | concat
        '3, 4'
    
    chain_with()
        Like itertools.chain, yields elements of the given iterable,
        then yields elements of its parameters
        >>> (1, 2, 3) | chain_with([4, 5], [6]) | concat
        '1, 2, 3, 4, 5, 6'
    
    take()
        Yields the given quantity of elemenets from the given iterable, like head
        in shell script.
        >>> (1, 2, 3, 4, 5) | take(2) | concat
        '1, 2'
    
    skip()
        Skips the given quantity of elements from the given iterable, then yields
        >>> (1, 2, 3, 4, 5) | skip(2) | concat
        '3, 4, 5'
    
    islice()
        Just the itertools.islice
        >>> (1, 2, 3, 4, 5, 6, 7, 8, 9) | islice(2, 8, 2) | concat
        '3, 5, 7'
    
    izip()
        Just the itertools.izip
        >>> (1, 2, 3, 4, 5, 6, 7, 8, 9)             | izip([9, 8, 7, 6, 5, 4, 3, 2, 1])             | concat
        '(1, 9), (2, 8), (3, 7), (4, 6), (5, 5), (6, 4), (7, 3), (8, 2), (9, 1)'
    
    aggregate()
        Works as python reduce
        >>> (1, 2, 3, 4, 5, 6, 7, 8, 9) | aggregate(lambda x, y: x * y)
        362880
    
        Simulate concat :
        >>> (1, 2, 3, 4, 5, 6, 7, 8, 9)             | aggregate(lambda x, y: str(x) + ', ' + str(y))
        '1, 2, 3, 4, 5, 6, 7, 8, 9'
    
    any()
        Returns True if any element of the given iterable satisfies the predicate
        >>> (1, 3, 5, 6, 7) | any(lambda x: x >= 7)
        True
    
        >>> (1, 3, 5, 6, 7) | any(lambda x: x > 7)
        False
    
    all()
        Returns True if all elements of the given iterable
        satisfies the given predicate
        >>> (1, 3, 5, 6, 7) | all(lambda x: x < 7)
        False
    
        >>> (1, 3, 5, 6, 7) | all(lambda x: x <= 7)
        True
    
    max()
        Returns the biggest element, using the given key function if
        provided (or else the identity)
    
        >>> ('aa', 'b', 'foo', 'qwerty', 'bar', 'zoog') | max(key=len)
        'qwerty'
        >>> ('aa', 'b', 'foo', 'qwerty', 'bar', 'zoog') | max()
        'zoog'
        >>> ('aa', 'b', 'foo', 'qwerty', 'bar', 'zoog') | max
        'zoog'
    
    min()
        Returns the smallest element, using the key function if provided
        (or else the identity)
    
        >>> ('aa', 'b', 'foo', 'qwerty', 'bar', 'zoog') | min(key=len)
        'b'
        >>> ('aa', 'b', 'foo', 'qwerty', 'bar', 'zoog') | min
        'aa'
    
    groupby()
        Like itertools.groupby(sorted(iterable, key = keyfunc), keyfunc)
        (1, 2, 3, 4, 5, 6, 7, 8, 9)             | groupby(lambda x: x % 2 and "Even" or "Odd")
                | select(lambda x: "%s : %s" % (x[0], (x[1] | concat(', '))))
                | concat(' / ')
        'Even : 1, 3, 5, 7, 9 / Odd : 2, 4, 6, 8'
    
    sort()
        Like Python's built-in "sorted" primitive.  Allows cmp (Python 2.x
        only), key, and reverse arguments. By default sorts using the
        identity function as the key.
    
        >>> "python" | sort | concat("")
        'hnopty'
        >>> [5, -4, 3, -2, 1] | sort(key=abs) | concat
        '1, -2, 3, -4, 5'
    
    reverse
        Like Python's built-in "reversed" primitive.
        >>> [1, 2, 3] | reverse | concat
        '3, 2, 1'
    
    permutations()
        Returns all possible permutations
        >>> 'ABC' | permutations(2) | concat(' ')
        "('A', 'B') ('A', 'C') ('B', 'A') ('B', 'C') ('C', 'A') ('C', 'B')"
    
        >>> range(3) | permutations | concat('-')
        '(0, 1, 2)-(0, 2, 1)-(1, 0, 2)-(1, 2, 0)-(2, 0, 1)-(2, 1, 0)'
    
    Euler project samples :
    
        # Find the sum of all the multiples of 3 or 5 below 1000.
        euler1 = (itertools.count() | select(lambda x: x * 3) | take_while(lambda x: x < 1000) | add)            + (itertools.count() | select(lambda x: x * 5) | take_while(lambda x: x < 1000) | add)            - (itertools.count() | select(lambda x: x * 15) | take_while(lambda x: x < 1000) | add)
        assert euler1 == 233168
    
        # Find the sum of all the even-valued terms in Fibonacci which do not exceed four million.
        euler2 = fib() | where(lambda x: x % 2 == 0) | take_while(lambda x: x < 4000000) | add
        assert euler2 == 4613732
    
        # Find the difference between the sum of the squares of the first one hundred natural numbers and the square of the sum.
        square = lambda x: x * x
        euler6 = square(itertools.count(1) | take(100) | add) - (itertools.count(1) | take(100) | select(square) | add)
        assert euler6 == 25164150

CLASSES
    Pipe
    
    class Pipe
     |  Represent a Pipeable Element :
     |  Described as :
     |  first = Pipe(lambda iterable: next(iter(iterable)))
     |  and used as :
     |  print [1, 2, 3] | first
     |  printing 1
     |  
     |  Or represent a Pipeable Function :
     |  It's a function returning a Pipe
     |  Described as :
     |  select = Pipe(lambda iterable, pred: (pred(x) for x in iterable))
     |  and used as :
     |  print [1, 2, 3] | select(lambda x: x * 2)
     |  # 2, 4, 6
     |  
     |  Methods defined here:
     |  
     |  __call__(self, *args, **kwargs)
     |  
     |  __init__(self, function)
     |  
     |  __ror__(self, other)

FUNCTIONS
    reduce(...)
        reduce(function, sequence[, initial]) -> value
        
        Apply a function of two arguments cumulatively to the items of a sequence,
        from left to right, so as to reduce the sequence to a single value.
        For example, reduce(lambda x, y: x+y, [1, 2, 3, 4, 5]) calculates
        ((((1+2)+3)+4)+5).  If initial is present, it is placed before the items
        of the sequence in the calculation, and serves as a default when the
        sequence is empty.

DATA
    __author__ = 'Julien Palard <[email protected]>'
    __credits__ = 'Jerome Schneider, for its Python skillz,\nand dalexande...
    __date__ = '10 Nov 2010'
    __version__ = '1.3'
    add = <pipe.Pipe instance>
    aggregate = <pipe.Pipe instance>
    all = <pipe.Pipe instance>
    any = <pipe.Pipe instance>
    as_list = <pipe.Pipe instance>
    as_tuple = <pipe.Pipe instance>
    average = <pipe.Pipe instance>
    chain = <pipe.Pipe instance>
    chain_with = <pipe.Pipe instance>
    concat = <pipe.Pipe instance>
    count = <pipe.Pipe instance>
    first = <pipe.Pipe instance>
    groupby = <pipe.Pipe instance>
    islice = <pipe.Pipe instance>
    izip = <pipe.Pipe instance>
    lineout = <pipe.Pipe instance>
    max = <pipe.Pipe instance>
    min = <pipe.Pipe instance>
    netcat = <pipe.Pipe instance>
    netwrite = <pipe.Pipe instance>
    permutations = <pipe.Pipe instance>
    reverse = <pipe.Pipe instance>
    select = <pipe.Pipe instance>
    skip = <pipe.Pipe instance>
    skip_while = <pipe.Pipe instance>
    sort = <pipe.Pipe instance>
    stdout = <pipe.Pipe instance>
    take = <pipe.Pipe instance>
    take_while = <pipe.Pipe instance>
    traverse = <pipe.Pipe instance>
    where = <pipe.Pipe instance>

VERSION
    1.3

DATE
    10 Nov 2010

AUTHOR
    Julien Palard <[email protected]>

CREDITS
    Jerome Schneider, for its Python skillz,
    and dalexander for contributing