Code Monkey home page Code Monkey logo

rumble's Introduction

LET'S GET READY

Build Status Coverage Status

This is a Python library for easily comparing function runtimes. It's well-tested on 2.6, 2.7, 3.3, 3.4, PyPy's Python 2 and 3 implementations.

Quickstart

Install with

pip install rumble

and see the examples directory here on GitHub for basic use.

Slightly Slower Start

You want to compare runtimes for different implementations of a function -- let's call it primes. It outputs a list of all primes up to and including its argument. The way you used to do this:

  • Create a file called functime.py. Paste in the different implementations of the function into that file -- let's say one uses a naive implementation that checks all numbers below its argument, and the other uses the Sieve of Eratosthenes.
  • Run a bunch of timeit commands:
    • python -m timeit -s 'import functime' 'check_all(50)', and then
    • python -m timeit -s 'import functime' 'sieve(50)', and then
    • python -m timeit -s 'import functime' 'check_all(100)', and then
    • python -m timeit -s 'import functime' 'sieve(100)'
  • Look back through your shell history to see what happened.

Isn't that hard? There's a lot of repetition, a lot of unnecessary name duplication, and some reasoning about local imports.

Now, you can do this:

from rumble import rumble

rumble.arguments(50)
rumble.arguments(100)

@rumble.contender
def check_all(n):
    ...

@rumble.contender
def sieve(n):
    ...

rumble.run()

and it prints out a table showing the runtimes of each of the contenders. I think it's much more readable than your shell history:

args: 100      usec    loops    best of
-----------  ------  -------  ---------
check_all    242.35     1000          3
sieve        138.30    10000          3

args: 500       usec    loops    best of
-----------  -------  -------  ---------
check_all    3539.53      100          3
sieve        2019.23      100          3

Use

See the examples directory for examples of use. At a high level:

  • Optionally, create a new Rumble object. If you'd rather, you can simply call methods from the module itself as well.
  • Decorate some functions as contenders in the Rumble.
  • Add some arguments with the arguments method. This method accepts arbitrary numbers of arguments, including keyword arguments.
    • You can also specify code to be executed before the timing runs with the _setup keyword argument.
    • (For now, each argument a must conform to the condition a == exec(repr(a)). So, for instance, [1, None] works, but lambda: None does not.)
  • Call the run method to time the functions and print your tables!
    • You can also call run(as_string=True) to return the output as a string

For now, the documentation consists of the examples, tests, method definitions, precious few docstrings, and this file. This will improve as part of getting this out of pre-alpha stage.

Contributing/Status

This is something I made for my personal use because I got annoyed with dealing with interactions between the shell and Python for answering questions like this. I'm also using it as a place to practice versioning, package maintenance, design, and testing.

If you use it, I want your feedback. I want to know problems you have that you think this could solve but doesn't yet, I want proposed solutions, I want pull requests, I want issues. Even the name is changeable at this point! And of course, you are welcome to fork and alter for your own purposes as allowed by the license. I'd love to see what you do with it if you want to share!

If you want to contribute, please consider contributing some tests alongside your changes. You'll notice in tests that each module has its own suite of unit tests. You'll notice in .travis.yml that each of those suites is run independently, and if its respective module does not have 100% coverage, the suite fails. Obviously, you don't have to make that happen for a casual contribution, but recognize that that's my endgame -- I'll write the tests myself if they aren't there. If you can make that easy, that'd be much appreciated! But, if you contribute at all: also much appreciated!

License

This code is licensed under the MIT License. Code in adaptiverun.py is derived from the Python 3.4 standard library.

rumble's People

Contributors

mambocab avatar

Stargazers

 avatar Ícaro avatar Ryan McGuire avatar

Watchers

James Cloos avatar  avatar  avatar

rumble's Issues

`time_this` api and `partial`s

I like the apparent of the current time_this API, at least for single-argument functions. However, I think there's a better API trying to get out.

Timer will take a functools.partial as an argument, but that introduces a lot of timing noise, which is probably not acceptable for this application. Thinking on it.

make run() call into a generator

#13 is part of a misguided attempt to print incremental results in once circumstance and return a big string in another. But, incremental results can only mean one thing: yield! So each time (from some inner function) yield the table as a string, and the outermost function can deal with the print-or-stringify grossness.

python 2 compatability

I haven't even tried to run this with 2.7.

  • get examples running on 2.7
  • write tests and get tox running

customizable table-rendering

There should be some way to configure how tables render. This'll require

  • a way to inject a new report_function
  • documentation of the data passed to it
    • including that groups are passed to it separately!

Name

I don't really like the simpletimeit name. And I'd need to get KR's approval before I said it was for humans 😛

Documentation

  • add docstrings
  • add comments to more complicated parts of the code
  • set up rst building locally
  • set up rst building on readthedocs

(Edited to reflect changes in plans)

improve data format on boundry to table

currently, the table generator gets an arguments string and a list of function-timingdata pairs. Really, it should be getting a table title and row name-timingdata pairs! Much more appropriate data format I believe.

fix data structures

adaptiverun currently assigns None to Report.timedfunction. We can do better here. I think a model could be a Result with a TimedFunction and a Report.

checking against a reference implementation

The TimedFunction class should be able to hold a reference function. For some random subset of the runs in a Timer run (or maybe just once) the function function's output should be checked against the reference.

The idea here is that you may want to tweak functions as you work. If you're constantly comparing to a known-correct implementation, you may catch bugs before they bite you.

simplify API

In the common case, the user has <5 functions they want to compare on the same inputs. They shouldn't have to deal with weird machinery for specifying arguments every time they decorate a function. Actually, for the moment, I can probably remove the per-function arguments feature entirely. Instead, the object should have an add_args method or something.

The internals of the object must be designed so that it doesn't matter what order you decorate and add arguments.

One nice thing here is that the "args/kwargs" and "stmt/setup" arguments are totally orthogonal, so we don't actually need to set a "mode" on this thing. Just observe what's coming in, and set the mode accordingly.

The purpose of groups is no longer clear to me, now that you can create multiple Rumble objects. That should be removed from the API.

`flake8`

I need to run flake8 and get things lookin' goooood.

estimate timing overhead

Have each run also run a noop and time that -- that will be an estimate of the overhead of the timing itself. This is important in particular for the partial implementation.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.