python-bonobo / bonobo Goto Github PK

View Code? Open in Web Editor NEW

1.6K 58.0 142.0 1.5 MB

Extract Transform Load for Python 3.5+

Home Page: https://www.bonobo-project.org/

License: Apache License 2.0

Makefile 2.02% Python 92.08% JavaScript 2.13% PowerShell 2.07% Batchfile 0.97% Shell 0.74%

data-processing extract-transform-load bonobo python3 automation parallelization

bonobo's Introduction

🐵 bonobo

Data-processing for humans.

Bonobo is an extract-transform-load framework for python 3.5+ (see comparisons with other data tools).

Bonobo uses plain old python objects (functions, generators and iterators), allows them to be linked together in a directed graph, and then executed using a parallelized strategy, without having to worry about the underlying complexity.

Developers can focus on writing simple and atomic operations, that are easy to unit-test by-design, while the focus of the framework is to apply them concurrently to rows of data.

One thing to note: write pure transformations and you'll be safe.

Bonobo is a young rewrite of an old python2.7 tool that ran millions of transformations per day for years on production. Although it may not yet be complete or fully stable (please, allow us to reach 1.0), the basics are there.

Bonobo is under heavy development, we're doing our best to keep the core as stable as possible while still moving forward. Please allow us to reach 1.0 stability and our sincere apologies for anything we break in the process (feel free to complain on issues, allowing us to correct breakages we did not expect)

Homepage: https://www.bonobo-project.org/ (Roadmap)

Documentation: http://docs.bonobo-project.org/

Contributing guide: http://docs.bonobo-project.org/en/latest/contribute/index.html

Issues: https://github.com/python-bonobo/bonobo/issues

Slack: https://bonobo-slack.herokuapp.com/

Release announcements: http://eepurl.com/csHFKL

Made with ♥ by Romain Dorgueil and contributors.

bonobo's People

Contributors

Stargazers

Watchers

Forkers

hartym jelloslinger dwillmer kwresearch devnull255 bssrdf niejn piotk strogo schevalier raster-blaster alekzvik vit- faic carloca traviscook21 cw-andrews automape tzubiri mouadhkaabachi jasoncrowe arimbr dalesys peterue userzimmermann anukat2015 winsmith knkski uduse oersted ivanistheone peinwu shenjiawei19 papasax dreispt messakali tate11 autoplectic simsoltech alenzhang86 hydrosquall fossabot techlord-rce benrudolph devopsmi biodun enzebo jjsaunier barleyj sksundaram-learning acknology mouedrao levic a-musing-moose maklori odscjames oagbaneje dparks-seattletimes jsubida codeinvain paulholio seanleelacdmh xpf100 kpilnacek gregwalla rogeriofalcone priya-gittest gozer flybywire yijxiang carlosveliz jonathaneunice black-monster euwen gufengxiaoyuehan rocktangcd kasei mdbenito shriganesh-kolhe strataggem pebbleshx pktippa cuulee craigbrownphd bidexbido akc0des abilian juthebest abdellahela unoqualsiasi willmartell mdjones imrehg oatmealdealer shriganesh-kolhe-cardinal javigoncompte m576f167 studiocredo algolink peregrine-io

bonobo's Issues

[cli] "bonobo version" should show where is the source

Enhance bonobo version to show where the package is installed.

Two options (need choice):

either we show it no matter what.
either we add a -v / --verbose flag that adds this information.

[core] Find a way to disable Console or Jupyter plugin even in the right context.

For now, jupyter or interactive console environments are detected, and respective output plugins are automatically added.

This is great, but maybe you do not want that (running in tests, running in another software, etc.).

This is now implemented using environment variables.

Normalize environment variable names. Is the BONOBO_ prefix really necessary ?
Document the settings module.
Tests? Not sure it is needed for this simple feature but could help non-regression later.

[core] Fatal errors handling. May need some interrupt-pipeline control flow tools.

Some fatal errors stay in an infinite loop, like when we have an extractor that raise a FileNotFoundError.

Whatever happens, there should be an interruption of pipeline if the full graph cannot be configured, for example if there is a file reader/writer that cannot open file.

Pass Parameters at Runtime?

#I looked, but didn't see, a way to pass parameters to bonobo at runtime. Specifically, I need the ability to use these at scale and to do that will need the ability to pass additional arguments.

Example:

bonobo run csvsanitizer 2773 inventory.txt

Is there anyway to do that? The closest thing I saw was possibly passing arguments as services.

Please note that though I used a command line example I would primarily be using the python api.

[config] Support for position arguments (options), required options are good candidates.

[stdlib] use terminal width for pprint instead of the default 80

Fix "Complexity" issue in bonobo/config/configurables.py

Cyclomatic complexity is too high in class Configurable. (11)

https://codeclimate.com/github/python-bonobo/bonobo/bonobo/config/configurables.py#issue_5907cbdd7869e70001000083

[windows] pip install -e fails with encoding problem, both in win10 powershell or MINGW64

Not professional windows user here, quite the contrary. Maybe noob mistake.

Maybe related to the way I installed git ? I asked for "clone using windows encoding, commit using unix". Gonna try again with full unix, maybe.

PS C:\bb36\bonobo> pip install -e .
Obtaining file:///C:/bb36/bonobo
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\bb36\bonobo\setup.py", line 48, in <module>
        long_description=read('README.rst'),
      File "C:\bb36\bonobo\setup.py", line 15, in read
        content = f.read().strip()
      File "c:\users\ieuser\appdata\local\programs\python\python36\lib\encodings\cp1252.py", line 23, in decode
        return codecs.charmap_decode(input,self.errors,decoding_table)[0]
    UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 2: character maps to <undefined>

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in C:\bb36\bonobo\

statistics are hard to read on the Ubuntu standard terminal

The standard terminal on a Ubuntu workstation has a dark background. Bonobo uses a dark green/grey for its statistics. So a less dark gray text color for the Bonobo statistics could be more readable. Maybe is possible to find a color that is a good trade off on all the platforms. Lighter green/grey maybe?

[examples] Broken example : examples/datasets/fablabs.py

$ bonobo run bonobo/examples/datasets/fablabs.py
Traceback (most recent call last):
  File "/Users/rd/.pyenv/versions/anaconda3-4.3.1/bin/bonobo", line 11, in <module>
    load_entry_point('bonobo', 'console_scripts', 'bonobo')()
  File "/Users/rd/Projects/Bonobo/bonobo/bonobo/commands/__init__.py", line 28, in entrypoint
    commands[args.pop('command')](**args)
  File "/Users/rd/Projects/Bonobo/bonobo/bonobo/commands/run.py", line 21, in execute
    exec(code, context)
  File "bonobo/examples/datasets/fablabs.py", line 60, in <module>
    OpenDataSoftAPI(dataset=API_DATASET, netloc=API_NETLOC, timezone='Europe/Paris'),
  File "/Users/rd/Projects/Bonobo/bonobo/bonobo/config.py", line 66, in __init__
    len(extraneous), 's' if len(extraneous) > 1 else '', ', '.join(map(repr, sorted(extraneous)))
TypeError: OpenDataSoftAPI() got 1 unexpected option: 'timezone'.

[docker] Needs a way to override easily the image name

[config] Fix the context processors mess

Context processors are too complicated.

There should be a simpler implementation, looking as much possible like Options and Services.

[cli] Option to check for dependencies on bonobo run ?

Maybe an option that checks requirements.txt, if present, and install (or update) dependencies prior to run.

bonobo run --install blah/foo.py

Fix "Similar code" issue in bonobo/execution/plugin.py

Similar code found in 2 other locations (mass = 32)

https://codeclimate.com/github/python-bonobo/bonobo/bonobo/execution/plugin.py#issue_5907cbdb7869e70001000070

[dx] Add a way to download (or generate) the example datasets.

For now, tutorial needs manual download like this:

curl https://raw.githubusercontent.com/python-bonobo/bonobo/master/bonobo/examples/datasets/coffeeshops.txt > `python -c 'import bonobo; print(bonobo.get_examples_path("datasets/coffeeshops.txt"))'`

This is not the best, and although it's not desirable to bundle the datasets within the eggs/wheels (which would add unnecessary files to the packaging), there should be an easy way to download the datasets. Github does not allow git archive, so probably we should add a command that download the missing files.

bonobo examples download ??? Just suggesting, not thought about it.

[cli] Multiple argument parsers when using "bonobo run"

If a transformation wants to define an argument parser, it will conflict because sys.argv is not "prepared" for children execution.

Example use case : oauth with google apis (see https://developers.google.com/sheets/api/quickstart/python), where there is "flags" defined.

[stdlib] More options for CsvWriter input

CsvWriter takes dictionaries as its input. In some cases, it would make sense to have a zip() output or even a value tuple. Let's think about options to support all those cases.

[core] Use inspect module instead of reinvented wheels

https://docs.python.org/3/library/inspect.html

Pretty much all the tools needed to inspect python objects, and that could replace a bit of code.

[pkg] Separate doc and test dependencies to speed up continuous integration

Is there any value of building the doc on travis/appveyor, knowing that readthedocs will do it anyway ?

Consider using Pipfile / pipenv ?

[doc] List of public datasets one can use to try bonobo (or even not just try)

[io] Migration to filesystem2 to have a "filesystem-as-a-service"

test and depend on "fs" (filesystem2) to provide a filesystem to transformations
how to have default services configuration, in examples, for a file, for a dir ...
re-support http/https ? Yes, but somehow differently, no real reason that filesystem supports http. Let's just fix the example that fails, for now.

[doc] there should be only one roadmap

Let's move the hack-ish and outdated roadmap from the readme to https://www.bonobo-project.org/roadmap

Different references (contributing, for example) should link there.

A detail view of each roadmap "chapter" would be great too.

[stdlib] Transformation factory

See https://github.com/hartym/bonobo/tree/transformation_factory

[stdlib] Extract from / Load to files standard types

Although using pandas function names was the first idea, I'm not so much inclined to use lowercase names anymore because we have to implement those as classes anyway. So there are two options, either we drop pandas names completely (which sounds not so unacceptable), or make them available through the bonobo.compat.pandas module, which will just alias things.

Extract from file ...

FileReader (default: one line is one row)
CsvReader (alias: read_csv)
ExcelReader (alias: read_excel)
JsonReader (alias: read_json)
HtmlReader (alias: read_html)
ClipboardReader (alias: read_clipboard)
PickleReader (alias: read_pickle)

Load to file ...

FileWriter (one row, one output line)
CsvWriter (alias: to_csv)
ExcelWriter (alias: to_excel)
JsonWriter (alias: to_json)
HtmlWriter (alias: to_html)
ClipboardWriter (alias: to_clipboard)
PickleWriter (alias: to_pickle)

Still need to stabilize / freeze this API.

Initial Update

Hi 👊

This is my first visit to this fine repo, but it seems you have been working hard to keep all dependencies updated so far.

Once you have closed this issue, I'll create separate pull requests for every update as soon as I find one.

That's it for now!

Happy merging! 🤖

[doc] Make readme look better

Look at other packages readme for inspiration

Why does rst readme show up bad on pypi? See how requests does it, there may be some transformations to make before release, or in setup.py.

[revert][commands] bonobo : alias cli command to bb ?

[console] fcntl Library Does Not Work in Native Windows

When trying to run one of the basic examples from the documentation, I hit the following stack trace:

Traceback (most recent call last):
  File "test.py", line 1, in <module>
    import bonobo
  File "C:\Users\mcopeland\Envs\bonobo\lib\site-packages\bonobo\__init__.py" line 17, in <module>
    from .core import __all__ as __all_core__
  File "C:\Users\mcopeland\Envs\bonobo\lib\site-packages\bonobo\core\__init_.py", line 3, in <module>
    from .bags import Bag, ErrorBag
  File "C:\Users\mcopeland\Envs\bonobo\lib\site-packages\bonobo\core\bags.py" line 3, in <module>
    from bonobo.util.tokens import Token
  File "C:\Users\mcopeland\Envs\bonobo\lib\site-packages\bonobo\util\__init__.py", line 6, in <module>
    import blessings
  File "C:\Users\mcopeland\Envs\bonobo\lib\site-packages\blessings\__init__.py" line 7, in <module>
    from fcntl import ioctl
ImportError: No module named 'fcntl'

It seems that the blessings library leverages fcntl which is used for file and IO control over Unix routines. However, this leaves Windows users out of luck.

The following links have some tidbits of information:

http://stackoverflow.com/questions/1422368/fcntl-substitute-on-windows
cs01/gdbgui#18

This issue seems to plague many other projects (google "fcntl python windows" for a more thorough list of resources) and I haven't found any other examples of how this issue has been handled in a satisfactory manner.

[doc] Examples do not explain well the value prop

The one pager and documentation has a lot of defects.

Few things that were not correctly understood by people discovering bonobo :

«toolz.chain with more code»
«there is no bonobo code here, only functions»
«the dag is a list»
«that's only conceptual, no real world usage»

To fix this, maybe

The website should show more than one example, we can keep the "tab" layout but maybe have 3 different exemples, maybe one with only builtin functions, one that extract a lot of data from an opendata provider (for example, SIREN data which is the french companies registry), and one which use databases.
Find a graphical way to explain parallelism? This one is tough.
Make at least one example with a graph that is not linear (a.k.a not just a list).

tbc.

Allow to --install on a file if a requirements.txt file is in the same directory.

https://github.com/hartym/bonobo/blob/develop/bonobo/commands/run.py

(See the raise RuntimeError('Cannot --install on a file (only available for dirs containing requirements.txt).') line)

Release management: resolve the circular dependency freeze problem

There is a problem with the release process, which is not very bad, but still may become in the future.

We have bonobo[X] (here X is one of docker, sqlalchemy, selenium, etc) depend on bonobo-X.
But bonobo-X depends on bonobo, too.

So we have to find a way to freeze the dependency, because obviously it's hard to be in the future at release time.

Maybe we should force usage of bonobo[X] (which was the favored way while discussing with people) and make sure the freezing is correct this way (freeze version of dep in bonobo for bonobo-X, which means release bonobo-X before bonobo while doing maintenance, and unfreeze bonobo in bonobo-X, that should only rely on bonobo "stable" (and anyway, it wont be encouraged to install extensions this way))

[config] Services and light dependency injection

Proposal / draft implementation: http://docs.bonobo-project.org/en/0.2/guide/services.html

There should be a way to configure default services for a directory, or a file, that works both when using python and bonobo run .
If there are missing services on lauch, then it should stop before transformation runs.
It should also work with bonobo runc (bonobo-docker). This is a bit more complicated, moved to 0.4

Release checklist

Tests
Documentation

[docs] Add a "where to go next" at the end of the tutorial

Could contain :

Release mailing list
Slack channel
Documentation / reference links
etc.

[core] CTRL-C / KeyboardInterrupt should work even if pressed only once.

For now, two keyboardinterrupts are needed to stop a graph execution.

[animals] Get some serious infos about bonobos, monkeys, apes, etc.

Looks like most people care much more about animal reign knowledge than what the actual tool do.

I should probably at least have a light background on that.

IOBuffer has no ability to flush stream (on KeyboardInterrupt, for example).

Exception ignored in: <bonobo.ext.console.IOBuffer object at 0x104e1b978>
AttributeError: 'IOBuffer' object has no attribute 'flush'

Test ticket from Code Climate

[doc] Add a FAQ section.

Collect comments and questions from hackernews, reddit, issues and mails and compile them in a FAQ section.

[doc] Basic documentation

Import things from rdc.etl
Create a better "Quick start"
Rewrite all outdated things.

[doc] Release documentation

Write a how-to for releases, based on http://rdc.li/r but with specific infos for bonobo.

Write a patch note somewhere
Checklist for QA
API doc update
Readthedocs versions add new tag by default, and we don't want that.
Post to slack, article ?

Related to this, we need a process to move to minor version +1

When do we create the branch?
What changes do we need, how to automate it (badges, documentation, etc ...)
More detailed and verbose release note

Post-release QA

Write a process for post-release QA (integration / system using pypi release and pip install)

[cli] bonobo run <path> / bonobo run -m <module>

$ bonobo run somedir
usage: bonobo run [-h] [--quiet] file
bonobo run: error: argument file: can't open 'somedir': [Errno 21] Is a directory: 'somedir'

Also, bonobo run -m .... should work.

[stdlib] PrettyPrint - Rewrite from scratch

Can't print objects, can't print title less (or looks crappy), etc ...

This will be rewritten from scratch. Idea is to print a bag, instead of the first argument, which is a bit unrealistic. One argument Bag is a special case, that should be taken care of, but only a special case.

Other tasks:

use terminal width for pprint instead of the default 80 (#36)

[config] Service injection for function-based transformations

There should be a way to define service dependencies for function-based transformations.

Maybe we can find a way to use the Service class as a decorator:

@Service('database.connection')
def query(db):
    # ...
    pass

Or something similar.

[qa] continuous integration for windows

We need to run the test suite on windows, at least to know potential problems like imports not working, etc.

AppVeyor looks like an option.

https://ci.appveyor.com/project/hartym/bonobo

[stdlib] XLS/XLSX Reader/Writer

Post 0.4 work needed

Allow to run directories or modules using "bonobo run": tests, docs.
Bonobo version command now shows where the package is installed, and an optional "--all/-a" flag show all extensions in the same way: tests, doc (#81)
Bonobo run flag "--install/-I" allow to pip install a requirements.txt file if run targets a directory: tests, docs (#71)
New PrettyPrinter and deprecate old crappy modules: More dev, tests, doc, deprecate old.

[psutil] error: command 'gcc' failed with exit status 1

I am getting the following error with pip install bonobo.

Building wheels for collected packages: psutil
  Running setup.py bdist_wheel for psutil ... error
  Complete output from command /Users/akash/installations/anaconda3/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/t2/tg4jlfqn715g4p9_5yh1kzsc0000gn/T/pip-build-yw2o18bv/psutil/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /var/folders/t2/tg4jlfqn715g4p9_5yh1kzsc0000gn/T/tmpjdqqg2odpip-wheel- --python-tag cp36:
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.macosx-10.7-x86_64-3.6
  creating build/lib.macosx-10.7-x86_64-3.6/psutil
  copying psutil/__init__.py -> build/lib.macosx-10.7-x86_64-3.6/psutil
  copying psutil/_common.py -> build/lib.macosx-10.7-x86_64-3.6/psutil
  copying psutil/_compat.py -> build/lib.macosx-10.7-x86_64-3.6/psutil
  copying psutil/_psbsd.py -> build/lib.macosx-10.7-x86_64-3.6/psutil
  copying psutil/_pslinux.py -> build/lib.macosx-10.7-x86_64-3.6/psutil
  copying psutil/_psosx.py -> build/lib.macosx-10.7-x86_64-3.6/psutil
  copying psutil/_psposix.py -> build/lib.macosx-10.7-x86_64-3.6/psutil
  copying psutil/_pssunos.py -> build/lib.macosx-10.7-x86_64-3.6/psutil
  copying psutil/_pswindows.py -> build/lib.macosx-10.7-x86_64-3.6/psutil
  creating build/lib.macosx-10.7-x86_64-3.6/psutil/tests
  copying psutil/tests/__init__.py -> build/lib.macosx-10.7-x86_64-3.6/psutil/tests
  copying psutil/tests/runner.py -> build/lib.macosx-10.7-x86_64-3.6/psutil/tests
  copying psutil/tests/test_bsd.py -> build/lib.macosx-10.7-x86_64-3.6/psutil/tests
  copying psutil/tests/test_linux.py -> build/lib.macosx-10.7-x86_64-3.6/psutil/tests
  copying psutil/tests/test_memory_leaks.py -> build/lib.macosx-10.7-x86_64-3.6/psutil/tests
  copying psutil/tests/test_misc.py -> build/lib.macosx-10.7-x86_64-3.6/psutil/tests
  copying psutil/tests/test_osx.py -> build/lib.macosx-10.7-x86_64-3.6/psutil/tests
  copying psutil/tests/test_posix.py -> build/lib.macosx-10.7-x86_64-3.6/psutil/tests
  copying psutil/tests/test_process.py -> build/lib.macosx-10.7-x86_64-3.6/psutil/tests
  copying psutil/tests/test_sunos.py -> build/lib.macosx-10.7-x86_64-3.6/psutil/tests
  copying psutil/tests/test_system.py -> build/lib.macosx-10.7-x86_64-3.6/psutil/tests
  copying psutil/tests/test_windows.py -> build/lib.macosx-10.7-x86_64-3.6/psutil/tests
  running build_ext
  building 'psutil._psutil_osx' extension
  creating build/temp.macosx-10.7-x86_64-3.6
  creating build/temp.macosx-10.7-x86_64-3.6/psutil
  creating build/temp.macosx-10.7-x86_64-3.6/psutil/arch
  creating build/temp.macosx-10.7-x86_64-3.6/psutil/arch/osx
  gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/akash/installations/anaconda3/include -arch x86_64 -I/Users/akash/installations/anaconda3/include -arch x86_64 -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=501 -DPSUTIL_OSX=1 -I/Users/akash/installations/anaconda3/include/python3.6m -c psutil/_psutil_osx.c -o build/temp.macosx-10.7-x86_64-3.6/psutil/_psutil_osx.o
  In file included from /usr/include/Availability.h:190:0,
                   from /usr/include/stdio.h:65,
                   from /Users/akash/installations/anaconda3/include/python3.6m/Python.h:25,
                   from psutil/_psutil_osx.c:9:
  /System/Library/Frameworks/CoreFoundation.framework/Headers/CFDateFormatter.h:53:34: error: 'introduced' undeclared here (not in a function)
       kCFISO8601DateFormatWithYear API_AVAILABLE(macosx(10.12), ios(10.0), watchos(3.0), tvos(10.0)) = (1UL << 0),
                                    ^
  /System/Library/Frameworks/CoreFoundation.framework/Headers/CFURL.h:777:39: error: 'deprecated' undeclared here (not in a function)
   const CFStringRef kCFURLLabelColorKey API_DEPRECATED("Use NSURLLabelColorKey", macosx(10.6, 10.12), ios(4.0, 10.0), watchos(2.0, 3.0), tvos(9.0, 10.0));
                                         ^
  /System/Library/Frameworks/CoreFoundation.framework/Headers/CFURL.h:777:39: error: 'message' undeclared here (not in a function)
   const CFStringRef kCFURLLabelColorKey API_DEPRECATED("Use NSURLLabelColorKey", macosx(10.6, 10.12), ios(4.0, 10.0), watchos(2.0, 3.0), tvos(9.0, 10.0));
                                         ^
  error: command 'gcc' failed with exit status 1

  ----------------------------------------
  Failed building wheel for psutil
  Running setup.py clean for psutil
Failed to build psutil
Installing collected packages: psutil, blessings, requests, stevedore, bonobo
  Found existing installation: psutil 5.1.3
    DEPRECATION: Uninstalling a distutils installed project (psutil) has been deprecated and will be removed in a future version. This is due to the fact that uninstalling a distutils project will only partially uninstall the project.
    Uninstalling psutil-5.1.3:
      Successfully uninstalled psutil-5.1.3
  Running setup.py install for psutil ... error
    Complete output from command /Users/akash/installations/anaconda3/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/t2/tg4jlfqn715g4p9_5yh1kzsc0000gn/T/pip-build-yw2o18bv/psutil/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /var/folders/t2/tg4jlfqn715g4p9_5yh1kzsc0000gn/T/pip-m1qlazv0-record/install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_py
    creating build
    creating build/lib.macosx-10.7-x86_64-3.6
    creating build/lib.macosx-10.7-x86_64-3.6/psutil
    copying psutil/__init__.py -> build/lib.macosx-10.7-x86_64-3.6/psutil
    copying psutil/_common.py -> build/lib.macosx-10.7-x86_64-3.6/psutil
    copying psutil/_compat.py -> build/lib.macosx-10.7-x86_64-3.6/psutil
    copying psutil/_psbsd.py -> build/lib.macosx-10.7-x86_64-3.6/psutil
    copying psutil/_pslinux.py -> build/lib.macosx-10.7-x86_64-3.6/psutil
    copying psutil/_psosx.py -> build/lib.macosx-10.7-x86_64-3.6/psutil
    copying psutil/_psposix.py -> build/lib.macosx-10.7-x86_64-3.6/psutil
    copying psutil/_pssunos.py -> build/lib.macosx-10.7-x86_64-3.6/psutil
    copying psutil/_pswindows.py -> build/lib.macosx-10.7-x86_64-3.6/psutil
    creating build/lib.macosx-10.7-x86_64-3.6/psutil/tests
    copying psutil/tests/__init__.py -> build/lib.macosx-10.7-x86_64-3.6/psutil/tests
    copying psutil/tests/runner.py -> build/lib.macosx-10.7-x86_64-3.6/psutil/tests
    copying psutil/tests/test_bsd.py -> build/lib.macosx-10.7-x86_64-3.6/psutil/tests
    copying psutil/tests/test_linux.py -> build/lib.macosx-10.7-x86_64-3.6/psutil/tests
    copying psutil/tests/test_memory_leaks.py -> build/lib.macosx-10.7-x86_64-3.6/psutil/tests
    copying psutil/tests/test_misc.py -> build/lib.macosx-10.7-x86_64-3.6/psutil/tests
    copying psutil/tests/test_osx.py -> build/lib.macosx-10.7-x86_64-3.6/psutil/tests
    copying psutil/tests/test_posix.py -> build/lib.macosx-10.7-x86_64-3.6/psutil/tests
    copying psutil/tests/test_process.py -> build/lib.macosx-10.7-x86_64-3.6/psutil/tests
    copying psutil/tests/test_sunos.py -> build/lib.macosx-10.7-x86_64-3.6/psutil/tests
    copying psutil/tests/test_system.py -> build/lib.macosx-10.7-x86_64-3.6/psutil/tests
    copying psutil/tests/test_windows.py -> build/lib.macosx-10.7-x86_64-3.6/psutil/tests
    running build_ext
    building 'psutil._psutil_osx' extension
    creating build/temp.macosx-10.7-x86_64-3.6
    creating build/temp.macosx-10.7-x86_64-3.6/psutil
    creating build/temp.macosx-10.7-x86_64-3.6/psutil/arch
    creating build/temp.macosx-10.7-x86_64-3.6/psutil/arch/osx
    gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/akash/installations/anaconda3/include -arch x86_64 -I/Users/akash/installations/anaconda3/include -arch x86_64 -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=501 -DPSUTIL_OSX=1 -I/Users/akash/installations/anaconda3/include/python3.6m -c psutil/_psutil_osx.c -o build/temp.macosx-10.7-x86_64-3.6/psutil/_psutil_osx.o
    In file included from /usr/include/Availability.h:190:0,
                     from /usr/include/stdio.h:65,
                     from /Users/akash/installations/anaconda3/include/python3.6m/Python.h:25,
                     from psutil/_psutil_osx.c:9:
    /System/Library/Frameworks/CoreFoundation.framework/Headers/CFDateFormatter.h:53:34: error: 'introduced' undeclared here (not in a function)
         kCFISO8601DateFormatWithYear API_AVAILABLE(macosx(10.12), ios(10.0), watchos(3.0), tvos(10.0)) = (1UL << 0),
                                      ^
    /System/Library/Frameworks/CoreFoundation.framework/Headers/CFURL.h:777:39: error: 'deprecated' undeclared here (not in a function)
     const CFStringRef kCFURLLabelColorKey API_DEPRECATED("Use NSURLLabelColorKey", macosx(10.6, 10.12), ios(4.0, 10.0), watchos(2.0, 3.0), tvos(9.0, 10.0));
                                           ^
    /System/Library/Frameworks/CoreFoundation.framework/Headers/CFURL.h:777:39: error: 'message' undeclared here (not in a function)
     const CFStringRef kCFURLLabelColorKey API_DEPRECATED("Use NSURLLabelColorKey", macosx(10.6, 10.12), ios(4.0, 10.0), watchos(2.0, 3.0), tvos(9.0, 10.0));
                                           ^
    error: command 'gcc' failed with exit status 1

    ----------------------------------------
  Rolling back uninstall of psutil
Command "/Users/akash/installations/anaconda3/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/t2/tg4jlfqn715g4p9_5yh1kzsc0000gn/T/pip-build-yw2o18bv/psutil/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /var/folders/t2/tg4jlfqn715g4p9_5yh1kzsc0000gn/T/pip-m1qlazv0-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/t2/tg4jlfqn715g4p9_5yh1kzsc0000gn/T/pip-build-yw2o18bv/psutil/

I have gcc 6.3 installed.
gcc (MacPorts gcc6 6.3.0_2) 6.3.0

Windows related issues

#134 -> no easy way to download example datasets (fixed by @parthiv20)
Console output is quirky