Zipline is not to be used as an installed python module? about zipline HOT 10 CLOSED

Thanks for taking the time to report. I tried to reproduce but couldn't. It is a little surprising because you execute the command with sudo.

Is there a more complete traceback perhaps?

from zipline.

Thanks for checking in! Yes there is. Disabling the locally installed module and running in either iPython notebook or via script when I get to:

my_algo = BuyApple() # Instantiate our algorithm
results_buy_apple = my_algo.run(data) # Backtest algorithm on dataframe.

I get

---------------------------------------------------------------------------
IOError                                   Traceback (most recent call last)
<ipython-input-5-7102635a4d33> in <module>()
      1 my_algo = BuyApple() # Instantiate our algorithm
----> 2 results_buy_apple = my_algo.run(data) # Backtest algorithm on dataframe.

/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/zipline/algorithm.pyc in run(self, source, start, end)
    170             self.transforms.append(sf)
    171 
--> 172         environment = create_trading_environment(start=start, end=end)
    173 
    174         # create transforms and zipline

/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/zipline/utils/factory.pyc in create_trading_environment(year, start, end)
     98 def create_trading_environment(year=2006, start=None, end=None):
     99     """Construct a complete environment with reasonable defaults"""
--> 100     benchmark_returns, treasury_curves = load_market_data()
    101 
    102     if start is None:

/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/zipline/utils/factory.pyc in load_market_data()
     55 Fetching data from Yahoo Finance.
     56 """.strip()
---> 57         data.loader.dump_benchmarks()
     58         fp_bm = open(benchmark_data_path, "rb")
     59 

/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/zipline/data/loader.pyc in dump_benchmarks()
     59     benchmark_path = os.path.join(os.path.dirname(__file__),
     60                                   "benchmark.msgpack")
---> 61     benchmark_fp = open(benchmark_path, "wb")
     62     benchmark_data = []
     63     for daily_return in get_benchmark_returns():

IOError: [Errno 13] Permission denied: '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/zipline/data/benchmark.msgpack'

data msgpacks aren't distribute with source.
Fetching data from Yahoo Finance.

So at that point it's trying to write benchmark.msgpack into that path which fails.

from zipline.

Ah, I see. One solution would then be to move the fetching into setup.py rather than during run-time.

from zipline.

Right, it looks we didn't consider the "installed into a root protected directory" properly, because of the use of virtualenv.

Since, these files should probably be updated over time (so that fresh data can be worked with), maybe we should install to a directory not in the library path? e.g. a ~/.zipline/data/ directory, which could then be made configurable later?

from zipline.

I see, yeah that'd work.

In general though I'm not a big fan of smaller tools installing things
outside of their assigned directory. What's wrong with:

1. Packaging the msgpack with the packaged version.
2. Checking if present and downloading to system dir in setup.py.

On Thu, Nov 8, 2012 at 1:32 PM, Eddie Hebert [email protected]:

Right, it looks we didn't consider the "installed into a root protected
directory" properly, because of the use of virtualenv.

Since, these files should probably be updated over time (so that fresh
data can be worked with), maybe we should install to a directory not in the
library path? e.g. a ~/.zipline/data/ directory, which could then be made
configurable later?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/11#issuecomment-10199194.

from zipline.

@twiecki

With regard to 1) Trying not to directly distribute the Yahoo Finance data, rather putting the onus on the developers' machines, so as to comply with my understanding of Yahoo's ToS.

2. We could do it in the setup.py so that it installs along side... but if we start providing tools for easy downloading of other data sets or ability to point at user provided data sets, we'd probably want to have the user lazily load that data iff they want to use that data. Though that is a use case we don't have yet.

I'm thinking there is a distinction between where the library and where the data it runs over exist. But can be convinced otherwise.

from zipline.

1. Ah, you are right. Disregard.

2. The problem is that even for the simplest example we require benchmark
   data (see #13). I think it should just work out of the box. Alternatively
   we could just allow running of zipline without benchmark data (as I
   mentioned in #13).

A mechanism to load data sets once they are needed would be really cool but
makes me think about http://en.wikipedia.org/wiki/YAGNI ;)

On Thu, Nov 8, 2012 at 2:12 PM, Eddie Hebert [email protected]:

@twiecki https://github.com/twiecki

With regard to 1) Trying not to directly distribute the Yahoo Finance
data, rather putting the onus on the developers' machines, so as to comply
with my understanding of Yahoo's ToS.

2. We could do it in the setup.py so that it installs along side... but if
   we start providing tools for easy downloading of other data sets or ability
   to point at user provided data sets, we'd probably want to have the user
   lazily load that data iff they want to use that data. Though that is a use
   case we don't have yet.

I'm thinking there is a distinction between where the library and where
the data it runs over exist. But can be convinced otherwise.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/11#issuecomment-10200660.

from zipline.

As a (brand spanking new) user I agree with point 2 here and YAGNI. It'd be neat but eventually the community would start coming with all kinds of unforeseen ways to pull in data especially since pandas/python readily facilitates this. The way we users use zipline will of course also go beyond what was originally planned. That's a given. :) Because of that, working "out of the box" may be challenging because we'll be reshaping the definition of that box even as you're making it. Which is what I'm doing which brought this problem to light. Heh.

So pandas made it easy to just pull my data in and I then assumed the system would Just Work™. I didn't know (previously) about the benchmark data requirement or the limits of that data.

For now I think I'll attempt to hack in my own benchmark data. Perhaps even a flat unchanging set of data or munging the existing data to allow my data to work with it and get an approximation. Then I can start testing strategies with either an approximated risk report for now or just a basic drawdown and sharpe report.

from zipline.

This has been fixed on master.

I'll update this ticket on the next release to PyPI.

from zipline.

Should be fixed with 0.5.2

from zipline.