Especially for state space models. Aim has always been to replace mean-field structure with something more appropriate, but in the meantime need to flag in the documentation and examples that this is imperfect because mean-field does not properly account for dependencies in the posterior.

Decided to do more work on this before releasing the beta:

• Fix the log-likelihood term
• Implement iterative GP
• Implement PITC pre-conditioning
• Refactor to allow custom covariance matrices
• Custom tolerance bounds for iterative GP

For some models - e.g. GARCH - parameters are constrained so optimization is smoother, but it means when they are printed, the standard errors are not printed alongside. Should ideally untransform them after optimization is complete, then pass them through the likelihood to get the Inverse Hessian -> standard errors

I know your feature list is long but maybe some property based testing a la Hypothesis would be good.

Bug in the recursions for the Kalman smoother; now fixed and will be part of the next release.

I have pandas 18.0 and python 3.5 installed. As these are both fairly new could this be the issue?

Here is the full error

(py35) C:\Users\admin>pip install pyflux
Collecting pyflux
  Downloading pyflux-0.3.6.tar.gz (869kB)
    100% |################################| 870kB 1.1MB/s
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\admin\AppData\Local\Temp\pip-build-3sckmt1h\pyflux\setup.py
", line 92, in <module>
        setup_package()
      File "C:\Users\admin\AppData\Local\Temp\pip-build-3sckmt1h\pyflux\setup.py
", line 53, in setup_package
        from numpy.distutils.core import setup
      File "C:\Anaconda\envs\py35\lib\site-packages\numpy\__init__.py", line 180
, in <module>
        from . import add_newdocs
      File "C:\Anaconda\envs\py35\lib\site-packages\numpy\add_newdocs.py", line
13, in <module>
        from numpy.lib import add_newdoc
      File "C:\Anaconda\envs\py35\lib\site-packages\numpy\lib\__init__.py", line
 8, in <module>
        from .type_check import *
      File "C:\Anaconda\envs\py35\lib\site-packages\numpy\lib\type_check.py", li
ne 11, in <module>
        import numpy.core.numeric as _nx
      File "C:\Anaconda\envs\py35\lib\site-packages\numpy\core\__init__.py", lin
e 34, in <module>
        from . import function_base
      File "C:\Anaconda\envs\py35\lib\site-packages\numpy\core\function_base.py"
, line 6, in <module>
        from .numeric import result_type, NaN, shares_memory, MAY_SHARE_BOUNDS,
TooHardError
    ImportError: cannot import name 'shares_memory'

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in C:\Users\admin\Ap
pData\Local\Temp\pip-build-3sckmt1h\pyflux\

Some models with complicated distributions - e.g. skew t - don't often play well. Possible solutions include better starting values, or warning the user more explicitly that something may not have gone well with estimation.

Curious if you've had a look at the statespace work going on in statsmodels? @ChadFulton

https://github.com/RJT1990/pyflux/blob/master/pyflux/gas/gasreg.py

Scores returned - bug, should return full array

Predict_is does not currently iterate over time (leading to constant predictions), and user has reported BBVI problems.

Potentially useful for some model types, e.g. GARCH

First state defaults to 0 - should be smoothed out; is causing BBVI to nan-out with some datasets.

Getting closer to a release that I'm satisfied to call a beta release. Here is my timeline:

First Half - Finalize existing models

• GP-NARX improvements - kernel creation and iterative estimation
• Gaussian State Space models - fix the smoothing recursion; review and clean up code.
• Non-Gaussian State Space models - review and clean up code
• VAR models - review and clean up code
• Metropolis-Hastings - fix scaling, review and clean up code

Second Half - Speed up existing models, clean code

• Cythonization
• Clean up backend (tsm.py, results.py, etc)

Problem: One of the difficulties with GAS models is that they often require good initial starting values for optimization to work effectively. Unlike ARIMA models, there is not an extensive history of computational implementation (and 'tricks of the trade').

Task: Find better starting values for GAS based models.

Potential ideas:

• Initialize the constant as a function of the sample mean of the data?
• Solve a simpler problem first, then use the results to help set starting parameters - e.g. estimate the parameters for a t-distribution (degrees of freedom, scale) and set these as starting values in the GAS based model.
• Simple grid-search of score and AR parameters - the set with the highest likelihood is chosen as the starting point?

Should be based on the EGARCH equivalents, but need to take into account additional skewness parameter and the effects of the parameter on the location of the predicted series.

Status quo: At the moment it calculates future dates using the daily date difference for the two most recent observations. Need to generalize for:

• Higher frequency time series - where the unit is seconds/minutes/hours
• Irregular time series - for example, a stock price series where there is no trading on particular days.

• Rewrite documentation
• Put notebooks on PyFlux website
• Finish Cythonization
• Review documented code
• GP-NARX and Non-Gaussian state space reform
• Ensure standard errors are shown for GARCH Based models
• Poisson overflow fix

A lot of places in the code there are references to index numbers of the latent variables; this makes for unclear, hard-to-read code. Should utilise dictionary keys and values instead.

Makes a hella lot of sense to use this instead of Monte Carlo gradients. Difficulty is that we're not using a automatic differentiation backend or utility, so would need to specify model gradients manually? Links into broader points about making models amenable to stochastic optimization (mini-batches) as well as using non-numerical gradients.

install fails on OS X 10.11.6 with gcc 5.0.0 20141005.

Compilation complains that file limits.h does not exist, but it is there.

jeff ~$: pip install pyflux
Collecting pyflux
Using cached pyflux-0.3.6.tar.gz
Requirement already satisfied (use --upgrade to upgrade): numpy in ./anaconda/lib/python2.7/site-packages (from pyflux)
Requirement already satisfied (use --upgrade to upgrade): pandas in ./anaconda/lib/python2.7/site-packages (from pyflux)
Requirement already satisfied (use --upgrade to upgrade): scipy in ./anaconda/lib/python2.7/site-packages (from pyflux)
Requirement already satisfied (use --upgrade to upgrade): matplotlib in ./anaconda/lib/python2.7/site-packages (from pyflux)
Requirement already satisfied (use --upgrade to upgrade): seaborn in ./anaconda/lib/python2.7/site-packages (from pyflux)
Requirement already satisfied (use --upgrade to upgrade): numdifftools in ./anaconda/lib/python2.7/site-packages (from pyflux)
Requirement already satisfied (use --upgrade to upgrade): patsy in ./anaconda/lib/python2.7/site-packages (from pyflux)
Requirement already satisfied (use --upgrade to upgrade): python-dateutil in ./anaconda/lib/python2.7/site-packages (from pandas->pyflux)
Requirement already satisfied (use --upgrade to upgrade): pytz>=2011k in ./anaconda/lib/python2.7/site-packages (from pandas->pyflux)
Requirement already satisfied (use --upgrade to upgrade): cycler in ./anaconda/lib/python2.7/site-packages (from matplotlib->pyflux)
Requirement already satisfied (use --upgrade to upgrade): pyparsing!=2.0.4,>=1.5.6 in ./anaconda/lib/python2.7/site-packages (from matplotlib->pyflux)
Requirement already satisfied (use --upgrade to upgrade): algopy>=0.4 in ./anaconda/lib/python2.7/site-packages (from numdifftools->pyflux)
Requirement already satisfied (use --upgrade to upgrade): setuptools>=9.0 in ./anaconda/lib/python2.7/site-packages/setuptools-20.3-py2.7.egg (from numdifftools->pyflux)
Requirement already satisfied (use --upgrade to upgrade): six in ./anaconda/lib/python2.7/site-packages (from patsy->pyflux)
Building wheels for collected packages: pyflux
Running setup.py bdist_wheel for pyflux ... error
Complete output from command /Volumes/ext/Users/jeff/anaconda/bin/python -u -c "import setuptools, tokenize;file='/private/var/folders/n7/cxk7jvfh8xld8w006s6g4lmr0000gp/T/pip-build-QzMJ4N/pyflux/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" bdist_wheel -d /var/folders/n7/cxk7jvfh8xld8w006s6g4lmr0000gp/T/tmpcxgdgcpip-wheel- --python-tag cp27:
running bdist_wheel
running build
running config_cc
unifing config_cc, config, build_clib, build_ext, build commands --compiler options
running config_fc
unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options
running build_src
build_src
building extension "pyflux.check_build._check_build" sources
building extension "pyflux.arma.arma_recursions" sources
building extension "pyflux.garch.garch_recursions" sources
building extension "pyflux.gas.gas_recursions" sources
building extension "pyflux.gpnarx.kernel_routines" sources
building extension "pyflux.inference.metropolis_sampler" sources
building extension "pyflux.inference.bbvi_routines" sources
building extension "pyflux.ssm.kalman" sources
building extension "pyflux.var.var_recursions" sources
build_src: building npy-pkg config files
running build_py
creating build
creating build/lib.macosx-10.5-x86_64-2.7
creating build/lib.macosx-10.5-x86_64-2.7/pyflux
copying pyflux/__init.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux
copying pyflux/covariances.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux
copying pyflux/data_check.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux
copying pyflux/latent_variables.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux
copying pyflux/results.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux
copying pyflux/setup.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux
copying pyflux/tsm.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux
creating build/lib.macosx-10.5-x86_64-2.7/pyflux/check_build
copying pyflux/__check_build/__init.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/check_build
copying pyflux/__check_build/setup.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/__check_build
creating build/lib.macosx-10.5-x86_64-2.7/pyflux/arma
copying pyflux/arma/__init.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/arma
copying pyflux/arma/arimax.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/arma
copying pyflux/arma/arma.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/arma
copying pyflux/arma/setup.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/arma
creating build/lib.macosx-10.5-x86_64-2.7/pyflux/distributions
copying pyflux/distributions/init.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/distributions
copying pyflux/distributions/distributions.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/distributions
copying pyflux/distributions/skewt.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/distributions
creating build/lib.macosx-10.5-x86_64-2.7/pyflux/garch
copying pyflux/garch/init.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/garch
copying pyflux/garch/egarch.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/garch
copying pyflux/garch/egarchm.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/garch
copying pyflux/garch/egarchmreg.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/garch
copying pyflux/garch/garch.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/garch
copying pyflux/garch/lmegarch.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/garch
copying pyflux/garch/lmsegarch.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/garch
copying pyflux/garch/segarch.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/garch
copying pyflux/garch/segarchm.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/garch
copying pyflux/garch/setup.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/garch
creating build/lib.macosx-10.5-x86_64-2.7/pyflux/gas
copying pyflux/gas/init.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/gas
copying pyflux/gas/gas.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/gas
copying pyflux/gas/gasllm.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/gas
copying pyflux/gas/gasllt.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/gas
copying pyflux/gas/gasmodels.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/gas
copying pyflux/gas/gasrank.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/gas
copying pyflux/gas/gasreg.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/gas
copying pyflux/gas/gasx.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/gas
copying pyflux/gas/scores.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/gas
copying pyflux/gas/setup.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/gas
creating build/lib.macosx-10.5-x86_64-2.7/pyflux/gpnarx
copying pyflux/gpnarx/init.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/gpnarx
copying pyflux/gpnarx/gpnarx.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/gpnarx
copying pyflux/gpnarx/gpnarxnew.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/gpnarx
copying pyflux/gpnarx/gpnarxnew2.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/gpnarx
copying pyflux/gpnarx/kernels.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/gpnarx
copying pyflux/gpnarx/setup.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/gpnarx
creating build/lib.macosx-10.5-x86_64-2.7/pyflux/inference
copying pyflux/inference/init.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/inference
copying pyflux/inference/bbvi.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/inference
copying pyflux/inference/metropolis_hastings.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/inference
copying pyflux/inference/norm_post_sim.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/inference
copying pyflux/inference/priors.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/inference
copying pyflux/inference/setup.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/inference
copying pyflux/inference/stoch_optim.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/inference
creating build/lib.macosx-10.5-x86_64-2.7/pyflux/output
copying pyflux/output/init.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/output
copying pyflux/output/tableprinter.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/output
creating build/lib.macosx-10.5-x86_64-2.7/pyflux/ssm
copying pyflux/ssm/init.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/ssm
copying pyflux/ssm/dynlin.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/ssm
copying pyflux/ssm/llm.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/ssm
copying pyflux/ssm/llt.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/ssm
copying pyflux/ssm/ndynlin.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/ssm
copying pyflux/ssm/nllm.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/ssm
copying pyflux/ssm/nllt.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/ssm
copying pyflux/ssm/setup.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/ssm
creating build/lib.macosx-10.5-x86_64-2.7/pyflux/tests
copying pyflux/tests/init.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/tests
copying pyflux/tests/nhst.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/tests
creating build/lib.macosx-10.5-x86_64-2.7/pyflux/var
copying pyflux/var/init.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/var
copying pyflux/var/setup.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/var
copying pyflux/var/var.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/var
running build_ext
customize UnixCCompiler
customize UnixCCompiler using build_ext
building 'pyflux.__check_build._check_build' extension
compiling C sources
C compiler: gcc -fno-strict-aliasing -I/Volumes/ext/Users/jeff/anaconda/include -arch x86_64 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes

creating build/temp.macosx-10.5-x86_64-2.7
creating build/temp.macosx-10.5-x86_64-2.7/pyflux
creating build/temp.macosx-10.5-x86_64-2.7/pyflux/__check_build
compile options: '-I/Volumes/ext/Users/jeff/anaconda/lib/python2.7/site-packages/numpy/core/include -I/Volumes/ext/Users/jeff/anaconda/include/python2.7 -c'
gcc: pyflux/__check_build/_check_build.c
In file included from /usr/local/lib/gcc/x86_64-apple-darwin14.0.0/5.0.0/include-fixed/syslimits.h:7:0,
from /usr/local/lib/gcc/x86_64-apple-darwin14.0.0/5.0.0/include-fixed/limits.h:34,
from /Volumes/ext/Users/jeff/anaconda/include/python2.7/Python.h:19,
from pyflux/__check_build/_check_build.c:4:
/usr/local/lib/gcc/x86_64-apple-darwin14.0.0/5.0.0/include-fixed/limits.h:168:61: fatal error: limits.h: No such file or directory
compilation terminated.
In file included from /usr/local/lib/gcc/x86_64-apple-darwin14.0.0/5.0.0/include-fixed/syslimits.h:7:0,
from /usr/local/lib/gcc/x86_64-apple-darwin14.0.0/5.0.0/include-fixed/limits.h:34,
from /Volumes/ext/Users/jeff/anaconda/include/python2.7/Python.h:19,
from pyflux/__check_build/_check_build.c:4:
/usr/local/lib/gcc/x86_64-apple-darwin14.0.0/5.0.0/include-fixed/limits.h:168:61: fatal error: limits.h: No such file or directory
compilation terminated.
error: Command "gcc -fno-strict-aliasing -I/Volumes/ext/Users/jeff/anaconda/include -arch x86_64 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Volumes/ext/Users/jeff/anaconda/lib/python2.7/site-packages/numpy/core/include -I/Volumes/ext/Users/jeff/anaconda/include/python2.7 -c pyflux/__check_build/_check_build.c -o build/temp.macosx-10.5-x86_64-2.7/pyflux/__check_build/_check_build.o" failed with exit status 1

Failed building wheel for pyflux
Running setup.py clean for pyflux
Failed to build pyflux
Installing collected packages: pyflux
Running setup.py install for pyflux ... error
Complete output from command /Volumes/ext/Users/jeff/anaconda/bin/python -u -c "import setuptools, tokenize;file='/private/var/folders/n7/cxk7jvfh8xld8w006s6g4lmr0000gp/T/pip-build-QzMJ4N/pyflux/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /var/folders/n7/cxk7jvfh8xld8w006s6g4lmr0000gp/T/pip-Y0Tz9Y-record/install-record.txt --single-version-externally-managed --compile:
running install
running build
running config_cc
unifing config_cc, config, build_clib, build_ext, build commands --compiler options
running config_fc
unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options
running build_src
build_src
building extension "pyflux.check_build._check_build" sources
building extension "pyflux.arma.arma_recursions" sources
building extension "pyflux.garch.garch_recursions" sources
building extension "pyflux.gas.gas_recursions" sources
building extension "pyflux.gpnarx.kernel_routines" sources
building extension "pyflux.inference.metropolis_sampler" sources
building extension "pyflux.inference.bbvi_routines" sources
building extension "pyflux.ssm.kalman" sources
building extension "pyflux.var.var_recursions" sources
build_src: building npy-pkg config files
running build_py
creating build
creating build/lib.macosx-10.5-x86_64-2.7
creating build/lib.macosx-10.5-x86_64-2.7/pyflux
copying pyflux/__init.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux
copying pyflux/covariances.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux
copying pyflux/data_check.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux
copying pyflux/latent_variables.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux
copying pyflux/results.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux
copying pyflux/setup.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux
copying pyflux/tsm.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux
creating build/lib.macosx-10.5-x86_64-2.7/pyflux/check_build
copying pyflux/__check_build/__init.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/check_build
copying pyflux/__check_build/setup.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/__check_build
creating build/lib.macosx-10.5-x86_64-2.7/pyflux/arma
copying pyflux/arma/__init.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/arma
copying pyflux/arma/arimax.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/arma
copying pyflux/arma/arma.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/arma
copying pyflux/arma/setup.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/arma
creating build/lib.macosx-10.5-x86_64-2.7/pyflux/distributions
copying pyflux/distributions/init.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/distributions
copying pyflux/distributions/distributions.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/distributions
copying pyflux/distributions/skewt.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/distributions
creating build/lib.macosx-10.5-x86_64-2.7/pyflux/garch
copying pyflux/garch/init.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/garch
copying pyflux/garch/egarch.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/garch
copying pyflux/garch/egarchm.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/garch
copying pyflux/garch/egarchmreg.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/garch
copying pyflux/garch/garch.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/garch
copying pyflux/garch/lmegarch.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/garch
copying pyflux/garch/lmsegarch.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/garch
copying pyflux/garch/segarch.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/garch
copying pyflux/garch/segarchm.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/garch
copying pyflux/garch/setup.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/garch
creating build/lib.macosx-10.5-x86_64-2.7/pyflux/gas
copying pyflux/gas/init.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/gas
copying pyflux/gas/gas.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/gas
copying pyflux/gas/gasllm.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/gas
copying pyflux/gas/gasllt.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/gas
copying pyflux/gas/gasmodels.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/gas
copying pyflux/gas/gasrank.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/gas
copying pyflux/gas/gasreg.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/gas
copying pyflux/gas/gasx.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/gas
copying pyflux/gas/scores.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/gas
copying pyflux/gas/setup.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/gas
creating build/lib.macosx-10.5-x86_64-2.7/pyflux/gpnarx
copying pyflux/gpnarx/init.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/gpnarx
copying pyflux/gpnarx/gpnarx.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/gpnarx
copying pyflux/gpnarx/gpnarxnew.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/gpnarx
copying pyflux/gpnarx/gpnarxnew2.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/gpnarx
copying pyflux/gpnarx/kernels.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/gpnarx
copying pyflux/gpnarx/setup.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/gpnarx
creating build/lib.macosx-10.5-x86_64-2.7/pyflux/inference
copying pyflux/inference/init.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/inference
copying pyflux/inference/bbvi.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/inference
copying pyflux/inference/metropolis_hastings.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/inference
copying pyflux/inference/norm_post_sim.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/inference
copying pyflux/inference/priors.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/inference
copying pyflux/inference/setup.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/inference
copying pyflux/inference/stoch_optim.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/inference
creating build/lib.macosx-10.5-x86_64-2.7/pyflux/output
copying pyflux/output/init.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/output
copying pyflux/output/tableprinter.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/output
creating build/lib.macosx-10.5-x86_64-2.7/pyflux/ssm
copying pyflux/ssm/init.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/ssm
copying pyflux/ssm/dynlin.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/ssm
copying pyflux/ssm/llm.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/ssm
copying pyflux/ssm/llt.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/ssm
copying pyflux/ssm/ndynlin.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/ssm
copying pyflux/ssm/nllm.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/ssm
copying pyflux/ssm/nllt.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/ssm
copying pyflux/ssm/setup.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/ssm
creating build/lib.macosx-10.5-x86_64-2.7/pyflux/tests
copying pyflux/tests/init.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/tests
copying pyflux/tests/nhst.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/tests
creating build/lib.macosx-10.5-x86_64-2.7/pyflux/var
copying pyflux/var/init.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/var
copying pyflux/var/setup.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/var
copying pyflux/var/var.py -> build/lib.macosx-10.5-x86_64-2.7/pyflux/var
running build_ext
customize UnixCCompiler
customize UnixCCompiler using build_ext
building 'pyflux.__check_build._check_build' extension
compiling C sources
C compiler: gcc -fno-strict-aliasing -I/Volumes/ext/Users/jeff/anaconda/include -arch x86_64 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes

creating build/temp.macosx-10.5-x86_64-2.7
creating build/temp.macosx-10.5-x86_64-2.7/pyflux
creating build/temp.macosx-10.5-x86_64-2.7/pyflux/__check_build
compile options: '-I/Volumes/ext/Users/jeff/anaconda/lib/python2.7/site-packages/numpy/core/include -I/Volumes/ext/Users/jeff/anaconda/include/python2.7 -c'
gcc: pyflux/__check_build/_check_build.c
In file included from /usr/local/lib/gcc/x86_64-apple-darwin14.0.0/5.0.0/include-fixed/syslimits.h:7:0,
                 from /usr/local/lib/gcc/x86_64-apple-darwin14.0.0/5.0.0/include-fixed/limits.h:34,
                 from /Volumes/ext/Users/jeff/anaconda/include/python2.7/Python.h:19,
                 from pyflux/__check_build/_check_build.c:4:
/usr/local/lib/gcc/x86_64-apple-darwin14.0.0/5.0.0/include-fixed/limits.h:168:61: fatal error: limits.h: No such file or directory
compilation terminated.
In file included from /usr/local/lib/gcc/x86_64-apple-darwin14.0.0/5.0.0/include-fixed/syslimits.h:7:0,
                 from /usr/local/lib/gcc/x86_64-apple-darwin14.0.0/5.0.0/include-fixed/limits.h:34,
                 from /Volumes/ext/Users/jeff/anaconda/include/python2.7/Python.h:19,
                 from pyflux/__check_build/_check_build.c:4:
/usr/local/lib/gcc/x86_64-apple-darwin14.0.0/5.0.0/include-fixed/limits.h:168:61: fatal error: limits.h: No such file or directory
compilation terminated.
error: Command "gcc -fno-strict-aliasing -I/Volumes/ext/Users/jeff/anaconda/include -arch x86_64 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Volumes/ext/Users/jeff/anaconda/lib/python2.7/site-packages/numpy/core/include -I/Volumes/ext/Users/jeff/anaconda/include/python2.7 -c pyflux/__check_build/_check_build.c -o build/temp.macosx-10.5-x86_64-2.7/pyflux/__check_build/_check_build.o" failed with exit status 1

----------------------------------------

Command "/Volumes/ext/Users/jeff/anaconda/bin/python -u -c "import setuptools, tokenize;file='/private/var/folders/n7/cxk7jvfh8xld8w006s6g4lmr0000gp/T/pip-build-QzMJ4N/pyflux/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /var/folders/n7/cxk7jvfh8xld8w006s6g4lmr0000gp/T/pip-Y0Tz9Y-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/n7/cxk7jvfh8xld8w006s6g4lmr0000gp/T/pip-build-QzMJ4N/pyflux/

ValueError: For numerical factors, num_columns must be an int

Suppose I want to forecast a series, using itself plus multiple other series as exogenous variables. Is it possible? can you provide a simple example?

Currently under progress and set to be in next release. The current Metropolis implementation is not fit for purpose, but BBVI seems to be working a lot better so I have promise that this can be a long term solution.

Now that a core model set is available, it is time to look to optimize the code to allow for quicker estimation and scalability to larger datasets.

Model

• ARIMA models
• GARCH models
• GAS models
• GAS Regression models
• GP-NARX models
• Gaussian State Space models
• Non-Gaussian State Space models
• VAR models

Inference

• Black Box Variational Inference
• Metropolis-Hastings

NaNs/infs out for large count data. Needs to be written to avoid overflow in the exponentiated terms.

Normalizing Nile data - BBVI works fine, but less well on non-normalized data. Need to investigate further.

could you give some example about ARIMAX model's predict function? I can not find usage in your docs.
I use it like this:
model = pf.ARIMAX(data=group,formula='passengerCount~1',ar=1,ma=1) x = model.fit() pre = model.predict(18) print(pre)

got error:
Error evaluating factor: TypeError: 'NoneType' object is not subscriptable
passengerCount~1

• Documentation : Introduction page
• Documentation : Probabilistic Thinking page
• Documentation : Inference page
• Documentation : Analysis and prediction page
• Documentation : New models and updated API
• ARIMA models - final checks / write tests / intervals / sample from posterior
• GAS models - fix prediction intervals / write tests / sample from posterior
• GARCH models - fix prediction intervals / write tests / sample from posterior
• Non-Gaussian State Space - fix prediction intervals / write tests / sample from posterior
• Gaussian State Space - fix prediction intervals / write tests / sample from posterior
• GP-NARX - fix prediction intervals / write tests / sample from posterior
• Neural Network AR - fix prediction intervals / write tests / sample from posterior
• VAR - fix prediction intervals / write tests / sample from posterior
• Metropolis - thinning option
• BBVI - clean up class, learning rate optimization

Hi,
First of all, I would like to congratulate/thank you for the amazing job and the great effort devoted to compile the tools in pyflux.
However, I have not had the chance to test it out since I am getting a compile-time error when installing it on Windows:

 gas_recursions.obj : warning LNK4197: export 'PyInit_gas_recursions' specified multiple times; using first specification
     Creating library build\temp.win-amd64-3.5\Release\pyflux\gas\gas_recursions.cp35-win_amd64.lib and object build\temp.win-amd64-3.5\Release\pyflux\gas\gas_recursions.cp35-win_amd64.exp
  Generating code
  c:\users\user\appdata\local\temp\pip-build-m71zj04b\pyflux\pyflux\gas\gas_recursions.c(2167) : fatal error C1001: An internal error has occurred in the compiler.
  (compiler file 'f:\dd\vctools\compiler\utc\src\p2\main.c', line 255)

Any clue? I will try to have a look at it when I have some spare time but I thought this error might be familiar to someone else out there. For those interested, the complete output is here.

Need to make a stochastic_model() method for each model type which allows for a subset of the data to be used. This can then be used for a mini-batch BBVI option, as well as open up the possibility of stochastic optimization options for larger datasets.

LLT and DynReg intervals do not work (undefined over some intervals of time) - needs fixing

Related to #10. Tests should be done on the GitHub side rather than locally. Steps:

• Set Up Travis (Issue #10 )
• Upload tests to a /tests folder within each model and inference folder

in http://www.pyflux.com/metropolis-hastings/

import numpy as np
import pyflux as pf
try:
    from pandas.io.data import DataReader
except:
    from pandas_datareader.data import DataReader
from datetime import datetime
import matplotlib.pyplot as plt
import matplotlib.mlab as mlab
%matplotlib inline 

fb = DataReader('FB',  'yahoo', datetime(2013,1,1), datetime(2016,6,10))
fb['Logged Adj Close'] = np.log(fb['Adj Close'].values)
plt.figure(figsize=(15,5))
plt.plot(fb.index[1:len(fb.index)],np.diff(fb['Adj Close'].values))
plt.ylabel('Returns')
plt.title('IBM Returns')
plt.show()

Problem:

• Silly using default scipy L-BFGS for optimization since it uses numerical gradients and line-search which require multiple inversions of the covariance matrix. Need to replace with analytical gradients.

More generally:

• Need to profile the entire code and find out where the bottlenecks are, and fix them.

Other potential options:

• Multiple starting points for GP-NARX -> pick the one which ends up in the best region.

Some models - such as GAS models - have likelihoods that are very sensitive to poor parameterization. This results in nan likelihoods, and thereby nan gradients for BBVI. Possible solutions include gradient clipping for GAS models, and a warm-up stage for BBVI that is specific to some model types.

Currently stored as transformed. Fixed -> will be in next release.

I'm trying to use PyFlux on a Windows 10 64-bit machine with Python 3.4 (Anaconda with NumPy 1.11.2 and SciPy 0.18.1). I installed PyFlux 0.4.0 using Christoph Gohlke's precompiled wheel. However when I run the ARIMA example from the documentation, when I fit the model, I get the following result:

C:\Anaconda3\lib\site-packages\scipy\stats\_continuous_distns.py:84: RuntimeWarning: overflow encountered in square
  return -x**2 / 2.0 - _norm_pdf_logC
ARIMA(4,0,4)                                                                                              
======================================================= ==================================================
Dependent Variable: sunspot.year                        Method: MLE                                       
Start Date: 1704                                        Log Likelihood: -1180.5324                        
End Date: 1988                                          AIC: 2381.0648                                    
Number of observations: 285                             BIC: 2417.5897                                    
==========================================================================================================
Latent Variable                          Estimate   Std Error  z        P>|z|    95% C.I.                 
======================================== ========== ========== ======== ======== =========================
Constant                                 2.6396     1.0944     2.412    0.0159   (0.4947 | 4.7846)        
AR(1)                                    1.8146     0.1035     17.5364  0.0      (1.6118 | 2.0174)        
AR(2)                                    -0.606     0.2661     -2.2776  0.0227   (-1.1275 | -0.0845)      
AR(3)                                    -0.8672    0.2707     -3.2031  0.0014   (-1.3979 | -0.3366)      
AR(4)                                    0.6084     0.1074     5.6638   0.0      (0.3978 | 0.8189)        
MA(1)                                    -0.6369    0.0958     -6.648   0.0      (-0.8246 | -0.4491)      
MA(2)                                    -0.5629    0.1497     -3.7595  0.0002   (-0.8563 | -0.2694)      
MA(3)                                    0.2923     0.1207     2.421    0.0155   (0.0557 | 0.5289)        
MA(4)                                    0.2367     0.0684     3.4602   0.0005   (0.1026 | 0.3707)        
Sigma                                    15.2314                                                          
==========================================================================================================

Do you have any thoughts on what could be going wrong and how it could be fixed?

As an alternative to diffuse initialization? Ideally would like to keep diffuse initialization though if doing MLE or MAP, so need to find a way to support both options.

Example is no longer working - ELBO gets stuck? Need to fix.

I want to fit a GARCH model on the first 70% of my time series and then test it for predicting the remained 30%. At each step I want to predict five steps ahead. Is it possible to shift and replace the model.data with some part of test data and then use this model for predicting future? My question is: is there any parameter that should be changed when I change and shift the model's training data to test the fitted model for the test data?

If the model is Bayesian, the intervals should account for the uncertainty in the parameters. Bootstrapping is an option for the frequentist approach.

Hi,
I am using pyflux in a project of mine on which I have configure readthedocs to automatically build the documentation. However, the build robot of readthedocs fails when it tries to install PyFlux. I don't know if this is a bug of readthedocs or of PyFlux, but I though you would like to know.

Goal is for API for non-Gaussian state space models to be the same as the GAS Models - and for a common API for distributions that carries across model types to prevent code duplication.

This needs for continuous integration etc - a travis.yml file. Should be a bit fiddly to set up but useful.

model = pf.ARIMA(data=xt.values[1:],ar=0,ma=1,integ=0)
m = model.fit()

create an error message, while if ar=any positive integer, it is learning without any problem

ValueError Traceback (most recent call last)
in ()
1 model = pf.ARIMA(data=xt.values[1:],ar=0,ma=1,integ=0)
----> 2 m = model.fit()
3 m.summary()

/home/vagrant/.virtualenvs/venvpy3/lib/python3.4/site-packages/pyflux/tsm.py in fit(self, method, **kwargs)
540
541 if method == 'MLE':
--> 542 return self._optimize_fit(self.neg_loglik,**kwargs)
543 elif method == 'PML':
544 return self._optimize_fit(self.neg_logposterior,**kwargs)

/home/vagrant/.virtualenvs/venvpy3/lib/python3.4/site-packages/pyflux/tsm.py in _optimize_fit(self, obj_type, **kwargs)
431
432 # Optimize using L-BFGS-B
--> 433 p = optimize.minimize(obj_type,phi,method='L-BFGS-B')
434
435 # Model check

/home/vagrant/.virtualenvs/venvpy3/lib/python3.4/site-packages/scipy/optimize/_minimize.py in minimize(fun, x0, args, method, jac, hess, hessp, bounds, constraints, tol, callback, options)
445 elif meth == 'l-bfgs-b':
446 return _minimize_lbfgsb(fun, x0, args, jac, bounds,
--> 447 callback=callback, **options)
448 elif meth == 'tnc':
449 return _minimize_tnc(fun, x0, args, jac, bounds, callback=callback,

/home/vagrant/.virtualenvs/venvpy3/lib/python3.4/site-packages/scipy/optimize/lbfgsb.py in _minimize_lbfgsb(fun, x0, args, jac, bounds, disp, maxcor, ftol, gtol, eps, maxfun, maxiter, iprint, callback, maxls, **unknown_options)
328 # minimization routine wants f and g at the current x
329 # Overwrite f and g:
--> 330 f, g = func_and_grad(x)
331 elif task_str.startswith(b'NEW_X'):
332 # new iteration

/home/vagrant/.virtualenvs/venvpy3/lib/python3.4/site-packages/scipy/optimize/lbfgsb.py in func_and_grad(x)
271 if jac is None:
272 def func_and_grad(x):
--> 273 f = fun(x, *args)
274 g = _approx_fprime_helper(x, fun, epsilon, args=args, f0=f)
275 return f, g

/home/vagrant/.virtualenvs/venvpy3/lib/python3.4/site-packages/scipy/optimize/optimize.py in function_wrapper(_wrapper_args)
287 def function_wrapper(_wrapper_args):
288 ncalls[0] += 1
--> 289 return function(*(wrapper_args + args))
290
291 return ncalls, function_wrapper

/home/vagrant/.virtualenvs/venvpy3/lib/python3.4/site-packages/pyflux/arma/arma.py in neg_loglik(self, beta)
282 """
283
--> 284 mu, Y = self._model(beta)
285 return -np.sum(ss.norm.logpdf(Y,loc=mu,scale=self.parameters.parameter_list[-1].prior.transform(beta[-1])))
286

/home/vagrant/.virtualenvs/venvpy3/lib/python3.4/site-packages/pyflux/arma/arma.py in _model(self, beta)
130
131 # Constant and AR terms
--> 132 mu = np.matmul(np.transpose(self.X),parm[0:-1-self.ma])
133
134 # MA terms

ValueError: shapes (221,) and (1,) not aligned: 221 (dim 0) != 1 (dim 0)

Hi have you any examples with cross-validation type methods here?

See Dustin Tran's recent blog entry http://dustintran.com/blog/on-bayesian-and-frequentist-latent-variables-and-parameters/ for the justification.

In the package, we refer to parameters in method names, as well as the Parameters class that is attached to each model type. These should be renamed to latent variables. Small task, but I think phrasing things in the best way possible is important.

Need to random search this in preoptimization stage. Noticed that a DAR model didn't optimize well today because of inappropriate starting values.

Would be nice to animate the learning process for dynamic regression coefficients.

ARIMAX models in particular need better initialization.