Issue loading data from FRED about cvxportfolio HOT 9 CLOSED

Hey Howard! Thanks, it looks like it's an SSL error on the FRED side. This is the error in your traceback:

File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/urllib/request.py", line 1347, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1000)>

Or this

File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/ssl.py", line 1320, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1000)

That's the first time I see it, but I did see an error with FRED on one of the GH test runners yesterday, it worked fine after re-running. FYI the endpoint is something like this https://fred.stlouisfed.org/graph/fredgraph.csv?id=DFF, it's an undocumented endpoint by FRED but it works fine. Maybe check if you're not using some proxy or vpn with cold SSL caches? Everything works on my side, and in the GH test runners.

from cvxportfolio.

FYI @hhenson, the example you're running only uses FRED to get the USD risk free (central bank) rate, Yahoo Finance is working fine for you it seems. You can get around that by disabling that part of the the market data download. Substitute this line

simulator = cvx.StockMarketSimulator(UNIVERSE)

with this

md = cvx.DownloadedMarketData(UNIVERSE, cash_key='cash')
simulator = cvx.StockMarketSimulator(market_data = md)

that sets the risk free rate to zero...

from cvxportfolio.

Also, this thread https://stackoverflow.com/questions/52805115/certificate-verify-failed-unable-to-get-local-issuer-certificate seems to suggest you have to run some post-installation script to enable SSL for your local Python, it looks like you're using one in the system libraries. I normally suggest using brew for installing Python on mac...

from cvxportfolio.

OK, was using a pyenv setup, changed to use the brew system installation and got it working!

from cvxportfolio.

FYI
I have gotten the performance test to run, these are the results:

FIRST RUN
BACK-TEST TOOK: 294.0837182998657
SIMULATOR + POLICY TIMES: 294.03591322898865
AVERAGE TIME PER ITERATION: 0.23373284040460146
RESULT:

#################################################################
Universe size 501
Initial timestamp 2019-05-13 13:30:00+00:00
Final timestamp 2024-05-10 13:30:00+00:00
Number of periods 1259
Initial value (USDOLLAR) 1.000e+06
Final value (USDOLLAR) 4.394e+06
Profit (USDOLLAR) 3.394e+06

Avg. return (annualized) 60.8%
Volatility (annualized) 78.0%
Avg. excess return (annualized) 58.8%
Avg. active return (annualized) 58.8%
Excess volatility (annualized) 78.0%
Active volatility (annualized) 78.0%

Avg. growth rate (annualized) 29.7%
Avg. excess growth rate (annualized) 27.6%
Avg. active growth rate (annualized) 27.6%

Avg. StocksTransactionCost 0bp
Max. StocksTransactionCost 9bp
Avg. StocksHoldingCost 1bp
Max. StocksHoldingCost 5bp

Sharpe ratio 0.75
Information ratio 0.75

Avg. drawdown -48.1%
Min. drawdown -88.3%
Avg. leverage 299.9%
Max. leverage 399.6%
Avg. turnover 3.9%
Max. turnover 150.0%

Avg. policy time 0.152s
Avg. simulator time 0.081s
Of which: market data 0.013s
Of which: result 0.054s
Total time 294.036s
#################################################################

SECOND RUN
BACK-TEST TOOK: 216.16064381599426
SIMULATOR + POLICY TIMES: 216.1089153289795
AVERAGE TIME PER ITERATION: 0.17178769104052424
RESULT:

#################################################################
Universe size 501
Initial timestamp 2019-05-13 13:30:00+00:00
Final timestamp 2024-05-10 13:30:00+00:00
Number of periods 1259
Initial value (USDOLLAR) 1.000e+06
Final value (USDOLLAR) 4.393e+06
Profit (USDOLLAR) 3.393e+06

Avg. return (annualized) 60.8%
Volatility (annualized) 78.0%
Avg. excess return (annualized) 58.8%
Avg. active return (annualized) 58.8%
Excess volatility (annualized) 78.0%
Active volatility (annualized) 78.0%

Avg. growth rate (annualized) 29.6%
Avg. excess growth rate (annualized) 27.6%
Avg. active growth rate (annualized) 27.6%

Avg. StocksTransactionCost 0bp
Max. StocksTransactionCost 9bp
Avg. StocksHoldingCost 1bp
Max. StocksHoldingCost 5bp

Sharpe ratio 0.75
Information ratio 0.75

Avg. drawdown -48.1%
Min. drawdown -88.3%
Avg. leverage 299.9%
Max. leverage 399.6%
Avg. turnover 3.9%
Max. turnover 150.0%

Avg. policy time 0.107s
Avg. simulator time 0.065s
Of which: market data 0.010s
Of which: result 0.044s
Total time 216.109s
#################################################################

This is on an M3 MacBook Pro with the Max chipset using the brew version of Python 3.11.9

from cvxportfolio.

FIRST RUN
BACK-TEST TOOK: 362.78375482559204
SIMULATOR + POLICY TIMES: 362.7362184524536
AVERAGE TIME PER ITERATION: 0.28834357587635423
RESULT:

#################################################################
Universe size 501
Initial timestamp 2019-05-13 13:30:00+00:00
Final timestamp 2024-05-10 13:30:00+00:00
Number of periods 1259
Initial value (USDOLLAR) 1.000e+06
Final value (USDOLLAR) 4.394e+06
Profit (USDOLLAR) 3.394e+06

Avg. return (annualized) 60.8%
Volatility (annualized) 78.0%
Avg. excess return (annualized) 58.8%
Avg. active return (annualized) 58.8%
Excess volatility (annualized) 78.0%
Active volatility (annualized) 78.0%

Avg. growth rate (annualized) 29.7%
Avg. excess growth rate (annualized) 27.6%
Avg. active growth rate (annualized) 27.6%

Avg. StocksTransactionCost 0bp
Max. StocksTransactionCost 9bp
Avg. StocksHoldingCost 1bp
Max. StocksHoldingCost 5bp

Sharpe ratio 0.75
Information ratio 0.75

Avg. drawdown -48.1%
Min. drawdown -88.3%
Avg. leverage 299.9%
Max. leverage 399.6%
Avg. turnover 3.9%
Max. turnover 150.0%

Avg. policy time 0.198s
Avg. simulator time 0.090s
Of which: market data 0.017s
Of which: result 0.053s
Total time 362.736s
#################################################################

SECOND RUN
BACK-TEST TOOK: 248.23834991455078
SIMULATOR + POLICY TIMES: 248.17573499679565
AVERAGE TIME PER ITERATION: 0.19727800874149098
RESULT:

#################################################################
Universe size 501
Initial timestamp 2019-05-13 13:30:00+00:00
Final timestamp 2024-05-10 13:30:00+00:00
Number of periods 1259
Initial value (USDOLLAR) 1.000e+06
Final value (USDOLLAR) 4.394e+06
Profit (USDOLLAR) 3.394e+06

Avg. return (annualized) 60.8%
Volatility (annualized) 78.0%
Avg. excess return (annualized) 58.8%
Avg. active return (annualized) 58.8%
Excess volatility (annualized) 78.0%
Active volatility (annualized) 78.0%

Avg. growth rate (annualized) 29.7%
Avg. excess growth rate (annualized) 27.6%
Avg. active growth rate (annualized) 27.6%

Avg. StocksTransactionCost 0bp
Max. StocksTransactionCost 9bp
Avg. StocksHoldingCost 1bp
Max. StocksHoldingCost 5bp

Sharpe ratio 0.75
Information ratio 0.75

Avg. drawdown -48.1%
Min. drawdown -88.3%
Avg. leverage 299.9%
Max. leverage 399.6%
Avg. turnover 3.9%
Max. turnover 150.0%

Avg. policy time 0.125s
Avg. simulator time 0.072s
Of which: market data 0.013s
Of which: result 0.044s
Total time 248.176s
#################################################################

This is on the M2 ULTRA Studio

from cvxportfolio.

No Issue

from cvxportfolio.

Very nice, second runs are the most interesting ones. That example uses an open-source solver (ECOS), which is very similar to MOSEK. Some (~10% I guess) improvement can come from optimizing the "result time", which could be done better (but it's very tricky to make sure the accounting is correct). Otherwise it is IMO about as fast as it can be done.

*edit: ECOS, not CLARABEL

from cvxportfolio.

@hhenson I just re-ran it myself, changing the solver to the new open-source one that is default in CVXPY 1.5; interesting to note that it gives slightly lower Sharpe, but it is a bit faster.

from cvxportfolio.