Issue When Running examples/sp500_ndx100.py about cvxportfolio HOT 5 CLOSED

Hello! Can you please post the output of

import cvxportfolio as cvx

md = cvx.MarketSimulator(['NVDA']).market_data

md.returns.loc[md.returns['NVDA'].isnull()]

on my machine there are no missing values for NVDA in 2001, there might have been a yfinance download issue on your end?

from cvxportfolio.

Thanks, I didn't get any warnings. This is the output:

Updating data
.
NVDA USDOLLAR
Date
2001-09-10 NaN 0.000137
2001-09-12 NaN 0.000139
2023-09-21 NaN NaN

from cvxportfolio.

from cvxportfolio.

I'm working on a fix, the Yahoo finance interface is not up to standards. I have done most testing on recent data (say, after 2010) and large cap US stocks. On those there are very rarely (I haven't seen any) missing rows in the data. I have seen bugs like this on low cap or foreign stocks, but this is the first time it is reported on a large cap US stock in recent history.

In the 2016 examples we were heavily filtering (any stock with missing data was excluded), at the time with quandl data (now defunct). The goal is to have an interface modular enough to plug a new data source with minimal code (alpha vantage, bloomberg, ...). Yahoo finance however is the only one that works without subscription (to my knowledge) and has acceptable quality, so I want to keep it as default (also for the examples). By the way, you can always use Cvxportfolio with user-provided returns and volumes (as it was done in the original interface), and filter in any way you wish.

What's your suggestion for dealing with a missing row like the one you had? All default estimators (expected return, covariances, transaction cost model parameters, ...) are robust to missing data, they are OK. The simulator however needs to make an assumption there. Do we propagate the last closing price (so on the missing days the returns are zero, but on the day before it might be larger in absolute value)? Do we just refuse to use that stock? (Which is basically what happens now.) Another possible fix from the user side is to do resampling (e.g., monthly) with the trading_frequency option to the simulator, which makes it robust to (some) missing data rows.

Thanks!

from cvxportfolio.

Thank you. It looks like a data download issue, it probably originated in a dependency package. We definitely need stricter checks on the downloaded data quality. In the meantime you can delete your cache folder $HOME/cvxportfolio_data, or just the NVDA file, and re-run the script.

I see, thanks.

from cvxportfolio.