Hi,

Please include a license to this project. I would like to contribute but can't now because there is no license.

Thanks

Xero

Not issues per se, except possibly in terms of complementing the documentation:

Is there also company/ticker status information? By not adding start, end and interval, like before?

Does the crumb expire, or can it be saved for use over time?

I will of course experiment with this.

Hello,

The current implementation of the load_yahoo_quote function cannot use dates prior to 1970/01/01 due to the limited range of time.mktime. Would it be possible/relevant to modify the load_yahoo_quote function so that it accepts earlier dates?

On paper I think calendar.timegm would do the trick - it outputs prior dates as negative numbers, which is also the format accepted for parameter 'period1' in the Yahoo URL for dates before 1970/01/01. However I've tried to modify the function accordingly (note the date format change):

tb = calendar.timegm(time.strptime(begindate+":4","%Y-%m-%d:%H"))
te = calendar.timegm(time.strptime(enddate+":18","%Y-%m-%d:%H"))

and get an 'invalid literal for int() with base 10' ValueError in exchange, which seems weird not only because both lines output correctly when run in isolation outside of the function, but because the error is marked at line 84, which is the '# Prepare the parameters and the URL' comment!

Hi, thank you for your project that helps me a lot. But it seems has problem when i want the result output as 'dataframe'.
Here is the error log:

Traceback (most recent call last):  
  File "test_yqd.py", line 25, in <module>  
    test()
  File "test_yqd.py", line 18, in test
    load_quote('QCOM')
  File "test_yqd.py", line 12, in load_quote
    print(yqd.load_yahoo_quote(ticker, '20170515', '20170517',format_output = 'dataframe'))
  File "~/code/python/venv/lib/python2.7/site-packages/yahoo_quote_download-0.1-py2.7.egg/yahoo_quote_download/yqd.py", line 117, in load_yahoo_quote
    adf = pd.DataFrame.from_records(nested_alines, columns=cols)
  File "/usr/local/lib/python2.7/site-packages/pandas/core/frame.py", line 1269, in from_records
    coerce_float=coerce_float)
  File "/usr/local/lib/python2.7/site-packages/pandas/core/frame.py", line 7461, in _to_arrays
    dtype=dtype)
  File "/usr/local/lib/python2.7/site-packages/pandas/core/frame.py", line 7540, in _list_to_arrays
    coerce_float=coerce_float)
  File "/usr/local/lib/python2.7/site-packages/pandas/core/frame.py", line 7598, in _convert_object_array
    con=len(content)))
AssertionError: 1 columns passed, passed data had 2 columns

It seems the code return dataframe may need to change to :

    if format_output == 'dataframe':
		#nested_alines = [line.split(',') for line in alines[1:]]
		nested_alines = [line.split(',') for line in alines.split('\n')[1:]]
		cols = alines.split('\n')[0].split(',')
		# print(cols)
		adf = pd.DataFrame.from_records(nested_alines[:-1], columns=cols)
		return adf

I'm using this tool since a couple of years and it's excellent; anyway it's a couple of days that I am experiencing this problem (unauthorized error). I tried it from different places and with different ip addresses but still not solving the problem.
Might it be due to changes on Yahoo's policies? thanks

Although yahoo_quote_download has been downloading for a while, the latest test_yqd.py is now generating:

HTTPError: HTTP Error 401: Unauthorized

Any suggestions?

Occasionally, the urlopen raise HTTPError with 401 Unauthorized response. I don't know what causes it (maybe too much request in a short time?)

At this time, I tried to catch those exceptions in my calling script and rerun the download after some timed sleep. But this is only a patch solution.

Any suggestions on what causes it?

I don't have enough reputation points to leave a comment on stackoverflow, hence this note.
First, thanks for addressing the problem.
Second, I'm using Python 2.7 and replaced
import urllib.request, urllib.parse, urllib.error
with
from six.moves import urllib

when I run your test I get and error
yqd.py", line 39, in _get_cookie_crumb
alines = f.read().decode()
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 59021: ordinal not in range(128)

Suggestions appreciated.

Could you kindly give an example on how to call the function. Failing to use it.

Thanks for the nice utility.

It would be nice to have the customary --version flag, to just show version number.

In addition, I opened a small PR for minor README fixes.

Written before I put thought into it. Please Close.

Running this:

yqdownload -t ^GSPC -f gspc.csv

generates this error

!!! ^GSPC: Invalid date 2021-07-23 in data

but the date seems perfectly valid

I'm trying to implement your workaround to obtain data for a web. I could get the crumb and overcome the cross domain limitations through a third party proxy. However I couldn't retrieve the data. Could you provide insights into how to do it or expand your solution for a javascript api?

This is what I've done so far:

// example.
var url="https://finance.yahoo.com/quote/%5EGSPC/history?p=%5EGSPC";

var cors_api_url = 'https://cors-anywhere.herokuapp.com/';

var requestParameters = {
method: "GET", //either POST or GET
url: url
};

doCORSRequest(requestParameters, function printResult(result) {
console.log(result);

var cs = result.indexOf('CrumbStore');
var cr = result.indexOf('crumb', cs + 10);
var cl = result.indexOf(':', cr + 5);
var q1 = result.indexOf('"', cl + 1);
var q2 = result.indexOf('"', q1 + 1);
var crumb = result.slice(q1 + 1,q2);

console.log("cs",cs);
console.log("cr",cr);
console.log("cl",cl);
console.log("q1",q1);
console.log("q2",q2);
console.log("crumb",crumb);

var url3 ="https://finance.yahoo.com/quote/XLF/history?period1=914299200&period2=1496376000&interval=1d&filter=history&frequency=1d"

var url2 = "https://query1.finance.yahoo.com/v7/finance/download/%5EGSPC?period1=1493730135&period2=1496408535&interval=1d&events=history&crumb="+crumb;

console.log(encodeURIComponent(url2));

var urlGoogleFinance = "https://www.google.com/finance/historical?q=XLY&startdate=2001-12-31&output=csv";
var urlGoogleFinance2 = "https://www.google.com/finance/historical?q=GE&startdate=1990-06-30&output=csv";

var baseUri = "https://query.yahooapis.com/v1/public/yql?q=";
var uriQuery = encodeURIComponent("SELECT * from json where url='"+url3+"'"/+" and columns='Date,Open,High,Low,Close,Volume' "/);
var totalUri = baseUri + uriQuery+"&format=json";

console.log("totalUri", totalUri);

Plotly.d3.json(totalUri, function(readData) {
//console.log("readData",readData);
console.log("readData",readData);
console.log("readData",readData.query.results.row[0]);
console.log("readData",readData.query.results);
});

});

// main function
function doCORSRequest(options, printResult) {
var x = new XMLHttpRequest();
console.log(" cors_api_url + options.url", cors_api_url + options.url);
x.open(options.method, cors_api_url + options.url);
x.onload = x.onerror = function() {
printResult(
options.method + ' ' + options.url + '\n' +
x.status + ' ' + x.statusText + '\n\n' +
(x.responseText || '')
);
};
if (/^POST/i.test(options.method)) {
x.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded');
}
x.send(options.data);
}