jugaad-py / jugaad-data Goto Github PK
View Code? Open in Web Editor NEWDownload live and historical data for Indian stock market
Home Page: https://marketsetup.in/documentation/jugaad-data/
Download live and historical data for Indian stock market
Home Page: https://marketsetup.in/documentation/jugaad-data/
Combined bhavcopy download for NSE and BSE for Cash segment
https://www.bseindia.com/download/BhavCopy/Equity/EQ040322_CSV.ZIP
https://www.bseindia.com/download/BhavCopy/Equity/EQ040322_CSV.ZIP
New Base URL to be updates is : https://nsearchives.nseindia.com/
jugaad-data/jugaad_data/nse/archives.py
Line 24 in c6a6427
class NSEArchives:
base_url = "https://archives.nseindia.com"
"""Conventions
NSELive().index_option_chain("NIFTY")
returns empty result set {}. Suddenly it stopped working and returns empty result set and doesn't return expiry date.
Running the bhavcopy_fo_save function always lead to timeout.
from jugaad_data.nse import bhavcopy_save, bhavcopy_fo_save
bhavcopy_fo_save(date(2022,12,3), "D:\\Speculation\\data\\NSE_data")
File "D:\anaconda3\lib\site-packages\requests\models.py", line 760, in generate
raise ConnectionError(e)
ConnectionError: HTTPSConnectionPool(host='www.nseindia.com', port=443): Read timed out.
The history url - https://www.nseindia.com/api/historical/cm/equity?symbol=RELIANCE&series=[%22EQ%22]&from=01-01-2019&to=31-01-2019 returns "Resource not found" if proper cookies not provided in the headers
How to replicate the issue without code-
Open this url (https://www.nseindia.com/api/historical/cm/equity?symbol=RELIANCE&series=[%22EQ%22]&from=01-01-2019&to=31-01-2019) in Incognito mode, you should see Resource not found error
How to resolve-
Now in the same incognito window go to (https://www.nseindia.com/get-quotes/equity?symbol=RELIANCE) and then go back to you should see the data (https://www.nseindia.com/api/historical/cm/equity?symbol=RELIANCE&series=[%22EQ%22]&from=01-01-2019&to=31-01-2019)
Traceback (most recent call last):
File "/home/mb/my/career/market/bhav/setup.py", line 8, in <module>
bhavcopy_fo_save(date(2021,7,20), ".")
File "/home/mb/my/career/market/bhav/venv/lib/python3.9/site-packages/jugaad_data/nse/archives.py", line 120, in bhavcopy_fo_save
text = self.bhavcopy_fo_raw(dt)
File "/home/mb/my/career/market/bhav/venv/lib/python3.9/site-packages/jugaad_data/nse/archives.py", line 13, in unzipper
with zipfile.ZipFile(file=fp) as zf:
File "/usr/lib/python3.9/zipfile.py", line 1257, in __init__
self._RealGetContents()
File "/usr/lib/python3.9/zipfile.py", line 1324, in _RealGetContents
raise BadZipFile("File is not a zip file")
zipfile.BadZipFile: File is not a zip file
Hi @sevakram , It looks like NSE India website API has issues and has very frequent downtime. It returns empty result set in the URI in which you are hitting to fetch the option chain. In the past few weeks, it has been a frustrating issue as the automation is failing by unable to fetch expiry date. Could you fetch the expiry dates from any other reliance source API please?
Thanks for your effort.
n = NSELive()
option_chain = n.index_option_chain("NIFTY")
expiry = option_chain['records']['expiryDates'][00]
or
option_chain = n.stock_quote_fno("NIFTY")
expiry = option_chain['expiryDates'][0]
Error in fetching expiry date
Hello Team,
This below code fetches data for ABBOTINDIA till 29th June 2021
df = stock_df(symbol = 'ABBOTINDIA', from_date=date(2021, 5, 1), to_date=date.today(),series="EQ")
But the same code fetches data till 28th June for 3MINDIA
df = stock_df(symbol = 'ABBOTINDIA', from_date=date(2021, 5, 1), to_date=date.today(),series="EQ")
While trying to extract data for ADANIGREEN, it gives OHLC data till June 08th, 2021
Hi, you might consider adding in a proxy variable when instantiating the NSELive or NSEHistory class to download data, the reason being if an individual is making a lot of requests to the NSE website for historical data or live prices - proxies come in handy so that the user does not get blocked.
I had my own version of the code to download all the data but your library is no doubt much more better.
Let me know if you are looking for more feedback!
https://2.python-requests.org/en/latest/user/advanced/#proxies
proposed functions-
def expiry_dates(year, month, day, instrument_type, symbol):
""" Returns all expiry dates being actively traded on a given date for a given instrument type eg. FUTIDX, OPTIDX and symbol in a given month"""
----
return list_list_of_dates
Daily Download of Full bhavcopy and FO File is working great.
Can you please add feature for downloading participant wise open interest file?
https://archives.nseindia.com/content/nsccl/fao_participant_oi_04052022.csv
https://archives.nseindia.com/content/nsccl/fao_participant_oi_04052022.csv
No response
Some issue happened
import os
os.listdir()
Exception: Could not find directory
I am trying to download the historical NIFTY50 index data. I am getting an error given below.
However, the individual stocks data download for the same period are working fine.
Please find the example code as well as the error snippet below.
from jugaad_data.nse import index_csv
# Download as csv
index_csv(symbol="NIFTY 50", from_date=date(2020,1,1),
to_date=date(2020,1,30), output=r"C:\Users\Kart\Documents\Workspace\pypen\index_data.csv")
Traceback (most recent call last):
File "C:\Users\Kart\Documents\Workspace\pypen\lib\site-packages\requests\models.py", line 971, in json
return complexjson.loads(self.text, **kwargs)
File "C:\Users\Kart\AppData\Local\Programs\Python\Python39\lib\json\__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "C:\Users\Kart\AppData\Local\Programs\Python\Python39\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\Kart\AppData\Local\Programs\Python\Python39\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Kart\Documents\Workspace\pypen\finance\bhavcopy.py", line 26, in <module>
index_csv(symbol="NIFTY 50", from_date=date(2020,1,1),
File "C:\Users\Kart\Documents\Workspace\pypen\lib\site-packages\jugaad_data\nse\history.py", line 336, in index_csv
raw = index_raw(symbol, from_date, to_date)
File "C:\Users\Kart\Documents\Workspace\pypen\lib\site-packages\jugaad_data\nse\history.py", line 318, in index_raw
return list(itertools.chain.from_iterable(chunks))
File "C:\Users\Kart\AppData\Local\Programs\Python\Python39\lib\concurrent\futures\_base.py", line 600, in result_iterator
yield fs.pop().result()
File "C:\Users\Kart\AppData\Local\Programs\Python\Python39\lib\concurrent\futures\_base.py", line 433, in result
return self.__get_result()
File "C:\Users\Kart\AppData\Local\Programs\Python\Python39\lib\concurrent\futures\_base.py", line 389, in __get_result
raise self._exception
File "C:\Users\Kart\AppData\Local\Programs\Python\Python39\lib\concurrent\futures\thread.py", line 52, in run
result = self.fn(*self.args, **self.kwargs)
File "C:\Users\Kart\Documents\Workspace\pypen\lib\site-packages\jugaad_data\util.py", line 109, in wrapper
j = function(**kw)
File "C:\Users\Kart\Documents\Workspace\pypen\lib\site-packages\jugaad_data\nse\history.py", line 312, in _index
return json.loads(self.r.json()['d'])
File "C:\Users\Kart\Documents\Workspace\pypen\lib\site-packages\requests\models.py", line 975, in json
raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Can we download multiple stock historical data at once and update it daily? For example,can we download all FnO stock data (mainly OHLC) and download in Excel for last one one year or so? If yes, how??
from datetime import date
from jugaad_data.nse import bhavcopy_save, bhavcopy_fo_save
# Download bhavcopy
bhavcopy_save(date(2020,1,1), "/path/to/directory")
# Download bhavcopy for futures and options
bhavcopy_fo_save(date(2020,1,1), "/path/to/directory")
# Download stock data to pandas dataframe
from jugaad_data.nse import stock_df
df = stock_df(symbol="SBIN", from_date=date(2020,1,1),
to_date=date(2020,1,30), series="EQ")
UPDATE: It's not working for official names from NSE either
You can reproduce the code as below:
current_date = date.today()
stock_df(symbol='INFY', from_date = current_date - timedelta(days = 365), to_date = current_date, series="EQ")
When you get the list from this website or by another as
# ! pip install nsetools
from nsetools import Nse
nse = Nse()
all_stocks = nse.get_stock_codes()
stocks = list(all_stocks.keys())[1:]
it won't give you for all. just a few, also sometimes it fails for HDFC too.
The exact error is
JSONDecodeError Traceback (most recent call last)
/tmp/ipykernel_2736/1058062146.py in <module>
1 current_date = date.today()
----> 2 stock_df(symbol='INFY', from_date = current_date - timedelta(days = 365), to_date = current_date, series="EQ").drop(drop,axis=1)
3
~/anaconda3/envs/finance/lib/python3.7/site-packages/jugaad_data/nse/history.py in stock_df(symbol, from_date, to_date, series)
168 if not pd:
169 raise ModuleNotFoundError("Please install pandas using \n pip install pandas")
--> 170 raw = stock_raw(symbol, from_date, to_date, series)
171 df = pd.DataFrame(raw)[stock_select_headers]
172 df.columns = stock_final_headers
~/anaconda3/envs/finance/lib/python3.7/site-packages/jugaad_data/nse/history.py in stock_raw(self, symbol, from_date, to_date, series)
106 chunks = ut.pool(self._stock, params)
107
--> 108 return list(itertools.chain.from_iterable(chunks))
109
110 def derivatives_raw(self, symbol, from_date, to_date, expiry_date, instrument_type, strike_price, option_type):
~/anaconda3/envs/finance/lib/python3.7/concurrent/futures/_base.py in result_iterator()
596 # Careful not to keep a reference to the popped future
597 if timeout is None:
--> 598 yield fs.pop().result()
599 else:
600 yield fs.pop().result(end_time - time.monotonic())
~/anaconda3/envs/finance/lib/python3.7/concurrent/futures/_base.py in result(self, timeout)
426 raise CancelledError()
427 elif self._state == FINISHED:
--> 428 return self.__get_result()
429
430 self._condition.wait(timeout)
~/anaconda3/envs/finance/lib/python3.7/concurrent/futures/_base.py in __get_result(self)
382 def __get_result(self):
383 if self._exception:
--> 384 raise self._exception
385 else:
386 return self._result
~/anaconda3/envs/finance/lib/python3.7/concurrent/futures/thread.py in run(self)
55
56 try:
---> 57 result = self.fn(*self.args, **self.kwargs)
58 except BaseException as exc:
59 self.future.set_exception(exc)
~/anaconda3/envs/finance/lib/python3.7/site-packages/jugaad_data/util.py in wrapper(*args, **kw)
107 if not os.path.exists(cache_dir):
108 os.makedirs(cache_dir)
--> 109 j = function(**kw)
110 with open(path, 'wb') as fp:
111 pickle.dump(j, fp)
~/anaconda3/envs/finance/lib/python3.7/site-packages/jugaad_data/nse/history.py in _stock(self, symbol, from_date, to_date, series)
73 }
74 self.r = self._get("stock_history", params)
---> 75 j = self.r.json()
76 return j['data']
77
~/anaconda3/envs/finance/lib/python3.7/site-packages/requests/models.py in json(self, **kwargs)
908 # used.
909 pass
--> 910 return complexjson.loads(self.text, **kwargs)
911
912 @property
~/anaconda3/envs/finance/lib/python3.7/json/__init__.py in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
346 parse_int is None and parse_float is None and
347 parse_constant is None and object_pairs_hook is None and not kw):
--> 348 return _default_decoder.decode(s)
349 if cls is None:
350 cls = JSONDecoder
~/anaconda3/envs/finance/lib/python3.7/json/decoder.py in decode(self, s, _w)
335
336 """
--> 337 obj, end = self.raw_decode(s, idx=_w(s, 0).end())
338 end = _w(s, end).end()
339 if end != len(s):
~/anaconda3/envs/finance/lib/python3.7/json/decoder.py in raw_decode(self, s, idx)
353 obj, end = self.scan_once(s, idx)
354 except StopIteration as err:
--> 355 raise JSONDecodeError("Expecting value", s, err.value) from None
356 return obj, end
JSONDecodeError: Expecting value: line 1 column 1 (char 0)
print(n.live_index('INDIA VIX')) returns {} even though 'INDIA VIX' shows up when using print(n.all_indices())
also how do i get data regarding usdinr fut ?
When I tried downloading NIFTY 50 index data of 2019, only 49 days data was available. Similar problem is there in 2017, 2018, 2019, 2020 and 2021.
Hi, Can you please add 2 more elements(Buy and Sell Qty) respectively to Live tick data which may display as follows:
tick_data = n.tick_data("HDFC")
tick_data['grapthData'][0:2]
[[1611566100000, 2635,2453,1235],
[1611566101000, 2636.25,2108,1987]]
Thanks in advance
sellQty and buyQty can be viewed in premarket data, same required for live tick data.
q = n.stock_quote("HDFC")
q['preOpenMarket']
{'preopen': [{'price': 2330.55, 'buyQty': 0, 'sellQty': 57},
{'price': 2400, 'buyQty': 0, 'sellQty': 7},
{'price': 2408.2, 'buyQty': 0, 'sellQty': 253},
{'price': 2449, 'buyQty': 0, 'sellQty': 10},
{'price': 2630.45, 'buyQty': 0, 'sellQty': 0, 'iep': True},
{'price': 2705.95, 'buyQty': 306, 'sellQty': 0},
{'price': 2710.5, 'buyQty': 8864, 'sellQty': 0},
{'price': 2718.9, 'buyQty': 16076, 'sellQty': 0},
{'price': 2750, 'buyQty': 100, 'sellQty': 0}],
'ato': {'buy': 1329, 'sell': 169},
'IEP': 2630.45,
'totalTradedVolume': 51122,
'finalPrice': 0,
'finalQuantity': 0,
'lastUpdateTime': '25-Jan-2021 09:07:41',
'totalBuyQuantity': 79071,
'totalSellQuantity': 58736,
'atoBuyQty': 1329,
'atoSellQty': 169}
No response
results returning from 2021 year. months are jumbled. dec 2021 missing.
from datetime import date as d
from jugaad_data.nse import index_df
# Download as pandas dataframe
df = index_df(symbol="NIFTY 50", from_date=d(2022,1,1),
to_date=d(2022,2,1))
print(df)
### Error snippet
![image](https://imgur.com/a/vkF8L7G)
Hi, I think better exception handling can be done in your code right now. For some reason, I cannot fetch any full_bhavcopy_save for 2019, and for prior years, I believe the data is not present anymore on https://archives.nseindia.com/products/content/sec_bhavdata_full_{ddmmyyyy}.csv
Are you planning to put any fix in there? Details of Exception below.
Traceback (most recent call last):
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\site-packages\urllib3\response.py", line 438, in _error_catcher
yield
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\site-packages\urllib3\response.py", line 519, in read
data = self._fp.read(amt) if not fp_closed else b""
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\http\client.py", line 458, in read
n = self.readinto(b)
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\http\client.py", line 502, in readinto
n = self.fp.readinto(b)
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\socket.py", line 669, in readinto
return self._sock.recv_into(b)
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\ssl.py", line 1241, in recv_into
return self.read(nbytes, buffer)
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\ssl.py", line 1099, in read
return self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\site-packages\requests\models.py", line 753, in generate
for chunk in self.raw.stream(chunk_size, decode_content=True):
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\site-packages\urllib3\response.py", line 576, in stream
data = self.read(amt=amt, decode_content=decode_content)
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\site-packages\urllib3\response.py", line 541, in read
raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\contextlib.py", line 131, in exit
self.gen.throw(type, value, traceback)
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\site-packages\urllib3\response.py", line 443, in _error_catcher
raise ReadTimeoutError(self._pool, None, "Read timed out.")
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='www.nseindia.com', port=443): Read timed out.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "bhavcopy_downloader.py", line 29, in
full_bhavcopy_save(dates, savepath)
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\site-packages\jugaad_data\nse\archives.py", line 91, in full_bhavcopy_save
text = self.full_bhavcopy_raw(dt)
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\site-packages\jugaad_data\nse\archives.py", line 81, in full_bhavcopy_raw
r = self.get("bhavcopy_full", yyyy=yyyy, mm=mm, dd=dd)
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\site-packages\jugaad_data\nse\archives.py", line 52, in get
self.r = self.s.get(url, timeout=self.timeout)
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\site-packages\requests\sessions.py", line 555, in get
return self.request('GET', url, **kwargs)
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\site-packages\requests\sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\site-packages\requests\sessions.py", line 677, in send
history = [resp for resp in gen]
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\site-packages\requests\sessions.py", line 677, in
history = [resp for resp in gen]
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\site-packages\requests\sessions.py", line 237, in resolve_redirects
resp = self.send(
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\site-packages\requests\sessions.py", line 697, in send
r.content
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\site-packages\requests\models.py", line 831, in content
self._content = b''.join(self.iter_content(CONTENT_CHUNK_SIZE)) or b''
File "C:\Users\Yash\anaconda3\envs\nse_env\lib\site-packages\requests\models.py", line 760, in generate
raise ConnectionError(e)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='www.nseindia.com', port=443): Read timed out.
I am getting the following error when installing the package. Should that be >7.1.2 ? Or what do I do about it?
I am running python 3.7.12 on win7.
jugaad-data 0.24 requires click==7.1.2, but you have click 8.1.3 which is incompatible.
pip install spyder==4.2.0
jugaad-data 0.24 requires click==7.1.2, but you have click 8.1.3 which is incompatible.
I want to get data for different timeframes like 5m, 10m, 1h, so on.
https://www.tradingview.com/chart/?symbol=NSE%3ANIFTY
No response
from jugaad_data.nse import NSELive
n = NSELive()
q = n.stock_quote("HDFC")
print(q['priceInfo'])
How can i get live data only and also collection of all top 50 companies means names of companies
from jugaad_data.nse import NSELive
n = NSELive()
q = n.stock_quote("HDFC")
print(q['priceInfo'])
from jugaad_data.nse import NSELive
n = NSELive()
q = n.stock_quote("HDFC")
print(q['priceInfo'])
For some reason the holidays list doesn't include 25/05/2020, I appended it manually for my use case.
You could add it to your library and complete it ๐
Hi - is there a plan to add support for FX derivatives? If not I'll be happy to contribute. Thanks.
Can't download option chain data. Screenshot attached.
from jugaad_data.nse import derivatives_csv, derivatives_df
df = derivatives_df(symbol="SBIN", from_date=date(2022,1,1), to_date=date(2022,2,24),
expiry_date=date(2022,3,31), instrument_type="OPTSTK", option_type="CE", strike_price=500)
print(df.head())
JSONDecodeError Traceback (most recent call last)
[<ipython-input-42-2a115219c3d7>](https://eei938yvh5k-496ff2e9c6d22116-0-colab.googleusercontent.com/outputframe.html?vrz=colab-20220328-060046-RC02_437851183#) in <module>()
1 from jugaad_data.nse import derivatives_csv, derivatives_df
2 df = derivatives_df(symbol="SBIN", from_date=date(2022,1,1), to_date=date(2022,2,24),
----> 3 expiry_date=date(2022,3,31), instrument_type="OPTSTK", option_type="CE", strike_price=500)
4 print(df.head())
11 frames
[/usr/lib/python3.7/json/decoder.py](https://eei938yvh5k-496ff2e9c6d22116-0-colab.googleusercontent.com/outputframe.html?vrz=colab-20220328-060046-RC02_437851183#) in raw_decode(self, s, idx)
353 obj, end = self.scan_once(s, idx)
354 except StopIteration as err:
--> 355 raise JSONDecodeError("Expecting value", s, err.value) from None
356 return obj, end
JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Hi, The library was working properly till day before yesterday and fetching the expiry dates as mentioned in the documentation. However, now it is through the error message and not fetching the expiry dates. Following is the code that I was using to fetch the current expiry date. It looks like in the results returned, there is no any key with the name 'records'. Only 'filtered' key with some values is being returned. Is it any API change in the NSE source or problem with Jugaad library? Please help.
n = NSELive()
option_chain = n.index_option_chain("BANKNIFTY")
expiry = option_chain['records']['expiryDates'][00]
jugad-data is using requests library does not support http2 but now nse support http2 protocol which faster and more efficient.
Kindly use httpx library instead of requests.
https://www.python-httpx.org/http2/
No response
NSE and BSE both do have their intraday data at EOD. Basically at EOD, we see an intraday chart we can hit the URL to get that intraday data and save it in local database/file.
https://www.nseindia.com/api/chart-databyindex?index=OPTIDXBANKNIFTY03-02-2022PE38000.00
Base URL in NSEArchies class:
base_url = "https://www1.nseindia.com"
Below is the code for downloading historical stock data for a company to a CSV file
from datetime import date from jugaad_data.nse import stock_csv stock_csv(symbol="NATIONALUM", from_date=date(2020,1,1), to_date=date(2023,1,18), series="EQ", _output="D:\\Stock\\NALUM.csv")
NSE Symbol : NATIONALUM
From Date : 1/1/2020
To Date : 1/18/2023
This should've pulled the data for all the 760 trading days in that time range into the CSV file. However the data is missing for the time period 10/6/2021 to 31/5/2022
This isn't a standalone issue for this particular company. All of them have the same issue of missing data for a particular period within the expected time range
nationalum.txt
NALUM.csv
from datetime import date
from jugaad_data.nse import stock_csv
# Download data and save to a csv file
stock_csv(symbol="NATIONALUM", from_date=date(2020,1,1),
to_date=date(2023,1,18), series="EQ", output="D:\\Stock\\NALUM.csv")
Not an error. Missing data in the output file as explained in the description
Can we have option to download historical option chain when we give symbol, trading date and expiry information ? it will be really helpful
Based on bhavcopy information
No response
I'm getting the below error when I run your code on Heroku and I tried on desktop its working fine buy on the cloud it is displaying the error:
2020-10-01T18:20:03.659025+00:00 app[worker.1]: started.......
2020-10-01T18:20:05.585950+00:00 app[worker.1]: Traceback (most recent call last):
2020-10-01T18:20:05.586335+00:00 app[worker.1]: File "script.py", line 420, in
2020-10-01T18:20:05.586914+00:00 app[worker.1]: schedule.run_pending()
2020-10-01T18:20:05.586954+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.6/site-packages/schedule/init.py", line 563, in run_pending
2020-10-01T18:20:05.587443+00:00 app[worker.1]: default_scheduler.run_pending()
2020-10-01T18:20:05.587478+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.6/site-packages/schedule/init.py", line 94, in run_pending
2020-10-01T18:20:05.587736+00:00 app[worker.1]: self._run_job(job)
2020-10-01T18:20:05.591719+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.6/site-packages/schedule/init.py", line 147, in _run_job
2020-10-01T18:20:05.592011+00:00 app[worker.1]: ret = job.run()
2020-10-01T18:20:05.592051+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.6/site-packages/schedule/init.py", line 466, in run
2020-10-01T18:20:05.592485+00:00 app[worker.1]: ret = self.job_func()
2020-10-01T18:20:05.592525+00:00 app[worker.1]: File "script.py", line 307, in nifty_500_stocks
2020-10-01T18:20:05.592856+00:00 app[worker.1]: series="EQ")
2020-10-01T18:20:05.592892+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.6/site-packages/jugaad_data/nse/init.py", line 170, in stock_df
2020-10-01T18:20:05.593144+00:00 app[worker.1]: raw = stock_raw(symbol, from_date, to_date, series)
2020-10-01T18:20:05.593179+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.6/site-packages/jugaad_data/nse/init.py", line 108, in stock_raw
2020-10-01T18:20:05.593417+00:00 app[worker.1]: return list(itertools.chain.from_iterable(chunks))
2020-10-01T18:20:05.593527+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.6/concurrent/futures/_base.py", line 586, in result_iterator
2020-10-01T18:20:05.594038+00:00 app[worker.1]: yield fs.pop().result()
2020-10-01T18:20:05.594076+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.6/concurrent/futures/_base.py", line 425, in result
2020-10-01T18:20:05.594459+00:00 app[worker.1]: return self.__get_result()
2020-10-01T18:20:05.594500+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
2020-10-01T18:20:05.600236+00:00 app[worker.1]: raise self._exception
2020-10-01T18:20:05.600244+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.6/concurrent/futures/thread.py", line 56, in run
2020-10-01T18:20:05.600439+00:00 app[worker.1]: result = self.fn(*self.args, **self.kwargs)
2020-10-01T18:20:05.600443+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.6/site-packages/jugaad_data/util.py", line 110, in wrapper
2020-10-01T18:20:05.600627+00:00 app[worker.1]: j = function(**kw)
2020-10-01T18:20:05.600632+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.6/site-packages/jugaad_data/nse/init.py", line 75, in _stock
2020-10-01T18:20:05.600801+00:00 app[worker.1]: j = self.r.json()
2020-10-01T18:20:05.600805+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.6/site-packages/requests/models.py", line 898, in json
2020-10-01T18:20:05.601212+00:00 app[worker.1]: return complexjson.loads(self.text, **kwargs)
2020-10-01T18:20:05.601237+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.6/json/init.py", line 354, in loads
2020-10-01T18:20:05.601530+00:00 app[worker.1]: return _default_decoder.decode(s)
2020-10-01T18:20:05.601534+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.6/json/decoder.py", line 339, in decode
2020-10-01T18:20:05.601754+00:00 app[worker.1]: obj, end = self.raw_decode(s, idx=_w(s, 0).end())
2020-10-01T18:20:05.601759+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.6/json/decoder.py", line 357, in raw_decode
2020-10-01T18:20:05.601972+00:00 app[worker.1]: raise JSONDecodeError("Expecting value", s, err.value) from None
2020-10-01T18:20:05.602034+00:00 app[worker.1]: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
2020-10-01T18:20:05.951067+00:00 heroku[worker.1]: Process exited with status 1
It appears that NSE may be blocking IP addresses. After a few hundred historical downloads, we get the below error for all dates:
Failed to download for below dates, these might be holidays, please check -
2015-01-01
2015-01-02
2015-01-05
2015-01-06
2015-01-07
2015-01-08
2015-01-09
2015-01-12
2015-01-13
2015-01-14
2015-01-15
2015-01-16
2015-01-19
Is it possible to add volume as it is very important and time period like 15 minutes, 1 hour, 1 day etc,
Getting error in Kaggle notebook
/opt/conda/lib/python3.7/site-packages/jugaad_data/util.py in wrapper(*args, **kw)
102 if not os.path.isfile(path):
103 if not os.path.exists(cache_dir):
--> 104 os.makedirs(cache_dir)
105 j = function(**kw)
106 with open(path, 'wb') as fp:
/opt/conda/lib/python3.7/os.py in makedirs(name, mode, exist_ok)
219 return
220 try:
--> 221 mkdir(name, mode)
222 except OSError:
223 # Cannot rely on checking for EEXIST, since the operating system
FileExistsError: [Errno 17] File exists: '/root/.cache/nsehistory-stock'
How often can the live data be fetched (like minute intervals). Also, could you please elaborate how to fetch that at a minute basis through code for say options.
The website: https://niftyindices.com is not working. Is this temporary or has there been a migration to some other url. If so, the code needs to be updated to reflect that.
Specifically, I'm accessing:
jugaad-data/jugaad_data/nse/history.py
Line 275 in c6a6427
The returned response code is 403.
nifty_50 = index_df(symbol="NIFTY 50",
from_date='2023-09-01',
to_date='2023-09-20')
Traceback (most recent call last):
File "/Users/uditg/anaconda3/envs/ny/lib/python3.11/site-packages/requests/models.py", line 971, in json
return complexjson.loads(self.text, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/uditg/anaconda3/envs/ny/lib/python3.11/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/uditg/anaconda3/envs/ny/lib/python3.11/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/uditg/anaconda3/envs/ny/lib/python3.11/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/uditg/Documents/dummy/ny.py", line 391, in <module>
main(args=parser.parse_args())
File "/Users/uditg/Documents/dummy/ny.py", line 301, in main
nifty_50 = index_df(symbol="NIFTY 50",
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/uditg/anaconda3/envs/ny/lib/python3.11/site-packages/jugaad_data/nse/history.py", line 371, in index_df
raw = index_raw(symbol, from_date, to_date)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/uditg/anaconda3/envs/ny/lib/python3.11/site-packages/jugaad_data/nse/history.py", line 321, in index_raw
return list(itertools.chain.from_iterable(chunks))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/uditg/anaconda3/envs/ny/lib/python3.11/concurrent/futures/_base.py", line 619, in result_iterator
yield _result_or_cancel(fs.pop())
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/uditg/anaconda3/envs/ny/lib/python3.11/concurrent/futures/_base.py", line 317, in _result_or_cancel
return fut.result(timeout)
^^^^^^^^^^^^^^^^^^^
File "/Users/uditg/anaconda3/envs/ny/lib/python3.11/concurrent/futures/_base.py", line 449, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File "/Users/uditg/anaconda3/envs/ny/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
raise self._exception
File "/Users/uditg/anaconda3/envs/ny/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/uditg/anaconda3/envs/ny/lib/python3.11/site-packages/jugaad_data/util.py", line 109, in wrapper
j = function(**kw)
^^^^^^^^^^^^^^
File "/Users/uditg/anaconda3/envs/ny/lib/python3.11/site-packages/jugaad_data/nse/history.py", line 315, in _index
return json.loads(self.r.json()['d'])
^^^^^^^^^^^^^
File "/Users/uditg/anaconda3/envs/ny/lib/python3.11/site-packages/requests/models.py", line 975, in json
raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
The error comes up only when one makes two or more calls to the fetch_stock_data() function. This has been coming at a higher frequency for longer periods of time. Tried it multiple times after creating a thread like in the Issue in October 2020 but not resolving.
from jugaad_data.nse import stock_csv, stock_df
def fetch_stock_data(symbol, start_year, start_month, start_day, end_year, end_month, end_day, series):
"""
Function to fetch historical stock data of specific symbols for a given range of dates
Input: Stock Symbol and the date range
Output: Stock Historical Data in form of a data frame
"""
df = stock_df(symbol=str(symbol),
from_date=date(start_year,start_day,start_month),to_date=date(end_year,end_month,end_day),
series=str(series))
return df
techm_df = fetch_stock_data("TECHM", 2010, 1, 1, 2021, 12, 31, "EQ")
\local\programs\python\python39\lib\json\__init__.py in loads(s, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
344 parse_int is None and parse_float is None and
345 parse_constant is None and object_pairs_hook is None and not kw):
--> 346 return _default_decoder.decode(s)
347 if cls is None:
348 cls = JSONDecoder
If there was a function to list all possible Stocks (symbols) and Fno entities it will be good.
Hi
Can you please tell me if we can get the tik by tik data.
It worked fine yesterday but today it is showing error. This is the only python library that working fine with historical data without much code. Please sort out this issue thanks.
hello,
code looks really good, is it possible to get full option chain with all strike price like nse so we can check where is the highest open interest on perticular stock.
it will be good to find right stocks.
thanks
nseindia
No response
Hi, is it possible to also fetch the index futures Live data, I see there is a function stock_quote_fno() which fails for BANKNIFTY and NIFTY, can a separate function be created to fetch these or add them to the existing ones?
The Dividend data can be got from NSE but some kind of string matching will be needed, the descriptions follow majorly some kind of template however cant be sure if all the entries follow them... Another thing might be needed is that Dividends need to be split adjusted since these are just mentioned as is when its announced however it might be difficult to make use of the data if its not split adjusted... Even the bonus/split details are provided in the same corp announcements table...
https://www.nseindia.com/companies-listing/corporate-filings-actions
No response
Hi the historical bhav copy files, will not take care of the any adjustments. Does the module take care of splits, etc?
Deliverable volume data is already provided on nse India 's new website and it should be added in the stock_df result also.
https://www.nseindia.com/report-detail/eq_security
No response
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.