0xrushi / tradingview-scraper Goto Github PK
View Code? Open in Web Editor NEWLicense: MIT License
License: MIT License
Hello! Sent you an email (at rushic24). Thank you.
Stack trace:
<some data>
~m~4~m~~h~1
~m~4~m~~h~2
~m~4~m~~h~3
~m~4~m~~h~4
~m~4~m~~h~5
~m~4~m~~h~6
~m~4~m~~h~7
Connection is already closed.
Traceback (most recent call last):
File "main.py", line 132, in <module>
generate_csv(a)
File "main.py", line 54, in generate_csv
out= re.search('"s":\[(.+?)\}\]', a).group(1)
AttributeError: 'NoneType' object has no attribute 'group'
Symbol: HOSE:SSI
. Similar stack trace for HOSE:TCB
Is there an api call to be able to get the drawings on a chart using user information and chart layout? Where can I find how to do this like api for websocket for example?
I tried executing modify_series with "r,start:end" epoch timestamp, but it doesn't seem to work. Has anyone successfully able to get the date range data from a stock?
First of all thanks for writing such beautiful code.
As you have already written code for getting candle / bar data using websocket, could you please enhance it for live streaming data also.
The difference I found is that for streaming data, every 10 seconds a message is sent ~m~5~m~~h~19
where the last digit changes.
Further it also seems that ~m~5~m~~h~99
then changes to ~m~6~m~~h~100
You can select any crypto data for the same like BTCUSDT so that it is live 24x7
Thanks in advance.
Is it possible to pull index data (BTC.D, and TOTAL crypto market cap)?
In your main code, csv is downloaded and it takes a lot of time.
Is it possible to get only last 1- to 20 bars for a timeframe of 1 minute or 5 / 15 minutes?
It must be continuous and stored in dataframe instead of csv.
Thanks for help.
hi guys, I did what you said, copying the message from websocket, but I can't get anything. Here is the message from the studio:
[chart_session,"st10","st1","sds_1","Script@tv-scripting-101!",{"text":"N10EngHTL9gXlYBxsYMAcg==_C9aX1yQzT6tD41AdX5pi1s3cIqMUTZ+znW6dFBocJwUfNSSEqS+ngdMg89WjGBKkLDping9inrGuhEWfeImdyASyKH2a1S8ut2OO4Eu1KVGbPbJXssL3Ryk9e2U12orZOeSbaFPFQQppJawn9dOlicRgpJNB0Vxfd7iZr45tOs4i1fblU3rIgUNHM9H6bd1EsoX7borDHcIGkHxEPMoL+BVMpkzOUD+RwE8FWTSQgsrbWOhnY4XzC+cjZLcWO5OTqj5YbgJidmztsdl1ssmXrR8hK+n9tL7Jb7N8kfFZOt1VlNFHItYS6LsPzoafCPA8IuvdQkWVoURymJJgMhGpSCRSkrUbl1oIq27J+dTScWCrI0BSpjgsVzkP010A","pineId":"PUB;7rviMJLWCGApT2hDsy9nLFnRYr2CG4zj","pineVersion":"1.0","in_0":{"v":'9',"f":true,"t":"integer"},"in_1":{"v":'20',"f":true,"t":"integer"},"in_2":{"v":'400',"f":true,"t":"integer"},"in_3":{"v":'200',"f":true,"t":"integer"}}])
I guess it is because of the last part, which directly modifies the parameters (it is a 4 ema indicator).
Thx guys!
for xi in x: xi= re.split('\[|:|,|\]', xi) print(xi) ind= int(xi[1]) ts= datetime.fromtimestamp(float(xi[4])).strftime("%Y/%m/%d, %H:%M:%S") employee_writer.writerow([ind, ts, float(xi[5]), float(xi[6]), float(xi[7]), float(xi[8]), float(xi[9])])
ts= datetime.fromtimestamp(float(xi[4])).strftime("%Y/%m/%d, %H:%M:%S")
I will try to solve it!
Thanks for your code, it works very well on my end. I was trying to stream from the chart session multiple tickers, but somehow, it only returns 1 ticker at a time. Was wondering if you have tried streaming multiple tickers before and got it to work?
This is what I tried so far, but has only managed to stream the first ticker:
def _add_chart_symbol(self, sym: str):
""" Chart symbol - Only required for the first symbol """
return "=" + json.dumps({"symbol": sym})
send(ws, "chart_create_session", [c_session, ""])
send(ws, "switch_timezone", [c_session, "Asia/Singapore"])
send(ws, "resolve_symbol", [
c_session, "symbol_1",
self._add_chart_symbol("BINANCE:ETHUSD")
])
send(ws, "create_series",
[c_session, "s1", "s1", "symbol_1", "15", 300])
send(ws, "resolve_symbol", [c_session, "ss_1", "BITSTAMP:BTCUSD"])
send(ws, "create_study", [c_session, "st1", "st1", "s1", "Overlay@tv-basicstudies-121", {"symbol": "BITSTAMP:BTCUSD"}])
Hi,
Is it possible to scrap ohlc candle data using the library? so far, I found the volume available in the provided examples but don't know if we can pull open close high low as well?
How can I modify the script to pull data for only a date range?
Someone know how to jump the captcha at login for premium account?
I tried reading the raw messages in the websocket, but couldn't figure it out.
raise WebSocketBadStatusException("Handshake status %d", status)
websocket._exceptions.WebSocketBadStatusException: Handshake status 403
I got this when I tried to run the code.
name 'outfilename' is not defined
Something missing in the latest commit 7093fb1bf2ebae67e971aafbc30e8c8b35939bdc
?
Great script btw! Thank you.
Hello, your python script is very useful and robust!
But how do you select a specific study by inspecting the tradingview webpage? I would like to scrap from the SuperTrend indicator (by KivancOzbilgic)
I see that you are scrapping volume information from "" sendMessage(ws, "create_study", [chart_session,"st1","st1","s1","Volume@tv-basicstudies-118",{"length":20,"col_prev_close":"false"}]) ""
What should I change in the code to get hourly/daily data?
import websocket
import time
import threading
import json
SOCKET = "wss://data.tradingview.com/socket.io/websocket"
headers = {
"Accept-Encoding": "gzip, deflate, br",
# "Accept-Language": "en-US,en;q=0.9",
# "Cache-Control": "no-cache",
# "Connection": "Upgrade",
"Host": "data.tradingview.com",
"Origin": "https://www.tradingview.com",
# "Pragma": "no-cache",
# "Sec-WebSocket-Extensions": "permessage-deflate; client_max_window_bits",
# "Sec-WebSocket-Key": "Qf9IDRKqcgNBrNs7X4FK9w==",
# "Sec-WebSocket-Version": 13,
# "Upgrade": "websocket",
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.111 Safari/537.36"
}
"""
~m~55~m~{"m":"chart_create_session","p":["cs_zEcm9GqyQdK0",""]}
'~m~52~m~{"m":"quote_create_session","p":["qs_dl0OygXkO4uu"]}'
~m~98~m~{"m":"quote_add_symbols","p":["qs_dl0OygXkO4uu","BINANCE:BTCUSDT",{"flags":["force_permission"]}]}
~m~68~m~{"m":"quote_fast_symbols","p":["qs_dl0OygXkO4uu","BINANCE:BTCUSDT"]}
~m~6~m~~h~157
~m~411~m~{"m":"quote_fast_symbols","p":["qs_x72fChUYomPp","MCX:GOLDGUINEAN2020","NSE:NIFTY","MCX:GOLDPETALN2020","NSE:BAJFINANCE","MCX:SILVERMQ2020","NSE:SBIN","NSE:BANKNIFTY","NSE:RBLBANK","NSE:INDIAVIX","NSE:INDUSINDBK","NSE:CIPLA","NSE:SUNTV","MCX:NATURALGAS1!","MCX:SILVERMIC1!","MCX:CRUDEOIL1!","MCX:GOLDM1!","NSE:BANKNIFTY1!","SGX:IN1!","OANDA:USDINR","NSE:DABUR","NSE:BERGEPAINT","NASDAQ:TSLA","BINANCE:BTCUSDT"]}
~m~68~m~{"m":"quote_fast_symbols","p":["qs_dl0OygXkO4uu","BINANCE:BTCUSDT"]}
~m~98~m~{"m":"quote_add_symbols","p":["qs_x72fChUYomPp","BINANCE:BTCUSDT",{"flags":["force_permission"]}]}
"""
def on_open(ws):
print('opened connection')
# def run(*args):
# for i in range(30):
# time.sleep(1)
# ws.send("Hello %d" % i)
# time.sleep(1)
# ws.close()
# print("thread terminating...")
# threading.start_new_thread(run, ())
time.sleep(2)
# ws.send('~m~524~m~{"m":"set_auth_token","p":[""]}')
ws.send('~m~54~m~{"m":"set_auth_token","p":["unauthorized_user_token"]}')
ws.send('~m~55~m~{"m":"chart_create_session","p":["cs_zEcm9GqyQdK0",""]}')
ws.send('~m~52~m~{"m":"quote_create_session","p":["qs_x72fChUYomPp"]}')
ws.send('~m~344~m~{"m":"quote_set_fields","p":["qs_x72fChUYomPp","ch","chp","current_session","description","local_description","language","exchange","fractional","is_tradable","lp","lp_time","minmov","minmove2","original_name","pricescale","pro_name","short_name","type","update_mode","volume","currency_code","logoid","currency-logoid","base-currency-logoid"]}')
ws.send('~m~98~m~{"m":"quote_add_symbols","p":["qs_x72fChUYomPp","BINANCE:BTCUSDT",{"flags":["force_permission"]}]}')
ws.send('~m~98~m~{"m":"quote_add_symbols","p":["qs_x72fChUYomPp","BINANCE:BNBUSDT",{"flags":["force_permission"]}]}')
ws.send('~m~91~m~{"m":"quote_add_symbols","p":["qs_x72fChUYomPp","NSE:SBIN",{"flags":["force_permission"]}]}')
ws.send('~m~98~m~{"m":"quote_fast_symbols","p":["qs_x72fChUYomPp","BINANCE:BTCUSDT","BINANCE:BNBUSDT", "NSE:SBIN"]}')
def on_close(ws):
print('closed connection')
def on_message(ws, message):
# p = message.split('~', -1)[4]
# data = json.loads(p)
# print(data)
# print(f'received message :: {message}')
if 'lp' in message:
p = message.split('~', -1)[4]
data = json.loads(p)
# print(data)
timestamp = time.strftime("%Y-%m-%d %H:%M:%S")
symbol = data['p'][1]['n']
ltp = data['p'][1]['v']['lp']
volume = data['p'][1]['v']['volume']
if symbol.upper() == "BINANCE:BTCUSDT":
print(f'tick :: {timestamp} :: {symbol} :: {ltp} :: {volume}')
def create_websocket_connection():
websocket.enableTrace(False)
ws = websocket.WebSocketApp(
SOCKET, on_message=on_message, on_open=on_open, on_close=on_close)
wst = threading.Thread(target=ws.run_forever)
wst.daemon = True
wst.start()
conn_timeout = 60
while not ws.sock.connected and conn_timeout:
time.sleep(1)
conn_timeout -= 1
while ws.sock is not None:
time.sleep(10)
print("buraya geldi")
if __name__ == "__main__":
while True:
create_websocket_connection()
print("buraya geldi2")
def get_auth_token():
sign_in_url = 'https://www.tradingview.com/accounts/signin/'
username = 'username'
password = 'password'
data = {"username": username, "password": password, "remember": "on"}
headers = {
'Referer': 'https://www.tradingview.com'
}
response = requests.post(url=sign_in_url, data=data, headers=headers)
auth_token = response.json()['user']['auth_token']
return auth_token
Then send a message through the tunnel
sendMessage(ws, "set_auth_token", ["auth_token"])
ws = create_connection(
'wss://data.tradingview.com/socket.io/websocket?from=chart/XXyour_chartXX/&date=XXXX_XX_XX-XX_XX',headers=headers)
result = ws.recv()
if re.search('~m~\\d+~m~~h~\\d+', result):
#Send ping message
ws.send(result)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.