Code Monkey home page Code Monkey logo

tradingview-scraper's Introduction

tradingview-scraper 2020

Install imports

pip install -r requirements.txt

Run

python main.py

Want to add something? file a PR and join this discord

https://discord.gg/pEssYdtXZx


Have a tradingview username and password?

credits: euvgub

def get_auth_token():
    sign_in_url = 'https://www.tradingview.com/accounts/signin/'
    username = 'username'
    password = 'password'
    data = {"username": username, "password": password, "remember": "on"}
    headers = {
        'Referer': 'https://www.tradingview.com'
    }
    response = requests.post(url=sign_in_url, data=data, headers=headers)
    auth_token = response.json()['user']['auth_token']    
    return auth_token

Then send a message through the tunnel sendMessage(ws, "set_auth_token", ["auth_token"])

ws = create_connection( 'wss://data.tradingview.com/socket.io/websocket?from=chart/XXyour_chartXX/&date=XXXX_XX_XX-XX_XX',headers=headers)

result = ws.recv()
if re.search('~m~\\d+~m~~h~\\d+', result):
            #Send ping message
            ws.send(result)

Buy Me A Coffee

tradingview-scraper's People

Contributors

0xrushi avatar jack-new-project avatar pisanvs avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

tradingview-scraper's Issues

OHLC DATA

Hi,

Is it possible to scrap ohlc candle data using the library? so far, I found the volume available in the provided examples but don't know if we can pull open close high low as well?

auth_token

def get_auth_token():
    sign_in_url = 'https://www.tradingview.com/accounts/signin/'
    username = 'username'
    password = 'password'
    data = {"username": username, "password": password, "remember": "on"}
    headers = {
        'Referer': 'https://www.tradingview.com'
    }
    response = requests.post(url=sign_in_url, data=data, headers=headers)
    auth_token = response.json()['user']['auth_token']    
    return auth_token

Then send a message through the tunnel
sendMessage(ws, "set_auth_token", ["auth_token"])

ws = create_connection(
'wss://data.tradingview.com/socket.io/websocket?from=chart/XXyour_chartXX/&date=XXXX_XX_XX-XX_XX',headers=headers)

result = ws.recv()
if re.search('~m~\\d+~m~~h~\\d+', result):
            #Send ping message
            ws.send(result)

Chart series - multiple symbols

Thanks for your code, it works very well on my end. I was trying to stream from the chart session multiple tickers, but somehow, it only returns 1 ticker at a time. Was wondering if you have tried streaming multiple tickers before and got it to work?

This is what I tried so far, but has only managed to stream the first ticker:

def _add_chart_symbol(self, sym: str):
    """ Chart symbol - Only required for the first symbol """
    return "=" + json.dumps({"symbol": sym})


send(ws, "chart_create_session", [c_session, ""])
send(ws, "switch_timezone", [c_session, "Asia/Singapore"])
send(ws, "resolve_symbol", [
    c_session, "symbol_1",
    self._add_chart_symbol("BINANCE:ETHUSD")
])
send(ws, "create_series",
     [c_session, "s1", "s1", "symbol_1", "15", 300])
send(ws, "resolve_symbol", [c_session, "ss_1", "BITSTAMP:BTCUSD"])
send(ws, "create_study", [c_session, "st1", "st1", "s1", "Overlay@tv-basicstudies-121", {"symbol": "BITSTAMP:BTCUSD"}])

Login captcha

Someone know how to jump the captcha at login for premium account?

NameError: name 'datetime' is not defined

for xi in x: xi= re.split('\[|:|,|\]', xi) print(xi) ind= int(xi[1]) ts= datetime.fromtimestamp(float(xi[4])).strftime("%Y/%m/%d, %H:%M:%S") employee_writer.writerow([ind, ts, float(xi[5]), float(xi[6]), float(xi[7]), float(xi[8]), float(xi[9])])

ts= datetime.fromtimestamp(float(xi[4])).strftime("%Y/%m/%d, %H:%M:%S")

I will try to solve it!

Hourly/daily

What should I change in the code to get hourly/daily data?

Error: `AttributeError: 'NoneType' object has no attribute 'group'`

Stack trace:

<some data>

~m~4~m~~h~1
~m~4~m~~h~2
~m~4~m~~h~3
~m~4~m~~h~4
~m~4~m~~h~5
~m~4~m~~h~6
~m~4~m~~h~7

Connection is already closed.
Traceback (most recent call last):
  File "main.py", line 132, in <module>
    generate_csv(a)
  File "main.py", line 54, in generate_csv
    out= re.search('"s":\[(.+?)\}\]', a).group(1)
AttributeError: 'NoneType' object has no attribute 'group'

Symbol: HOSE:SSI. Similar stack trace for HOSE:TCB

Code for getting live streaming data from trading view by sending heartbeat every 10 seconds

First of all thanks for writing such beautiful code.
As you have already written code for getting candle / bar data using websocket, could you please enhance it for live streaming data also.

The difference I found is that for streaming data, every 10 seconds a message is sent ~m~5~m~~h~19 where the last digit changes.
Further it also seems that ~m~5~m~~h~99 then changes to ~m~6~m~~h~100

You can select any crypto data for the same like BTCUSDT so that it is live 24x7

Thanks in advance.

How can I scrape data 24/7? I found such a solution, but it did not seem reliable.

import websocket
import time
import threading
import json

SOCKET = "wss://data.tradingview.com/socket.io/websocket"

headers = {
    "Accept-Encoding": "gzip, deflate, br",
    # "Accept-Language": "en-US,en;q=0.9",
    # "Cache-Control": "no-cache",
    # "Connection": "Upgrade",
    "Host": "data.tradingview.com",
    "Origin": "https://www.tradingview.com",
    # "Pragma": "no-cache",
    # "Sec-WebSocket-Extensions": "permessage-deflate; client_max_window_bits",
    # "Sec-WebSocket-Key": "Qf9IDRKqcgNBrNs7X4FK9w==",
    # "Sec-WebSocket-Version": 13,
    # "Upgrade": "websocket",
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.111 Safari/537.36"
}

"""
~m~55~m~{"m":"chart_create_session","p":["cs_zEcm9GqyQdK0",""]}
'~m~52~m~{"m":"quote_create_session","p":["qs_dl0OygXkO4uu"]}'
~m~98~m~{"m":"quote_add_symbols","p":["qs_dl0OygXkO4uu","BINANCE:BTCUSDT",{"flags":["force_permission"]}]}
~m~68~m~{"m":"quote_fast_symbols","p":["qs_dl0OygXkO4uu","BINANCE:BTCUSDT"]}
~m~6~m~~h~157
~m~411~m~{"m":"quote_fast_symbols","p":["qs_x72fChUYomPp","MCX:GOLDGUINEAN2020","NSE:NIFTY","MCX:GOLDPETALN2020","NSE:BAJFINANCE","MCX:SILVERMQ2020","NSE:SBIN","NSE:BANKNIFTY","NSE:RBLBANK","NSE:INDIAVIX","NSE:INDUSINDBK","NSE:CIPLA","NSE:SUNTV","MCX:NATURALGAS1!","MCX:SILVERMIC1!","MCX:CRUDEOIL1!","MCX:GOLDM1!","NSE:BANKNIFTY1!","SGX:IN1!","OANDA:USDINR","NSE:DABUR","NSE:BERGEPAINT","NASDAQ:TSLA","BINANCE:BTCUSDT"]}
~m~68~m~{"m":"quote_fast_symbols","p":["qs_dl0OygXkO4uu","BINANCE:BTCUSDT"]}
~m~98~m~{"m":"quote_add_symbols","p":["qs_x72fChUYomPp","BINANCE:BTCUSDT",{"flags":["force_permission"]}]}

"""



def on_open(ws):
    print('opened connection')
    # def run(*args):
    #     for i in range(30):
    #         time.sleep(1)
    #         ws.send("Hello %d" % i)
    #     time.sleep(1)
    #     ws.close()
    #     print("thread terminating...")
    # threading.start_new_thread(run, ())
    time.sleep(2)
    # ws.send('~m~524~m~{"m":"set_auth_token","p":[""]}')
    ws.send('~m~54~m~{"m":"set_auth_token","p":["unauthorized_user_token"]}')
    ws.send('~m~55~m~{"m":"chart_create_session","p":["cs_zEcm9GqyQdK0",""]}')
    ws.send('~m~52~m~{"m":"quote_create_session","p":["qs_x72fChUYomPp"]}')
    ws.send('~m~344~m~{"m":"quote_set_fields","p":["qs_x72fChUYomPp","ch","chp","current_session","description","local_description","language","exchange","fractional","is_tradable","lp","lp_time","minmov","minmove2","original_name","pricescale","pro_name","short_name","type","update_mode","volume","currency_code","logoid","currency-logoid","base-currency-logoid"]}')

    ws.send('~m~98~m~{"m":"quote_add_symbols","p":["qs_x72fChUYomPp","BINANCE:BTCUSDT",{"flags":["force_permission"]}]}')
    ws.send('~m~98~m~{"m":"quote_add_symbols","p":["qs_x72fChUYomPp","BINANCE:BNBUSDT",{"flags":["force_permission"]}]}')
    ws.send('~m~91~m~{"m":"quote_add_symbols","p":["qs_x72fChUYomPp","NSE:SBIN",{"flags":["force_permission"]}]}')
    ws.send('~m~98~m~{"m":"quote_fast_symbols","p":["qs_x72fChUYomPp","BINANCE:BTCUSDT","BINANCE:BNBUSDT", "NSE:SBIN"]}')


def on_close(ws):
    print('closed connection')


def on_message(ws, message):
    # p = message.split('~', -1)[4]
    # data = json.loads(p)
    # print(data)
    # print(f'received message :: {message}')
    if 'lp' in message:
        p = message.split('~', -1)[4]
        data = json.loads(p)
        # print(data)
        timestamp = time.strftime("%Y-%m-%d %H:%M:%S")
        symbol = data['p'][1]['n']
        ltp = data['p'][1]['v']['lp']
        volume = data['p'][1]['v']['volume']
        if symbol.upper() == "BINANCE:BTCUSDT":
            print(f'tick :: {timestamp} :: {symbol} :: {ltp} :: {volume}')



def create_websocket_connection():
    websocket.enableTrace(False)
    ws = websocket.WebSocketApp(
        SOCKET, on_message=on_message, on_open=on_open, on_close=on_close)
    wst = threading.Thread(target=ws.run_forever)
    wst.daemon = True
    wst.start()

    conn_timeout = 60
    while not ws.sock.connected and conn_timeout:
        time.sleep(1)
        conn_timeout -= 1

    while ws.sock is not None:
        time.sleep(10)
    print("buraya geldi")

if __name__ == "__main__":
    while True:
        create_websocket_connection()
        print("buraya geldi2")
        
        
        
        
        
        
        

Error : Not working !!!

raise WebSocketBadStatusException("Handshake status %d", status)
websocket._exceptions.WebSocketBadStatusException: Handshake status 403

Get Drawings

Is there an api call to be able to get the drawings on a chart using user information and chart layout? Where can I find how to do this like api for websocket for example?

Date Range data

I tried executing modify_series with "r,start:end" epoch timestamp, but it doesn't seem to work. Has anyone successfully able to get the date range data from a stock?

Index data

Is it possible to pull index data (BTC.D, and TOTAL crypto market cap)?

Create Study

Hello, your python script is very useful and robust!
But how do you select a specific study by inspecting the tradingview webpage? I would like to scrap from the SuperTrend indicator (by KivancOzbilgic)

I see that you are scrapping volume information from "" sendMessage(ws, "create_study", [chart_session,"st1","st1","s1","Volume@tv-basicstudies-118",{"length":20,"col_prev_close":"false"}]) ""

Thank you very much!
supertrend

Date range

How can I modify the script to pull data for only a date range?

name 'outfilename' is not defined

I got this when I tried to run the code.
name 'outfilename' is not defined
Something missing in the latest commit 7093fb1bf2ebae67e971aafbc30e8c8b35939bdc?

Great script btw! Thank you.

email

Hello! Sent you an email (at rushic24). Thank you.

Create study

hi guys, I did what you said, copying the message from websocket, but I can't get anything. Here is the message from the studio:

[chart_session,"st10","st1","sds_1","Script@tv-scripting-101!",{"text":"N10EngHTL9gXlYBxsYMAcg==_C9aX1yQzT6tD41AdX5pi1s3cIqMUTZ+znW6dFBocJwUfNSSEqS+ngdMg89WjGBKkLDping9inrGuhEWfeImdyASyKH2a1S8ut2OO4Eu1KVGbPbJXssL3Ryk9e2U12orZOeSbaFPFQQppJawn9dOlicRgpJNB0Vxfd7iZr45tOs4i1fblU3rIgUNHM9H6bd1EsoX7borDHcIGkHxEPMoL+BVMpkzOUD+RwE8FWTSQgsrbWOhnY4XzC+cjZLcWO5OTqj5YbgJidmztsdl1ssmXrR8hK+n9tL7Jb7N8kfFZOt1VlNFHItYS6LsPzoafCPA8IuvdQkWVoURymJJgMhGpSCRSkrUbl1oIq27J+dTScWCrI0BSpjgsVzkP010A","pineId":"PUB;7rviMJLWCGApT2hDsy9nLFnRYr2CG4zj","pineVersion":"1.0","in_0":{"v":'9',"f":true,"t":"integer"},"in_1":{"v":'20',"f":true,"t":"integer"},"in_2":{"v":'400',"f":true,"t":"integer"},"in_3":{"v":'200',"f":true,"t":"integer"}}])

I guess it is because of the last part, which directly modifies the parameters (it is a 4 ema indicator).

Thx guys!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.