Code Monkey home page Code Monkey logo

Comments (10)

twopirllc avatar twopirllc commented on May 13, 2024 1

Hello @22Vignesh-M,

Your speed depends on your sources, dimensions and hardware of course. Pandas TA just runs Pandas compatible TA features comparable with TA Lib. It is agnostic to how you want to process the data. You might want to check other Pandas Frameworks that are built for multiprocessing, event processing, streaming, ETL, et al for more efficient solutions.

I have not tried this out, but you can try to help speed up your processing using a temporary DataFrame that is updated every tick and is slightly larger than the your indicator of maximum period. For example, like this example pseudo code:

import pandas as pd
import pandas_ta as ta

momo_bands_sma_ta = [
    {"kind":"sma", "length": 50},
    {"kind":"sma", "length": 200},
    {"kind":"bbands", "length": 20},
    {"kind":"macd"},
    {"kind":"rsi"},
    {"kind":"log_return", "cumulative": True}, # Creates new column: "CUMLOGRET_1"
    {"kind":"sma", "close": "CUMLOGRET_1", "length": 5, "suffix": "CUMLOGRET"},
]
momo_bands_sma_strategy = ta.Strategy(
    "Momo, Bands and SMAs and Cumulative Log Returns", # name
    momo_bands_sma_ta, # ta
    "MACD and RSI Momo with BBANDS and SMAs 50 & 200 and Cumulative Log Returns" # description
)

# Since SMA requires 200 periods above, we will set last to 200
last = 200
# Indicator length buffer
last_buffer = 10
rows = last + last_buffer

histdf = YourHistoricalSource()
has_new_tick = True

while has_new_tick:
    lastdf = histdf.tail(rows).copy() # Get the last 210 rows from histdf
    lastdf.ta.strategy(momo_bands_sma_strategy) # Process the Strategy with lastdf
    histdf.append(lastdf.iloc[-1], ignore_index=True) # Append new TA to histdf
    histdf.to_<Storage>() # Save to your Storage if you require it

    new_tick = YourTickSource() # Request new tick for your Broker
    if new_tick has "ohlcv": # Or whatever satisfies your requirements
        has_new_tick = True
    else:
        has_new_tick = False

Let me know what you find or come up with. Hope this helps!

Don't forget to ⭐ if you find the library useful.

Thanks,
KJ

from pandas-ta.

pbrumblay avatar pbrumblay commented on May 13, 2024 1

Thank you for this discussion. This was something I stumbled on and you folks helped me out. Here's a simple example I'm going to run with: https://gist.github.com/pbrumblay/b8fee2a425be6bfc88fc43ff15fb0199 (not using strategy yet but I will...).

I'm not going to worry about optimizing just yet but if pandas is really copying the whole DF there might be a performance problem in there depending on DF size (as @twopirllc mentioned above).

from pandas-ta.

22Vignesh-M avatar 22Vignesh-M commented on May 13, 2024

Hi @twopirllc,

This is exactly what I wanted, but I'm getting some fluctuations in values for some indicators which depend on the OHLC form the beginning of the data frame.

Example Screenshots:

Computations on full Data Frame:
pandas_ta_computed_full

Computations on Live Ticks:
pandas_ta_computed_tick

Regards,
Vignesh Murugan

from pandas-ta.

twopirllc avatar twopirllc commented on May 13, 2024

Hello @22Vignesh-M,

Thanks for replying and providing more info.

However, can you share some of your code so I can see what is happening? It looks like your MACD is incomplete, it requires a minimum of 26 periods. I would at least look back 30 periods/rows maybe even 50 since you have a small subset of indicators.

Regards,
KJ

from pandas-ta.

twopirllc avatar twopirllc commented on May 13, 2024

Hello @22Vignesh-M,

I don't know what you use for charting, but mplfinance just added Animation Support. Check it out, maybe it will help. 🤷‍♂️

Regards,
KJ

from pandas-ta.

22Vignesh-M avatar 22Vignesh-M commented on May 13, 2024

Hello @twopirllc,

Here is the code that I have tried for live tick calculation.

import pandas as pd
import pandas_ta as ta
from alphaVantageAPI.alphavantage import AlphaVantage

#AlphaVantage
def farm(ticker = 'AAPL', drop=['dividend', 'split_coefficient']):
    AV = AlphaVantage(api_key="YOUR API KEY", premium=False, clean=True, output_size='full')
    df = AV.data(symbol=ticker, function='D')
    df.set_index(pd.DatetimeIndex(df['date']), inplace=True) if not df.ta.datetime_ordered else None
    df.drop(['dividend', 'split_coefficient'], axis=1, inplace=True) if 'dividend' in df.columns and 'split_coefficient' in df.columns else None
    df.name = ticker
    return df

#Historical Data
ticker = 'GOOGL'
df = farm(ticker)

#Live Tick Iterator
def live_df_simulation(df, start, n, end):
    while True:
        if start+n < end:
            yield df.iloc[start: start+n]
            start += 1
        else:
            yield pd.DataFrame()

#Strategy Definition
strategy = [
    {"kind":"sma"},
    {"kind":"bbands"},
    {"kind":"macd"},
    {"kind":"rsi"},
    {"kind":"supertrend"},
    {"kind":"mfi"}
]

strategy = ta.Strategy('live_indicators', strategy, 'technical indicators live calculation')
df_hist = df.copy(deep=True)

#Calclating Indicators
df_hist.ta.strategy(strategy)

#Live Tick Simulation
start = 300 #Starts from index 300
end = len(df)-105
n = 30 

#df_slice iterates through 300-399, 301-400, 302-401 .... 3918-4017
df_slice = live_df_simulation(df, start, n, end)

#Live DataFrame
df_live = pd.DataFrame()
tmp = next(df_slice)

#Live Calculation
while tmp.empty != True:
    tmp.ta.strategy(strategy)
    df_live = df_live.append(tmp.iloc[-1])
    tmp = next(df_slice)
    

df_hist = df_hist[['open', 'high', 'low', 'close']+['SUPERT_7_3.0', 'RSI_14', 'SMA_10', 'MFI_14', 'MACD_12_26_9']]
df_live = df_live[['SUPERT_7_3.0', 'RSI_14', 'SMA_10', 'MFI_14', 'MACD_12_26_9']]

#Comparison b/w single_run calculation and live_tick calculation
comparision = df_hist.merge(df_live, how='inner', left_index=True, right_index=True)

#if we increase the number of rows n=100 or 200, there is a change in values only after the decimal point

And lots of thanks for suggesting a charting solution,
I'm currently using bokeh for live streaming and charting.

Thanks & Regards,
Vignesh Murugan

from pandas-ta.

twopirllc avatar twopirllc commented on May 13, 2024

Hello @22Vignesh-M,

Thanks for getting back to me with some example code to help. I'll take a look as soon as I can.

I'm current using bokeh live streaming and charting.

Excellent choice.

Thanks,
KJ

from pandas-ta.

twopirllc avatar twopirllc commented on May 13, 2024

@pbrumblay,

Thanks for using Pandas TA! Glad it helped you out.

Thanks,
KJ

from pandas-ta.

sasha-id avatar sasha-id commented on May 13, 2024

I guess most efficient way to achive streaming processing is to utilize:

from pandas-ta.

rahulmr avatar rahulmr commented on May 13, 2024

@twopirllc @22Vignesh-M This is extremely cool stuff. It will be great guys if you can create example for alpaca paper trading account. It will be of great use, once you code it for alpaca others can code similarly for other brokers. If I happen to do this myself, I will definitely post it here. Also, the watchlist example but for live market with alpaca example. Alpaca Paper trading is available for free and even oanda demo account is free.

from pandas-ta.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.