Code Monkey home page Code Monkey logo

autodd_rev2's Introduction

About AutoDD Rev 2

AutoDD = Automatically does the "due diligence" for you. If you want to know what stocks people are talking about on reddit, this little program might help you.

Original author - Fufu Fang https://github.com/fangfufu

Rev 2 Author - Steven Zhu https://github.com/kaito1410 gobbedy https://github.com/gobbedy Napo2k https://github.com/Napo2k

The original AutoDD produced a simple table of the stock ticker and the number of threads talking about the ticker.

Version 2 of AutoDD adds some options and features that the original did not have.

- ability to display a change in results (ie, an increase or decrease of score from the previous day to today)

- ability to pull additional stock information from yahoo finance (such as open and close price, volume, average volume, etc)

- ability to pull results from multiple subreddits (pennystocks, RobinHoodPennyStocks, stocks, Daytrading, etc)

- added score system to calculate a score for each ticker based on the number of occurrences, DD, Catalyst, or technical analysis flair, and number of upvotes

- Can be run with a windows scheduler to run the program at a set time everyday

Requirements

Python (tested on python 3.8.1) https://www.python.org/downloads/release/python-381/

Pip - python get-pip.py https://phoenixnap.com/kb/install-pip-windows#:~:text=PIP%20is%20automatically%20installed%20with,9%2B%20and%20Python%203.4%2B.

psaw - pip install psaw https://pypi.org/project/psaw/

praw - pip install praw https://pypi.org/project/praw/

tabulate - pip install tabulate https://pypi.org/project/tabulate/

pandas - pip install pandas https://pypi.org/project/pandas/

The requirements can be installed by running install_requirements.bat / install_requirements.sh

Running

Watch the setup video here https://www.youtube.com/watch?v=YwfwJYjnBFU

To set up the dependencies on Windows 10

1. Install python 3.8 and make sure you add python to the path variable during installation
2. Run install_requirements.bat, it should open a terminal and install the dependencies
3. If all dependencies are installed successfully, run run_auto_dd.bat
4. After 1-2 minutes, you should find a table_records.txt file in the AutoDD folder
5. To generate a new table, simply run run_auto_dd.bat again, it will append a new table to the table_records.txt file

To set up the dependencies on Linux/MacOSX

1. Install python 3.8 and Pip3 https://medium.com/swlh/installing-python-and-pip-on-mac-72b7639a58
2. Run install_requirements.sh, it should open a terminal and install the dependencies
3. If all dependencies are installed successfully, run run_auto_dd.sh
4. After 1-2 minutes, you should find a table_records.txt file in the AutoDD folder
5. To generate a new table, simply run run_auto_dd.bat again, it will append a new table to the table_records.txt file

For Advanced Users:

1. Simply open the terminal (powershell or command prompt on windows, terminal on linux/MacOSX) and navigate to the AutoDD folder, then type:
	
	python main.py -h
	
2. Follow the help document and set up the optional parameters as you'd like. 

Columns Explained

Code - Ticker Name

24H Total - Score of the ticker from the last XX hours. By default, this column shows the score from the last 24 hours. If you change the interval for example --interval 48, then this show score of the tickers for the last 48 hours

Recent - Score of the ticker from the recent half of the interval. By default, Recent shows the score from the last 12 hours. If you change the interval for example --interval 48, then recent show data from the last 24 hours

Prev - Score of the ticker from the last X - 2X hour period. By default, Prev shows the score from the last 24-48 hour period. If you change the interval for example --interval 48, then recent show data from the 48-96 hour period

Change - Shows increase or decrease in the score of the ticker. A positive number means that there was more discussion recently than the previous interval period.

Rockets - Number of Rocket Emojis on the submission

Price - Current stock price

1DayChange% - Percentage change in todays price compared to yesterday

50DayChange% - Percentage change in todays price compared to the last 50 day average

Vol/3MonthAvg - Ratio of the most recent trading days' volumn to the average in the last 3 month

Float Shares - Number of tradable shares of the ticker

Short/Float% - Amount of short shares / avaliable float for the ticker in percentage

Industry - Industry of the company if available

Example output

Default Output:

Alt text

Allsub Option Output:

Alt text

Yahoo Option Output:

Alt text

Options

In terminal, type:

python main.py -h

This will produce the following help text:

usage: main.py [-h] [--interval [INTERVAL]] [--min [MIN]] [--adv] [--sub [SUB]] [--sort [SORT]] [--filename [FILENAME]]

AutoDD Optional Parameters

optional arguments:
-h, --help            show this help message and exit
--interval [INTERVAL]
						Choose a time interval in hours to filter the results, default is 24 hours
--sub [SUB]           Choose a different subreddit to search for tickers in, default is pennystocks
--min [MIN]           Filter out results that have less than the min score, default is 10
--minprice [MINPRICE]
						Filter out results less than the min price set, default is 0
--maxprice [MAXPRICE]
						Filter out results more than the max price set, default is 9999999
--advanced            Using this parameter shows advanced yahoo finance information on the ticker
--sort [SORT]         Sort the results table by descending order of score, 1 = sort by total score, 2 = sort by recent score, 3 = sort by previous score, 4 = sort by change in score, 5 = sort by # of rocket emojis
--allsub              Using this parameter searchs from one subreddit only, default subreddit is r/pennystocks.
--psaw                Using this parameter selects psaw (push-shift) as the reddit scraper over praw (reddit-api)
--no-threads          Disable multi-tasking (enabled by default). Multi-tasking speeds up downloading of data.
--csv                 Using this parameter produces a table_records.csv file, rather than a .txt file
--filename [FILENAME]
						Change the file name from table_records to whatever you wish

Interval (Time interval)

1. Choose a time interval N in hours to filter the results, default is 24 hours

2. The score in the Total column shows the score for each ticker in the last N hours

3. The score in the Recent column shows the score for each ticker in the last N/2 hours, default to 12h

4. The score in the Prev column shows the score for each ticker in the last N/2 - N hours, default is 12h - 24h

5. The score in the other subreddit columns shows the score for each ticker in the last 24 hours

Min (Minimum score)

1. Filter out results that have less than the min score in the Title column, default is 10

Sub (Subreddit Selection)

1. Choose a different subreddit to search for tickers in, by default, it searches both pennystocks and robinhoodpennystocks

3. You can choose to run this on any subreddit you want, there are no limits. For example --sub=WallStreetBets

MaxPrice (Maximum Price Limit)

1. Filter out tickers that have a current price of greater than the set limit, default is 9999999 (does not filter out anything)

MinPrice (Minimum score)

1. Filter out tickers that have a current price of less than the set limit, default is 0 (does not filter out anything)

Advanced (Yahoo Finance Key Metrics)

1. Using this parameter shows yahoo finance information, running yahoo mode is slower

2. This options shows additional yahoo information on the ticker, such as open price, day low, day high, forward PE, beta, volume, etc.

Sort

1. Sort the results by descending order of score, by default the table shows the highest total score first

2.  pass in values 1, 2, 3, or 4

3. 1 = sort by total score, 2 = sort by recent score, 3 = sort by previous score, 4 = sort by change in score, 5 = sort by change in # of rocket emojis

Allsub (All Subreddit toggle)

1. Using this parameter shows scores on the other subreddits such as RobinHoodPennyStocks, Stocks, WallStreetBets, etc

Psaw (Push-Shift toggle)

1. Using this parameter chooses push-shift to retieve subreddit data

No-threads (Multi-threading Off toggle)

1. Disable multi-tasking (enabled by default). Multi-tasking speeds up downloading of data.

Csv

1. Outputs table_records.csv file

Filename

1. choose a different filename, this programs saves the table results to table_records.txt in the same folder as the AutoDD.py program

Troubleshoot

ModuleNotFoundError: No module named 'something'

- This means the dependency was not installed correctly, try running: 

	pip install 'something'
	
- Another possibility is that python is using the wrong version, try:
	
	python3 main.py

AutoDD.py not found

- This means the terminal can't find the python script, either navigate to the AutoDD folder using terminal or

	python path-to-autodd-folder/AutoDD.py
	ie. python C:/AutoDD_Folder/AutoDD.py

Scheduler (Tested on Windows)

  1. Create a .bat file and type in:

    python path-to-AutoDD\AutoDD.py --whatever options you want to configure

  2. Open windows Task Scheduler

  3. Create a basic task

  4. Fill in the name and description

  5. Choose a trigger that works for you, mine is every day

  6. Choose "Start a program" and put in the path to your .bat file

    • ie. "C:\AutoDD-folder\run_auto_dd.bat"
  7. That's it, just check table_records.txt or the file name that you've selected and it will have the table ready

Developers/Advanced Users

I'm a C++ main, so excuse my python code/inefficencies with handling tables and lists in python.

I've put a couple global variables for some advanced users to allow for easy modifications:

# dictionary of possible subreddits to search in with their respective table column name
subreddit_dict = {'pennystocks' : 'pnystks',
				  'RobinHoodPennyStocks' : 'RHPnnyStck',
				  'Daytrading' : 'daytrade',
				  'StockMarket' : 'stkmrkt',
				  'stocks' : 'stocks'}

# dictionary of ticker financial information to get from yahoo
financial_measures = {'currentPrice' : 'Price', 'quickRatio': 'QckRatio', 'currentRatio': 'CrntRatio', 'targetMeanPrice': 'trgtmean', 'recommendationKey': 'recommadtn'}

# dictionary of ticker summary information to get from yahoo
summary_measures = {'previousClose' : 'prvCls', 'open': 'open', 'dayLow': 'daylow', 'dayHigh': 'dayhigh', 'payoutRatio': 'pytRatio', 'forwardPE': 'forwardPE', 'beta': 'beta', 'bidSize': 'bidSize', 'askSize': 'askSize', 'volume': 'volume', 'averageVolume': 'avgvolume', 'averageVolume10days': 'avgvlmn10', 'fiftyDayAverage': '50dayavg', 'twoHundredDayAverage': '200dayavg'}


# note: the following scoring system is tuned to calculate a "popularity" score
# feel free to make adjustments to suit your needs

# x base point of for a ticker that appears on a subreddit title or text body that fits the search criteria
base_points = 4

# x bonus points for each flair matching 'DD' or 'Catalyst' of for a ticker that appears on the subreddit
bonus_points = 2

# every x upvotes on the thread counts for 1 point (rounded down)
upvote_factor = 2	

License

AutoDD - Automatically does the "due diligence" for you. 
Copyright (C) 2020  Fufu Fang, kaito1410, Napo2k

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program.  If not, see <https://www.gnu.org/licenses/>.

autodd_rev2's People

Contributors

drsig avatar gobbedy avatar kaito1410 avatar napo2k avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

autodd_rev2's Issues

redundancy?

Am I right that or row[1][1] >= min_val is redundant in the code below?

tbl = [row for row in tbl if row[1][0] >= min_val or row[1][1] >= min_val]

If I'm not mistaken row[1][0] is the total score (prev + current) for the ticker, and row[1][1] is the current score.

I'm removing the redundant part in my version of the code, just making sure I haven't misunderstood something?

Error Code 429 pushshift.io

On first time running, recieve the following:

C:\Users*\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\psaw\PushshiftAPI.py:192: UserWarning: Got non 200 code 429
warnings.warn("Got non 200 code %s" % response.status_code)
C:\Users\Shokkatweej\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\psaw\PushshiftAPI.py:180: UserWarning: Unable to connect to pushshift.io. Retrying after backoff.
warnings.warn("Unable to connect to pushshift.io. Retrying after backoff.")

And then stalls at 'Getting quick stats...'

error if no selftext

Me again :)

I just got this error:

Traceback (most recent call last):
  File "C:\Users\guillaume\PycharmProjects\AutoDD_Rev2\main.py", line 101, in <module>
    main()
  File "C:\Users\guillaume\PycharmProjects\AutoDD_Rev2\main.py", line 65, in main
    prev_tbl, prev_rockets = get_freq_list(results_from_api[1])
  File "C:\Users\guillaume\PycharmProjects\AutoDD_Rev2\AutoDD.py", line 171, in get_freq_list
    rocket_tickers = selftext_extracted.union(title_extracted)
UnboundLocalError: local variable 'selftext_extracted' referenced before assignment

I assume this happens when there is no selftext. Easy fix is to add the line pointed to below

        # search the text body for the ticker/tickers
        selftext_extracted = set()   # <------------- INSERT
        if hasattr(i, 'selftext'):
            selftext = ' ' + i.selftext + ' '
            selftext_extracted = set(re.findall(pattern, selftext))

Division by zero exception found

Hi,

found this exception by running script today(13:00 CET, 2020/1/26 on Win10).

Getting submissions...
Searching for tickers...
Populating results...
Counting rockets...
Getting quick stats...
Traceback (most recent call last):
  File "main.py", line 101, in <module>
    main()
  File "main.py", line 92, in main
    results_tbl = getQuickStats(results_tbl)
  File "C:\Users\Rado\Desktop\AutoDD\AutoDD_Rev2-main\AutoDD.py", line 489, in getQuickStats
    change_50day = ((float(price) - float(avg50day))/float(avg50day))*100
ZeroDivisionError: float division by zero

My temporal "fix" was:

 if price is not None and price != 0:
                if avg50day > 0:
                    change_50day = ((float(price) - float(avg50day))/float(avg50day))*100
                else:
                    change_50day = 0

on line 489 in AutoDD.py

Definition of key_stats_measures is missing

It looks like in the latest code the definition of key_stats_measures dict disappeared which is causing an error when running. I added it back in at lines 64/65 and everything worked as it should.

issue with rocket count?

Hello,

rockets seem to be counted in an odd way. Let's take an example. Suppose a reddit submission has the following title:
"PLEASE BUY AAPL, MSFT, GME NOW"

then for that submission you would have

rocket_dict['PLEASE'] += 6
rocket_dict['BUY'] += 6
rocket_dict['AAPL'] += 6
rocket_dict['MSFT'] += 6
rocket_dict['GME'] += 6
rocket_dict['NOW'] += 6

Shouldn't it be +=1 for each one?

Could you please clarify what rockets count?

PS. I should have finished the refactored code by Monday or so if you're interested. Speedup is in the order of 100x when there are many tickers.

Table Columns Incorrect

python3 main.py --allsub

Maybe I am missing something in the documentation but I don't believe the numbers in specific columns are adding up correctly. For example, the price of ZOM has never been 186. (It looks like price is under Industry???)

Is this a bug?

Screen Shot 2021-01-13 at 3 17 10 PM

fiftyDayAverage not returned by yahoo, causes error

I just tried the code, it's awesome!

Although I just tried to run this command:
main.py --interval 2 --sub wallstreetbets

And got this error output:

Traceback (most recent call last):
  File "C:\Users\guillaume\PycharmProjects\AutoDD_Rev2\main.py", line 101, in <module>
    main()
  File "C:\Users\guillaume\PycharmProjects\AutoDD_Rev2\main.py", line 92, in main
    results_tbl = getQuickStats(results_tbl)
  File "C:\Users\guillaume\PycharmProjects\AutoDD_Rev2\AutoDD.py", line 491, in getQuickStats
    if avg50day > 0:
TypeError: '>' not supported between instances of 'NoneType' and 'int'

After some debugging I was able to narrow it down to the ticker "WIN" for which yahoo does not return 'fiftyDayAverage'. I wrote a little standalone to isolate the error:

from yahooquery import Ticker
ticker=Ticker('WIN')
data=ticker.summary_detail.get('WIN')
if data.get('fiftyDayAverage') is None:
    print("Houston...")

I doubt "WIN" was meant as a stock ticker in the reddit submission, but regardless, it is a valid ticker, so this error should not occur?

In the meantime I added "WIN" to the list of banned words locally. Works as a temp fix.

Returning Empty Tables

I am not sure what happened. The program was working perfectly until I tried to use it today. Now it just returns nothing. I attempted a new install and it still reproduced the same result. There are no error logs so it's a little difficult to isolate the problem.

Screen Shot 2021-01-27 at 12 24 46 PM

Has Reddit changed something on their end?

Issue running on Ubuntu/macOS

Hey there,

I'm getting the following error trying to run the main.py on both Ubuntu and macOS.

Traceback (most recent call last):
  File "main.py", line 101, in <module>
    main()
  File "main.py", line 98, in main
    print_tbl(results_tbl, args.filename, args.allsub, args.yahoo, args.csv)
  File "/app/auto-dd/AutoDD.py", line 378, in print_tbl
    writer.writerow(row)
UnicodeEncodeError: 'ascii' codec can't encode character '\u2014' in position 87: ordinal not in range(128)

Do you have any idea on what I can do to get it running? It seems to have trouble printing the table after gathering the data. I can get it running on Windows though.

Thanks in advance!

Running with -sub wallstreetbets slow

Hi there,

First of all thank you for your excellent work.

Running main.py with wallstreetbets sub option is incredibly slow. My thought is that because of the huge dataset, but processing one entry a second is not ideal.

Thank you frou your app.

undercounting of symbols?

I just went through your code, I love it and will try my hand at using it tomorrow.

I just found what seems to be an undercounting of tickers:

        # title_extracted is a set, duplicate tickers from the same title counted once only
        for k in title_extracted:

            if k in all_dict:
                all_dict[k] += increment
            else:
                all_dict[k] = increment

        # avoid counting additional point for the tickers found in the text body
        # only search the text body if ticker was not found in the title
        if len(title_extracted) > 0:
                continue

        for m in selftext_extracted:

            if m in all_dict:
                all_dict[m] += increment
            else:
                all_dict[m] = increment

Doesn't the above miss out on instances when a ticker appears in the selftext but not in the title? (eg say the title contains AAPL and the selftext contains MSFT, it would skip MSFT).

Since rocket_tickers is already a set, shouldn't the above be replaced by the following?

        for k in rocket_tickers:

            if k in all_dict:
                all_dict[k] += increment
            else:
                all_dict[k] = increment

Thanks again for sharing this code.

Yahoo 403 Client Error: Forbidden URL

Kaito, you are the patron saint of day-traders and swing-traders!

It seems, Yahoo pushed our nose to the glass:
image

I hope you can resolve it. 🤘🏼

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.