mariostoev / finviz Goto Github PK

The FinViz site seems to indicate that "real-time" quotes are in fact possible if you buy their premium subscription. Your description says that theres a 15 minute delay without any qualification.

Can you confirm that with a premium subscription, there is still a 15 minute delay?

To reproduce error:

1. pip install finviz
2. run following code

from finviz.screener import Screener

filters = ['exch_nasd', 'idx_sp500']  # Shows companies in NASDAQ which are in the S&P500
stock_list = Screener(filters=filters, order='price')  # Get the first 50 results sorted by price ascending

# Export the screener results to .csv 
stock_list.to_csv()

gives following error:

   'cssselect does not seem to be installed. '
ImportError: cssselect does not seem to be installed. See http://packages.python.org/cssselect/

https://finviz.com/api/quote.ashx?rev=1597869829851&ticker=CL&instrument=futures&timeframe=d&type=new

`---------------------------------------------------------------------------
HTTPError Traceback (most recent call last)
in
----> 1 finviz.get_insider('АAPL')

~/anaconda3/envs/arcticw/lib/python3.7/site-packages/finviz/main_func.py in get_insider(ticker)
34 """
35
---> 36 page_parsed, _ = http_request_get(url=STOCK_URL, payload={'t': ticker}, parse=True)
37 table = page_parsed.cssselect('table[class="body-table"]')[0]
38 headers = table[0].xpath('td//text()')

~/anaconda3/envs/arcticw/lib/python3.7/site-packages/finviz/helper_functions/request_functions.py in http_request_get(url, session, payload, parse)
19 content = requests.get(url, params=payload, verify=False)
20
---> 21 content.raise_for_status() # Raise HTTPError for bad requests (4xx or 5xx)
22
23 if parse:

~/anaconda3/envs/arcticw/lib/python3.7/site-packages/requests/models.py in raise_for_status(self)
938
939 if http_error_msg:
--> 940 raise HTTPError(http_error_msg, response=self)
941
942 def close(self):

HTTPError: 404 Client Error: Not Found for url: https://finviz.com/quote.ashx?t=%D0%90APL`

The API has been auto-blocked by finviz to stop the requests.

I'm aware of this issue and I'm working on quick-fixing it. When I'm done I'll update the project on GitHub, as well as, PyPi.

Wondering if anyone else has had this problem lately or if it's something I'm missing?

raise client_error(req.connection_key, exc) from exc
aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host finviz.com:443 ssl:None [Connect call failed ('IP_ADDRESS_HERE', 443)]

The error happens when the Scraper() function is used. Thoughts?

To start with, this is the filter I apply when screening for stocks: filters = ['sh_curvol_o300', 'ta_highlow52w_b0to10h', 'ind_stocksonly']. When I print the length of the Screener object, the result is 515 (as I write this). However, when I try to iterate through the object and print the ticker symbols only, the iteration stops before it's completed. As a result, I can see only first tickers.
Any idea why is this happening? Or even better: is there a working solution?

Thank you very much.

First for loop 'exch_nyse' works - but 'exch-nasd' does not? Can anyone else connect using 'exch-nasd' as a filter?

Exception has occurred: ClientConnectorError
Cannot connect to host finviz.com:443 ssl:default [The semaphore timeout period has expired]

def getTickers(price,marketcap):
df = pd.DataFrame()
for exchange in ['exch_nyse','exch_nasd']:
filters = [price,marketcap,exchange]
stock_list = pd.DataFrame(Screener(filters=filters).data)

First of all congrats on the great work you are doing. I wanted to ask you do you plan to support FinViz groups API? What I mean is shown here:

https://finviz.com/groups.ashx?g=sector&v=120&o=pe

Thanks a keep up!

As column names can not start with a digit, the function "export_to_db" in helper_functions.save_data does not convert column names to valid names and thus columns like "50DHigh" are invalid and lines#52 raises a sqlite3.OperationalError

While Finviz Elite filter provide pre-market data starting 8AM EST, however the services return 0 results set , until somewhere around 10:30AM EST?

Hi,

Thank you for sharing this tool, very useful!
I installed finviz with Python 3.7 today, and ran the sample code for screener:

from finviz.screener import Screener

filters = ['exch_nasd', 'idx_sp500']  # Shows companies in NASDAQ which are in the S&P500
# Get the first 50 results sorted by price ascending
stock_list = Screener(filters=filters, order='price')

I get the following error. Could you please help? Thanks!

  File "M:/ Analytics/z/Data/Equity/finviz_scrape.py", line 12, in <module>
    stock_list = Screener(filters=filters, order='price')

  File "C:\Users\\AppData\Local\Continuum\anaconda3\lib\site-packages\finviz\screener.py", line 65, in __init__
    self.data = self.__search_screener()

  File "C:\Users\\AppData\Local\Continuum\anaconda3\lib\site-packages\finviz\screener.py", line 222, in __search_screener
    pages_data = async_connector.run_connector()

  File "C:\Users\\AppData\Local\Continuum\anaconda3\lib\site-packages\finviz\helper_functions\request_functions.py", line 61, in run_connector
    loop.run_until_complete(self.__async_scraper())

  File "C:\Users\\AppData\Local\Continuum\anaconda3\lib\asyncio\base_events.py", line 560, in run_until_complete
    self.run_forever()

  File "C:\Users\\AppData\Local\Continuum\anaconda3\lib\asyncio\base_events.py", line 515, in run_forever
    raise RuntimeError('This event loop is already running')

RuntimeError: This event loop is already running

Hi, I have been using your finviz module all along, but recently it stopped working.

Here is the error I get. Do you have any suggestions to help me fix this?
requests.exceptions.HTTPError: 403 Client Error: Forbidden: Access is denied. for url:

Thanks!

I think the v in the screener url payload might stand for View instead of Table. When v=210, the data is not presented as a table of alphanumeric data but charts. Also, the last number in v=210 (0) is to choose whether the filter options are shown (1), or the custom settings (2). Note the custom settings seems to only work when we choose the custom table like this: v=152.

In custom, it is possible to change the columns in the table.

Here is an example of url with custom settings:
https://www.finviz.com/screener.ashx?v=150&c=0,1,2,3,4,5,6,7,8,24,65,66
where v=15[0-2] and c is the custom settings payload parameter character.

Hi, do you have some historical saves of screeners? Thx!

Would be great to add this api feature.
Thanks!

requests.exceptions.HTTPError: 403 Client Error: Forbidden: Access is denied. for url: https://finviz.com/screener.ashx?v=Overview&t=&f=sh_price_u50%2Ccap_largeunder%2Cexch_nyse&o=&s=&c=

get_all_news() returns empty list. How i can fix it? I need date and time of publication.

I need help with implementing a feature, where the data from the scraper would be saved into a SQL table. If you're willing to help, you can type your own code into a new function inside of the finviz-api/finviz/save_data.py file.

Love the tool!

Noticed that when I use the function:

stock_list.get_charts(period='d', chart_type='c', size='l', ta=True)

...I get a bunch of empty/nothing .jpg files in the newly created charts folder? I've tried using 'm' and both True and False for ta, still the same. I also tried changing the .jpg to .jpeg (just in case) in the scraper_functions.py file too. Don't know if it matters, but I'm running Windows 10, Python 3.6. Any ideas on things to try? Thanks!

Can someone please post a list of filter strings for the screener? What's the one for the DOW for example? Where do I find them?

Thanks in advance.

Attempting to run finviz screener on Python 3.6.5 and tornado 4.5.3.

filters = ['exch_nasd','idx_sp500']
screener = finviz.screener.Screener(filters=filters, order= 'price')

Returns error: RuntimeError: There is no current event loop in thread 'ThreadPoolExecutor-0_0'.

Thoughts?

I tried to make discord bot using finviz library and that what i got for Screener
Command raised an exception: AssertionError:

Hi can someone give me full get_charts example i tried it but get errors 'str object has no attribute data'

In the latest revision, get all news requires a ticker. If you go back to the version 1.2.X you can copy the get_all_news function from the main_func, then update finviz, then paste it in the new main_func to get it to work properly.

Adding cryptocurrency support would be fairly simple - you'd have to create separate methods in finviz.main_func to contain the code to:

1. Scrape the table data from crypto performance webpage using finviz.helper_functions.scraper_functions.get_table.
2. Download tables from crypto charts webpage (there are several timeframes) by getting all the the table images' URLs and passing them to finviz.helper_functions.scraper_functions.download_chart_image. You should be able to download the bigger chart from the individual cryptocurrency pages, too.
3. Get price for individual cryptos directly from their exclusive page

Hello, I started using finviz module to load stock price data, and version is 1.3.1.
My python version is 3.7.6.
When testing with terminal innate python, finviz.get_stock worked, but when I use in my work code in vscode, such attribute error keep shows and I have no idea what to do.
Please help me fixing this problem.
This is my code.

import finviz

print("Enter tickers in uppercase letters (A B C D ...)")
stock_list = input().split()

for stock in stock_list:
    data = finviz.get_stock(stock)
    data = data['price']
    print(data)

This error appear when I try to get charts with get_charts(). I just upgrade to actual version, Before I had no such error

I've installed via pip and git and receive the same error when trying to use the Screener function

from finviz.screener import Screener


filters = ['f=cap_microunder','sh_curvol_o200','sh_float_u100','sh_price_u2','sh_relvol_o3','ta_perf_dup','ta_sma20_pa','ft=4']
stock_list = Screener(filters=filters, table='Performance', order='price')

print(stock_list)

ModuleNotFoundError: No module named 'finviz.screener'; 'finviz' is not a package

Bug report:

# python3 example.py
Traceback (most recent call last):
  File "example.py", line 3, in <module>
    from finviz.helper_functions.save_data import export_to_db, export_to_csv
  File "/mnt/c/Users/consultant/Documents/software/finviz/finviz/__init__.py", line 1, in <module>
    from finviz.screener import Screener
  File "/mnt/c/Users/consultant/Documents/software/finviz/finviz/screener.py", line 1, in <module>
    from finviz.helper_functions.request_functions import Connector, http_request_get
  File "/mnt/c/Users/consultant/Documents/software/finviz/finviz/helper_functions/request_functions.py", line 3, in <module>
    from user_agent import generate_user_agent
ModuleNotFoundError: No module named 'user_agent'

filters2 = ['exch_nasd']

screener = Screener(filters=filters2, table='Performance', order='price')
stocklist = []
for stock in screener:
    stocklist.append(stock['Ticker'])
print(len(stocklist))

I run the above code and it returns 280 stocks, while on finviz with this https://finviz.com/screener.ashx?v=111&f=exch_nasd&ft=4 it shows 3205 stocks.

Am I missing something?

Still getting used to GitHub, but I used this and there were a lot of errors, so I fixed them. Here's my code. Sorry for my ignorance, I wanted to provide the information, but I don't know the proper way to present it.

`from urllib.request import urlopen
from lxml import html
from table_format import table_format
import asyncio
import aiohttp
import urllib
import requests
import os
import csv

TABLE = {
'Overview': ['110', 0],
'Valuation': ['120', 1],
'Ownership': ['130', 2],
'Performance': ['140', 3],
'Custom': ['150', 4],
'Financial': ['160', 5],
'Technical': ['170', 7]
}

def parse(url):

"""
Returns the html content of a web page.
"""

request = urllib.request.Request(url)
htmll = urlopen(request).read()
pageContent = html.fromstring(htmll)

return pageContent

def httprequest(payload):

"""
Parses the url by given parameters and sends
request to the website.
"""

payload['user-agent'] = '''Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36
                        (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36'''
request = requests.get('https://finviz.com/screener.ashx', params=payload)

return request.url

def page(url, quantity):

"""
Returns a list with the URL's of the all pages and
the headers of the table.
"""

pageContent = parse(url)

try:
    total_pages = int([i.text.split('/')[1] for i in pageContent.cssselect('option[value="1"]')][0])
except AttributeError:  # No results found
    return None

URLS = []
# Using basic level arithmetic sequence to fetch urls
for i in range(1, total_pages + 1):
    sequence = 1 + (i - 1) * 20
    if sequence - 20 <= quantity < sequence:  # PROBLEM WITH QUANTITY = 1
        break
    else:
        URLS.append(url + '&r={}'.format(str(sequence)))

return URLS

def parse_row(row):

"""
Scrapes each element in a row
"""

row_data = []

for tags in row:
    if tags.text is not None:
        row_data.append(tags.text)
    else:
        row_data.append([span.text for span in tags.cssselect('span')][0])

return row_data

def get_data(htmll, headers, quantity):

"""
Iterates over the screener's table and saves each
row's data into a dictionary.
"""

datasets = []
pageContent = html.fromstring(htmll) #htmll replaced content
all_rows = [i.cssselect('a') for i in pageContent.cssselect('tr[valign="top"]')[1:]]

for row in all_rows:
    if int(row[0].text) is quantity:
        values = dict(zip(table_format[0], parse_row(row)))
        datasets.append(values)
        break
    else:
        values = dict(zip(table_format[0], parse_row(row)))
        datasets.append(values)

return datasets

def screener(tickers=[], filters=[], order='',
signal='', todir=os.getcwd(), table='Overview',
export='sql', quantity=20):

"""
With given specific filters, screener() scrapes table data
from Finviz and creates a file containing all of
the information.
"""

payload = {
    'v': TABLE[table][0],  # Overview search
    't': ','.join(tickers),
    'f': ','.join(filters),
    'o': order,
    's': signal
}

headers = [i for i in table_format[TABLE[table][1]]]
URLS = page(httprequest(payload), quantity)
DATA = []

async def get_page(url, client):
    """ #original code:
    session = aiohttp.ClientSession()
    response = yield from(aiohttp.request('GET', url, compress=True))
    """
    async with aiohttp.request('GET', url, compress=True) as response:
        assert response.status == 200
        return await response.text()

async def print_data(url, headers, quantity):
    async with aiohttp.ClientSession() as client:
        page = await get_page(url, client)
        data = get_data(page, headers, quantity)
        DATA.append(data)

loop = asyncio.get_event_loop()

if URLS is not None:
    loop.run_until_complete(asyncio.wait([print_data(i, headers, quantity) for i in URLS]))
else:
    print("URLS is None")

DATA = DATA[0]
data_keys = DATA[0].keys()
with open('screener_results.csv', 'w', newline='') as output_file:
    dict_writer = csv.DictWriter(output_file, data_keys)
    dict_writer.writeheader()
    dict_writer.writerows(DATA)

return DATA`

Having the same issue with the Portfolio function:
using finviz 1.3.4

Traceback (most recent call last):
File "c:\Users\parke\OneDrive\Desktop\StockTrading in Python\v.1.0\finviz porfolio.py", line 37, in
portfolio = Portfolio(USERNAME,PASSWORD)
File "C:\Users\parke\AppData\Local\Programs\Python\Python38\lib\site-packages\finviz\portfolio.py", line 41, in init
auth_response.raise_for_status()
File "C:\Users\parke\AppData\Local\Programs\Python\Python38\lib\site-packages\requests\models.py", line 941, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden: Access is denied. for url: https://finviz.com/login_submit.ashx

Originally posted by @parker-moore in #45 (comment)

Hi There,

How can I use screener with Finvize Elite account (similar to portfolio) to receive real-time data instead of delayed data?

Thanks,

Nima

Does this unofficial API contain any methods that allow for screening stocks by industry? If not that would be great.

Hi, it's not an issue, I just want to add some feature.

I am not a professional coder, but I offer to you adding the function of downloading charts from BigCharts.com
I just add new def get_bigcharts and a little fix in scraper_functions.py. You can use it to download charts from Screener results

All code is here,
Gist

The function doesn't work for price_from to price_to

import finviz
finviz.get_analyst_price_targets('aapl')
[{'date': 'Jun-19-20', 'category': 'Reiterated', 'analyst': 'Jefferies', 'rating': 'Buy', 'price_from': '2020-06-19', 'price_to': 'Reiterated'}, {'date': 'Jun-17-20', 'category': 'Reiterated', 'analyst': 'RBC Capital Mkts', 'rating': 'Outperform', 'price_from': '2020-06-17', 'price_to': 'Reiterated'}, {'date': 'Jun-16-20', 'category': 'Reiterated', 'analyst': 'Wedbush', 'rating': 'Outperform', 'price_from': '2020-06-16', 'price_to': 'Reiterated'}, {'date': 'Jun-16-20', 'category': 'Reiterated', 'analyst': 'Citigroup', 'rating': 'Buy', 'price_from': '2020-06-16', 'price_to': 'Reiterated'}, {'date': 'Jun-11-20', 'category': 'Upgrade', 'analyst': 'HSBC Securities', 'rating': 'Reduce → Hold', 'price_from': '2020-06-11', 'price_to': 'Upgrade'}]

Since 9/9/2020, there is an issue in Screener that it returns much fewer stocks than usual. In particular, it started return ~100 stocks. Before that date it used to return ~7500 stocks.

Here is the code that I'm running:
stock_list = Screener(table='Performance', order='price')

Here is a screenshot of number of elements(stocks) returned by this code sorted by date:

Thanks

What might be causing this error?

Traceback (most recent call last):
  File "./robin_stocks.API.py", line 163, in <module>
    result = rhLib.apiRun(args.method, args.parameters)
  File "/home/master/www/robinbot/rhLib.py", line 232, in apiRun
    result= eval(method + parameters)  # <class 'dict'>
  File "<string>", line 1, in <module>
  File "/home/master/www/robinbot/rhLib.py", line 457, in fullQuote
    import finviz
  File "/usr/local/lib/python3.5/dist-packages/finviz/__init__.py", line 1, in <module>
    from finviz.screener import Screener
  File "/usr/local/lib/python3.5/dist-packages/finviz/screener.py", line 113
    values = f'tickers: {tuple(self._tickers)}\n' \
                                                ^
SyntaxError: invalid syntax

My Python script merely uses finvizData = finviz.get_stock(symbol) , but the Syntax Error happens right at import finviz wherever it's placed.

I did
pip install finviz
under all versions installed locally or globally (Python 2.7.12 built-in to Ubuntu 16.04.6 LTS, Python 3.5 and Python 3.6).

Would be nice if this worked in Google colab:

https://colab.research.google.com/drive/1vLn2Vf2-t0TsHA-rOoJshNjcYUgSmkMh?usp=sharing

Hi,

I'm currently facing some issues running the below screener class code using Python Flask:

`from finviz.screener import Screener

filters = ['exch_nyse', 'idx_dji'] # Shows companies in NASDAQ which are in the S&P500
stock_list = Screener(filters=filters, table='Valuation', order='price') # Get the performance table and sort it by price ascending
`

And i get the following error. Could anyone help please? Thanks!!

Traceback (most recent call last):
File "/Users/asherchew/Desktop/Projects/hello_flask/env/lib/python3.8/site-packages/flask/app.py", line 2464, in call
return self.wsgi_app(environ, start_response)
File "/Users/asherchew/Desktop/Projects/hello_flask/env/lib/python3.8/site-packages/flask/app.py", line 2450, in wsgi_app
response = self.handle_exception(e)
File "/Users/asherchew/Desktop/Projects/hello_flask/env/lib/python3.8/site-packages/flask/app.py", line 1867, in handle_exception
reraise(exc_type, exc_value, tb)
File "/Users/asherchew/Desktop/Projects/hello_flask/env/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/Users/asherchew/Desktop/Projects/hello_flask/env/lib/python3.8/site-packages/flask/app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "/Users/asherchew/Desktop/Projects/hello_flask/env/lib/python3.8/site-packages/flask/app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/Users/asherchew/Desktop/Projects/hello_flask/env/lib/python3.8/site-packages/flask/app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/Users/asherchew/Desktop/Projects/hello_flask/env/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/Users/asherchew/Desktop/Projects/hello_flask/env/lib/python3.8/site-packages/flask/app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "/Users/asherchew/Desktop/Projects/hello_flask/env/lib/python3.8/site-packages/flask/app.py", line 1936, in dispatch_request
return self.view_functionsrule.endpoint
File "/Users/asherchew/Desktop/Projects/hello_flask/playground/test_8.py", line 15, in test_finviz
stock_list = Screener(filters=filters, table='Valuation', order='price') # Get the performance table and sort it by price ascending
File "/Users/asherchew/Desktop/Projects/hello_flask/env/lib/python3.8/site-packages/finviz/screener.py", line 120, in init
self.data = self.__search_screener()
File "/Users/asherchew/Desktop/Projects/hello_flask/env/lib/python3.8/site-packages/finviz/screener.py", line 325, in __search_screener
pages_data = async_connector.run_connector()
File "/Users/asherchew/Desktop/Projects/hello_flask/env/lib/python3.8/site-packages/finviz/helper_functions/request_functions.py", line 78, in run_connector
loop = asyncio.get_event_loop()
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/asyncio/events.py", line 639, in get_event_loop
raise RuntimeError('There is no current event loop in thread %r.'
RuntimeError: There is no current event loop in thread 'Thread-3'.

I was just testing a few of the functions in my iPython notebook and got this:

AttributeError: module 'finviz' has no attribute 'get_analyst_price_targets'

Very strange - I copied the example in the tutorial and all of the other functions seem to work fine.

Any idea? I'm using the latest version - pulled it yesterday.

Before, I could pull data for the entire universe of tickers (about 7500 tickers) using the following code:

from finviz.screener import Screener
stock_list = Screener(filters=[], table='Ownership')
print(len(stock_list[::])) # this used to print 7500, now it prints 680 only

However since 1-2 days ago, I'm only getting a subset of tickers back. I checked which tickers are coming back, and there's certain letters of the alpha bet excluded (e.g. any ticker starting with T).

If we save it to CSV, the same thing.. Only ~700 rows get saved:

stock_list.to_csv("stock.csv") # only 680 rows, any stocks starting with T are missing.

Even when getting only the sp500, it returns only 100 tickers:

stock_list = Screener(filters=['idx_sp500'], table='Ownership')
len(stock_list[::]) # returns 100

Thanks.

Hey bro! Really nice work with your code!
You should add requirements.txt

pip install cssselect
pip install requests
pip install aiohttp
pip install lxml
pip install user-agent

Hello,

I'd like to start adding custom table settings like Shares Float, Outstanding Shares, etc. to the custom table in the FinViz module. I've tried doing a few different things like stock_list.add(Shares_Float = True), but so far I haven't found anything that works. Is there a better way to do this?

It can be quicker to type when we want to download the screener data using a single line shell command