Got some error on this line of the code in Sentiment.py line 788
stream.filter(follow=useridlist, languages=['en'])

Hi.

After a bit of testing I finally managed to get it working... partially.
When running any command I get the " raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x10c6a76a0>: Failed to establish a new connection: [Errno 61] Connection refused. " - error.
I think this might occur cause I use python3 instead of python2.

The full output when running the "python3 sentiment.py -s TSLA -k 'Elon Musk',Musk,Tesla,SpaceX --debug" command is :

   _                     _                 
 _| |_ _           _   _| |_ _     _   _   
|   __| |_ ___ ___| |_|   __|_|___| |_| |_ 
|__   |  _| . |  _| '_|__   | | . |   |  _|
|_   _|_| |___|___|_,_|_   _|_|_  |_|_|_|  
  |_|                   |_|   |___|                
      :) = +$   :( = -$    v0.1-b.12
 https://github.com/shirosaidev/stocksight

2021-01-21 07:22:38,527 [INFO][stocksight] Creating new Elasticsearch index or using existing stocksight
2021-01-21 07:22:38,530 [WARNING][elasticsearch] PUT http://localhost:9200/stocksight [status:N/A request:0.003s]
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 169, in _new_conn
conn = connection.create_connection(
File "/usr/local/lib/python3.9/site-packages/urllib3/util/connection.py", line 96, in create_connection
raise err
File "/usr/local/lib/python3.9/site-packages/urllib3/util/connection.py", line 86, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 61] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/elasticsearch/connection/http_urllib3.py", line 115, in perform_request
response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, **kw)
File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 755, in urlopen
retries = retries.increment(
File "/usr/local/lib/python3.9/site-packages/urllib3/util/retry.py", line 506, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/usr/local/lib/python3.9/site-packages/urllib3/packages/six.py", line 735, in reraise
raise value
File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
httplib_response = self._make_request(
File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 394, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 234, in request
super(HTTPConnection, self).request(method, url, body=body, headers=headers)
File "/usr/local/Cellar/[email protected]/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1255, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/local/Cellar/[email protected]/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1301, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/local/Cellar/[email protected]/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1250, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/local/Cellar/[email protected]/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1010, in _send_output
self.send(msg)
File "/usr/local/Cellar/[email protected]/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 950, in send
self.connect()
File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 200, in connect
conn = self._new_conn()
File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 181, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x1053adaf0>: Failed to establish a new connection: [Errno 61] Connection refused
2021-01-21 07:22:38,532 [DEBUG][elasticsearch] > {"mappings": {"tweet": {"properties": {"author": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}, "location": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}, "language": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}, "friends": {"type": "long"}, "followers": {"type": "long"}, "statuses": {"type": "long"}, "date": {"type": "date"}, "message": {"type": "string", "fields": {"english": {"type": "string", "analyzer": "english"}, "keyword": {"type": "keyword"}}}, "tweet_id": {"type": "long"}, "polarity": {"type": "float"}, "subjectivity": {"type": "float"}, "sentiment": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}}}, "newsheadline": {"properties": {"date": {"type": "date"}, "location": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}, "message": {"type": "string", "fields": {"english": {"type": "string", "analyzer": "english"}, "keyword": {"type": "keyword"}}}, "polarity": {"type": "float"}, "subjectivity": {"type": "float"}, "sentiment": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}}}}}
2021-01-21 07:22:38,533 [WARNING][elasticsearch] PUT http://localhost:9200/stocksight [status:N/A request:0.001s]
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 169, in _new_conn
conn = connection.create_connection(
File "/usr/local/lib/python3.9/site-packages/urllib3/util/connection.py", line 96, in create_connection
raise err
File "/usr/local/lib/python3.9/site-packages/urllib3/util/connection.py", line 86, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 61] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/elasticsearch/connection/http_urllib3.py", line 115, in perform_request
response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, **kw)
File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 755, in urlopen
retries = retries.increment(
File "/usr/local/lib/python3.9/site-packages/urllib3/util/retry.py", line 506, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/usr/local/lib/python3.9/site-packages/urllib3/packages/six.py", line 735, in reraise
raise value
File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
httplib_response = self._make_request(
File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 394, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 234, in request
super(HTTPConnection, self).request(method, url, body=body, headers=headers)
File "/usr/local/Cellar/[email protected]/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1255, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/local/Cellar/[email protected]/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1301, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/local/Cellar/[email protected]/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1250, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/local/Cellar/[email protected]/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1010, in _send_output
self.send(msg)
File "/usr/local/Cellar/[email protected]/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 950, in send
self.connect()
File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 200, in connect
conn = self._new_conn()
File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 181, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x10539fd90>: Failed to establish a new connection: [Errno 61] Connection refused
2021-01-21 07:22:38,533 [DEBUG][elasticsearch] > {"mappings": {"tweet": {"properties": {"author": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}, "location": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}, "language": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}, "friends": {"type": "long"}, "followers": {"type": "long"}, "statuses": {"type": "long"}, "date": {"type": "date"}, "message": {"type": "string", "fields": {"english": {"type": "string", "analyzer": "english"}, "keyword": {"type": "keyword"}}}, "tweet_id": {"type": "long"}, "polarity": {"type": "float"}, "subjectivity": {"type": "float"}, "sentiment": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}}}, "newsheadline": {"properties": {"date": {"type": "date"}, "location": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}, "message": {"type": "string", "fields": {"english": {"type": "string", "analyzer": "english"}, "keyword": {"type": "keyword"}}}, "polarity": {"type": "float"}, "subjectivity": {"type": "float"}, "sentiment": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}}}}}
2021-01-21 07:22:38,534 [WARNING][elasticsearch] PUT http://localhost:9200/stocksight [status:N/A request:0.001s]
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 169, in _new_conn
conn = connection.create_connection(
File "/usr/local/lib/python3.9/site-packages/urllib3/util/connection.py", line 96, in create_connection
raise err
File "/usr/local/lib/python3.9/site-packages/urllib3/util/connection.py", line 86, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 61] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/elasticsearch/connection/http_urllib3.py", line 115, in perform_request
response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, **kw)
File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 755, in urlopen
retries = retries.increment(
File "/usr/local/lib/python3.9/site-packages/urllib3/util/retry.py", line 506, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/usr/local/lib/python3.9/site-packages/urllib3/packages/six.py", line 735, in reraise
raise value
File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
httplib_response = self._make_request(
File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 394, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 234, in request
super(HTTPConnection, self).request(method, url, body=body, headers=headers)
File "/usr/local/Cellar/[email protected]/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1255, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/local/Cellar/[email protected]/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1301, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/local/Cellar/[email protected]/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1250, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/local/Cellar/[email protected]/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1010, in _send_output
self.send(msg)
File "/usr/local/Cellar/[email protected]/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 950, in send
self.connect()
File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 200, in connect
conn = self._new_conn()
File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 181, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x10539fa90>: Failed to establish a new connection: [Errno 61] Connection refused
2021-01-21 07:22:38,535 [DEBUG][elasticsearch] > {"mappings": {"tweet": {"properties": {"author": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}, "location": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}, "language": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}, "friends": {"type": "long"}, "followers": {"type": "long"}, "statuses": {"type": "long"}, "date": {"type": "date"}, "message": {"type": "string", "fields": {"english": {"type": "string", "analyzer": "english"}, "keyword": {"type": "keyword"}}}, "tweet_id": {"type": "long"}, "polarity": {"type": "float"}, "subjectivity": {"type": "float"}, "sentiment": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}}}, "newsheadline": {"properties": {"date": {"type": "date"}, "location": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}, "message": {"type": "string", "fields": {"english": {"type": "string", "analyzer": "english"}, "keyword": {"type": "keyword"}}}, "polarity": {"type": "float"}, "subjectivity": {"type": "float"}, "sentiment": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}}}}}
2021-01-21 07:22:38,536 [WARNING][elasticsearch] PUT http://localhost:9200/stocksight [status:N/A request:0.001s]
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 169, in _new_conn
conn = connection.create_connection(
File "/usr/local/lib/python3.9/site-packages/urllib3/util/connection.py", line 96, in create_connection
raise err
File "/usr/local/lib/python3.9/site-packages/urllib3/util/connection.py", line 86, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 61] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/elasticsearch/connection/http_urllib3.py", line 115, in perform_request
response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, **kw)
File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 755, in urlopen
retries = retries.increment(
File "/usr/local/lib/python3.9/site-packages/urllib3/util/retry.py", line 506, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/usr/local/lib/python3.9/site-packages/urllib3/packages/six.py", line 735, in reraise
raise value
File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
httplib_response = self._make_request(
File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 394, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 234, in request
super(HTTPConnection, self).request(method, url, body=body, headers=headers)
File "/usr/local/Cellar/[email protected]/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1255, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/local/Cellar/[email protected]/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1301, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/local/Cellar/[email protected]/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1250, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/local/Cellar/[email protected]/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1010, in _send_output
self.send(msg)
File "/usr/local/Cellar/[email protected]/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 950, in send
self.connect()
File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 200, in connect
conn = self._new_conn()
File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 181, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x10539fa30>: Failed to establish a new connection: [Errno 61] Connection refused
2021-01-21 07:22:38,536 [DEBUG][elasticsearch] > {"mappings": {"tweet": {"properties": {"author": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}, "location": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}, "language": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}, "friends": {"type": "long"}, "followers": {"type": "long"}, "statuses": {"type": "long"}, "date": {"type": "date"}, "message": {"type": "string", "fields": {"english": {"type": "string", "analyzer": "english"}, "keyword": {"type": "keyword"}}}, "tweet_id": {"type": "long"}, "polarity": {"type": "float"}, "subjectivity": {"type": "float"}, "sentiment": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}}}, "newsheadline": {"properties": {"date": {"type": "date"}, "location": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}, "message": {"type": "string", "fields": {"english": {"type": "string", "analyzer": "english"}, "keyword": {"type": "keyword"}}}, "polarity": {"type": "float"}, "subjectivity": {"type": "float"}, "sentiment": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}}}}}
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 169, in _new_conn
conn = connection.create_connection(
File "/usr/local/lib/python3.9/site-packages/urllib3/util/connection.py", line 96, in create_connection
raise err
File "/usr/local/lib/python3.9/site-packages/urllib3/util/connection.py", line 86, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 61] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/elasticsearch/connection/http_urllib3.py", line 115, in perform_request
response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, **kw)
File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 755, in urlopen
retries = retries.increment(
File "/usr/local/lib/python3.9/site-packages/urllib3/util/retry.py", line 506, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/usr/local/lib/python3.9/site-packages/urllib3/packages/six.py", line 735, in reraise
raise value
File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
httplib_response = self._make_request(
File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 394, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 234, in request
super(HTTPConnection, self).request(method, url, body=body, headers=headers)
File "/usr/local/Cellar/[email protected]/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1255, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/local/Cellar/[email protected]/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1301, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/local/Cellar/[email protected]/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1250, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/local/Cellar/[email protected]/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1010, in _send_output
self.send(msg)
File "/usr/local/Cellar/[email protected]/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 950, in send
self.connect()
File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 200, in connect
conn = self._new_conn()
File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 181, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x10539fa30>: Failed to establish a new connection: [Errno 61] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Users/casperdavidsen/Downloads/stocksight/sentiment.py", line 862, in
es.indices.create(index=args.index, body=mappings, ignore=[400, 404])
File "/usr/local/lib/python3.9/site-packages/elasticsearch/client/utils.py", line 73, in _wrapped
return func(*args, params=params, **kwargs)
File "/usr/local/lib/python3.9/site-packages/elasticsearch/client/indices.py", line 106, in create
return self.transport.perform_request('PUT', _make_path(index),
File "/usr/local/lib/python3.9/site-packages/elasticsearch/transport.py", line 312, in perform_request
status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
File "/usr/local/lib/python3.9/site-packages/elasticsearch/connection/http_urllib3.py", line 124, in perform_request
raise ConnectionError('N/A', str(e), e)
elasticsearch.exceptions.ConnectionError: ConnectionError(<urllib3.connection.HTTPConnection object at 0x10539fa30>: Failed to establish a new connection: [Errno 61] Connection refused) caused by: NewConnectionError(<urllib3.connection.HTTPConnection object at 0x10539fa30>: Failed to establish a new connection: [Errno 61] Connection refused)

First of all I would like to say this is a great idea and I cannot wait to be up and running with this :) I have been trying to get setup but I seem to be running into a few issues --

1. When running sentiment.py I am getting this, I would like to point out as I am used to working with Python I am no expert & I apologize in advance if I sound like a noob XD

`
LookupError:

Resource punkt not found.
Please use the NLTK Downloader to obtain the resource:

import nltk
nltk.download('punkt')

Attempted to load tokenizers/punkt/PY3/english.pickle

Searched in:
- '/Users/zachmayer/nltk_data'
- '/Library/Frameworks/Python.framework/Versions/3.7/nltk_data'
- '/Library/Frameworks/Python.framework/Versions/3.7/share/nltk_data'
- '/Library/Frameworks/Python.framework/Versions/3.7/lib/nltk_data'
- '/usr/share/nltk_data'
- '/usr/local/share/nltk_data'
- '/usr/lib/nltk_data'
- '/usr/local/lib/nltk_data'
- ''

`

2. What would be the steps in order to get the GUI dashboard up and running as well with Kibana? I would like to work on a detailed guide for other users that are not too familiar with it so more people can benefit from this project!

Thanks!!!

2018-12-12 11:32:06,850 [DEBUG][elasticsearch] < {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Rejecting mapping update to [stocksight] as the final mapping would have more than 1 type: [tweet, stock]"}],"type":"illegal_argument_exception","reason":"Rejecting mapping update to [stocksight] as the final mapping would have more than 1 type: [tweet, stock]"},"status":400}
2018-12-12 11:32:06,850 [ERROR][stocksight] Exception: can't get stock data, trying again later, reason is TransportError(400, 'illegal_argument_exception', 'Rejecting mapping update to [stocksight] as the final mapping would have more than 1 type: [tweet, stock]')

Also occurs when trying to scrape the newsheadlines

Elasticsearch 6.5.3
Python 3.6.6

Hey, I'm not able to move past the 'compose' step. The docker has started but I'm not able to shell into the container. It would be very helpful if you could make a tutorial on how to set up and use stocksight from scratch.

This is where I'm stuck at:

System Information:
OS: Windows 10 Professional
Docker Version: 19.0.8
Python Version: 3.7.2

Expected Behavior:

1. pull down https://github.com/shirosaidev/stocksight/releases/tag/v0.1-b.10 source code.
2. set config.py elasticsearch_host with the altered container name: "stocksight-01-b10_elasticsearch_1"
3. set the other config values as appropriate
4. docker-compose build, docker-compose up
5. docker exec -it stocksight-01-b10_stocksight_1 bash
6. run a command to pull back data into kibana (i.e. python sentiment.py -s TSLA -k 'Elon Musk',Musk,Tesla,SpaceX --debug)
7. The command runs successfully and relevant data is pulled into your kibana instance

Actual Behavior

1. pull down https://github.com/shirosaidev/stocksight/releases/tag/v0.1-b.10 source, code.
2. set config.py as with the altered container name: stocksight-01-b10_elasticsearch_1
3. set the other config values as appropriate
4. docker-compose build, docker-compose up
5. docker exec -it stocksight-01-b10_stocksight_1 bash
6. run a command to pull back data into kibana (i.e. python sentiment.py -s TSLA -k 'Elon Musk',Musk,Tesla,SpaceX --debug)
7. the command yields the following error: "name 'elasticsearch_port' is not defined stocksight", even though it is clearly defined in the config.yml as 9200 as it should be.

Temporary Resolution
I hard-coded the port on line 800 of sentiment.py:
es = Elasticsearch(hosts=[{'host': elasticsearch_host, 'port': 9200}],

This seemed to fix the issue for my docker setup. I'm not sure why it's not pulling that port number from the config.yml on my end, but it seems to pull the other config.json imported variables just fine. I suspect is might have something to do with variables named similarly in different places, but I haven't done analysis to conclude that's exactly the problem.

I will try to do submit pull request if I get a moment to look into this and find a more permanent resolution.

Everything seems fine but cant get the initial command to run.

py sentiment.py -s TSLA -k 'Elon Musk',Musk,Tesla,SpaceX --debug
File "C:\Users\Trading\Documents\stocksight-0.1-b.12\sentiment.py", line 27
from config import
^
SyntaxError: invalid syntax

C:\Users\Trading\Documents\stocksight-0.1-b.12>

Hi,

Amazing idea!

But While running stock price, it is not adding data to stocksight index and throwing below error

2019-03-17 19:38:38,731 [DEBUG][elasticsearch] < {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Rejecting mapping update to [stocksight] as the final mapping would have more than 1 type: [tweet, stock]"}],"type":"illegal_argument_exception","reason":"Rejecting mapping update to [stocksight] as the final mapping would have more than 1 type: [tweet, stock]"},"status":400}
2019-03-17 19:38:38,731 [ERROR][stocksight] Exception: can't get stock data, trying again later, reason is TransportError(400, 'illegal_argument_exception', 'Rejecting mapping update to [stocksight] as the final mapping would have more than 1 type: [tweet, stock]')

Could you please help me to fix it.

Thanks,
Anshul

Any chance of throwing a Django GUI on top of this? 😃

im just trying to get this repo to run i cant get passt this error:

elasticsearch5.exceptions.ConnectionError: ConnectionError(<urllib3.connection.HTTPConnection object at 0x7f6f35d0eb38>: Failed to establish a new connection: [Errno 111] Connection refused) caused by: NewConnectionError(<urllib3.connection.HTTPConnection object at 0x7f6f35d0eb38>: Failed to establish a new connection: [Errno 111] Connection refused)

caused by:

2018-11-20 15:26:36,085 [INFO][stocksight] Creating new Elasticsearch index or using existing stocksight
2018-11-20 15:26:36,087 [WARNING][elasticsearch] PUT http://localhost:9200/stocksight [status:N/A request:0.001s]
```
i also tried to configure elastic in the call but no luck.

Any suggestions? 

Any Help for this ??

I am having trouble regarding:
"Load 'stocksight' index in Kibana and import export.json file for visuals/dashboard."

I already started Kibana and Elasticsearch in docker images and I am successfully running sentiment.py with keywords. I can load the stocksight index in kibana but where do I import the export.json file. Could you please elaborate on how to do so?

After cloning this repo and running the appropriate docker-compose commands to get the service(s) running, I had a very difficult time figuring out how to access the Kibana user interface. The port is not documented in the README, and I ended up discovering it in the docker-compose file. After I was able to access the Kibana user interface, I'm prompted with a request for an "index pattern". I have no clue what this is, or what it needs to be, and I don't see any reference to it in the README.

I am facing this issue upon executing the sentiment.py file. Any Suggestions or solutions??

2023-08-08 14:05:45,887 [INFO][stocksight] Creating new Elasticsearch index or using existing stocksight
2023-08-08 14:05:50,462 [WARNING][elasticsearch] PUT http://localhost:9200/stocksight [status:N/A request:4.576s]
Traceback (most recent call last):
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\connection.py", line 203, in _new_conn
sock = connection.create_connection(
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\util\connection.py", line 85, in create_connection
raise err
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\util\connection.py", line 73, in create_connection
sock.connect(sa)
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\elasticsearch\connection\http_urllib3.py", line 115, in perform_request
response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, **kw)
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\connectionpool.py", line 844, in urlopen
retries = retries.increment(
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\util\retry.py", line 445, in increment
raise reraise(type(error), error, _stacktrace)
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\util\util.py", line 39, in reraise
raise value
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\connectionpool.py", line 790, in urlopen
response = self._make_request(
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\connectionpool.py", line 496, in _make_request
conn.request(
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\connection.py", line 395, in request
self.endheaders()
File "C:\Users\moham\AppData\Local\Programs\Python\Python39\lib\http\client.py", line 1280, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "C:\Users\moham\AppData\Local\Programs\Python\Python39\lib\http\client.py", line 1040, in _send_output
self.send(msg)
File "C:\Users\moham\AppData\Local\Programs\Python\Python39\lib\http\client.py", line 980, in send
self.connect()
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\connection.py", line 243, in connect
self.sock = self._new_conn()
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\connection.py", line 218, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x000001B681972C40>: Fail
ed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it
2023-08-08 14:05:50,748 [DEBUG][elasticsearch] > {"mappings": {"stock": {"properties": {"symbol": {"type": "k
eyword"}, "price_last": {"type": "float"}, "date": {"type": "date"}, "change": {"type": "float"}, "price_high": {"type": "float"}, "price_low": {"type": "float"}, "vol": {"type": "integer"}}}}}
2023-08-08 14:05:55,561 [WARNING][elasticsearch] PUT http://localhost:9200/stocksight [status:N/A request:4.811s]
Traceback (most recent call last):
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\connection.py", line 203, in _new_conn
sock = connection.create_connection(
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\util\connection.py", line 85, in create_connection
raise err
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\util\connection.py", line 73, in create_connection
sock.connect(sa)
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\elasticsearch\connection\http_urllib3.py", line 115, in perform_request
response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, **kw)
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\connectionpool.py", line 844, in urlopen
retries = retries.increment(
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\util\retry.py", line 445, in increment
raise reraise(type(error), error, _stacktrace)
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\util\util.py", line 39, in reraise
raise value
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\connectionpool.py", line 790, in urlopen
response = self._make_request(
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\connectionpool.py", line 496, in _make_request
conn.request(
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\connection.py", line 395, in request
self.endheaders()
File "C:\Users\moham\AppData\Local\Programs\Python\Python39\lib\http\client.py", line 1280, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "C:\Users\moham\AppData\Local\Programs\Python\Python39\lib\http\client.py", line 1040, in _send_output
self.send(msg)
File "C:\Users\moham\AppData\Local\Programs\Python\Python39\lib\http\client.py", line 980, in send
self.connect()
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\connection.py", line 243, in connect
self.sock = self._new_conn()
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\connection.py", line 218, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x000001B681972E50>: Fail
ed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it
2023-08-08 14:05:55,685 [DEBUG][elasticsearch] > {"mappings": {"stock": {"properties": {"symbol": {"type": "k
eyword"}, "price_last": {"type": "float"}, "date": {"type": "date"}, "change": {"type": "float"}, "price_high": {"type": "float"}, "price_low": {"type": "float"}, "vol": {"type": "integer"}}}}}
2023-08-08 14:06:00,331 [WARNING][elasticsearch] PUT http://localhost:9200/stocksight [status:N/A request:4.646s]
Traceback (most recent call last):
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\connection.py", line 203, in _new_conn
sock = connection.create_connection(
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\util\connection.py", line 85, in create_connection
raise err
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\util\connection.py", line 73, in create_connection
sock.connect(sa)
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\elasticsearch\connection\http_urllib3.py", line 115, in perform_request
response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, **kw)
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\connectionpool.py", line 844, in urlopen
retries = retries.increment(
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\util\retry.py", line 445, in increment
raise reraise(type(error), error, _stacktrace)
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\util\util.py", line 39, in reraise
raise value
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\connectionpool.py", line 790, in urlopen
response = self._make_request(
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\connectionpool.py", line 496, in _make_request
conn.request(
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\connection.py", line 395, in request
self.endheaders()
File "C:\Users\moham\AppData\Local\Programs\Python\Python39\lib\http\client.py", line 1280, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "C:\Users\moham\AppData\Local\Programs\Python\Python39\lib\http\client.py", line 1040, in _send_output
self.send(msg)
File "C:\Users\moham\AppData\Local\Programs\Python\Python39\lib\http\client.py", line 980, in send
self.connect()
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\connection.py", line 243, in connect
self.sock = self._new_conn()
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\connection.py", line 218, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x000001B681989100>: Fail
ed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it
2023-08-08 14:06:00,396 [DEBUG][elasticsearch] > {"mappings": {"stock": {"properties": {"symbol": {"type": "k
eyword"}, "price_last": {"type": "float"}, "date": {"type": "date"}, "change": {"type": "float"}, "price_high": {"type": "float"}, "price_low": {"type": "float"}, "vol": {"type": "integer"}}}}}
2023-08-08 14:06:05,012 [WARNING][elasticsearch] PUT http://localhost:9200/stocksight [status:N/A request:4.614s]
Traceback (most recent call last):
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\connection.py", line 203, in _new_conn
sock = connection.create_connection(
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\util\connection.py", line 85, in create_connection
raise err
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\util\connection.py", line 73, in create_connection
sock.connect(sa)
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\elasticsearch\connection\http_urllib3.py", line 115, in perform_request
response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, **kw)
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\connectionpool.py", line 844, in urlopen
retries = retries.increment(
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\util\retry.py", line 445, in increment
raise reraise(type(error), error, _stacktrace)
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\util\util.py", line 39, in reraise
raise value
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\connectionpool.py", line 790, in urlopen
response = self._make_request(
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\connectionpool.py", line 496, in _make_request
conn.request(
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\connection.py", line 395, in request
self.endheaders()
File "C:\Users\moham\AppData\Local\Programs\Python\Python39\lib\http\client.py", line 1280, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "C:\Users\moham\AppData\Local\Programs\Python\Python39\lib\http\client.py", line 1040, in _send_output
self.send(msg)
File "C:\Users\moham\AppData\Local\Programs\Python\Python39\lib\http\client.py", line 980, in send
self.connect()
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\connection.py", line 243, in connect
self.sock = self._new_conn()
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\connection.py", line 218, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x000001B681989280>: Fail
ed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it
2023-08-08 14:06:05,017 [DEBUG][elasticsearch] > {"mappings": {"stock": {"properties": {"symbol": {"type": "k
eyword"}, "price_last": {"type": "float"}, "date": {"type": "date"}, "change": {"type": "float"}, "price_high": {"type": "float"}, "price_low": {"type": "float"}, "vol": {"type": "integer"}}}}}
Traceback (most recent call last):
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\connection.py", line 203, in _new_conn
sock = connection.create_connection(
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\util\connection.py", line 85, in create_connection
raise err
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\util\connection.py", line 73, in create_connection
sock.connect(sa)
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\elasticsearch\connection\http_urllib3.py", line 115, in perform_request
response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, **kw)
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\connectionpool.py", line 844, in urlopen
retries = retries.increment(
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\util\retry.py", line 445, in increment
raise reraise(type(error), error, _stacktrace)
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\util\util.py", line 39, in reraise
raise value
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\connectionpool.py", line 790, in urlopen
response = self._make_request(
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\connectionpool.py", line 496, in _make_request
conn.request(
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\connection.py", line 395, in request
self.endheaders()
File "C:\Users\moham\AppData\Local\Programs\Python\Python39\lib\http\client.py", line 1280, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "C:\Users\moham\AppData\Local\Programs\Python\Python39\lib\http\client.py", line 1040, in _send_output
self.send(msg)
File "C:\Users\moham\AppData\Local\Programs\Python\Python39\lib\http\client.py", line 980, in send
self.connect()
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\connection.py", line 243, in connect
self.sock = self._new_conn()
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\urllib3\connection.py", line 218, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x000001B681989280>: Fail
ed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "D:\Recent Projects\stocks_prediction\stocksight\stockprice.py", line 234, in
es.indices.create(index=args.index, body=mappings, ignore=[400, 404])
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\elasticsearch\client\utils.py", line 73, in _wrapped
return func(*args, params=params, **kwargs)
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\elasticsearch\client\indices.py", line 106, in create
return self.transport.perform_request('PUT', _make_path(index),
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\elasticsearch\transport.py", line 312, in perform_request
status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
File "D:\Recent Projects\stocks_prediction\venv\lib\site-packages\elasticsearch\connection\http_urllib3.py", line 124, in perform_request
raise ConnectionError('N/A', str(e), e)
elasticsearch.exceptions.ConnectionError: ConnectionError(<urllib3.connection.HTTPConnection object at 0x0000
01B681989280>: Failed to establish a new connection: [WinError 10061] No connection could be made because the
target machine actively refused it) caused by: NewConnectionError(<urllib3.connection.HTTPConnection object
at 0x000001B681989280>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it)

can someone help me with this .. i tried to change the "localhost" to my device hostname and changing the port also and nothing works

Hey, had some time today so decided to give this a go, but I get an error running the script.

2019-05-18 11:40:36,349 [INFO][stocksight] Creating new Elasticsearch index or using existing stocksight 2019-05-18 11:40:36,364 [INFO][elasticsearch] PUT http://localhost:9200/stocksight [status:400 request:0.014s] 2019-05-18 11:40:36,364 [DEBUG][elasticsearch] > {"mappings": {"newsheadline": {"properties": {"polarity": {"type": "float"}, "sentiment": {"fields": {"keyword": {"type": "keyword"}}, "type": "string"}, "location": {"fields": {"keyword": {"type": "keyword"}}, "type": "string"}, "subjectivity": {"type": "float"}, "date": {"type": "date"}, "message": {"fields": {"keyword": {"type": "keyword"}, "english": {"type": "string", "analyzer": "english"}}, "type": "string"}}}, "tweet": {"properties": {"subjectivity": {"type": "float"}, "date": {"type": "date"}, "message": {"fields": {"keyword": {"type": "keyword"}, "english": {"type": "string", "analyzer": "english"}}, "type": "string"}, "friends": {"type": "long"}, "statuses": {"type": "long"}, "polarity": {"type": "float"}, "sentiment": {"fields": {"keyword": {"type": "keyword"}}, "type": "string"}, "language": {"fields": {"keyword": {"type": "keyword"}}, "type": "string"}, "author": {"fields": {"keyword": {"type": "keyword"}}, "type": "string"}, "tweet_id": {"type": "long"}, "followers": {"type": "long"}, "location": {"fields": {"keyword": {"type": "keyword"}}, "type": "string"}}}}} 2019-05-18 11:40:36,365 [DEBUG][elasticsearch] < {"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"No handler for type [string] declared on field [sentiment]"}],"type":"mapper_parsing_exception","reason":"Failed to parse mapping [tweet]: No handler for type [string] declared on field [sentiment]","caused_by":{"type":"mapper_parsing_exception","reason":"No handler for type [string] declared on field [sentiment]"}},"status":400} 2019-05-18 11:40:36,366 [INFO][stocksight] Looking up Twitter user ids from usernames...

Info regarding elastic search

andy@shirosaidev-stocksight:~/stocksight$ curl http://localhost:9200/
{
"name" : "Cluster_01_Node_001",
"cluster_name" : "ES_Cluster_01",
"cluster_uuid" : "FKGqityWS1mJyc_s46Ukqg",
"version" : {
"number" : "6.7.2",
"build_flavor" : "default",
"build_type" : "deb",
"build_hash" : "56c6e48",
"build_date" : "2019-04-29T09:05:50.290371Z",
"build_snapshot" : false,
"lucene_version" : "7.7.0",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},
"tagline" : "You Know, for Search"
}

The fresh installation of the stack and executing the python code throws an expetion

Hi there,
I would really like to try out you program. I have everything installed as described but after running;
python sentiment.py -k TSLA,'Elon Musk',Musk,Tesla --debug

I receive the following error:

`2019-02-02 02:19:54,186 [WARNING][stocksight] Exception: exception caused by:

Resource punkt not found.
Please use the NLTK Downloader to obtain the resource:

import nltk
nltk.download('punkt')

Attempted to load tokenizers/punkt/english.pickle

Searched in:
- '/home/user/nltk_data'
- '/usr/nltk_data'
- '/usr/share/nltk_data'
- '/usr/lib/nltk_data'
- '/usr/share/nltk_data'
- '/usr/local/share/nltk_data'
- '/usr/lib/nltk_data'
- '/usr/local/lib/nltk_data'
- u''

Traceback (most recent call last):
File "sentiment.py", line 796, in
stream.filter(track=keywords, languages=['en'])
File "/home/user/.local/lib/python2.7/site-packages/tweepy/streaming.py", line 453, in filter
self._start(is_async)
File "/home/user/.local/lib/python2.7/site-packages/tweepy/streaming.py", line 368, in _start
self._run()
File "/home/user/.local/lib/python2.7/site-packages/tweepy/streaming.py", line 300, in _run
six.reraise(*exc_info)
File "/home/user/.local/lib/python2.7/site-packages/tweepy/streaming.py", line 269, in _run
self._read_loop(resp)
File "/home/user/.local/lib/python2.7/site-packages/tweepy/streaming.py", line 331, in _read_loop
self._data(next_status_obj)
File "/home/user/.local/lib/python2.7/site-packages/tweepy/streaming.py", line 303, in _data
if self.listener.on_data(data) is False:
File "sentiment.py", line 127, in on_data
tokens = nltk.word_tokenize(text_for_tokens)
File "/home/user/.local/lib/python2.7/site-packages/nltk/tokenize/init.py", line 143, in word_tokenize
sentences = [text] if preserve_line else sent_tokenize(text, language)
File "/home/user/.local/lib/python2.7/site-packages/nltk/tokenize/init.py", line 104, in sent_tokenize
tokenizer = load('tokenizers/punkt/{0}.pickle'.format(language))
File "/home/user/.local/lib/python2.7/site-packages/nltk/data.py", line 868, in load
opened_resource = _open(resource_url)
File "/home/user/.local/lib/python2.7/site-packages/nltk/data.py", line 993, in open
return find(path, path + ['']).open()
File "/home/user/.local/lib/python2.7/site-packages/nltk/data.py", line 699, in find
raise LookupError(resource_not_found)
LookupError:

Resource punkt not found.
Please use the NLTK Downloader to obtain the resource:

import nltk
nltk.download('punkt')

Attempted to load tokenizers/punkt/english.pickle

Searched in:
- '/home/user/nltk_data'
- '/usr/nltk_data'
- '/usr/share/nltk_data'
- '/usr/lib/nltk_data'
- '/usr/share/nltk_data'
- '/usr/local/share/nltk_data'
- '/usr/lib/nltk_data'
- '/usr/local/lib/nltk_data'
- u''
**********************************************************************`

I have issued:
pip install nltk
and even issued:
apt-get install python-numpy python-nltk

And tried NLTK manually:

`
Python 2.7.15rc1 (default, Nov 12 2018, 14:31:15)
[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.

Import nltk
File "", line 1
Import nltk
^
SyntaxError: invalid syntax
`
I'm not sure what I'm missing here, and hope you can help me get everything running as it's supposed to.
I'm running Ubuntu 18.04 with the latest updates.
Kind regards,
Ronald.

Can you provide , maybe export your dashboard ?
I don't have experience with Kibana + Elasticsearch I made this run but I can't show any data.

2019-05-18 13:18:17,457 [ERROR][stocksight] Exception: can't get stock data, trying again later, reason is TransportError(400, 'illegal_argument_exception', 'Rejecting mapping update to [stocksight] as the final mapping would have more than 1 type: [tweet, stock]')

Running on a Ubuntu 18.04 TLS
andy@shirosaidev-stocksight:~/stocksight$ python3.6 stockprice.py -s TSLA --debug

Kibana 6.7.2
ElasticSearch 6.7.2

Hi,
I was trying to build the project and after configuring it as suggested in "How to use" section and running the first example with

python sentiment.py -s TSLA -k "Elon Musk,Musk,Tesla,SpaceX" --debug

I receive the next error log

2020-06-09 00:24:18,768 [INFO][stocksight] Creating new Elasticsearch index or using existing stocksight
2020-06-09 00:24:18,775 [WARNING][elasticsearch] PUT http://localhost:9200/stocksight [status:N/A request:0.007s]
Traceback (most recent call last):
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/connection.py", line 160, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/util/connection.py", line 84, in create_connection
    raise err
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/util/connection.py", line 74, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/elasticsearch5/connection/http_urllib3.py", line 115, in perform_request
    response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, **kw)
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 725, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/util/retry.py", line 379, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/packages/six.py", line 735, in reraise
    raise value
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 677, in urlopen
    chunked=chunked,
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 392, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3.7/http/client.py", line 1252, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.7/http/client.py", line 1298, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.7/http/client.py", line 1247, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.7/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/usr/lib/python3.7/http/client.py", line 966, in send
    self.connect()
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/connection.py", line 187, in connect
    conn = self._new_conn()
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/connection.py", line 172, in _new_conn
    self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fa6386eb710>: Failed to establish a new connection: [Errno 111] Connection refused
2020-06-09 00:24:18,776 [DEBUG][elasticsearch] > {"mappings": {"tweet": {"properties": {"author": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}, "location": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}, "language": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}, "friends": {"type": "long"}, "followers": {"type": "long"}, "statuses": {"type": "long"}, "date": {"type": "date"}, "message": {"type": "string", "fields": {"english": {"type": "string", "analyzer": "english"}, "keyword": {"type": "keyword"}}}, "tweet_id": {"type": "long"}, "polarity": {"type": "float"}, "subjectivity": {"type": "float"}, "sentiment": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}}}, "newsheadline": {"properties": {"date": {"type": "date"}, "location": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}, "message": {"type": "string", "fields": {"english": {"type": "string", "analyzer": "english"}, "keyword": {"type": "keyword"}}}, "polarity": {"type": "float"}, "subjectivity": {"type": "float"}, "sentiment": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}}}}}
2020-06-09 00:24:18,783 [WARNING][elasticsearch] PUT http://localhost:9200/stocksight [status:N/A request:0.006s]

       _                     _                 
     _| |_ _           _   _| |_ _     _   _   
    |   __| |_ ___ ___| |_|   __|_|___| |_| |_ 
    |__   |  _| . |  _| '_|__   | | . |   |  _|
    |_   _|_| |___|___|_,_|_   _|_|_  |_|_|_|  
      |_|                   |_|   |___|                
          :) = +$   :( = -$    v0.1-b.11
    GitHub repo https://github.com/shirosaidev/stocksight
    StockSight website https://stocksight.diskoverspace.com
            

Traceback (most recent call last):
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/connection.py", line 160, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/util/connection.py", line 84, in create_connection
    raise err
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/util/connection.py", line 74, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/elasticsearch5/connection/http_urllib3.py", line 115, in perform_request
    response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, **kw)
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 725, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/util/retry.py", line 379, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/packages/six.py", line 735, in reraise
    raise value
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 677, in urlopen
    chunked=chunked,
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 392, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3.7/http/client.py", line 1252, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.7/http/client.py", line 1298, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.7/http/client.py", line 1247, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.7/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/usr/lib/python3.7/http/client.py", line 966, in send
    self.connect()
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/connection.py", line 187, in connect
    conn = self._new_conn()
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/connection.py", line 172, in _new_conn
    self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fa6386eba50>: Failed to establish a new connection: [Errno 111] Connection refused
2020-06-09 00:24:18,783 [DEBUG][elasticsearch] > {"mappings": {"tweet": {"properties": {"author": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}, "location": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}, "language": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}, "friends": {"type": "long"}, "followers": {"type": "long"}, "statuses": {"type": "long"}, "date": {"type": "date"}, "message": {"type": "string", "fields": {"english": {"type": "string", "analyzer": "english"}, "keyword": {"type": "keyword"}}}, "tweet_id": {"type": "long"}, "polarity": {"type": "float"}, "subjectivity": {"type": "float"}, "sentiment": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}}}, "newsheadline": {"properties": {"date": {"type": "date"}, "location": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}, "message": {"type": "string", "fields": {"english": {"type": "string", "analyzer": "english"}, "keyword": {"type": "keyword"}}}, "polarity": {"type": "float"}, "subjectivity": {"type": "float"}, "sentiment": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}}}}}
2020-06-09 00:24:18,790 [WARNING][elasticsearch] PUT http://localhost:9200/stocksight [status:N/A request:0.007s]
Traceback (most recent call last):
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/connection.py", line 160, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/util/connection.py", line 84, in create_connection
    raise err
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/util/connection.py", line 74, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/elasticsearch5/connection/http_urllib3.py", line 115, in perform_request
    response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, **kw)
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 725, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/util/retry.py", line 379, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/packages/six.py", line 735, in reraise
    raise value
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 677, in urlopen
    chunked=chunked,
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 392, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3.7/http/client.py", line 1252, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.7/http/client.py", line 1298, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.7/http/client.py", line 1247, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.7/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/usr/lib/python3.7/http/client.py", line 966, in send
    self.connect()
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/connection.py", line 187, in connect
    conn = self._new_conn()
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/connection.py", line 172, in _new_conn
    self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fa638640cd0>: Failed to establish a new connection: [Errno 111] Connection refused
2020-06-09 00:24:18,790 [DEBUG][elasticsearch] > {"mappings": {"tweet": {"properties": {"author": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}, "location": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}, "language": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}, "friends": {"type": "long"}, "followers": {"type": "long"}, "statuses": {"type": "long"}, "date": {"type": "date"}, "message": {"type": "string", "fields": {"english": {"type": "string", "analyzer": "english"}, "keyword": {"type": "keyword"}}}, "tweet_id": {"type": "long"}, "polarity": {"type": "float"}, "subjectivity": {"type": "float"}, "sentiment": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}}}, "newsheadline": {"properties": {"date": {"type": "date"}, "location": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}, "message": {"type": "string", "fields": {"english": {"type": "string", "analyzer": "english"}, "keyword": {"type": "keyword"}}}, "polarity": {"type": "float"}, "subjectivity": {"type": "float"}, "sentiment": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}}}}}
2020-06-09 00:24:18,800 [WARNING][elasticsearch] PUT http://localhost:9200/stocksight [status:N/A request:0.009s]
Traceback (most recent call last):
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/connection.py", line 160, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/util/connection.py", line 84, in create_connection
    raise err
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/util/connection.py", line 74, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/elasticsearch5/connection/http_urllib3.py", line 115, in perform_request
    response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, **kw)
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 725, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/util/retry.py", line 379, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/packages/six.py", line 735, in reraise
    raise value
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 677, in urlopen
    chunked=chunked,
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 392, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3.7/http/client.py", line 1252, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.7/http/client.py", line 1298, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.7/http/client.py", line 1247, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.7/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/usr/lib/python3.7/http/client.py", line 966, in send
    self.connect()
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/connection.py", line 187, in connect
    conn = self._new_conn()
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/connection.py", line 172, in _new_conn
    self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fa638640f90>: Failed to establish a new connection: [Errno 111] Connection refused
2020-06-09 00:24:18,800 [DEBUG][elasticsearch] > {"mappings": {"tweet": {"properties": {"author": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}, "location": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}, "language": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}, "friends": {"type": "long"}, "followers": {"type": "long"}, "statuses": {"type": "long"}, "date": {"type": "date"}, "message": {"type": "string", "fields": {"english": {"type": "string", "analyzer": "english"}, "keyword": {"type": "keyword"}}}, "tweet_id": {"type": "long"}, "polarity": {"type": "float"}, "subjectivity": {"type": "float"}, "sentiment": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}}}, "newsheadline": {"properties": {"date": {"type": "date"}, "location": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}, "message": {"type": "string", "fields": {"english": {"type": "string", "analyzer": "english"}, "keyword": {"type": "keyword"}}}, "polarity": {"type": "float"}, "subjectivity": {"type": "float"}, "sentiment": {"type": "string", "fields": {"keyword": {"type": "keyword"}}}}}}}
Traceback (most recent call last):
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/connection.py", line 160, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/util/connection.py", line 84, in create_connection
    raise err
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/util/connection.py", line 74, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/elasticsearch5/connection/http_urllib3.py", line 115, in perform_request
    response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, **kw)
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 725, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/util/retry.py", line 379, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/packages/six.py", line 735, in reraise
    raise value
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 677, in urlopen
    chunked=chunked,
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 392, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3.7/http/client.py", line 1252, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.7/http/client.py", line 1298, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.7/http/client.py", line 1247, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.7/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/usr/lib/python3.7/http/client.py", line 966, in send
    self.connect()
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/connection.py", line 187, in connect
    conn = self._new_conn()
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/urllib3/connection.py", line 172, in _new_conn
    self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fa638640f90>: Failed to establish a new connection: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/david/git/stocksight/sentiment.py", line 871, in <module>
    es.indices.create(index=args.index, body=mappings, ignore=[400, 404])
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/elasticsearch5/client/utils.py", line 73, in _wrapped
    return func(*args, params=params, **kwargs)
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/elasticsearch5/client/indices.py", line 107, in create
    params=params, body=body)
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/elasticsearch5/transport.py", line 312, in perform_request
    status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
  File "/home/david/git/stocksight/venv/lib/python3.7/site-packages/elasticsearch5/connection/http_urllib3.py", line 124, in perform_request
    raise ConnectionError('N/A', str(e), e)
elasticsearch5.exceptions.ConnectionError: ConnectionError(<urllib3.connection.HTTPConnection object at 0x7fa638640f90>: Failed to establish a new connection: [Errno 111] Connection refused) caused by: NewConnectionError(<urllib3.connection.HTTPConnection object at 0x7fa638640f90>: Failed to establish a new connection: [Errno 111] Connection refused)

Do I forgot something?

Thanks in advanced

how much time will the sentiment.py file runs.
during its execution I cant access the stocksight index in kibana.
Also stockprice got an error "Exception: can't get stock data, trying again later, reason is unsupported operand type(s) for -: 'NoneType' and 'float'"

Hi, I have stalled every requirement is needed but I have several errors. I attach you a picture about it. Thanks you very much!

Hi,

I really like the project, however it would be awesome if this would be dockerized.

On running the command:
python stockprice.py -s TSLA --debug

I got this error. Please help me to solve this one:

2019-05-09 11:41:25,510 [�[1;32mINFO�[1;0m][stocksight] Adding stock data to Elasticsearch...
2019-05-09 11:41:25,562 [�[1;31mWARNING�[1;0m][elasticsearch] POST http://localhost:9200/stocksight/stock [status:400 request:0.048s]
2019-05-09 11:41:25,566 [�[1;33mDEBUG�[1;0m][elasticsearch] > {"symbol": "TSLA", "price_last": 244.75999450683594, "date": "2019-05-09T06:11:25", "change": 0.01634246728233062, "price_high": 244.88389587402344, "price_low": 244.66140747070312, "vol": 97861}
2019-05-09 11:41:25,570 [�[1;33mDEBUG�[1;0m][elasticsearch] < {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Rejecting mapping update to [stocksight] as the final mapping would have more than 1 type: [tweet, stock]"}],"type":"illegal_argument_exception","reason":"Rejecting mapping update to [stocksight] as the final mapping would have more than 1 type: [tweet, stock]"},"status":400}
2019-05-09 11:41:25,574 [�[1;41mERROR�[1;0m][stocksight] Exception: can't get stock data, trying again later, reason is TransportError(400, 'illegal_argument_exception', 'Rejecting mapping update to [stocksight] as the final mapping would have more than 1 type: [tweet, stock]')
2019-05-09 11:41:25,574 [�[1;32mINFO�[1;0m][stocksight] Will get stock data again in 120 sec...

I was able to run sentiment.py however I got ERROR:401. I got to know ERROR:401 is caused due to bad request but I don't know what is causing this issue.

[INFO][stocksight] Writing twitter user ids to text file ./twitteruserids.txt
Traceback (most recent call last):
File "sentiment.py", line 761, in
f.write(i + '\n')
TypeError: can't concat str to bytes

How to fix this?

My name is Luis, I'm a big-data machine-learning developer, I'm a fan of your work, and I usually check your updates.

I was afraid that my savings would be eaten by inflation. I have created a powerful tool that based on past technical patterns (volatility, moving averages, statistics, trends, candlesticks, support and resistance, stock index indicators).
All the ones you know (RSI, MACD, STOCH, Bolinger Bands, SMA, DEMARK, Japanese candlesticks, ichimoku, fibonacci, williansR, balance of power, murrey math, etc) and more than 200 others.

The tool creates prediction models of correct trading points (buy signal and sell signal, every stock is good traded in time and direction).
For this I have used big data tools like pandas python, stock market libraries like: tablib, TAcharts ,pandas_ta... For data collection and calculation.
And powerful machine-learning libraries such as: Sklearn.RandomForest , Sklearn.GradientBoosting, XGBoost, Google TensorFlow and Google TensorFlow LSTM.

With the models trained with the selection of the best technical indicators, the tool is able to predict trading points (where to buy, where to sell) and send real-time alerts to Telegram or Mail. The points are calculated based on the learning of the correct trading points of the last 2 years (including the change to bear market after the rate hike).

I think it could be useful to you, to improve, I would like to give it to you, and if you are interested in improving and collaborating I am also willing, and if not I would like to file it in the drawer.

Hello, I don't have any experience with elasticsearch or kibana etc. Could anyone explain to me what these errors mean?

2019-09-15 07:16:56,930 [INFO][stocksight] Creating new Elasticsearch index or using existing stocksight
2019-09-15 07:16:56,931 [WARNING][elasticsearch] PUT http://localhost:9200/stocksight [status:N/A request:0.000s]
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 159, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw)
File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 80, in create_connection
raise err
File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 70, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/elasticsearch5/connection/http_urllib3.py", line 115, in perform_request
response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, **kw)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 638, in urlopen
_stacktrace=sys.exc_info()[2])
File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 343, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/usr/lib/python3/dist-packages/six.py", line 693, in reraise
raise value
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 600, in urlopen
chunked=chunked)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 354, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/lib/python3.7/http/client.py", line 1244, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.7/http/client.py", line 1290, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.7/http/client.py", line 1239, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.7/http/client.py", line 1026, in _send_output
self.send(msg)
File "/usr/lib/python3.7/http/client.py", line 966, in send
self.connect()
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 181, in connect
conn = self._new_conn()
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 168, in _new_conn
self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f6336d89cd0>: Failed to establish a new connection: [Errno 111] Connection refused
2019-09-15 07:16:56,933 [WARNING][elasticsearch] PUT http://localhost:9200/stocksight [status:N/A request:0.000s]
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 159, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw)
File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 80, in create_connection
raise err
File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 70, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/elasticsearch5/connection/http_urllib3.py", line 115, in perform_request
response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, **kw)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 638, in urlopen
_stacktrace=sys.exc_info()[2])
File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 343, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/usr/lib/python3/dist-packages/six.py", line 693, in reraise
raise value
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 600, in urlopen
chunked=chunked)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 354, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/lib/python3.7/http/client.py", line 1244, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.7/http/client.py", line 1290, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.7/http/client.py", line 1239, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.7/http/client.py", line 1026, in _send_output
self.send(msg)
File "/usr/lib/python3.7/http/client.py", line 966, in send
self.connect()
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 181, in connect
conn = self._new_conn()
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 168, in _new_conn
self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f6336d47810>: Failed to establish a new connection: [Errno 111] Connection refused
2019-09-15 07:16:56,934 [WARNING][elasticsearch] PUT http://localhost:9200/stocksight [status:N/A request:0.000s]
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 159, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw)
File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 80, in create_connection
raise err
File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 70, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/elasticsearch5/connection/http_urllib3.py", line 115, in perform_request
response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, **kw)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 638, in urlopen
_stacktrace=sys.exc_info()[2])
File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 343, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/usr/lib/python3/dist-packages/six.py", line 693, in reraise
raise value
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 600, in urlopen
chunked=chunked)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 354, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/lib/python3.7/http/client.py", line 1244, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.7/http/client.py", line 1290, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.7/http/client.py", line 1239, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.7/http/client.py", line 1026, in _send_output
self.send(msg)
File "/usr/lib/python3.7/http/client.py", line 966, in send
self.connect()
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 181, in connect
conn = self._new_conn()
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 168, in _new_conn
self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f6336d89b10>: Failed to establish a new connection: [Errno 111] Connection refused
2019-09-15 07:16:56,935 [WARNING][elasticsearch] PUT http://localhost:9200/stocksight [status:N/A request:0.000s]
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 159, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw)
File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 80, in create_connection
raise err
File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 70, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/elasticsearch5/connection/http_urllib3.py", line 115, in perform_request
response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, **kw)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 638, in urlopen
_stacktrace=sys.exc_info()[2])
File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 343, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/usr/lib/python3/dist-packages/six.py", line 693, in reraise
raise value
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 600, in urlopen
chunked=chunked)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 354, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/lib/python3.7/http/client.py", line 1244, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.7/http/client.py", line 1290, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.7/http/client.py", line 1239, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.7/http/client.py", line 1026, in _send_output
self.send(msg)
File "/usr/lib/python3.7/http/client.py", line 966, in send
self.connect()
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 181, in connect
conn = self._new_conn()
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 168, in _new_conn
self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f6336cd8c10>: Failed to establish a new connection: [Errno 111] Connection refused
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 159, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw)
File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 80, in create_connection
raise err
File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 70, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/elasticsearch5/connection/http_urllib3.py", line 115, in perform_request
response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, **kw)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 638, in urlopen
_stacktrace=sys.exc_info()[2])
File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 343, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/usr/lib/python3/dist-packages/six.py", line 693, in reraise
raise value
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 600, in urlopen
chunked=chunked)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 354, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/lib/python3.7/http/client.py", line 1244, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.7/http/client.py", line 1290, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.7/http/client.py", line 1239, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.7/http/client.py", line 1026, in _send_output
self.send(msg)
File "/usr/lib/python3.7/http/client.py", line 966, in send
self.connect()
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 181, in connect
conn = self._new_conn()
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 168, in _new_conn
self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f6336cd8c10>: Failed to establish a new connection: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "stockprice.py", line 242, in
es.indices.create(index=args.index, body=mappings, ignore=[400, 404])
File "/usr/local/lib/python3.7/dist-packages/elasticsearch5/client/utils.py", line 73, in _wrapped
return func(*args, params=params, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/elasticsearch5/client/indices.py", line 107, in create
params=params, body=body)
File "/usr/local/lib/python3.7/dist-packages/elasticsearch5/transport.py", line 312, in perform_request
status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
File "/usr/local/lib/python3.7/dist-packages/elasticsearch5/connection/http_urllib3.py", line 124, in perform_request
raise ConnectionError('N/A', str(e), e)
elasticsearch5.exceptions.ConnectionError: ConnectionError(<urllib3.connection.HTTPConnection object at 0x7f6336cd8c10>: Failed to establish a new connection: [Errno 111] Connection refused) caused by: NewConnectionError(<urllib3.connection.HTTPConnection object at 0x7f6336cd8c10>: Failed to establish a new connection: [Errno 111] Connection refused)