AttributeError: module 'types' has no attribute 'ListType' about scrapy-elasticsearch HOT 7 CLOSED

@vreen what version of Python do you use?

from scrapy-elasticsearch.

@vreen I pushed a new build, please give it a test and let me know (0aec4cc)

from scrapy-elasticsearch.

@jayzeng I am using Python 3.5.2
Your commit fixes the 'ListType' issue but heads to the next one:

Traceback (most recent call last):
File "/home/spl/Code/python_env/myenv/lib/python3.5/site-packages/twisted/internet/defer.py", line 587, in _runCallbacks
current.result = callback(current.result, _args, *_kw)
File "/home/spl/Code/python_env/myenv/lib/python3.5/site-packages/scrapyelasticsearch/scrapyelasticsearch.py", line 112, in process_item
self.index_item(item)
File "/home/spl/Code/python_env/myenv/lib/python3.5/site-packages/scrapyelasticsearch/scrapyelasticsearch.py", line 94, in index_item
item_id = hashlib.sha1(unique_key).hexdigest()
TypeError: Unicode-objects must be encoded before hashing

from scrapy-elasticsearch.

@vreen never tested it w/ Python 3, I will make whatever changes required to make it compatible. Stay tuned

from scrapy-elasticsearch.

@vreen pushed the fix 4c35d75 and cut a new release (0.8.5), give it a test and let me know if it works.

from scrapy-elasticsearch.

@jayzeng Line 94 has to be modified accordingly:
item_id = hashlib.sha1(unique_key.encode('utf-8')).hexdigest()

After that, this nice Pipeline works with Python 3.5!
Thanks for having an eye on this.

from scrapy-elasticsearch.

@vreen sorry I linked the wrong commit for the fix, here it is 8aa92f6, it is essentially the same as what you did, except it is added in the get_unique_key(), which is more appricate. Either way, I pushed my change to pypi, feel free to pull the latest.

Have fun :)

from scrapy-elasticsearch.