avinassh / haxor Goto Github PK

View Code? Open in Web Editor NEW

157.0 7.0 34.0 87 KB

Unofficial Python wrapper for official Hacker News API

License: MIT License

Python 99.28% Makefile 0.72%

haxor's People

Stargazers

Watchers

haxor's Issues

Explore pyproject and poetry

Update requests dependency

Is it possible to make haxor support requests>=2.4.0 to solve this dependency conflict I have?

haxor 1.1 has requirement requests==2.0.0, but you'll have requests 2.18.4 which is incompatible.

ImportError: No module named 'pypandoc'

When installing haxor I get this error:

Complete output from command python setup.py egg_info:
Traceback (most recent call last):
  File "/tmp/pip-install-zl_kpo7s/haxor/setup.py", line 6, in <module>
    import pypandoc
ImportError: No module named 'pypandoc'

The error won’t occur if I install pypandoc before haxor.

The project’s requirements-dev.txt contains pypandoc but requirements.txt does not. Is this intended?

Speeding up performance?

Hi avinassh,

I'm running into performance issues running the examples. It seems the HN API route can be considerably slower than just screen scraping.

Is there a way for me to speed this up? I wonder if the connection to Firebase is being reused--seems like with requests this should be done automatically?

Using timeit here are my results:

for story_id in hn.top_stories(limit=10):
    print hn.get_item(story_id)

timeit: 4.0882999897 sec

who_is_hiring = hn.get_item(8394339)

for comment_id in who_is_hiring.kids:
    comment = hn.get_item(comment_id)
    if 'python' in comment.text.lower():
        print comment.item_id

timeit: 125.318946123 sec

Thanks!

Use PyPI Publish action

https://github.com/marketplace/actions/pypi-publish

Update requests dependency

Consider updating Requests dependency due to security issue which impacts versions <= 2.19.1. More on https://nvd.nist.gov/vuln/detail/CVE-2018-18074.

RecursionError when running tests

Running tests:

pip install -r requirements-dev.txt
python setup.py develop
pytest tests

produces the following error

RecursionError: maximum recursion depth exceeded while calling a Python object

Python version: 3.6.5

Not compatible with python 3.11 due to possible bug in aiohttp

my code:

from hackernews import HackerNews
hn = HackerNews()

output:

:!python /Users/A78751003/projects/py/hn/main.py
Traceback (most recent call last):
  File "/Users/A78751003/projects/py/hn/main.py", line 1, in <module>
    from hackernews import HackerNews
  File "/Users/A78751003/projects/py/hn/.venv/lib/python3.11/site-packages/hackernews/__init__.py", line 19, in <module>
    import aiohttp
  File "/Users/A78751003/projects/py/hn/.venv/lib/python3.11/site-packages/aiohttp/__init__.py", line 6, in <module>
    from .client import *  # noqa
    ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/A78751003/projects/py/hn/.venv/lib/python3.11/site-packages/aiohttp/client.py", line 16, in <module>
    from . import client_exceptions, client_reqrep
  File "/Users/A78751003/projects/py/hn/.venv/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 18, in <module>
    from . import hdrs, helpers, http, multipart, payload
  File "/Users/A78751003/projects/py/hn/.venv/lib/python3.11/site-packages/aiohttp/helpers.py", line 23, in <module>
    import async_timeout
  File "/Users/A78751003/projects/py/hn/.venv/lib/python3.11/site-packages/async_timeout/__init__.py", line 10, in <module>
    class timeout:
  File "/Users/A78751003/projects/py/hn/.venv/lib/python3.11/site-packages/async_timeout/__init__.py", line 40, in timeout
    @asyncio.coroutine
     ^^^^^^^^^^^^^^^^^
AttributeError: module 'asyncio' has no attribute 'coroutine'. Did you mean: 'coroutines'?

According to stackoverflow that is because asyncio dropped the decorator with python 3.11.

can we update the requests version?

it messes with some of my other packages

Outdated version on pip

Hi, I noticed that the version on pip is outdated as it does not support the descendants field added recently.

You might want to update it. Temporarily I can solve it by using the github url as the dependency.

Error while using hn.get_user when expand=True

Whenever I am using usertp = hn.get_user('dhouston', expand=True) it is giving error RuntimeError: This event loop is already running

Missing parentheses in call to 'print' on Python 3.4.3 pip install

Hi, I'm getting the following error trying to install on Python 3.4.3, but it works fine on 2.7.10:

$ pip install haxor

Collecting haxor
  Using cached haxor-0.3.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 20, in <module>
      File "/private/var/folders/rp/8nf53wk57bgd12w6fr31rrwh0000gn/T/pip-build-yuswawpl/haxor/setup.py", line 12
        print long_description
                             ^
    SyntaxError: Missing parentheses in call to 'print'

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/rp/8nf53wk57bgd12w6fr31rrwh0000gn/T/pip-build-yuswawpl/haxor

TimeoutError Top Stories

Calling https://hacker-news.firebaseio.com/v0/topstories.json?print=pretty from within the browser works fine, however if I try to leverage your wrapper

hn = HackerNews()
top_stories = hn.top_stories()
top_stories

I´ll run into ProtocolError: ('Connection aborted.', TimeoutError(10060,...) for most of the time.

Any hints?
Thanks.

`asyncio.get_event_loop()`: DeprecationWarning: There is no current event loop

While running tests, following is printed:

=============================== warnings summary ===============================
tests/test_ask_stories.py::TestAskStories::test_ask_stories
  /home/runner/work/haxor/haxor/hackernews/__init__.py:139: DeprecationWarning: There is no current event loop
    loop = asyncio.get_event_loop()

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

https://github.com/avinassh/haxor/actions/runs/7166888744/job/19511836841

on stackoverflow: https://stackoverflow.com/questions/73361664/asyncio-get-event-loop-deprecationwarning-there-is-no-current-event-loop

Update to latest standards of Python 2024

I am pretty sure I am not using the asyncio as intended
Possibly explore the right ways to do async requests and alternatives
types / mypy support
Move to Poetry for packaging and publishing (#36)

Optimisations where async can be used. e.g.

haxor/hackernews/__init__.py

Lines 193 to 200 in d12bd42

    
           if expand: 
        
               item.by = self.get_user(item.by) 
        
               item.kids = self.get_items_by_ids(item.kids) if item.kids else None 
        
               item.parent = self.get_item(item.parent) if item.parent else None 
        
               item.poll = self.get_item(item.poll) if item.poll else None 
        
               item.parts = ( 
        
                   self.get_items_by_ids(item.parts) if item.parts else None 
        
               )

Issue while fetching comments

hn = HackerNews()
who_is_hiring = hn.get_item(12202865)
for comment_id in who_is_hiring.kids:
    print('processing:- '+str(comment_id))
    comment = hn.get_item(comment_id) #this line caused the error
    if comment.text is not None:
        cleantext = BeautifulSoup(comment.text.lower(),'lxml').text.strip()
        comments.append(cleantext)

Error is:

Traceback (most recent call last):
  File "/anaconda3/anaconda/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 331, in _make_request
    httplib_response = conn.getresponse(buffering=True)
TypeError: getresponse() got an unexpected keyword argument 'buffering'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/anaconda3/anaconda/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 516, in urlopen
    body=body, headers=headers)
  File "/anaconda3/anaconda/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 333, in _make_request
    httplib_response = conn.getresponse()
  File "/anaconda3/anaconda/lib/python3.5/http/client.py", line 1197, in getresponse
    response.begin()
  File "/anaconda3/anaconda/lib/python3.5/http/client.py", line 297, in begin
    version, status, reason = self._read_status()
  File "/anaconda3/anaconda/lib/python3.5/http/client.py", line 266, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/anaconda3/anaconda/lib/python3.5/site-packages/requests/adapters.py", line 362, in send
    timeout=timeout
  File "/anaconda3/anaconda/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 559, in urlopen
    _pool=self, _stacktrace=stacktrace)
  File "/anaconda3/anaconda/lib/python3.5/site-packages/requests/packages/urllib3/util/retry.py", line 245, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/anaconda3/anaconda/lib/python3.5/site-packages/requests/packages/urllib3/packages/six.py", line 309, in reraise
    raise value.with_traceback(tb)
  File "/anaconda3/anaconda/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 516, in urlopen
    body=body, headers=headers)
  File "/anaconda3/anaconda/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 333, in _make_request
    httplib_response = conn.getresponse()
  File "/anaconda3/anaconda/lib/python3.5/http/client.py", line 1197, in getresponse
    response.begin()
  File "/anaconda3/anaconda/lib/python3.5/http/client.py", line 297, in begin
    version, status, reason = self._read_status()
  File "/anaconda3/anaconda/lib/python3.5/http/client.py", line 266, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
requests.packages.urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))

During handling of the above exception, another exception occurred:

	if expand:
	item.by = self.get_user(item.by)
	item.kids = self.get_items_by_ids(item.kids) if item.kids else None
	item.parent = self.get_item(item.parent) if item.parent else None
	item.poll = self.get_item(item.poll) if item.poll else None
	item.parts = (
	self.get_items_by_ids(item.parts) if item.parts else None
	)

avinassh / haxor Goto Github PK

haxor's People

Stargazers

Watchers

Forkers

haxor's Issues

Recommend Projects

Recommend Topics

Recommend Org