nibrag / aiosocks Goto Github PK

View Code? Open in Web Editor NEW

114.0 114.0 24.0 99 KB

SOCKS proxy client for asyncio

License: Apache License 2.0

Python 100.00%

aiosocks's People

Contributors

Stargazers

Watchers

aiosocks's Issues

"limit_per_host" and "enable_cleanup_closed" is not used in aiohttp.TCPConnector anymore

For the ProxyConnector Class,
the super().init() inside init() function is still passing args "limit_per_host" and "enable_cleanup_closed",these args are not used in aiohttp.TCPConnector anymore.

I think these two args should be removed from aiosocks too.

Add support for ssl_handshake_timeout kwarg in create_connection()

As of version 3.7, asyncio.create_connection() has a ssl_handshake_timeout kwarg. This kwarg should be supported by aiosocks.create_connection() and passed through.

aiohttp not work got an unexpected keyword argument 'resolve'

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-2-7e6defeb3e54> in <module>()

~/Documents/pip/qphantom-core/QPhantom/net/__init__.py in __init__(self, retry_limit, retry_pause, timeout, proxy, loop)
     55         if self.proxy is not None:
     56             from aiosocks.connector import ProxyConnector, ProxyClientRequest
---> 57             self.connector = ProxyConnector(remote_resolve=True)
     58         else:
     59             self.connector = None

~/.opt/anaconda/lib/python3.6/site-packages/aiosocks/connector.py in __init__(self, verify_ssl, fingerprint, resolve, use_dns_cache, family, ssl_context, local_addr, resolver, keepalive_timeout, force_close, limit, limit_per_host, enable_cleanup_closed, loop, remote_resolve)
     54             force_close=force_close, limit=limit,  loop=loop,
     55             limit_per_host=limit_per_host, use_dns_cache=use_dns_cache,
---> 56             enable_cleanup_closed=enable_cleanup_closed)
     57 
     58         self._remote_resolve = remote_resolve

TypeError: __init__() got an unexpected keyword argument 'resolve'

https://docs.aiohttp.org/en/stable/client_reference.html#aiohttp.TCPConnector

Support aiohttp 2.3.1 +

aiohttp 2.3 introduced explicit support for ssl.*Error

As well in upcoming 2.3.2 (bugfix release) will be private api changes for Connectors aio-libs/aiohttp#2446

It guess aiosocks integration will be broken

@nibrag What about to move aiosocks connector to aiohttp itself?

aiosocks.LoginAuthenticationFailed is not raised on auth error

Instead, aiosocks.SocksError is raised saying there was an auth error.

AFAICS, this happens because in aiosocks.protocols.BaseSocksProtocol.negotiate, on line 58, exception aiosocks.SocksError is caught, which is a parent of aiosocks.LoginAuthenticationFailed and a brand new aiosocks.SocksError is created and raised.

No idea why that is done but the result is that aiosocks.LoginAuthenticationFailed never actually reaches the code using aiosocks.

Is this intentional?

Импорт ProxyConnector

Добавь возможность импортировать не только так:

from aiosocks.connector import ProxyConnector

но и так:

import aiosocks
a = aiosocks.connector.ProxyConnector

cookies not used in session?

switching from TCPConnector to Socks(5)Connector, I tried: a) session.cookies_jar.update_cookies, b) session.connector.cookies[key] = value

the calls succeeded, but response shows a login page instead.

wondering whether I'm using it improperly or it just ignores cookie...

Https and aiosocks not work if use uvloop

Http + aisocks + uvloop + aiohttp - OK
HttpS + aisocks + uvloop + aiohttp - Fail
aiohttp - 1.0.3
uvloop - 0.5.4
asyncio - 3.4.3

source_traceback: Object created at (most recent call last):
File "./test.py", line 478, in
loop.run_until_complete(main())
File "/usr/local/python/3.5.1/lib/python3.5/site-packages/aiosocks/protocols.py", line 106, in connection_made
super().connection_made(transport)
File "/usr/local/python/3.5.1/lib/python3.5/asyncio/streams.py", line 228, in connection_made
self._loop.create_task(res)
Traceback (most recent call last):
File "uvloop/task.pyx", line 126, in uvloop.loop.BaseTask._fast_step (uvloop/loop.c:99441)
File "/usr/local/python/3.5.1/lib/python3.5/site-packages/aiosocks/protocols.py", line 80, in negotiate
self._transport._closing = True
AttributeError: attribute '_closing' of 'uvloop.loop.UVBaseTransport' objects is not writable

aiohttp is a dependency

I want to use aiosocks but don't need HTTP access in my application. Your module requires aiohttp but does not list it as a dependency, and really it shouldn't be a dependency in my case.

aiohttp should be marked as a dependency of this project (easy), or removed as a requirement in the code (harder, but not impossible).

ProxyConnector is missing 'ttl_dns_cache' parameter

'ttl_dns_cache' was introduced in TCPConnector in aiohttp 2.0.8. It is missing in ProxyConnector.
Pull request #26 proposed.

ImportError: aiosocks.SocksConnector require aiohttp library

aiohttp = 3.6.2
aiosocks = 0.2.6

Traceback (most recent call last):
  File "/Users/joey.orlando/.pyenv/versions/standout_specialties_data_scraping-3.7.0/lib/python3.7/site-packages/aiosocks/connector.py", line 4, in <module>
    from aiohttp.client_exceptions import certificate_errors, ssl_errors
ImportError: cannot import name 'certificate_errors' from 'aiohttp.client_exceptions' (/Users/joey.orlando/.pyenv/versions/standout_specialties_data_scraping-3.7.0/lib/python3.7/site-packages/aiohttp/client_exceptions.py)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "step2.py", line 13, in <module>
    from aiosocks.connector import ProxyConnector, ProxyClientRequest
  File "/Users/joey.orlando/.pyenv/versions/standout_specialties_data_scraping-3.7.0/lib/python3.7/site-packages/aiosocks/connector.py", line 6, in <module>
    raise ImportError('aiosocks.SocksConnector require aiohttp library')
ImportError: aiosocks.SocksConnector require aiohttp library

host/port and sock can not be specified at the same time

create_connection with sock only raises error create_connection() missing 1 required positional argument: 'proxy'.
With proxy as None: proxy must be Socks4Addr() or Socks5Addr() tuple
With both sock and proxy: host/port and sock can not be specified at the same time

PyPI Package Broken

Looks like the setup.py on the PyPI package is broken

pip3.6 install -U aiosocks
Collecting aiosocks
  Downloading aiosocks-0.2.1.tar.gz
    Complete output from command python setup.py egg_info:
    running egg_info
    creating pip-egg-info/aiosocks.egg-info
    writing pip-egg-info/aiosocks.egg-info/PKG-INFO
    writing dependency_links to pip-egg-info/aiosocks.egg-info/dependency_links.txt
    writing top-level names to pip-egg-info/aiosocks.egg-info/top_level.txt
    writing manifest file 'pip-egg-info/aiosocks.egg-info/SOURCES.txt'
    reading manifest file 'pip-egg-info/aiosocks.egg-info/SOURCES.txt'
    writing manifest file 'pip-egg-info/aiosocks.egg-info/SOURCES.txt'
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-_fxbxjzi/aiosocks/setup.py", line 38, in <module>
        s
    NameError: name 's' is not defined

There's no line 38 in the current commit so it was probs a mistake.

aiosocks <-> aiohttp not working

The following code throws this error. Adapted from the example in the readme file.

TypeError: update_proxy() takes 3 positional arguments but 4 were given

import asyncio
import aiohttp
import aiosocks
from aiosocks.connector import ProxyConnector, ProxyClientRequest


async def load_github_main():
    connector = ProxyConnector(remote_resolve=True)
    proxy = 'socks5://127.0.0.1:8910'
    async with aiohttp.ClientSession(connector=connector, request_class=ProxyClientRequest) as session:
        async with session.get('http://github.com/', proxy=proxy) as resp:
            if resp.status == 200:
                print(await resp.text())


if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    loop.run_until_complete(load_github_main())
    loop.close()

Add trove classifiers to add metadata to JSON API

Hi,

Many thanks for the project!

How do you feel about adding trove classifiers to your setup.py about Python versions so API calls can be used to find that information?
e.g. My build system checks https://pypi.org/pypi/aiosocks/json to see if it's Python 3 only or not.

PR coming.

Suggestion

Create a new released of the current version, because the released #0.2.6 have the error "ImportError: aiosocks.SocksConnector require aiohttp library" thank you.

TypeError when remote_resolve is disabled

When remote_resolve is disabled, a TypeError occurs here because aiohttp returns an islice which it erroneously tries to access by index.

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/aiohttp/client.py", line 637, in __aenter__
    self._resp = yield from self._coro
  File "/usr/local/lib/python3.6/site-packages/aiohttp/client.py", line 231, in _request
    conn = yield from self._connector.connect(req)
  File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 378, in connect
    proto = yield from self._create_connection(req)
  File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 685, in _create_connection
    _, proto = yield from self._create_proxy_connection(req)
  File "/usr/local/lib/python3.6/site-packages/aiosocks/connector.py", line 70, in _create_proxy_connection
    return await self._create_socks_connection(req)
  File "/usr/local/lib/python3.6/site-packages/aiosocks/connector.py", line 80, in _create_socks_connection
    dst = dst_hosts[0]['host'], dst_hosts[0]['port']
TypeError: 'itertools.islice' object is not subscriptable

Occasional InvalidStateError

If I put in a loop a function which creates a Thread which connects to proxy (one proxie at each iteration) and disconnects immediately, sometimes I get errors like this:

Exception in callback Future.set_result(True)
handle: <Handle Future.set_result(True) created at D:\!_Work\Python38\lib\site-packages\aiosocks\protocols.py:92>
source_traceback: Object created at (most recent call last):
  File "D:\!_Work\Python38\lib\concurrent\futures\thread.py", line 80, in _worker
    work_item.run()
  File "D:\!_Work\Python38\lib\concurrent\futures\thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "D:\!_Work\Python38\lib\asyncio\runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "D:\!_Work\Python38\lib\asyncio\base_events.py", line 603, in run_until_complete
    self.run_forever()
  File "D:\!_Work\Python38\lib\asyncio\base_events.py", line 570, in run_forever
    self._run_once()
  File "D:\!_Work\Python38\lib\asyncio\base_events.py", line 1851, in _run_once
    handle._run()
  File "D:\!_Work\Python38\lib\asyncio\events.py", line 81, in _run
    self._context.run(self._callback, *self._args)
  File "D:\!_Work\Python38\lib\site-packages\aiosocks\protocols.py", line 92, in negotiate
    self._loop.call_soon(self._waiter.set_result, True)
Traceback (most recent call last):
  File "D:\!_Work\Python38\lib\asyncio\events.py", line 81, in _run
    self._context.run(self._callback, *self._args)
asyncio.exceptions.InvalidStateError: invalid state

For some reason, _waiter is already cancelled when we are trying to set result to it, which lead to error.

aiohttp 3 incompatibility

Please, add aiohttp 3.0.0 support.

Thank you.

Passing socks4/5 connectors to session.<REQUEST> instead of ClientSession(connector=aiosocks.Socks5Addr(...))

I am trying to create a few thousand async requests using socks4/5 servers.
Instead of passing your connectors to ClientSession(), how to pass the proxy to the session.request directly without recreating sessions for every socks proxy i use, like aiohttp allows passing http proxies. Example-
async with aiohttp.ClientSession() as session: async with session.get("http://python.org", proxy="http://some.proxy.com") as resp:

Alternatively, how can i asynchronously create sessions with different proxies for each request.
Here's my code.
`import asyncio
from aiohttp import ClientSession
import time

async def fetch(i, url, session, proxy):
async with session.get(url, proxy=proxy, timeout=60) as resp:
print(i, resp.status)
data = await resp.json()
print(i, data['origin'])
return data

async def bound_fetch(i, sem, url, session, proxy):
# Getter function with semaphore.
async with sem:
await fetch(i, url, session, proxy)

async def run(r):
url = "https://httpbin.org/get?show_env"
tasks = []
sem = asyncio.Semaphore(1000)
proxy = "http://83.239.58.162:8080"

async with ClientSession() as session:
	for i in range(r):
		# print(i)
		task = asyncio.ensure_future(bound_fetch(i, sem, url, session, proxy))
		tasks.append(task)

	responses = asyncio.gather(*tasks)
	await responses

t1 = time.time()
number = 10000
loop = asyncio.get_event_loop()

future = asyncio.ensure_future(run(number))
loop.run_until_complete(future)
print("TIME:",time.time() - t1)`

Missing argument in exception

Failed when connecting to a broken socks5 proxy with aiohttp:

session = aiohttp.ClientSession(
    connector=ProxyConnector(remote_resolve=False),
    request_class=ProxyClientRequest,
    headers={'User-Agent': user_agent}
)
auth = aiosocks.Socks5Auth(login, password=password)
response = await session.get(url, params=params, proxy=proxy_addr, proxy_auth=auth)

Traceback (most recent call last):
  File "/root/vktools/env/lib/python3.6/site-packages/vktools/utils/command.py", line 29, in worker
    loop.run_until_complete(coro_f(*args, **kwargs))
  File "/usr/lib/python3.6/asyncio/base_events.py", line 467, in run_until_complete
    return future.result()
  File "/root/vktools/env/lib/python3.6/site-packages/vktools/cli.py", line 78, in run_group_finder
    vk_session_pool = await vktools.api.create_pool(accounts)
  File "/root/vktools/env/lib/python3.6/site-packages/vktools/api/__init__.py", line 49, in create_pool
    await session.authorize(acc['login'], acc['password'], scope='offline')
  File "/root/vktools/env/lib/python3.6/site-packages/vktools/api/session.py", line 81, in authorize
    doc = await self.get_auth_page(scope)
  File "/root/vktools/env/lib/python3.6/site-packages/vktools/api/session.py", line 116, in get_auth_page
    status, resp_text = await self._driver.get(self.AUTH_URL, params=params)
  File "/root/vktools/env/lib/python3.6/site-packages/vktools/api/drivers.py", line 65, in get
    proxy=self.proxy, proxy_auth=self.auth)
  File "/root/vktools/env/lib/python3.6/site-packages/aiohttp/helpers.py", line 104, in __await__
    ret = yield from self._coro
  File "/root/vktools/env/lib/python3.6/site-packages/aiohttp/client.py", line 269, in _request
    conn = yield from self._connector.connect(req)
  File "/root/vktools/env/lib/python3.6/site-packages/aiohttp/connector.py", line 393, in connect
    proto = yield from self._create_connection(req)
  File "/root/vktools/env/lib/python3.6/site-packages/aiohttp/connector.py", line 738, in _create_connection
    _, proto = yield from self._create_proxy_connection(req)
  File "/root/vktools/env/lib/python3.6/site-packages/aiosocks/connector.py", line 70, in _create_proxy_connection
    return await self._create_socks_connection(req)
  File "/root/vktools/env/lib/python3.6/site-packages/aiosocks/connector.py", line 108, in _create_socks_connection
    raise aiohttp.ClientProxyConnectionError(*exc.args)
TypeError: __init__() missing 1 required positional argument: 'os_error'

TypeError when raise aiohttp.ClientProxyConnectionError

aiohttp: 2.3.2
aiosocks: 0.2.4

in connector.py:108

raise aiohttp.ClientProxyConnectionError(*exc.args)

throw

TypeError: __init__() missing 1 required positional argument: 'os_error'

because aiohttp.ClientProxyConnectionError, which inherited from aiohttp.ClientConnectorError,
has constructor:

    def __init__(self, connection_key, os_error):
        self._conn_key = connection_key
        self._os_error = os_error
        super().__init__(os_error.errno, os_error.strerror)

Does not work it

My code

import asyncio
import aiohttp
import aiosocks
from aiosocks.connector import ProxyConnector, ProxyClientRequest

Error:

Traceback (most recent call last):
  File "F:\Anaconda3\lib\site-packages\aiosocks\connector.py", line 4, in <module>
    from aiohttp.client_exceptions import certificate_errors, ssl_errors
ImportError: cannot import name 'certificate_errors'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "asds.py", line 4, in <module>
    from aiosocks.connector import ProxyConnector, ProxyClientRequest
  File "F:\Anaconda3\lib\site-packages\aiosocks\connector.py", line 6, in <module>
    raise ImportError('aiosocks.SocksConnector require aiohttp library')
ImportError: aiosocks.SocksConnector require aiohttp library

What to do?

First socks request takes much more time when compared to other requests

I've noticed that the first request always takes much more time when compared to the later requests of the same session. Below is the code to reproduce this behaviour:

async def connect_aiosocks(session, url):
    for i in range(10):
        try:
            start_time = time.time()
            async with async_timeout.timeout(7):
                async with session.get(url, proxy=f'socks4://72.11.148.222:56533') as response:
                    print(f'Request {i} {int(round((time.time() - start_time) * 1000))}ms')
        except asyncio.TimeoutError:
            continue
        except Exception as ex:
            print(ex)
            continue

async def fetch_aiosocks():
    conn = ProxyConnector(remote_resolve=False)
    async with aiohttp.ClientSession(connector=conn, request_class=ProxyClientRequest) as session:
        await connect_aiosocks(session, 'http://example.com')

if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    loop.run_until_complete(fetch_aiosocks())

Output is the following:

Is this the intended behaviour?

I've tested this behaviour with the normal aiohttp library with http proxies, and it doesn't happen?

nibrag / aiosocks Goto Github PK

aiosocks's People

Contributors

Stargazers

Watchers

Forkers

aiosocks's Issues

Recommend Projects

Recommend Topics

Recommend Org