Code Monkey home page Code Monkey logo

requests_tor's People

Contributors

deedy5 avatar hs-157 avatar scream4ik avatar sherlynnw-fac avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

requests_tor's Issues

TypeError: get() got an unexpected keyword argument 'params'

Hi,
I have just tried your package. It seems you get request don't accept params, headers and other arguments from requests library.
For example this code:

    r = rt.get("https://oss.uredjenazemlja.hr/rest/katHr/parcelInfo",
                    params={
                        "id": 2786835,
                        "status": encode(number, int(12414714730308))
                    },
                    headers={
                        "Origin": "https://www.katastar.hr",
                        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36"
                    }

returns TypeError: get() got an unexpected keyword argument 'params'

Examples don't run

Python 3.9.5 (tags/v3.9.5:0a7dcbd, May 3 2021, 17:27:52) [MSC v.1928 64 bit (AMD64)] on win32
from requests_tor import RequestsTor
rt = RequestsTor(tor_ports=(9000, 9001, 9002, 9003, 9004), autochange_id=1)
urls = (f'https://habr.com/ru/post/{x}' for x in range(1, 50))
r = rt.get_urls(urls)
print(r[-1].text)
Traceback (most recent call last):
File "", line 6, in
File "C:\Users\jbard\AppData\Local\Programs\Python\Python39\lib\site-packages\requests_tor.py", line 105, in get_urls
temp_results = [resp for resp in
File "C:\Users\jbard\AppData\Local\Programs\Python\Python39\lib\site-packages\requests_tor.py", line 105, in
temp_results = [resp for resp in
File "C:\Users\jbard\AppData\Local\Programs\Python\Python39\lib\concurrent\futures_base.py", line 608, in result_iterator
yield fs.pop().result()
File "C:\Users\jbard\AppData\Local\Programs\Python\Python39\lib\concurrent\futures_base.py", line 445, in result
return self.__get_result()
File "C:\Users\jbard\AppData\Local\Programs\Python\Python39\lib\concurrent\futures_base.py", line 390, in __get_result
raise self._exception
File "C:\Users\jbard\AppData\Local\Programs\Python\Python39\lib\concurrent\futures\thread.py", line 52, in run
result = self.fn(*self.args, **self.kwargs)
File "C:\Users\jbard\AppData\Local\Programs\Python\Python39\lib\site-packages\requests_tor.py", line 77, in get
return self.request(method, url, **kwargs)
File "C:\Users\jbard\AppData\Local\Programs\Python\Python39\lib\site-packages\requests_tor.py", line 69, in request
resp = requests.request(method, url, **kwargs, proxies=proxies)
File "C:\Users\jbard\PycharmProjects\Parallel\venv\lib\site-packages\requests\api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Users\jbard\PycharmProjects\Parallel\venv\lib\site-packages\requests\sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "C:\Users\jbard\PycharmProjects\Parallel\venv\lib\site-packages\requests\sessions.py", line 655, in send
r = adapter.send(request, **kwargs)
File "C:\Users\jbard\PycharmProjects\Parallel\venv\lib\site-packages\requests\adapters.py", line 412, in send
conn = self.get_connection(request.url, proxies)
File "C:\Users\jbard\PycharmProjects\Parallel\venv\lib\site-packages\requests\adapters.py", line 309, in get_connection
proxy_manager = self.proxy_manager_for(proxy)
File "C:\Users\jbard\PycharmProjects\Parallel\venv\lib\site-packages\requests\adapters.py", line 182, in proxy_manager_for
manager = self.proxy_manager[proxy] = SOCKSProxyManager(
File "C:\Users\jbard\PycharmProjects\Parallel\venv\lib\site-packages\requests\adapters.py", line 47, in SOCKSProxyManager
raise InvalidSchema("Missing dependencies for SOCKS support.")
requests.exceptions.InvalidSchema: Missing dependencies for SOCKS support.

Tor ip will change after every God damn request.

Hi, Tor ip will change after every God damn request.

ports = []
for id in range(1000):
    ports.append(5000 + id)

rt = RequestsTor(tor_ports=ports, autochange_id=0)
url = 'https://httpbin.org/anything'
first = rt.get(url)
print(first.text)
second = rt.get(url)
print(second.text)

Will it work in Colab?

I get an error:

from requests_tor import RequestsTor

# If you use the Tor browser
rt = RequestsTor()

# If you use the Tor
# rt = RequestsTor(tor_ports=(9050,), tor_cport=9051)

url = 'https://httpbin.org/anything'
r = rt.get(url)
print(r.text)

urls = ['https://foxnews.com', 'https://nbcnews.com', 'https://wsj.com/news/world',
        'https://abcnews.go.com', 'https://cbsnews.com',  'https://nytimes.com',
        'https://usatoday.com','https://reuters.com/world', 'http://bbc.com/news',
        'https://theguardian.com/world', 'https://cnn.com', 'https://apnews.com']
r = rt.get_urls(urls)
print(r[-1].text)
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
[/usr/local/lib/python3.7/dist-packages/socks.py](https://localhost:8080/#) in connect(self, dest_pair, catch_errors)
    786             # Initial connection to proxy server.
--> 787             super(socksocket, self).connect(proxy_addr)
    788 

OSError: [Errno 99] Cannot assign requested address

During handling of the above exception, another exception occurred:

ProxyConnectionError                      Traceback (most recent call last)
19 frames
[/usr/local/lib/python3.7/dist-packages/urllib3/contrib/socks.py](https://localhost:8080/#) in _new_conn(self)
     87                 timeout=self.timeout,
---> 88                 **extra_kw
     89             )

[/usr/local/lib/python3.7/dist-packages/socks.py](https://localhost:8080/#) in create_connection(dest_pair, timeout, source_address, proxy_type, proxy_addr, proxy_port, proxy_rdns, proxy_username, proxy_password, socket_options)
    208     if err:
--> 209         raise err
    210 

[/usr/local/lib/python3.7/dist-packages/socks.py](https://localhost:8080/#) in create_connection(dest_pair, timeout, source_address, proxy_type, proxy_addr, proxy_port, proxy_rdns, proxy_username, proxy_password, socket_options)
    198 
--> 199             sock.connect((remote_host, remote_port))
    200             return sock

[/usr/local/lib/python3.7/dist-packages/socks.py](https://localhost:8080/#) in wrapper(*args, **kwargs)
     46                 self.setblocking(True)
---> 47             return function(*args, **kwargs)
     48         except Exception as e:

[/usr/local/lib/python3.7/dist-packages/socks.py](https://localhost:8080/#) in connect(self, dest_pair, catch_errors)
    799                 log.debug("%s due to: %s", msg, error)
--> 800                 raise ProxyConnectionError(msg, error)
    801             else:

ProxyConnectionError: Error connecting to SOCKS5 proxy localhost:9150: [Errno 99] Cannot assign requested address

During handling of the above exception, another exception occurred:

NewConnectionError                        Traceback (most recent call last)
[/usr/local/lib/python3.7/dist-packages/urllib3/connectionpool.py](https://localhost:8080/#) in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
    599                                                   body=body, headers=headers,
--> 600                                                   chunked=chunked)
    601 

[/usr/local/lib/python3.7/dist-packages/urllib3/connectionpool.py](https://localhost:8080/#) in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
    342         try:
--> 343             self._validate_conn(conn)
    344         except (SocketTimeout, BaseSSLError) as e:

[/usr/local/lib/python3.7/dist-packages/urllib3/connectionpool.py](https://localhost:8080/#) in _validate_conn(self, conn)
    838         if not getattr(conn, 'sock', None):  # AppEngine might not have  `.sock`
--> 839             conn.connect()
    840 

[/usr/local/lib/python3.7/dist-packages/urllib3/connection.py](https://localhost:8080/#) in connect(self)
    300         # Add certificate verification
--> 301         conn = self._new_conn()
    302         hostname = self.host

[/usr/local/lib/python3.7/dist-packages/urllib3/contrib/socks.py](https://localhost:8080/#) in _new_conn(self)
    109                         self,
--> 110                         "Failed to establish a new connection: %s" % error
    111                     )

NewConnectionError: <urllib3.contrib.socks.SOCKSHTTPSConnection object at 0x7f10527dc7d0>: Failed to establish a new connection: [Errno 99] Cannot assign requested address

During handling of the above exception, another exception occurred:

MaxRetryError                             Traceback (most recent call last)
[/usr/local/lib/python3.7/dist-packages/requests/adapters.py](https://localhost:8080/#) in send(self, request, stream, timeout, verify, cert, proxies)
    498                     retries=self.max_retries,
--> 499                     timeout=timeout,
    500                 )

[/usr/local/lib/python3.7/dist-packages/urllib3/connectionpool.py](https://localhost:8080/#) in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
    637             retries = retries.increment(method, url, error=e, _pool=self,
--> 638                                         _stacktrace=sys.exc_info()[2])
    639             retries.sleep()

[/usr/local/lib/python3.7/dist-packages/urllib3/util/retry.py](https://localhost:8080/#) in increment(self, method, url, response, error, _pool, _stacktrace)
    398         if new_retry.is_exhausted():
--> 399             raise MaxRetryError(_pool, url, error or ResponseError(cause))
    400 

MaxRetryError: SOCKSHTTPSConnectionPool(host='httpbin.org', port=443): Max retries exceeded with url: /anything (Caused by NewConnectionError('<urllib3.contrib.socks.SOCKSHTTPSConnection object at 0x7f10527dc7d0>: Failed to establish a new connection: [Errno 99] Cannot assign requested address'))

During handling of the above exception, another exception occurred:

ConnectionError                           Traceback (most recent call last)
[<ipython-input-3-70877019d4e9>](https://localhost:8080/#) in <module>()
      8 
      9 url = 'https://httpbin.org/anything'
---> 10 r = rt.get(url)
     11 print(r.text)
     12 

[/usr/local/lib/python3.7/dist-packages/requests_tor.py](https://localhost:8080/#) in get(self, url, **kwargs)
     88     def get(self, url, **kwargs):
     89         method = 'GET'
---> 90         return self.request(method, url, **kwargs)
     91 
     92     def post(self, url, **kwargs):

[/usr/local/lib/python3.7/dist-packages/requests_tor.py](https://localhost:8080/#) in request(self, method, url, **kwargs)
     79 
     80         kwargs["headers"] = kwargs.get("headers", TOR_HEADERS)
---> 81         resp = requests.request(method, url, **kwargs, proxies=proxies)
     82         if self.verbose:
     83             print(f"SocksPort={port} status={resp.status_code} url={resp.url}")

[/usr/local/lib/python3.7/dist-packages/requests/api.py](https://localhost:8080/#) in request(method, url, **kwargs)
     57     # cases, and look like a memory leak in others.
     58     with sessions.Session() as session:
---> 59         return session.request(method=method, url=url, **kwargs)
     60 
     61 

[/usr/local/lib/python3.7/dist-packages/requests/sessions.py](https://localhost:8080/#) in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
    585         }
    586         send_kwargs.update(settings)
--> 587         resp = self.send(prep, **send_kwargs)
    588 
    589         return resp

[/usr/local/lib/python3.7/dist-packages/requests/sessions.py](https://localhost:8080/#) in send(self, request, **kwargs)
    699 
    700         # Send the request
--> 701         r = adapter.send(request, **kwargs)
    702 
    703         # Total elapsed time of the request (approximately)

[/usr/local/lib/python3.7/dist-packages/requests/adapters.py](https://localhost:8080/#) in send(self, request, stream, timeout, verify, cert, proxies)
    563                 raise SSLError(e, request=request)
    564 
--> 565             raise ConnectionError(e, request=request)
    566 
    567         except ClosedPoolError as e:

ConnectionError: SOCKSHTTPSConnectionPool(host='httpbin.org', port=443): Max retries exceeded with url: /anything (Caused by NewConnectionError('<urllib3.contrib.socks.SOCKSHTTPSConnection object at 0x7f10527dc7d0>: Failed to establish a new connection: [Errno 99] Cannot assign requested address'))

Not working

# trying code;
from requests_tor import RequestsTor

rt = RequestsTor()

rt.check_ip()

Error ->

�Traceback (most recent call last):
  File "C:\Users\deniz\AppData\Local\Programs\Python\Python310\lib\site-packages\socks.py", line 787, in connect
    super(socksocket, self).connect(proxy_addr)
ConnectionRefusedError: [WinError 10061] The connection could not be established because the target machine actively refused.

Please help.

sending post request concurrently

Is your feature request related to a problem? Please describe.
I want to send post request concurrently like we do in get_urls() function

Describe the solution you'd like
similar to get_urls like to see post_urls function with additional fields like data and headers

Describe alternatives you've considered
If this kind feature can't be implemented can you mention it here that how can i send post request concurrently using RequestTor object.

Additional context
Add any other context or screenshots about the feature request here.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.