Simple REST client for python 3.8+
License: MIT License
Simple REST client for python 3.8+.
pip install simple-rest-clientCheck out documentation at Read the Docs website.
The client allows you to set json_encode_body on both API (global to the API?) and on the resources (local to the resource?). This works fine in a scenario where you have many non-JSON resources and a few JSON resources. In that case you can set json_encode_body=False in the API and True in the resources.
The problem, due to this line is that you can't do the opposite. If the encoding is set to True in the API then all resources use JSON encoding.
An option would be to default the argument to None:
def add_resource(
self,
...
json_encode_body=None,
):
...
resource = resource_class(
...
json_encode_body=json_encode_body if json_encode_body is not None or self.json_encode_body
)Slightly less elegant and slightly less consistent with the other calls. A better way, if it works, is to default the argument to whatever the API flag is:
def add_resource(
self,
...
json_encode_body=self.json_encode_body,
):
...
resource = resource_class(
...
json_encode_body=json_encode_body
)I have api url with dash, like "api/device-types"
So impossibe to use api.device-types.list
may be add something like resource_url parameter?
I'm having problems with big response payloads
<class 'httpcore.RemoteProtocolError'> - peer closed connection without sending complete message body
No matter what value I set in the timeout parameter I always get the same error.
For example with 100 or 10000000 the requests always stop at the same time
I have updated to 1.0.6 and find myself not having any builtin exception being returned when the remote connection is impossible.
I tested how it react when shutting down my RESt server but it throws
Traceback (most recent call last):
File "/home/god_kane/Documents/virtualenv/flagship-client/lib/python3.6/site-packages/flask/app.py", line 2446, in wsgi_app
response = self.full_dispatch_request()
File "/home/god_kane/Documents/virtualenv/flagship-client/lib/python3.6/site-packages/flask/app.py", line 1951, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/home/god_kane/Documents/virtualenv/flagship-client/lib/python3.6/site-packages/flask/app.py", line 1820, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/home/god_kane/Documents/virtualenv/flagship-client/lib/python3.6/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/home/god_kane/Documents/virtualenv/flagship-client/lib/python3.6/site-packages/flask/app.py", line 1949, in full_dispatch_request
rv = self.dispatch_request()
File "/home/god_kane/Documents/virtualenv/flagship-client/lib/python3.6/site-packages/flask/app.py", line 1935, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/home/god_kane/Documents/python_projects/flagship-client/routes/login.py", line 26, in auth_customer
if rest.authenticate(form.customer_login.data, form.customer_pass.data, allow_admin=False):
File "/home/god_kane/Documents/virtualenv/flagship-client/lib/python3.6/site-packages/flagshipcore/flagship/libserver.py", line 126, in authenticate
'login': login, 'pass': password, 'customer': allow_customer, 'admin': allow_admin})
File "/home/god_kane/Documents/virtualenv/flagship-client/lib/python3.6/site-packages/simple_rest_client/resource.py", line 101, in action_method
return make_request(self.client, request)
File "/home/god_kane/Documents/virtualenv/flagship-client/lib/python3.6/site-packages/simple_rest_client/decorators.py", line 28, in wrapper
response = f(*args, **kwargs)
File "/home/god_kane/Documents/virtualenv/flagship-client/lib/python3.6/site-packages/simple_rest_client/request.py", line 25, in make_request
client_response = client_method(request.url, **client_options)
File "/home/god_kane/Documents/virtualenv/flagship-client/lib/python3.6/site-packages/httpx/_client.py", line 835, in post
timeout=timeout,
File "/home/god_kane/Documents/virtualenv/flagship-client/lib/python3.6/site-packages/httpx/_client.py", line 601, in request
request, auth=auth, allow_redirects=allow_redirects, timeout=timeout,
File "/home/god_kane/Documents/virtualenv/flagship-client/lib/python3.6/site-packages/httpx/_client.py", line 621, in send
request, auth=auth, timeout=timeout, allow_redirects=allow_redirects,
File "/home/god_kane/Documents/virtualenv/flagship-client/lib/python3.6/site-packages/httpx/_client.py", line 648, in send_handling_redirects
request, auth=auth, timeout=timeout, history=history
File "/home/god_kane/Documents/virtualenv/flagship-client/lib/python3.6/site-packages/httpx/_client.py", line 684, in send_handling_auth
response = self.send_single_request(request, timeout)
File "/home/god_kane/Documents/virtualenv/flagship-client/lib/python3.6/site-packages/httpx/_client.py", line 719, in send_single_request
timeout=timeout.as_dict(),
File "/home/god_kane/Documents/virtualenv/flagship-client/lib/python3.6/site-packages/httpcore/_sync/connection_pool.py", line 153, in request
method, url, headers=headers, stream=stream, timeout=timeout
File "/home/god_kane/Documents/virtualenv/flagship-client/lib/python3.6/site-packages/httpcore/_sync/connection.py", line 65, in request
self.socket = self._open_socket(timeout)
File "/home/god_kane/Documents/virtualenv/flagship-client/lib/python3.6/site-packages/httpcore/_sync/connection.py", line 86, in _open_socket
hostname, port, ssl_context, timeout
File "/home/god_kane/Documents/virtualenv/flagship-client/lib/python3.6/site-packages/httpcore/_backends/sync.py", line 139, in open_tcp_stream
return SyncSocketStream(sock=sock)
File "/usr/lib/python3.6/contextlib.py", line 99, in __exit__
self.gen.throw(type, value, traceback)
File "/home/god_kane/Documents/virtualenv/flagship-client/lib/python3.6/site-packages/httpcore/_exceptions.py", line 12, in map_exceptions
raise to_exc(exc) from None
httpcore._exceptions.ConnectError: [Errno 111] Connection refused
Why no simple_rest_client.exceptions is returned at all?
Python is 3.6
httpcore is 0.9.1
simple-rest is 1.0.6
It would be great to be able to add comments for methods and class and have the ability to get the documentation from __doc__ method attribut. Something like this:
class UserResource(Resource):
"""
My doc...
"""
actions = {
"search": {"method": "GET", "url": "users/search","doc":"Search users"},
"retrieve": {"method": "GET", "url": "users/{}", "doc": "Search a user X. X should be a username"},
}
instagram_api.add_resource(resource_name="users", resource_class=UserResource)
help(instagram_api.users)
help(instagram_api.users.search)
help(instagram_api.users.retrieve)Hi,
This is not an issue but feature request :)
Could you add proxy support? It will be easier to debug requests.
Thank you in advance.
Problem:
recently there was resource name mangling added to add_resource() method which blocks usingn this lib for third-party API with resource names with hyphens.
Example:
Valid Resource URI:
http://some-rest-api/resource-name
what lib does:
from simple_rest_client.api import API
api = API('http://some-rest-api/')
api.add_resource(resource_name='resource-name')
api.get_resource_list()
# ['resource_name']
api.resource_name.list()
# Error on GET for url 'http://some-rest-api/resource_name'
I understand why it was done,
but this completely breaks compatibility of this lib with any ext API which uses such resource names.
So maybe this mechanics should be converted into an additional helper - add "sanitized" attribute only if there is no such already?
I am working with large size json body and i have encountered that python-simple-rest-client just takes around *4 memory than i would expected.
I have decided to reimplement my http client using requests directly and looks like i have solved the problem.
After debugging i cam to realise that the pure Json library is ineffective in terms of memory and this is what caused my memory to blow.
I read your code and looks like you are using json_encoder which also uses the pure json lib (unless you choose to change it - which in your case you didn't).
My recommendation:
It would be great a way to add a custom user agent to the client
Hi, Thanks for your beautiful lib.
In the file upload example, there is maybe a little error. If I understood correctly, content is passed to create function using the "body" named argument.
in the example, we found post_test_server_api.file_upload.create(files=files) intead of post_test_server_api.file_upload.create(body=files)
Hi.
Thanks for your great work. It helps me a lot.
But now I need some help. I‘d want to make a request to a api who needs a special form of authentication. In the request header I have to send a signature who is generated on every request.
Here is the copy for generating the signature. Input is an requests.Request.prepare() object.
def prepare_request(request, adp_token, private_key):
url_parsed = urlparse(request.url)
path = url_parsed[2]
query = url_parsed[4]
url = path[:]
if query:
url += f"?{query}"
headers = sign_request(
url, request.method, request.body, adp_token, private_key
)
for item in headers:
request.headers[item] = headers[item]
return request
def sign_request(url, method, body, adp_token, private_key):
date = datetime.utcnow().isoformat("T") + "Z"
data = f"{method}\n{url}\n{date}\n"
if body:
data += body
data += "\n"
data += adp_token
key = rsa.PrivateKey.load_pkcs1(private_key)
cipher = rsa.pkcs1.sign(data.encode(), key, "SHA-256")
signed_encoded = base64.b64encode(cipher)
signature = f"{signed_encoded.decode()}:{date}"
return {
"x-adp-token": adp_token,
"x-adp-alg": "SHA256withRSA:1.0",
"x-adp-signature": signature
}
Is it possible to make such requests?
@zmbbb report:
Hey, really like your lib and the elegant interface.
I encountered an exception (see below) when I received an empty response from the server, in this case a "204 NO CONTENT". The lib forwarded the empty response to JSON for decoding which raised the exception. As a quick fix, I added a check whether client_response.text is True. This works for me, for empty and non-empty responses.
The exception:
[...]
File "/usr/local/lib/python3.5/dist-packages/simple_rest_client/resource.py", line 107, in action_method
return make_request(self.session, request)
File "/usr/local/lib/python3.5/dist-packages/simple_rest_client/decorators.py", line 39, in wrapper
response = f(*args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/simple_rest_client/request.py", line 30, in make_request
body = json.loads(client_response.text)
File "/usr/local/lib/python3.5/dist-packages/json_encoder/json/init.py", line 229, in loads
**kw
File "/usr/lib/python3.5/json/init.py", line 332, in loads
return cls(**kw).decode(s)
File "/usr/lib/python3.5/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.5/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Hi there! Thanks for this package! ❤️
Just troubleshot an error like this:
Traceback (most recent call last):
File "parse-email-config.py", line 45, in <module>
response = mailcow_api.aliases.list()
File "/Users/patcon/.virtualenvs/hypha-demo-emails-ZwNMEhuU/lib/python3.6/site-packages/simple_rest_client/resource.py", line 102, in action_method
return make_request(self.client, request)
File "/Users/patcon/.virtualenvs/hypha-demo-emails-ZwNMEhuU/lib/python3.6/site-packages/simple_rest_client/decorators.py", line 28, in wrapper
response = f(*args, **kwargs)
File "/Users/patcon/.virtualenvs/hypha-demo-emails-ZwNMEhuU/lib/python3.6/site-packages/simple_rest_client/request.py", line 27, in make_request
content_type = client_response.headers.get("Content-Type", "")
AttributeError: 'coroutine' object has no attribute 'headers'
sys:1: RuntimeWarning: coroutine 'Client.get' was never awaited
Seems it's resolved by locking your version below 0.8.0, which was released 8 hours ago :)
https://github.com/allisson/python-simple-rest-client/blob/master/setup.py#L28
Hi, thanks for this library, it looks really promising!
Do you plan to add support for file uploads, similarly to what unittest provides?
self.client.post("/upload", data={
"name": "filename",
"file": file
}, headers={})Currently, when I try to do this, I get an error:
TypeError: <tempfile._TemporaryFileWrapper object at 0x7f797f6a44e0> is not JSON serializable
I'd be happy to write a PR if you'd review it.
File "/usr/local/lib/python3.7/site-packages/simple_rest_client/api.py", line 3, in
from .resource import Resource
File "/usr/local/lib/python3.7/site-packages/simple_rest_client/resource.py", line 8, in
from .request import make_async_request, make_request
File "/usr/local/lib/python3.7/site-packages/simple_rest_client/request.py", line 3, in
from .decorators import handle_async_request_error, handle_request_error
File "/usr/local/lib/python3.7/site-packages/simple_rest_client/decorators.py", line 5, in
from httpx import exceptions
ImportError: cannot import name 'exceptions' from 'httpx' (/usr/local/lib/python3.7/site-packages/httpx/init.py)
Can you expose the auth support from the underlying requests library through this?
NameError: name 'NotFoundError' is not defined
Great project @allisson. How do I extend a Resource's actions without replacing the default endpoints entirely? I.e. I've seen the documentation for the Github example with create EventResource with custom actions -- but defining the actions that way eliminates the default create/destroy etc.
API(api_root_url='https://test.loc/api')I expect one of 2 options:
https://test.loc/api/resource).I like the first one a lot better. If there are any objections, please shout.
A resource URL without root URL path is used, i.e. https://test.loc/resource.
This line.
A common form of authentication for several web APIs is OAuth. The requests-oauthlib library allows Requests users to easily make OAuth 1
Hello,
I came across your library, it looks nice, I wonder if there is any way to limit the maximum of concurrent requests sent to a server?
My use case, is that servers I target have per IP concurrent requests limit and while client side a single user could send multiple requests in parallel or multiple users could send multiple requests from the same machine.
That would be for this library https://github.com/SciQLop/spwc.
Best regards,
Alexis.
I am trying to connect to a REST API with this client but I get this error when I try to list projects. What does this error mean? When I use curl it works fine.
Code:
api = API(
api_root_url='...api_url...',
params={},
headers={
'Authorization': 'Bearer [...]')
},
timeout=30,
append_slash=False,
json_encode_body=True,
)
api.add_resource(resource_name='projects')
api.projects.actions # runs ok!
api.projects.list() # Failure!
Error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/anaconda3/envs/test/lib/python3.8/site-packages/simple_rest_client/resource.py", line 101, in action_method
return make_request(self.client, request)
File "/usr/local/anaconda3/envs/test/lib/python3.8/site-packages/simple_rest_client/decorators.py", line 35, in wrapper
response = f(*args, **kwargs)
File "/usr/local/anaconda3/envs/test/lib/python3.8/site-packages/simple_rest_client/request.py", line 25, in make_request
client_response = client_method(request.url, **client_options)
File "/usr/local/anaconda3/envs/test/lib/python3.8/site-packages/httpx/_client.py", line 804, in get
return self.request(
File "/usr/local/anaconda3/envs/test/lib/python3.8/site-packages/httpx/_client.py", line 640, in request
return self.send(
File "/usr/local/anaconda3/envs/test/lib/python3.8/site-packages/httpx/_client.py", line 670, in send
response = self._send_handling_redirects(
File "/usr/local/anaconda3/envs/test/lib/python3.8/site-packages/httpx/_client.py", line 699, in _send_handling_redirects
response = self._send_handling_auth(
File "/usr/local/anaconda3/envs/test/lib/python3.8/site-packages/httpx/_client.py", line 736, in _send_handling_auth
response = self._send_single_request(request, timeout)
File "/usr/local/anaconda3/envs/test/lib/python3.8/site-packages/httpx/_client.py", line 759, in _send_single_request
(
File "/usr/local/anaconda3/envs/test/lib/python3.8/contextlib.py", line 131, in __exit__
self.gen.throw(type, value, traceback)
File "/usr/local/anaconda3/envs/test/lib/python3.8/site-packages/httpx/_exceptions.py", line 359, in map_exceptions
raise mapped_exc(message, **kwargs) from None # type: ignore
httpx._exceptions.ProxyError: 503 Error
How to include authentication in the requests being sent. Is there an example I can refer to?
Appreciate your help on this.
Due to httpx 0.19.0 stricter enforcement around client scoping it's no longer possible to do two subsequent requests since the resource is reusing a closed client.
I'm not sure if the best fix is to create new httpx instances each time (allows for concurrent calls) or to just not use the context manager like they did here.
Just installed python-simple-rest-client by using pip inside a virtualenv and tried the following:
(venv) [alceu@localhost harbor-cleanup]$ python
Python 3.4.5 (default, Dec 11 2017, 14:22:24)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-16)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from simple_rest_client.api import API
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/alceu/Projetos/harbor-cleanup/venv/lib/python3.4/site-packages/simple_rest_client/api.py", line 1, in <module>
from .resource import Resource
File "/home/alceu/Projetos/harbor-cleanup/venv/lib/python3.4/site-packages/simple_rest_client/resource.py", line 119
async def action_method(self, *args, body=None, params=None,
^
SyntaxError: invalid syntax
>>>
I'm using Python 3.4.5 and Virtualenv 15.1.0.
Hi! I've been using simple-rest-client for https://github.com/dearkafka/woot and I liked the logic so far, but I just hacked it a little bit for better __repr__ and stuff. But I was thinking that maybe how async is currently made is not the best way? the thing is right now async client will be closed every time and that means that user need to re-init presumable the whole class of api with resource, right? that does not sound so great, also sync logic allows to be re-used easily. I hacked this too in my repo but thats not so elegant since I wanted to avoid redefining your own code (https://github.com/dearkafka/woot/blob/aebb9d5051fe73101e0ab01ac335d12aa044160d/woot/resources.py#L137)
what do you think about this issue?
The primary key in example is ONLY one field:
response = api.users.retrieve(2, body=None, params={}, headers={})
How to code if the the primary key combinate two fields?
I am using httpx 0.13.1 and 0.13.2, and simple-rest seems to have trouble translating their TimeoutException:
File "env/lib/python3.8/site-packages/simple_rest_client/resource.py", line 131, in action_method
return await make_async_request(client, request)
File "env/lib/python3.8/site-packages/simple_rest_client/decorators.py", line 44, in wrapper
except (httpx.TimeoutException,) as exc:
AttributeError: module 'httpx' has no attribute 'TimeoutException'
The original exception thrown is:
File "env/lib/python3.8/site-packages/httpx/_client.py", line 1222, in send_handling_auth
response = await self.send_single_request(request, timeout)
File "env/lib/python3.8/site-packages/httpx/_client.py", line 1254, in send_single_request
) = await transport.request(
File "env/lib/python3.8/site-packages/httpcore/_async/connection_pool.py", line 152, in request
response = await connection.request(
File "env/lib/python3.8/site-packages/httpcore/_async/connection.py", line 65, in request
self.socket = await self._open_socket(timeout)
File "env/lib/python3.8/site-packages/httpcore/_async/connection.py", line 85, in _open_socket
return await self.backend.open_tcp_stream(
File "env/lib/python3.8/site-packages/httpcore/_backends/auto.py", line 38, in open_tcp_stream
return await self.backend.open_tcp_stream(hostname, port, ssl_context, timeout)
File "env/lib/python3.8/site-packages/httpcore/_backends/asyncio.py", line 233, in open_tcp_stream
return SocketStream(
File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__
self.gen.throw(type, value, traceback)
File "env/lib/python3.8/site-packages/httpcore/_exceptions.py", line 12, in map_exceptions
raise to_exc(exc) from None
httpcore._exceptions.ConnectTimeout
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.