mosquito / aiohttp-xmlrpc Goto Github PK

Hi,

This is a great project and exactly what I am looking for. However when I did some test on server code, it responds inconsistently with python native xmlrpc library. Here is my parameter:

<param>
    <value>
        <i4>100</i4>
    </value>
</param>
<param>
    <value>
        <i4>200</i4>
    </value>
</param>
<param>
    <value>
        <struct>
            <member>
                <name>key1</name>
                <value>
                    <i4>1</i4>
                </value>
            </member>
            <member>
                <name>key2</name>
                <value>
                    <i4>2</i4>
                </value>
            </member>
        </struct>
    </value>
</param>

aiohttp-xmlrpc will parse it as one argument list with 2 elements(100 and 200) and one named argument with 2 keys(key1 and key2), while python3's native xmlrpc will parse it as one argument list with 3 elements ((100, 200, {'key1': 1, 'key2': 2}))
Here is the code I use for native xmlrpc:

from xmlrpc.server import SimpleXMLRPCServer

def ping():
    return 'Pong'

def echo( *args, **kwargs):
    return (args, kwargs)

def test():
    return None

def args(*args):
    return len(args)

def kwargs(**kwargs):
    return len(kwargs)

def args_kwargs(*args, **kwargs):
    print(args)
    print(kwargs)
    return len(args) + len(kwargs)

def exception():
    raise Exception("YEEEEEE!!!")

server = SimpleXMLRPCServer(("localhost", 8080),allow_none=True)
print("Listening on port 8080...")
server.register_function(ping, "ping")
server.register_function(echo, "echo")
server.register_function(test, "test")
server.register_function(args, "args")
server.register_function(kwargs, "kwargs")
server.register_function(args_kwargs, "args_kwargs")

server.serve_forever()

server on aiohttp-xmlrpc one:

from aiohttp import web
from aiohttp_xmlrpc import handler

class XMLRPCExample(handler.XMLRPCView):
    def rpc_ping(self):
        return 'Pong'

    def rpc_echo(self, *args, **kwargs):
        return (args, kwargs)

    def rpc_test(self):
        return None

    def rpc_args(self, *args):
        return len(args)

    def rpc_kwargs(self, **kwargs):
        return len(kwargs)

    def rpc_args_kwargs(self, *args, **kwargs):
        print(args)
        print(kwargs)
        return len(args) + len(kwargs)

    def rpc_exception(self):
        raise Exception("YEEEEEE!!!")

def start_server():
    app = web.Application()
    app.router.add_route('*', '/', XMLRPCExample)
    web.run_app(app)


if __name__ == "__main__":
    start_server()

I called the same method of args_kwargs on client side. But results are 4 for aiohttp-xmlrpc and 3 for xmlrpc.
(BTW, I use https://xmlrpc.devzing.com/ tool to call rpc service which is running locally and tunnelled by ngork).

I think xmlrpc is doing the right way. Maybe there are some configurations I missed or some tests I did incorrectly?

Hello,
I got a response fomr xmlrpc server like this:

<methodResponse>
  <params>
    <param>
      <value>
        <struct>
          <member>
            <name>stdout</name>
            <value>
              <base64>DQpNaWNyb3NvZnQgV2luZG93cyBbVmVyc2lvbiA2LjEuNzYwMV0NCg==</base64>
            </value>
          </member>
          <member>
            <name>stderr</name>
            <value>
              <base64/>
            </value>
          </member>
          <member>
            <name>rc</name>
            <value>
              <i4>0</i4>
            </value>
          </member>
        </struct>
      </value>
    </param>
  </params>
</methodResponse>

But when parsing the response, a TypeError raised as following. Since there's an empty base64 tag in the response, the value passed to Binary.fromstring() was None. Can we just return None or b'' in this scenario (if the value is None)?

Traceback (most recent call last):
  File "check.py", line 258, in <module>
    check_io_tools()
  File "/cygdrive/c/workspaces/io_tools_checker/venv/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/cygdrive/c/workspaces/io_tools_checker/venv/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/cygdrive/c/workspaces/io_tools_checker/venv/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/cygdrive/c/workspaces/io_tools_checker/venv/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "check.py", line 254, in check_io_tools
    loop.run_until_complete(check_io_tools_on_hosts(hosts))
  File "/usr/lib/python3.6/asyncio/base_events.py", line 467, in run_until_complete
    return future.result()
  File "check.py", line 232, in check_io_tools_on_hosts
    await future
  File "/usr/lib/python3.6/asyncio/tasks.py", line 458, in _wait_for_one
    return f.result()  # May raise f.exception().
  File "check.py", line 225, in check_io_tools_on_host
    log('OS: {}'.format(await rpc_h.os))
  File "check.py", line 89, in os
    ret = await self.server['Command.cmd']({'command': ['cmd', '/c', 'ver']})
  File "/cygdrive/c/workspaces/io_tools_checker/venv/lib/python3.6/site-packages/aiohttp_xmlrpc-0.7.3-py3.6.egg/aiohttp_xmlrpc/client.py", line 104, in __remote_call
    return self._parse_response(r, method_name)
  File "/cygdrive/c/workspaces/io_tools_checker/venv/lib/python3.6/site-packages/aiohttp_xmlrpc-0.7.3-py3.6.egg/aiohttp_xmlrpc/client.py", line 74, in _parse_response
    return xml2py(result[0])
  File "/cygdrive/c/workspaces/io_tools_checker/venv/lib/python3.6/site-packages/aiohttp_xmlrpc-0.7.3-py3.6.egg/aiohttp_xmlrpc/common.py", line 164, in xml2py
    return XML2PY_TYPES.get(value.tag)(value)
  File "/cygdrive/c/workspaces/io_tools_checker/venv/lib/python3.6/site-packages/aiohttp_xmlrpc-0.7.3-py3.6.egg/aiohttp_xmlrpc/common.py", line 139, in xml2struct
    map(xml2py, p.xpath("./member/value/*"))
  File "/cygdrive/c/workspaces/io_tools_checker/venv/lib/python3.6/site-packages/aiohttp_xmlrpc-0.7.3-py3.6.egg/aiohttp_xmlrpc/common.py", line 164, in xml2py
    return XML2PY_TYPES.get(value.tag)(value)
  File "/cygdrive/c/workspaces/io_tools_checker/venv/lib/python3.6/site-packages/aiohttp_xmlrpc-0.7.3-py3.6.egg/aiohttp_xmlrpc/common.py", line 151, in <lambda>
    'base64': lambda x: Binary.fromstring(x.text),
  File "/cygdrive/c/workspaces/io_tools_checker/venv/lib/python3.6/site-packages/aiohttp_xmlrpc-0.7.3-py3.6.egg/aiohttp_xmlrpc/common.py", line 26, in fromstring
    return cls(base64.b64decode(data))
  File "/cygdrive/c/workspaces/io_tools_checker/venv/lib/python3.6/base64.py", line 80, in b64decode
    s = _bytes_from_decode_data(s)
  File "/cygdrive/c/workspaces/io_tools_checker/venv/lib/python3.6/base64.py", line 46, in _bytes_from_decode_data
    "string, not %r" % s.__class__.__name__) from None
TypeError: argument should be a bytes-like object or ASCII string, not 'NoneType'
Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x6fffef9c940>
Unclosed connector
connections: ['[(<aiohttp.client_proto.ResponseHandler object at 0x6fffef9c9b0>, 1324445.127690263)]']
connector: <aiohttp.connector.TCPConnector object at 0x6fffef9c9e8>

Although it's nice that the ServerProxy comes with its predefined UserAgent, I'd need to use my own. Could you please make it so it is an argument to __init__?

<methodResponse>
	<params>
		<param>
			<value><int>0</int></value>
		</param>
		<param>
			<value>
				<string>SUCCESS</string>
			</value>
		</param>
		<param>
			<value>
				<int>81880860</int>
			</value>
		</param>
	</params>
</methodResponse>

Schema verification fails and result still be a single value from params (first one).

http://xmlrpc-c.sourceforge.net/introspection.html

The stdlib xmlrpc library has support for system.multicall. It would be nice to have the same feature in this library.

What are the thoughts of dropping 3.4 support and mainly @asyncio.coroutine decorator now we're in 2020 and many versions of Python ahead. This would be replaced with async def coroutine creation.

3.4 is not the default anymore on most popular Linux distros too.

Hi hi,

Thanks for the work of late.

Was there a reason no wheel was uploaded for 1.2.0? If not, I feel uploading one is worth it for some scenarios.

Hi all!

In my company, we are really interested to use aiohttp-xmlrpc library to create a python client talking with a LAVA master.

However, we figured out that aiohttp-xmlrpc seems to not support the "nested remote method calls". Unfortunately, LAVA is using only nested methods as you can see in the link here above.

My first question is do you plan to support that in the future?
My second question is: are you interested in integrating this support if we submit it in a pull requests?

Thank you by advance for your answers!

Library parses local XXE entities but seems does not allow use them inside XML document. See responses from examples/server.py before and after patch that disables entities resolving in lxml:

xxe.xml

<?xml version="1.0"?>
<!DOCTYPE external  [
<!ENTITY ee SYSTEM "file:///dev/random">
]>
<methodCall>
<methodName>x&ee;</methodName>
  <params>
  </params>
</methodCall>

Before patch

$ curl -X POST -H "Content-type: application/xml" -d @xxe.xml http://127.0.0.1:8080/ | grep -Eo "<string>.*</string>"
<string>XMLSyntaxError('Input is not proper UTF-8, indicate encoding !\nBytes: 0xDA 0x08 0xCE 0x99, line 1, column 1',)</string>

After patch

$ curl -X POST -H "Content-type: application/xml" -d @r2-xxe.xml http://127.0.0.1:8080/ | -Eo "<string>.*</string>"
<string>&lt;[-32500] ApplicationError(Method 'x' not found)&gt;</string>

So, attacker has limited abilities to leak portions of binary files, may be something from /proc/self

Patch

diff --git a/aiohttp_xmlrpc/handler.py b/aiohttp_xmlrpc/handler.py
index 5a443c0..809b489 100644
--- a/aiohttp_xmlrpc/handler.py
+++ b/aiohttp_xmlrpc/handler.py
@@ -123,7 +124,8 @@ class XMLRPCView(View):

     @staticmethod
     def _parse_xml(xml_string):
-        root = etree.fromstring(xml_string)
+        parser = etree.XMLParser(resolve_entities=False)
+        root = etree.fromstring(xml_string, parser)
         schema.assertValid(root)
         return root

I'm having difficulty closing the connections cleanly. I'm getting the following errors after calling methods. Mainly the specific client session. I think we need to do that around the post.

2020-01-12 20:09:24,125 INFO: Syncing all packages.
Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x110d400d0>

• I'm manually calling (as I can't use the ServerProxy().close() in the state it's in):

client = ServerProxy(self.xmlrpc_url, loop=self.loop)
...
await client.client.close()

Fixes Needed

It seems later versions of aiohttp the close method has moved to a coroutine. We should fix that.

Would you be open to moving to using the context managers around aiohttp calls as this will probably clean up everything cleanly and remove the error.

Example raw response:

<?xml version="1.0" encoding="ISO-8859-1"?>
<methodResponse>
  <params>
    <param>
      <value>
        <struct>
          <member>
            <name>foo</name>
            <value>
              <string/>
            </value>
          </member>
        </struct>
      </value>
    </param>
  </params>
</methodResponse>

Example output:

{'foo': 'None'}

Expected output:

{'foo': ''}

When I do pip3.5 instal aiohhtp_xmlrpc in version 0.4.0 I get error: FileNotFound(xmlrpc.rng)

May by you forget to add them to install? Because if I add thoise file to install package it's work and work fine =)

Hello,

It seems PyPI is confused by the versioning of this package. On legacy PyPI the last version is 0.7.3 but on warehouse the version is 1.3.

Is also seems like the 1.3 can no longer be installed and is depreciated. This confuses tools like pyup.io. This might be a bug on PyPI or in the package configuration.