phpxmlrpc in Docker container wrong charset about phpxmlrpc HOT 10 CLOSED

Hello.

1. I'd go for ubuntu 14.04 or 16.04 if you can - 12.04 is close to EOL (https://www.ubuntu.com/info/release-end-of-life)

2. the php-xmlrpc library does it best to handle charset encoding on its own, in order to be portable. It does not need any os package or php extension to work - but having php extension mbstring enabled is probably a good idea

3. I do not think that Docker has any impact

4. Things that might have an impact:

• the mbstring settings. Do you have it enabled to do transparent encoding/decoding of charset?
• version of php and version of phpxmlrpc in use: at some point both the language and the lib switched to have UTF8 as default 'internal' charset
• how do you use the library? via the legacy-wrapper (I assume), or using the new API (available since phpxmlrpc 4.0)?
• does your app use UTF8 as charset for its internal data, or latin1?

5. is the xmlrpc server you are talking to public? If it is, I could test making calls to it and verify what it sends back...

from phpxmlrpc.

Hi,

thank you for the reply. Let me answer this point-by-point :)

1. we use Ubuntu 12.04, as PHP 5.3 is default there and there is no need to compile it (which failed on multiple occasions with different Ubuntu / PHP versions for us in the past)

2. mbstring is indeed enabled

3. I was more thinking about the underlying OS, whichever that may be. Docker containers are known to be optimized for size, thus will lack some features from normal OS images.

4. this is what I have:

• mbstring transparent encoding, and even function overloading is enabled and set to UTF-8
• PHP version is: PHP 5.3.10-1ubuntu3.26 with Suhosin-Patch (cli) (built: Feb 13 2017 20:37:53)
• phpxmlrpc is the latest from GitHub
• the library, as far as I know, is used in the old style (legacy wrapper), as we used it with version 3x before for some time now
• the app indeed uses UTF-8 charset and our translations are appearing correctly in UTF-8, it's only the communication with XML-RPC server that comes back with the wrong character encoding somehow

5. the server is not public but I'm really almost certain that it's not the cause, as it works with normal OS installations, just not with Docker - so the server is sending the correct data

If you'd be interested, I can attach a Dockerfile for you, so you can build the image and try our the library in Docker directly.

from phpxmlrpc.

At the end, this does not seem a Docker issue. I've just tried to use the library on Ubuntu 16.04 with XAMPP (LAMP) 5.6.28-1 (https://sourceforge.net/projects/xampp/files/XAMPP%20Linux/5.6.28/) and I still get the incorrect charsets. This is with all the settings I mentioned in my previous comment.

Unfortunately, I can't be of much more help, as I've exhausted my debugging capabilities on this issue. If you could try on the same setup, that would be much appreciated.

Thank you and have a nice rest of the day :)

from phpxmlrpc.

I suspect that the problems relate to the fact that you use mbstring configuration to have php transparently convert the character set of the data it receives from the server and/or sends to its backend.

To be honest, the lib was not designed for supporting mbstring transparent conversion. If it ever did in your previous setup, it might have been out of luck rather than conscious design.

As a very quick test, I set mbstring.internal_encoding = UTF-8 and mbstring.func_overload = 7 in php.ini for a php 5.6 installation and immediately a large part of the unit tests broke.

So one more round of questions:

• did you use mbstring transparent conversion in your previous environment as well?
• can you send me the whole set of mbstring settings that you use?
• also, any of the xmlrpc globals that you set in your code (eg. $GLOBALS['xmlrpc_internalencoding'])
• last but not least: it would be nice you could grab a copy of the webserver response, including the http headers (gotten eg. from a network sniffer or by turning up the debug level of the xmlrpc client)

from phpxmlrpc.

Hi, and thank you for your answers and your time :)

I did not use automatic function overloading in the last setup, and neither in this one. Here's a link to my complete phpinfo() - https://jsfiddle.net/h6ude4wd/

This is the way we init RPC connection:

include_once('xmlrpc.inc');

$GLOBALS['xmlrpc_internalencoding'] = 'UTF-8';
$GLOBALS['xmlrpc_defencoding'] = "UTF-8";
$GLOBALS['xmlrpc_null_apache_encoding'] = TRUE;

$this->xml_rpc_connection = new xmlrpc_client(conf('xml_rpc_folder'), conf('xml_rpc_url'), conf('xml_rpc_port'), conf('xml_rpc_protocol'));
$this->xml_rpc_connection->setDebug(0);
$this->xml_rpc_connection->request_charset_encoding='UTF-8';

As for globals, you can see what we set in the code above. And at the moment, I'm unable to easily debug the RPC server (it's all internal-internal communication and I only have a PHP SDK that communicates with the server, and it's a real programming "heaven"... so it's hard for me to even get to any RPC headers right now).

If any of this helps or you still need those headers, please let me know and I'll do my best to provide the information.

Thanks again for your effort in helping me with this issue!

from phpxmlrpc.

As a very quick test, I set mbstring.internal_encoding = UTF-8 and mbstring.func_overload = 7 in php.ini for a php 5.6 installation and immediately a large part of the unit tests broke.

@gggeek perhaps include a script that can be used to verify these sorts of conditions. It would save you and the users of the library a lot of time and effort in debugging these conditions.

from phpxmlrpc.

@martinambrus can you try adding this line in between setting the xmlrpc globals and creating xmlrpc values and doing the call ?

PhpXmlRpc\PhpXmlRpc::importGlobals();

from phpxmlrpc.

@gggeek Did as instructed. Results are still the same.

from phpxmlrpc.

I've just installed PHP 5.6 locally according to this link: https://askubuntu.com/questions/756181/installing-php-5-6-on-xenial-16-04/756186#756186
... and the encoding problem persist. It would seem a colleague of mine, however is on Ubuntu as well and got the library actually working! So, I'm going to compare phpinfo() from his machine with mine and see if I can sniff something out.

from phpxmlrpc.

Solved. It would seem that the following setting in the old functions wrapper, xmlrpc.inc was the cause:

PhpXmlRpc\PhpXmlRpc::$xmlrpc_internalencoding = 'ISO-8859-1'; // old default
... changing the encoding to UTF-8 solved it.
PhpXmlRpc\PhpXmlRpc::$xmlrpc_internalencoding = 'UTF-8'; // old default

Thank you, @gggeek for your support and time devoted to this amazingly peculiar problem!

from phpxmlrpc.