Code Monkey home page Code Monkey logo

msl-loadlib's Introduction

MSL-LoadLib

Documentation Status pypi github tests

This package loads a shared library in Python. It is basically just a thin wrapper around ctypes (for libraries that use the __cdecl or __stdcall calling convention), Python for .NET (for libraries that use Microsoft .NET, CLR), Py4J (for Java .jar or .class files) and comtypes (for libraries that use the Component Object Model).

However, the primary advantage is that it is possible to communicate with a 32-bit library from 64-bit Python.

MSL-LoadLib is a pure-python package, but, Python for .NET depends on the .NET Common Language Runtime (CLR) on Windows and Mono Runtime on Linux/macOS, and Py4J depends on having a Java Virtual Machine installed.

Install

To install MSL-LoadLib run:

pip install msl-loadlib

Alternatively, using the MSL Package Manager run:

msl install loadlib

Optional dependencies:

To set up your environment on Linux, please follow the instructions on the prerequisites section of the documentation.

Examples

If you are loading a 64-bit library in 64-bit Python (or a 32-bit library in 32-bit Python), then you can directly load the library using LoadLibrary.

The following examples load a 64-bit library in a 64-bit Python interpreter. If you are using a 32-bit Python interpreter then replace the 64 with 32 in the filename.

Import the LoadLibrary class and the directory where the example libraries are located

>>> from msl.loadlib import LoadLibrary
>>> from msl.examples.loadlib import EXAMPLES_DIR

If the file extension is not included then a default extension, .dll (Windows), .so (Linux) or .dylib (macOS), is used.

Load a C++ library and call the add function

>>> cpp = LoadLibrary(EXAMPLES_DIR + '/cpp_lib64')
>>> cpp.lib.add(1, 2)
3

Load a FORTRAN library and call the factorial function

>>> fortran = LoadLibrary(EXAMPLES_DIR + '/fortran_lib64')

With a FORTRAN library you must pass values by reference using ctypes, and, since the returned value is not of type int we must configure ctypes for a value of type double to be returned

>>> from ctypes import byref, c_int, c_double
>>> fortran.lib.factorial.restype = c_double
>>> fortran.lib.factorial(byref(c_int(37)))
1.3763753091226343e+43

Load Java byte code and call the cos function

>>> java = LoadLibrary(EXAMPLES_DIR + '/Trig.class')
>>> java.lib.Trig.cos(1.234)
0.33046510807172985

Python interacts with the Java Virtual Machine via a local network socket and therefore the connection needs to be closed when you are done using the Java library

>>> java.gateway.shutdown()

Load a .NET library and call the reverse_string function, we must specify that the library type is a .NET library by passing in the 'net' argument

>>> net = LoadLibrary(EXAMPLES_DIR + '/dotnet_lib64.dll', 'net')
>>> net.lib.StringManipulation().reverse_string('abcdefghijklmnopqrstuvwxyz')
'zyxwvutsrqponmlkjihgfedcba'

To load a Component Object Model (COM) library pass in the library's Program ID. NOTE: This example will only work on Windows.

Here we load the FileSystemObject library and include the 'com' argument to indicate that it is a COM library.

>>> com = LoadLibrary('Scripting.FileSystemObject', 'com')

We then use the library to create, edit and close a text file

>>> fp = com.lib.CreateTextFile('a_new_file.txt')
>>> fp.WriteLine('This is a test')
0
>>> fp.Close()
0

Inter-process communication is used to access a 32-bit shared library from a module that is running within a 64-bit Python interpreter. The procedure uses a client-server protocol where the client is a subclass of msl.loadlib.Client64 and the server is a subclass of msl.loadlib.Server32. See the tutorials for examples on how to implement inter-process communication.

Documentation

The documentation for MSL-LoadLib can be found here.

msl-loadlib's People

Contributors

fake-name avatar jborbely avatar thuesdays avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

msl-loadlib's Issues

.NET Assembly with "." in the Namespace

This issue was discovered while investigating issue #6

Running

from msl.loadlib import LoadLibrary

net = LoadLibrary('Microsoft.Dynamics.BusinessConnectorNet.dll', 'net')

for item in dir(net.lib):
    obj = getattr(net.lib, item)
    print(item + '\n  value: {}, type: {}, id: {:#x}'.format(obj, type(obj), id(obj)))

Prints

...
MappedNoAlloc
  value: MappedNoAlloc, type: <class '.MappedNoAlloc'>, id: 0x1b7c3f3f518
Microsoft.Dynamics
  value: <module 'Microsoft'>, type: <class 'CLR.ModuleObject'>, id: 0x1b7ab2a24a8
Microsoft.Dynamics.BusinessConnectorNet
  value: <module 'Microsoft'>, type: <class 'CLR.ModuleObject'>, id: 0x1b7ab2a24a8
NameValueCollectionWrapper
  value: NameValueCollectionWrapper, type: <class 'System.RuntimeType'>, id: 0x1b7c4027c18
OneToNScanner
  value: OneToNScanner, type: <class '.OneToNScanner'>, id: 0x1b7c3fca048
...

The Microsoft.Dynamics and Microsoft.Dynamics.BusinessConnectorNet attributes of net.lib point to the same object, which is the Microsoft module.

Therefore, with the current implementation (v0.3.2), walking through the Assembly.GetTypes() array generates confusing syntax because

  1. The net.lib.Microsoft.Dynamics.BusinessConnectorNet attribute does not exist.
  2. getattr(net.lib, 'Microsoft.Dynamics.BusinessConnectorNet') returns a reference to <module 'Microsoft'> and not the Microsoft.Dynamics.BusinessConnectorNet Namespace.

No Timeout when trying to to execute on the 32-bit server from 64bit client

I faced an issue that despite the fact that I set timeout for invocation of 32-bit server from 64bit client the timeout was not happening. The issue was found in below file and code:

msl\loadlib\client64.py :

        # start the 32-bit server
        self._proc = subprocess.Popen(cmd, stderr=subprocess.PIPE, stdout=subprocess.PIPE)
        try:
            utils.wait_for_server(host, port, timeout)
        except ConnectionTimeoutError as err:
            self._proc.wait() #  line 201

As per the above code even though there is a connection timeout exception occurs the code waits for INFINITY time for the subprocess to be killed as per below code:

lib\subprocess.py:

       def wait(self, timeout=None, endtime=None):  
  ................................................................................
            if timeout is None:
                timeout_millis = _winapi.INFINITE
            else:
                timeout_millis = int(timeout * 1000)
            if self.returncode is None:
                result = _winapi.WaitForSingleObject(self._handle,
                                                    timeout_millis)
                if result == _winapi.WAIT_TIMEOUT:
                    raise TimeoutExpired(self.args, timeout)
                self.returncode = _winapi.GetExitCodeProcess(self._handle)
            return self.returncode

Hence the code waits for infinity time period - I have made a fix for same it works but not sure if it is the correct one:

Just change line 201 in msl\loadlib\client64.py from self._proc.wait() to **self._proc.wait(timeout=timeout)**

Interprocess communication thread-safe?

What is the best way to have multiple threads sending requests to the 32 bit process? I have a multi-threaded 64-bit application that needs to interface with a 32-bit API so I used this module to communicate with a 32-bit process running the 32-bit DLL/API but I notice that when multiple threads send requests, I get a crash with the following message:

 File "E:\Anaconda3\lib\site-packages\msl\loadlib\client64.py", line 265, in re
quest32
    self._conn.request('GET', method32)
  File "E:\Anaconda3\lib\http\client.py", line 1229, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "E:\Anaconda3\lib\http\client.py", line 1240, in _send_request
    self.putrequest(method, url, **skips)
  File "E:\Anaconda3\lib\http\client.py", line 1098, in putrequest
    raise CannotSendRequest(self.__state)
http.client.CannotSendRequest: Request-sent

Any advice on how to implement thread-safe inter-process communication? I was thinking perhaps a thread that queues up commands and sends requests, but I'm not sure where to start. I was hoping the inter-process communication implementation here would be able to do this automatically, but apparently does not.

Version 0.7 works with "Microsoft.Dynamics.BusinessConnectorNet.dll" but 0.8 does not (win10 64bit with py3.6 32bit)

Hey,

thanks for the nice library. I just had to migrate my setup to a new laptop and after some fiddling around I found out that somehow only the older 0.7 version works "out of the box".

Output with version 0.7 on python 3.6 32bit:

from msl.loadlib import LoadLibrary
net = LoadLibrary("Microsoft.Dynamics.BusinessConnectorNet", "net")
Traceback (most recent call last):
File "C:\Python36\lib\site-packages\msl\loadlib\load_library.py", line 230, in init
self._assembly = clr.System.Reflection.Assembly.LoadFile(self._path)
System.IO.FileLoadException: Mixed mode assembly is built against version 'v2.0.50727' of the runtime and cannot be loaded in the 4.0 runtime without additional configuration information.

at System.Reflection.RuntimeAssembly.nLoadFile(String path, Evidence evidence)
at System.Reflection.Assembly.LoadFile(String path)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "", line 1, in
File "C:\Python36\lib\site-packages\msl\loadlib\load_library.py", line 242, in init
raise IOError(msg)
OSError: The library appears to be from a .NET Framework version < 4.0.
The useLegacyV2RuntimeActivationPolicy property was added to C:\Python36\python.exe.config
to fix the "System.IO.FileLoadException: Mixed mode assembly..." error.
Rerun the script, or shutdown and restart the interactive console, to see
if Python can now load the .NET library.

When doing the same thing again it now "just works".

With version 0.8 although, I only always got:

System.IO.FileNotFoundException: Unable to find assembly 'Microsoft.Dynamics.BusinessConnectorNet'.
at Python.Runtime.CLRModule.AddReference(String name)

So apparently no helpful "useLegacyV2RuntimeActivationPolicy property was added to C:\Python36\python.exe.config" magic was done.

Sorry if this is expected behaviour now.

BR.

Installation depends on pythonnet

I am trying to install from pip on a fresh instalation of xubuntu but if fails when trying to install pythonnet.

I was under the impression that .net framework support was optional, but it appears that its a hard requirement.

Connection is refused occasionally

I use msl-load lib within a project of mine. However occasionally it fails connecting client and server with:

self.*conn.request('POST', 'protocol={}&path={}'.format(self.pickleprotocol, self.*pickle_path))
File "http\client.py", line 1239, in request
File "http\client.py", line 1285, in sendrequest
File "http\client.py", line 1234, in endheaders
File "http\client.py", line 1026, in sendoutput
File "http\client.py", line 964, in send
File "http\client.py", line 936, in connect
File "socket.py", line 724, in create_connection
File "socket.py", line 713, in create_connectionConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it

I use the latest release version of ms-lib (0.9.0) with python 3.6 here. The exact same code runs fine like 9 out of 10 times, but occasionally it fails establishing the client serer connection. I was guessing that this is a port issue, but that doesn't seem to be the case, as picking a specific free port leads to the same results. The host is a windows 10 64bit Machine. Any clue what is happening there?

How to properly bundle using pyinstaller?

Hi,

I have been going through the docs for the past week and still unable to figure it out properly.

My project structure is like this (simplified) :
main.py
my_api/
- server.py
- client.py
- some_other_stuff_server_depends_on.py

I (presumably correctly) add the server executable like so: --add-data ".\virt\Lib\site-packages\msl\loadlib\server32-windows.exe;." , but at runtime the server cannot start because the client can't find the server file in a bundle.

I don't think I can just use --add-data to simply add server.py because then I would also have to manually add all the dependencies etc, so I feel like there's a better way. Maybe I misunderstand the way the bundling works.

I have prepared a simple test project to illustrate my setup, which is in the echo_project.zip.

I would really appreciate some pointers. And great work with the library, it's very useful!

Platform: Windows 10
Python version: 3.11
Virtual env: yes

32-bit dll not found (or one of its dependencies)

Hi! It's been a while since I last used your library. It has helped me a lot in the past. Recently, I wanted to use a script that I used to use a lot in the past, and always worked fine, and I kept getting this error:

Instantiating 'PSLInGaAs32' raised the following exception:

Traceback (most recent call last):
  File "PyInstaller\loader\pyimod03_ctypes.py", line 53, in __init__
  File "ctypes\__init__.py", line 376, in __init__
FileNotFoundError: Could not find module 'C:\Generic\Path\To\snakecamlinkcontrol.dll' (or one of its dependencies). Try using the full path with constructor syntax.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "start_server32.py", line 183, in main
  File "C:\Generic\Path\To\PSLInGaAs32.py", line 22, in __init__
    super(PSLInGaAs32, self).__init__("./snakecamlinkcontrol.dll", 'cdll', host, port)
  File "server32.py", line 72, in __init__
  File "load_library.py", line 162, in __init__
  File "PyInstaller\loader\pyimod03_ctypes.py", line 55, in __init__
pyimod03_ctypes.install.<locals>.PyInstallerImportError: Failed to load dynlib/dll 'C:\\Generic\\Path\\To\\snakecamlinkcontrol.dll'. Most likely this dynlib/dll was not found when the application was frozen.

Cannot start the 32-bit server.

After spending quite a few hours to debug the problem, I found out that downgrading from version 0.10.0 to version 0.9.0 solved the issue.
I tried to find what changed, and if it was something I should change on my code to "update" it, but to no avail. So I ask you: what could be the problem? I suspect it is a problem that many people will have, so it would be good to understand what is causing it.

Note: Using Python 3.11.4 on Windows 11.
I also tried using the full path and moving the dll to a "simpler" location, but nothing helped.

Thank you!

Edit: I understand it could possibly be due to how the included 32-bit python was frozen, but honestly, I don't see how a simple user of the library could go on to fix this. Re-freezing seems to be the way to go, but without understanding exactly what to change during the process, it can be a very tough challenge, so downgrading is just simpler and saves a lot of time at this point.

provide conda package

Hi, first of all thank you for this wonderful library!

is there any chance of having this package uploaded also to a conda repository, e.g conda-forge?

Thanks for the attention.

No module named loadlib

Perhaps I'm overlooking something, or don't have the proper dependencies.

Installed are:
python2.7.5
python3.5.3

Error:

Python 2.7.5 (default, Aug  2 2016, 04:20:16) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import msl
>>> from msl.loadlib import *
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named loadlib

Not clear exception

HI, I am trying to connect not net assembly and having the following result:

_____________________________ test_msl_connector ______________________________

    def test_msl_connector():
        """Get Axapta object via msl-loadlib package."""
>       connectorLL = LoadLibrary(DLL_PATH, 'net')

test_main.py:12:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <[AttributeError("'LoadLibrary' object has no attribute '_lib'") raised in repr()] LoadLibrary object at 0
x4152c18>
path = WindowsPath('C:/Users/AZ/Desktop/test_bom-mcs/Microsoft.Dynamics.BusinessConnectorNet.dll')
libtype = 'net'

    def __init__(self, path, libtype='cdll'):

        _path = path

        # a reference to the .NET Runtime Assembly
        self._assembly = None

        # assume a default extension if no extension was provided
        if not os.path.splitext(_path)[1]:
            _path += DEFAULT_EXTENSION

        self._path = os.path.abspath(_path)
        if not os.path.isfile(self._path):
            # for find_library use the original 'path' value since it may be a library name
            # without any prefix like lib, suffix like .so, .dylib or version number
            self._path = ctypes.util.find_library(path)
            if self._path is None:  # then search sys.path and os.environ['PATH']
                success = False
                search_dirs = sys.path + os.environ['PATH'].split(os.pathsep)
                for directory in search_dirs:
                    p = os.path.join(directory, _path)
                    if os.path.isfile(p):
                        self._path = p
                        success = True
                        break
                if not success:
                    raise IOError('Cannot find the shared library "{}"'.format(path))

        if libtype == 'cdll':
            self._lib = ctypes.CDLL(self._path)
        elif libtype == 'windll':
            self._lib = ctypes.WinDLL(self._path)
        elif libtype == 'oledll':
            self._lib = ctypes.OleDLL(self._path)
        elif libtype == 'net' and self.is_pythonnet_installed():
            import clr
            try:
                # By default, pythonnet can only load libraries that are for .NET 4.0+.
                #
                # When MSL-LoadLib is installed, the useLegacyV2RuntimeActivationPolicy
                # property should have been enabled automatically to allow for loading
                # assemblies from previous .NET Framework versions.
                self._assembly = clr.System.Reflection.Assembly.LoadFile(self._path)

            except clr.System.IO.FileLoadException as err:
                # Example error message that can occur if the library is for .NET <4.0,
                # and the useLegacyV2RuntimeActivationPolicy is not enabled:
                #
                # " Mixed mode assembly is built against version 'v2.0.50727' of the
                #  runtime and cannot be loaded in the 4.0 runtime without additional
                #  configuration information. "
                #
                # To solve this problem, a <python-executable>.config file must exist and it must
                # contain a useLegacyV2RuntimeActivationPolicy property that is set to be "true".
                if "Mixed mode assembly" in str(err):
                    status, msg = self.check_dot_net_config(sys.executable)
                    if not status == 0:
                        raise IOError(msg)
                    else:
                        update_msg = 'Checking .NET config returned "{}" '.format(msg)
                        update_msg += 'and still cannot load library.\n'
                        update_msg += str(err)
                        raise IOError(update_msg)
                raise IOError('The above "System.IO.FileLoadException" is not handled.\n')

            # the shared library must be available in sys.path
            head, tail = os.path.split(self._path)
            sys.path.insert(0, head)

            # don't include the library extension
            clr.AddReference(os.path.splitext(tail)[0])

            # import namespaces, create instances of classes or reference a System.Type[] object
            dotnet = {}
>           for t in self._assembly.GetTypes():
E           System.Reflection.ReflectionTypeLoadException: Unable to load one or more of the requested types. Ret
rieve the LoaderExceptions property for more information.
E              at System.Reflection.RuntimeModule.GetTypes(RuntimeModule module)
E              at System.Reflection.Assembly.GetTypes()

..\..\.virtualenvs\test_bom-mcs-bwfslhpz\lib\site-packages\msl\loadlib\load_library.py:132: ReflectionTypeLoadExc
eption

I can't understand it. What happens?

loading alternate server32.exe

Hi All

I have built a custom server32.exe with a more current version of python ect. When I followed the docs it asked to place the files in the site-packages/msl/loadlib folder.
This makes things a little awkward for distributing my solution. I would like to put the new server32.exe file in the same folder as my server32.py file.
Is there a way to specify the filename and location of the server32.exe to use.

Marco

32-bit server not shutting down

I have a Python script using msl-loadlib to access a 32bit-dll for communication with external hardware. Everything works smooth just running this script.

I wanted to add a little tkinter UI for better usability. When calling the script from the GUI, I run into issues, because after execution of the procedure, the 32-bit server will not shut down. This might be intended, since the main thread / GUI is still running. Is there a way to manually call a deconstructor to trigger closing down the server?

Working in virtualenv Client64

Is there any functional problem in Client64.py if I switch from sys.getsitepackages() to sys.path?
Because virtualenv manages it's own version of site pypa/virtualenv/#737
Client64.py Line 127:

      # _append_sys_path = site.getsitepackages()
        _append_sys_path = sys.path
        if append_sys_path is not None:
            if isinstance(append_sys_path, str):
                _append_sys_path.append(append_sys_path.strip())
            elif isinstance(append_sys_path, (list, tuple)):
                _append_sys_path.extend(append_sys_path)
            else:
                raise TypeError('append_sys_path must be a str, list or tuple')
        cmd.extend(['--append-sys-path', ';'.join(_append_sys_path).strip()])

I just blindly tried to change it and it worked in my case.

kill a "zombie" server

The 32-bit server can continue to run even after the program running the client's script has terminated.

Consider fetching some metadata (e.g., process id) from the server when the client connects.

ImportError: bad magic number in 'app': b'U\r\r\n'

Background: I was using msl-loadlib 0.7.0 two years ago with Python 3.6 and in a frozen environment using https://github.com/marcelotduarte/cx_Freeze. Now with msl-loadlib 0.7.0 with a newer Python version 3.8 I stumbled across #21. I subsequently updated msl-loadlib (tested with 0.8.0. and 0.9.0) and now the following code (in spirit of https://msl-loadlib.readthedocs.io/en/latest/interprocess_communication.html):

Client64.__init__(
    self, module32="app.can_wrapper", append_sys_path=vendor_path,
)

errors with

msl.loadlib.exceptions.ConnectionTimeoutError: Timeout after 10.0 seconds. Could not connect to 127.0.0.1:63307
ImportError: bad magic number in 'app': b'U\r\r\n'
The missing module must be in sys.path (see the --append-sys-path option)

The magic number in the exception is the one from the 3.8 env installation:

>>> import importlib.util
>>> importlib.util.MAGIC_NUMBER
b'U\r\r\n'

The code in question works well when executed within a regular Python script (unfrozen env) with all mentioned msl-loadlib versions.

Are there any best practices for using msl-loadlib in frozen environments? Do you have any idea whether it is the import machinery of msl-loadlib or the input machinery of cx_Freeze that is causing this?

edit: I was vaguely recalling something similar and indeed found an old issue I reported with cx_Freeze, please see marcelotduarte/cx_Freeze#525

warning: subprocess is still running (Client64)

The following warning can be displayed

"subprocess %s is still running" % self.pid, ResourceWarning, source=self

This warning results from the way the subprocess.Popen call is invoked in Client64.__init__

Server32Error thrown

I was happily building up a client/server solution to a thirdparty c dll.
I was able to use the msl libray to execute many of the methods in the third party dll (32bit) from within 64 bit python and thought I was making good progress.
I am now dead in my tracks with an exception in Server32

File 'c:\WinPython\WPy64-31110\python-3.11.1.amd64\Lib\site-packages\numpy\init.py', line 124, in
from numpy.config import show as show_config
AttributeError: module 'os' has no attribute 'add_dll_directory'

Oddly I am not calling or using any numpy features in my client or server code which inherits form Client64 or Server32.
Any suggestions as I am at a total loss on this.

Marco

Can't use with External Dlls

I am trying to use this package to interface with 32 bit dll's for a Texas Instruments API, but the server fails to start with the message:

"
raise PyInstallerImportError(name) from base_error
pyimod03_ctypes.install..PyInstallerImportError: Failed to load dynlib/dll 'C:\Users\ahlquist3\Anaconda3\envs\MainEnv\Lib\site-packages\msl\examples\loadlib\PortabilityLayer.dll'. Most likely this dynlib/dll was not found when the application was frozen.

Cannot start the 32-bit server.
"

From numerous stack exchange, this appears to be due to issues with ctypes.cdll() and the suggestions say to refreze the executable with a spec file pointing to the corresponding .dlls needed.

However, when running freeze_server32.main(spec='path-to-spec') it fails with the following info:

2563 INFO: PyInstaller: 4.8
2563 INFO: Python: 3.9.12 (conda)
2585 INFO: Platform: Windows-10-10.0.19045-SP0
2585 INFO: UPX is not available.
2585 INFO: Removing temporary files and cleaning cache in C:\Users\ahlquist3\AppData\Local\pyinstaller
script 'C:\Users\ahlquist3\Anaconda3\Lib\site-packages\msl\loadlib\server32-windows.py' not found
1
b''
Traceback (most recent call last):

File ~\Anaconda3\Lib\site-packages\msl\loadlib\NewServer.py:10 in
freeze_server32.main(spec='C:\Users\ahlquist3\Anaconda3\Lib\site-packages\msl\loadlib\server32-alt.spec')

File ~\Anaconda3\lib\site-packages\msl\loadlib\freeze_server32.py:191 in main
check_call(cmd)

File ~\Anaconda3\lib\subprocess.py:373 in check_call
raise CalledProcessError(retcode, cmd)

CalledProcessError: Command '['C:\Users\ahlquist3\Anaconda3\python.exe', '-m', 'PyInstaller', '--distpath', 'C:\Users\ahlquist3\Anaconda3\Lib\site-packages\msl\loadlib', '--workpath', 'C:\Users\AHLQUI~1\AppData\Local\Temp\tmpciuv22kn', '--noconfirm', '--clean', 'C:\Users\ahlquist3\Anaconda3\Lib\site-packages\msl\loadlib\server32-alt.spec']' returned non-zero exit status 1.

I am eager to find a way around all of these issues - any help is immensely appreciated!

Relevant files for the DLPR200API Installer.exe can be downloaded at: https://dlinnovations.com/resources/product-downloads/ - This includes the PortabilityLayer.dll and other .dll, .lib and .h files

TypeError: cannot pickle 'xxx' object

Durante a utilização da biblioteca usando o recurso de servidor e server tento acessar uma dll .net em 32btis a mesma consegue ser carrega porem ao chamar um objeto acontece um erro.

image

Quando utilizo x64 para x64 o erro não consegue apenas ao utilizar a função x86 para x64

my_client.py

from msl.loadlib import Client64

class MyClient(Client64):
    """Call a function in 'my_lib.dll' via the 'MyServer' wrapper."""

    def __init__(self):
        # Specify the name of the Python module to execute on the 32-bit server (i.e., 'my_server')
        super(MyClient, self).__init__(module32='my_server')

    def INIT_(self):
        return self.request32('library')
    
c = MyClient()

bm = c.INIT_()

INCA = bm.GetOpenedExperiment()

pass

`

my_server.py

from msl.loadlib import Server32
from msl.loadlib import LoadLibrary

class MyServer(Server32):

  def __init__(self, host, port, **kwargs):
      super(MyServer, self).__init__('C:\\ETAS\\INCA7.2\\cebra\\incacom.dll', 'net', host, port)

  def library(self):
      return self.lib.de.etas.cebra.toolAPI.Inca.Inca()

´

UnsupportedOperation: fileno

I am trying to run the example given to "Access a 32-bit library in 64-bit Python". But i am getting this error.

File "c:\programdata\anaconda3\envs\pytorch\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 827, in runfile
execfile(filename, namespace)

File "c:\programdata\anaconda3\envs\pytorch\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 110, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)

File "D:/Coding/c/f/msl-loadlib-master/msl_example.py", line 9, in
cpp=Cpp64()

File "C:\Users\v\AppData\Roaming\Python\Python37\site-packages\msl\examples\loadlib\cpp64.py", line 30, in init
super(Cpp64, self).init(module32='cpp32', append_sys_path=os.path.dirname(file))

File "C:\Users\v\AppData\Roaming\Python\Python37\site-packages\msl\loadlib\client64.py", line 182, in init
self._proc = subprocess.Popen(cmd, stderr=sys.stderr, stdout=sys.stdout)

File "c:\programdata\anaconda3\envs\pytorch\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 143, in init
super(SubprocessPopen, self).init(*args, **kwargs)

File "c:\programdata\anaconda3\envs\pytorch\lib\subprocess.py", line 728, in init
errread, errwrite) = self._get_handles(stdin, stdout, stderr)

File "c:\programdata\anaconda3\envs\pytorch\lib\subprocess.py", line 1057, in _get_handles
c2pwrite = msvcrt.get_osfhandle(stdout.fileno())

UnsupportedOperation: fileno

Please support. Thanks.

Why is having to define the client class even a thing?

I'm struggling to understand the point of having to define the entire ServerXX and ClientXX classes.

Basically, is there any reason you can't basically completely define your ClientXX class like:

import functools
from msl.loadlib import Client64

class ClassName(Client64):
	def __init__(self):
		super().__init__(module32='module_name', append_sys_path=os.path.dirname(__file__))

	def __getattr__(self, name):
		return functools.partial(self.request32, name)

The only thing that might need examining is properly passing exceptions from the client context back (and that appears to work fine), but I don't see any reason to bother duplicating all the methods from the client in the server.

Loading Framework and Core based library at the same time.

I have few library/module created in .NETFramework which I am using in python by loading via pythonnet. Most of them are 64bit but 1 or 2 is 32bit as there were no other option. I am using msl loadlib to do that using via rpa, between 64bit python to 32 bit library. This is working fine so far.

I have a requirement to upgrade all libraries to latest core base due to increased performance and improvements. At the same time I need to ensure backward compatibility. Given this scenario, I learned that I can load either .NETFramework on Core at a time in a python via pythonnet.

Due to this constraint, I am wandering is there any way I can load Core based 64bit library in RPC mode (just like 32bit).
msl loadlib lacks 2 thing to make this solution possible,

  1. mslload lib can not start/available 64 bit based server like Server32
  2. mslload lib itself works only with .NETFramework only, so I can not load Core as far as I am loading msl loadlib.

Please help me to resolve this problem. or is there any other alternative I can try.

Thanks in advance.

AttributeError: 'WindowsPath' object has no attribute 'rfind'

This issue was discovered while investigating issue #6

Instantiating the LoadLibrary class raises an AttributeError if the path parameter is created using the pathlib module and the interpreter is Python 3.5. See here for the reference.

The error results from passing a <class 'pathlib.WindowsPath'> object to os.path.splitext.

repo server32 not a valid exexutable

I did a git pull/install and when I run my code I get an error which oddly enough does not happen if I build my own 32server.

----------------------snip start
Exception has occurred: ConnectionTimeoutError
Timeout after 10.0 seconds. Could not connect to 127.0.0.1:18879
Importing 'CobraPyServer' on the 32-bit server raised the following exception:

Traceback (most recent call last):
File "PyInstaller\loader\pyimod03_ctypes.py", line 77, in init
File "ctypes_init_.py", line 376, in init
OSError: [WinError 193] %1 is not a valid Win32 application

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

File "start_server32.py", line 151, in main
File "importlib_init_.py", line 126, in import_module
File "", line 1206, in _gcd_import
File "", line 1178, in _find_and_load
File "", line 1149, in _find_and_load_unlocked
File "", line 690, in _load_unlocked
File "", line 940, in exec_module
File "", line 241, in call_with_frames_removed
File "C:\Users\ma\OneDrive - RWDI\Documents\Visual Studio Code\Projects\CobraPy\CobraPyServer.py", line 7, in
import numpy as np
File "c:\WinPython\WPy64-31110\python-3.11.1.amd64\Lib\site-packages\numpy_init
.py", line 138, in
from . import _distributor_init
File "c:\WinPython\WPy64-31110\python-3.11.1.amd64\Lib\site-packages\numpy_distributor_init.py", line 26, in
WinDLL(os.path.abspath(filename))
File "PyInstaller\loader\pyimod03_ctypes.py", line 79, in init
pyimod03_ctypes.install..PyInstallerImportError: Failed to load dynlib/dll 'c:\WinPython\WPy64-31110\python-3.11.1.amd64\Lib\site-packages\numpy\.libs\libopenblas.FB5AE2TYXYH2IJRDKGDGQ3XBKLKTF43H.gfortran-win_amd64.dll'. Most likely this dynlib/dll was not found when the application was frozen.

Cannot start the 32-bit server.
File "C:\Users\ma\OneDrive - RWDI\Documents\Visual Studio Code\Projects\CobraPy\CobraPy.py", line 7, in init
super(CobraPy, self).init(module32='CobraPyServer.py')
File "C:\Users\ma\OneDrive - RWDI\Documents\Visual Studio Code\Projects\CobraPy\Example.py", line 13, in
cobraPy = CobraPy()
^^^^^^^^^
msl.loadlib.exceptions.ConnectionTimeoutError: Timeout after 10.0 seconds. Could not connect to 127.0.0.1:18879
Importing 'CobraPyServer' on the 32-bit server raised the following exception:

Traceback (most recent call last):
File "PyInstaller\loader\pyimod03_ctypes.py", line 77, in init
File "ctypes_init_.py", line 376, in init
OSError: [WinError 193] %1 is not a valid Win32 application

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "start_server32.py", line 151, in main
File "importlib_init_.py", line 126, in import_module
File "", line 1206, in _gcd_import
File "", line 1178, in _find_and_load
File "", line 1149, in _find_and_load_unlocked
File "", line 690, in _load_unlocked
File "", line 940, in exec_module
File "", line 241, in call_with_frames_removed
File "C:\Users\ma\OneDrive - RWDI\Documents\Visual Studio Code\Projects\CobraPy\CobraPyServer.py", line 7, in
import numpy as np
File "c:\WinPython\WPy64-31110\python-3.11.1.amd64\Lib\site-packages\numpy_init
.py", line 138, in
from . import _distributor_init
File "c:\WinPython\WPy64-31110\python-3.11.1.amd64\Lib\site-packages\numpy_distributor_init.py", line 26, in
WinDLL(os.path.abspath(filename))
File "PyInstaller\loader\pyimod03_ctypes.py", line 79, in init
pyimod03_ctypes.install..PyInstallerImportError: Failed to load dynlib/dll 'c:\WinPython\WPy64-31110\python-3.11.1.amd64\Lib\site-packages\numpy\.libs\libopenblas.FB5AE2TYXYH2IJRDKGDGQ3XBKLKTF43H.gfortran-win_amd64.dll'. Most likely this dynlib/dll was not found when the application was frozen.

Cannot start the 32-bit server.
--------------------------------------------snip end

How can this be resolved?

Marco

Failing to install module

Hi, could you help me with the installation please?
I'm trying to install it to CentOS 6.8 server with python27 from software collection, but setup is failing at this step:
error: [Errno 2] No such file or directory: '/opt/rh/python27/root/usr/lib64/python2.7/site-packages/msl/loadlib/server32-linux'

But actually all files are located under /opt/rh/python27/root/usr/lib/python2.7/site-packages/msl/ directory, lib, not lib64
Thanks in advance.

Question to labview32.dll

If my Vi has an output labview string - how i get his back to my python64 environment

Input double - Output String (Labview32)
Python 64 ? string?

Cannot start the 32-bit server.

Hi,
I'm writting a project using msl to load a 32-bits librairy in my 64-bits python environment, but I still have this error (see below).
Do you have any idea why I get this error? I tried to re-freeze the 32-bits server without succes.

Error

  File "g:/Projets/Process_Integration/smaract/connexion_smar_client64.py", line 14, in <module>
    smaract = smaract_class_client64()
  File "g:/Projets/Process_Integration/smaract/connexion_smar_client64.py", line 6, in __init__
    super(smaract_class_client64, self).__init__(module32='connexion_smar_server32.py', timeout=30)
  File "C:\Users\lmleb\AppData\Local\Programs\Python\Python37\lib\site-packages\msl\loadlib\client64.py", line 199, in __init__
    utils.wait_for_server(host, port, timeout)
  File "C:\Users\lmleb\AppData\Local\Programs\Python\Python37\lib\site-packages\msl\loadlib\utils.py", line 283, in wait_for_server
    'Timeout after {:.1f} seconds. Could not connect to {}:{}'.format(timeout, host, port)
msl.loadlib.exceptions.ConnectionTimeoutError: Timeout after 30.0 seconds. Could not connect to 127.0.0.1:64299
Instantiating the 32-bit server raised the following exception:
  PyInstallerImportError: Failed to load dynlib/dll 'g:\\Projets\\Process_Integration\\smar\\lib\\Control.dll'. Most likely this dynlib/dll was not found when the application was frozen.
Cannot start the 32-bit server.

Background files

Here are my clien64.py and server64.py files:

  • file_client64.py:
from msl.loadlib import Client64

class smar_class_client64(Client64):
    def __init__(self) -> None:
        super(smar_class_client64, self).__init__(module32='file_server32.py')
        
    def getpos(self): # Method doing things...
        return self.request32('getpos')

if __name__ == "__main__":
    smaract = smar_class_client64()
    print(smar.getpos())
  • file_server32.py:
from msl.loadlib import Server32

class smar_class_server32(Server32):
    def __init__(self, host, port) -> None:
        super(smar_class_server32, self).__init__('lib\\Control.dll', 'cdll', host, port)

    def getpos(self):
        return 0

Environnment

  • Windows 11
  • Python 3.7.9 64 bits
  • VS Code

with Pyinstaller..

Hello
32bit dll is being used by 64-bit Python.
I'm going to turn the program into an exe with a 64-bit installer.

image

When I didn't make an exe and just executed it, it went well.
This error has occurred.Error.
Is there a solution?

pythonnet usage

i noticed this comment:

# pythonnet can only load libraries that are .NET 4.0+

it is possible to load .NET 2.0 assemblies using useLegacyV2RuntimeActivationPolicy="true" in app.config file for the loading executable (e.g. python.exe.config).

python.exe.config associated with wrong python.exe when using a virtual environment

When msl-loadlib is installed in a virtual environment and a .NET library, that was built using .NET < v4.0, is loaded the python.exe.config file that is created is saved in the same directory where python.exe is located for the virtual environment. The python.exe.config file should be saved to the directory where the base python.exe file is located (i.e., the directory of the python executable that was used to create the virtual environment).

Cannot load library with python3.8, but works with python3.6

Hello again :)

Thanks for maintaining this great library still.
I'm wondering if this is something this library can improve/fix or does it have to do with some incompatibility with the newer python version and the dll I try to load.

My code is doing nothing more than:

from msl.loadlib import LoadLibrary

net = LoadLibrary("DLL_name", 'net')
...
 Checking .NET config returned "The useLegacyV2RuntimeActivationPolicy property is enabled" and still cannot load library.
Mixed mode assembly is built against version 'v2.0.50727' of the runtime and cannot be loaded in the 4.0 runtime without additional configuration information.
   at System.Reflection.RuntimeAssembly.nLoadFile(String path, Evidence evidence)
   at System.Reflection.Assembly.LoadFile(String path)

Appreciate any input. 😄

Console window pops up when using pythonw

When using pythonw.exe (instead of python.exe) to make a call, a console window pops up to run the server32-windows.exe process.

You can recreate the behavior using the attached code.
msl_console.txt

When running under python.exe (python.exe msl_console.txt), you get the expected result of the console getting the path, then time twice.

When running under pythonw.exe (pythonw.exe msl_console.txt), the expected result should be that nothing is printed to the active console.
The observed behavior is that a console window pops up and flashes a few times to run the server32-windows.exe process.

This is an issue when developping GUI apps since the users will be bombarded with console windows when using the app
msl_console
.

server32-windows.exe not copied when package is installed on 32-bit python environment

'server32-windows.exe' file is not copied when package is installed on 32-bit python environment. Everything works flawlessly after manual copying of the file. Installation on 64-bit python works and copies the file mentioned.

My aim is to have a code which works during transition from 32-bit to 64-bit on both versions of environment as various teams sharing it can make the transition at different times.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.