zeromq / pyczmq Goto Github PK

Which i assume was removed in 303bf148e1fb19f25276f33e55ba37b522ed7230

$ python ./test_zcert.py
Traceback (most recent call last):
  File "./test_zcert.py", line 7, in <module>
    from pyczmq import zctx, zcert
  File "/usr/local/lib/python2.7/dist-packages/pyczmq/__init__.py", line 3, in <module>
    from . import (
  File "/usr/local/lib/python2.7/dist-packages/pyczmq/zsocket.py", line 215, in <module>
    probe_router = C.zsocket_probe_router
  File "/usr/local/lib/python2.7/dist-packages/cffi/api.py", line 463, in __getattr__
    make_accessor(name)
  File "/usr/local/lib/python2.7/dist-packages/cffi/api.py", line 459, in make_accessor
    make_accessor_locked(name)
  File "/usr/local/lib/python2.7/dist-packages/cffi/api.py", line 422, in make_accessor_locked
    raise AttributeError('%s: %s' % (name, e))
AttributeError: zsocket_probe_router: "function 'zsocket_probe_router' not found in library 'libczmq.so.1': /usr/lib/x86_64-linux-gnu/libczmq.so.1: undefined symbol: zsocket_probe_router"

Error:
c:\local\python36>.\python.exe -tt c:\temp\sample.py
Traceback (most recent call last):
File "c:\temp\sample.py", line 1, in
from pyczmq import zctx, zmsg, zpoller, zsocket, zframe
File "c:\local\python36\lib\site-packages\pyczmq_init_.py", line 1, in
from _cffi import C, ffi
ModuleNotFoundError: No module named '_cffi'

To reproduce, a 1 line file with statement:
from pyczmq import zctx, zmsg, zpoller, zsocket, zframe

OS: Windows 10 Pro, AMD64
Python: 3.6.8 and 3.7.3

pip report in 3.7.3 env, lists cffi module exists:
Package Version

cffi 1.12.3
h5py 2.9.0
nose 1.3.7
numpy 1.16.4
pip 19.1.1
pycparser 2.19
pyczmq 0.0.4
pyzmq 18.0.2
setuptools 41.0.1
six 1.12.0

pip verbose for CFFI shows supported envs:
Name: cffi
Version: 1.12.3
Summary: Foreign Function Interface for Python calling C code.
Home-page: http://cffi.readthedocs.org
Author: Armin Rigo, Maciej Fijalkowski
Author-email: [email protected]
License: MIT
Location: c:\local\python\lib\site-packages
Requires: pycparser
Required-by: pyczmq
Metadata-Version: 2.1
Installer: pip
Classifiers:
Programming Language :: Python
Programming Language :: Python :: 2
Programming Language :: Python :: 2.6
Programming Language :: Python :: 2.7
Programming Language :: Python :: 3
Programming Language :: Python :: 3.2
Programming Language :: Python :: 3.3
Programming Language :: Python :: 3.4
Programming Language :: Python :: 3.5
Programming Language :: Python :: 3.6
Programming Language :: Python :: Implementation :: CPython
Programming Language :: Python :: Implementation :: PyPy

Tried to run the same 1-line sample.py file with python 3.6.8 -- same problem.

Adding pip packages dir to path explicitly does not solve the issue.
set PATH=c:\local\python36\Lib\site-packages;%PATH%

output of pip verbose cffi under 3.6.8:
c:\local\python36>Scripts\pip.exe show --verbose -f cffi
Name: cffi
Version: 1.12.3
Summary: Foreign Function Interface for Python calling C code.
Home-page: http://cffi.readthedocs.org
Author: Armin Rigo, Maciej Fijalkowski
Author-email: [email protected]
License: MIT
Location: c:\local\python36\lib\site-packages
Requires: pycparser
Required-by: pyczmq
Metadata-Version: 2.1
Installer: pip
Classifiers:
Programming Language :: Python
Programming Language :: Python :: 2
Programming Language :: Python :: 2.6
Programming Language :: Python :: 2.7
Programming Language :: Python :: 3
Programming Language :: Python :: 3.2
Programming Language :: Python :: 3.3
Programming Language :: Python :: 3.4
Programming Language :: Python :: 3.5
Programming Language :: Python :: 3.6
Programming Language :: Python :: Implementation :: CPython
Programming Language :: Python :: Implementation :: PyPy
Entry-points:
[distutils.setup_keywords]
cffi_modules = cffi.setuptools_ext:cffi_modules
Files:
cffi_backend.cp36-win_amd64.pyd
cffi-1.12.3.dist-info\INSTALLER
cffi-1.12.3.dist-info\LICENSE
cffi-1.12.3.dist-info\METADATA
cffi-1.12.3.dist-info\RECORD
cffi-1.12.3.dist-info\WHEEL
cffi-1.12.3.dist-info\entry_points.txt
cffi-1.12.3.dist-info\top_level.txt
cffi_init.py
cffi_pycache__init_.cpython-36.pyc
cffi_pycache_\api.cpython-36.pyc
cffi_pycache_\backend_ctypes.cpython-36.pyc
cffi_pycache_\cffi_opcode.cpython-36.pyc
cffi_pycache_\commontypes.cpython-36.pyc
cffi_pycache_\cparser.cpython-36.pyc
cffi_pycache_\error.cpython-36.pyc
cffi_pycache_\ffiplatform.cpython-36.pyc
cffi_pycache_\lock.cpython-36.pyc
cffi_pycache_\model.cpython-36.pyc
cffi_pycache_\pkgconfig.cpython-36.pyc
cffi_pycache_\recompiler.cpython-36.pyc
cffi_pycache_\setuptools_ext.cpython-36.pyc
cffi_pycache_\vengine_cpy.cpython-36.pyc
cffi_pycache_\vengine_gen.cpython-36.pyc
cffi_pycache_\verifier.cpython-36.pyc
cffi \ _cffi_errors.h
cffi \ _cffi_include.h
cffi \ _embedding.h
cffi\api.py
cffi\backend_ctypes.py
cffi\cffi_opcode.py
cffi\commontypes.py
cffi\cparser.py
cffi\error.py
cffi\ffiplatform.py
cffi\lock.py
cffi\model.py
cffi\parse_c_type.h
cffi\pkgconfig.py
cffi\recompiler.py
cffi\setuptools_ext.py
cffi\vengine_cpy.py
cffi\vengine_gen.py
cffi\verifier.py

Listing pip details for pyczmq, shows requirement for "_cffi.py".

c:\local\python36>Scripts\pip.exe show --verbose -f pyczmq
Name: pyczmq
Version: 0.0.4
Summary: pyczmq cffi wrapper
Home-page: https://github.com/michelp/pyczmq
Author: Michel Pelletier
Author-email: [email protected]
License: LGPL v3
Location: c:\local\python36\lib\site-packages
Requires: cffi
Required-by:
Metadata-Version: 1.0
Installer:
Classifiers:
Entry-points:
Files:
pyczmq-0.0.4-py3.6.egg-info\PKG-INFO
pyczmq-0.0.4-py3.6.egg-info\SOURCES.txt
pyczmq-0.0.4-py3.6.egg-info\dependency_links.txt
pyczmq-0.0.4-py3.6.egg-info\requires.txt
pyczmq-0.0.4-py3.6.egg-info\top_level.txt
pyczmq_init_.py
pyczmq_pycache__init_.cpython-36.pyc
pyczmq_pycache__cffi.cpython-36.pyc
pyczmq_pycache_\types.cpython-36.pyc
pyczmq_pycache_\zauth.cpython-36.pyc
pyczmq_pycache_\zbeacon.cpython-36.pyc
pyczmq_pycache_\zcert.cpython-36.pyc
pyczmq_pycache_\zcertstore.cpython-36.pyc
pyczmq_pycache_\zctx.cpython-36.pyc
pyczmq_pycache_\zframe.cpython-36.pyc
pyczmq_pycache_\zloop.cpython-36.pyc
pyczmq_pycache_\zmq.cpython-36.pyc
pyczmq_pycache_\zmsg.cpython-36.pyc
pyczmq_pycache_\zpoller.cpython-36.pyc
pyczmq_pycache_\zsocket.cpython-36.pyc
pyczmq_pycache_\zstr.cpython-36.pyc
pyczmq \ _cffi.py
pyczmq\types.py
pyczmq\zauth.py
pyczmq\zbeacon.py
pyczmq\zcert.py
pyczmq\zcertstore.py
pyczmq\zctx.py
pyczmq\zframe.py
pyczmq\zloop.py
pyczmq\zmq.py
pyczmq\zmsg.py
pyczmq\zpoller.py
pyczmq\zsocket.py
pyczmq\zstr.py

CFFI module does not list "_cffi.py" file.
What's missing?

sample.txt

I'm using czmq e831fffe5 + libzmq ecb9770, pyczmq 8ad9d50 (all latest and greatest)

$ python
Python 2.7.6 (default, Jan 11 2014, 17:06:02)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.

from pyczmq import zmq, zctx, zsocket, zmsg, zframe, zbeacon, zstr
Traceback (most recent call last):
File "", line 1, in
File "build/bdist.linux-i686/egg/pyczmq/zmsg.py", line 66, in
File "build/bdist.linux-i686/egg/pyczmq/_cffi.py", line 17, in cdef
File "/usr/local/lib/python2.7/dist-packages/cffi/api.py", line 102, in cdef
self._parser.parse(csource, override=override)
File "/usr/local/lib/python2.7/dist-packages/cffi/cparser.py", line 154, in parse
self._internal_parse(csource)
File "/usr/local/lib/python2.7/dist-packages/cffi/cparser.py", line 159, in _internal_parse
ast, macros = self._parse(csource)
File "/usr/local/lib/python2.7/dist-packages/cffi/cparser.py", line 126, in _parse
self.convert_pycparser_error(e, csource)
File "/usr/local/lib/python2.7/dist-packages/cffi/cparser.py", line 148, in convert_pycparser_error
raise api.CDefError(msg)
cffi.api.CDefError: cannot parse "int zmsg_push (zmsg_t *self, zframe_t *frame);"
:8:30: before: zframe_t

if I add

cdef('typedef struct _zframe_t zframe_t;')

to zmsg.py, the error's gone; not sure this is the right fix though.

-m

Reported by Michael Haberler:

btw there's another regression relative to czmq master:

tests.test_zauth.test_zauth ... python: zsockopt.c:455: zsocket_set_curve_secretkey_bin: Assertion `rc == 0 || zmq_errno () == (156384712 + 53)' failed.
Aborted (core dumped)

it wont accept either - it should be possible to use zbeacon with non-string frames too

not sure why - happens to me on jessie as well as lucid:

mah@wheezy:~/src/czmq$ gdb python
GNU gdb (GDB) 7.6.1 (Debian 7.6.1-1)
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "i486-linux-gnu".
For bug reporting instructions, please see:
http://www.gnu.org/software/gdb/bugs/...
Reading symbols from /usr/bin/python2.7...(no debugging symbols found)...done.
(gdb) r
Starting program: /usr/bin/python
warning: Could not load shared library symbols for linux-gate.so.1.
Do you need "set solib-search-path" or "set sysroot"?
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/i686/cmov/libthread_db.so.1".
Python 2.7.5+ (default, Sep 17 2013, 17:31:54)
[GCC 4.8.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import pyczmq
pyczmq.zmq.version()
(4, 1, 0)
from pyczmq import zbeacon
service_beacon = zbeacon.new(10042)

Program received signal SIGSEGV, Segmentation fault.
zctx__socket_new (self=self@entry=0x273a, type=type@entry=0) at zctx.c:326
326 if (!self->context) {
(gdb) backtrace
#0 zctx__socket_new (self=self@entry=0x273a, type=type@entry=0) at zctx.c:326
#1 0xb774ef89 in zctx__socket_pipe (self=self@entry=0x273a) at zctx.c:364
#2 0xb7762c9e in zthread_fork (ctx=ctx@entry=0x273a, thread_fn=thread_fn@entry=0xb774a390 <s_agent_task>, args=args@entry=0x0)

at zthread.c:182

#3 0xb774ac51 in zbeacon_new (ctx=0x273a, port_nbr=137740368) at zbeacon.c:108
#4 0xb79ea58a in ffi_call_SYSV () from /usr/lib/i386-linux-gnu/libffi.so.5
#5 0xb79ea20d in ffi_call () from /usr/lib/i386-linux-gnu/libffi.so.5
#6 0xb7fccead in cdata_call (cd=0x87079e0, args=0x870dbac, kwds=0x0) at c/_cffi_backend.c:2323
#7 0x08151f9d in PyEval_EvalFrameEx ()
#8 0x081a5392 in ?? ()
#9 0x08153536 in PyEval_EvalFrameEx ()
#10 0x08189972 in PyEval_EvalCodeEx ()
#11 0x08151ac4 in PyEval_EvalFrameEx ()
#12 0x08189972 in PyEval_EvalCodeEx ()
#13 0x081efa97 in PyEval_EvalCode ()
#14 0x081f419d in ?? ()
#15 0x080ac9f9 in PyRun_InteractiveOneFlags ()
#16 0x080acb2e in PyRun_InteractiveLoopFlags ()
#17 0x080ad2ca in PyRun_AnyFileExFlags ()
#18 0x080adf5e in Py_Main ()
#19 0x080ae03d in main ()

(gdb)

The CZMQ v2 APIs have been declared deprecated with the 3.0.0 release in October 2014: https://github.com/zeromq/czmq#toc3-8224

Following the C4.1 process we are moving toward retiring those APIs - i.e. deleting them. This will break pyczmq.

The solution would be to update pyczmq to use the new v3 APIs from CZMQ.

Currently using ABI level compatibility that opens the zmq and czmq libraries with dlopen(). CFFI provides an API level mode where a small stub python extension is built at initial import time to export the library APIs to Python. It has many portability and version management advantages but requires a compiler. Precompiled platform binaries can be optionally distributed.

Support for both is also possible and fairly trivial, any thoughts @claws or @hintjens?

Running on Ubuntu 13.10 (saucy), pyczmq tests fail for me:

nosetests
.........E.....
======================================================================
ERROR: tests.test_zloop.test_zloop
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/data/src/pyczmq/tests/test_zloop.py", line 37, in test_zloop
    cancel_timer_id = zloop.timer(l, 1000, 1, on_timer_event, None)
  File "/data/src/pyczmq/pyczmq/_cffi.py", line 22, in inner_f
    val = f(*args)
  File "/data/src/pyczmq/pyczmq/zloop.py", line 94, in timer
    return C.zloop_timer(loop, delay, times, handler, ffi.new_handle(arg))
AttributeError: 'FFI' object has no attribute 'new_handle'

----------------------------------------------------------------------
Ran 15 tests in 2.153s

FAILED (errors=1)

I'm using libsodium, libzmq, and czmq cloned from github yesterday (2014-01-15), pyczmq from github today (2014-01-16), all installed to a dedicated prefix (/usr/local/zmq). No virtualenv involved. Everything else is from recent Ubuntu packages. Specifically:

$ dpkg-query -W | grep "^python"
python  2.7.5-5ubuntu1
[...]
python-cffi 0.6-1
[...]
$ dpkg-query -W | grep "ffi" 
libffi-dev:amd64    3.0.13-4
libffi6:amd64   3.0.13-4
[...]

pyczmq isn't compatible with libzmq < 4 or older versions of czmq either I suppose (at a minumum, I think libzmq would need the zocket ipv6 stuff)

This is fine, I just thought it might be worthwhile to mention somewhere the versioning requirements. It wasn't clear to me when I first tried to install it (admittedly, I didn't follow instructions)

from _cffi import C, ffi

ImportError: No module named '_cffi'