Comments (25)
So far I haven't been able to reproduce this on my FreeBSD boxes. I have one 9.x and one 10.x 64 bit (both installed ZMQ::FFI cleanly) and one 10.x 32 bit (though the 32bit one is in the process of rebuilding so I haven't gotten there yet). The libffi on those machines is more recent that 3.0.13 though, I will try the older version next. I can also try on my netbsd VM if I can remember how to install packages in that environment :)
I noticed the kFreeBSD failures were similar to the Linux failures (and should be fixed with the Math::BigInt upgrade), so that points to the userland rather than the kernel.
from perlzmq.
@plicease thanks so much for your help, let me know what you find. I've released 1.01 with the newer Math::BigInt requirement.
from perlzmq.
Since adding the newer Math::BigInt requirement test failures only appear to be on BSD now (interestingly only FreeBSD and NetBSD flavors):
http://matrix.cpantesters.org/?dist=ZMQ-FFI+1.02
And they all exhibit the same symptom ("No subtests run"), so at least it's consistent...
from perlzmq.
Note that the the failures for both Net and FreeBSD are on unthreaded Perls for 1.02.
I haven't been able to do any testing on this yet, but I hope to soon.
from perlzmq.
Also the Perl that I built for FreeBSD was threaded too, which may explain why I wasn't able to reproduce it.
from perlzmq.
I think I have this figured out. I was able to reproduce this on NetBSD once I installed zmq since the Perl on my NetBSD VM is unthreaded. It is, as I suspected dumping core:
netbsd64% perl -Mblib t/fd.t
Abort (core dumped)
The problem is that zmq uses pthreads:
netbsd64% ldd /usr/pkg/lib/libzmq.so.4.0.0
/usr/pkg/lib/libzmq.so.4.0.0:
-lsodium.13 => /usr/pkg/lib/libsodium.so.13
-lc.12 => /usr/lib/libc.so.12
-lrt.1 => /usr/lib/librt.so.1
-lgcc_s.1 => /usr/lib/libgcc_s.so.1
-lpthread.1 => /usr/lib/libpthread.so.1
-lstdc++.7 => /usr/lib/libstdc++.so.7
-lm.0 => /usr/lib/libm.so.0
I was able to get the test suite to pass by setting LD_PRELOAD:
netbsd64% env LD_PRELOAD=/usr/lib/libpthread.so.1 prove -b
t/close.t ....... ok
t/device.t ...... ok
t/errors.t ...... ok
t/fd.t .......... ok
t/fork.t ........ ok
t/gc.t .......... ok
t/multipart.t ... ok
t/options.t ..... ok
t/proxy.t ....... ok
t/pubsub.t ...... ok
t/router-req.t .. ok
t/send_recv.t ... ok
t/unbind.t ...... ok
t/unicode.t ..... ok
All tests successful.
Files=14, Tests=37, 2 wallclock secs ( 0.03 usr 0.03 sys + 0.79 cusr 0.22 csys = 1.07 CPU)
Result: PASS
Solutions as I see it include:
- do not support unthreaded perls on Free/NetBSD
- document the LD_PRELOAD hack and modify the test suite to set it on Free/NetBSD (I am not sure how to do that exactly with MM, but presumably it is possible).
Here are some references:
https://rt.perl.org/Public/Bug/Display.html?id=122906
http://stackoverflow.com/questions/13587325/threads-vs-pthread-in-perl
This does seem to be a general Perl issue and not a FFI / Platypus issue, so that at least is good :)
from perlzmq.
To clarify, an unthreaded Perl without the LD_PRELOAD hack WOULD work, IF the Perl had been linked with pthreads (which is possible with manual intervention by the person building Perl itself). This is probably half baked, and may not be portable, but I was able to detect if pthreads had been initalized with this snipit:
use strict;
use warnings;
use 5.010;
use FFI::Platypus;
my $ffi = FFI::Platypus->new;
$ffi->find_lib( lib => 'pthread' );
$ffi->attach( pthread_self => [] => 'opaque' );
if(pthread_self() == -1)
{
warn 'pthreads were not initalized';
}
else
{
say pthread_self();
}
Let me know if you want me to test whatever solution you come up with.
from perlzmq.
Not sure why I didn't think of it last night but I think the portable way to determine if pthreads is already baked into the Perl you are using is use find_symbol
to search for pthread_self
in the current process.
It breaks down if someone else has already loaded a library linked to pthreads, but them having done that is probably broken anyway.
use strict;
use warnings;
use 5.010;
use FFI::Platypus;
my $ffi = FFI::Platypus->new;
$ffi->lib( undef );
if($ffi->find_symbol('pthread_self'))
{
say "have pthreads";
}
else
{
say "no pthreads";
}
from perlzmq.
Awesome @plicease, thanks for the update.
Just so I'm clear, the reason this isn't an issue on Linux is because the libc/pthread/dlopen semantics are different? i.e. on Linux pthread init can happen lazily on a subsequent dlopen, but on BSD it only happens on initial loading of the executable?
That is, they're both honoring the NEEDED entry in the ELF dynamic section, it's just a matter of whether pthreads is being initialized on the lazy dlopen or not?
$ readelf -d /usr/lib/libzmq.so
Dynamic section at offset 0xa2d28 contains 31 entries:
Tag Type Name/Value
0x0000000000000001 (NEEDED) Shared library: [libsodium.so.13]
0x0000000000000001 (NEEDED) Shared library: [librt.so.1]
0x0000000000000001 (NEEDED) Shared library: [libpthread.so.0] <===
0x0000000000000001 (NEEDED) Shared library: [libstdc++.so.6]
0x0000000000000001 (NEEDED) Shared library: [libm.so.6]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1]
Regarding how to address, yeah I'm thinking go with the find_symbol
check you mentioned. Put that in Makefile.PL and bail out with the "unthreaded Perls not supported" warning on BSD... will generate NA on CPAN Testers, and immediately alert people when they try to install the module.
from perlzmq.
Let me know if you want me to test whatever solution you come up with.
@plicease I added the find_symbol
check and pushed it to the pthread-self-check
branch. If you have time mind giving it a quick whirl? It should just bail installing on an unthreaded BSD perl.
from perlzmq.
diff --git a/dist.ini b/dist.ini
index 07fd08e..94058a4 100644
--- a/dist.ini
+++ b/dist.ini
@@ -12,11 +12,11 @@ lib = zmq
[MakeMaker::Awesome]
delimiter = |
+header = |use FFI::Platypus;
header = |# Can't currently support unthreaded BSD perls
header = |# See GH #13
header = |if ($^O =~ m/bsd/i) {
-header = | !FFI::Platypus->new
-header = | ->lib(undef)
+header = | !FFI::Platypus->new(lib => [undef])
header = | ->find_symbol('pthread_self')
header = | && exit;
header = |}
Almost, need to use FFI::Platypus
first and lib
is a attribute rather than a method so it returns the value instead of the $ffi
instance. Once I made that change it worked. Or rather didn't work in the right way on NetBSD.
Fails with exit 0 on NetBSD without pthreads and installs on FreeBSD64 with a threaded Perl. Also installed on NetBSD with the LD_PRELOAD
hack.
from perlzmq.
fixed, thanks. All of the outstanding ZMQ::FFI
CPAN tester failures were on unthreaded BSD perls, so here's hoping I have a 100% pass rate after this :)
from perlzmq.
well, no luck... still failing on bsd. Checked the Makefile.PL
in 1.04 and it has the correct blurb from above, so I'm stumped.
http://www.cpantesters.org/distro/Z/ZMQ-FFI.html?oncpan=1&distmat=1&version=1.04&grade=3
@plicease you think just check $Config{usethreads}
instead? Yeah it breaks for the custom Perl/LD_PRELOAD hack use cases, but those are probably pretty out there use cases anyways.
grr this action-at-a-distance stuff is hard to test. I'll reach out to the CPAN Testers and see if they can tell me if a) their Perl is definitely sans threads, and b) why the Makefile.PL didn't bail like it was supposed to.
from perlzmq.
@eserte are these your CPAN Tester boxes? If so mind helping me debug this?
http://www.cpantesters.org/cpan/report/d586ea36-d25b-11e4-99a7-833ee0bfc7aa
http://www.cpantesters.org/cpan/report/d493c1a8-d25b-11e4-99a7-833ee0bfc7aa
http://www.cpantesters.org/cpan/report/d3afd4c0-d25b-11e4-99a7-833ee0bfc7aa
http://www.cpantesters.org/cpan/report/d2c7fb32-d25b-11e4-99a7-833ee0bfc7aa
from perlzmq.
@calid It's up to you, and I agree that breaking LD_PRELOAD is not a big deal, but the problem with $Config{usethreads}
is that it also breaks people without a threaded Perl, but specified -lpthreads
when they build Perl, which is definitely something that people do.
It does seem to be working for NetBSD at least. Admittedly I wasn't able to test FreeBSD because my Perl on my FreeBSD system is threaded. For FreeBSD you could check $Config{ldflags}
for -pthread
:
freebsd% perl -V:ldflags
ldflags='-pthread -Wl,-E -fstack-protector -L/usr/local/lib';
At some point I do want to build an unthreaded Perl on FreeBSD that I can do some testing on.
from perlzmq.
oh yeah, $Config{ldflags}
... yeah that's probably a better option as it wouldn't kill the manually linked pthread case.
I'll probably hold off on further changes until more CPAN Tester reports come in... who knows, maybe those 4 are just a one off and it's working as expected everywhere else.
from perlzmq.
Also it wouldn't hurt to let me get an unthreaded FreeBSD perl. It would be good to verify that -pthreads
is missing from $Config{ldflags}
there.
from perlzmq.
Definitely, thanks again for your help
from perlzmq.
Yes, the test reports mentioned above come from my boxes. The long runtime (wallclock secs > 3200) indicates that the tests were killed because they were hanging (inactivity timeout is 200s times 16 tests = 3200s). And yes, plicease's observation is correct that it seems to fail only for unthreaded FreeBSD perls without -pthread
compiled in. Because this is a frequent problem (also for other CPAN distribution) I tend to manually hack -pthread
into newer perls, but unfortunately this is not officially supported by perl's configure
mechanism.
Right now I am running your module with such perls, so expect a couple of PASSes for the combination FreeBSD + unthreaded perl.
from perlzmq.
@eserte thanks. One question: the newest ZMQ::FFI
version has a check in Makefile.PL
to exit if it doesn't find pthread_self
in the running perl, any idea why the tests started at all? If perl was built without -pthread
that symbol should be missing and the Makefile should have exited 0?
From the 1.04 Makefile.PL on CPAN:
use FFI::Platypus;
# Can't currently support unthreaded BSD perls
# See GH #13
if ($^O =~ m/bsd/i) {
!FFI::Platypus->new(lib => [undef])
->find_symbol('pthread_self')
&& exit;
}
from perlzmq.
pthread_self seems to be part of libc:
$ strings /lib/libc.so.7 | grep pthread_self
_pthread_self
from perlzmq.
@plicease thanks. @eserte I've published a developer release 1.05_01
with @plicease's special freebsd check. Mind trying it and letting me know if it works as intended? If so I'll go ahead and publish 1.06
and I think we can finally consider this issue put to rest :)
The direct link is at: http://www.cpan.org/authors/id/C/CA/CALID/ZMQ-FFI-1.05_01.tar.gz
from perlzmq.
I published 1.05_02
and added an error message for unthreaded bsd perls, hopefully I didn't botch the check.
http://www.cpan.org/authors/id/C/CA/CALID/ZMQ-FFI-1.05_02.tar.gz
from perlzmq.
1.05_02 looks good, it seems that the problematic perl versions are properly skipped on my systems.
from perlzmq.
I've published 1.06
with the fix, thanks guys for the help.
from perlzmq.
Related Issues (20)
- Segfault when using EV version 3 HOT 1
- user linger value is clobbered HOT 1
- segfault during global destruction HOT 1
- test hangs with some versions of Socket HOT 2
- zmq_poll function missing ? HOT 11
- Sockets do not keep alive contexts. HOT 4
- support exporting zmq constants directly from ZMQ::FFI
- Add missing socket api methods to SocketRole requires HOT 1
- DEALER-REP hangs forever HOT 9
- Faster get/set on sockets HOT 1
- using proxy results in high cpu usage HOT 4
- Context object still keeps around socket objects after they go out of scope HOT 3
- Unable to build a release with SELinux enabled on system HOT 1
- Unable to build a release with a perl compiled with clang HOT 1
- zmq-ffi: t/proxy.t occasionally hangs HOT 26
- zmq_bind: Protocol not supported HOT 2
- Fails to install on CentOS HOT 4
- FFI::Platypus should be specified as configure_requires HOT 1
- Tests will fail in perl-5.38.0
- dealer with AE::io don't get any event when watch_write is 0 HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from perlzmq.