perlffi / ffi-platypus Goto Github PK

Expect

To be able to make ffi calls in object destructors.

Actual

Segfaults when destructors make ffi calls during global destruction.

Reproduce

This turns out to be extremely hard to reliably reproduce (given that in global destruction order is undefined). For me I'm able to make this consistently happen by using EV version 3, the latest ZMQ::FFI, the latest FFI::Platypus, Perl 5.20.1, and running this test. But YMMV.

With the conditions above I get:

calid@arch ~
[✔]▶ perl /tmp/bad_ev.t
1..0
# No tests run!
argument type not supported (0) at /home/calid/.perlbrew/libs/perl-5.20.1@tmp/lib/perl5/ZMQ/FFI/ZMQ3/Socket.pm line 404 during global destruction.
argument type not supported (1) at /home/calid/.perlbrew/libs/perl-5.20.1@tmp/lib/perl5/ZMQ/FFI/ZMQ3/Socket.pm line 404 during global destruction.
argument type not supported (2) at /home/calid/.perlbrew/libs/perl-5.20.1@tmp/lib/perl5/ZMQ/FFI/ZMQ3/Socket.pm line 404 during global destruction.
Segmentation fault (core dumped)

calid@arch ~
[SIGSEGV]▶

Full output

Discussion

After digging into this, essentially FFI::Platypus is cleaning things up before I have a chance to make my ffi calls.

Here's some trimmed down output showing the issue (I added warns to the underlying cleanup routines and rebuilt Platypus.so with debug on):

DLOPEN libzmq.so, HANDLE = 29702432 at /home/calid/.perlbrew/libs/perl-5.20.1@tmp/lib/perl5/x86_64-linux/FFI/Platypus.pm line 421.
...
DCLOSE 29702432 at /home/calid/.perlbrew/libs/perl-5.20.1@tmp/lib/perl5/x86_64-linux/FFI/Platypus.pm line 512 during global destruction.
FFI::Platypus::Type::DESTROY during global destruction.
FFI::Platypus::Type::DESTROY during global destruction.
FFI::Platypus::Type::DESTROY during global destruction.
FFI::Platypus::Function::DESTROY during global destruction.
FFI::Platypus::Type::DESTROY during global destruction.
FFI::Platypus::Function::DESTROY during global destruction.
FFI::Platypus::Function::DESTROY during global destruction.
FFI::Platypus::Type::DESTROY during global destruction.
FFI::Platypus::Function::DESTROY during global destruction.
DCLOSE 1712251184 at /home/calid/.perlbrew/libs/perl-5.20.1@tmp/lib/perl5/x86_64-linux/FFI/Platypus.pm line 512 during global destruction.
About to do ffi_call from Platypus.xs at lib/ZMQ/FFI/ZMQ3/Socket.pm line 404 during global destruction.
argument type not supported (0) at lib/ZMQ/FFI/ZMQ3/Socket.pm line 404 during global destruction.
argument type not supported (1) at lib/ZMQ/FFI/ZMQ3/Socket.pm line 404 during global destruction.
argument type not supported (2) at lib/ZMQ/FFI/ZMQ3/Socket.pm line 404 during global destruction.
argument type not supported (3) at lib/ZMQ/FFI/ZMQ3/Socket.pm line 404 during global destruction.
Segmentation fault (core dumped)

Full output

In this case I'm pretty sure the immediate cause of the segfault is libzmq getting dlclosed and unloaded before zeromq shutdown is complete, resulting in this stack trace:

Program received signal SIGSEGV, Segmentation fault.
[Switching to LWP 31279]
0x00007ffff488488c in ?? ()
(gdb) bt
#0  0x00007ffff488488c in ?? ()
#1  0x00f3b91000000001 in ?? ()
#2  0x0000000000000000 in ?? ()
(gdb) thr
[Current thread is 2 (LWP 31279)]
(gdb) info thr
  Id   Target Id         Frame
* 2    LWP 31279 "perl"  0x00007ffff488488c in ?? ()
  1    LWP 31275 "perl"  0x00007ffff6fb856d in write () from /usr/lib/libc.so.6

I'm assuming thread 2 is the zmq background thread, which is still spinning doing things, but as the ?? () indicates libzmq itself has been unloaded from memory... so as soon as that background thread goes to do... anything, kaboom.

But dlclose is not the only issue. Here's the result when I completely comment out the dlclose code in dl.xs:

FFI::Platypus::Type::DESTROY during global destruction.
FFI::Platypus::Function::DESTROY during global destruction.
About to do ffi_call from Platypus.xs at lib/ZMQ/FFI/ZMQ3/Socket.pm line 404 during global destruction.
argument type not supported (0) at lib/ZMQ/FFI/ZMQ3/Socket.pm line 404 during global destruction.
argument type not supported (1) at lib/ZMQ/FFI/ZMQ3/Socket.pm line 404 during global destruction.
argument type not supported (2) at lib/ZMQ/FFI/ZMQ3/Socket.pm line 404 during global destruction.
argument type not supported (3) at lib/ZMQ/FFI/ZMQ3/Socket.pm line 404 during global destruction.
Segmentation fault (core dumped)

and the corresponding stacktrace:

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff6f6d55e in __memcpy_sse2_unaligned () from /usr/lib/libc.so.6
(gdb) bt
#0  0x00007ffff6f6d55e in __memcpy_sse2_unaligned () from /usr/lib/libc.so.6
#1  0x00007ffff6172c0e in ffi_call () from /usr/lib/libffi.so.6
#2  0x00007ffff637c747 in ffi_pl_sub_call (cv=0xf274e8) at include/ffi_platypus_call.h:552
#3  0x00000000004a5530 in Perl_pp_entersub ()
#4  0x000000000049e373 in Perl_runops_standard ()
#5  0x0000000000435597 in Perl_call_sv ()
#6  0x00000000004ae256 in S_curse ()
#7  0x00000000004aeb7d in Perl_sv_clear ()
#8  0x00000000004aeeff in Perl_sv_free2 ()
#9  0x00000000004a6962 in S_visit ()
#10 0x00000000004af30c in Perl_sv_clean_objs ()
#11 0x0000000000437758 in perl_destruct ()
#12 0x000000000041dcd4 in main ()
(gdb)

The argument type not supported warning appears to be because all the FFI::Platypus::Type objects are getting destroyed before subsequent ffi calls. Here's the result after I comment out the DESTROY code in Type.xs

FFI::Platypus::Function::DESTROY during global destruction.
About to do ffi_call from Platypus.xs at lib/ZMQ/FFI/ZMQ3/Socket.pm line 404 during global destruction.
Segmentation fault (core dumped)

Still segfaults. But if I also comment out the DESTROY in Function.xs:

About to do ffi_call from Platypus.xs at lib/ZMQ/FFI/ZMQ3/Socket.pm line 404 during global destruction.
About to do ffi_call from Platypus.xs at lib/ZMQ/FFI/ErrorHelper.pm line 46 during global destruction.
About to do ffi_call from Platypus.xs at lib/ZMQ/FFI/ZMQ3/Socket.pm line 453 during global destruction.
About to do ffi_call from Platypus.xs at lib/ZMQ/FFI/ErrorHelper.pm line 46 during global destruction.
About to do ffi_call from Platypus.xs at lib/ZMQ/FFI/ZMQ3/Context.pm line 155 during global destruction.
# No segfault

Success! so only when none of the Platypus cleanup routines run during global destruction was ZMQ::FFI able to do its own cleanup without exploding.

Solution?

The two solutions that come to mind:

1. Never make ffi calls in destructors (client-side fix)
2. FFI::Platypus doesn't do any cleanup if global destruction is in effect (module-side fix)

It's not entirely clear which is the right solution. For now I'll go with the first one and try to avoid any ffi calls in the ZMQ::FFI destructors if global destruction is in effect. In the case of zeromq this is a little tricky since there's a whole linger/close/destroy dance you're supposed to do. I guess in the worst case users would always have to do this explicitly themselves and could no longer rely on the ZMQ::FFI object destructors by default, but that would be sad.

Alternatively, is it possible for FFI::Platypus to skip its cleanup routines during global destruction? Going off my own experiments, as long as FFI::Platypus doesn't try to cleanup during global destruction ffi calls remain safe. If we're in global destruction the process is closing up shop anyways, so perhaps it doesn't matter if the ffi cleanup is skipped?

FYI I am aware of the pattern of setting native cleanup functions to a package's *DESTROY symbol table entry directly (see zeromq/perlzmq/issues/19). Initially I hoped that would be the fix, but after digging into the issue more it doesn't look like this will be a cure-all (for example it doesn't address FFI::Platypus::Type objects getting prematurely destroyed.

ps

Sorry for the length of this issue report!

This may have been the elusive threaded perl / platypus crash reported by someone at YAPC::NA that I was unable to reproduce. Or maybe just another bug. I can get it to seg fault with debian 8 + perl 5.10.0.

dare% perl -Mblib t/threads.t 
1..2
Segmentation fault

t/ffi_platypus_type_pointer_size_buffer.t tests a practical example of using the custom types API, but we really need to test each function individually.

Use ExtUtils::CppGuess if it is installed.

I don't want to make it a hard prereq for Platypus (at least not yet).

Extension writer can make ExtUtils::CppGuess a prereq if they are doing C++.

Now that Alien::Base generates a config.site (new in 0.015) based on the Perl in use Alien::FFI generates what seems to be a usable libffi on OS X for the system Perl (which is a hybrid 32/64 bit exe). However, it does not pass two tests:

urquhart% prove -bv t/ffi_platypus_abi.t t/ffi_platypus_declare_abi.t 
t/ffi_platypus_abi.t .......... 
1..2
not ok 1 - has a default ABI

#   Failed test 'has a default ABI'
#   at t/ffi_platypus_abi.t line 12.
    ok 1 - string
    ok 2 - integer
    1..2
ok 2 - bogus
# Looks like you failed 1 test of 2.
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/2 subtests 
t/ffi_platypus_declare_abi.t .. 
1..2
not ok 1 - has a default ABI

#   Failed test 'has a default ABI'
#   at t/ffi_platypus_declare_abi.t line 10.
    ok 1 - string
    ok 2 - integer
    1..2
ok 2 - bogus
# Looks like you failed 1 test of 2.
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/2 subtests 

Test Summary Report
-------------------
t/ffi_platypus_abi.t        (Wstat: 256 Tests: 2 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
t/ffi_platypus_declare_abi.t (Wstat: 256 Tests: 2 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
Files=2, Tests=4,  0 wallclock secs ( 0.04 usr  0.01 sys +  0.07 cusr  0.01 csys =  0.13 CPU)
Result: FAIL

They are actually the same test, but via different interfaces.

Because the ABI probe uses /tmp to build and run executables, it fails if /tmp is mounted with noexec:

https://gist.github.com/plicease/f08c2cb43a9c459a4a33207f8a5f1834

Basically a series of multiple arguments get the same ffi type right now. It doesn't matter as far as the arguments API, but it means that libffi might be using the wrong type. This probably is okay for integers on intel, but may barf on big endian or floats.

This memory, allocated in include/ffi_platypus_call.h is never freed:

Newx(closure, 1, ffi_pl_closure); /* FIXME: leak */

When I was writing the code the idea was to attach the memory to an array ref or something stored as an inside out attribute on the FFI::Platypus::Closure object, and free it from that class' DESTROY method.

Variable length arrays should be allowed for everything except for return values.

This shouldn't be necessary:

https://github.com/plicease/FFI-Platypus-Lang-Fortran/blob/d7f79c2df4afad7a5fc2b74f6fe71aa4cd5e3d8e/examples/var_array.pl

This is important on Windows where the Windows API uses stdcall instead of cdecl.

To be comprehensive it would be desirable to use any arbitrary ABI, but it seems the only place they are defined is as an enum in ffitarget.h ... not easy to extract, and we'd probably want to verify the ABIs as well.

The problem is that we need to know the size of things like an int and a pointer, so what we detect is which ever mode is the default when installed. The user could change the mode after install. Currently this is not supported, so I am marking this as an enhancement. When I first looked at this using arch -32 perl ... would select the 32 bit mode for Perl, but the config it used was for 64 bit. The correct way to specify 32/64bits appears to be via the VERSIONER_PERL_PREFER_32_BIT:

https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man1/perl.1.html

Bonus points for also supporting PowerPC from an Intel build, though I doubt it is possible, at lest using a probe as we do currently. Since PowerPC isn't supported by any currently supported version of OS X this is a very low priority.

Reopening in the right place from here:

PerlFFI/FFI-Platypus-Lang-CPP#8

This test cases consumes memory, until it runs out:

use FFI::Platypus;
use FFI::TinyCC;

sub c {
}

my $tcc = FFI::TinyCC->new;

$tcc->compile_string(<<'EOF');
void c_with_arg(void (*f)(void))
{
    f();
}
EOF

my $ffi = FFI::Platypus->new;
$ffi->type('()->void', 'FType');

my $c = $ffi->closure(\&c);

my $a = $tcc->get_symbol('c_with_arg');

while(1) {
    $ffi->function($a => ['FType'] => 'int')->call($c);
}

If you cast the closure to an opaque before the loop then you are good:

use FFI::Platypus;
use FFI::TinyCC;

sub c {
}

my $tcc = FFI::TinyCC->new;

$tcc->compile_string(<<'EOF');
void c_with_arg(void (*f)(void))
{
    f();
}
EOF

my $ffi = FFI::Platypus->new;
$ffi->type('()->void', 'FType');

my $c = $ffi->closure(\&c);
my $c_ptr = $ffi->cast('()->void' => 'opaque', $c);

my $a = $tcc->get_symbol('c_with_arg');

while(1) {
    $ffi->function($a => ['opaque'] => 'int')->call($c_ptr);
}

Possible fixes:

1. Instead of pushing the closure data onto a list, cache it in a hash where the key is the signature. Could the signature be the pointer to the FFI::Platypus::Type? Probably good enough since the type should be reused in the FFI::Platypus instance.
2. Document as in the CAVEATS section, with the above alternate suggestion. This is reasonable if the use case is not common.

For example, FFI::Platypus::Lang::Pascal fails like this on OS X:

/usr/local/bin/fpc -Px86_64 myunit.pas
Free Pascal Compiler version 2.6.4 [2014/02/26] for x86_64
Copyright (c) 1993-2014 by Florian Klaempfl and others
Target OS: Darwin for x86_64
Compiling myunit.pas
Assembling (pipe) myunit.s
57 lines compiled, 0.1 sec 
/usr/local/bin/fpc -Px86_64 test.pas
Free Pascal Compiler version 2.6.4 [2014/02/26] for x86_64
Copyright (c) 1993-2014 by Florian Klaempfl and others
Target OS: Darwin for x86_64
Compiling test.pas
Assembling (pipe) test.s
Linking libtest.dylib
8 lines compiled, 0.1 sec 
no dylib in /Users/ollisg/.cpanm/work/1422984005.6881/FFI-Platypus-Lang-Pascal-0.01/libtest at lib/Module/Build/FFI/Pascal.pm line 187.
FAIL
! Installing FFI::Platypus::Lang::Pascal failed. See /Users/ollisg/.cpanm/work/1422984005.6881/build.log for details. Retry with --force to force install it.
urquhart% ls libtest
libtest.dylib*  myunit.o  myunit.pas  myunit.ppu  test.o  test.pas

because fpc produces a .dylib instead of a .bundle.

%Config disappoints again.

I think probably the only thing that it can do is warn.

Some doco:

http://caml.inria.fr/pub/docs/manual-ocaml-4.00/manual033.html

Seems that newXSproto does not return a CV* on Perl 5.8.8.

I did not notice this because travis uses 5.8.9 for its perl 5.8. Most Linux distros that shipped with a 5.8.x that are still in use shipped with 5.8.8. Almost none shipped with 5.8.9 as far as I can tell because 5.8.9 came out after 5.10. The usefulness of travis testing with 5.8 is thus reduced (although it is not completely useless).

I picked 5.8.1 as the minimum requirement because I believe it is the oldest version supported by the tool chain. It is hard to say exactly if this is true. It is mentioned here:

https://github.com/Perl-Toolchain-Gang/toolchain-site/blob/master/lancaster-consensus.md

Though that document is quite old. Both ExtUtils::MakeMaker and Module::Build specify Perl 5.6 as a requirement. We are not going to support 5.6.

We have to decide if we need to support 5.8.8 based on how much effort / work it will be. I would like to support 5.8.8 as long as it is supported by the tool chain. One resolution for this issue would be to bump the version requirement to 5.8.9. Hardly seems worth supporting 5.8.9 though, since it is used almost nowhere and prevents the use of 5.10 features.

cc -Iinclude -Ixs -I/home/ollisg/perl5/perlbrew/perls/perl-5.8.8/lib/5.8.8/x86_64-linux/CORE -DXS_VERSION="0.04" -DVERSION="0.04" -fpic -c -fno-strict-aliasing -pipe -Wdeclaration-after-statement -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2 -o lib/FFI/Platypus.o lib/FFI/Platypus.c
lib/FFI/Platypus.xs: In function ‘XS_FFI__Platypus__Function_attach’:
lib/FFI/Platypus.xs:389:23: warning: passing argument 1 of ‘Perl_newXS’ discards ‘const’ qualifier from pointer target type
       cv = newXS(perl_name, ffi_pl_sub_call, path_name);
                       ^
In file included from /home/ollisg/perl5/perlbrew/perls/perl-5.8.8/lib/5.8.8/x86_64-linux/CORE/perl.h:3950:0,
                 from lib/FFI/Platypus.xs:2:
/home/ollisg/perl5/perlbrew/perls/perl-5.8.8/lib/5.8.8/x86_64-linux/CORE/proto.h:829:6: note: expected ‘char *’ but argument is of type ‘const char *’
 PERL_CALLCONV CV* Perl_newXS(pTHX_ char* name, XSUBADDR_t f, char* filename);
      ^
lib/FFI/Platypus.xs:389:51: warning: passing argument 3 of ‘Perl_newXS’ discards ‘const’ qualifier from pointer target type
       cv = newXS(perl_name, ffi_pl_sub_call, path_name);
                                                   ^
In file included from /home/ollisg/perl5/perlbrew/perls/perl-5.8.8/lib/5.8.8/x86_64-linux/CORE/perl.h:3950:0,
                 from lib/FFI/Platypus.xs:2:
/home/ollisg/perl5/perlbrew/perls/perl-5.8.8/lib/5.8.8/x86_64-linux/CORE/proto.h:829:6: note: expected ‘char *’ but argument is of type ‘ffi_pl_string’
 PERL_CALLCONV CV* Perl_newXS(pTHX_ char* name, XSUBADDR_t f, char* filename);
      ^
lib/FFI/Platypus.xs:391:42: warning: passing argument 1 of ‘Perl_newXS’ discards ‘const’ qualifier from pointer target type
       cv = newXSproto(perl_name, ffi_pl_sub_call, path_name, proto);
                                          ^
In file included from /home/ollisg/perl5/perlbrew/perls/perl-5.8.8/lib/5.8.8/x86_64-linux/CORE/perl.h:3950:0,
                 from lib/FFI/Platypus.xs:2:
/home/ollisg/perl5/perlbrew/perls/perl-5.8.8/lib/5.8.8/x86_64-linux/CORE/proto.h:829:6: note: expected ‘char *’ but argument is of type ‘const char *’
 PERL_CALLCONV CV* Perl_newXS(pTHX_ char* name, XSUBADDR_t f, char* filename);
      ^
lib/FFI/Platypus.xs:391:68: warning: passing argument 3 of ‘Perl_newXS’ discards ‘const’ qualifier from pointer target type
       cv = newXSproto(perl_name, ffi_pl_sub_call, path_name, proto);
                                                                    ^
In file included from /home/ollisg/perl5/perlbrew/perls/perl-5.8.8/lib/5.8.8/x86_64-linux/CORE/perl.h:3950:0,
                 from lib/FFI/Platypus.xs:2:
/home/ollisg/perl5/perlbrew/perls/perl-5.8.8/lib/5.8.8/x86_64-linux/CORE/proto.h:829:6: note: expected ‘char *’ but argument is of type ‘ffi_pl_string’
 PERL_CALLCONV CV* Perl_newXS(pTHX_ char* name, XSUBADDR_t f, char* filename);
      ^
lib/FFI/Platypus.xs:391:10: error: void value not ignored as it ought to be
       cv = newXSproto(perl_name, ffi_pl_sub_call, path_name, proto);
          ^
error building lib/FFI/Platypus.o from 'lib/FFI/Platypus.c' at /home/ollisg/.perlbrew/libs/perl-5.8.8@dev/lib/perl5/ExtUtils/CBuilder/Base.pm line 175.
FAIL
! Installing FFI::Platypus failed. See /home/ollisg/.cpanm/work/1421169697.1280/build.log for details. Retry with --force to force install it.

The declarative syntax should be something like this:

  my $closure = sticky closure { ... }

sticky can just be a function (xsub or Perl) that increments the ref count. Now you can pass it into a function that expects it without having to save the code ref anywhere.

  foo(1,2, sticky closure { ... } , 4, 5, ..);

Consider this pattern:

$ffi->type('record($tm_size)' => 'tm');
$ffi->attach( [ localtime => '_new' ] => ['time_t*'] => 'tm' );

sub new { _new(\$_[0]) }

If we could tell Platypus to ignore the first argument (or better yet, use it when blessing) then we could simplify to:

$ffi->type('record($tm_size)' => 'tm');
$ffi->attach( [ localtime => 'new' ] => ['ignore_classname', 'time_t*'] => 'tm' );

There are (today) a few ways of dealing with C structs (or their equivalents in other languages):

• Perl's pack/unpack
• Convert::Binary::C
• Inline::Struct
• bundle your extension with a ffi directory or use FFI::TinyCC to write accessors in

Some examples of these would be great.

Also a better solution would be great too. Perhaps something like ruby ffi's layout system could be borrowed/stolen.

https://gist.github.com/plicease/3d3ea2dba23af456d2ccbaa00d9ceb3d

It looks like the most immediate problem is that it isn't able to resolve libc symbols. It may just be a matter of detecting $^O eq 'msys' as PERL_OS_WINDOWS (inc/My/AutoConf.pm) or there may be more to it than that.

There may be other things once we fix that of course :)

In windows OS, ActiveState Perl provides nice utility called ppm, which will download the pre built perl modules from the ActiveState build environment and install in local machines.

This means, ActiveState perl user do not need to have development environment in the local machine to install perl modules.

As of now, FFI-Platypus is not available via ActiveState perl ppm.

http://code.activestate.com/ppm/FFI-Platypus/

Build Log: http://ppm4.activestate.com/MSWin32-x86-64int/5.20/2000/P/PL/PLICEASE/FFI-Platypus-0.23.d/log-20150216T061859.txt

From the build log, it appears that, ActivePerl severs have only 420 seconds for a module to build and test. It looks like FFI::Platypus test takes longer than 420s and it gets aborted and module is not available to consume via Active State perl PPM tool.

Its is possible to install this module via CPAN + having gcc in the windows PC but if its available via Active Perl PPM, it would reach more developer on windows.

Please optimize the test / remove low priority test / do not run all the test by default , So it would get build in Active State PPM server environment.

Parse::nm does a pretty good job at this on some platforms, and am currently using it for FFI::Platypus::Lang::CPP, but it has series drawbacks:

• cpantesters failures on platforms that I care about like MSWin32 and Solaris. Also fails on OpenBSD
• depends on Regexp::Assemble which is unmaintained and fails tests thanks to POD errors. sigh

Parsing nm isn't that hard so I propose a FFI::DLL::Symbols or hopefully a better name for that. I did a survey of my weird platform VMs and physical hardware and this is what I found:

• nm -P works on Linux, kFreeBSD, FreeBSD (9 and 10), NetBSD and Solaris (-P is for POSIX) and is the easiest format to parse (split on white space)
• OpenBSD (of course) has its own stupid format sigh
• dumpbin /exports foo.dll works with MSVC++
• Strawberry and Cygwin ??? one link claims that this can be done with objdump, but I get "no symbols" when I try to use it on a .dll

Config::AutoConf->check_prog_cc first appeared in Config::AutoConf v0.306_001. Might as well bump it to the latest (v0.309).

Diff.

This is a combination feature request / call for some help. Basically I'm working on a GPGME::FFI and ran into a little snag. GPGME has the following type:

struct _gpgme_engine_info
{
  struct _gpgme_engine_info *next;
  gpgme_protocol_t protocol;
  char *file_name;
  char *version;
  const char *req_version;
  char *home_dir;
};

There are two issues:

1. How can I (or is it even possible) to have a pointer to the same typed record?
2. If I have a predefined type (for example gpgme_protocol_t is an int) how can I reference it in the record?

If either of the above are not yet doable, maybe just create bugs for them.

This requires adding support for MSVC++ to Alien::FFI.

This should be POSSIBLE since I managed to add MSVC++ support to FFI::Raw, though it won't be pretty.

These sv's (from include/ffi_platypus_call.h) need to be allocated with malloc / Newx

        SV *sv[count]; /* TODO: could be large shouldn't alloate on the stack */

currently we check that the thing is a reference, but we should also double check that it is a reference to a scalar.

        if(SvROK(arg)) /* TODO: and a scalar ref */

Perhaps as a standard custom type.

http://www.cpantesters.org/cpan/report/e676fb22-b804-11e4-aa90-775b7e3e1735

I was able to reproduce this on Linux 64bit. 5.8.9 does not appear to have the same issue.

Notice also that there is a warning (both 5.8.8 and 5.8.9) that does not appear in more recent Perls:

t/ffi_platypus_record.t .................... 1/8 Variable "@destroy" will not stay shared at (eval 103) line 6.

There could be some advantages to supporting multiple implementations down the road. Examples:

• Win32::API may provide better performance or features on Windows platforms
• GNU ffcall may support some platforms that libffi does not (skeptical)
• Alternate implementations of Perl, such as p2, gperl or moe or even Perl 6

I had this in mind when I was designing the interface, but I haven't been implementing it in a particularly pluggable way unfortunately. Tasks include:

• Moving the libffi specific Perl code into FFI::Platypus::Impl::Libffi
• FFI::Platypus becomes a subclass of FFI::Platypus::Impl::...
• split the XS and C code into libffi specific stuff and non-libffi specific stuff. For example ffi_platypus_call.h is decidedly libffi specific. The record code should not be.
• Remove the libffi objects from the structs in ffi_platypus.h. Right now we are using the size and type fields for logic. This is probably the most significant task.

Libraries that use pthreads on Free/NetBSD may crash, see:

zeromq/perlzmq#13

At the moment this does not seem to be a FFI / Platypus issue, but a general Perl issue, but it is frustrating if you see the crashes and do not know what it is, so this should be documented perhaps as a FAQ or Trouble Shooting entry.

Also consider adding an option (off by default) for Module::Build::FFI to set LD_PRELOAD to pthreads on platforms that need it when running ./Build test. I am a bit leery of this because without a big DANGER WILL ROBINSON statement module using Foo::FFI it will be passing the buck to the module and the user using Foo::FFI. Perhaps if coupled with detection code that detects if pthreads have been loaded with a helpful diagnostic this could be workable.

use strict;
use warnings;
use 5.010;
use FFI::Platypus;

my $ffi = FFI::Platypus->new;
$ffi->find_lib( lib => 'pthread' );

$ffi->attach( pthread_self => [] => 'opaque' );

if(pthread_self() == -1)
{
  warn 'pthreads were not initalized';
}
else
{
  say pthread_self();
}

https://rt.perl.org/Public/Bug/Display.html?id=122906
http://stackoverflow.com/questions/13587325/threads-vs-pthread-in-perl
http://man7.org/linux/man-pages/man7/pthreads.7.html

in include/ffi_platypus_call.h

    /*
     * FIXME: wrong number of arguments should be supported.
     * too few and we should fill with undef.  too many may
     * be a warning.
     */
    if(items-(EXTRA_ARGS) != self->ffi_cif.nargs)
      croak("wrong number of arguments (expected %d, got %d)", self->ffi_cif.nargs, items-(EXTRA_ARGS) );

This is a misfeature copied from FFI::Raw. If the developer needs, s/he can use prototypes instead.

Docs say FFI::CheckLib is the way to go to identify libraries -- why not just integrate it directly like:

$ffi->lib(find_lib => 'archive');

The intuitive way:

http://blogs.perl.org/users/graham_ollis/2015/01/practical-ffi-with-platypus.html#comment-1531375

Existing way:

http://blogs.perl.org/users/graham_ollis/2015/01/practical-ffi-with-platypus.html#comment-1531383

No reason we can't support both.

I implemented some really basic assembly support, in that if there is a .s file it will feed it into cc along with any .c or .cpp files. I am pretty clueless about Assembly so this could almost certainly be better.

@awwaiid was demoing some assembly code at dc.pm.org last night and it occurred to me that anyone wanting to bundle assembly code would probably want to use nasm (or maybe yasm?) instead of cc. Also there exists Alien::nasm now, so Module::Build::FFI could inject it into the build prereqs if it sees a .asm file.

perhaps as a standard custom type.

Basically this:

$ffi->attach(foo=>[]);

should be the same as this:

$ffi->attach(foo=>[] => 'void');

libffi needs pthreads:

openbsd64% prove -bv t/closure_die.t
t/closure_die.t ..
1..2
/home/ollisg/perl5/perlbrew/perls/perl-5.18.2g/bin/perl:/usr/local/lib/libffi.so.0.0: undefined symbol 'pthread_mutex_lock'
lazy binding failed!
Failed 2/2 subtests

Test Summary Report
-------------------
t/closure_die.t (Wstat: 139 Tests: 0 Failed: 0)
  Non-zero wait status: 139
  Parse errors: Bad plan.  You planned 2 tests but ran 0.
Files=1, Tests=0,  0 wallclock secs ( 0.02 usr  0.01 sys +  0.04 cusr  0.00 csys =  0.07 CPU)
Result: FAIL

The "fix" such as it is for FFI::Raw is here:

ghedo/p5-FFI-Raw#29

The documentation is definitely easier to understand if you know C. This betrays my background with C and C++ obviously. It would be great to have some guides specific to Platypus with these and other languages:

• Fortran
• Rust
• Go
• Assembly

Also:

• C++ (I theorize this may be possible with c++filt or maybe clang has an API)

Also interspersing the documentation generally with some comments that will help non C programmers would be great.

I get this on both FreeBSD on i386 and Strawberry Perl 32 bit:

freebsd32% prove -bv t/type_sint64.t
t/type_sint64.t .. 
1..14
ok 1 - add(-1,2) = 1
ok 2 - inc(\$i,4) = \1
ok 3 - i=1
ok 4 - inc(\-3,4) = \1
not ok 5 - sum([-5..4]) = -5

#   Failed test 'sum([-5..4]) = -5'
#   at t/type_sint64.t line 38.
#          got: '4294967291'
#     expected: '-5'
ok 6 - array increment
ok 7 - null() == undef
ok 8 - is_null(undef) == 1
ok 9 - is_null(22) == 0
ok 10 - static_array = [-1,2,-3,4,-5,6,-7,8,-9,10]
ok 11 - null2() == undef
not ok 12 - call_closure(-2) = -4
ok 13 - call_closure(2) = 0
ok 14 - extra test

#   Failed test 'call_closure(-2) = -4'
#   at t/type_sint64.t line 57.
#          got: '4294967292'
#     expected: '-4'
# Looks like you failed 2 tests of 14.
Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/14 subtests 

Test Summary Report
-------------------
t/type_sint64.t (Wstat: 512 Tests: 14 Failed: 2)
  Failed tests:  5, 12
  Non-zero exit status: 2
Files=1, Tests=14,  0 wallclock secs ( 0.01 usr  0.01 sys +  0.02 cusr  0.01 csys =  0.05 CPU)
Result: FAIL

There is a work around for complex types passes as arguments (pass as an array of two elements or record of two members). This, however, does not work for return types. More recent versions of libffi seem to support complex types, but not the version that is bundled with Debian. long double is supported in older versions of libffi.

There are a couple of places that we croak() in include/ffi_platypus_call.h without making sure that memory is free'd. We need to either make those warnings, or make sure we free the memory.

Currently we check to see that it is a reference, but not the type of reference.

        if(SvROK(arg)) /* TODO: and an array ref */

Things to do to get the word out about FFI / Platypus:

• Remove FFI from the top of metacpan.org, or failing that SEE ALSO it / reimplement in something more modern (ticket: https://rt.cpan.org/Ticket/Display.html?id=101880) (update now maintained CPAN module with SEE ALSOs pointing to FFI::Platypus, FFI::Raw and Win32::API)
• Get FFI::Platypus on the libffi home page (ticket: https://github.com/atgreen/libffi/issues/131)
• Convince other CPAN authors to port their existing modules to Platypus (one success here: gugod/Phash-FFI#1), another good candidate would be the zmq interface (ported)
• blog about FFI::Platypus (examples: http://blogs.perl.org/users/graham_ollis/2015/01/practical-ffi-with-platypus.html , http://npointsolutions.blogspot.in/2015/02/forgin-function-interface-with-perl.html)
• Add additional language plugins. When I created the Rust plugin I had two rust programmers star it almost immediately. This is double plus good since it also gets the word out about Perl 5.
• Talk about Platypus at local and regional events. I will be giving a presentation at DCBPW (http://dcbpw.org/dcbpw2015/talk/6008) and YAPC::NA (http://www.yapcna.org/yn2015/talk/6072).
• Favorite FFI::Platypus in metacpan.org (with your PAUSE id)
• Recruit people to take over some of the language plugins / custom type modules, to get people invested in FFI infrastructure
• get the ppm build for this thing working. Right now it gets bogged down trying to install PkgConfig.pm see #37 Work arounds in PkgConfig.pm would be acceptable.

Hey @plicease,

Here is a simplified test case for segfaulting while using FFI::Platypus::Record to interface with libtcod. I'm running this on OS X against libtcod v1.5.1 from Homebrew (the games keg). I'm using perl v5.18.2 and FFI::Platypus v0.37. The segfault specifically occurs when trying to build a TcodColor object retrieved via the api.

https://gist.github.com/anonymous/fc337a8ec8083de00b19

Cheers,
Fitz

libtcod docs:
libtcod api documentation
console drawing api

You can extract constants from a .h file via C::Scan from memory I used this with Archive::Libarchive::FFI at some point, though I think it got replaced later on. Would be a good example + discussion for the documentation. The downside of course to C::Scan is that you may need development packages installed to use it.

C::Scan also gets some other information that might be relevant.

basically expose this to a custom type callback:

#define ffi_pl_arguments_count(arguments)                 (arguments->count)
#define ffi_pl_arguments_set_pointer(arguments, i, value) (arguments->slot[i].pointer = value)
#define ffi_pl_arguments_get_pointer(arguments, i)        (arguments->slot[i].pointer)
#define ffi_pl_arguments_set_string(arguments, i, value)  (arguments->slot[i].string  = value)

#define ffi_pl_arguments_set_sint8(arguments, i, value)   (arguments->slot[i].sint8   = value)
#define ffi_pl_arguments_get_sint8(arguments, i)          (arguments->slot[i].sint8)
#define ffi_pl_arguments_set_uint8(arguments, i, value)   (arguments->slot[i].uint8   = value)
#define ffi_pl_arguments_get_uint8(arguments, i)          (arguments->slot[i].uint8)
#define ffi_pl_arguments_set_sint16(arguments, i, value)  (arguments->slot[i].sint16  = value)
#define ffi_pl_arguments_get_sint16(arguments, i)         (arguments->slot[i].sint16)
#define ffi_pl_arguments_set_uint16(arguments, i, value)  (arguments->slot[i].uint16  = value)
#define ffi_pl_arguments_get_uint16(arguments, i)         (arguments->slot[i].uint16)
#define ffi_pl_arguments_set_sint32(arguments, i, value)  (arguments->slot[i].sint32  = value)
#define ffi_pl_arguments_get_sint32(arguments, i)         (arguments->slot[i].sint32)
#define ffi_pl_arguments_set_uint32(arguments, i, value)  (arguments->slot[i].uint32  = value)
#define ffi_pl_arguments_get_uint32(arguments, i)         (arguments->slot[i].uint32)
#define ffi_pl_arguments_set_sint64(arguments, i, value)  (arguments->slot[i].sint64  = value)
#define ffi_pl_arguments_get_sint64(arguments, i)         (arguments->slot[i].sint64)
#define ffi_pl_arguments_set_uint64(arguments, i, value)  (arguments->slot[i].uint64  = value)
#define ffi_pl_arguments_get_uint64(arguments, i)         (arguments->slot[i].uint64)

#define ffi_pl_arguments_set_float(arguments, i, value)  (arguments->slot[i].xfloat  = value)
#define ffi_pl_arguments_get_float(arguments, i)         (arguments->slot[i].xfloat)
#define ffi_pl_arguments_set_double(arguments, i, value)  (arguments->slot[i].xdouble  = value)
#define ffi_pl_arguments_get_double(arguments, i)         (arguments->slot[i].xdouble)

later these same "functions" should be available from a C/XS custom type.