nokogiri fails health_check -- extensions use the wrong zlib about omnibus-software HOT 28 CLOSED

I tried adding explicit libxslt and libxml2 dependencies, but still get the same error. Any leads on a fix (or hack)?

from omnibus-software.

yes. you'll need to add libz.so.1 to the whitelist thats in heath_check.rb in the omnibus gem. i guess i should actually make this against the omnibus-ruby project instead.

from omnibus-software.

I can't seem to find this 'heath_check.rb' file in any of the omnibus repos..?

EDIT: Oh, haha HEATH_check

from omnibus-software.

sorry. typo. health_check.rb

from omnibus-software.

Hm. That whitelist add didn't work for me. Here's my health_check backtrace:

[health_check] Executing `find /opt/rumm2/ -type f | egrep '.(dylib|bundle)$' | xargs otool -L > otool.out 2>/dev/null`
[health_check] *** Health Check Failed, Summary follows:
[health_check] *** The following Omnibus-built libraries have unsafe or unmet dependencies:
[health_check]     --> /opt/rumm2//embedded/lib/ruby/gems/1.9.1/gems/nokogiri-1.5.10/ext/nokogiri/nokogiri.bundle
[health_check]     --> /opt/rumm2//embedded/lib/ruby/gems/1.9.1/gems/nokogiri-1.5.10/lib/nokogiri/nokogiri.bundle
[health_check] *** The following Omnibus-built binaries have unsafe or unmet dependencies:
[health_check] *** The following libraries cannot be guaranteed to be on target systems:
[health_check]     --> /usr/lib/libexslt.0.dylib
[health_check]     --> /usr/lib/libxslt.1.dylib
[health_check]     --> /usr/lib/libxml2.2.dylib
[health_check] *** The precise failures were:
[health_check]     --> /opt/rumm2//embedded/lib/ruby/gems/1.9.1/gems/nokogiri-1.5.10/ext/nokogiri/nokogiri.bundle
[health_check]     DEPENDS ON: libexslt.0.dylib
[health_check]       COUNT: 1
[health_check]       PROVIDED BY: /usr/lib/libexslt.0.dylib
[health_check]       FAILED BECAUSE: Unsafe dependency
[health_check]     --> /opt/rumm2//embedded/lib/ruby/gems/1.9.1/gems/nokogiri-1.5.10/ext/nokogiri/nokogiri.bundle
[health_check]     DEPENDS ON: libxslt.1.dylib
[health_check]       COUNT: 1
[health_check]       PROVIDED BY: /usr/lib/libxslt.1.dylib
[health_check]       FAILED BECAUSE: Unsafe dependency
[health_check]     --> /opt/rumm2//embedded/lib/ruby/gems/1.9.1/gems/nokogiri-1.5.10/ext/nokogiri/nokogiri.bundle
[health_check]     DEPENDS ON: libxml2.2.dylib
[health_check]       COUNT: 1
[health_check]       PROVIDED BY: /usr/lib/libxml2.2.dylib
[health_check]       FAILED BECAUSE: Unsafe dependency
[health_check]     --> /opt/rumm2//embedded/lib/ruby/gems/1.9.1/gems/nokogiri-1.5.10/lib/nokogiri/nokogiri.bundle
[health_check]     DEPENDS ON: libexslt.0.dylib
[health_check]       COUNT: 1
[health_check]       PROVIDED BY: /usr/lib/libexslt.0.dylib
[health_check]       FAILED BECAUSE: Unsafe dependency
[health_check]     --> /opt/rumm2//embedded/lib/ruby/gems/1.9.1/gems/nokogiri-1.5.10/lib/nokogiri/nokogiri.bundle
[health_check]     DEPENDS ON: libxslt.1.dylib
[health_check]       COUNT: 1
[health_check]       PROVIDED BY: /usr/lib/libxslt.1.dylib
[health_check]       FAILED BECAUSE: Unsafe dependency
[health_check]     --> /opt/rumm2//embedded/lib/ruby/gems/1.9.1/gems/nokogiri-1.5.10/lib/nokogiri/nokogiri.bundle
[health_check]     DEPENDS ON: libxml2.2.dylib
[health_check]       COUNT: 1
[health_check]       PROVIDED BY: /usr/lib/libxml2.2.dylib
[health_check]       FAILED BECAUSE: Unsafe dependency

from omnibus-software.

yep for OS X you'll need to it under -- MAC_WHITELIST_LIBS
but it looks like you're having trouble with xml2 and xslt libraries. in your nokogiri.rb file are you specifying the correct --with-xslt-lib and --with-xml2-lib directories?

from omnibus-software.

Fixed in osx (at least) adding libexslt.0.dylib, libxslt.1.dylib and libxml2.2.dylib to MAC_WHITELIST_LIBS.

from omnibus-software.

@cgallimore Tried this same technique with a health_check failure in ubuntu. Did not fix it (made it worse)

from omnibus-software.

hmm. check again. i've had plenty of success with this. i created a PR against omnibus-ruby for this, but for Ubuntu all you need is /libz.so.\d/

from omnibus-software.

Hm. I added nokogiri as an explicit dependency and it built.

from omnibus-software.

can you post that?

from omnibus-software.

Bumped into this with nokogiri depending on libz.so.1 outside embedded on ubuntu also.

Is the 'correct' solution to modify the heath check? or is it to build a zlib in embedded?

from omnibus-software.

@jessereynolds If it's asking for libz.so.1 during the health check then modify the whitelist. If nokogiri is failing to build, then you might need zlib.

from omnibus-software.

@hayesmp so when you say 'modify the whitelist', do you mean add /lib/x86_64-linux-gnu/libz.so.1 to the whitelist using the whitlist_file method in our nokogiri definition?

nokogiri 1.6.0 is building fine but failing health check with a lot of messages like this:

[health_check]     --> /opt/flapjack/embedded/lib/ruby/gems/1.9.1/gems/nokogiri-1.6.0/ext/nokogiri/tmp/x86_64-linux-gnu/ports/libxslt/1.1.26/libxslt-1.1.26/libexslt/.libs/libexslt.so.0.8.15
[health_check]     DEPENDS ON: libz.so.1
[health_check]       COUNT: 1
[health_check]       PROVIDED BY: /lib/x86_64-linux-gnu/libz.so.1 (0x00007f5fe2df0000)
[health_check]       FAILED BECAUSE: Unsafe dependency

I guess, for an ubuntu-precise64 package (which is the target of this project), can we always rely on their being a libz.so.1 at /lib/x86_64-linux-gnu/libz.so.1 ? If so then whitelist. If not, then we need to pass some additional flags to nokogiri to have it use the zlib we've built within the project?

from omnibus-software.

Older version of nokogiri doesn't seem to have this issue (1.5.4). 1.6.0 seems to have problems linking certain files to the system's zlib, rather than the embedded one.

from omnibus-software.

Good find Robby! I'll try my project with 1.5.4.

from omnibus-software.

I'm unable to revert to nokogiri 1.5.4 due to issues with it, alas.

I've tried adding /lib/x86_64-linux-gnu/libz.so.1 to @whitelist_files using whitelist_file but this isn't having the desired effect.

Is the only solution really to hack the omnibus-ruby gem?

from omnibus-software.

As far as i can tell, yes. I opened a PR in omnibus-ruby to to just that, but it was closed as won't fix.

from omnibus-software.

The strange thing is that the nokogiri lib itself links to the proper (embedded) libz, but other seemingly random things do not.

Ex.
This looks correct:

 [vagrant@albert-omnibus-build-lab nokogiri-1.6.0]$ ldd lib/nokogiri/nokogiri.so  |grep libz
    libz.so.1 => /opt/albert/embedded/lib/libz.so.1 (0x00007f65309d7000)

This looks incorrect:

    [vagrant@albert-omnibus-build-lab nokogiri-1.6.0]$ ldd ext/nokogiri/tmp/x86_64-redhat-linux/ports/libxml2/2.8.0/libxml2-2.8.0/doc/examples/.libs/testWriter | grep libz
    libz.so.1 => /lib64/libz.so.1 (0x000000323be00000)

It's as if nokogiri is ignoring the --with-*-lib commands for certain files.

from omnibus-software.

I actually got around the nokogiri failure by adding nokogiri as a dependency in my software definition.

from omnibus-software.

@hayesmp That will use the omnibus-software hosted "nokogiri", which is using v1.5.4. 1.5.4 definitely works, it's 1.6.0 we're having issues with.

from omnibus-software.

@robbydyer Ah, yeah.

from omnibus-software.

Add NOKOGIRI_USE_SYSTEM_LIBRARIES to env to avoid this

dependency "libxml2"
dependency "libxslt"

env = {
  "PATH" => "#{install_dir}/embedded/bin:#{ENV["PATH"]}",
  "LD_FLAGS" => "-L#{install_dir}/embedded/lib -I#{install_dir}/embedded/include",
  "LD_RUN_PATH" => "#{install_dir}/embedded/lib",
  "LD_LIBRARY_PATH" => "#{install_dir}/embedded/lib",
  "NOKOGIRI_USE_SYSTEM_LIBRARIES" => "1"
}

build do
  bundle "install", :env => env
end

from omnibus-software.

We would gladly review a Pull Request as @Meecr0b suggested

from omnibus-software.

I'm still seeing this with latest omnibus-software and omnibus-ruby 3.1.1

    --> /opt/flapjack/embedded/service/gem/ruby/1.9.1/gems/nokogiri-1.6.1/ports/x86_64-linux-gnu/libxslt/1.1.26/lib/libxslt.so.1.1.26
    DEPENDS ON: libz.so.1
      COUNT: 1
      PROVIDED BY: /lib/x86_64-linux-gnu/libz.so.1 (0x00007f666a86b000)
      FAILED BECAUSE: Unsafe dependency

from omnibus-software.

something is installing a gem which pulls in nokogiri 1.6.1 which is not getting compiled to use the system libraries. whatever definition is installing the gem needs to have 'dependency "nokogiri"' in it, in order to pull in the nokogiri definition from omnibus-software first and install it.

you may need to override the version and install exactly nokogiri 1.6.1 via omnibus in order to satisfy the requirements of the gem that is pulling in nokogiri.

from omnibus-software.

@lamont-granquist thanks, yeah i just realised my software definition for flapjack is installing nokogiri again (as a gem dependency). I'll look into why it's not happy with the nokogiri already installed from the declared dependency.

from omnibus-software.

there seems to be some bundler/rubygems bug with depsolving where it
likes to install gems that are already satisfied by deps. you can try
to beat on different versions of those until you find a solution, but
we gave up and just installed the version of nokogiri that the gem
install /really/ wanted to install.

from omnibus-software.