vips dzsave core dumps about libvips HOT 10 CLOSED

Sometimes it just says CORE DUMPED but actually doesn't dump anything, I also tried different tile sizes:

ubuntu@ip-10-28-209-79:~$ date && vips dzsave Leica-1.scn p256_8 --layout google --tile-size 256 --background '221 221 221 255' && date
Tue Dec 18 16:55:03 UTC 2012
Segmentation fault (core dumped)

ubuntu@ip-10-28-209-79:~$ date && vips dzsave Leica-1.scn p255_9 --layout google --tile-size 255 --background '221 221 221 255' && date
Tue Dec 18 16:56:09 UTC 2012
Segmentation fault (core dumped)

ubuntu@ip-10-28-209-79:~$ date && vips dzsave Leica-1.scn p200_10 --layout google --tile-size 200 --background '221 221 221 255' && date
Tue Dec 18 16:57:52 UTC 2012
*** glibc detected *** vips: double free or corruption (fasttop): 0x0000000001609a50 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x7eb96)[0x7f07895b5b96]
...
...
7fff7f1ff000-7fff7f200000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
Aborted (core dumped)

The one tile size that tends to actually do the job is 512px:

ubuntu@ip-10-28-209-79:~$ date && vips dzsave Leica-1.scn p512_11 --layout google --tile-size 512 --background '221 221 221 255' && date
Tue Dec 18 16:58:42 UTC 2012
Tue Dec 18 17:01:10 UTC 2012
ubuntu@ip-10-28-209-79:~$ 

from libvips.

Hi, looks like it's a problem with google layout. If I compile with debugging I see:

$ vips dzsave wtc.jpg x --layout google
** VIPS:ERROR:type.c:164:vips_area_unref: assertion failed: (area->count > 0)
Aborted (core dumped)

I'll investigate tomorrow.

from libvips.

This commit fixes it for me:

jcupitt@8b67d53

The problem was that google maps output copies the background property for each tile it makes, and these tiles are made and freed in a background thread. There need to be locks on the reference counts used to manage array property lifetimes.

Thanks for reporting this nasty thing.

from libvips.

Hello,
thanks for quick reply. But it seem that the issue is still there!
Somehow it feels like it's happening less often with 512 tiles than with 256 tiles (less tiles, less memory lost?)

from libvips.

I haev recompiled, then ldconfig, then restarted the virtual machine and it is still there I will terminate the virtual machine and create new one where I will compile on a clean slate and will report back.

from libvips.

Oh dear. I'll download Leica-1.scn and try with that tomorrow. This is the openslide test image I guess?

Does it work if you try with a regular jpg image?

J

from libvips.

I haven't tried with regular image I will be using with digital slides, so I am testing it with slides.
After re-install of the machine it seems to work fine, actually I am quite impressed with it:

ubuntu@ip-10-38-191-150:~$ rm -rf p && date && vips dzsave Leica-1.scn p --layout google --tile-size 64 && date
Wed Dec 19 17:03:49 UTC 2012
Wed Dec 19 17:14:23 UTC 2012
ubuntu@ip-10-38-191-150:~$ du -bs p/
698403463   p/
ubuntu@ip-10-38-191-150:~$ find p/ -type f | wc -l
461920

I just generated half a million files (altogether 700MB of them) in less than 9 minutes! 👍

I still can't get it to work where I need to use it though :(
Is there any way to build it for 64bit linux system so that it's portable? One big binary I could just run?

Right now what I do is hunt down libraries it requires one by one using ldd and th ncpy them to ~/lib, same for binaries which go to ~/bin and then I change PATH and LD_LIBRARY_PATH. And pack this whole thing and ship it to other system where it is spawned when needed. I still can't seem to be able to get it to work reliably though :(

from libvips.

Glad it's working!

There are so many ways to deploy, it's hard to know what would be best for you without knowing a lot of detail.

The simplest is just to build vips on each machine, or perhaps have a single VM image you send out to the servers.

Next is to use jhbuild to make a private tree, then zip that up and copy to the target machines.

If this is Debian or Redhat, you can make your own repository, put a private vips binary in there and deploy with apt or redcarpet. SuSE has something similar I think.

It's not really my field though. There's a sysadmin stackoverflow now, perhaps they have a guide?

from libvips.

Hi, we are having trouble installing libvips on CentOS 6,

Any chance you will put a new tarball on the website soon?

This is what we are getting:

./bootstrap.sh
cp: cannot stat `/usr/share/aclocal/isc-posix.m4': No such file or directory
testing for glibtoolize ...
using libtoolize
Copying file mkinstalldirs
Copying file po/Makefile.in.in

Please add the files
  codeset.m4 gettext.m4 glibc21.m4 iconv.m4 isc-posix.m4 lcmessage.m4
  progtest.m4
from the /usr/share/aclocal directory to your autoconf macro directory
or directly to your aclocal.m4 file.
You will also need config.guess and config.sub, which you can get from
ftp://ftp.gnu.org/pub/gnu/config/.

configure.in:91: installing `./config.guess'
configure.in:91: installing `./config.sub'
configure.in:12: installing `./install-sh'
configure.in:12: installing `./missing'
gtk-doc.make:55: GTK_DOC_BUILD_HTML does not appear in AM_CONDITIONAL
doc/reference/Makefile.am:89:   `gtk-doc.make' included from here
gtk-doc.make:60: GTK_DOC_BUILD_PDF does not appear in AM_CONDITIONAL
doc/reference/Makefile.am:89:   `gtk-doc.make' included from here
libvips/Makefile.am: installing `./depcomp'
swig/vipsCC/Makefile.am:13: installing `./py-compile'

and then this when compiling from source (the old 7.30.6 from web):

checking for THREADS... ./configure: line 22756: syntax error near unexpected token `else'
./configure: line 22756: `else'

I finally feel quite comfortable getting VIPS to work on Debian based distributions, but this is Amazon Linux which is very close to Cent OS 6 and I have noc lue what is going on, it's like a different world :( .

from libvips.

I've opened a new issue for this one:

https://github.com/jcupitt/libvips/issues/47

It'll make it easier for other people to find if they have the same problem.

from libvips.