Can't load multiple libraries exposed through server about go-nfs HOT 15 CLOSED

The device number, Blocks, and IO Blocks are also different.

from go-nfs.

that you can access files implies that the file handles provided by NFS are distinct. There is not a direct passing of an underlying filesystem inode over NFS, nor is the underlying filesystem layout of Blocks translated over the virtual file system.

from go-nfs.

link loaders are abstracted from the physical layout of data on disk, the inode/block information are at that layer, so should be distinct from issues loading linked libraries.

The immediate thing I wonder about is that libraries very often are linked using absolute paths, so depending on your mount point you may fail to open the expected data as the library will have been 'relocated' / mounted at a different absolute path from when the link was created.

from go-nfs.

In this case (bash), the libraries are linked with sonames, so this is not an/the issue. To completely rule this out, I tried sudo chroot mount bash -i, which still failed.

I think a problem is that the loader can't get version information, for some reason.

from go-nfs.

i'm guessing this is using https://github.com/DUOLabs333/overlay-nfs - can you provide an example of the code you're attempting to compile/link in the overlayfs so I can see if i can reproduce?

from go-nfs.

I exposed /usr/lib with overlay-nfs /=RO ~/test_mount. Then, I'm running bash with LD_LIBRARY_PATH=~/test_mount/usr/lib /bin/bash.

from go-nfs.

While diffing debug logs, I found that with LD_LIBRARY_PATH=~/test_mount/usr/lib /bin/bash, the linker finds /usr/lib/lib{c,ncursesw}.so, but skips them both (it doesn't do anything with them). libc has mbrtowc, but since it skips it, it says it's not found.

Of course, now the question is, why is it skipping the files.

from go-nfs.

do you happen to have selinux running? i wonder if it could be something like containers/podman#8985

from go-nfs.

No, but that wouldn't explain why some work and some don't (I would kind of get why libc wouldn't work, but why doesn't libncursesw work?)

from go-nfs.

I have also noticed that the readelf output of test_mount/usr/lib/libc.so.6 has extra tabs at the end vs. the output for /usr/lib/libc.so.6:

stapsdt              0x00000054       NT_STAPSDT (SystemTap probe descriptors)            Provider: libc
    Name: memory_mallopt_trim_threshold
    Location: 0x0000000000093014, Base: 0x000000000015e008, Semaphore: 0x0000000000000000
    Arguments: 8@x20 8@[x0] -4@[x0, 72]
  stapsdt              0x00000047       NT_STAPSDT (SystemTap probe descriptors)            Provider: libc
    Name: memory_mallopt_arena_max
    Location: 0x000000000009303c, Base: 0x000000000015e008, Semaphore: 0x0000000000000000
    Arguments: 8@x20 8@[x0, 32]
  stapsdt              0x00000058       NT_STAPSDT (SystemTap probe descriptors)            Provider: libc
    Name: memory_mallopt_mmap_threshold
    Location: 0x0000000000093054, Base: 0x000000000015e008, Semaphore: 0x0000000000000000
    Arguments: 8@x20 8@[x0, 16] -4@[x0, 72]
  stapsdt              0x00000046       NT_STAPSDT (SystemTap probe descriptors)            Provider: libc
    Name: memory_mallopt_perturb
    Location: 0x0000000000093070, Base: 0x000000000015e008, Semaphore: 0x0000000000000000
    Arguments: -4@x20 -4@[x0, 8]

vs.

stapsdt              0x00000054  NT_STAPSDT (SystemTap probe descriptors)     Provider: libc
    Name: memory_mallopt_trim_threshold
    Location: 0x0000000000093014, Base: 0x000000000015e008, Semaphore: 0x0000000000000000
    Arguments: 8@x20 8@[x0] -4@[x0, 72]
  stapsdt              0x00000047  NT_STAPSDT (SystemTap probe descriptors)     Provider: libc
    Name: memory_mallopt_arena_max
    Location: 0x000000000009303c, Base: 0x000000000015e008, Semaphore: 0x0000000000000000
    Arguments: 8@x20 8@[x0, 32]
  stapsdt              0x00000058  NT_STAPSDT (SystemTap probe descriptors)     Provider: libc
    Name: memory_mallopt_mmap_threshold
    Location: 0x0000000000093054, Base: 0x000000000015e008, Semaphore: 0x0000000000000000
    Arguments: 8@x20 8@[x0, 16] -4@[x0, 72]
  stapsdt              0x00000046  NT_STAPSDT (SystemTap probe descriptors)     Provider: libc
    Name: memory_mallopt_perturb
    Location: 0x0000000000093070, Base: 0x000000000015e008, Semaphore: 0x0000000000000000
    Arguments: -4@x20 -4@[x0, 8]

from go-nfs.

Even more interesting, I can force the loading of libraries with LD_PRELOAD and it will be loaded correctly.

from go-nfs.

This makes me think that it's more likely to be an issue with the configured "ld search path" and/or the linker cache, than an issue with enumerating or reading from the mounted filesystem.

from go-nfs.

It seems that only the first library gets loaded --- if I preload libc, libreadline no longer gets loaded. However, this only happens if I preload libc from the mounted directory.

from go-nfs.

I've been looking at the linker source code, and going back to the inode number issue, there are multiple checks to see if two inode numbers are the same --- it's possible that the loader sees that two files' inodes are 0, and assumes they're the same file, skipping them. This would explain why I can only load one library.

from go-nfs.

This fixed the issue --- by adding

stat, ok :=info.Sys().(*syscall.Stat_t)
if ok{
   f.Fileid=stat.Ino
}

to ToFileAttribute, I was able to load the libraries.

from go-nfs.