Build failure on Fedora 23 about securefs HOT 23 CLOSED

It looks like a bug in libstdc++. The solution is probably disabling inline assembly in the dependency crypto++. I will install a Fedora virtual machine later to further investigate this issue.

from securefs.

@rfjakob

I just installed a fresh Fedora 23 with g++ 5.3.1-2 and fuse-devel and built securefs successfully. Can you share more about how to reproduce your build failure?

from securefs.

Here is the complete build log: https://gist.github.com/rfjakob/24584dde58bd3d4cea1e

The problem seems to be that mm_clmulepi64_si128 and friends are already defined in /usr/lib/gcc/x86_64-redhat-linux/5.3.1/include/wmmintrin.h, and cryptopp defines them again. Do you have the wmmintrin.h file on your VM?

Edit: Wrong file. It is wmmintrin.h, not x86intrin.h.

from securefs.

@rfjakob

It has both wmmintrin.h and x86intrin.h, in /usr/lib/gcc/x86_64-redhat-linux/5.3.1/include as well. Can you dump the contents of your wmmintrin.h and x86intrin.h? The build log suggests that the functions they define are not inline, thus violating One Definition Rule.

You may also try building a vanilla Crypto++ downloaded from the official website. Mine includes some modifications in order to be built with flag --std=c++11.

from securefs.

wmmintrin.h: https://gist.github.com/rfjakob/79cae8b3ac49621fffd5

I tried building crypto++ and it works. However, they use this build command for vmac.c

g++ -DNDEBUG -g -O2 -march=native -pipe -std=c++11 -c vmac.cpp

while you use

g++ -g -march=native -mtune=native -std=c++11 -pipe -Wall -Wextra -pedantic -pthread -O3 -DNDEBUG -ffunction-sections -fdata-sections -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -march=native -pipe -c vmac.cpp

I have tracked it down to -std=c++11. Adding this to the vanilla build command causes the errors I see.

from securefs.

I now have also tried it in a fresh Fedora 23 VM (64 bit). I get the same error.

from securefs.

@rfjakob

I now have also tried it in a fresh Fedora 23 VM (64 bit). I get the same error.

I installed another Fedora Workstation (x64) with default packages set, and I still built the program successfully. Can you give more details? Like the specific URL from which you download the installation media as well as the packages you installed over the base system?

from securefs.

I am using the Fedora Server Netinstall ISO, because it is a lot smaller
than the other ISOs. Download it here:
https://download.fedoraproject.org/pub/fedora/linux/releases/23/Server/x86_64/iso/Fedora-Server-netinst-x86_64-23.iso

Select "Minimal Install" from Software Selection

Once the installation is complete, I think it is a good idea to take a
snapshot of the VM so we can revert quickly to the initial state.

Use

ip addr show

to get the IP address and connect via SSH. Then,

sudo dnf install fuse fuse-devel git make gcc gcc-c++
git clone https://github.com/netheril96/securefs.git
cd securefs
make

You get the error as described above:

from securefs.

@rfjakob

I followed your approach and still built successfully. I now think it is probably something wrong with gcc combined with your specific CPU model (the usage of AES-NI and optimization flag --march=native is highly CPU specific). I have pushed to the master branch a flag to disable AES-NI. Please update and try build with make securefs NOASM=1 to see if the issue goes away.

from securefs.

Remember to run make deepclean before trying to rebuild.

from securefs.

Sorry, did not fix it:

git clean -dxf
make securefs NOASM=1
[...]
cpu.h:51:1: error: redefinition of ‘__m128i _mm_clmulepi64_si128(__m128i, __m128i, int)’

BTW I am running a "Sandy Bridge" Pentium G630 WITHOUT aes instructions.

from securefs.

@rfjakob

OK, I fumbled around and was able to reproduce the issue by deleting the lines

#if !defined(__GNUC__) || (defined(__AES__) && defined(__PCLMUL__)) || defined(__INTEL_COMPILER)
#include <wmmintrin.h>
#else
.....
#endif

I guess the lack of AES-NI brought you to the other branch. Then I encountered the same error.

And the solution is building with clang++. I built it successfully with clang after reproducing the issue.

It is probably a regression bug in GCC 5.3.

In a nutshell, please try

sudo dnf install clang
CC=clang CXX=clang++ make securefs

from securefs.

Yes, I got it to build!

Performance looks very good as well, congratulations!

from securefs.

FYI, I have added securefs to the comparison page at https://nuetzlich.net/gocryptfs/comparison/ . Tell me if I got something wrong. Best regards!

from securefs.

@rfjakob

The lines of code is off. It should be 5469. The sources inline some of third party dependencies, including cryptopp, json and cppformat, which should be excluded from the line count.

from securefs.

Case closed.

from securefs.

About the line count: How do I get this number? Can you give me a cloc command line?

Stupid question: Why do you include all those libraries, anyway? I have not checked all of them, but it seems that most are available as regular packages in Debian and Fedora

crypto++:
https://apps.fedoraproject.org/packages/cryptopp-devel
https://packages.debian.org/jessie/libcrypto++-dev

tclap:
https://apps.fedoraproject.org/packages/tclap
https://packages.debian.org/jessie/libtclap-dev

from securefs.

I don't take care to separate my own sources from third party dependencies, so there are no easy way to calculate the line count. I manually list all the files I have written myself and count them. The list may change in the future if I consolidate certain files or add new features.

cat sources/btree_dir.h sources/commands.h sources/exceptions.h sources/file_table.h sources/files.h sources/logger.h sources/operations.h sources/streams.h sources/utils.h sources/xattr_compat.h sources/btree_dir.cpp sources/commands.cpp sources/file_table.cpp sources/files.cpp sources/logger.cpp sources/operations.cpp sources/streams.cpp sources/utils.cpp | wc -l

All crypto++ packages are built with c++98, while my sources are in c++11. The ABI of the two language versions are not compatible, and the issue is especially serious given the heavy usage of templates in crypto++. So it must be built with --std=c++11 for my program to be linked against. That is why I directly put its sources under my own.

The reason for inlining other libraries is my laziness. I don't want to deal with the fact that certain packages exist on Fedora but not Debian, or exist on Ubuntu 14.04 but not on 12.04, or that the version on the user's system is too old or too new. Finding the location of these libraries automatically on a myriad configuration of operating systems is also something I'd like to avoid.

from securefs.

By the way, I am amazed by how much you find out about this project as shown on your comparison page. Parameters like max filename length are not documented anywhere, yet you find out. Do you read the source code? And understand it in that short amount of time?

from securefs.

I have updated the lines of code number acc. to your file list!

No, I do not read any code for the comparison. I just mount it and test the limitations.

from securefs.

But its really nice that you can have the full 255 characters. This is a frequent complaint about encfs and ecryptfs as well.

A downside may be that when people put their files onto dropbox, they can get conflicts on the B+ files, which can be difficult to resolve.

from securefs.

Hi,

I have been looking for a solution to encrypt my dropbox files and found securefs which looks most promising to me. Are there users who use securefs with Dropbox for daily use?

from securefs.

@hanthethanh You should open a new issue for it.

from securefs.