berkus / snappy Goto Github PK

What steps will reproduce the problem?
1. Read http://code.google.com/p/snappy/ Note: "Benchmarks against a few other 
compression libraries (zlib, LZO, LZF, FastLZ, and QuickLZ) are included in the 
source code distribution."
2. Download source according http://code.google.com/p/snappy/source/checkout
3. Read README
4. Guess how to compile and try it Note: "
checking for zlibVersion in -lz... yes
checking for lzo1x_1_15_compress in -llzo2... yes" in configure output.
5. Try benchmark according README

I would expect something like http://pastebin.com/hj0tLuXW

But I got: --lzo: No such file or directory

I also tried --zlib, --LZO, -LZO, ...

README should contain exact step by step instructions how to achieve promised 
results.

The code says it all:

struct BenchmarkCompareCPUTime {
  bool operator() (const BenchmarkRun& a, const BenchmarkRun& b) const {
    return a.real_time_us < b.real_time_us;
  }
};

Also, it would be slightly more elegant to use nth_element than sort.

IMHO, the build system could do with a little work:

* The various bits generated from and added by the autotools shouldn't be 
committed. autoreconf -i works really well these days. That's INSTALL 
Makefile.in aclocal.m4 compile config.guess config.h.in config.sub configure 
depcomp install-sh ltmain.sh missing mkinstalldirs.

configure.ac:

* Needs to call AC_SUBST([LIBTOOL_DEPS]) or else the rule to rebuild libtool in 
Makefile.am won't work.

* A lot of macro calls are underquoted. It'll probably work fine, but it's poor 
style.

* The dance with EXTRA_LIBSNAPPY_LDFLAGS seems odd. It'd be more conventional 
to do something like:

    SNAPPY_LTVERSION=snappy_version
    AC_SUBST([SNAPPY_LTVERSION])

and set the -version-info flag directly in Makefile.am. If it's to allow the 
user to provide custom LDFLAGS, it's unnecessary: LDFLAGS is part of 
libsnappy_la_LINK. Here's the snippet from Makefile.in:

    libsnappy_la_LINK = $(LIBTOOL) --tag=CXX $(AM_LIBTOOLFLAGS) \
            $(LIBTOOLFLAGS) --mode=link $(CXXLD) $(AM_CXXFLAGS) \
            $(CXXFLAGS) $(libsnappy_la_LDFLAGS) $(LDFLAGS) -o $@

* There should be an AC_ARG_WITH for gflags, because automagic dependencies 
aren't cool: http://www.gentoo.org/proj/en/qa/automagic.xml

* Shell variables starting with ac_ are in autoconf's namespace. Setting things 
like ac_have_builtin_ctz is therefore equally uncool. See 
http://www.gnu.org/s/hello/manual/autoconf/Macro-Names.html :

> To ensure that your macros don't conflict with present or future Autoconf 
macros, you should prefix your own macro names and any shell variables they use 
with some other sequence. Possibilities include your initials, or an 
abbreviation for the name of your organization or software package.

* Use AS_IF instead of directly using the shell's `if`: 
http://www.gnu.org/software/hello/manual/autoconf/Limitations-of-Builtins.html#L
imitations-of-Builtins and 
http://www.gnu.org/s/hello/manual/autoconf/Common-Shell-Constructs.html .

* Consider adding -Wall to either AUTOMAKE_OPTIONS in Makefile.am or as an 
argument to AM_INIT_AUTOMAKE. If you don't mind using a modern automake (1.11 
or later), also call AM_SILENT_RULES([yes]). Even MSYS has automake-1.11 these 
days.

Makefile.am:

* Adding $(GTEST_CPPFLAGS) to both snappy_unittest_CPPFLAGS and 
snappy_unittest_CXXFLAGS is redundant. See this part of Makefile.in:

    snappy_unittest-snappy-test.o: snappy-test.cc
    @am__fastdepCXX_TRUE@   $(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(snappy_unittest_CPPFLAGS) $(CPPFLAGS) $(snappy_unittest_CXXFLAGS) $(CXXFLAGS) -MT snappy_unittest-snappy-test.o -MD -MP -MF $(DEPDIR)/snappy_unittest-snappy-test.Tpo -c -o snappy_unittest-snappy-test.o `test -f 'snappy-test.cc' || echo '$(srcdir)/'`snappy-test.cc
    @am__fastdepCXX_TRUE@   $(am__mv) $(DEPDIR)/snappy_unittest-snappy-test.Tpo $(DEPDIR)/snappy_unittest-snappy-test.Po
    @AMDEP_TRUE@@am__fastdepCXX_FALSE@      source='snappy-test.cc' object='snappy_unittest-snappy-test.o' libtool=no @AMDEPBACKSLASH@
    @AMDEP_TRUE@@am__fastdepCXX_FALSE@      DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@
    @am__fastdepCXX_FALSE@  $(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(snappy_unittest_CPPFLAGS) $(CPPFLAGS) $(snappy_unittest_CXXFLAGS) $(CXXFLAGS) -c -o snappy_unittest-snappy-test.o `test -f 'snappy-test.cc' || echo '$(srcdir)/'`snappy-test.cc

* snappy_unittest should be in check_PROGRAMS, not noinst_PROGRAMS. That way, 
it's built as part of `make check`, not `make all`.

What steps will reproduce the problem?
1. Building snappy without using standard ./configure; make steps 
 (e.g., using Visual Studio, etc.)

What is the expected output? What do you see instead?
An error, config.h is not found, occurs at snappy-stubs-internal.h 

What version of the product are you using? On what operating system?
snappy-1.0.1, and the latest revision 22

Please provide any additional information below.

Please wrap config.h import with HAVE_CONFIG_H guard to achieve the portability.
#ifdef HAVE_CONFIG_H
#include "config.h"
#endif 

By applying the attached patch, I can successfully build snappy in Windows with 
Visual C++, mingw-g++, etc. 

Fixed typo
Had no instead of not in phrase " * Robust: The Snappy decompressor is designed 
not to crash in the face of
"

r21 seems to have chopped out the first couple lines of the explanatory comment 
after the license header in some files (snappy-internal.h, snappy.h).

http://www.gnu.org/software/hello/manual/autoconf/Autoconf-Language.html

"""
Arguments should be enclosed within the quote characters ‘[’ and ‘]’, 
and be separated by commas.
"""

The page goes on to provide an example where quoting isn't require and goes on 
to say:

"""
Cautious Autoconf users would keep the quotes, but many Autoconf users find 
such precautions annoying...
"""

Personally, I am a cautious autoconf user, and wanted to bring this to your 
attention.

If zlib, LZO, etc. are found, these are used for snappy_unittest. However, 
they're also linked into libsnappy.so, creating possibly false dependencies in 
distribution packages. We should

  a) Stop linking libsnappy.so against these libs (the unit test alone should do so)
  b) Have an option to not detect third-party compression libraries at all, or
  c) Both a) and b).

The attached patch mirrors the implementation of Snappy_Compress and 
Snappy_Uncompress from port/port_posix.h to port/port_osx.h, making compression 
available in OS X and iOS builds.

trivial typo

Many of the source files state that they are licensed under the Apache License 
Version 2.0.  However, some of the source files have no licensing statement at 
all (snappy-sinksource.h, snappy-sinksource.cc, snappy-stubs-public.h).  
Additionally, the COPYING file has terms in it that are not the Apache 2.0 
terms; rather they are a modified BSD license, similar to the one used by 
google-perftools.

IMHO, The various bits generated from and added by the autotools shouldn't be 
committed. autoreconf -i works really well these days. For this project, that's:

INSTALL
Makefile.in
aclocal.m4
autom4te.cache
compile
config.guess
config.h.in
config.sub
configure
depcomp
install-sh
ltmain.sh
missing
mkinstalldirs

The main problem with this is that it causes the diffs to become really 
cluttered, especially if the local developer is using a different autoconf or 
automake version. Better, IMHO, to put up a list of tested autotools versions 
and a working bootstrap script (or a note to call autoreconf -i, if that's 
sufficient here).

Hello,

It would be really great to have C bindings for snappy for projects which 
prefer plain C instead of C++.

The microbenchmark usually segfaults under GNU/Hurd, because two consecutive 
calls to gettimeofday() on that platform can return the same value (it is only 
updated ever so often), and we divide by the elapsed time.

Listing snappy_unittest in check_PROGRAMS means that it's only built as part of 
`make check`. `make all` builds things that are declared `noinst`, which is not 
the correct time to build a unittest, IMHO. See 
http://www.gnu.org/software/automake/manual/html_node/Uniform.html#index-check_0
05f-141

In particular, this stops it from trying to compile snappy_unittest (and 
possibly failing) if gtest isn't available.

What steps will reproduce the problem?
1. svn checkout on an intel mac running snow leopard
2. run ./autogen.sh 

What is the expected output? What do you see instead?
autogen fails, even after I change libtoolize to glibtoolize

What version of the product are you using? On what operating system?
rev. 21, mac os snow leopard

Please provide any additional information below.

no -> not

What steps will reproduce the problem?
1. run snappy-unittest

What is the expected output? What do you see instead?
  How to disable assert, so there is no warning when I run 
  snappy-unittest. 

What version of the product are you using? On what operating system?
    snappy-1.0.1

Please provide any additional information below.


./snappy_unittest --run_microbenchmarks=false

Running microbenchmarks.
WARNING: Compiled with assertions enabled, will be slow.

As far as I see, COPYING contains the BSD license text. Since snappy is 
licensed under ASL 2.0, the text should be replaced accordingly.

Later in this document, section 2.2.3 refers to 11 as a 4-byte offset, not a 
3-byte offset.  From looking at the code and unit test, it appears that 4-byte 
is correct.

I also changed the heading of 2.2.3 to be consistent with 2.2.1 and 2.2.2 
("offset" instead of "offsets")

What steps will reproduce the problem?
1. Try to build snappy with Solaris Studio compilers

What is the expected output? What do you see instead?
Link error:

Undefined           first referenced
 symbol                 in file
char*snappy::internal::CompressFragment(const char*,unsigned,char*,unsigned 
short*,const int) snappy_unittest-snappy_unittest.o


What version of the product are you using? On what operating system?
Latest code in trunk


Please provide any additional information below.

I've created a patch to fix the problem.

I ported the code to pure C as part of an effort to use it in the Linux kernel. 
When I tested the code as a regular user space dynamic library against snappy's 
tester, I was surprised to find out it was slower, although the inner loop was 
exactly the same machine code. In the tester, I found that for every contender, 
the output buffers are resized prior to compress/uncompress calls. Except for 
snappy. The change accounted for the differences in compression performance.

I ask you to apply the attached patch that fixes this problem. Attached also 
are logs of benchmark runs with and without the patch. Benchmarks were run on a 
desktop pc with X shutdown and cpu clock manually set to not change. Core 2 Duo 
2.33Ghz, Gentoo ~amd64, Glibc 2.13, GCC 4.6.0, just -O2.

With the code as is:

testdata/house.jpg                       :
LZO:    [b 1M] bytes 126958 -> 127173 100.2%  comp  23.9 MB/s  uncomp 1663.8 
MB/s
SNAPPY: [b 4M] bytes 126958 -> 126803 99.9%  comp 2409.3 MB/s  uncomp 8523.6 
MB/s
testdata/html                            :
LZO:    [b 1M] bytes 102400 ->  21027 20.5%  comp 135.2 MB/s  uncomp 494.6 MB/s
SNAPPY: [b 4M] bytes 102400 ->  24140 23.6%  comp 419.1 MB/s  uncomp 850.1 MB/s
testdata/kennedy.xls                     :
LZO:    [b 1M] bytes 1029744 -> 357315 34.7%  comp 157.8 MB/s  uncomp 622.9 MB/s
SNAPPY: [b 4M] bytes 1029744 -> 425735 41.3%  comp 351.8 MB/s  uncomp 512.4 MB/s
testdata/mapreduce-osdi-1.pdf            :
LZO:    [b 1M] bytes  94330 ->  76999 81.6%  comp  29.5 MB/s  uncomp 946.8 MB/s
SNAPPY: [b 4M] bytes  94330 ->  77477 82.1%  comp 837.6 MB/s  uncomp 1969.4 MB/s

With patch applied:

testdata/house.jpg                       :
LZO:    [b 1M] bytes 126958 -> 127173 100.2%  comp  24.0 MB/s  uncomp 1661.1 
MB/s
SNAPPY: [b 4M] bytes 126958 -> 126803 99.9%  comp 2312.3 MB/s  uncomp 8502.0 
MB/s
testdata/html                            :
LZO:    [b 1M] bytes 102400 ->  21027 20.5%  comp 135.7 MB/s  uncomp 494.0 MB/s
SNAPPY: [b 4M] bytes 102400 ->  24140 23.6%  comp 404.6 MB/s  uncomp 851.0 MB/s
testdata/kennedy.xls                     :
LZO:    [b 1M] bytes 1029744 -> 357315 34.7%  comp 157.6 MB/s  uncomp 622.9 MB/s
SNAPPY: [b 4M] bytes 1029744 -> 425735 41.3%  comp 343.4 MB/s  uncomp 513.0 MB/s
testdata/mapreduce-osdi-1.pdf            :
LZO:    [b 1M] bytes  94330 ->  76999 81.6%  comp  29.5 MB/s  uncomp 954.0 MB/s
SNAPPY: [b 4M] bytes  94330 ->  77477 82.1%  comp 812.0 MB/s  uncomp 1971.7 MB/s

What steps will reproduce the problem?

  1. checkout the trunk
  2. 'autoreconf' fails with: Libtool library used but `LIBTOOL' is undefined


What version of the product are you using? On what operating system?

  CentOS 5.5 64
  Autoconf 2.68

Adding 'AC_PROG_LIBTOOL' at the end of configure.ac fixes the problem.

Fix a typo.

Snappy looks like a great idea and we'd be interested in using it in Mozilla, 
unfortunately the Apache license is not compatible with the current version of 
MPL. What are the chances of snappy switching to BSD or another MPL-compatible 
license? 

lzf_compress has a hash parameter on NetBSD; find attached patch which 
compiles/runs. Tested NetBSD -current on AMD64.

Cheers,

-bch

Attached find a svn diff to snappy-internal.h FindMatchLength that slightly 
improves performance, ~0-3.5%.  The usual caveats apply- the compilers and 
compiler versions used, along with the microarchitecture the code runs on, may 
or may not result in an improvement for your combination(s).  All I can say is 
that for me (Mac OS X, Core 2 T9550 @ 2.66GHz, gcc 4.2.1), it results in an 
improvement.

Basically, the change is to modify FindMatchLength so it doesn't explicitly 
keep track of the number of matched bytes, but instead increments pointers, and 
the number of characters matched is the incremented pointer - where the match 
started.  Additionally, the primary match check was modified from a 64-bit 
comparison that when it fails, it computers the XOR of the two 64-bit values.  
Instead, the modified code performs the XOR first, and then checks if the XOR'd 
result is == 0.  If the XOR'd result is != 0, there is no need to (possibly 
re-read) and XOR the values.

A non-diffed version of FindMatchLength is below:


static inline int FindMatchLength(const char* s1,
                                  const char* s2,
                                  const char* s2_limit) {
  DCHECK_GE(s2_limit, s2);
  const char *ms1 = s1, *ms2 = s2;

  // Find out how long the match is. We loop over the data 64 bits at a
  // time until we find a 64-bit block that doesn't match; then we find
  // the first non-matching bit and use that to calculate the total
  // length of the match.
  while (PREDICT_TRUE(ms2 < s2_limit - 8l)) {
    uint64 x = UNALIGNED_LOAD64(ms1) ^ UNALIGNED_LOAD64(ms2);
    if (x == 0ul) {
      ms1 += 8ul;
      ms2 += 8ul;
    } else {
      // On current (mid-2008) Opteron models there is a 3% more
      // efficient code sequence to find the first non-matching byte.
      // However, what follows is ~10% better on Intel Core 2 and newer,
      // and we expect AMD's bsf instruction to improve.
      int matching_bits = Bits::FindLSBSetNonZero64(x);
      ms1 += matching_bits >> 3;
      return ms1 - s1;
    }
  }
  while (PREDICT_TRUE(ms2 < s2_limit)) {
    if (PREDICT_TRUE(*ms1 == *ms2)) {
      ++ms1;
      ++ms2;
    } else {
      return ms1 - s1;
    }
  }
  return ms1 - s1;
}

This library would likely be directly useful to a lot more people if a simple 
command line program to compress/decompress from stdin to stdout was included.

I want to test how snappy performs against other compression librarys. I saw 
that there is support fo this. Because there seems no windows support and it 
looks complicated to compile with all librarys on windows, I tested in a Linux 
Mint Debian VM.

After compiling snappy, the test terminates after a std::bad_alloc exception.
I recompiled without everything (even without google-test and gflags)
The result is the same (minus the gtest table)

I have the output of snappy_unittest and configure attached.

Shell variables starting with ac_ are in autoconf's namespace. Setting things 
like ac_have_builtin_ctz is therefore uncool. See 
http://www.gnu.org/s/hello/manual/autoconf/Macro-Names.html .

I refer to the two AC_TRY_COMPILE tests that set ac_have_builtin_ctz and 
ac_have_builtin_expect.

> To ensure that your macros don't conflict with present or future Autoconf 
macros, you should prefix your own macro names and any shell variables they use 
with some other sequence. Possibilities include your initials, or an 
abbreviation for the name of your organization or software package.

Since there's no really appropriate category int the manual's list of macro 
categories, I'd call them something like snappy_have_builtin_ctz and 
snappy_have_builtin_expect.

I see that caching these tests is a TODO item. When using AC_CACHE_CHECK, 
they'd rename to something like snappy_cv_have_builtin_ctz and 
snappy_cv_have_builtin_expect, respectively.

The dance with EXTRA_LIBSNAPPY_LDFLAGS seems odd. It'd be more conventional to 
do something like:

    SNAPPY_LTVERSION=1:0:0
    AC_SUBST([SNAPPY_LTVERSION])

and set this directly in libsnappy_la_LDFLAGS in Makefile.am. I'm not sure why 
it has been defined with m4_define.

If it's to allow the user to provide custom LDFLAGS, it's unnecessary: LDFLAGS 
is part of libsnappy_la_LINK. Here's the (generated) snippet from Makefile.in:

    libsnappy_la_LINK = $(LIBTOOL) --tag=CXX $(AM_LIBTOOLFLAGS) \
            $(LIBTOOLFLAGS) --mode=link $(CXXLD) $(AM_CXXFLAGS) \
            $(CXXFLAGS) $(libsnappy_la_LDFLAGS) $(LDFLAGS) -o $@

I'd love to see a JNI wrapper and/or a pure Java implementation of the codec.

Use AS_IF instead of directly using the shell's `if`: 
http://www.gnu.org/software/hello/manual/autoconf/Limitations-of-Builtins.html#L
imitations-of-Builtins and 
http://www.gnu.org/s/hello/manual/autoconf/Common-Shell-Constructs.html .

Using AS_IF ensures that any required macro arguments in the then and else 
branches are expanded in advance of the actual if statement. I don't think 
they're technically required here, but I believe their liberal use to be better 
autoconf style. This is a personal opinion thing, though.

Microsoft Visual Studio 2005, 2008 and MinGW from opensuse does not have file 
<sys/resource.h>.

And on MinGW does not building shared library (dll). Only static.

What steps will reproduce the problem?
1. Download code according http://code.google.com/p/snappy/source/checkout
2. Read README
3. Try to compile according information read from README

I would expect step by step instructions how to compile.

It should be something like

./autogen.sh && ./configure && make

It is just guess, it is exactly information which is mandatory in README!

Adding $(GTEST_CPPFLAGS) to both snappy_unittest_CPPFLAGS and 
snappy_unittest_CXXFLAGS is redundant. See this part of Makefile.in:

    snappy_unittest-snappy-test.o: snappy-test.cc
    @am__fastdepCXX_TRUE@   $(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(snappy_unittest_CPPFLAGS) $(CPPFLAGS) $(snappy_unittest_CXXFLAGS) $(CXXFLAGS) -MT snappy_unittest-snappy-test.o -MD -MP -MF $(DEPDIR)/snappy_unittest-snappy-test.Tpo -c -o snappy_unittest-snappy-test.o `test -f 'snappy-test.cc' || echo '$(srcdir)/'`snappy-test.cc
    @am__fastdepCXX_TRUE@   $(am__mv) $(DEPDIR)/snappy_unittest-snappy-test.Tpo $(DEPDIR)/snappy_unittest-snappy-test.Po
    @AMDEP_TRUE@@am__fastdepCXX_FALSE@      source='snappy-test.cc' object='snappy_unittest-snappy-test.o' libtool=no @AMDEPBACKSLASH@
    @AMDEP_TRUE@@am__fastdepCXX_FALSE@      DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@
    @am__fastdepCXX_FALSE@  $(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(snappy_unittest_CPPFLAGS) $(CPPFLAGS) $(snappy_unittest_CXXFLAGS) $(CXXFLAGS) -c -o snappy_unittest-snappy-test.o `test -f 'snappy-test.cc' || echo '$(srcdir)/'`snappy-test.ccest.cc

Note that the CPPFLAGS and the CXXFLAGS are both copied into the compiler 
command line.

From reading gtest.m4 (which defines GTEST_LIB_CHECK), it appears that the 
correct course of action is to set:

    snappy_unittest_CPPFLAGS = $(GTEST_CPPFLAGS)
    snappy_unittest_CXXFLAGS = $(gflags_CFLAGS) $(GTEST_CXXFLAGS)

because $(GTEST_CPPFLAGS) and $(GTEST_CXXFLAGS) are both defined.

Note also that because $(gflags_CFLAGS) comes from pkg-config, it could set 
flags that make sense for the compiler but not the preprocessor, so I believe 
that it belongs in the CXXFLAGS, too.

Currently Snappy can be build only for arch of the system where it is being 
built.

When building in 64bits, it should be possible to be able to do a 32bits build.

The following patch simplifies configure.ac and snappy-stubs-public.h.in

I'm no expert with the autotools, but I believe this change doesn't change any 
behaviour.

cheers

Consider adding -Wall (and possibly also -Werror) to either AUTOMAKE_OPTIONS in 
Makefile.am or as an argument to AM_INIT_AUTOMAKE.

If you don't mind using a modern automake (1.11 or later), you can also call 
AM_SILENT_RULES([yes]), but makes the generated makefiles require the 
"non-POSIX, but in practice rather widely supported Makefile construct of 
nested variable expansion ‘$(var1$(V))’" (cite: 
http://www.gnu.org/software/automake/manual/html_node/Options.html#Options ). 
Even MSYS has automake-1.11 these days.

If the optional gflags package is missing on the build system, "make check" 
fails because gtest isn't initialized properly. This can easily be fixed with 
the attached patch.

Hi, after revision 28 the build goes much better on Windows, thanks for those 
changes.

However, it's still not enough for me. I'm using Visual Studio 2010 and this is 
how I configure/compile, via Cygwin:

$ export CXX=cl
$ export CC=cl
$ ./configure
$ make

I got the error:

c:\cygwin\home\fdmanana\snappy\snappy-1.0.1\snappy-test.h(261) : error C2079: 
'snappy::CycleTimer::start_' uses undefined struct 'snappy::timeval'
c:\cygwin\home\fdmanana\snappy\snappy-1.0.1\snappy-test.h(244) : error C3861: 
'gettimeofday': identifier not found
c:\cygwin\home\fdmanana\snappy\snappy-1.0.1\snappy-test.h(248) : error C2079: 
'stop' uses undefined struct 'snappy::timeval'
c:\cygwin\home\fdmanana\snappy\snappy-1.0.1\snappy-test.h(249) : error C3861: 
'gettimeofday': identifier not found
c:\cygwin\home\fdmanana\snappy\snappy-1.0.1\snappy-test.h(251) : error C2228: 
left of '.tv_sec' must have class/struct/union
        type is 'int'
c:\cygwin\home\fdmanana\snappy\snappy-1.0.1\snappy-test.h(251) : error C2228: 
left of '.tv_sec' must have class/struct/union
        type is 'int'
c:\cygwin\home\fdmanana\snappy\snappy-1.0.1\snappy-test.h(252) : error C2228: 
left of '.tv_usec' must have class/struct/union
        type is 'int'
c:\cygwin\home\fdmanana\snappy\snappy-1.0.1\snappy-test.h(252) : error C2228: 
left of '.tv_usec' must have class/struct/union
        type is 'int'
make[1]: *** [snappy_unittest-snappy_unittest.obj] Error 2
make[1]: Leaving directory `/home/fdmanana/snappy/snappy-1.0.1'
make: *** [all] Error 2

The attached patch fixes this particular issue - sys/time.h doesn't exist on 
Windows neither does the function gettimeofday (required by snappy-test.h).

However, after applying this patch I then get the following error (I've 
included the full configure output and make output in case it helps):

fdmanana ~/snappy/snappy-1.0.1 > ./configure
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /usr/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking build system type... i686-pc-cygwin
checking host system type... i686-pc-cygwin
checking for style of include used by make... GNU
checking for gcc... cl
checking whether the C compiler works... yes
checking for C compiler default output file name... conftest.exe
checking for suffix of executables... .exe
checking whether we are cross compiling... no
checking for suffix of object files... obj
checking whether we are using the GNU C compiler... no
checking whether cl accepts -g... yes
checking for cl option to accept ISO C89... none needed
checking dependency style of cl... none
checking for a sed that does not truncate output... /usr/bin/sed
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for fgrep... /usr/bin/grep -F
checking for non-GNU ld... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 8192
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... file_magic ^x86 archive 
import|^x86 DLL
checking for ar... ar
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from cl object... ok
checking how to run the C preprocessor... cl -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... no
checking for inttypes.h... no
checking for stdint.h... yes
checking for unistd.h... no
checking for dlfcn.h... no
checking for objdir... .libs
checking for cl option to produce PIC... -DDLL_EXPORT -DPIC
checking if cl PIC flag -DDLL_EXPORT -DPIC works... yes
checking if cl static flag  works... yes
checking if cl supports -c -o file.obj... no
checking if cl supports -c -o file.obj... (cached) no
checking if we can lock with hard links... yes
checking whether the cl linker (/usr/bin/ld) supports shared libraries... yes
checking dynamic linker characteristics... Win32 ld.exe
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking whether we are using the GNU C++ compiler... no
checking whether cl accepts -g... yes
checking dependency style of cl... none
checking whether we are using the GNU C++ compiler... (cached) no
checking whether cl accepts -g... (cached) yes
checking dependency style of cl... (cached) none
checking how to run the C++ preprocessor... cl -E
checking whether the cl linker (/usr/bin/ld) supports shared libraries... yes
checking for cl option to produce PIC...  -DPIC
checking if cl PIC flag  -DPIC works... yes
checking if cl static flag  works... yes
checking if cl supports -c -o file.obj... no
checking if cl supports -c -o file.obj... (cached) no
checking if we can lock with hard links... yes
checking whether the cl linker (/usr/bin/ld) supports shared libraries... yes
checking dynamic linker characteristics... Win32 ld.exe
checking how to hardcode library paths into programs... immediate
checking whether byte ordering is bigendian... no
checking for stdint.h... (cached) yes
checking stddef.h usability... yes
checking stddef.h presence... yes
checking for stddef.h... yes
checking sys/mman.h usability... no
checking sys/mman.h presence... no
checking for sys/mman.h... no
checking sys/resource.h usability... no
checking sys/resource.h presence... no
checking for sys/resource.h... no
checking sys/time.h usability... no
checking sys/time.h presence... no
checking for sys/time.h... no
checking windows.h usability... yes
checking windows.h presence... yes
checking for windows.h... yes
checking for mmap... no
checking for 'gtest-config'... checking for gtest-config... no
no
checking for pkg-config... /usr/bin/pkg-config
checking pkg-config is at least version 0.9.0... yes
checking for gflags... no
checking if the compiler supports __builtin_expect... no
checking if the compiler supports __builtin_ctzll... no
checking for zlibVersion in -lz... no
checking for lzo1x_1_15_compress in -llzo2... no
checking for lzf_compress in -llzf... no
checking for fastlz_compress in -lfastlz... no
checking for qlz_compress in -lquicklz... no
configure: creating ./config.status
config.status: creating Makefile
config.status: creating snappy-stubs-public.h
config.status: creating config.h
config.status: executing depfiles commands
config.status: executing libtool commands
fdmanana ~/snappy/snappy-1.0.1 > make
make  all-am
make[1]: Entering directory `/home/fdmanana/snappy/snappy-1.0.1'
source='snappy.cc' object='snappy.lo' libtool=yes \
        DEPDIR=.deps depmode=none /bin/sh ./depcomp \
        /bin/sh ./libtool --tag=CXX   --mode=compile cl -DHAVE_CONFIG_H -I.     -g -c -o snappy.lo snappy.cc
libtool: compile:  cl -DHAVE_CONFIG_H -I. -g -c snappy.cc  -DPIC
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 16.00.30319.01 for 80x86
Copyright (C) Microsoft Corporation.  All rights reserved.

cl : Command line warning D9002 : ignoring unknown option '-g'
snappy.cc
C:\Program Files\Microsoft Visual Studio 10.0\VC\INCLUDE\xlocale(323) : warning 
C4530: C++ exception handler used, but unwind semantics are not enabled. 
Specify /EHsc
libtool: compile: mv -f "snappy.obj" ".libs/snappy.obj"
libtool: compile:  cl -DHAVE_CONFIG_H -I. -g -c snappy.cc >/dev/null 2>&1
source='snappy-sinksource.cc' object='snappy-sinksource.lo' libtool=yes \
        DEPDIR=.deps depmode=none /bin/sh ./depcomp \
        /bin/sh ./libtool --tag=CXX   --mode=compile cl -DHAVE_CONFIG_H -I.     -g -c -o snappy-sinksource.lo snappy-sinksource.cc
libtool: compile:  cl -DHAVE_CONFIG_H -I. -g -c snappy-sinksource.cc  -DPIC
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 16.00.30319.01 for 80x86
Copyright (C) Microsoft Corporation.  All rights reserved.

cl : Command line warning D9002 : ignoring unknown option '-g'
snappy-sinksource.cc
libtool: compile: mv -f "snappy-sinksource.obj" ".libs/snappy-sinksource.obj"
libtool: compile:  cl -DHAVE_CONFIG_H -I. -g -c snappy-sinksource.cc >/dev/null 
2>&1
source='snappy-stubs-internal.cc' object='snappy-stubs-internal.lo' libtool=yes 
\
        DEPDIR=.deps depmode=none /bin/sh ./depcomp \
        /bin/sh ./libtool --tag=CXX   --mode=compile cl -DHAVE_CONFIG_H -I.     -g -c -o snappy-stubs-internal.lo snappy-stubs-internal.cc
libtool: compile:  cl -DHAVE_CONFIG_H -I. -g -c snappy-stubs-internal.cc  -DPIC
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 16.00.30319.01 for 80x86
Copyright (C) Microsoft Corporation.  All rights reserved.

cl : Command line warning D9002 : ignoring unknown option '-g'
snappy-stubs-internal.cc
C:\Program Files\Microsoft Visual Studio 10.0\VC\INCLUDE\xlocale(323) : warning 
C4530: C++ exception handler used, but unwind semantics are not enabled. 
Specify /EHsc
libtool: compile: mv -f "snappy-stubs-internal.obj" 
".libs/snappy-stubs-internal.obj"
libtool: compile:  cl -DHAVE_CONFIG_H -I. -g -c snappy-stubs-internal.cc 
>/dev/null 2>&1
source='snappy-c.cc' object='snappy-c.lo' libtool=yes \
        DEPDIR=.deps depmode=none /bin/sh ./depcomp \
        /bin/sh ./libtool --tag=CXX   --mode=compile cl -DHAVE_CONFIG_H -I.     -g -c -o snappy-c.lo snappy-c.cc
libtool: compile:  cl -DHAVE_CONFIG_H -I. -g -c snappy-c.cc  -DPIC
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 16.00.30319.01 for 80x86
Copyright (C) Microsoft Corporation.  All rights reserved.

cl : Command line warning D9002 : ignoring unknown option '-g'
snappy-c.cc
C:\Program Files\Microsoft Visual Studio 10.0\VC\INCLUDE\xlocale(323) : warning 
C4530: C++ exception handler used, but unwind semantics are not enabled. 
Specify /EHsc
libtool: compile: mv -f "snappy-c.obj" ".libs/snappy-c.obj"
libtool: compile:  cl -DHAVE_CONFIG_H -I. -g -c snappy-c.cc >/dev/null 2>&1
/bin/sh ./libtool --tag=CXX   --mode=link cl  -g -version-info 1:1:0  -o 
libsnappy.la -rpath /usr/local/lib snappy.lo snappy-sinksource.lo 
snappy-stubs-internal.lo snappy-c.lo
libtool: link: warning: undefined symbols not allowed in i686-pc-cygwin shared 
libraries
libtool: link: lib -OUT:.libs/libsnappy.lib  snappy.obj snappy-sinksource.obj 
snappy-stubs-internal.obj snappy-c.obj
Microsoft (R) Library Manager Version 10.00.30319.01
Copyright (C) Microsoft Corporation.  All rights reserved.

libtool: link: ( cd ".libs" && rm -f "libsnappy.la" && ln -s "../libsnappy.la" 
"libsnappy.la" )
source='snappy_unittest.cc' object='snappy_unittest-snappy_unittest.obj' 
libtool=no \
        DEPDIR=.deps depmode=none /bin/sh ./depcomp \
        cl -DHAVE_CONFIG_H -I.      -g -c -o snappy_unittest-snappy_unittest.obj `if test -f 'snappy_unittest.cc'; then
cygpath -w 'snappy_unittest.cc'; else cygpath -w './snappy_unittest.cc'; fi`
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 16.00.30319.01 for 80x86
Copyright (C) Microsoft Corporation.  All rights reserved.

cl : Command line warning D9035 : option 'o' has been deprecated and will be 
removed in a future release
cl : Command line warning D9002 : ignoring unknown option '-g'
snappy_unittest.cc
C:\Program Files\Microsoft Visual Studio 10.0\VC\INCLUDE\xlocale(323) : warning 
C4530: C++ exception handler used, but unwind semantics are not enabled. 
Specify /EHsc
C:\Program Files\Microsoft SDKs\Windows\v7.0A\include\winnt.h(1140) : warning 
C4005: 'ARRAYSIZE' : macro redefinition
        c:\cygwin\home\fdmanana\snappy\snappy-1.0.1\snappy-stubs-internal.h(73) : see previous definition of 'ARRAYSIZE'
c:\cygwin\home\fdmanana\snappy\snappy-1.0.1\snappy_unittest.cc(165) : warning 
C4715: 'snappy::MinimumRequiredOutputSpace' : not all control paths return a 
value
source='snappy-test.cc' object='snappy_unittest-snappy-test.obj' libtool=no \
        DEPDIR=.deps depmode=none /bin/sh ./depcomp \
        cl -DHAVE_CONFIG_H -I.      -g -c -o snappy_unittest-snappy-test.obj `if test -f 'snappy-test.cc'; then cygpath
-w 'snappy-test.cc'; else cygpath -w './snappy-test.cc'; fi`
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 16.00.30319.01 for 80x86
Copyright (C) Microsoft Corporation.  All rights reserved.

cl : Command line warning D9035 : option 'o' has been deprecated and will be 
removed in a future release
cl : Command line warning D9002 : ignoring unknown option '-g'
snappy-test.cc
C:\Program Files\Microsoft Visual Studio 10.0\VC\INCLUDE\xlocale(323) : warning 
C4530: C++ exception handler used, but unwind semantics are not enabled. 
Specify /EHsc
C:\Program Files\Microsoft SDKs\Windows\v7.0A\include\winnt.h(1140) : warning 
C4005: 'ARRAYSIZE' : macro redefinition
        c:\cygwin\home\fdmanana\snappy\snappy-1.0.1\snappy-stubs-internal.h(73) : see previous definition of 'ARRAYSIZE'
/bin/sh ./libtool --tag=CXX   --mode=link cl  -g   -o snappy_unittest.exe 
snappy_unittest-snappy_unittest.obj snappy_unittest-snappy-test.obj libsnappy.la
libtool: link: cl -g -o .libs/snappy_unittest.exe 
snappy_unittest-snappy_unittest.obj snappy_unittest-snappy-test.obj  
./.libs/libsnappy.lib
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 16.00.30319.01 for 80x86
Copyright (C) Microsoft Corporation.  All rights reserved.

cl : Command line warning D9035 : option 'o' has been deprecated and will be 
removed in a future release
cl : Command line warning D9002 : ignoring unknown option '-g'
Microsoft (R) Incremental Linker Version 10.00.30319.01
Copyright (C) Microsoft Corporation.  All rights reserved.

/out:snappy_unittest-snappy_unittest.exe
/out:.libs/snappy_unittest.exe
snappy_unittest-snappy_unittest.obj
snappy_unittest-snappy-test.obj
./.libs/libsnappy.lib
LINK : fatal error LNK1181: cannot open input file 
'snappy_unittest-snappy_unittest.obj'
make[1]: *** [snappy_unittest.exe] Error 2
make[1]: Leaving directory `/home/fdmanana/snappy/snappy-1.0.1'
make: *** [all] Error 2
fdmanana ~/snappy/snappy-1.0.1 > ls snappy_unittest.
snappy_unittest.cc   snappy_unittest.obj
fdmanana ~/snappy/snappy-1.0.1 > make check
source='snappy_unittest.cc' object='snappy_unittest-snappy_unittest.obj' 
libtool=no \
        DEPDIR=.deps depmode=none /bin/sh ./depcomp \
        cl -DHAVE_CONFIG_H -I.      -g -c -o snappy_unittest-snappy_unittest.obj `if test -f 'snappy_unittest.cc'; then
cygpath -w 'snappy_unittest.cc'; else cygpath -w './snappy_unittest.cc'; fi`
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 16.00.30319.01 for 80x86
Copyright (C) Microsoft Corporation.  All rights reserved.

cl : Command line warning D9035 : option 'o' has been deprecated and will be 
removed in a future release
cl : Command line warning D9002 : ignoring unknown option '-g'
snappy_unittest.cc
C:\Program Files\Microsoft Visual Studio 10.0\VC\INCLUDE\xlocale(323) : warning 
C4530: C++ exception handler used, but unwind semantics are not enabled. 
Specify /EHsc
C:\Program Files\Microsoft SDKs\Windows\v7.0A\include\winnt.h(1140) : warning 
C4005: 'ARRAYSIZE' : macro redefinition
        c:\cygwin\home\fdmanana\snappy\snappy-1.0.1\snappy-stubs-internal.h(73) : see previous definition of 'ARRAYSIZE'
c:\cygwin\home\fdmanana\snappy\snappy-1.0.1\snappy_unittest.cc(165) : warning 
C4715: 'snappy::MinimumRequiredOutputSpace' : not all control paths return a 
value
source='snappy-test.cc' object='snappy_unittest-snappy-test.obj' libtool=no \
        DEPDIR=.deps depmode=none /bin/sh ./depcomp \
        cl -DHAVE_CONFIG_H -I.      -g -c -o snappy_unittest-snappy-test.obj `if test -f 'snappy-test.cc'; then cygpath
-w 'snappy-test.cc'; else cygpath -w './snappy-test.cc'; fi`
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 16.00.30319.01 for 80x86
Copyright (C) Microsoft Corporation.  All rights reserved.

cl : Command line warning D9035 : option 'o' has been deprecated and will be 
removed in a future release
cl : Command line warning D9002 : ignoring unknown option '-g'
snappy-test.cc
C:\Program Files\Microsoft Visual Studio 10.0\VC\INCLUDE\xlocale(323) : warning 
C4530: C++ exception handler used, but unwind semantics are not enabled. 
Specify /EHsc
C:\Program Files\Microsoft SDKs\Windows\v7.0A\include\winnt.h(1140) : warning 
C4005: 'ARRAYSIZE' : macro redefinition
        c:\cygwin\home\fdmanana\snappy\snappy-1.0.1\snappy-stubs-internal.h(73) : see previous definition of 'ARRAYSIZE'
/bin/sh ./libtool --tag=CXX   --mode=link cl  -g   -o snappy_unittest.exe 
snappy_unittest-snappy_unittest.obj snappy_unittest-snappy-test.obj libsnappy.la
libtool: link: cl -g -o .libs/snappy_unittest.exe 
snappy_unittest-snappy_unittest.obj snappy_unittest-snappy-test.obj  
./.libs/libsnappy.lib
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 16.00.30319.01 for 80x86
Copyright (C) Microsoft Corporation.  All rights reserved.

cl : Command line warning D9035 : option 'o' has been deprecated and will be 
removed in a future release
cl : Command line warning D9002 : ignoring unknown option '-g'
Microsoft (R) Incremental Linker Version 10.00.30319.01
Copyright (C) Microsoft Corporation.  All rights reserved.

/out:snappy_unittest-snappy_unittest.exe
/out:.libs/snappy_unittest.exe
snappy_unittest-snappy_unittest.obj
snappy_unittest-snappy-test.obj
./.libs/libsnappy.lib
LINK : fatal error LNK1181: cannot open input file 
'snappy_unittest-snappy_unittest.obj'
make: *** [snappy_unittest.exe] Error 2


Do you any idea of why the file is missing?
Is there anything I can do to help fix this?

Thank you very much

Makefile.am contains a rule to rebuild libtool if $(LIBTOOL_DEPS) is out of 
date, but LIBTOOL_DEPS is never AC_SUBST'd:

See http://www.gnu.org/software/libtool/manual/html_node/LT_005fINIT.html

Automagic dependencies (where the program automatically enables/disables 
features based on the result of configure tests) aren't cool: 
http://www.gentoo.org/proj/en/qa/automagic.xml

It's quite alright to default to automagic dependencies, but if the user 
specifies --with-gflags or --without-gflags, the correct behaviour is to abort 
configure if the flags library wasn't found (--with-gflags) or to never use it, 
even if it exists (--without-gflags).

This is particularly important on source-based distributions like gentoo, where 
optional dependencies are much more common.

12

Tried to compile on PPC Mac OS X 10.4.11 and got 'Symbol not found: 
_UNALIGNED_LOAD32'

It looks like the google-ctemplate project has already solved this portability 
issue:
http://code.google.com/p/google-ctemplate/source/browse/trunk/src/template_strin
g.cc#85

What steps will reproduce the problem?
1. run ./configure && make on a host with the SunPro C++ compiler
2.
3.

What is the expected output? What do you see instead?

I'd like to see it compile, but it fails for several reasons (can't find 
byteswap.h or bswap_(16,32,64) functions, a few syntax errors, and the linkers 
inability to match function signatures when the constness of arguments vary 
from the declaration.

What version of the product are you using? On what operating system?

snappy-1.0.3 on SunOS 5.10 with Sun C++ 5.10

Please provide any additional information below.

I'm attaching a patch.

Snappy does not come with a description of the bitstream, neither in English 
nor in C. The decompression code is obfuscated by optimizations. It is useful 
when working with a file format to have a clear description of its workings, 
and (unoptimized) C is usually more unambiguous than English.

While working on the Linux Kernel port, I wrote both:
https://github.com/zeevt/csnappy/blob/master/csnappy_simple.c

Please include something like that (or better!) with the official distribution. 
If you want copyright assignment I can do that too.

asd

the previous code read like this "if this is not defined, undefine it"

autogen.sh should not reset PATH unconditionally; this causes problems for 
people who pull in their autotools from other sources. (This is currently due 
to internal issues in our internal release pipeline.)