kesinger / miniz Goto Github PK

Summary: The function mz_zip_extract_archive_file_to_heap() can fail to locate 
an archive member, because it uses binary search in the central directory, even 
if that central directory was never sorted because the   
MZ_ZIP_FLAG_DO_NOT_SORT_CENTRAL_DIRECTORY flag was specified when initializing 
the archive reader.

The problem is caused by the following two lines in mz_zip_reader_locate_file():

 if (((flags & (MZ_ZIP_FLAG_IGNORE_PATH | MZ_ZIP_FLAG_CASE_SENSITIVE) == 0) && (!pComment) && (pZip->m_pState->m_sorted_central_dir_offsets.m_p))
    return mz_zip_reader_locate_file_binary_search(pZip, pName);

The check whether pZip->m_pState->m_sorted_central_dir_offsets.m_p is 
initialized is probably meant to determine whether the central directory was 
sorted.  However, the pZip->m_pState->m_sorted_central_dir_offsets array is 
*always* initialized, irrespective of whether the central directory was sorted 
or not (see the definition of  mz_zip_reader_read_central_dir()).

Hence, a different way of checking whether the central directory has been 
sorted should be implemented.

What steps will reproduce the problem?
1. Create a ZIP file with entries in this order: C, B, A.
2. Open the ZIP file with MZ_ZIP_FLAG_DO_NOT_SORT_CENTRAL_DIRECTORY.
3. Attempt to locate entry C (no flags).

What is the expected output? What do you see instead?
I expect to find entry C at index 0. Instead, it is not found.

What version of the product are you using? On what operating system?
miniz 1.14 on Windows 7 (x64).

Please provide any additional information below.
The mz_zip_reader_locate_file() function does not seem to detect that 
MZ_ZIP_FLAG_DO_NOT_SORT_CENTRAL_DIRECTORY was used, and assumes the central 
directory is sorted. This causes the subsequent binary search for C to fail.

There seems to be a check for whether m_sorted_central_dir_offsets.m_p, but 
m_sorted_central_dir_offsets is apparently always initialized, regardless of 
whether or not the central directory was actually sorted.

What steps will reproduce the problem?
1. Use tdm-mingw32 to compile miniz on win32

What is the expected output? What do you see instead?

Excepting quiet compilation,
got some undefined references instead.

What version of the product are you using? On what operating system?
1.11 / win32

Please provide any additional information below.

My solution:
add 
#define __forceinline inline
#include <time.h> 
at the beginning of the source.

and
mingw32 doesn't have _fseeki64/_ftelli64,
therefore I changed the corresponding macro to ftello64/fseeko64 as 
alternatives. They seems work the same way. I hope :)


      #if defined(__MINGW32__)

         #define MZ_FTELL64 ftello64

         #define MZ_FSEEK64 fseeko64

      #else

         #define MZ_FTELL64 _ftelli64

         #define MZ_FSEEK64 _fseeki64

      #endif

Defining MINIZ_NO_TIME generates a compiler error (unknown type) because 
mz_zip_set_file_times uses the type time_t without time.h being included.

miniz is designed as a single file to be included by another source file via 
#include "miniz.c" as stated in the example applications. This works fine when 
miniz is used within one project. When two projects using miniz unexpected 
results arise.

Situation:

Shared library A includes miniz.c. Application B also includes miniz.c. 
Application A will load B.


Problem:

When B loads library A there are two copies of the symbols related to miniz. 
Depending on the OS, B might fail to load A. Also, depending on the OS the 
miniz that is part of A might be used by B or vice versa. This can be a major 
issue if A and B have different versions (especially if they are not ABI or API 
compatible) of miniz included.


Solutions:

1) Make all parts within miniz.c static. This will make everything in miniz.c 
only visible to what is including it. This will prevent the miniz symbols from 
being exposed publicly.

This could be facilitated by setting a define similar to MINIZ_NO_ARCHIVE_APIS. 
Something like:

#ifdef MINIZ_STATIC
# define MINIZ_STATIC static
#else
# define MINIZ_STATIC
#endif

Then any any function that would be considered public would be prefixed by 
MINIZ_STATIC:

MINIZ_STATIC int mz_uncompress(...

2) Break miniz.c appart into miniz.h and miniz.c to facilitate building and 
installing miniz as a shared library.

What steps will reproduce the problem?
1. Create a ZIP with an entry with a larger compressed size than uncompressed 
size. (Not sure if this is a necessary condition.)
2. Call mz_reader_extract_to_mem_noalloc with a non-NULL user buffer.

What is the expected output? What do you see instead?
I expect the extracted output, but the procedure fails instead.

What version of the product are you using? On what operating system?
1.14 on Visual Studio 2010 (32-bit). Also replicated with GCC on Cygwin.

Please provide any additional information below.

The problem seems to be on line 3539:

  else if (pUser_read_buf)
  {
    // Use a user provided read buffer.
    if (!user_read_buf_size)
      return MZ_FALSE;
    pRead_buf = (mz_uint8 *)pUser_read_buf;
    read_buf_size = user_read_buf_size;
    read_buf_avail = 0;
    comp_remaining = file_stat.m_uncomp_size;
  }

Judging from the surrounding code, comp_remaining = file_stat.m_comp_size, not 
m_uncomp_size. This would explain why cases with a NULL pUser_read_buf works 
fine.

What steps will reproduce the problem?
1. port the example buftest.c and xzminidec.c from the xz-embedded library to 
miniz with a 1 byte input and output buffer

What is the expected output? What do you see instead?
I see duplicated bytes in the output

What version of the product are you using? On what operating system?
v111b

Please provide any additional information below.
I think porting the test/examples/demos from the xz-embedded or the xz library 
proper and/or other decompression libraries and including them into miniz would 
be a good idea.

It seems that this code:

#if !defined(MINIZ_NO_TIME) && !defined(MINIZ_NO_ARCHIVE_APIS)
#include <time.h>
#endif

Occurs in "miniz.c" *before* the big block of commented-out #defines, meaning 
that the include happens regardless of how you've chosen to configure miniz for 
your project. This looks like a bug to me. I'm very new as a miniz user though, 
so I could be mistaken.

I noticed this in version v114_r1.7.

What steps will reproduce the problem?
1. #define MINIZ_NO_STDIO

What is the expected output? What do you see instead?
'mz_zip_reader_extract_to_file': identifier not found

Please provide any additional information below.
Problem appears to be that mz_zip_reader_extract_file_to_file should not be 
defined in MINIZ_NO_STDIO builds

Solution appears to be to wrap mz_zip_reader_extract_file_to_file in #ifndef 
MINIZ_NO_STDIO
#endif

Problem:
========
Compilation of miniz.c using 

    g++ -std=c++11 file.cc 

results in errors of the sort:

    miniz.c:3393:27: warning: always_inline function might not be inlinable [-Wattributes]



Solution:
=========

<   #define MZ_FORCEINLINE inline __attribute__((__always_inline__))

---
>   #define MZ_FORCEINLINE __attribute__((__always_inline__))

See 
http://stackoverflow.com/questions/4434140/narrowing-conversions-in-c0x-is-it-ju
st-me-or-does-this-sound-like-a-breakin

It seems to me this might actually be a compiler bug because clang thinks 
"\0\0\04\02\06"[num_chans] isn't a constant expression. However, this fixes the 
problem:

@@ -2765,6 +2765,7 @@ mz_uint tdefl_create_comp_flags_from_zip_params(int 
level, int window_bits, int
 // http://altdevblogaday.org/2011/04/06/a-smaller-jpg-encoder/.
 void *tdefl_write_image_to_png_file_in_memory(const void *pImage, int w, int h, int num_chans, size_t *pLen_out)
 {
+  static const mz_uint8 chans[] = {0x00, 0x00, 0x04, 0x02, 0x06};
   tdefl_compressor *pComp = (tdefl_compressor *)MZ_MALLOC(sizeof(tdefl_compressor)); tdefl_output_buffer out_buf; int i, bpl = w * num_chans, y, z; mz_uint32 c; *pLen_out = 0;
   if (!pComp) return NULL;
   MZ_CLEAR_OBJ(out_buf); out_buf.m_expandable = MZ_TRUE; out_buf.m_capacity = 57+MZ_MAX(64, (1+bpl)*h); if (NULL == (out_buf.m_pBuf = (mz_uint8*)MZ_MALLOC(out_buf.m_capacity))) { MZ_FREE(pComp); return NULL; }
@@ -2778,7 +2779,7 @@ void *tdefl_write_image_to_png_file_in_memory(const void 
*pImage, int w, int h,
   *pLen_out = out_buf.m_size-41;
   {
     mz_uint8 pnghdr[41]={0x89,0x50,0x4e,0x47,0x0d,0x0a,0x1a,0x0a,0x00,0x00,0x00,0x0d,0x49,0x48,0x44,0x52,
-      
0,0,(mz_uint8)(w>>8),(mz_uint8)w,0,0,(mz_uint8)(h>>8),(mz_uint8)h,8,"\0\0\04\02\
06"[num_chans],0,0,0,0,0,0,0,
+      
0,0,(mz_uint8)(w>>8),(mz_uint8)w,0,0,(mz_uint8)(h>>8),(mz_uint8)h,8,chans[num_ch
ans],0,0,0,0,0,0,0,
       (mz_uint8)(*pLen_out>>24),(mz_uint8)(*pLen_out>>16),(mz_uint8)(*pLen_out>>8),(mz_uint8)*pLen_out,0x49,0x44,0x41,0x54};
     c=(mz_uint32)mz_crc32(MZ_CRC32_INIT,pnghdr+12,17); for (i=0; i<4; ++i, c<<=8) ((mz_uint8*)(pnghdr+29))[i]=(mz_uint8)(c>>24);
     memcpy(out_buf.m_pBuf, pnghdr, 41);

Compiling with clang++ and the `-fsanitize=unsigned-integer-overflow` flag 
(http://clang.llvm.org/docs/UsersManual.html) exposes a few unsigned overflow 
issues:

../src/miniz.c:1929:7: runtime error: unsigned integer overflow: 0 - 1 cannot 
be represented in type 'mz_uint' (aka 'unsigned int')
../src/miniz.c:964:35: runtime error: unsigned integer overflow: 0 - 1 cannot 
be represented in type 'size_t' (aka 'unsigned long')


Originally described at https://github.com/developmentseed/node-blend/issues/27

using a 32bit unsigned int instead of a 64bit unsigned long fixes the problem.

-mz_ulong mz_crc32(mz_ulong crc, const unsigned char *ptr, size_t buf_len);
+mz_uint32 mz_crc32(mz_uint32 crc, const unsigned char *ptr, size_t buf_len);

also the typedef for mz_uint32 must be moved to preceed this.

What steps will reproduce the problem?
1. Complie attached test_crash.c with latest miniz.
2. Run it against attached test.zip.

What is the expected output? What do you see instead?
Expected is MANIFEST.MF contents since test.zip is a valid archive.
However it fails to decompress file contents.

What version of the product are you using? On what operating system?
Miniz 1.11b, Win7 and GNU/Linux (gcc 4.1.2)

Please provide any additional information below.

I've discovered that failure occurs in tinfl_decompress() routine:

...
        TINFL_HUFF_DECODE(26, dist, &r->m_tables[1]);
        num_extra = s_dist_extra[dist]; dist = s_dist_base[dist];
        if (num_extra) { mz_uint extra_bits; TINFL_GET_BITS(27, extra_bits, num_extra); dist += extra_bits; }

        dist_from_out_buf_start = pOut_buf_cur - pOut_buf_start;
        if ((dist > dist_from_out_buf_start) && (decomp_flags & TINFL_FLAG_USING_NON_WRAPPING_OUTPUT_BUF))
        {
          TINFL_CR_RETURN_FOREVER(37, TINFL_STATUS_FAILED);
        }
...


dist occures to be 2049 which is much greater than dist_from_out_buf_start thus 
decompression fails.

What steps will reproduce the problem?
1.using example3 to compress the *.txt to *.zip
2.and then decompress *.zip by using winzip/winrar/7zip
3.fail

What is the expected output? What do you see instead?

can decompress by other software
What version of the product are you using? On what operating system?
 miniz.c v1.14 , windows xp

Please provide any additional information below.