Code Monkey home page Code Monkey logo

msgpack-c's Introduction

MessagePack

MessagePack is an efficient binary serialization format. It's like JSON. but fast and small.

This repository manages specification of MessagePack format. See Spec for the specification.

Implementation projects have each own repository. See msgpack.org website to find implementations and their documents.

If you'd like to show your msgpack implementation to the msgpack.org website, please follow the website document.

msgpack-c's People

Contributors

advect avatar al11090 avatar cableramki avatar derwolfe avatar dgryski avatar drrlvn avatar frsyuki avatar gfx avatar haneefmubarak avatar herbrechtsmeier avatar jamessan avatar jpetso avatar kazuki avatar kzk avatar llchan avatar makamaka avatar methane avatar moriyoshi avatar muga avatar n1tehawk avatar nobu-k avatar redboltz avatar repeatedly avatar tanakh avatar tarruda avatar tbeu avatar tokuhirom avatar whoshuu avatar xerial avatar ygj6 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

msgpack-c's Issues

AddressSanitizer: alloc-dealloc-mismatch (malloc vs operator delete)

#include <msgpack.hpp>
#include <iostream>
#include <string>

int main(void) {
        // serializes multiple objects using msgpack::packer.
        msgpack::sbuffer buffer;

        msgpack::packer<msgpack::sbuffer> pk(&buffer);
        pk.pack(std::string("Log message ... 1"));
        pk.pack(std::string("Log message ... 2"));
        pk.pack(std::string("Log message ... 3"));

        // deserializes these objects using msgpack::unpacker.
        msgpack::unpacker pac;

        // feeds the buffer.
        pac.reserve_buffer(buffer.size());
        memcpy(pac.buffer(), buffer.data(), buffer.size());
        pac.buffer_consumed(buffer.size());

        // now starts streaming deserialization.
        msgpack::unpacked result;
        while(pac.next(&result)) {
            std::cout << result.get() << std::endl;
        }

        // results:
        // $ g++ stream.cc -lmsgpack -o stream
        // $ ./stream
        // "Log message ... 1"
        // "Log message ... 2"
        // "Log message ... 3"
}

Running this sample program under ASan produces the following:

"Log message ... 1"
=================================================================
==1007==ERROR: AddressSanitizer: alloc-dealloc-mismatch (malloc vs operator delete) on 0x625000007900
    #0 0x423114 in operator delete(void*) ??:?
    #1 0x43818e in std::auto_ptr<msgpack::zone>::reset(msgpack::zone*) /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.2/include/g++-v4/backward/auto_ptr.h:244
    #2 0x4340ac in msgpack::unpacker::next(msgpack::unpacked*) /usr/include/msgpack/unpack.hpp:234
    #3 0x4313e4 in main /tmp/msgpack-wat/code3.cpp:24
    #4 0x7f36399afbf4 in ?? ??:0
0x625000007900 is located 0 bytes inside of 8248-byte region [0x625000007900,0x625000009938)
allocated by thread T0 here:
    #0 0x422b74 in malloc ??:?
    #1 0x43cdd1 in msgpack_zone_new /home/xyz/msgpack-c/src/zone.c:198
==1007==HINT: if you don't care about these warnings you may set ASAN_OPTIONS=alloc_dealloc_mismatch=0
==1007==ABORTING

How to detect invalid data when using the streaming feature?

I'm currently using the msgpack_unpack_next to deserialize data coming from the network. From what I understand, this function will return true when it successfully parses an object from the data currently available to the unpacker.

But what happens if non-valid msgpack data comes from the network? In other words, how can I tell if msgpack_unpack_next failed because it still needs more data or because it received invalid msgpack data?

Segmentation fault

I have a corrupted msgpacked file: http://yadi.sk/d/mbAiGDy2GqRtU . Msgpack-c 0.5.7 (from Ubuntu 12.10) and ffd0525 fails to segmentation fault on it with following trace:

#0  msgpack_zone_malloc_expand (zone=0x608130, size=<optimized out>) at zone.c:90
#1  0x00007ffff7bd55e0 in msgpack_zone_malloc_no_align (size=<optimized out>, zone=<optimized out>) at msgpack/zone.h:96
#2  msgpack_zone_malloc (size=<optimized out>, zone=<optimized out>) at msgpack/zone.h:108
#3  template_callback_array (u=0x7fffffffcfc0, o=0x7fffffffd1a0, n=<optimized out>) at unpack.c:108
#4  template_execute (off=<synthetic pointer>, len=39730849, data=0x7ffff177c028 "\224", <incomplete sequence \335>, ctx=0x7fffffffcfc0) at msgpack/unpack_template.h:328
#5  msgpack_unpack (data=0x7ffff177c028 "\224", <incomplete sequence \335>, len=39730849, off=0x0, result_zone=<optimized out>, result=0x7fffffffd880) at unpack.c:424
#6  0x0000000000401353 in msgpack::unpack (result=0x7fffffffd910, data=0x7ffff177c028 "\224", <incomplete sequence \335>, len=39730849, offset=0x0) at /usr/include/msgpack/unpack.hpp:314
#7  0x000000000040164a in main (argc=1, argv=0x7fffffffda28) at ../untitled/main.cpp:22

At the same time msgpack-python allocates over 13 GB of memory and throws the exception:

Traceback (most recent call last):
  File "test.py", line 5, in <module>
    data = msgpack.unpack(f, use_list=0)
  File "_msgpack.pyx", line 246, in msgpack._msgpack.unpack (msgpack/_msgpack.c:3123)
  File "_msgpack.pyx", line 234, in msgpack._msgpack.unpackb (msgpack/_msgpack.c:2861)
TypeError: unhashable type: 'dict'

Issue Configuring and building for Linux running on Cold Fire Board / Atomic Opertions not supported ?

dev@dev-desktop:/Downloads/msgpack-0.5.4$ ./configure --host=m68k-linux CC=m68k-linux-gnu-gcc CXX=m68k-linux-gnu-g++ CFLAGS=-march=m68k
configure: WARNING: If you wanted to set the --build type, don't use --host.
If a cross compiler is detected then cross compile mode will be used.
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for m68k-linux-strip... no
checking for strip... strip
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking for m68k-linux-gcc... m68k-linux-gnu-gcc
checking whether the C compiler works... no
configure: error: in /home/dev/Downloads/msgpack-0.5.4': configure: error: C compiler cannot create executables Seeconfig.log' for more details.
dev@dev-desktop:
/Downloads/msgpack-0.5.4$ ./configure --host=m68k-linux CC=m68k-linux-gnu-gcc CXX=m68k-linux-gnu-g++
configure: WARNING: If you wanted to set the --build type, don't use --host.
If a cross compiler is detected then cross compile mode will be used.
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for m68k-linux-strip... no
checking for strip... strip
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking for m68k-linux-gcc... m68k-linux-gnu-gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... yes
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether m68k-linux-gnu-gcc accepts -g... yes
checking for m68k-linux-gnu-gcc option to accept ISO C89... none needed
checking for style of include used by make... GNU
checking dependency style of m68k-linux-gnu-gcc... gcc3
checking if C++ API is enabled...
checking whether we are using the GNU C++ compiler... yes
checking whether m68k-linux-gnu-g++ accepts -g... yes
checking dependency style of m68k-linux-gnu-g++... gcc3
checking whether m68k-linux-gnu-gcc and cc understand -c and -o together... yes
checking build system type... i686-pc-linux-gnu
checking host system type... m68k-unknown-linux-gnu
checking for a sed that does not truncate output... /bin/sed
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for fgrep... /bin/grep -F
checking for ld used by m68k-linux-gnu-gcc... /opt/freescale/usr/local/gcc-4.4.217-eglibc-2.11.217/m68k-linux/m68k-linux-gnu/bin/ld
checking if the linker (/opt/freescale/usr/local/gcc-4.4.217-eglibc-2.11.217/m68k-linux/m68k-linux-gnu/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... no
checking for m68k-linux-dumpbin... no
checking for m68k-linux-link... no
checking for dumpbin... no
checking for link... link -dump -symbols
configure: WARNING: using cross tools not prefixed with host triplet
checking the name lister (link -dump -symbols) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
checking for /opt/freescale/usr/local/gcc-4.4.217-eglibc-2.11.217/m68k-linux/m68k-linux-gnu/bin/ld option to reload object files... -r
checking for m68k-linux-objdump... no
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for m68k-linux-ar... no
checking for ar... ar
checking for m68k-linux-strip... strip
checking for m68k-linux-ranlib... no
checking for ranlib... ranlib
checking command to parse link -dump -symbols output from m68k-linux-gnu-gcc object... failed
checking how to run the C preprocessor... m68k-linux-gnu-gcc -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking whether we are using the GNU C++ compiler... (cached) yes
checking whether m68k-linux-gnu-g++ accepts -g... (cached) yes
checking dependency style of m68k-linux-gnu-g++... (cached) gcc3
checking how to run the C++ preprocessor... m68k-linux-gnu-g++ -E
checking for objdir... .libs
checking if m68k-linux-gnu-gcc supports -fno-rtti -fno-exceptions... no
checking for m68k-linux-gnu-gcc option to produce PIC... -fPIC -DPIC
checking if m68k-linux-gnu-gcc PIC flag -fPIC -DPIC works... yes
checking if m68k-linux-gnu-gcc static flag -static works... yes
checking if m68k-linux-gnu-gcc supports -c -o file.o... yes
checking if m68k-linux-gnu-gcc supports -c -o file.o... (cached) yes
checking whether the m68k-linux-gnu-gcc linker (/opt/freescale/usr/local/gcc-4.4.217-eglibc-2.11.217/m68k-linux/m68k-linux-gnu/bin/ld) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking for ld used by m68k-linux-gnu-g++... /opt/freescale/usr/local/gcc-4.4.217-eglibc-2.11.217/m68k-linux/m68k-linux-gnu/bin/ld
checking if the linker (/opt/freescale/usr/local/gcc-4.4.217-eglibc-2.11.217/m68k-linux/m68k-linux-gnu/bin/ld) is GNU ld... yes
checking whether the m68k-linux-gnu-g++ linker (/opt/freescale/usr/local/gcc-4.4.217-eglibc-2.11.217/m68k-linux/m68k-linux-gnu/bin/ld) supports shared libraries... yes
checking for m68k-linux-gnu-g++ option to produce PIC... -fPIC -DPIC
checking if m68k-linux-gnu-g++ PIC flag -fPIC -DPIC works... yes
checking if m68k-linux-gnu-g++ static flag -static works... yes
checking if m68k-linux-gnu-g++ supports -c -o file.o... yes
checking if m68k-linux-gnu-g++ supports -c -o file.o... (cached) yes
checking whether the m68k-linux-gnu-g++ linker (/opt/freescale/usr/local/gcc-4.4.217-eglibc-2.11.217/m68k-linux/m68k-linux-gnu/bin/ld) supports shared libraries... yes
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking dependency style of m68k-linux-gnu-gcc... gcc3
checking if debug option is enabled...
checking for _sync* atomic operations...
configure: error: _sync* atomic operations are not supported.

Note that gcc < 4.1 is not supported.

If you are using gcc >= 4.1 and the default target CPU architecture is "i386", try to
add CFLAGS="--march=i686" and CXXFLAGS="-march=i686" options to ./configure as follows:

$ ./configure CFLAGS="-march=i686" CXXFLAGS="-march=i686"

dev@dev-desktop:/Downloads/msgpack-0.5.4$ m68k-linux-gnu-gcc -v
Using built-in specs.
Target: m68k-linux-gnu
Configured with: /scratch/maxim/cf-2010q1-lite/src-linux/gcc-4.4-2010q1/configure --build=i686-pc-linux-gnu --host=i686-pc-linux-gnu --target=m68k-linux-gnu --enable-threads --disable-libmudflap --disable-libssp --disable-libstdcxx-pch --with-arch=cf --with-gnu-as --with-gnu-ld --with-specs='%{O2:%{!fno-remove-local-statics: -fremove-local-statics}} %{O*:%{O|O0|O1|O2|Os:;:%{!fno-remove-local-statics: -fremove-local-statics}}}' --enable-languages=c,c++ --enable-shared --disable-lto --enable-symvers=gnu --enable-__cxa_atexit --with-pkgversion='Sourcery G++ Lite 4.4-217' --with-bugurl=https://support.codesourcery.com/GNUToolchain/ --disable-nls --prefix=/opt/freescale/usr/local/gcc-4.4.217-eglibc-2.11.217/m68k-linux --with-sysroot=/opt/freescale/usr/local/gcc-4.4.217-eglibc-2.11.217/m68k-linux/m68k-linux-gnu/libc --with-build-sysroot=/scratch/maxim/cf-2010q1-lite/install-linux/m68k-linux-gnu/libc --with-gmp=/scratch/maxim/cf-2010q1-lite/obj-linux/host-libs-4.4-217-m68k-linux-gnu-i686-pc-linux-gnu/usr --with-mpfr=/scratch/maxim/cf-2010q1-lite/obj-linux/host-libs-4.4-217-m68k-linux-gnu-i686-pc-linux-gnu/usr --with-ppl=/scratch/maxim/cf-2010q1-lite/obj-linux/host-libs-4.4-217-m68k-linux-gnu-i686-pc-linux-gnu/usr --with-host-libstdcxx='-static-libgcc -Wl,-Bstatic,-lstdc++,-Bdynamic -lm' --with-cloog=/scratch/maxim/cf-2010q1-lite/obj-linux/host-libs-4.4-217-m68k-linux-gnu-i686-pc-linux-gnu/usr --enable-libgomp --enable-poison-system-directories --with-build-time-tools=/scratch/maxim/cf-2010q1-lite/install-linux/m68k-linux-gnu/bin --with-build-time-tools=/scratch/maxim/cf-2010q1-lite/install-linux/m68k-linux-gnu/bin
Thread model: posix
gcc version 4.4.1 (Sourcery G++ Lite 4.4-217)
dev@dev-desktop:
/Downloads/msgpack-0.5.4$
The above tool chain uses the https://sourcery.mentor.com/GNUToolchain/subscription3129?lite=ColdFire

Serializing Class Hierarchy

What is the recommended way to serialize the state of a derived class along with all the states of it's parents? Looks like MSGPACK_DEFINE doesn't directly support invoking the base class. Shall a macro with name like MSGPACK_DEFINE_BASE allowing to specify a list of base classes that has to be serialized too?

Troubles decoding complex structure

The following structure fails to unpack. Calls to packing routines are padded for convenience:

int main(int argc, char ** argv)
{
  msgpack_sbuffer mp_buf;
    msgpack_sbuffer_init(&mp_buf);
    msgpack_packer *    pk = msgpack_packer_new(&mp_buf, msgpack_sbuffer_write);

    msgpack_pack_array(pk, 1);
     msgpack_pack_array(pk, 2);
      msgpack_pack_array(pk, 2);
       msgpack_pack_uint32(pk, 1358252950);
       msgpack_pack_uint32(pk, 436731042);
      msgpack_pack_array(pk, 5);
       msgpack_pack_uint32(pk, 8);
       msgpack_pack_uint32(pk, 33664);
       msgpack_pack_uint32(pk, 804);
       msgpack_pack_array(pk, 12);
        msgpack_pack_uint8(pk, 0);
        msgpack_pack_raw(pk, 0);
        msgpack_pack_raw_body(pk, "", 0);
        msgpack_pack_raw(pk, 0);
        msgpack_pack_raw_body(pk, "", 0);
        msgpack_pack_raw(pk, 0);
        msgpack_pack_raw_body(pk, "", 0);
        msgpack_pack_raw(pk, 0);
        msgpack_pack_raw_body(pk, "", 0);
        msgpack_pack_raw(pk, 0);
        msgpack_pack_raw_body(pk, "", 0);
        msgpack_pack_raw(pk, 0);
        msgpack_pack_raw_body(pk, "", 0);
        msgpack_pack_raw(pk, 0);
        msgpack_pack_raw_body(pk, "", 0);
        msgpack_pack_raw(pk, 0);
        msgpack_pack_raw_body(pk, "", 0);
        msgpack_pack_raw(pk, 0);
        msgpack_pack_raw_body(pk, "", 0);
        msgpack_pack_raw(pk, 0);
        msgpack_pack_raw_body(pk, "", 0);
        msgpack_pack_raw(pk, 0);
        msgpack_pack_raw_body(pk, "", 0);
       msgpack_pack_array(pk, 3);
        msgpack_pack_uint8(pk, 1);
        msgpack_pack_array(pk, 10);
         msgpack_pack_uint32(pk, 0);
         msgpack_pack_uint32(pk, 804);
         msgpack_pack_uint32(pk, 806);
         msgpack_pack_uint32(pk, 33664);
         msgpack_pack_raw(pk, 16);
         msgpack_pack_raw_body(pk, "multipart/report", 16);
         msgpack_pack_raw(pk, 0);
         msgpack_pack_raw_body(pk, "", 0);
         msgpack_pack_raw(pk, 0);
         msgpack_pack_raw_body(pk, "", 0);
         msgpack_pack_raw(pk, 0);
         msgpack_pack_raw_body(pk, "", 0);
         msgpack_pack_raw(pk, 0);
         msgpack_pack_raw_body(pk, "", 0);
         msgpack_pack_raw(pk, 0);
         msgpack_pack_raw_body(pk, "", 0);
        msgpack_pack_array(pk, 3);
         msgpack_pack_array(pk, 3);
          msgpack_pack_uint8(pk, 1);
          msgpack_pack_array(pk, 10);
           msgpack_pack_uint32(pk, 843);
           msgpack_pack_uint32(pk, 888);
           msgpack_pack_uint32(pk, 890);
           msgpack_pack_uint32(pk, 1014);
           msgpack_pack_raw(pk, 10);
           msgpack_pack_raw_body(pk, "text/plain", 10);
           msgpack_pack_raw(pk, 0);
           msgpack_pack_raw_body(pk, "", 0);
           msgpack_pack_raw(pk, 0);
           msgpack_pack_raw_body(pk, "", 0);
           msgpack_pack_raw(pk, 0);
           msgpack_pack_raw_body(pk, "", 0);
           msgpack_pack_raw(pk, 0);
           msgpack_pack_raw_body(pk, "", 0);
           msgpack_pack_raw(pk, 0);
           msgpack_pack_raw_body(pk, "", 0);
          msgpack_pack_array(pk, 0);
         msgpack_pack_array(pk, 3);
          msgpack_pack_uint8(pk, 1);
          msgpack_pack_array(pk, 10);
           msgpack_pack_uint32(pk, 1048);
           msgpack_pack_uint32(pk, 1106);
           msgpack_pack_uint32(pk, 1108);
           msgpack_pack_uint32(pk, 1184);
           msgpack_pack_raw(pk, 23);
           msgpack_pack_raw_body(pk, "message/feedback-report", 23);
           msgpack_pack_raw(pk, 0);
           msgpack_pack_raw_body(pk, "", 0);
           msgpack_pack_raw(pk, 0);
           msgpack_pack_raw_body(pk, "", 0);
           msgpack_pack_raw(pk, 0);
           msgpack_pack_raw_body(pk, "", 0);
           msgpack_pack_raw(pk, 0);
           msgpack_pack_raw_body(pk, "", 0);
           msgpack_pack_raw(pk, 0);
           msgpack_pack_raw_body(pk, "", 0);
          msgpack_pack_array(pk, 0);
         msgpack_pack_array(pk, 3);
          msgpack_pack_uint8(pk, 1);
          msgpack_pack_array(pk, 10);
           msgpack_pack_uint32(pk, 1218);
           msgpack_pack_uint32(pk, 1267);
           msgpack_pack_uint32(pk, 1269);
           msgpack_pack_uint32(pk, 33622);
           msgpack_pack_raw(pk, 14);
           msgpack_pack_raw_body(pk, "message/rfc822", 14);
           msgpack_pack_raw(pk, 0);
           msgpack_pack_raw_body(pk, "", 0);
           msgpack_pack_raw(pk, 0);
           msgpack_pack_raw_body(pk, "", 0);
           msgpack_pack_raw(pk, 12);
           msgpack_pack_raw_body(pk, "untitled.eml", 12);
           msgpack_pack_raw(pk, 0);
           msgpack_pack_raw_body(pk, "", 0);
           msgpack_pack_raw(pk, 0);
           msgpack_pack_raw_body(pk, "", 0);
          msgpack_pack_array(pk, 1);
           msgpack_pack_array(pk, 3);
            msgpack_pack_uint8(pk, 1);
            msgpack_pack_array(pk, 10);
             msgpack_pack_uint32(pk, 1269);
             msgpack_pack_uint32(pk, 3092);
             msgpack_pack_uint32(pk, 3094);
             msgpack_pack_uint32(pk, 33622);
             msgpack_pack_raw(pk, 15);
             msgpack_pack_raw_body(pk, "multipart/mixed", 15);
             msgpack_pack_raw(pk, 6);
             msgpack_pack_raw_body(pk, "binary", 6);
             msgpack_pack_raw(pk, 0);
             msgpack_pack_raw_body(pk, "", 0);
             msgpack_pack_raw(pk, 0);
             msgpack_pack_raw_body(pk, "", 0);
             msgpack_pack_raw(pk, 0);
             msgpack_pack_raw_body(pk, "", 0);
             msgpack_pack_raw(pk, 0);
             msgpack_pack_raw_body(pk, "", 0);
            msgpack_pack_array(pk, 2);
             msgpack_pack_array(pk, 3);
              msgpack_pack_uint8(pk, 1);
              msgpack_pack_array(pk, 10);
               msgpack_pack_uint32(pk, 3175);
               msgpack_pack_uint32(pk, 3308);
               msgpack_pack_uint32(pk, 3310);
               msgpack_pack_uint32(pk, 6306);
               msgpack_pack_raw(pk, 21);
               msgpack_pack_raw_body(pk, "multipart/alternative", 21);
               msgpack_pack_raw(pk, 6);
               msgpack_pack_raw_body(pk, "binary", 6);
               msgpack_pack_raw(pk, 0);
               msgpack_pack_raw_body(pk, "", 0);
               msgpack_pack_raw(pk, 0);
               msgpack_pack_raw_body(pk, "", 0);
               msgpack_pack_raw(pk, 0);
               msgpack_pack_raw_body(pk, "", 0);
               msgpack_pack_raw(pk, 0);
               msgpack_pack_raw_body(pk, "", 0);
              msgpack_pack_array(pk, 2);
               msgpack_pack_array(pk, 3);
                msgpack_pack_uint8(pk, 1);
                msgpack_pack_array(pk, 10);
                 msgpack_pack_uint32(pk, 3391);
                 msgpack_pack_uint32(pk, 3545);
                 msgpack_pack_uint32(pk, 3547);
                 msgpack_pack_uint32(pk, 4180);
                 msgpack_pack_raw(pk, 10);
                 msgpack_pack_raw_body(pk, "text/plain", 10);
                 msgpack_pack_raw(pk, 6);
                 msgpack_pack_raw_body(pk, "binary", 6);
                 msgpack_pack_raw(pk, 12);
                 msgpack_pack_raw_body(pk, "windows-1251", 12);
                 msgpack_pack_raw(pk, 0);
                 msgpack_pack_raw_body(pk, "", 0);
                 msgpack_pack_raw(pk, 0);
                 msgpack_pack_raw_body(pk, "", 0);
                 msgpack_pack_raw(pk, 6);
                 msgpack_pack_raw_body(pk, "inline", 6);
                msgpack_pack_array(pk, 0);
               msgpack_pack_array(pk, 3);
                msgpack_pack_uint8(pk, 1);
                msgpack_pack_array(pk, 10);
                 msgpack_pack_uint32(pk, 4213);
                 msgpack_pack_uint32(pk, 4348);
                 msgpack_pack_uint32(pk, 4350);
                 msgpack_pack_uint32(pk, 6267);
                 msgpack_pack_raw(pk, 9);
                 msgpack_pack_raw_body(pk, "text/html", 9);
                 msgpack_pack_raw(pk, 6);
                 msgpack_pack_raw_body(pk, "binary", 6);
                 msgpack_pack_raw(pk, 12);
                 msgpack_pack_raw_body(pk, "windows-1251", 12);
                 msgpack_pack_raw(pk, 0);
                 msgpack_pack_raw_body(pk, "", 0);
                 msgpack_pack_raw(pk, 0);
                 msgpack_pack_raw_body(pk, "", 0);
                 msgpack_pack_raw(pk, 6);
                 msgpack_pack_raw_body(pk, "inline", 6);
                msgpack_pack_array(pk, 0);
             msgpack_pack_array(pk, 3);
              msgpack_pack_uint8(pk, 1);
              msgpack_pack_array(pk, 10);
               msgpack_pack_uint32(pk, 6339);
               msgpack_pack_uint32(pk, 6742);
               msgpack_pack_uint32(pk, 6744);
               msgpack_pack_uint32(pk, 33581);
               msgpack_pack_raw(pk, 14);
               msgpack_pack_raw_body(pk, "message/rfc822", 14);
               msgpack_pack_raw(pk, 6);
               msgpack_pack_raw_body(pk, "binary", 6);
               msgpack_pack_raw(pk, 0);
               msgpack_pack_raw_body(pk, "", 0);
               msgpack_pack_raw(pk, 135);
               msgpack_pack_raw_body(pk, "Fw:  subject.eml", 135);
               msgpack_pack_raw(pk, 0);
               msgpack_pack_raw_body(pk, "", 0);
               msgpack_pack_raw(pk, 10);
               msgpack_pack_raw_body(pk, "attachment", 10);
              msgpack_pack_array(pk, 1);
               msgpack_pack_array(pk, 3);
                msgpack_pack_uint8(pk, 1);
                msgpack_pack_array(pk, 10);
                 msgpack_pack_uint32(pk, 6744);
                 msgpack_pack_uint32(pk, 8195);
                 msgpack_pack_uint32(pk, 8197);
                 msgpack_pack_uint32(pk, 33581);
                 msgpack_pack_raw(pk, 15);
                 msgpack_pack_raw_body(pk, "multipart/mixed", 15);
                 msgpack_pack_raw(pk, 6);
                 msgpack_pack_raw_body(pk, "binary", 6);
                 msgpack_pack_raw(pk, 0);
                 msgpack_pack_raw_body(pk, "", 0);
                 msgpack_pack_raw(pk, 0);
                 msgpack_pack_raw_body(pk, "", 0);
                 msgpack_pack_raw(pk, 0);
                 msgpack_pack_raw_body(pk, "", 0);
                 msgpack_pack_raw(pk, 0);
                 msgpack_pack_raw_body(pk, "", 0);
                msgpack_pack_array(pk, 2);
                 msgpack_pack_array(pk, 3);
                  msgpack_pack_uint8(pk, 1);
                  msgpack_pack_array(pk, 10);
                   msgpack_pack_uint32(pk, 8278);
                   msgpack_pack_uint32(pk, 8411);
                   msgpack_pack_uint32(pk, 8413);
                   msgpack_pack_uint32(pk, 10033);
                   msgpack_pack_raw(pk, 21);
                   msgpack_pack_raw_body(pk, "multipart/alternative", 21);
                   msgpack_pack_raw(pk, 6);
                   msgpack_pack_raw_body(pk, "binary", 6);
                   msgpack_pack_raw(pk, 0);
                   msgpack_pack_raw_body(pk, "", 0);
                   msgpack_pack_raw(pk, 0);
                   msgpack_pack_raw_body(pk, "", 0);
                   msgpack_pack_raw(pk, 0);
                   msgpack_pack_raw_body(pk, "", 0);
                   msgpack_pack_raw(pk, 0);
                   msgpack_pack_raw_body(pk, "", 0);
                  msgpack_pack_array(pk, 2);
                   msgpack_pack_array(pk, 3);
                    msgpack_pack_uint8(pk, 1);
                    msgpack_pack_array(pk, 10);
                     msgpack_pack_uint32(pk, 8494);
                     msgpack_pack_uint32(pk, 8648);
                     msgpack_pack_uint32(pk, 8650);
                     msgpack_pack_uint32(pk, 8984);
                     msgpack_pack_raw(pk, 10);
                     msgpack_pack_raw_body(pk, "text/plain", 10);
                     msgpack_pack_raw(pk, 6);
                     msgpack_pack_raw_body(pk, "binary", 6);
                     msgpack_pack_raw(pk, 12);
                     msgpack_pack_raw_body(pk, "windows-1251", 12);
                     msgpack_pack_raw(pk, 0);
                     msgpack_pack_raw_body(pk, "", 0);
                     msgpack_pack_raw(pk, 0);
                     msgpack_pack_raw_body(pk, "", 0);
                     msgpack_pack_raw(pk, 6);
                     msgpack_pack_raw_body(pk, "inline", 6);
                    msgpack_pack_array(pk, 0);
                   msgpack_pack_array(pk, 3);
                    msgpack_pack_uint8(pk, 1);
                    msgpack_pack_array(pk, 10);
                     msgpack_pack_uint32(pk, 9017);
                     msgpack_pack_uint32(pk, 9151);
                     msgpack_pack_uint32(pk, 9153);
                     msgpack_pack_uint32(pk, 9994);
                     msgpack_pack_raw(pk, 9);
                     msgpack_pack_raw_body(pk, "text/html", 9);
                     msgpack_pack_raw(pk, 6);
                     msgpack_pack_raw_body(pk, "binary", 6);
                     msgpack_pack_raw(pk, 12);
                     msgpack_pack_raw_body(pk, "windows-1251", 12);
                     msgpack_pack_raw(pk, 0);
                     msgpack_pack_raw_body(pk, "", 0);
                     msgpack_pack_raw(pk, 0);
                     msgpack_pack_raw_body(pk, "", 0);
                     msgpack_pack_raw(pk, 6);
                     msgpack_pack_raw_body(pk, "inline", 6);
                    msgpack_pack_array(pk, 0);
                 msgpack_pack_array(pk, 3);
                  msgpack_pack_uint8(pk, 1);
                  msgpack_pack_array(pk, 10);
                   msgpack_pack_uint32(pk, 10066);
                   msgpack_pack_uint32(pk, 10445);
                   msgpack_pack_uint32(pk, 10447);
                   msgpack_pack_uint32(pk, 33542);
                   msgpack_pack_raw(pk, 14);
                   msgpack_pack_raw_body(pk, "message/rfc822", 14);
                   msgpack_pack_raw(pk, 6);
                   msgpack_pack_raw_body(pk, "binary", 6);
                   msgpack_pack_raw(pk, 0);
                   msgpack_pack_raw_body(pk, "", 0);
                   msgpack_pack_raw(pk, 131);
                   msgpack_pack_raw_body(pk, " subject.eml", 131);
                   msgpack_pack_raw(pk, 0);
                   msgpack_pack_raw_body(pk, "", 0);
                   msgpack_pack_raw(pk, 10);
                   msgpack_pack_raw_body(pk, "attachment", 10);
                  msgpack_pack_array(pk, 1);
                   msgpack_pack_array(pk, 3);
                    msgpack_pack_uint8(pk, 1);
                    msgpack_pack_array(pk, 10);
                     msgpack_pack_uint32(pk, 10447);
                     msgpack_pack_uint32(pk, 11525);
                     msgpack_pack_uint32(pk, 11527);
                     msgpack_pack_uint32(pk, 33542);
                     msgpack_pack_raw(pk, 15);
                     msgpack_pack_raw_body(pk, "multipart/mixed", 15);
                     msgpack_pack_raw(pk, 6);
                     msgpack_pack_raw_body(pk, "binary", 6);
                     msgpack_pack_raw(pk, 0);
                     msgpack_pack_raw_body(pk, "", 0);
                     msgpack_pack_raw(pk, 0);
                     msgpack_pack_raw_body(pk, "", 0);
                     msgpack_pack_raw(pk, 0);
                     msgpack_pack_raw_body(pk, "", 0);
                     msgpack_pack_raw(pk, 0);
                     msgpack_pack_raw_body(pk, "", 0);
                    msgpack_pack_array(pk, 2);
                     msgpack_pack_array(pk, 3);
                      msgpack_pack_uint8(pk, 1);
                      msgpack_pack_array(pk, 10);
                       msgpack_pack_uint32(pk, 11608);
                       msgpack_pack_uint32(pk, 11741);
                       msgpack_pack_uint32(pk, 11743);
                       msgpack_pack_uint32(pk, 14824);
                       msgpack_pack_raw(pk, 21);
                       msgpack_pack_raw_body(pk, "multipart/alternative", 21);
                       msgpack_pack_raw(pk, 6);
                       msgpack_pack_raw_body(pk, "binary", 6);
                       msgpack_pack_raw(pk, 0);
                       msgpack_pack_raw_body(pk, "", 0);
                       msgpack_pack_raw(pk, 0);
                       msgpack_pack_raw_body(pk, "", 0);
                       msgpack_pack_raw(pk, 0);
                       msgpack_pack_raw_body(pk, "", 0);
                       msgpack_pack_raw(pk, 0);
                       msgpack_pack_raw_body(pk, "", 0);
                      msgpack_pack_array(pk, 2);
                       msgpack_pack_array(pk, 3);
                        msgpack_pack_uint8(pk, 1);
                        msgpack_pack_array(pk, 10);
                         msgpack_pack_uint32(pk, 11824);
                         msgpack_pack_uint32(pk, 11978);
                         msgpack_pack_uint32(pk, 11980);
                         msgpack_pack_uint32(pk, 12649);
                         msgpack_pack_raw(pk, 10);
                         msgpack_pack_raw_body(pk, "text/plain", 10);
                         msgpack_pack_raw(pk, 6);
                         msgpack_pack_raw_body(pk, "binary", 6);
                         msgpack_pack_raw(pk, 12);
                         msgpack_pack_raw_body(pk, "windows-1251", 12);
                         msgpack_pack_raw(pk, 0);
                         msgpack_pack_raw_body(pk, "", 0);
                         msgpack_pack_raw(pk, 0);
                         msgpack_pack_raw_body(pk, "", 0);
                         msgpack_pack_raw(pk, 6);
                         msgpack_pack_raw_body(pk, "inline", 6);
                        msgpack_pack_array(pk, 0);
                       msgpack_pack_array(pk, 3);
                        msgpack_pack_uint8(pk, 1);
                        msgpack_pack_array(pk, 10);
                         msgpack_pack_uint32(pk, 12682);
                         msgpack_pack_uint32(pk, 12817);
                         msgpack_pack_uint32(pk, 12819);
                         msgpack_pack_uint32(pk, 14785);
                         msgpack_pack_raw(pk, 9);
                         msgpack_pack_raw_body(pk, "text/html", 9);
                         msgpack_pack_raw(pk, 6);
                         msgpack_pack_raw_body(pk, "binary", 6);
                         msgpack_pack_raw(pk, 12);
                         msgpack_pack_raw_body(pk, "windows-1251", 12);
                         msgpack_pack_raw(pk, 0);
                         msgpack_pack_raw_body(pk, "", 0);
                         msgpack_pack_raw(pk, 0);
                         msgpack_pack_raw_body(pk, "", 0);
                         msgpack_pack_raw(pk, 6);
                         msgpack_pack_raw_body(pk, "inline", 6);
                        msgpack_pack_array(pk, 0);
                     msgpack_pack_array(pk, 3);
                      msgpack_pack_uint8(pk, 1);
                      msgpack_pack_array(pk, 10);
                       msgpack_pack_uint32(pk, 14857);
                       msgpack_pack_uint32(pk, 15267);
                       msgpack_pack_uint32(pk, 15269);
                       msgpack_pack_uint32(pk, 33503);
                       msgpack_pack_raw(pk, 14);
                       msgpack_pack_raw_body(pk, "message/rfc822", 14);
                       msgpack_pack_raw(pk, 6);
                       msgpack_pack_raw_body(pk, "binary", 6);
                       msgpack_pack_raw(pk, 0);
                       msgpack_pack_raw_body(pk, "", 0);
                       msgpack_pack_raw(pk, 142);
                       msgpack_pack_raw_body(pk, "Fw: Fw: Fw: subject.eml", 142);
                       msgpack_pack_raw(pk, 0);
                       msgpack_pack_raw_body(pk, "", 0);
                       msgpack_pack_raw(pk, 10);
                       msgpack_pack_raw_body(pk, "attachment", 10);
                      msgpack_pack_array(pk, 1);
                       msgpack_pack_array(pk, 3);
                        msgpack_pack_uint8(pk, 1);
                        msgpack_pack_array(pk, 10);
                         msgpack_pack_uint32(pk, 15269);
                         msgpack_pack_uint32(pk, 16637);
                         msgpack_pack_uint32(pk, 16639);
                         msgpack_pack_uint32(pk, 33503);
                         msgpack_pack_raw(pk, 15);
                         msgpack_pack_raw_body(pk, "multipart/mixed", 15);
                         msgpack_pack_raw(pk, 6);
                         msgpack_pack_raw_body(pk, "binary", 6);
                         msgpack_pack_raw(pk, 0);
                         msgpack_pack_raw_body(pk, "", 0);
                         msgpack_pack_raw(pk, 0);
                         msgpack_pack_raw_body(pk, "", 0);
                         msgpack_pack_raw(pk, 0);
                         msgpack_pack_raw_body(pk, "", 0);
                         msgpack_pack_raw(pk, 0);
                         msgpack_pack_raw_body(pk, "", 0);
                        msgpack_pack_array(pk, 2);
                         msgpack_pack_array(pk, 3);
                          msgpack_pack_uint8(pk, 1);
                          msgpack_pack_array(pk, 10);
                           msgpack_pack_uint32(pk, 16719);
                           msgpack_pack_uint32(pk, 16851);
                           msgpack_pack_uint32(pk, 16853);
                           msgpack_pack_uint32(pk, 19647);
                           msgpack_pack_raw(pk, 21);
                           msgpack_pack_raw_body(pk, "multipart/alternative", 21);
                           msgpack_pack_raw(pk, 6);
                           msgpack_pack_raw_body(pk, "binary", 6);
                           msgpack_pack_raw(pk, 0);
                           msgpack_pack_raw_body(pk, "", 0);
                           msgpack_pack_raw(pk, 0);
                           msgpack_pack_raw_body(pk, "", 0);
                           msgpack_pack_raw(pk, 0);
                           msgpack_pack_raw_body(pk, "", 0);
                           msgpack_pack_raw(pk, 0);
                           msgpack_pack_raw_body(pk, "", 0);
                          msgpack_pack_array(pk, 2);
                           msgpack_pack_array(pk, 3);
                            msgpack_pack_uint8(pk, 1);
                            msgpack_pack_array(pk, 10);
                             msgpack_pack_uint32(pk, 16933);
                             msgpack_pack_uint32(pk, 17087);
                             msgpack_pack_uint32(pk, 17089);
                             msgpack_pack_uint32(pk, 17669);
                             msgpack_pack_raw(pk, 10);
                             msgpack_pack_raw_body(pk, "text/plain", 10);
                             msgpack_pack_raw(pk, 6);
                             msgpack_pack_raw_body(pk, "binary", 6);
                             msgpack_pack_raw(pk, 12);
                             msgpack_pack_raw_body(pk, "windows-1251", 12);
                             msgpack_pack_raw(pk, 0);
                             msgpack_pack_raw_body(pk, "", 0);
                             msgpack_pack_raw(pk, 0);
                             msgpack_pack_raw_body(pk, "", 0);
                             msgpack_pack_raw(pk, 6);
                             msgpack_pack_raw_body(pk, "inline", 6);
                            msgpack_pack_array(pk, 0);
                           msgpack_pack_array(pk, 3);
                            msgpack_pack_uint8(pk, 1);
                            msgpack_pack_array(pk, 10);
                             msgpack_pack_uint32(pk, 17701);
                             msgpack_pack_uint32(pk, 17836);
                             msgpack_pack_uint32(pk, 17838);
                             msgpack_pack_uint32(pk, 19609);
                             msgpack_pack_raw(pk, 9);
                             msgpack_pack_raw_body(pk, "text/html", 9);
                             msgpack_pack_raw(pk, 6);
                             msgpack_pack_raw_body(pk, "binary", 6);
                             msgpack_pack_raw(pk, 12);
                             msgpack_pack_raw_body(pk, "windows-1251", 12);
                             msgpack_pack_raw(pk, 0);
                             msgpack_pack_raw_body(pk, "", 0);
                             msgpack_pack_raw(pk, 0);
                             msgpack_pack_raw_body(pk, "", 0);
                             msgpack_pack_raw(pk, 6);
                             msgpack_pack_raw_body(pk, "inline", 6);
                            msgpack_pack_array(pk, 0);
                         msgpack_pack_array(pk, 3);
                          msgpack_pack_uint8(pk, 1);
                          msgpack_pack_array(pk, 10);
                           msgpack_pack_uint32(pk, 19679);
                           msgpack_pack_uint32(pk, 20085);
                           msgpack_pack_uint32(pk, 20087);
                           msgpack_pack_uint32(pk, 33465);
                           msgpack_pack_raw(pk, 14);
                           msgpack_pack_raw_body(pk, "message/rfc822", 14);
                           msgpack_pack_raw(pk, 6);
                           msgpack_pack_raw_body(pk, "binary", 6);
                           msgpack_pack_raw(pk, 0);
                           msgpack_pack_raw_body(pk, "", 0);
                           msgpack_pack_raw(pk, 138);
                           msgpack_pack_raw_body(pk, "Fw: Fw: subject.eml", 138);
                           msgpack_pack_raw(pk, 0);
                           msgpack_pack_raw_body(pk, "", 0);
                           msgpack_pack_raw(pk, 10);
                           msgpack_pack_raw_body(pk, "attachment", 10);
                          msgpack_pack_array(pk, 1);
                           msgpack_pack_array(pk, 3);
                            msgpack_pack_uint8(pk, 1);
                            msgpack_pack_array(pk, 10);
                             msgpack_pack_uint32(pk, 20087);
                             msgpack_pack_uint32(pk, 21313);
                             msgpack_pack_uint32(pk, 21315);
                             msgpack_pack_uint32(pk, 33465);
                             msgpack_pack_raw(pk, 15);
                             msgpack_pack_raw_body(pk, "multipart/mixed", 15);
                             msgpack_pack_raw(pk, 6);
                             msgpack_pack_raw_body(pk, "binary", 6);
                             msgpack_pack_raw(pk, 0);
                             msgpack_pack_raw_body(pk, "", 0);
                             msgpack_pack_raw(pk, 0);
                             msgpack_pack_raw_body(pk, "", 0);
                             msgpack_pack_raw(pk, 0);
                             msgpack_pack_raw_body(pk, "", 0);
                             msgpack_pack_raw(pk, 0);
                             msgpack_pack_raw_body(pk, "", 0);
                            msgpack_pack_array(pk, 2);
                             msgpack_pack_array(pk, 3);
                              msgpack_pack_uint8(pk, 1);
                              msgpack_pack_array(pk, 10);
                               msgpack_pack_uint32(pk, 21396);
                               msgpack_pack_uint32(pk, 21529);
                               msgpack_pack_uint32(pk, 21531);
                               msgpack_pack_uint32(pk, 24448);
                               msgpack_pack_raw(pk, 21);
                               msgpack_pack_raw_body(pk, "multipart/alternative", 21);
                               msgpack_pack_raw(pk, 6);
                               msgpack_pack_raw_body(pk, "binary", 6);
                               msgpack_pack_raw(pk, 0);
                               msgpack_pack_raw_body(pk, "", 0);
                               msgpack_pack_raw(pk, 0);
                               msgpack_pack_raw_body(pk, "", 0);
                               msgpack_pack_raw(pk, 0);
                               msgpack_pack_raw_body(pk, "", 0);
                               msgpack_pack_raw(pk, 0);
                               msgpack_pack_raw_body(pk, "", 0);
                              msgpack_pack_array(pk, 2);
                               msgpack_pack_array(pk, 3);
                                msgpack_pack_uint8(pk, 1);
                                msgpack_pack_array(pk, 10);
                                 msgpack_pack_uint32(pk, 21612);
                                 msgpack_pack_uint32(pk, 21766);
                                 msgpack_pack_uint32(pk, 21768);
                                 msgpack_pack_uint32(pk, 22387);
                                 msgpack_pack_raw(pk, 10);
                                 msgpack_pack_raw_body(pk, "text/plain", 10);
                                 msgpack_pack_raw(pk, 6);
                                 msgpack_pack_raw_body(pk, "binary", 6);
                                 msgpack_pack_raw(pk, 12);
                                 msgpack_pack_raw_body(pk, "windows-1251", 12);
                                 msgpack_pack_raw(pk, 0);
                                 msgpack_pack_raw_body(pk, "", 0);
                                 msgpack_pack_raw(pk, 0);
                                 msgpack_pack_raw_body(pk, "", 0);
                                 msgpack_pack_raw(pk, 6);
                                 msgpack_pack_raw_body(pk, "inline", 6);
                                msgpack_pack_array(pk, 0);
                               msgpack_pack_array(pk, 3);
                                msgpack_pack_uint8(pk, 1);
                                msgpack_pack_array(pk, 10);
                                 msgpack_pack_uint32(pk, 22420);
                                 msgpack_pack_uint32(pk, 22555);
                                 msgpack_pack_uint32(pk, 22557);
                                 msgpack_pack_uint32(pk, 24409);
                                 msgpack_pack_raw(pk, 9);
                                 msgpack_pack_raw_body(pk, "text/html", 9);
                                 msgpack_pack_raw(pk, 6);
                                 msgpack_pack_raw_body(pk, "binary", 6);
                                 msgpack_pack_raw(pk, 12);
                                 msgpack_pack_raw_body(pk, "windows-1251", 12);
                                 msgpack_pack_raw(pk, 0);
                                 msgpack_pack_raw_body(pk, "", 0);
                                 msgpack_pack_raw(pk, 0);
                                 msgpack_pack_raw_body(pk, "", 0);
                                 msgpack_pack_raw(pk, 6);
                                 msgpack_pack_raw_body(pk, "inline", 6);
                                msgpack_pack_array(pk, 0);
                             msgpack_pack_array(pk, 3);
                              msgpack_pack_uint8(pk, 1);
                              msgpack_pack_array(pk, 10);
                               msgpack_pack_uint32(pk, 24481);
                               msgpack_pack_uint32(pk, 24883);
                               msgpack_pack_uint32(pk, 24885);
                               msgpack_pack_uint32(pk, 33426);
                               msgpack_pack_raw(pk, 14);
                               msgpack_pack_raw_body(pk, "message/rfc822", 14);
                               msgpack_pack_raw(pk, 6);
                               msgpack_pack_raw_body(pk, "binary", 6);
                               msgpack_pack_raw(pk, 0);
                               msgpack_pack_raw_body(pk, "", 0);
                               msgpack_pack_raw(pk, 134);
                               msgpack_pack_raw_body(pk, "Fw: subject.eml", 134);
                               msgpack_pack_raw(pk, 0);
                               msgpack_pack_raw_body(pk, "", 0);
                               msgpack_pack_raw(pk, 10);
                               msgpack_pack_raw_body(pk, "attachment", 10);
                              msgpack_pack_array(pk, 1);
                               msgpack_pack_array(pk, 3);
                                msgpack_pack_uint8(pk, 1);
                                msgpack_pack_array(pk, 10);
                                 msgpack_pack_uint32(pk, 24885);
                                 msgpack_pack_uint32(pk, 26090);
                                 msgpack_pack_uint32(pk, 26092);
                                 msgpack_pack_uint32(pk, 33426);
                                 msgpack_pack_raw(pk, 15);
                                 msgpack_pack_raw_body(pk, "multipart/mixed", 15);
                                 msgpack_pack_raw(pk, 6);
                                 msgpack_pack_raw_body(pk, "binary", 6);
                                 msgpack_pack_raw(pk, 0);
                                 msgpack_pack_raw_body(pk, "", 0);
                                 msgpack_pack_raw(pk, 0);
                                 msgpack_pack_raw_body(pk, "", 0);
                                 msgpack_pack_raw(pk, 0);
                                 msgpack_pack_raw_body(pk, "", 0);
                                 msgpack_pack_raw(pk, 0);
                                 msgpack_pack_raw_body(pk, "", 0);
                                msgpack_pack_array(pk, 2);
                                 msgpack_pack_array(pk, 3);
                                  msgpack_pack_uint8(pk, 1);
                                  msgpack_pack_array(pk, 10);
                                   msgpack_pack_uint32(pk, 26172);
                                   msgpack_pack_uint32(pk, 26304);
                                   msgpack_pack_uint32(pk, 26306);
                                   msgpack_pack_uint32(pk, 29382);
                                   msgpack_pack_raw(pk, 21);
                                   msgpack_pack_raw_body(pk, "multipart/alternative", 21);
                                   msgpack_pack_raw(pk, 6);
                                   msgpack_pack_raw_body(pk, "binary", 6);
                                   msgpack_pack_raw(pk, 0);
                                   msgpack_pack_raw_body(pk, "", 0);
                                   msgpack_pack_raw(pk, 0);
                                   msgpack_pack_raw_body(pk, "", 0);
                                   msgpack_pack_raw(pk, 0);
                                   msgpack_pack_raw_body(pk, "", 0);
                                   msgpack_pack_raw(pk, 0);
                                   msgpack_pack_raw_body(pk, "", 0);
                                  msgpack_pack_array(pk, 2);
                                   msgpack_pack_array(pk, 3);
                                    msgpack_pack_uint8(pk, 1);
                                    msgpack_pack_array(pk, 10);
                                     msgpack_pack_uint32(pk, 26386);
                                     msgpack_pack_uint32(pk, 26540);
                                     msgpack_pack_uint32(pk, 26542);
                                     msgpack_pack_uint32(pk, 27202);
                                     msgpack_pack_raw(pk, 10);
                                     msgpack_pack_raw_body(pk, "text/plain", 10);
                                     msgpack_pack_raw(pk, 6);
                                     msgpack_pack_raw_body(pk, "binary", 6);
                                     msgpack_pack_raw(pk, 12);
                                     msgpack_pack_raw_body(pk, "windows-1251", 12);
                                     msgpack_pack_raw(pk, 0);
                                     msgpack_pack_raw_body(pk, "", 0);
                                     msgpack_pack_raw(pk, 0);
                                     msgpack_pack_raw_body(pk, "", 0);
                                     msgpack_pack_raw(pk, 6);
                                     msgpack_pack_raw_body(pk, "inline", 6);
                                    msgpack_pack_array(pk, 0);
                                   msgpack_pack_array(pk, 3);
                                    msgpack_pack_uint8(pk, 1);
                                    msgpack_pack_array(pk, 10);
                                     msgpack_pack_uint32(pk, 27234);
                                     msgpack_pack_uint32(pk, 27369);
                                     msgpack_pack_uint32(pk, 27371);
                                     msgpack_pack_uint32(pk, 29344);
                                     msgpack_pack_raw(pk, 9);
                                     msgpack_pack_raw_body(pk, "text/html", 9);
                                     msgpack_pack_raw(pk, 6);
                                     msgpack_pack_raw_body(pk, "binary", 6);
                                     msgpack_pack_raw(pk, 12);
                                     msgpack_pack_raw_body(pk, "windows-1251", 12);
                                     msgpack_pack_raw(pk, 0);
                                     msgpack_pack_raw_body(pk, "", 0);
                                     msgpack_pack_raw(pk, 0);
                                     msgpack_pack_raw_body(pk, "", 0);
                                     msgpack_pack_raw(pk, 6);
                                     msgpack_pack_raw_body(pk, "inline", 6);
                                    msgpack_pack_array(pk, 0);
                                 msgpack_pack_array(pk, 3);
                                  msgpack_pack_uint8(pk, 1);
                                  msgpack_pack_array(pk, 10);
                                   msgpack_pack_uint32(pk, 29414);
                                   msgpack_pack_uint32(pk, 29792);
                                   msgpack_pack_uint32(pk, 29794);
                                   msgpack_pack_uint32(pk, 33388);
                                   msgpack_pack_raw(pk, 14);
                                   msgpack_pack_raw_body(pk, "message/rfc822", 14);
                                   msgpack_pack_raw(pk, 6);
                                   msgpack_pack_raw_body(pk, "binary", 6);
                                   msgpack_pack_raw(pk, 0);
                                   msgpack_pack_raw_body(pk, "", 0);
                                   msgpack_pack_raw(pk, 130);
                                   msgpack_pack_raw_body(pk, "subject.eml", 130);
                                   msgpack_pack_raw(pk, 0);
                                   msgpack_pack_raw_body(pk, "", 0);
                                   msgpack_pack_raw(pk, 10);
                                   msgpack_pack_raw_body(pk, "attachment", 10);
                                  msgpack_pack_array(pk, 1);
                                   msgpack_pack_array(pk, 3);
                                    msgpack_pack_uint8(pk, 1);
                                    msgpack_pack_array(pk, 10);
                                     msgpack_pack_uint32(pk, 29794);
                                     msgpack_pack_uint32(pk, 30925);
                                     msgpack_pack_uint32(pk, 30927);
                                     msgpack_pack_uint32(pk, 33388);
                                     msgpack_pack_raw(pk, 21);
                                     msgpack_pack_raw_body(pk, "multipart/alternative", 21);
                                     msgpack_pack_raw(pk, 6);
                                     msgpack_pack_raw_body(pk, "binary", 6);
                                     msgpack_pack_raw(pk, 0);
                                     msgpack_pack_raw_body(pk, "", 0);
                                     msgpack_pack_raw(pk, 0);
                                     msgpack_pack_raw_body(pk, "", 0);
                                     msgpack_pack_raw(pk, 0);
                                     msgpack_pack_raw_body(pk, "", 0);
                                     msgpack_pack_raw(pk, 0);
                                     msgpack_pack_raw_body(pk, "", 0);
                                    msgpack_pack_array(pk, 2);
                                     msgpack_pack_array(pk, 3);
                                      msgpack_pack_uint8(pk, 1);
                                      msgpack_pack_array(pk, 10);
                                       msgpack_pack_uint32(pk, 31008);
                                       msgpack_pack_uint32(pk, 31162);
                                       msgpack_pack_uint32(pk, 31164);
                                       msgpack_pack_uint32(pk, 31647);
                                       msgpack_pack_raw(pk, 10);
                                       msgpack_pack_raw_body(pk, "text/plain", 10);
                                       msgpack_pack_raw(pk, 6);
                                       msgpack_pack_raw_body(pk, "binary", 6);
                                       msgpack_pack_raw(pk, 12);
                                       msgpack_pack_raw_body(pk, "windows-1251", 12);
                                       msgpack_pack_raw(pk, 0);
                                       msgpack_pack_raw_body(pk, "", 0);
                                       msgpack_pack_raw(pk, 0);
                                       msgpack_pack_raw_body(pk, "", 0);
                                       msgpack_pack_raw(pk, 6);
                                       msgpack_pack_raw_body(pk, "inline", 6);
                                      msgpack_pack_array(pk, 0);
                                     msgpack_pack_array(pk, 3);
                                      msgpack_pack_uint8(pk, 1);
                                      msgpack_pack_array(pk, 10);
                                       msgpack_pack_uint32(pk, 31680);
                                       msgpack_pack_uint32(pk, 31815);
                                       msgpack_pack_uint32(pk, 31817);
                                       msgpack_pack_uint32(pk, 33349);
                                       msgpack_pack_raw(pk, 9);
                                       msgpack_pack_raw_body(pk, "text/html", 9);
                                       msgpack_pack_raw(pk, 6);
                                       msgpack_pack_raw_body(pk, "binary", 6);
                                       msgpack_pack_raw(pk, 12);
                                       msgpack_pack_raw_body(pk, "windows-1251", 12);
                                       msgpack_pack_raw(pk, 0);
                                       msgpack_pack_raw_body(pk, "", 0);
                                       msgpack_pack_raw(pk, 0);
                                       msgpack_pack_raw_body(pk, "", 0);
                                       msgpack_pack_raw(pk, 6);
                                       msgpack_pack_raw_body(pk, "inline", 6);
                                      msgpack_pack_array(pk, 0);

    msgpack_unpacked    msg;

    int ok;
    size_t offset = 0;
    msgpack_unpacked_init(&msg);
    if(msgpack_unpack_next(&msg, mp_buf.data, mp_buf.size, &offset))
        ok = 1;

    return ok;
}

Error unpacking list with some instances of floating-point numbers.

I persisted a serialized list of floating-point numbers using python:

import msgpack

with open('msgpack.tst', 'wb') as file:
    file.write(msgpack.packb([1.0,  2.0]))

and then tested reading it from a very simple c code:

#include <msgpack.h>
#include <stdio.h>
#include <string.h>
int main(int argc, char *argv[]){
  int BUFFER_SIZE = 1024;
  FILE* source;
  int count = 0;
  char* buffer  = (char*) malloc(BUFFER_SIZE);
  source = fopen("msgpack.tst","rb");
  if(source){
      n = fread(buffer,BUFFER_SIZE, 1, source);
      count = strlen(buffer);
  }else{
     printf("Failed to open file\n");
     return 1;
  }

  /* deserializes it. */
  msgpack_unpacked msg;
  msgpack_unpacked_init(&msg);
  printf("buffer len: %d   value:%s\n",count, buffer);
  bool success = msgpack_unpack_next(&msg, buffer, count, NULL);
  if (success){
       printf("Success unpacking!\n");
       /* prints the deserialized object. */
       msgpack_object obj = msg.data;
       msgpack_object_print(stdout, obj);
       puts("\n");
  }else{
       printf("Unpacking failed!\n");
  }

}

To my surprise the unpacking fails:

buffer len: 12   value:๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ@
Unpacking failed!

But if my python code serializes 'true' floating-point numbers as bellow:

import msgpack

with open('msgpack.tst', 'wb') as file:
    file.write(msgpack.packb([1.2345,  2.9876]))

Then the C code success unpacking this list of floating-point numbers:

buffer len: 19   value:๏ฟฝ๏ฟฝ?๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝn๏ฟฝ๏ฟฝ๏ฟฝ@๏ฟฝ๏ฟฝ๏ฟฝ,<๏ฟฝ
Sucess unpacking!
[1.234500, 2.987600]

Persisting lists of strings and list of integers also works fine.

Regards,

PS : Using Ubuntu 12.04.1, msgpack version is (0,2,2) for python and 0.5.5 for C.

valgrind errors on stream.cc example

Hi,

valgrind reports errors on stream.cc example. Could someone please-please review the code (I am new to the library, I cannot do this myself yet).

Sasha

int64 unpacking failure.

Pretty critical thing. High probability of int64 value being broken.
Reproduced with Ubuntu's package 0.5.7-2.

Here is the test snippet:

int64_t test_i64 = 0xFFF7FFFFFFFFFFFFLL;
msgpack_sbuffer buf;
msgpack_sbuffer_init(&buf);
msgpack_packer * pk = msgpack_packer_new(&buf, msgpack_sbuffer_write);
msgpack_pack_array(pk, 2);
msgpack_pack_int32(pk, 0);
msgpack_pack_int64(pk, test_i64);
msgpack_packer_free(pk);
uint32_t upk_pos = 0;
msgpack_unpacked msg;
msgpack_unpacked_init(&msg);
if(msgpack_unpack_next(&msg, buf.data, buf.size, &upk_pos))
    assert(msg.data.via.array.ptr[1].via.i64 == test_i64);
msgpack_sbuffer_destroy(&buf);

0xFFF7FFFFFFFFFFFFLL is a specific value I found to be broken after unpacking. 53th bit will always be set to 1 for some reason. That's the cause of the problem. Also I found that int64 must be packed inside of an array to reproduce the bug. Everything said above is truth for uint64 as well.

Parsing incomplete chunks

Is there a builtin facility to parse data in chunks? Eg: When parsing data coming from a tcp socket, not all data of a msgpack document may be readily available. To illustrate better, consider this simple scenario:

Lets say I receive a chunk of bytes 'a' which contains 50% of a msgpack document. If I tell messagepack to parse, it will fail because the data is incomplete. The most obvious way to handle this is to wait for another chunk 'b' which contains the other 50% , concatenate both chunks into a new memory location and try again, but this will start reparsing from the beginning.

My question is: is there a way to store partial 'parsing state' so when 'b' arrives I can just feed 'b' to the unpacker and it will just work? If so, could you provide a simple example?

C++11: Make some non-copyable types move-only.

A little feature request: I want to write a function that looks something like this:

msgpack::unpacker get_data() {
    msgpack::unpacker up(SIZE);
    size_t len = some_read_func(up.buffer(), SIZE);
    up.buffer_consumed(len);
    return std::move(up);
}

Instead I ended up writing something like this:

std::string get_data() {
  char buf[SIZE];
  size_t len = some_read_func(buf, SIZE);
  return std::string(buf, buf+len);
}

...

std::string dataStr = get_data();
msgpack::unpacker up(dataStr.size());
memcpy(up.buffer(), dataStr.data(), dataStr.size());

Non-copyable types should at least be movable. Seems only the poc/0.6 branch has C++11 support, but I didn't see this feature in that branch either. I'm not sure of the best way to represent a "moved" object. Tentatively, I tried this:

inline unpacker::unpacker(unpacker&& up) : msgpack_unpacker(up)
{
    bzero(&up, sizeof(up));
}

inline unpacker::~unpacker()
{
    if(buffer() != nullptr && ctx != nullptr) {
        msgpack_unpacker_destroy(this);
    }
}

This seems to work, even if it feels naiive. I think the same should apply to packer, zone, and maybe object if it manages any memory on its own.

fbuffer and sbuffer seem to have good excuses for being non-movable in that a packer might point to them, but why does the packer not hold the buffer itself?

I could submit a PR if requested, but since I am having troubles installing zlib setting up my system for compiling/testing msgpack, it would be largely untested.

Stop copying files in preprocess script

Some files like /src/msgpack/pack_template.h are copied by /preprocess script. We cannot directly modify copied files because they're copied everytime the preprocess script runs and they're ignored by git (.gitignore has their file paths). This is very confusing. I think we can safely stop copying those files.

I guess this process came from the old msgpack repository, which the same files including pack_template.h had to be copied to multiple directories for a few language bindings depending on those C programs. However, we don't need it now.

Can not work with int(when value>48)?

I want to use std::string instead of msgpack::sbuffer as the internel scrip, see my code here.

But I get a error value 0 after unpack it, what's the reason here?

Besides, other type can work well such as double...

Compilation problem

Hi all,

I am trying to compile any of the examples, but it fails. For example, if I try to complie simple.c, I got the following:

$ gcc -lmsgpack simple.c -o simple
/tmp/cctYARIo.o: In function main': simple.c:(.text+0x5cb): undefined reference tomsgpack_zone_init'
simple.c:(.text+0x5f7): undefined reference to msgpack_unpack' simple.c:(.text+0x61c): undefined reference tomsgpack_object_print'
simple.c:(.text+0x634): undefined reference to `msgpack_zone_destroy'
collect2: ld returned 1 exit status

The gcc version is 4.6.3). Any insights on how to solve this issue? Thanks in advance.

Serialization of 64-bit floats broken on iOS ARM 32-bit

This has to do with the __ARM_EABI__ conditional in unpack_template.h / pack_template.h.

If I set a build setting __ARM_EABI__ = 1, then it works, but this kinda sucks. Is there another way to conditionalize this hack for that platform without messing up the build for iOS?

Won't compile without GCC

configure requires a gcc style compiler. It assumes (incorrectly) that -Wall is a valid option. The configure.in template should not assume CFLAGS options, but allow the user to set valid options with configure directly.

I'm testing on AIX with the native compiler.

Whats the latest release?

The repository was updated 10 days ago but the sourceforge files and github tags havent been updated in a few years. Where can I download the latest release?

Inconsistent deserialization behaviors

There're different deserialization behavoirs of STL containers. With vector or deque, operator >> completely replaces their content. For example,

vector<int> v;
v.push_back(1); v.push_back(2); v.push_back(3);
obj >> v; // obj has [4, 5]

v will be [4, 5] in this code. All elements that v previously had are removed. On the other hand, with map or set, operator >> for them doesn't clear their elements.

map<string, int> m;
m["1"] = 1;
obj >> m; // obj has {"2": 2, "3": 3}

In this case, m will be something like {"1": 1, "2": 2, "3": 3}.

We need to discuss which behavior is correct (I personally prefer the former). Moreover, we might not be able to fix this bug until next major version up since we have to change the current behavoir of some functions.

Thank you for the report, Jubatus team!

Compilation error on mac os mountain lion 10.8.2

Steps:
bslima:$ git clone [email protected]:msgpack/msgpack.git
bslima:$ cd msgpack/cpp/
bslima:$ ./bootstrap
bslima:$ ./configure
bslima$ make
make all-recursive
Making all in src
/bin/sh ../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I.. -O3 -Wall -g -MT unpack.lo -MD -MP -MF .deps/unpack.Tpo -c -o unpack.lo unpack.c
libtool: compile: gcc -DHAVE_CONFIG_H -I. -I.. -O3 -Wall -g -MT unpack.lo -MD -MP -MF .deps/unpack.Tpo -c unpack.c -fno-common -DPIC -o .libs/unpack.o
unpack.c:463: internal compiler error: Bus error: 10
Please submit a full bug report,
with preprocessed source if appropriate.
See URL:http://developer.apple.com/bugreporter for instructions.
make[2]: *** [unpack.lo] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

suggestion for examples/very_simple.c

Thanks for writing such a neat piece of software!

I was struggeling a little bit with examples/simple.c, trying to get started. After fiddling around for a bit, I got the snippet below to work. I thought might be useful to include among the other examples.

#include <msgpack.h>
#include <stdio.h>

static inline int my_callback(void* data, const char* buf, unsigned int len)
{
  // print buffer in hex
  for(int i = 0 ; i < len ; i++)
    printf("%02x ", 0xFF & buf[i]);
  printf("\n");
}

int main() 
{
  msgpack_packer pk;
  msgpack_packer_init(&pk, 0, my_callback);

  msgpack_pack_int(&pk, 13);
  msgpack_pack_float(&pk, 0.1);
  msgpack_pack_map(&pk, 1);
  msgpack_pack_int(&pk, 9);
  msgpack_pack_false(&pk);


  return 0;
}

compilation error on mcosx 10.8.3

hi ihave problem compiling new release of msgpac-c

this are my steps:
git clone git://github.com/msgpack/msgpack-c.git -> OK
cd msgpack-c -> OK
./bootstrap -> OK
make ->ERROR

this is the output of make:
make all-recursive
Making all in src
/bin/sh ../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I.. -O3 -Wall -g -MT unpack.lo -MD -MP -MF .deps/unpack.Tpo -c -o unpack.lo unpack.c
libtool: compile: gcc -DHAVE_CONFIG_H -I. -I.. -O3 -Wall -g -MT unpack.lo -MD -MP -MF .deps/unpack.Tpo -c unpack.c -fno-common -DPIC -o .libs/unpack.o
libtool: compile: gcc -DHAVE_CONFIG_H -I. -I.. -O3 -Wall -g -MT unpack.lo -MD -MP -MF .deps/unpack.Tpo -c unpack.c -o unpack.o >/dev/null 2>&1
make[2]: *** [unpack.lo] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

any idea???

clear_chunk_list accesses freed memory; could crash or corrupt heap

The Clang static analyzer points out that the function clear_chunk_list accesses memory after it's been freed, in the line:

cl->head->next = NULL;

The malloc block pointed to by cl->head has already been freed up above in the line

free(c);

The consequences of this are pretty dire. Writing into a free block is likely to corrupt heap structures (depending on the malloc implementation). It could crash immediately if the VM page was freed. Or if the block has already been handed to a malloc call on another thread, it would corrupt another program heap block.

It looks as though the fix is to change the offending line to

cl->head = NULL

although I'm not exactly sure what this function is supposed to do. If the zone is supposed to remain usable, restored to the state it was initially in after init_chunk_list, then this isn't the right fix. Instead it should probably be freeing only the chunks that come after the first one, which would mean modifying the while loop slightly.

unpacker uses default assignment operator

unpacker has a private copy constructor (line 160 unpack.hpp) but does not define a private assignment operator.

copying an unpacker object causes memory corruption errors and should not be allowed (for the same reason the copy constructor is private)

Packer => class PackStream : std::ostream?

It occurred to me that sbuffer functions very similarly to std::streambuf and the only feature missing from Packer is the << operator. Also, fbuffer is similar to a C version of std::filebuf. The equivalent of sbuffer::write() is std::streambuf::sputn. std::ostream also allows the specification of an external buffer to write to. (http://en.cppreference.com/w/cpp/io/basic_ostream/basic_ostream)

As things are, I found that I could replace every instance of sbuffer with std::ostringstream, which defines the write() method. Technically (although this isn't what I'm proposing), sbuffer and fbuffer could be entirely removed by using std::ostringstream and std::ofstream, respectively. However, I don't fully understand zbuffer, but I imagine it could be implemented as a child class of std::streambuf.

To be honest, I haven't done enough research to determine how the best way to integrate this into msgpack would be or if it's even a good idea, I don't know how much code would need to be written, and it's not as if the currently implemented system needs any fixing. But I thought I'd at least put it out there.

I should actually have some time this week to investigate, but as noted before, I can't run the unit tests because of the zlib dependency.

build on Windows

Moved from jira

March 17 - 7:00 PM

Damian commented on MSGPACK-84 (msgpack_vc.postbuild.bat fails with The system cannot find the file specified.) saying:
As a quick workaround I took the header files from my fedora build.

March 17 - 12:00 PM

Damian commented on MSGPACK-84 (msgpack_vc.postbuild.bat fails with The system cannot find the file specified.) saying:
Now I have a problem with zone.hpp which is not found. I can find a file called zone.hpp.erb which I also suppose needs the autotools from Linux?

Damian commented on MSGPACK-84 (msgpack_vc.postbuild.bat fails with The system cannot find the file specified.) saying:
So now my new .bat file looks like this:

IF NOT EXIST include MKDIR include
IF NOT EXIST include\msgpack MKDIR include\msgpack
IF NOT EXIST include\msgpack\type MKDIR include\msgpack\type
IF NOT EXIST include\msgpack\type\tr1 MKDIR include\msgpack\type\tr1
copy pack_define.h include\msgpack
copy pack_template.h include\msgpack
copy unpack_define.h include\msgpack
copy unpack_template.h include\msgpack
copy sysdep.h include\msgpack
copy src\msgpack.h include
copy src\msgpack\sbuffer.h include\msgpack
copy src\msgpack\version.h include\msgpack
copy src\msgpack\vrefbuffer.h include\msgpack
copy src\msgpack\zbuffer.h include\msgpack
copy src\msgpack\pack.h include\msgpack
copy src\msgpack\unpack.h include\msgpack
copy src\msgpack\object.h include\msgpack
copy src\msgpack\zone.h include\msgpack
copy src\msgpack.hpp include
copy src\msgpack\sbuffer.hpp include\msgpack
copy src\msgpack\vrefbuffer.hpp include\msgpack
copy src\msgpack\zbuffer.hpp include\msgpack
copy src\msgpack\pack.hpp include\msgpack
copy src\msgpack\unpack.hpp include\msgpack
copy src\msgpack\object.hpp include\msgpack
copy src\msgpack\zone.hpp include\msgpack
copy src\msgpack\type.hpp include\msgpack
copy src\msgpack\type\bool.hpp include\msgpack\type
copy src\msgpack\type\deque.hpp include\msgpack\type
copy src\msgpack\type\fixint.hpp include\msgpack\type
copy src\msgpack\type\float.hpp include\msgpack\type
copy src\msgpack\type\int.hpp include\msgpack\type
copy src\msgpack\type\list.hpp include\msgpack\type
copy src\msgpack\type\map.hpp include\msgpack\type
copy src\msgpack\type\nil.hpp include\msgpack\type
copy src\msgpack\type\pair.hpp include\msgpack\type
copy src\msgpack\type\raw.hpp include\msgpack\type
copy src\msgpack\type\set.hpp include\msgpack\type
copy src\msgpack\type\string.hpp include\msgpack\type
copy src\msgpack\type\vector.hpp include\msgpack\type
copy src\msgpack\type\tuple.hpp include\msgpack\type
copy src\msgpack\type\define.hpp include\msgpack\type
copy src\msgpack\type\tr1\unordered_map.hpp include\msgpack\type\tr1
copy src\msgpack\type\tr1\unordered_set.hpp include\msgpack\type\tr1\

Damian commented on MSGPACK-84 (msgpack_vc.postbuild.bat fails with The system cannot find the file specified.) saying:
Some of the header files are in d:\dev\GitHub\msgpack-c so that is easy to solve, by copying from there instead.

But version.h.in needs to run though configure which I don't have on Windows. I can manual create the version.h file from version.h.in

March 17 - 11:00 AM

Damian created MSGPACK-84 (msgpack_vc.postbuild.bat fails with The system cannot find the file specified.)

Windows build is broken

It is not possible to build this project as described according to the instructions below, which are found in the downloaded README as well as other msgpack-c sites:

"On Windows, open msgpack_vc8.vcproj or msgpack_vc2008 file and build it using batch build. DLLs are built on lib folder,
and the headers are built on include folder."

The build failure errors are to do with the fact that 1) the project does not have the correct include file path configured to find msgpack\sysdep.h or any other header file in that directory (easily fixed), and 2) many header files cannot be found because they are only available as ruby template files, with a .erb extension:

zone.hpp.erb
define.hpp.erb
tuple.hpp.erb

...these and other header files can apparently only be produced by running some preprocessing tool which is apparently not provided in the download. These build errors cannot be resolved without a functional pre-processing tool.

In the absence of this tool, is it possible to generate the missing .hpp files and make them available for download? Any other workaround?

This problem can be found in both the current snapshot of the source code, as well as 0.5.4, the version that is recommended to build the source from on Windows.

unit tests

How do I run the unit tests when I build from git source myself?

I can see it needs gtest but I can't find any executable after build in test directory.

Would be nice if this was documented anywhere?

Can not link on 32-bit suse

$ gcc -lmsgpack example.c
/usr/local/lib/libmsgpack.so: undefined reference to `__sync_sub_and_fetch_4'
collect2: ld returned 1 exit status

$ gcc -v
Using built-in specs.
Target: i586-suse-linux
Configured with: ../configure --enable-threads=posix --prefix=/usr --with-local-prefix=/usr/local --infodir=/usr/share/info --mandir=/usr/share/man --libdir=/usr/lib --libexecdir=/usr/lib --enable-languages=c,c++,objc,fortran,obj-c++,java,ada --enable-checking=release --with-gxx-include-dir=/usr/include/c++/4.1.2 --enable-ssp --disable-libssp --disable-libgcj --with-slibdir=/lib --with-system-zlib --enable-shared --enable-__cxa_atexit --enable-libstdcxx-allocator=new --program-suffix= --enable-version-specific-runtime-libs --without-system-libunwind --with-cpu=generic --host=i586-suse-linux
Thread model: posix
gcc version 4.1.2 20070115 (prerelease) (SUSE Linux)

what i should do ? Thanks!

unordered_map and unordered_set

I noticed the std::tr1::unordered_map/set is supported, but not the new std::unordered_map/set. Could you please update it?

MSGPACK_DEFINE: 32 limit for arg numbers

for c++98, we can't extend it easily, all we can do is just write extra specialization for more args.

but for c++11, we can use variadic template to break it, although, there is limit, but it's compiler's business.

namespace msgpack {

template< typename Packer >
struct aux_packer {
explicit aux_packer(Packer & pk)
: m_pk(pk)
{
}

template< typename ... args_t >
void operator()(const args_t& ... args)
{
    m_pk.pack_array(sizeof...(args_t));
    auto s = {
        (m_pk.pack(args), 1)...
    };
    (void)s;
}

void operator()()
{
    m_pk.pack_array(1);
}

private:
aux_packer(const aux_packer & rhs) = delete;
aux_packer & operator = (const aux_packer & rhs) = delete;
private:
Packer & m_pk;
};

class aux_unpacker1 {
public:
explicit aux_unpacker1(msgpack::object & obj)
: m_o(obj)
{
}

template< typename ... args_t >
void operator()(args_t& ... args)
{
    if(m_o.type != type::ARRAY) { throw type_error(); }

    if (m_o.via.array.size != sizeof...(args_t)) {
        throw type_error();
    }

    size_t idx = 0;
    auto s = {
        (m_o.via.array.ptr[idx++].convert(&args), 1)...
    };

    (void)s;
}

void operator()()
{
    if(m_o.type != type::ARRAY) { throw type_error(); }
}

private:
aux_unpacker1(const aux_unpacker1 & rhs) = delete;
aux_unpacker1 & operator = (const aux_unpacker1 & rhs) = delete;
private:
msgpack::object & m_o;
};

template
struct aux_unpacker2 {
explicit aux_unpacker2(MSGPACK_OBJECT & o, msgpack::zone & z)
: m_o(o)
, m_z(z)
{
}

template< typename ... args_t >
void operator()(args_t& ... args)
{
    m_o.type = type::ARRAY;

    m_o.via.array.ptr = (object*)m_z.malloc(
        sizeof(object) * sizeof...(args_t));
    m_o.via.array.size = sizeof...(args_t);

    size_t idx = 0;
    auto s = {
        (m_o.via.array.ptr[idx++] = object(args, m_z), 1)...
    };

    (void)s;
}
void operator()()
{
    m_o.type = type::ARRAY;
    m_o.via.array.ptr = NULL;
    m_o.via.array.size = 0;
}

private:
aux_unpacker2(const aux_unpacker2 & rhs) = delete;
aux_unpacker2 & operator = (const aux_unpacker2 & rhs) = delete;
private:
MSGPACK_OBJECT & m_o;
msgpack::zone & m_z;
};

} /* msgpack */

define MSG_PACKER_DEFINE(...) \

template <typename Packer> \
void msgpack_pack(Packer& pk) const \
{ \
    msgpack::aux_packer<Packer> tmp(pk); \
    tmp(__VA_ARGS__); \
} \
void msgpack_unpack(msgpack::object o) \
{ \
    msgpack::aux_unpacker1 tmp(o); \
    tmp(__VA_ARGS__); \
}\
template <typename MSGPACK_OBJECT> \
void msgpack_object(MSGPACK_OBJECT* o, msgpack::zone* z) const \
{ \
    msgpack::aux_unpacker2<MSGPACK_OBJECT> tmp(*o, *z); \
    tmp(__VA_ARGS__); \
}

Can not work with vector<double>(pack with STL string).

see the problem @:
stackoverflow.com/questions/21018336/why-msgpack-c-do-not-support-vectorfloat-or-vectordouble-in-my-case

I compile the piece of my code with:
'g++-4.7 demo.cpp -std=c++11 -lmsgpack', and got the runtime error below:

'terminate called after throwing an instance of 'msgpack::type_error'
what(): std::bad_cast'

In short, I transform the packed sbuffer object to std::string(which I need). It works with double, int, vector, but can not work with vector, vector...

compilation of sample code fails

In sample code in readme.md, there are too few include statements.

Sample code start with below:

#include <msgpack.hpp>
#include <vector>

int main(void) {

But it should start with below as Quick start guide.

#include <msgpack.hpp>
#include <vector>
#include <string>
#include <iostream>

int main(void) {

And also, stating to compile c++ code with gcc as readme.md is ill.

gcc -lmsgpack myapp.c -o myapp

It should be:

g++ -lmsgpack myapp.c -o myapp

or

gcc -lmsgpack myapp.c -lstdc++ -o myapp

C++ unpacker leaks memory

From what I can tell by the comments in unpack.hpp, unpacker objects are meant to live for the lifetime of the input source. However, using it to implement MessagePack-RPC, it leaks memory.

Consider this example:

#include <msgpack.hpp>

int main(int argc, char **argv)
{
    // Build a buffer containing a sample msgpack-rpc message
    msgpack::sbuffer sbuf;
    msgpack::packer<msgpack::sbuffer> packer(&sbuf);
    packer.pack_array(4);
    packer.pack(0);
    packer.pack(123456);
    packer.pack(std::string("method"));
    packer.pack_array(2);
    packer.pack(std::string("strarg"));
    packer.pack(42);

    // Simulate receiving the message from the network
    msgpack::unpacker pac;
    while (1) {
        pac.reserve_buffer(sbuf.size());
        memcpy(pac.buffer(), sbuf.data(), sbuf.size());
        pac.buffer_consumed(sbuf.size());

        msgpack::unpacked result;
        while (pac.next(&result)) {
            msgpack::object obj = result.get();
            result.zone().release();
        }
    }
    return 0;
}

Running this will leak until OOM killer dispatches the process.

Am I using the class wrong? The only documentation I can find (comments) say this is how it should be used.

serializing an object with > 32 levels of nesting fails to work

I created this issue initially in msgpack-node (see msgpack/msgpack-node#8) but reopening here for wider discussion since the behavior is driven by the c library.

packing/unpacking have different behaviors when you exceed 32 levels of nesting in a map object... the object will pack fine, but fail to unpack.

The issue is the stack size is defined as 32 in /deps/msgpack/msgpack/unpack_define.h:

#ifndef MSGPACK_EMBED_STACK_SIZE
#define MSGPACK_EMBED_STACK_SIZE 32
#endif

Two requests:

a) Can we get this bumped up to higher default, 32 seems rather low?

b) Can we make the behavior consistent for both packing and unpacking? As described in the example for the issue above, packing will work correctly for > 32 levels of nesting, but unpacking leads to a parse error. At the very least, the behavior should be consistent (anything I pack without errors I should be able to unpack)

error while configuring without c++ support

./configure --disable-cxx --enable-static --disable-shared

configure: error: conditional "am__fastdepCXX" was never defined.
Usually this means the macro was only invoked conditionally.

use-after-free bug in zone.c

See my comments in the code below. Very surprising that this hasn't caused widespread crashes (and hasn't been caught before this).

static inline void clear_chunk_list(msgpack_zone_chunk_list* cl, size_t chunk_size)
{
    msgpack_zone_chunk* c = cl->head;         <============ first value of c is the incoming cl->head
    while(true) {
        msgpack_zone_chunk* n = c->next;
        if(n != NULL) {
            free(c);             //                 <===== original cl->head gets free'd here
            c = n;
        } else {
            break;
        }
    }
    cl->head->next = NULL;  //               <=====  same cl->head pointer is being deref'd here.
    cl->free = chunk_size;
    cl->ptr  = ((char*)cl->head) + sizeof(msgpack_zone_chunk);
}

no manpages / docs

manpages aren't all that difficult to do anymore, especially with tools like md2man and they provide an excellent form of documentation. I would write them myself, except that I don't know what a good amount of the functions do and can't find any documentation.

If some documentation on each of the functions (in C) exist somewhere, I would love to begin writing manpages on a separate branch (maybe man ๐Ÿ˜ธ ).

Better support for str/bin discrepancy

The new spec introduces some new types for str/bin. Support for these types can be improved.

  • There should be some way from the C API to deliberately emit a str vs a bin type. Currently the only pack function in userspace is msg_pack_raw, which as best as I can tell always emits str.
  • There should be some way to detect whether a msgpack_object was decoded from a str or a bin type. Currently both str and bin formats get decoded to an object with object.type==MSGPACK_OBJECT_RAW.

The first problem is simple enough. As to the second problem, the decode logic seems be buried in macro hell, and I'm not really confident enough that I understand the existing flow to make an attempt.

This bug is important enough that in the absence of a simple solution I will end up writing a new library in the target language.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.