The MessagePack extension allows DuckDB to directly read data from files storing MessagePack map values.

This is just a toy project. Do not use for production systems.

• DuckDB version 0.10.0
• msgpack for C++ version 6.0.0 or above

export CMAKE_BUILD_PARALLEL_LEVEL=6 # parallelize build
# if you use vcpkg
export VCPKG_TOOLCHAIN_PATH="/path/to/your/vcpkg/installation"
# if you installed vcpkg on the same directory as this extension:
# export VCPKG_TOOLCHAIN_PATH=`pwd`/vcpkg/scripts/buildsystems/vcpkg.cmake

make (release/debug)

msgpack_extension is already linked into the built duckdb binary. You can use the extension without load.

./build/release/duckdb

Run DuckDB CLI:

duckdb -unsigned  # allow unsigned extensions

Then, load the extension:

LOAD 'build/release/extension/msgpack_maps/msgpack_maps.duckdb_extension';

msgpack files can be read by read_msgpack function. It supports only scan of map values.

select * from read_msgpack('data/nation.msgpack', columns={n_nationkey: 'INTEGER', n_name: 'VARCHAR', n_regionkey: 'SMALLINT', n_comment: 'VARCHAR'});