Code Monkey home page Code Monkey logo

node-blend's Introduction

node-blend

This module can re-encode one or more images of the same size. It supports stiching multiple images together into a single image, alpha-compositing, color quantization, and various compression options to produce highly optimized output.

Build Status

Usage

var blend = require('blend');
var image1; // Contains a compressed PNG image as a buffer.
var image2;
blend([ image1, image2 ], function(err, result) {
    // result contains the blended result image compressed as PNG.
});

blend([ image1, image2 ], {
    format: 'jpeg',
    quality: 90
}, function(err, result) {
    // result contains the blended result image compressed as JPEG.
});

blend([
    { buffer: images[1], x: 20, y: 10 },
    { buffer: images[0], x: -30, y: 90 }
], {
    width: 256,
    height: 256
}, function(err, data) {
    // result contains the blended result image compressed as JPEG.
});

Options

The first argument is an array of either Buffers containing image data, or Objects with the following potential properties:

  • buffer: Buffer containing image data
  • x: image offset in the X dimension
  • y: image offset in the Y dimension

The second argument is an optional options Object with the following potential properties:

  • format: jpeg, png, or webp
  • quality: integer indicating the quality of the final image. Meaning and range differs per format. For JPEG and webp the range is from 0-100. It defaults to 80. The lower the number the lower image quality and smaller the final image size. For PNG range is from 2-256. It means the # of colors to reduce the image to using. The lower the number the lower image quality and smaller the final image size.
  • width: integer, default 0: final width of blended image. If options provided with no width value it will default to 0
  • height: integer, default 0: final width of blended image. If options provided with no height value it will default to 0
  • reencode: boolean, default false
  • matte: when alpha is used this is the color to initialize the buffer to (reencode will be set to true automatically when a matte is supplied)
  • compression: level of compression to use when format is png. The higher value indicates higher compression and implies slower encodeing speeds. The lower value indicates faster encoding but larger final images. Default is 6. If the encoder is libpng then the valid range is between 1 and 9. If the encoder is miniz then the valid range is between 1 and 10. The reason for this difference is that miniz has a special "UBER" compression mode that tries to be extremely small at the potential cost of being extremely slow.
  • palette: pass a blend.Palette object to be used to reduced PNG images to a fixed array of colors
  • mode: octree or hextree - the PNG quantization method to use, from Mapnik: https://github.com/mapnik/mapnik/wiki/OutputFormats. Octree only support a few alpha levels, but is faster while Hextree supports many alpha levels.
  • encoder: libpng or miniz - the PNG encoder to use. libpng is standard while miniz is experimental but faster.

Installation

npm install @mapbox/blend@latest

Development

To run tests for this module, run npm install --dev to install the testing framework, then npm test. Tests require Imagemagick for its compare utility.

node-blend's People

Contributors

flippmoke avatar ian29 avatar kkaefer avatar kyleamathews avatar millzpaugh avatar springmeyer avatar taketime avatar tmcw avatar wilhelmberg avatar willwhite avatar wrynearson avatar yhahn avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

node-blend's Issues

valgrind warning in png reader

This looks to me like a png or zlib bug, but logging here until that it totally clear:

==8144== Conditional jump or move depends on uninitialised value(s)
==8144==    at 0x73164E0: inflateReset2 (in /lib/x86_64-linux-gnu/libz.so.1.2.3.4)
==8144==    by 0x73165D8: inflateInit2_ (in /lib/x86_64-linux-gnu/libz.so.1.2.3.4)
==8144==    by 0x6EA60D3: png_create_read_struct_2 (in /lib/x86_64-linux-gnu/libpng12.so.0.46.0)
==8144==    by 0x6EA6296: png_create_read_struct (in /lib/x86_64-linux-gnu/libpng12.so.0.46.0)
==8144==    by 0x6C5A94B: PNGImageReader::PNGImageReader(unsigned char*, unsigned long) (reader.cpp:9)
==8144==    by 0x6C5B873: ImageReader::create(unsigned char*, unsigned long) (reader.cpp:210)
==8144==    by 0x6C6ABBE: Work_Blend(uv_work_s*) (blend.cpp:273)
==8144==    by 0x5C3E4B: ??? (in /usr/bin/nodejs)
==8144==    by 0x5A57E99: start_thread (pthread_create.c:308)
==8144==    by 0x5D60CBC: clone (clone.S:112)
==8144== 

Also, this patch did not fix:

diff --git a/src/reader.cpp b/src/reader.cpp
index 5b37f1a..8efbd63 100644
--- a/src/reader.cpp
+++ b/src/reader.cpp
@@ -5,8 +5,10 @@
 PNGImageReader::PNGImageReader(unsigned char* src, size_t len) :
     ImageReader(src, len), depth(0), color(-1), interlace(PNG_INTERLACE_NONE) {
     // Decode PNG header.
-    png = png_create_read_struct(PNG_LIBPNG_VER_STRING, (png_voidp)this, errorHandler, warningHandler);
+    //png = png_create_read_struct(PNG_LIBPNG_VER_STRING, (png_voidp)this, errorHandler, warningHandler);
+    png = png_create_read_struct(PNG_LIBPNG_VER_STRING,0,0,0);
     assert(png);
+    png_set_error_fn(png, (png_voidp)this, errorHandler, warningHandler);
     info = png_create_info_struct(png);
     assert(info);

make hextree usage optional

hextree is now default on for images with alpha. We should make this an option. When only binary alpha is needed octree is faster.

supporting libjpeg-turbo

libjpeg-turbo is able to emulate different versions of libjpeg. node-blend requires at least the libjpef v8 interface (which added in memory source reading). So if libjpeg-turbo is built with the --with-jpeg8 flag then node-blend can be built against it. If not the libjpeg-turbo will, by default, report that it is jpeg6 and node-blends configure checks will catch this and bail.

Port to Windows or shift to node-mapnik?

tm2 now depends on node-blend for @camilleanne's awesome printing support. So we either need windows packages for node-blend, or we need to port the last remaining bit to node-mapnik that node-blend does and node-mapnik does not: offsets for stitching images. @camilleanne - for context: node-blend is basically Mapnik bits extracted into a standalone module which made sense at the time because we did not have Mapnik around to depend on. That is no longer the case.

Anyway, I'm feeling like the latter (porting stitching offsets) would be the least amount of work and a nice api to have in node-mapnik.

/cc @mikemorris @yhahn - thoughts?

webp image differences between versions

Looks like ubuntu precise has webp 0.1.3 while osx homebrew brings in 0.3.0 so naturally we have some pretty significant image differences. Logging here as documentation for test changes to work around this

Stacked blending

Currently one can pass an array of blend operations at once.
However, there may be the case where the input images are asynchronously processed and
the finished images should be already blended in while still processing the other images.

Fix miniz compiler warnings

These look serious (seen on ubuntu raring) and may be the cause of some incorrect behavior (#13):

../src/miniz.c: In function ‘void tdefl_find_match(tdefl_compressor*, mz_uint, mz_uint, mz_uint, mz_uint*, mz_uint*)’:
../src/miniz.c:2254:42: warning: enumeral and non-enumeral type in conditional expression [enabled by default]
../src/miniz.c: In function ‘mz_bool tdefl_compress_fast(tdefl_compressor*)’:
../src/miniz.c:2382:19: warning: enumeral and non-enumeral type in conditional expression [enabled by default]
../src/miniz.c:2410:19: warning: enumeral and non-enumeral type in conditional expression [enabled by default]
../src/miniz.c: In function ‘mz_bool tdefl_compress_normal(tdefl_compressor*)’:
../src/miniz.c:2564:22: warning: enumeral and non-enumeral type in conditional expression [enabled by default]
../src/miniz.c: In function ‘void* tdefl_write_image_to_png_file_in_memory(const void*, int, int, int, size_t*)’:
../src/miniz.c:2781:100: warning: narrowing conversion of ‘"\000\000\004\002\006"[num_chans]’ from ‘const char’ to ‘mz_uint8 {aka unsigned char}’ inside { } is ill-formed in C++11 [-Wnarrowing]
In file included from ../src/png_io.hpp:32:0,
                 from ../src/blend.cpp:4:
../src/miniz_png.hpp: In member function ‘void MiniZ::PNGWriter::finishChunk(size_t)’:
../src/miniz_png.hpp:87:69: warning: narrowing conversion of ‘(crc >> 24)’ from ‘mz_uint32 {aka unsigned int}’ to ‘mz_uint8 {aka unsigned char}’ inside { } is ill-formed in C++11 [-Wnarrowing]
../src/miniz_png.hpp:87:69: warning: narrowing conversion of ‘(crc >> 16)’ from ‘mz_uint32 {aka unsigned int}’ to ‘mz_uint8 {aka unsigned char}’ inside { } is ill-formed in C++11 [-Wnarrowing]
../src/miniz_png.hpp:87:69: warning: narrowing conversion of ‘(crc >> 8)’ from ‘mz_uint32 {aka unsigned int}’ to ‘mz_uint8 {aka unsigned char}’ inside { } is ill-formed in C++11 [-Wnarrowing]
../src/miniz_png.hpp:87:69: warning: narrowing conversion of ‘crc’ from ‘mz_uint32 {aka unsigned int}’ to ‘mz_uint8 {aka unsigned char}’ inside { } is ill-formed in C++11 [-Wnarrowing]
In file included from ../src/miniz_png.hpp:17:0,
                 from ../src/png_io.hpp:32,
                 from ../src/blend.cpp:4:
../src/miniz.c: At global scope:
../src/miniz.c:2440:28: warning: always_inline function might not be inlinable [-Wattributes]
../src/miniz.c:2432:28: warning: always_inline function might not be inlinable [-Wattributes]
../src/miniz.c:2230:28: warning: always_inline function might not be inlinable [-Wattributes]
../src/miniz.c:977:14: warning: ‘void* def_realloc_func(void*, void*, size_t, size_t)’ defined but not used [-Wunused-function]

CERT_UNTRUSTED build failure: can't get binary from s3

node-pre-gyp http GET https://mapbox-node-binary.s3.amazonaws.com/mapnik/v3.1.2/node-v11-linux-x64.tar.gz
node-pre-gyp http CERT_UNTRUSTED (falling back to source compile with node-gyp) 
node-pre-gyp verb command build [ 'rebuild' ]

incorrect rgb -> hsl conversion

Based on http://stackoverflow.com/a/9493060 our rgb2hsl conversion has a bug which is fixed by:

--- a/src/tint.hpp
+++ b/src/tint.hpp
@@ -15,7 +15,7 @@ static void rgb2hsl(unsigned red, unsigned green, unsigned blue,
     double gamma = max + min;
     h = 0.0, s = 0.0, l = gamma / 2.0;
     if (delta > 0.0) {
-        s = l > 0.5 ? delta / (2.0 - gamma) : delta / gamma;
+        s = l > 0.5 ? delta / (2.0 - delta) : delta / gamma;
         if (max == r && max != g) h = (g - b) / delta + (g < b ? 6.0 : 0.0);
         if (max == g && max != b) h = (b - r) / delta + 2.0;
         if (max == b && max != r) h = (r - g) / delta + 4.0;

This code was originally ported from
https://github.com/mapbox/node-tint/blob/6a966803dfa4a6fcafc760a337311ccc127496cc/lib/tint.js#L47 which looks to have this issue originally: cutting-room-floor/node-tint@2fa2054

With this change, this input image:

base_raw

becomes:

fixed

Instead of:

out

Testcase:

var blend = require('.');
var fs = require('fs');
var image = fs.readFileSync('/Users/dane/Desktop/base_raw.jpg');
var tint = blend.parseTintString('0x1;0x1;0x.5;0x1');
blend([ {buffer:image,tint:tint} ], function(err, result) {
    fs.writeFileSync('out.png',result);
});

0.10.0 package broken

Tonight we started getting the following when running npm install blend:

npm ERR! Error: shasum check failed for /root/tmp/npm-13946-tJlkDzte/1397172714912-0.743328359676525/tmp.tgz
npm ERR! Expected: 98b4c9a90514c5cd970c16f7efe806aab1906799
npm ERR! Actual:   ae16adc5474fb70de8ea4c368b8e78f73079a066
npm ERR! From:     https://registry.npmjs.org/blend/-/blend-0.10.0.tgz
npm ERR!     at /usr/local/lib/node_modules/npm/node_modules/sha/index.js:38:8
npm ERR!     at ReadStream.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/sha/index.js:85:7)
npm ERR!     at ReadStream.EventEmitter.emit (events.js:117:20)
npm ERR!     at _stream_readable.js:920:16
npm ERR!     at process._tickCallback (node.js:415:13)
npm ERR! If you need help, you may report this *entire* log,
npm ERR! including the npm and node versions, at:
npm ERR!     <http://github.com/npm/npm/issues>

npm ERR! System Linux 3.2.0-58-virtual
npm ERR! command "/usr/local/bin/node" "/usr/local/bin/npm" "install" "-g"
npm ERR! cwd /usr/local/src/tilestream-pro
npm ERR! node -v v0.10.26
npm ERR! npm -v 1.4.3
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR!     /usr/local/src/tilestream-pro/npm-debug.log
npm ERR! not ok code 0

Thinking that the package became corrupt in the registry, we unpublished the package, but when we went to republish npm wouldn't allow it. Now the release is stuck in a pretty inconsistent state where npm info reports it exists but it actually doesn't.

signed integer overflow issues

clang++'s -fsanitize=integer tool (http://clang.llvm.org/docs/UsersManual.html) has highlighted two places where unsigned ints are used but numbers can go negative.

Once case is the composite code: https://github.com/developmentseed/node-blend/blob/master/src/blend.cpp#L343-L351 where clang outputs things like:

../src/blend.cpp:324:27: runtime error: unsigned integer overflow: 2304 - 60690 cannot be represented in type 'unsigned int'
../src/blend.cpp:324:33: runtime error: unsigned integer overflow: 4294908910 * 139 cannot be represented in type 'unsigned int'
../src/blend.cpp:324:38: runtime error: unsigned integer overflow: 4286851642 + 15536640 cannot be represented in type 'unsigned int

And another is in miniz:

    ◦ should support miniz MZ_UBER_COMPRESSION: ../src/miniz.c:964:35: runtime error: unsigned integer overflow: 0 - 1 cannot be represented in type 'size_t' (aka 'unsigned long')
../src/miniz.c:1929:7: runtime error: unsigned integer overflow: 0 - 1 cannot be represented in type 'mz_uint' (aka 'unsigned int')
../src/miniz.c:1929:7: runtime error: unsigned integer overflow: 0 - 1 cannot be represented in type 'mz_uint' (aka 'unsigned int')
../src/miniz.c:964:35: runtime error: unsigned integer overflow: 0 - 1 cannot be represented in type 'size_t' (aka 'unsigned long')
../src/miniz.c:964:35: runtime error: unsigned integer overflow: 0 - 1 cannot be represented in type 'size_t' (aka 'unsigned long')

Tested with ./configure --debug and this patch:

diff --git a/wscript b/wscript
index e5c6c22..176a4eb 100644
--- a/wscript
+++ b/wscript
@@ -191,7 +191,8 @@ def build(bld):
         obj.cxxflags = ["-O0","-g", "-DDEBUG", "-D_FILE_OFFSET_BITS=64",
                         "-D_LARGEFILE_SOURCE", "-Wall", "-Wno-unused-value",
                         "-Wno-unused-function", "-mfpmath=sse", "-march=core2",
-                        "-funroll-loops"]
+                        "-funroll-loops","-fsanitize=integer"]
+        obj.linkflags = ['/opt/llvm2/lib/clang/3.3/lib/darwin/libclang_rt.ubsan_osx.a']
     else:
         obj.cxxflags = ["-O3","-DNDEBUG", "-D_FILE_OFFSET_BITS=64",
                         "-D_LARGEFILE_SOURCE", "-Wall", "-Wno-unused-value",

Support transparency modes

Currently node-blend defaults to trans_mode = -1 which means that the best possible transparency level is attempted for both the octree and hextree encoders. In cases where we do not need transparency in the final image output, moderate size and speed benefits should be able to be gained from being able to tweak this setting.

optimizing tinting

node-blend now supports tinting on rgb pixels using rgb->hsl->rgb conversions.

The code can likely be made faster. Ideas are:

  • NONE MORE FOUND: Look for more obvious slow parts of the hsl->rgb code to optimize (like replacing fmod).
  • DONE in f9b542b: Try caching results of hsl->rgb transforms, so that if an image has repeated colors the conversion can be skipped

mismatched malloc/delete in palettes branch

Not sure the level of severity of this, but valgrind reports it correctly, that we are doing a malloc that is not matched with a free but rather with a delete().

==6693== Mismatched free() / delete / delete []
==6693==    at 0x4C2A4BC: operator delete(void*) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==6693==    by 0x6C708B5: ImageData<unsigned int>::~ImageData() (image_data.hpp:131)
==6693==    by 0x6C6B465: Work_Blend(uv_work_s*) (blend.cpp:366)
==6693==    by 0x5C3E4B: ??? (in /usr/bin/nodejs)
==6693==    by 0x5A57E99: start_thread (pthread_create.c:308)
==6693==    by 0x5D60CBC: clone (clone.S:112)
==6693==  Address 0x7924040 is 0 bytes inside a block of size 1,680,000 alloc'd
==6693==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==6693==    by 0x6C6B29A: Work_Blend(uv_work_s*) (blend.cpp:341)
==6693==    by 0x5C3E4B: ??? (in /usr/bin/nodejs)
==6693==    by 0x5A57E99: start_thread (pthread_create.c:308)
==6693==    by 0x5D60CBC: clone (clone.S:112)

dense_hash_map should use unsigned as key since white rgba will overflow int

max int is usually:

1 <<32
4294967296

But white will overflow this:

255 | (255<<8) | (255<<16) | (255 << 24)
4294967295

Causing the dense_hash_map to crash (in release mode) or assert (in debug mode) since the overflow will likely (with clang++ at least) convert to -1:

Assertion failed: ((!settings.use_empty() || !equals(key, get_key(val_info.emptyval))) && "Inserting the empty key"), function find_or_insert, file ../deps/sparsehash/internal/densehashtable.h, line 985.
Abort trap: 6

Clarify license

The LICENSE file is BSDish, but most files in src have an LGPL header.

HiDPI support

This protocol doesn't seem to support retina requests.
Below a partial configuration for tessera. With something like http://127.0.0.1:8089/blend/11/326/[email protected] we get a 512x512 tile (good) however the blend data is made on the 256x256 top left part.
Note:
http://staging.tile.stamen.com/toner-background/11/326/[email protected]
returns an expected 512x512 tile in good resolution.

{
       "/blend": {
                "source": {
                        "protocol": "blend:",
                        "query": {
                                "layers": [{
                                        "source": "tilejson+http://staging.tile.stamen.com/toner-background/index.json",
                                        "opacity": 0.5,
                                        "filters": "color-to-alpha(#008800)",
                                        "offset": [5, -5]
                                }, {
                                        "source": "tilejson+http://staging.tile.stamen.com/toner-lines/index.json",
                                        "filters": "invert agg-stack-blur(1,1)"
                                }, {
                                        "source": "tilejson+http://staging.tile.stamen.com/toner-labels/index.json",
                                        "comp-op": "over"
                                }],
                                "format": "png32:z=1"
                        }
                }
        }
}

Failing to install package node-pre

Trying to install this package for the first time and I am having problems, does anyone know whats up?
I'm on a window 10 machine and I have already installed windows build tools before. My node version is v8.9.1 and my NPM version is 6.4.1. Here's my error message below, I left out some of the stack trace but I can post more if needed

node-pre-gyp ERR! Tried to download(403): https://mapbox-node-binary.s3.amazonaws.com/mapnik/v3.5.14/Release/node-v57-win32-x64.tar.gz
node-pre-gyp ERR! Pre-built binaries not found for [email protected] and [email protected] (node-v57 ABI, unknown) (falling back to source compile with node-gyp)
'mapnik-config' is not recognized as an internal or external command,
operable program or batch file.
gyp: Call to 'mapnik-config --ldflags' returned exit status 1 while in binding.gyp. while trying to load binding.gyp
gyp ERR! configure error
gyp ERR! stack Error: gyp failed with exit code: 1
gyp ERR! stack at ChildProcess.onCpExit (C:\Users\Noah\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\lib\configure.js:345:16)
gyp ERR! stack at emitTwo (events.js:126:13)
gyp ERR! stack at ChildProcess.emit (events.js:214:7)
gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:198:12)
gyp ERR! System Windows_NT 10.0.17134
gyp ERR! command "C:\Program Files\nodejs\node.exe" "C:\Users\Noah\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js" "configure" "--fallback-to-build" "--module=C:\Users\Noah\Documents\Mosaic\node_modules\mapnik\lib\binding\node-v57-win32-x64\mapnik.node" "--module_name=mapnik" "--module_path=C:\Users\Noah\Documents\Mosaic\node_modules\mapnik\lib\binding\node-v57-win32-x64" "--python=C:\Users\Noah\.windows-build-tools\python27\python.exe"
gyp ERR! cwd C:\Users\Noah\Documents\Mosaic\node_modules\mapnik
gyp ERR! node -v v8.9.1
gyp ERR! node-gyp -v v3.8.0
gyp ERR! not ok

lightness values outside valid range

current parseTintString implementation turns 4c2d00 into:

{ h: [ 0.0986842105263158, 0.0986842105263158 ],
  s: [ 1, 1 ],
  l: [ -0.7019607843137254, 1 ] }

Notice the lightness value less that 0. This as caught by the validation newly added to mapnik (mapnik/mapnik@eaeccc3) when running the blendnik-js branch.

Differences in node-tint vs node-blend image results

While visually identical in nearly all cases, image magicks compare -metric PSNR gives differences commonly as high as 50-60.

This issue stands to document and explain this before overwriting the test results (that were originally ported directly fron node-tint).

test webp alpha support

There may be some bugs lurking. Webp alpha support is more advanced in Mapnik and we may need to port fixes back to node-blend.

parseTintString should be pickier

parseTintString("jkfldsjaklfdjsazzz") yields

{ h: [ NaN, NaN ] }

If a tint string is invalid, the returned value should be null - detecting nan'y objects is too much work to find errors.

mapnik-config: not found

During installation of blend package from npm, build fails with:

[...]
/bin/sh: 1: mapnik-config: not found
gyp: Call to 'mapnik-config --cflags' returned exit status 127 while in binding.gyp. while trying to load binding.gyp
gyp ERR! configure error
[...]

Ideally, the mapnik(-config) dependency is just stripped from this module.

avoid using std::ostringstream

The blend baton in the palettes branch (and soon master) now uses a std::ostringstream for the write buffer.

Ideally we should eventually factor this out, replacing it with some custom data structure that does not incur the overhead of std::locale locking/reference counting (i.e - is not a c++ formatted string class).

This should help performance in a noticeable way.

On OSX running benchmark/bench-stich.js on master gives 53.78078950198989 while palettes branch is slower (without using miniz) at 49.44620253164557

My wild guess at this point is that this slight speed regression is due to locale locking, but this needs a closer look. This ticket is a reminder.

Memory leaks in buffer usage

Using this testcase:

var fs = require('fs');
var blend = require('./lib/blend.node');

blend.blend([fs.readFileSync('test/fixture/1.png'),fs.readFileSync('test/fixture/2.png')], {
    width: 700,
    height: 600,
    quality: 256
}, function(err, data) {
    fs.writeFileSync('./out.png', data);
    console.log('done!');
});

On the palettes branch I get: https://gist.github.com/3990098

The odd one is:

==8475== 44,901 (56 direct, 44,845 indirect) bytes in 1 blocks are definitely lost in loss record 35 of 37
==8475==    at 0x4C2B1C7: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==8475==    by 0x57EBA5: node::Buffer::New(v8::Arguments const&) (in /usr/bin/nodejs)
==8475==    by 0x6CA9A6: ??? (in /usr/bin/nodejs)
==8475==    by 0x3A8B6406361: ???
==8475==    by 0x3A8B640C792: ???
==8475==    by 0x3A8B640CA49: ???
==8475==    by 0x3A8B6406275: ???
==8475==    by 0x6EE982: ??? (in /usr/bin/nodejs)
==8475==    by 0x6EED79: v8::internal::Execution::New(v8::internal::Handle<v8::internal::JSFunction>, int, v8::internal::Handle<v8::internal::Object>*, bool*) (in /usr/bin/nodejs)
==8475==    by 0x6B0F31: v8::Function::NewInstance(int, v8::Handle<v8::Value>*) const (in /usr/bin/nodejs)
==8475==    by 0x57ED3F: node::Buffer::New(char*, unsigned long) (in /usr/bin/nodejs)
==8475==    by 0x6C6B779: Work_AfterBlend(uv_work_s*) (blend.cpp:385)

minor test failures with subtle image differences on ubuntu precise (12.04)

master branch at 2a2a9f9. Node v0.8.11.

$ ldd lib/blend.node 
    linux-vdso.so.1 =>  (0x00007fff7d9ff000)
    libpng12.so.0 => /lib/x86_64-linux-gnu/libpng12.so.0 (0x00007f58badbd000)
    libjpeg.so.8 => /usr/lib/x86_64-linux-gnu/libjpeg.so.8 (0x00007f58bab6d000)
    libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f58ba86c000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f58ba656000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f58ba299000)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f58ba081000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f58b9d87000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f58bb21d000)

Seeing some minor test failures due to image differences. Here is one example:

And the test result:

ubuntu@ip-10-196-0-40:~/node-blend$ ./node_modules/.bin/mocha 

  ․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․

  ✖ 5 of 58 tests failed:

  1) JPEG writing should write a JPEG:
     Error: Images not equal (36.7957):
/home/ubuntu/node-blend/test/fixture/results/6.result.jpg
/home/ubuntu/node-blend/test/fixture/results/6.jpg
      at ChildProcess.exports.imageEqualsFile (/home/ubuntu/node-blend/test/support/utilities.js:38:27)
      at ChildProcess.EventEmitter.emit (events.js:96:17)
      at Process._handle.onexit (child_process.js:678:10)

  2) JPEG writing should write a JPEG with 50% quality:
     Error: Images not equal (37.947):
/home/ubuntu/node-blend/test/fixture/results/7.result.jpg
/home/ubuntu/node-blend/test/fixture/results/7.jpg
      at ChildProcess.exports.imageEqualsFile (/home/ubuntu/node-blend/test/support/utilities.js:38:27)
      at ChildProcess.EventEmitter.emit (events.js:96:17)
      at Process._handle.onexit (child_process.js:678:10)

  3) JPEG writing should write a PNG that is explicitly marked as such:
     Error: Images not equal (41.6011):
/home/ubuntu/node-blend/test/fixture/results/8.result.png
/home/ubuntu/node-blend/test/fixture/results/8.png
      at ChildProcess.exports.imageEqualsFile (/home/ubuntu/node-blend/test/support/utilities.js:38:27)
      at ChildProcess.EventEmitter.emit (events.js:96:17)
      at Process._handle.onexit (child_process.js:678:10)

  4) JPEG writing should write a PNG with an empty parameters hash:
     Error: Images not equal (41.6011):
/home/ubuntu/node-blend/test/fixture/results/9.result.png
/home/ubuntu/node-blend/test/fixture/results/9.png
      at ChildProcess.exports.imageEqualsFile (/home/ubuntu/node-blend/test/support/utilities.js:38:27)
      at ChildProcess.EventEmitter.emit (events.js:96:17)
      at Process._handle.onexit (child_process.js:678:10)

  5) JPEG writing should blend color JPEGs:
     Error: Images not equal (41.6011):
/home/ubuntu/node-blend/test/fixture/results/5.result.png
/home/ubuntu/node-blend/test/fixture/results/5.png
      at ChildProcess.exports.imageEqualsFile (/home/ubuntu/node-blend/test/support/utilities.js:38:27)
      at ChildProcess.EventEmitter.emit (events.js:96:17)
      at Process._handle.onexit (child_process.js:678:10)

support png strategies

node-blend does not currently expose, as an option, the strategy of PNG encoding. But this could be done. In my testing in Mapnik the default (Z_DEFAULT_STRATEGY) has always been the fastest, so this is a low priority item unless some specific use case arises.

undefined behavior in miniz implementation

When testing the extra clang/gcc flags to trap undefined behavior:

diff --git a/wscript b/wscript
index 3051a01..7f5d30b 100644
--- a/wscript
+++ b/wscript
@@ -161,7 +161,7 @@ def configure(conf):

 def build(bld):
     obj = bld.new_task_gen("cxx", "shlib", "node_addon")
-    obj.cxxflags = ["-O3", "-D_FILE_OFFSET_BITS=64", "-D_LARGEFILE_SOURCE", "-Wall", "-Wno-unused-value", "-Wno-unused-function", "-mfpmath=sse", 
+    obj.cxxflags = ["-O3","-fcatch-undefined-behavior","-ftrapv","-fwrapv", "-D_FILE_OFFSET_BITS=64", "-D_LARGEFILE_SOURCE", "-Wall", "-Wno-unused
         "-funroll-loops", "-fomit-frame-pointer"]
     obj.target = TARGET
     obj.source = ["src/reader.cpp", "src/blend.cpp", "src/palette.cpp"]

Running node benchmark/bench-stitch.js hangs on OSX which is a pretty strong indication that undefined behavior is present (and the threadpool of node is preventing clang from reporting it leading to a hang):

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.