/index.html and /examples/ndvi/index.html give out errors on running (Safari, Chrome, Firefox on Mac)

Am I doing something wrong since I believe nobody else is getting these errors?

On Safari [Mac]

/index.html

The following pops up every time a module is called

[Error] IndexSizeError (DOM Exception 1): The index is not in the allowed range.
getImageData (image-sequencer.js:184439)
draw (image-sequencer.js:184439)
output (image-sequencer.js:184203)
(anonymous function) (image-sequencer.js:184551)
emit (image-sequencer.js:26578)
(anonymous function) (image-sequencer.js:12768)
run (image-sequencer.js:163900)
drainQueue (image-sequencer.js:163870)

The following error comes up only when "Add Step is clicked"

[Error] TypeError: undefined is not an object (evaluating 'options.el.html')
draw (image-sequencer.js:184409)
run (image-sequencer.js:184221)
(anonymous function) (index.html:78)
dispatch (image-sequencer.js:60435)
handle (image-sequencer.js:60247)

/examples/ndvi/index.html

The following pops up every time "Add step is clicked"

[Error] TypeError: undefined is not an object (evaluating 'options.el.html')
draw (image-sequencer.js:184409)
run (image-sequencer.js:184221)
(anonymous function) (index.html:76)
dispatch (image-sequencer.js:60435)
handle (image-sequencer.js:60247)

On Chrome [Mac]

Both files

On clicking "Add Step"

Uncaught TypeError: Cannot read property 'html' of undefined
at Object.draw (image-sequencer.js:184409)
at Object.run (image-sequencer.js:184221)
at HTMLButtonElement. (index.html:76)
at HTMLButtonElement.dispatch (image-sequencer.js:60435)
at HTMLButtonElement.elemData.handle (image-sequencer.js:60247)

On Firefox [Mac]

Both files

On clicking "Add Step"

ReferenceError: event is not defined image-sequencer.js:184383:11
onImage/reader.onload /dist/image-sequencer.js:184383:11

If it's possible, can you check if you can break out any of your longer functions into sub files? If you need shared scope, maybe you'll have to pass them in via the function parameters, so they'll be less inter-referential.

But this will make your code into smaller, more independent and easier (one thing per function) parts. So I think it's worth it in some cases!

Image Sequencer requires DataURI input when in Browser mode.
I think we should allow local images and AJAX images via XHR Requests too.

@jywarren you were once suggesting changes in the UI module. I thought upon them, and find them really interesting. Also, the current system has forward compatibility issues (Adding new features to it is really complicated. The code quality isn't very good). I have been working on the new model, with a much cleaner syntax and code and working on it all day. Will create a PR when ready.

Some takeaway features are:

• User defined variables : Users can define a variable for any step. This can be used to store the reference of an HTML Element, for further use in the other events (Like you suggested earlier) .

      sequencer.setUI(
          onSetup: function(step){
              step.image = document.createElement('img'); // create a user defined variable step.image
              step.image.setAttribute( 'id', 'IS-'+step.ID ); // access the default variable step.ID
              ... // append to DOM and other stuff
          },
          onComplete: function(step){
              step.src = step.output;
          },
          onRemove: function(step){
              step.remove();
          },
      );

• Unified variables : Earlier there were too many unnecessary variables and objects in the picture : options, identity, etc. Now all come under the same banner steps. This brings in clarity and helps maintain code, and is good for new contributors!
• Easier to understand and more User Friendly.

I am sorry, the "Bootstrapping Phase" doesn't seem to end very soon, major functionality changes happening every day... :(

Now that we have a UI system, I'd like to take the "starter module" idea from the previous demo, and start by plugging it into the new sequencer code.

This is the first step in getting a demo that runs on the new code. But do you think we should package up the selector more than including it in the demo code? I had originally seen it as a kind of "special module" but if it really doesn't fit into the architecture, we can just provide it in something like demo/initial-image.js

Check image similarity using image-diff

Thinking ahead again (so, lower priority) but it'd be great to integrate the webgl-distort library:

https://github.com/jywarren/webgl-distort

This, like fisheyegl, uses WebGL canvas.

It probably needs to be packaged up for compatibility with require or something, but it is also on NPM:

https://www.npmjs.com/package/webgl-distort

As described in #6, in comments, after merging #6, I still got:

Uncaught TypeError: Cannot read property 'src' of undefined
    at PixelManipulation (image-sequencer.js:184462)

on this line: https://github.com/publiclab/image-sequencer/blob/master/src/modules/PixelManipulation.js#L17

when adding a new step via the HTML ui. But then when I drop a new image into the image-select UI, all steps run correctly.

This may have to do with sequencer.run()

We should allow users on NodeJS to save output to actual binary files!
Agree?
Some gitignored directory like ./output/ could be used for the same

Closed #1 as it is a bit out of date now, but I still would like tests which incorporate a "before" and "after" image, as attempted in this code:

https://github.com/jywarren/image-sequencer/blob/3d43eceff9a51ed2f85c3ce42277017c7890e435/test/image-sequencer.js#L31-L59

At simplest, we could perform an operation, then reverse it -- like, for example, inverting and inverting again -- and then confirm that the image is the same after the test, perhaps using:

• https://www.npmjs.com/package/blink-diff
• smaller and faster: https://www.npmjs.com/package/@schornio/pixelmatch

Or even as an initial step, do an operation on an image and expect() that it's not the same -- just confirm that /something/ happened. Then move on to more complex tests of actual test images.

If this is in another PR, please link it here, thanks!

Some modules might require the filetype.
I hadn't noticed that options.format is undefined, so:

options.format = options.format || "jpg";

always results in options.format to be equal to jpg.
Also, we can't only rely on extension of the file. For example, a jpg file renamed as png would work perfectly fine otherwise, but when we set the src for img, we do:

img.src = 'data:image/' + options.format + ';base64,' + buffer.read().toString();

so this might result in a faulty image.
Should we import the module file-type?

How addSteps, removeSteps, etc. work currently :
If no image is specified in the addSteps or removeSteps arguments, then the operation is performed on all images.

Recently we allowed chaining of methods which enables us to do this:

    sequencer.loadImage('image.jpg').addSteps('ndvi-red').run()

As you can see, no image is specified with addSteps. This means that ndvi-red will be added to ALL images present in the sequencer.

What should be done:

• If chained, apply the steps only to the image which is being chained by storing which steps are being chained and the image associated in the object returned by these methods?

or

• Force the user to specify the image.

Still traveling so i'll get to your other issues when i can, but I wanted to suggest that persisted data from each module could be stored in a namespaced hash, such as:

metadata = {
'ndvi-red': { ... },
'exif': { ... }
}

That object would be accessible to each module, and the UI functions as well. Haven't thought through how to pass it but perhaps it's just a property of each image object that the sequencer stores? Any thoughts?

When a step is added to the sequencer, It gets pushed into steps[], and then the code runs through ALL the steps, running the modules again and again.
I think we should change that. This is making the program unnecessarily slow.

Shouldn't we change this? What do you think, @jywarren ?

The Issue

Now PixelManipulation works perfectly fine for JPEG images. Issues come up when we use PNG/GIF images. I am currently using a petty fix so that this works for all images. This is what I do in PixelManipulation.js :

options.format = "jpeg";

I am hardcoding the format to "jpeg" whenever a module using PixelManipulation is used.
So this is working for all images. But as a result, we can't work with images having an alpha channel.

This is what is happening: (Have spent about 2 days on this)

• savePixels() generates a readable stream -- r
• base64-stream generates a writable stream -- w
• r.pipe(w) pipes r to w
• the event listener 'finished' is attached to the pipe which is responsible for converting the image data to a DataURI.

In JPEG images this process works perfectly fine. But for PNG/GIF, the 'finished' event never gets triggered (I've tested this.)

Surprisingly, the same code worked for a 16x16 pixel PNG image.

Possible Explanation(s)

I am aware that save-pixels uses different parsers for PNG, JPEG, GIF. This may be causing some issue. But also, The code works fine when the writable stream is as follows

var w = require('fs').createWriteStream('output.png')

instead of:

var w = base64.encode();

So this may be an issue with the base64-stream.

Overview

• Works fine for JPEG
• Issue saving PNG or GIF images
• Temporary fix applied : convert all images to JPEG
• Implication : We can't work with alpha channel images

Another Fix (not a good idea, didn't do this)

Create a temporary binary PNG/GIF file using fs, convert it into a DataURI using urify and then destroy the temporary binary. Didn't do this because this won't work for Browsers.

Thanks to #73, GIFs are fully supported by PixelManipulation!

But It doesn't really work as expected. I inverted the following GIF using IS :

This is the result:

It doesn't apply to all frames, I guess. Also, now the GIF doesn't play infinitely. It just plays once.
(Try opening the image in a browser tab and then Cmd+Shift+R or Ctrl+Shift+R)

Uh oh, looks like this broke the build... Any idea why? #30

https://travis-ci.org/publiclab/image-sequencer/builds/248752455?utm_source=email&utm_medium=notification

I know, we've had this discussion about node-canvas earlier, but since quite a lot of algorithms are based on canvas, Do you think we should include node-canvas in image sequencer anyway?

in the docs, we see how to use run() but not what it returns -- i'm assuming it's the final image state, right? Just good to be explicit, thanks!

Original

Image Sequencer Output

Full Image

Infragram Output

Full Image

Is the algorithm we are using for ndvi-red correct?

    function changePixel(r, g, b, a) {
      var ndvi = 255 * (b - r) / (1.00 * b + r);
      return [ndvi, ndvi, ndvi, a];
    }

Now that Image Sequencer has been published to npm, I think the README.md should only consist of Usage instructions and all the development notes should be in CONTRIBUTING.md.
Am I right in thinking so?
Also because README.md has become very long over the time.

Should README.md have a small navigation or "jump to..." section?

we can use PixelManipulation, and copy this directly from Infragram.org (https://infragram.org/sandbox/):

https://github.com/publiclab/infragram-js/blob/master/public/infragram.js#L88-L114

that method returns a new function which maps an incoming value (0-255) to an outgoing value, and can be run on each pixel.

Colormap functions are based on a nested array:

https://github.com/publiclab/infragram-js/blob/master/public/infragram.js#L212-L214

var c = segmented_colormap([[0, [0, 0, 255], [38, 195, 195]], [0.5, [0, 150, 0], [255, 255, 0]], [0.75, [255, 255, 0], [255, 50, 50]]]);
c(0) // => [0, 0, 255]

then we can make lots of different segmented_colormaps! Gradient colormaps too, but later; this is simpler for now.

Also the Fastie colormap:

colormap_fastie = segmented_colormap([[0, [255, 255, 255], [0, 0, 0]], [0.167, [0, 0, 0], [255, 255, 255]], [0.33, [255, 255, 255], [0, 0, 0]], [0.5, [0, 0, 0], [140, 140, 255]], [0.55, [140, 140, 255], [0, 255, 0]], [0.63, [0, 255, 0], [255, 255, 0]], [0.75, [255, 255, 0], [255, 0, 0]], [0.95, [255, 0, 0], [255, 0, 255]]]);
 ```

When a Colorbar module is applied, It Makes the entire page (HTML) slow. Also after adding a colorbar module, results of adding other modules is undesirable.

Should we implement a better alternative?

A module which crops images according to passed width and height values.

Thinking how we might take a set of parameters required and provide (in the module) a description of what they should be and their defaults -- for use either by programmers, or by the HTML interface when you'd be able to configure the step as a user, using text inputs or sliders or menus.

Thoughts on this?

This would have to use a CORS enabled tile source so as to allow canvas manipulation of images from another domain. But it'd be cool -- on the fly image processing on a web map.

http://martyjs.org/api/top-level-api/index.html#isBrowser

Current logic is at:

There are Cross Origin Resource Sharing Canvas errors on Safari, IE, Opera, Mobile based-browsers.

This is because they not implemented CORS enabled images yet. MDN Source
The entire code of Image Sequencer depends on CORS Enabled images.

CORS enabled images are supported only on Desktop based Chrome and Firefox
Other desktop based browsers and support by any mobile-based browser is questionable as of now.

Can we do something about this? Because not doing so would render "Image Sequencer" useless for a huge chunk of Desktop users and all Moblie users?

We decided to have logging. SO, what data should be logged? Where does the logged data go?
Does it remain accessible only for that session or should it get stored in a LOG file (for Node.js)?

I think the following should be logged:

• Module Name & Options
• Image details (dimensions,format)
• Success/Failure in generating image

Please let me know what all I am missing.
Thanks :-)

if each method loadImages(), addStep() etc returns the sequencer itself, we can chain, like:

sequencer.loadImages(image_name, image_src)
         .addStep(...)
         .run();

That's also a really succinct way to demonstrate the entire usage cycle of the library, and would be great to include in the README!

I found quite a few npm packages for the same:

• QRCode Reader (Uses Canvas)
• jsQR (Does Not Use Canvas)
• JS QR Code (Uses Canvas)

Since jsQR is a pure JS port of the best QR Code decoding library ZXing, and it does not use Canvas or WebGL, I think We should use that. Nad I shall go ahead with it's implementation.

Should we include grunt build as a build job?
anything else we should include? Once the PhantomJS tests are ready, we should include them too.

I was implementing FishEyeGL and noticed that we need a different folder structure for that.
FishEyeGL has shaders. So there have to be multiple files.
So instead of this:

.
└─ src
    └─ modules
        ├─ Crop.js
        ├─ NdviRed.js
        └─ FishEyeGL.js

Should we move into this:

.
└─ src
    └─ modules
        ├─ Crop
        │   └─ Module.js
        ├─ NdviRed
        │   └─ Module.js
        └─ FishEyeGL
            ├─ Module.js (Connects Image Sequencer to the external module - fisheyegl.js)
            ├─ fisheyegl.js
            └─ shaders
                ├─ fragment.glfs
                ├─ method.glfs
                └─ vertex.glfs

Benfits of this folder structure :

• Multiple files per module can be stored.
• Easier to port external modules - We can have the external module file and our own Module.js which will act as an API between the external module file and hence convert the external module into an ImageSequencer "Module"
• Easier to maintain long code for complex modules like "segmented-colormap" (I know, this isn't that complex, but modularization will make the code more readable.)

Aha, very interesting! If you repeat clicking too long, the image degrades; we can address this in a follow-up issue but it's really cool to see, actually! Must be decimal/integer issues.

Kind of fascinating actually I wonder if it's conversing values from decimal to integer and back over and over. We could actually test for this, to be honest, by running the invert 20 times.

As to a solution, I think there are non drifting decimal math implementations we could use from other libraries.

Also, don't you think that the steps after histogram should use the output from the step BEFORE histogram?
It doesn't quite make sense to apply a module on the "histogram" output.
So what we can do is, the histogram module shows the histogram of the image but passes on ITS INPUT as its output to OTHER modules.

Similarly for the QR Code module?

This would be really great to show in the README, too -- to demonstrate that it's possible to use this in an everyday commandline setting:

https://www.npmjs.com/package/commander

I think it'd have to accept some kind of command string, maybe JSON like:

imagesequencer --help
imagesequencer -s "crop:cropOptions,..." image.png

This may need more thought but it'd be a cool idea to have a section in the docs that's like From the commandline

I was working on #1. I was able to get a basic version running on node.
I was wondering if we could use callbacks/promises in the code?

Just reading through this again and trying to think as a newcomer --

Should we add loadImage() as well as loadImages()? I guess I feel the initial default assumption should be that the library works on a single image, and multiple images is a additional mode. That's sort of conceptually simpler too, as a starting point. The README could then say use loadImage(...) then have a section called For multiple images, use loadImages(). I feel this could be clearer, what do you think?

In .travis.yml :

language: node_js
node_js:
  - '4'
  - '5'
env:
  - CXX=g++-4.8

Are these the only Node versions we should be testing builds on? what about v6, v7, v8?
I run v7.4.0 on my computer.

➜  image-sequencer git:(documentation) node -v
v7.4.0

Hi, @ccpandhare - whats the latest?

I wanted to add this as a feature request down the line -- it's a good one to try to develop a test for:

https://github.com/jywarren/fisheyegl

could be a great module. I want to suggest a test that takes a checkerboard image and corrects fisheye, then diff-compares it with a correct checkerboard.

There is some problem with the savePixels part of the ImageManipulation.js with all images.
Some pixels don't get saved and also the saturation of the image, somehow, decreases.

Before:

After:

In fact, the "After" image is corrupted, those at the bottom are empty pixels and not white pixels.
The module used with ImageManipulation.js was DoNothingPix.js. This module just includes ImageManipulation.js without changing any pixels. This module was made specifically for this purpose.

I believe the number of empty pixels isn't proportional to image size. This is because larger images have smaller area of "empty" pixels. For example:
Before:

After:

One might be tempted to think that the "After" image of Sundar Pichai is perfectly fine, but upon close inspection, it comes out that there is a thin strip of empty pixels at the bottom, and also, the image is slightly desaturated.

Maybe this is a flaw with save-pixels module (the desaturation part).
We are surely missing something due to which we are gettng the blank strip of pixels.

I am not sure what we could put up on the gh-pages branch. As of now, ImageSequencer doesn't have a browser UI, although it does work on browser. So what should I update on the gh-pages branch? It won't be a very good demo, for sure...

low priority, but kind of interesting -- i think this is currently still waiting for the newly published npmjs module to update, but could/should work?

https://runkit.com/jywarren/image-sequencer

Nice to give people a Node demo sandbox to work with :-)

And confirm that the library runs the same in both nodejs and phantomjs -- this will be cool! You could do it in a separate file called browserSpec or something.

This is my plan for the summer. Please suggest any suggestions which you may deem fit @jywarren @ebarry. There is still a lot to be written here, will keep adding more points with time. This is by no means, exhaustive.
Link to my proposal

Phase 1 : Core functionality and JSON Input Support

In this phase I will build upon the core fuctions of Image Sequencer (addSteps, removeSteps, insertSteps, loadImage)

• Create function addSteps which accepts data in many formats (JSON and otherwise) as described in the development wiki (PR #24)
• Create the system to be used for handling images : Instead of choosing Sharp or Jimp or lwip, I stuck to plain old DataURIs in custom Image objects. As the mentioned modules have huge functionalities which are not relevant to Image Sequencer. (PR #24)
• Create the framework for using multiple images : Finally enables Image Sequencer to handle multiple images at once. (PR #24)
• Similarly, create removeSteps, insertSteps, loadImages and modify run (PR #24)
• Update Usage Instructions in Readme.md

Phase 2 : Unit Tests, Modularization, Unified Input Parsing

• Unit Tests for the methods addSteps, removeSteps, insertSteps, run (PR #28)
• Image Based Unit Tests : Double Inversion Test, NDVI Test (PR #30)
• Unified Input Parser : A Single Module to parse inputs in different formats - reduces source size and helps manage code. (PR #28)
• Modularization of large functions which were earlier a part of src/ImageSequencer.js (PR #28)

Phase 3 : Module Logs, custom I/O, headless testing, Chaining, replaceImage

• Headless Browser Testing using the tape-run module. (PR #55)
• Method Chaining (PR #44, #53)
• Added Invert Module (PR #30)
• replaceImage Created, and along with it, its gh-pages demo. (PR #35)
• Create a module log. This will store logs from various modules. Useful for debugging. (PR #37)
• Allow modules to interact with the user, accept and output data apart from images. (example: data of a QR Code)

Phase 4 : User Interface, Support external Images

• Create Module Interface (PR #67 ISS #85)
• Create the User Interface for node based and browser based implementations of Image Sequencer. (Across various PRs)
• local image support on Browser (PR #81)
• URL image support on Browser (PR #81)
• URL image support on Node.JS (PR #81)

Phase 5 : Additional New Features

• Added method .exportBinary() to export images as binary (NodeJS) (PR #90)
• Image testing using pixel-by-pixel comparison with a tolerance level (PR #89)
• Describing module requirements using an info.json file for every module. (PR #92)
• Implemented Complete demo (PR #92)
• Added Basic Command Line Interface (CLI) (PR #95)
• Publish demos to GH-Pages (PR #96)

Phase 6 : Module Implementation & NPM

This phase involves implementing the various modules as mentioned in the Readme.md and my project proposal.

• Publish to NPM
• FisheyeGl (PR #61)
• Crop (PR #56)
• Invert
• Segmented Colormap (PR #57)
• QR Code Module (PR #72)

Just to kick this off, i had some thoughts:

Each module should have a default interface, which perhaps just doesn't do anything. But we should be able to add an interface, and connect up:

• onAdd or onSetup - runs when the module is added - can be used to create/append HTML
• onTrigger or onDraw - runs when the module is run
• beforeDraw vs. afterDraw? These could display for example a spinning icon when a process starts, and hide the spinner on completion. Alternately, we could also use onComplete.
• we should make some methods available to any HTML features we construct inside onSetup -- maybe pass in some methods like onSetup(stuff)

This will be a good way to integrate this project's abilities into other sites. Like, replace an image on a webpage.

We discussed on #62 that modules should provide their own basic UI. This can be done by creating a wrapper UI module which requires in the actual module and also contains the UI elements.

What do we mean by a basic UI? A basic UI could just mean <input type="text" /> elements for all input fields or a slider as in FIsheyeGl. It's not very clear as to what should be done - or what the purpose is.

What are our expectations from the module in terms of UI? What UI Elements should it return? Ideally, Image Sequencer is supposed to have multiple steps, and not one entire page for just one step (Which has been the case for the demos I have recently made).

This is why I thought that the module can return the elements that it wants, but again the positioning and styling will highly depend on the implementation (one-step, multi-step, multi-image, et cetra.) , so it is really hard to expect the module to return a fit-all UI which can simply be injected into the front end code.

I think still a little clarity is needed on this front.

@jywarren I have been working on the removeStep().
We need a unique ID for every step for this to be done. I am aware that we already have one. It is defined in /src/UserInterface.js. This is a problem. Because /src/UserInterface.js is imported only for browser implementations.
Hence it is a unique ID only for the Browser UI, right?

We'll need a unique ID for every step. Either now, or in the future.

For the purpose of testing my code, I have implemented a sequential Unique ID system : 0,1,2,... which is implemented in /src/ImageSequencer.js

My question is : should we use a sequential one, or the one which is already implemented in /src/UserInterface.js ?

1. When I upload an image in Firefox (Mac) Image does not upload and I get this issue in the console

2. No module works when I select it from the select dropdown and then click on add step , I get the following issue on the console

I tested it in dev mode, not on the github hosted versions