add maxBuffer option to avoid "stdout maxBuffer exceeded" error

Hello,

I have to run a deployment shell script, which at one point prompt for y/n question, the shell task continues as expected, till script prints done message, than it fails to emit stop.

gulp_shell.task([
'pkill -f proxy-server.js',
'pkill -f http-server',
'nohup node proxy-server.js &',
'nohup http-server -c-1 src -P http://localhost:5050 &'
],{
ignoreErrors: true
}
)

The command no 4 never executes.

I did walkthrough the library code and found that it invokes child_process.exec and waits for process finished callback to execute the subsequent command in the list.

In my case 3rd command is long running process which will never exit and so 4th command will never execute.

I think for such long running tasks , one can listen to process started signal and move to next task in the list.

I'm using shell to run npm run browserify where the actual command is browserify --debug ./src/index.js | exorcist --base ./js ./js/index.js.map > ./js/index.js.

The error handling of shell is function (error, stdout, stderr) { if (error) ... }. When the above command fails, the arguments are like:

error: null
stdout: empty string
stderr: the actual error message

Running only the browserify command will throw the error as expected. Piping it to exorcist will reproduce what I described above. But the error is from the browserify part, not from exorcist. So I think it's about piping.

Please checkout my fork on branch browserify. Run gulp bug will reproduce the bug.

What is the preferred way of executing something like ./tools/mytool.exe without changing current working dir to tools?

I'm running a gulp task invoked from a MSBUILD target. This executes in a context where I believe there is no standard in.

I'm getting the following error:

1>  [16:45:28] Error: Implement me. Unknown stdin file type!
1>      at process.stdin (node.js:558:17)
1>      at async.eachSeries.self.emit.gutil.PluginError.plugin (D:\Source\team2\Core\Source\SumTotalUI\SumTotal.UI.MVC\node_modules\gulp-shell\index.js:65:14)
1>      at D:\Source\team2\Core\Source\SumTotalUI\SumTotal.UI.MVC\node_modules\gulp-shell\node_modules\async\lib\async.js:187:20
1>      at iterate (D:\Source\team2\Core\Source\SumTotalUI\SumTotal.UI.MVC\node_modules\gulp-shell\node_modules\async\lib\async.js:265:13)
1>      at Object.async.forEachOfSeries.async.eachOfSeries (D:\Source\team2\Core\Source\SumTotalUI\SumTotal.UI.MVC\node_modules\gulp-shell\node_modules\async\lib\async.js:284:9)
1>      at Object.async.forEachSeries.async.eachSeries (D:\Source\team2\Core\Source\SumTotalUI\SumTotal.UI.MVC\node_modules\gulp-shell\node_modules\async\lib\async.js:220:22)
1>      at DestroyableTransform._transform (D:\Source\team2\Core\Source\SumTotalUI\SumTotal.UI.MVC\node_modules\gulp-shell\index.js:35:11)
1>      at DestroyableTransform.Transform._read (D:\Source\team2\Core\Source\SumTotalUI\SumTotal.UI.MVC\node_modules\through2\node_modules\readable-stream\lib\_stream_transform.js:172:10)
1>      at DestroyableTransform.Transform._write (D:\Source\team2\Core\Source\SumTotalUI\SumTotal.UI.MVC\node_modules\through2\node_modules\readable-stream\lib\_stream_transform.js:160:12)
1>      at doWrite (D:\Source\team2\Core\Source\SumTotalUI\SumTotal.UI.MVC\node_modules\through2\node_modules\readable-stream\lib\_stream_writable.js:326:12)

Here's the gulp task:

gulp.task('optimize-requirejs', shell.task([
    'r.js.cmd -o app.build.js'
]));

gulp.task('exec', ['clean'], function() {
  gutil.log("start execution");
  shell.task(
      "node scripts/script.js"
  );
  gutil.log("end execution");
});

Does not report any errors, but also does not seem to execute the script.

Hi! First of all thank you for making such an awesome utility gulp module. I have been very happily using it for many months.

I am currently working with a project (using Samsung Tizen Wearable SDK) which for some reason forces me to run several commands in a particular folder and none others.

In order to complete the build process at one go (i.e. gulp build), I need to be able to 'cd to/this/path/' and found such a feature is not supported yet with gulp-shell.

Do you have a plan to support this? Or could you suggest a workaround?

Currently I am doing:

1. gulp build1
2. cd to/this/path (manually)
3. gulp build2

I want to do gulp build and go grab a coffee.

An option to specify additional keys for the template, not only "file"

google/material-design-lite#790

People are using this in weird ways, can you add a clarification to the README that this module should only be used if:

• you need to run one command per file in the stream
• you need templating per file in the stream

and to use child_process otherwise?

Empty globs will start throwing errors in 4.0 as well. Thanks!

When I increase the number of tasks from 4 to 5, I got

(node) warning: possible EventEmitter memory leak detected. 11 listeners added. Use emitter.setMaxListeners() to increase limit.
Trace
    at ReadStream.EventEmitter.addListener (events.js:160:15)
    at ReadStream.Stream.pipe (stream.js:118:10)
    at /tmp/test/node_modules/gulp-shell/index.js:51:21
    at iterate (/tmp/test/node_modules/gulp-shell/node_modules/async/lib/async.js:149:13)
    at Object.async.eachSeries (/tmp/test/node_modules/gulp-shell/node_modules/async/lib/async.js:165:9)
    at Transform._transform (/tmp/test/node_modules/gulp-shell/index.js:35:11)
    at Transform._read (/tmp/test/node_modules/gulp-shell/node_modules/through2/node_modules/readable-stream/lib/_stream_transform.js:184:10)
    at Transform._write (/tmp/test/node_modules/gulp-shell/node_modules/through2/node_modules/readable-stream/lib/_stream_transform.js:172:12)
    at doWrite (/tmp/test/node_modules/gulp-shell/node_modules/through2/node_modules/readable-stream/lib/_stream_writable.js:237:10)
    at writeOrBuffer (/tmp/test/node_modules/gulp-shell/node_modules/through2/node_modules/readable-stream/lib/_stream_writable.js:227:5)

And the gulpfile.js

var gulp = require('gulp');
var shell = require('gulp-shell');

gulp.task('test1', shell.task([
    "find . -type f -name '*.txt' > /dev/null "
]));

gulp.task('test2', shell.task([
    "find . -type f -name '*.txt' > /dev/null "
]));

gulp.task('test3', shell.task([
    "find . -type f -name '*.txt' > /dev/null "
]));

gulp.task('test4', shell.task([
    "find . -type f -name '*.txt' > /dev/null "
]));

gulp.task('test5', shell.task([
    "find . -type f -name '*.txt' > /dev/null "
]));

gulp.task('test', ['test1', 'test2', 'test3', 'test4', 'test5' ]);

node -v => v0.10.28

Lodash has been updated to version 3.10.0. There are a few other libs that are out of date, but it would be nice to get lodash updated to current. Would you consider changing support from ~ to ^ for lodash?

My team needs to be able to use values from file to set the cwd option.

I am running serveral gulp tasks, under docker containers, inside Jenkins jobs.

If I disable pseudo-tty allocation (required for jenkins for capturing the output of jobs), running a gulp shell task does not exit when using the shorthand notation.

Here are the commands I am running:

# -T option disables pseudo-tty allocation
docker-compose run -T api sh -c 'gulp migration'

gulp.task('migration', shell.task('mongo db:27017/ctrldb ./migrations/channels.js'));

In my case gulp-shell is last in pipeline. gulp.dest is not needed because the utility, that gulp-shell runs, already outputs the final file.

Also, the script should do something with the final file after gulp-shell finishes. So, i use .once('end', function. But it seems that .pipe(shell(...)) does not emit 'end'.

What can i do? May be it is better to make gulp-shell work instead of gulp.dest?

Hi,
I think I'm seeing an issue in windows with '..' is not recognized as an internal or external command,

Here is the gulp file.

var gulp = require('gulp');
var shell = require('gulp-shell');
var handleErrors = require('../util/handleErrors');

gulp.task('jshint', shell.task([
  '../node_modules/.bin/jsxhint --harmony "../src/**" "./src/app/**" --exclude ../src/utils/modernizr.custom.js'
])).on('error', handleErrors);

and the output.

chris@C005 /e/code/material-ui/docs (master)
$ gulp
[13:41:36] Using gulpfile e:\code\material-ui\docs\gulpfile.js
[13:41:36] Starting 'setWatch'...
[13:41:36] Finished 'setWatch' after 64 μs
[13:41:36] Starting 'jshint'...
[13:41:36] Starting 'markup'...
[13:41:36] Starting 'fontIcons'...
[13:41:36] Starting 'css'...
'..' is not recognized as an internal or external command,
operable program or batch file.
[13:41:36] 'jshint' errored after 47 ms
[13:41:36] Error in plugin 'gulp-shell'
Message:
    Command `../node_modules/.bin/jsxhint --verbose --harmony "../src/**" "./src/app/**" --exclude ../src/utils/modernizr.custom.js` failed with exit code 1
[13:41:36] Finished 'css' after 40 ms
[13:41:36] Finished 'markup' after 62 ms
[13:41:36] Finished 'fontIcons' after 63 ms

Running manually on the shell is fine.

chris@C005 /e/code//material-ui/docs (master)
$ ../node_modules/.bin/jsxhint --harmony "../src/**" "./src/app/**" --exclude ../src/utils/modernizr.custom.js

Any suggestions would be most welcome.

Thanks,
Chris

I'd like to do this:

gulp.task('git-rev', function() {
  var shell = require('gulp-shell');
  var defineModule = require('gulp-define-module');
  return gulp.src('')
    .pipe(shell([
      'git rev-parse HEAD'
    ]))
    .pipe(defineModule('commonjs'))
    .pipe(rename('sha.js'))
    .pipe(gulp.dest('./www/js/multiapp/config/'));
});

I get find: missing argument to -exec'` from both of these:

gulp.task('php-lint', shell.task([
    'find $(pwd)/app -name \'*.php\' -exec php -l {} \;',
],{
  verbose:true,
}));
gulp.task('php-lint',function () {
  return gulp.src('').pipe(shell([
      'find $(pwd)/app -name \'*.php\' -exec php -l {} \;',
  ]));
});

Also:

/bin/sh: 1: Syntax error: "|" unexpected

for:

find $(pwd)/resources/php -name \'*.php\' -exec php -l {} \; | grep -v "No syntax errors detected in"

I'm using webstorm 9 EAP (it has gulp integration) - and I'm pretty sure the issue below is webstorm related, but I'm stuck and wondering if you may be able to help nudge me in the right direction here.

I have the following 'gulpfile.js':

var gulp = require('gulp'),
    shell = require('gulp-shell');

gulp.task('test',  shell.task(
    'ls'
));

When I run task from my terminal, it works fine. When I run from webstorm I get:

/usr/local/bin/node /usr/local/lib/node_modules/gulp/bin/gulp.js --color --gulpfile /Users/ryan/projects/readfulapp-webservices/gulpfile.js test
[22:24:14] Using gulpfile /Users/ryan/projects/readfulapp-webservices/gulpfile.js
[22:24:14] Starting 'test'...
/bin/sh: ls: command not found
[22:24:14] 'test' errored after 11 ms
[22:24:14] Error in plugin 'gulp-shell'
Message:
    Command failed: /bin/sh: ls: command not found

Details:
    killed: false
    code: 127
    signal: null

Process finished with exit code 1

Any idea what could be going on here? Don't spend much time on this as I'm pretty sure its not a bug in your code.

I am running a command (npm test) on save, if there is an error thrown it kills the gulp process. Is there anyway to keep this from happening? Or, if there is an error and process is killed, can it be automatically restarted?

Now it's a bit tricky to follow what has changed.

Thanks!

I’m trying to add a task that runs tail -f on a log file, but exclides some lines (| grep -v).

Unfortunatelly this does not work as expected:

gulp.task(
    'log',
    function () {
        return gulp.src('').pipe(shell('tail -n0 -f /var/log/nginx/access.log | grep -Pv "/(img|css|js)/"'));
    }
);

However these do:

// no piping
… shell('tail -n0 -f /var/log/nginx/access.log') …
// piping to cat
… shell('tail -n0 -f /var/log/nginx/access.log | cat') …
// no -f attribute to tail
… shell('tail -n0 /var/log/nginx/access.log | grep -Pv "/(img|css|js)/"') …

Isn’t this possible with gulp-shell or am I maybe just doing something wrong?

I keep encountering this error, I presume it has something to do with the EOL of a Windows/DOS OS mixing up with when I run it from my mac OSX? It tries to run node/r rather than node.

[11:17:15] Error in plugin 'gulp-shell'
Message:
    Command failed: env: node\r: No such file or directory

Details:
    killed: false
    code: 127
    signal: null

Hello there.

Been having issues in getting gulp-shell to read keyboard input.

example:

#!/bin/sh
echo "type something: ";
read VARIABLE_NAME
echo $VARIABLE_NAME

It successfully prints "type something", but then when you start typing, it doesn't read any input. Same goes for bash scripts. Is this expected behavior? Or perhaps a limitation of node?

When I use gulp.shell with Gulp 4, what is the correct syntax? All attempts I've made using Gulp 4 seem to fail with the following gulp error:

Did you forget to signal async completion?

Here's a simple example of Gulp 3 task which worked:

gulp.task('npmInstall', shell.task[
  'echo Installing dependencies',
  'npm install my-package'
]);

I had hoped to just do something like:

gulp.task('npmInstall', gulp.series(()=> {
  shell.task[
    'echo Installing dependencies',
    'npm install my-package'
  ];
}));

but this is an example of where I get this error. I've tried a few other approaches but nothing seems to work. :(

I had a CLI tool that was not finishing properly. Node's child_process module has a timeout option. We should allow the user to pass a timeout in and set that on the child process. This would help my situation and possibly others.

https://nodejs.org/api/child_process.html

The Mocha's error logging is colorful. But the error handling will only print the text.

I did not notice this days before. I'm not sure now if I should use the native npm run instead of gulp.

Duplicate of https://github.com/robrich/gulp-exec

// @robrich

Great tool, however, it seems that when using wildcards in my gulp.src, only the first 16 files are being passed on to the shell. Any reason for this, and any way to override it?

I'm using gulp-shell for a number of tasks that are run in a Bamboo CI plan. However, these tasks always seem to hang my Bamboo agents, forcing me to manually abort the builds.

The gulp task prints that it is finished, but it apparently never returns control to the build agent. The only way I've been able to get this to work is by adding process.exit(0); in a stream end event handler for the offending gulp tasks. This does not happen with any gulp tasks that do not use gulp-shell.

Any thoughts? Here's an example:

gulp.task('lint', function() {
  return gulp.src([
      '**/*.py',
      '!node_modules/**/*.py',
    ], {
      read: false
    })
    .pipe(shell([
      'echo \"Linting <%= file.path %>...\"',
      'pylint <%= file.path %> -E'
    ]))
    .on('error', function() {
      gutil.log(gutil.colors.red('LINTING FAILED'));
      process.exit(1);
    })
    .on('end', function() {
      process.exit(0); // Force bamboo task to end
    });
});

Hi there - thanks for the util :)

Is there any way to get access to the child process? I need to explicitly kill the process but by looking at the source, it doesn't seem like that is possible..

Do stuff:

gulp.task('client-gulp-build', shell.task(['bower install', 'gulp --harmony build'], {
  cwd: './node_modules/project-client',
}));

In project-client task:

gulp.task('build', function(callback) {
  return runSequence(
    'clean',
    'client-init',
    ['build-widget', 'build-images', 'build-custom'],
    callback);
});

In 0.4.1-0.4.3 version of gulp-shell 'client-gulp-build' task executes all commands, but gulp-shell task doesn't complete itself. Next task doesn't start.

In 0.4.0 version all works correctly.

I have a simple task that copies a couple files, but it never exists. When I gulp with this task included I have the hit enter for the gulp process to exit.

I tried it with several simple commands and always get this result.

Am I missing something in the docs?

When i try to install or:

npm outdated
gulp                 3.8.6   3.8.6   3.8.8  gulp
gulp-load-plugins    0.5.3   0.5.3   0.6.0  gulp-load-plugins
gulp-concat          2.3.3   2.3.3   2.4.0  gulp-concat
gulp-notify          1.4.0   1.4.0   1.5.1  gulp-notify
gulp-uglify          0.3.1   0.3.1   1.0.1  gulp-uglify
gulp-shell         MISSING   0.2.9   0.2.9  gulp-shell
gulp-minify-css      0.3.6   0.3.6   0.3.8  gulp-minify-css
glob                3.2.11  3.2.11   4.0.5  gulp-load > glob
has-ansi             0.1.0   0.1.0   1.0.0  chalk > has-ansi
strip-ansi           0.3.0   0.3.0   2.0.0  chalk > strip-ansi
supports-color       0.2.0   0.2.0   1.1.0  chalk > supports-color
event-stream        3.0.20  3.0.20   3.1.7  gulp-replace > event-stream
safefs               3.0.6   3.0.6   3.1.1

Example gulpfile, run from windows command promt (Windows 7, node v0.12.2, gulp-shell v0.4.1)

var gulp = require('gulp');
var shell = require('gulp-shellexec');
gulp.task('_testexec', shell.task('cmd /k "set Pa"'));

Shows that there are Path and PATH variables, which causes problems for programs that were run from gulp-shell. For example, .NET Process.Start method will fail with an exception:

Item has already been added. Key in dictionary: 'Path' Key being added: 'PATH' ---> > System.ArgumentException: Item has already been added. Key in dictionary: 'Path' Key being added: > 'PATH'
at System.Collections.Hashtable.Insert(Object key, Object nvalue, Boolean add)
at System.Diagnostics.ProcessStartInfo.get_EnvironmentVariables()

It seems that it's caused by line 29 in index.js:

options.env = _.extend({}, process.env, {PATH: PATH}, options.env)

I use the following replacement:

var pathKey = _.findKey(process.env, function(val, key){return key.toLowerCase() === 'path'}) || 'PATH';
options.env = _.extend({}, process.env, options.env);
options.env[pathKey] = PATH;

can i pipe the output of a command to a standard gulp pipe (browserify for example)?

thx in advance :)

I'm having following issue:

gulp.task("exec", shell.task("node script.js")); // does not report an error when script.js does process.exit(1)

when

node script.js

following correctly fails:

gulp.task("exec", shell.task("exit /b 1"));

Haven't tried to repro with a simple script yet

… I'm dumb. Ignore this.

If I run a command that writes to the console (stdout & stderr) I can suppress the output using options.quiet: true.

That's OK because I don't want to fill up my screen with this info, but if there's a problem, I want to be able to view that info to troubleshoot.

How about a new option: redirect
It would take a filename (relative to process.cwd()) and send the results of that output to it.

It looked like this could be done in the command and using a template, e.g.:

shell.task('npm install > <%= logfile %> 2>&1', {
    cwd: workspace_loc,
    templateData: {
        logfile: process.cwd() + path.sep + 'npm-install.log'
        }
    });

But that's a lot uglier, less readable, and makes a lot more boilerplate than:

shell.task('npm install', {cwd: workspace_loc, redirect: process.cwd() + path.sep + 'npm-install.log'});

Indeed, with a bit further testing, it seems like the redirect causes the task to indicate it's completed as soon as it starts writing to the file rather than once it actually completes, so it isn't really a solution.

Could you please give me an options.code callback hook-up? I'd like to run a function any time a command succeeds or fails (it doesn't matter which) and get its exit code to drive notifications with. See lantiga/ki#22 about one kind of use case in the context of datomiki.

I have the task below and for some reason I keep getting this error, when it is actually a Cordova project.

events.js:72
        throw er; // Unhandled 'error' event
              ^
Error: Command failed: Current working directory is not a Cordova-based project.

    at ChildProcess.exithandler (child_process.js:648:15)
    at ChildProcess.emit (events.js:98:17)
    at maybeClose (child_process.js:756:16)
    at Socket.<anonymous> (child_process.js:969:11)
    at Socket.emit (events.js:117:20)
    at Pipe.close (net.js:465:12)

I've logged out the cwd using gulp-debug and ran a pwd using gulp-shell and when the command runs, i'm in the right directory. If i run the command manually it works fine, just not with this module.

gulp
  .task('android', ['www'], function() {

    gulp
      .src('./')
      .pipe(shell('cordova build android'))
    ;
  })

Hello,

I am having an issue with gulp-shell. I have the following task :

gulp.task('styleguide-kss',['compile-styleguide'], shell.task([
'kss-node <%= source %> <%= destination %> --template <%= template %> --css <%= cssfile %>'
], {
templateData: {
source: paths.styleguide.source,
destination: 'styleguide/docs',
template: paths.styleguide.template,
cssfile: 'css/styles.css'
}
}
));

It fails with the message that command failed with code 1. If I run the command manually via kss-node (the same command in the output messsage exactly) it works fine.

Any idea what the problem might be?

Best regards

I saw #32, but it looks like that is resolved and updating to v0.4.x of gulp-shell doesn't fix my issue. Here is my current gulpfile.js

#!/usr/bin/env gulp
'use strict';

var gulp = require('gulp'),
    shell = require('gulp-shell');

gulp.task('shorthand', shell.task([
  'echo hello',
  'echo world'
]));

I also tried this without shorthand and it still doesn't work:

#!/usr/bin/env gulp
'use strict';

var gulp = require('gulp'),
    shell = require('gulp-shell');

gulp.task('shorthand', function () {
  return gulp
  .src('')
  .pipe(shell([
    'echo hello',
    'echo world'
  ]));
});

Correct me if I am wrong--I also don't see any tests that pertain to this, so the fact that the tests pass don't mean much to this problem.

Thanks!

On Windows the command "mkdir -p somedir" will end up creating both the folders "-p" and "somedir", it doesn't seem to pass -p as a parameter.

gulp.task('phpcs', function(){
    return gulp.src(phpFiles, {base: './'})   
        .pipe(shell(['phpcs -n --standard="' + composer_modules + 'ruleset.xml" "<%= file.path %>"']))
    ;
});

Specific command ran:

phpcs -n --standard="C:/Users/username/AppData/Roaming/Composer/vendor/bin/phpcs-ruleset
.xml" "C:\Program Files\WAMP\www\gulp\index.php"

gulp and gulp-shell run the command above successfully (correct output is seen) but the final message gulp-shell issues to the command line is that the command has failed (even though everything executed fine).

I'm executing a node script with node index.js but gulp-shell doesn't preserve colors in stdout. It would be helpful to keep them in if possible.

error occured in ci(quickbuild), not in cli.
work well at 0.2.5.

my guess is maybe quickbuild read the child process log from std, and gulp-shell's stdin reading code conflicted?

throw er; // Unhandled 'error' event
^
Error: read ECONNRESET
at errnoException (net.js:904:11)
at Pipe.onread (net.js:558:19)

Created a local fork and after doing the npm install, ran the tests and received a couple errors. The error is due to the fact that the test requires ' \n '

[gulp] 'test' errored after 543 ms Command failed: 1 failing

1. gulp-shell(commands, options) options cwd should use the process current working directory when cwd is not passed:
   Uncaught AssertionError: expected '/Users/mikee/documents/projects/gulp-shell\n' to contain '/Users/mikee/Documents/Projects/gulp-shell'
   at Assertion.prop.(anonymous function) (/Users/mikee/Documents/Projects/gulp-shell/node_modules/should/lib/should.js:60:14)
   at Socket.process.stdout.write (/Users/mikee/Documents/Projects/gulp-shell/test/index.js:11:20)
   at Socket.ondata (stream.js:51:26)
   at Socket.EventEmitter.emit (events.js:117:20)
   at Socket. (stream_readable.js:746:14)
   at Socket.EventEmitter.emit (events.js:92:17)
   at emitReadable (_stream_readable.js:408:10)
   at emitReadable (_stream_readable.js:404:5)
   at readableAddChunk (_stream_readable.js:165:9)
   at Socket.Readable.push (_stream_readable.js:127:10)
   at Pipe.onread (net.js:528:21)

stream.js:94
      throw er; // Unhandled stream error in pipe.
            ^
Error: read ECONNRESET
    at errnoException (net.js:904:11)
    at Pipe.onread (net.js:558:19)
[06:30:44] 'deploy:scripts:libs' errored after 16 s
[06:30:44] Error in plugin 'gulp-shell'
Message:
    Command `gulp scripts:libs` failed with exit code 8
[06:30:44] 'deploy:tick' errored after 23 s
[06:30:44] Error in plugin 'gulp-shell'
Message:
    Command `gulp deploy:scripts:libs` failed with exit code 1

Fatal error: local() encountered an error (return code 1) while executing 'gulp deploy:tick'

node v0.10.33
npm v2.11.1
gulp-shell v0.4.1

I used this version the other day and it was accepting stdin and now appears to be hanging. Any thoughts?