sellout / external-program Goto Github PK

It was discovered on ecl-11.1.1-606449eb-linux-x86 and Quicklisp 2012-03-07 (external-program-20111001-git)

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

This repository currently has no open or pending branches.

Detected dependencies

None detected

• Check this box to trigger a request for Renovate to run again on this repository

SBCL has a :pty keyword for run-program, which make it possible to talk to other REPL like program possible, for example, python, psql, sqlite, etc. Although it's not in CCL and ECL, but maybe possible implement in a similar way by calling c functions fork, execvp like SBCL does.
I think it's a useful feature. It's of course possible to use these interpreter's low level c library, but need a lot of effort, like library RCL, LTK). Talk to interpreter to directly is a less efficient but quick way. Is this task possible (run a external python, talk to it interactively in CL) in current external-program in current external-program? Thank you

Currently tests call things like cd and which, which don’t exist on Windows. Need to make them more portable.

There is presently no way to pass environment variables such as SSH_AUTH_SOCK=/tmp/ssh-BLAH/. They are upcased when symbols, if passed as |/tmp/ssh-BLAH/| the pipes remain, etc. Perhaps I'm missing something simple...

I've recently added external-progrma to cl-test-grid.

As you can see for example in this report - http://common-lisp.net/project/cl-test-grid/pivot_lib-lisp_ql.html - there are one or more failed tests in almost all the lisps.

Please consider adding :description, :author and :license information to your ASDF system(s). This will greatly help Quicklisp users and make it easier for them to report bugs.

More information:

http://blog.quicklisp.org/2015/05/looking-for-more-metadata.html
https://www.quicklisp.org

I was using run to start gnuplot, which returned an io stream, and not a numeric value. In the cond statement at the end of run, the (< result 0) test was signaling an error, since a stream is not a numeric value.

I had to add a clause (not (null result)) before the (< result 0) to circumvent this.

This duplicates the (t ...) clause, but I did not want to move it from its current spot.

Hello.

I once tried to use external-program, and collected some notes about problems I saw.

• sb-ext:run-program on Windows doesn't accept :environment nor :env keyword arguments.
  external-program always passes the :environment argument to sb-ext:run-program,
  therefore external-program:run always fails on SBCL Windows with an error about
  illegal keyword passed to sb-ext:run-program, event if we don't pass :environment to
  external-program:run

• Quoting the program arguments against shell interpretation is
  inconsistent.

  For example on CCL (external-program:run "some-other-lisp.exe" '("--eval" "(+ 1 2)"))
  doesn't work because of spaces in the expression;
  ''("--eval" ""(+ 1 2)"") works.

  But on CLISP quoting is not necessary, and
  (external-program:run "some-lisp" '("--eval" "(+ 1 2)"))
  works fine.

  The issue is complicated by the fact that external-program docs do not
  specify how arguments are interpreted.

  I may be wrong, but I find more convenient the approach similar
  to Unix' execve, i.e. arguments are just strings which are not interpreted
  anyhow and work as if they just passed as the argv parameter of the called
  program function main (i..e the how it works on CLISP now).

• On CLISP the method external-program:run
  incorrectly interprets the exit status
  returned by CLISP function ext:run-program.

  The rules of ext:run-program:
  case 1
  if :input or :output were T,
  than ext:run-program return value
  NIL means the program exited successfully
  with status 0. Othersise ext:run-program
  returns a positive integer.
  case 2
  if :stream was specified for both
  input and output, 3 streams are returned.

  The external-program:run method interprets
  the ext:run-program return value absolutely
  differently.

• (minor) the API doc is not very clear what is the format
  of exit status returned by external-program:run.
  My assumption is that it should be in the same form
  as returned by external-program:process-status.

Note, none of these issues is real blocker for me; just reporting here for your information.

In SBCL implementation external-program is looking for /usr/bin/env which is nonexistent in Windows.

The issue is outlined here: http://trac.clozure.com/ccl/ticket/1015

A short summary: If you signal a dead process with e.g. sbcl, nothing happens. If you do it with ccl, you get a simple-error. Checking if the process is alive and killing it only then creates a race condition. Thus :error-if-exited was added, which defaults to t.

I do not know how other implementations (than SBCL and CCL) handle this matter.

I am sorry for bothering you but I just don't know where to ask. I am using some foreign CL code to create an animation using external-program to pipe cairo surfaces into an ffmpeg process which was started using external-program:start.

While on macOS 10.13. everything is working on 10.15. I get the following error:

Couldn't write to #<SB-SYS:FD-STREAM for "descriptor 15" {1001A962D3}>:
  Broken pipe

I am using SBCL 1.5.9. on both machines, ffmpeg is installed properly via macports and the process is :RUNNING according to external-program:process-status. I assume, that there was some change in the inter process communication or security settings but wasn't able to debug this further. Is anyone using macOS here and can confirm the issue?

Any hint will be gratefully appreciated!

(external-program::make-shell-string "ls" '(".") '(("PATH" . "")) nil)

yields

    0[11]: (EXTERNAL-PROGRAM::MAKE-SHELL-STRING "ls" (".") (("PATH" . ""))  NIL)
    1[11]: (EXTERNAL-PROGRAM::REFORMAT-ENVIRONMENT (("PATH" . "")))
    1[11]: returned ("PATH=''")
    0[11]: returned " #<Printer Error, obj=#x10000000af9: `\"PATH=''\"' is not of the expected type `LIST'>"

because REFORMAT-ENVIRONMENT flattens the list of conses.

RUN on ACL (and probably abcl) breaks on this; for lispworks the environment is never passed down, so it is not affected. Other backends do not use make-shell-string.

cmu-20c_release-20c__20c_unicode_-linux-x86.
quicklisp 2012-07-03

• (external-program:start "echo" '("hello"))

Function with declared result type NIL returned:
(PCL:FAST-METHOD EXTERNAL-PROGRAM:START (T T))
[Condition of type KERNEL:SIMPLE-CONTROL-ERROR]

Restarts:
0: [ABORT] Return to Top-Level.

Debug (type H for help)

("LAMBDA (.KEYARGS-START. .VALID-KEYS. G3150)" # #
"echo" ("hello") ...)
Source: (FUNCALL (THE FUNCTION #'(PCL:FAST-METHOD EXTERNAL-PROGRAM:START #))
(PCL::FAST-METHOD-CALL-PV-CELL #:G3150)
(PCL::FAST-METHOD-CALL-NEXT-METHOD-CALL #:G3150)
PCL::.ARG0.
...)

On Allegro CL the RUN method does not obtain input from a stream
correctly. Just creating an echo stream does not work, one needs to
copy the data to the provided stream in the :input form of excl.osi:with-command-io.
Test:

(with-input-from-string (s (format nil "Print~%2 lines~%")) 
 (external-program:run "cat" '() :input s :output T :error T))

does not print anything.

It is written here https://github.com/sellout/external-program/wiki/API that #'run “runs the specified program in an external (Unix) process, returning a process object if successful.” which is not relevant since it returns #'process-status application to the process object.

Also the link in the README.markdown (“Read the API documention for details.”) does not point to the API page.

site/ has broken links (and it seems much older than the html on common-lisp.net too)

– reported by Xach

(external-program:run "cat" nil :input *standard-input*)

You see, no matter how I hit enter, cat won't print anything.
Is it a bug? How to work it around?

the new test should-allow-spaces-in-args fails on ACL (as well as some other tests, but this one is new, therefore I report it):

https://cl-test-grid.appspot.com/blob?key=234009

This line:
https://github.com/sellout/external-program/blob/master/src/allegro.lisp#L11
Is not correct in Allegro. The system module is always loaded. If there is a patch for it, you can't just load it with require, you need to rebuild the lisps with update.exe or update.sh.
A customer of ours wrote in about a failure loading your package and this was the problem. Thanks.

I can already run a program asynchronously (yay!). But then how do I know when it's done? (without resorting to a loop in which I constantly call process-status)

There are mechanisms in place for this kind of task, like sb-ext:process-wait for sbcl. A look at the source of UIOP in https://github.com/fare/asdf/blob/master/uiop/run-program.lisp (where this logic is made available as part of the function %wait-process-result, but not a separate function) suggests that the following should work:

  #+clozure (ccl::external-process-wait process)
  #+(or cmu scl) (ext:process-wait process)
  #+sbcl (sb-ext:process-wait process)

I have external-program installed via quicklisp. If I load it via ASDF, I get:

% sbcl --eval "(asdf:operate 'asdf:load-op :external-program)" --quit
This is SBCL 1.3.7.nixos, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.

SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses.  See the CREDITS and COPYING files in the
distribution for more information.
WARNING:
   redefining RUN (#<SB-PCL:SYSTEM-CLASS COMMON-LISP:T>
                   #<SB-PCL:SYSTEM-CLASS COMMON-LISP:T>) in DEFMETHOD
WARNING:
   redefining START (#<SB-PCL:SYSTEM-CLASS COMMON-LISP:T>
                     #<SB-PCL:SYSTEM-CLASS COMMON-LISP:T>) in DEFMETHOD
WARNING:
   redefining PROCESS-INPUT-STREAM (#<SB-PCL:SYSTEM-CLASS COMMON-LISP:T>) in DEFMETHOD
WARNING:
   redefining PROCESS-OUTPUT-STREAM (#<SB-PCL:SYSTEM-CLASS COMMON-LISP:T>) in DEFMETHOD
WARNING:
   redefining PROCESS-ERROR-STREAM (#<SB-PCL:SYSTEM-CLASS COMMON-LISP:T>) in DEFMETHOD
WARNING:
   redefining PROCESS-STATUS (#<SB-PCL:SYSTEM-CLASS COMMON-LISP:T>) in DEFMETHOD
WARNING:
   redefining PROCESS-ID (#<SB-PCL:SYSTEM-CLASS COMMON-LISP:T>) in DEFMETHOD
WARNING:
   redefining PROCESS-P (#<SB-PCL:SYSTEM-CLASS COMMON-LISP:T>) in DEFMETHOD
% 

This does not happen for other packages (except for those that depend on external-program). Is something wrong with the build system perhaps?