msoap / shell2http Goto Github PK

with -cache=60 option, any requests like /path?url=xxx will be cached with 60s, but the query argument url varies every single request. I guess the cache is implemented based on the request path, not the uri; this behaves badly, because it should not be cached at this way at all.

Cache bases on request uri may be reasonable.

Awesome program, but I am having a problem in Windows. I am trying to have shell2http run a console executable that requires parameters and uses libraries (dll) in the same directory as the executable.
If I use the below to start shell2http:
shell2http -form /Test "C:\Test\Test.exe Arg1 Arg2 Arg3"
and
http://localhost:8080/Test
It works fine.

But

If I use the below to start shell2http:
shell2http -form /Test "C:\Test\Text.exe %v_arg1% %v_arg2% %v_arg3%"
and
http://localhost:8080/Test/Text.exe?arg1=Arg1&arg2=Arg2&arg3=Arg3

then I get an error that the library dlls cannot be found (even though C:\Test is in the PATH environment variable.

Am I doing something wrong?

Thank you for your help!

How to upload a script then excute? I need some example codes. Thank u.

Option like: -basic-auth login:password

This would allow shell2http to be used with inetd-like scenarios, eg. Systemd Socket activation, or Launchd socket activation.

This is especially useful if you want your application to bind to a privileged port (eg. 80), but don't want the application to run as root, but also don't want to set up a separate proxy process to proxy 80 -> 8080.

So in that case you would have a managing process (like launchd, systemd) open the socket on the privileged port and have it passed to the process instead.

Launchd:
https://en.wikipedia.org/wiki/Launchd#Socket_activation_protocol
https://github.com/sstephenson/launch_socket_server/blob/master/src/launch_socket_server.go

Systemd:
http://0pointer.de/blog/projects/socket-activation.html
https://www.darkcoding.net/software/systemd-socket-activation-in-go/
https://github.com/coreos/go-systemd/tree/master/examples/activation/httpserver

I am open to doing a PR if you would find the idea compatible with shell2http.

For work in CGI-mode.
Header like: X-HTTP-CODE: 403
Status: 400 Bad Request

Hi @msoap
Thanks for this tool.

I have the same use case described in #51. I followed your solution but I only get 200 response when running it in a Docker container.

docker run --name shell-server --rm -p 8080:8080 shell2http-with-scripts -show-errors -cgi -include-stderr /health './error.sh > /dev/null || echo "Status: 503\n\nError\n"' /ok './ok.sh || echo "Status: 503"'

The response for the above:

curl -v localhost:8080/health
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8080 (#0)
> GET /health HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.54.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Server: shell2http 1.13
< X-Shell2http-Exit-Code: 0
< Date: Tue, 31 Dec 2019 11:55:39 GMT
< Content-Length: 23
< Content-Type: text/plain; charset=utf-8
< 
Status: 503\n\nError\n
* Connection #0 to host localhost left intact

Do you know why?

Hi,
I really like you implementation. this simple yet powerful shell can really help me.
I'm having trouble with the "-form" option.

no matter what i try the $v_xx doesn't work.

shell2http -form /test "echo $v_xx"
running this with the URL "http://localhost:8080/test?xx=123"
the output I get is "$v_xx"

any ideas ?

Using shell2http 1.4 (a fantastic tool - thanks a lot for creating this!) we are observing the following behaviour: that the response for tasks that return a non-zero exit code is simply something like:

exec error: exit status 2

In order to actually see the error message and yet still know that the task has actually failed, we are resorting to something along the lines of:

/my/task '(do_stuff; echo Exit code: $?; exit 0)'

Would this be a good candidate for a command-line flag? Or is there an easier way to achieve the desired result (i.e. having stdout and/or stderr returned even for non-zero exit codes) without the hackery in the example above?

Great tool. I'm relatively new to docker and am having trouble passing flags to a container I've built from your image that's running on Ubuntu.

Here is my Dockerfile:
FROM msoap/shell2http
CMD ["-form /date 'date'"]

After building, I run the container with this command:
docker run --rm -p 7777:8080 test-server

I get this error:
flag provided but not defined: -form /date 'date'

Am I missing something super simple? Without the "-from" everything works. Thanks!

Instead of the application using timestamps, apply the unix philosophy in applying timestamps to the standard output/error: leave it to another tool instead.

When I just wrap everything in a shell script, I would still like to use uniform date format for my entire shell script, with something like moreutils/ts.
With shell2http doing it's own timestamps I cannot do that right now.

So I propose to introduce a --no-timestamps flag to disable shell2http adding timestamps to it's stdout/stderr.

If I have a shell like this "shell_script --help", and started the server like this:
shell2http /testing 'shell_script $v_arg1'

It doesn't work if I run it in my browser like this:
http://localhost:8080/testing?arg1=--help

How to send arguments of the shell script correctly?

When I got my shiny new MacMini with the M1 chip, migration assistant transferred my entire homebrew installation from my Intel mac. Somewhat surprisingly, it almost all "just worked" under Rosetta. The only exception I have found so far is shell2http.

Workaround: Build a native M1 version of go according to the instructions here.
Then in this directory, do:

make build
codesign -s - ./shell2http

Voila!

Hi there,
Great tool you've made. I am trying to use it with a specific package say jq, or even good old curl. Both are present on my system but missing in shell2http's script contexts. I've check with PATHs but nothinhg there. It seems that everything is running contained, which is nice and secure. How could I install those packages without calling the apt-get in my script (which would be overkill) ?
Thanks fo your help

Found when try upload image for #23

Hi,
I am trying to create mysql heartbeat on http using this tool.
So I want to execute something as below:

shell2http /showStatus '/var/lib/snapd/snap/bin/mysql.client --user=root --password=root -h 127.0.0.1 -e "select 1+1"'

I installed sudo snap install --beta mysql as well but couldn't figure out.
Can you please help.
Note: actual database in on same instance.

Need for redirect by
Location: url
header

Thank you for this great tool, we use it as health check endpoints for a variety of services. Is it possible to change the HTTP status code from '200' to let's say '500' when executed script returns a exit code other then '0' ? This would allow a health check status to be validated based solely on HTTP status code rather than string output. This would cover certain health check validators that lack string output validation.

HI

i have the following docker-compose file , but don't get multiple commds to run ...

version: '3.7'

networks:
 proxy-net:
   external: true
   
services:
  shell2http_swarm_master:
    image: bpmspace/docker_bpmspace_base:shell2http
    **command: /bin/sh "/app/shell2http -form /date date /ps "ps aux" /form 'echo $v_from, $v_to'"**
    hostname: shell2http_swarm_master
    deploy:
      labels:
        - traefik.enable=true
        - traefik.http.routers.shell2http_swarm_master.rule=Host(`shell2http_swarm_master.bpmspace.net`)
        - traefik.http.services.shell2http_swarm_master-service.loadbalancer.server.port=8080
      mode: replicated
      replicas: 1
      placement:
        constraints:
          - node.role == manager
    networks:
      - proxy-net   

  shell2http_swarm_node:
    image: bpmspace/docker_bpmspace_base:shell2http
    hostname: shell2http_swarm_node
    deploy:
      labels:
        - traefik.enable=true
        - traefik.http.routers.shell2http_swarm_node.rule=Host(`shell2http_swarm_node.bpmspace.net`)
        - traefik.http.services.shell2http_swarm_node-service.loadbalancer.server.port=8080
      mode: replicated
      replicas: 2
      placement:
        constraints:
          - node.role == worker
    networks:
      - proxy-net   

The error messages is " task: non-zero exit (1)"

When I start the service again without the "command" line it works! BUT i only have 1 command "/date" since my Dockerfile is your test.Dockerfile

Do you have any hint how to "add" new commands via compose file? My image is created via GITHUB -> webhooks -> dockerhub ... so it is no option to add all the commands i need in in the Dockerfile and in a DockerSwarm the "build:" in docker-compose is not supported ....

thanks rob

PS when i try to execute the command in the sh of the container it tells me:

/ # ./app/shell2http -form /date date /ps "ps aux" /form 'echo $v_from, $v_to'
2020/01/05 15:30:40 register: /date (date)
2020/01/05 15:30:40 register: /ps (ps aux)
2020/01/05 15:30:40 register: /form (echo $v_from, $v_to)
2020/01/05 15:30:40 register: / (index page)
2020/01/05 15:30:40 listen tcp :8080: bind: address already in use

I installed the SNAP on my Ubuntu 20.04

I executed several of the "Examples" for instance:

$ shell2http /top "top -l 1 | head -10"
2020/09/11 11:33:32 register: /top (top -l 1 | head -10)
2020/09/11 11:33:32 register: / (index page)
2020/09/11 11:33:32 listen http://localhost:8080/

Am I supposed to see something more than the above? Maybe I am misunderstanding the use of this tool?

Thanks for any enlightenment.

Brian

On windows, shell2http -shell="powershell" /shell 'something' returns "loading managed windows powershell failed with error 8009001d" for http request.

when I'm calling bash scripts, which contains docker command I get an exception:
docker: not found
Note: I've installed shell2https with snap, like:
sudo snap install shell2http

Broken parse http-headers from cat file (in CGI mode).
When linking several javascript files, looks like some of them did not load properly.
For example, js file :

var x = new XMLHttpRequest();
var cowlist;
x.open("GET", "http://localhost:8080/array", true);
x.onload = function (){
    str = x.responseText;
    cowlist = str.split(' ');
}
x.send(null);

did not load at all.

Example run:
shell2http -cgi /script ‘cat script.js’

For use another shell:
-shell="bash /c"
-shell="" -- dont use shell, execute command directly

Hi,

I have a script that fails and that's ok, I'm getting following:

/bin/mkdir: cannot create directory ‘/a/b/c’: No such file or directory
/bin/ln: failed to create symbolic link ‘/c/b’ -> ‘’: No such file or directory
/bin/ln: failed to create symbolic link ‘/c/b’ -> ‘’: No such file or directory

The issue in here are quotes... instead of ASCII shell2http returns unicode's quote (U+2019) what is wrong

Is there any quick fix for this?

For longer running commands, such as wget or pv with progress indication, is it possible to flush output and make it visible to the webbrowser while running?

ubuntu@ip-172-31-4-243:~$ go build github.com/msoap/shell2http
go/src/github.com/msoap/shell2http/shell2http.go:4:2: cannot find package "context" in any of:
/usr/lib/go-1.6/src/context (from $GOROOT)
/home/ubuntu/go/src/context (from $GOPATH)

When running the program like so

shell2http -port=80 / "tail /root/server/logs/logfile.log"

it gives the following output when a request is received

tail: cannot open '/root/server/logs/logfile.log' for reading: Permission denied
2020/04/01 13:09:57 exec error:  exit status 1

but the command

tail /root/server/logs/logfile.log

can be executed normally outside shell2http.

OS Ubuntu 18.04
The program is being executed by the root user.
The file and directories and are owned by user and group root.
The permissions are as follows:
root: 700
server: 775
logs: 644
logfile.log: 644

I've managed to make shell2http execute the command by setting permissions 744 on the "logs" folder but I'm confused for the reason that it's working.
From what I understood while searching for the answer, directories must have the "x" permission to allow acessing it's files (StackExchange/AskUbuntu), so theoretically my shell is wrong for letting me access the contents of the logs folder, unless there is something special about the root user.

Shouldn't the 2 situations execute the command in the same way?

It may be that this question is more Ubuntu/Linux than about shell2http, so please let me know if here is the appropriate place for it.

For example for get uploaded file

Currently my bash script is
#!/bin/bash
PWD=/root
SHELL=/bin/bash
PATH=some path variables
cd $PWD
shell2http -port=80 -export-all-vars and my commands

Well. I actually dont understand how shell2http work with enviroment vars. What does -export-all-vars do?

It looks like that the recent version doesn't support registering resources and specifying the allowed HTTP method for each.

For example:

POST /run/command - to execute
GET /run/command - to get the status

Hello,

I get a Failed to connect to bus: No such file or directory when trying to run a systemctl command to grab the status of a service:

$ ./shell2http /test 'systemctl is-active --user --quiet myservice && echo "myservice is active"'

shell2http output:

Failed to connect to bus: No such file or directory
2020/11/21 11:51:49 exec error:  exit status 1

I tried launching shell2http as root and using su in the command but it still fails in the same way.

Is it possible to send back systemctl output ?

Because koding/cache now depends of mongodb client library.

Hi,

how to install this on Raspberry Pi?

Thank you in advance.

Regards,
amilino

When I run this command I get this error message:

shell2http -export-all-vars -shell="bash" -form /unblock-ip "csf -g $v_ip"

[root@acesso ~]# shell2http -export-all-vars -shell="bash" -form /unblock-ip "csf -g $v_ip"
2021/03/17 14:26:29 register: /unblock-ip (csf -g )
2021/03/17 14:26:29 register: / (index page)
2021/03/17 14:26:29 listen http://localhost:8080/
2021/03/17 14:26:31 /unblock-ip - 404
bash: csf: command not found

I can affirm that this command exists.

Hi !
Is there a way to have a tar.gz release ?
I plan to add your binary like that in my Dockerfile :
ADD https://github.com/msoap/shell2http/releases/download/1.7/shell2http-1.7.linux.amd64.tar.gz /
But it has to be a direct link or a tar.gz

Thanx for reading

I want it for my Google Cloud Run thing to be sure it'll not reuse the instance.

C:\Selva\software\shell2http>shell2http -form /starttest "C:\Selva\software\shell2http\script.bat"
2020/03/02 14:17:56 register: /starttest (C:\Selva\software\shell2http\script.bat)
2020/03/02 14:17:56 register: / (index page)
2020/03/02 14:17:56 listen http://:8080/
2020/03/02 14:18:05 localhost:8080 [::1]:55677 GET /starttest "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.122 Safari/537.36"
2020/03/02 14:18:05 exec error: exit status 1

Getting above error when we tired an api for executing a bat file
http://localhost:8080/starttest

In seconds:
-timeout 30

For example, can't work redirects:
shell2http -cgi -cache 10 /r 'echo "Location: /date\n";' /date date

will make that tool super-super-cool, for now it's only super-cool :)

in my Arch Linux environment, i have run.sh with content below:

#!/usr/bin/env bash
export HTML_INDEX="site/index.html"
export MAIN_JS="site/main.js"

shell2http -export-all-vars -cgi -form / "cat $HTML_INDEX" \
/echo 'echo "$v_msg"' \
/call2 'echo "Content-Type: application/json";echo ;echo "{\"error\": \"ok\"}"' \
/main.js 'echo "Content-Type: application/javascript";echo ;cat $MAIN_JS' 

in the script above i can set correct Content-Type response header of the /main.js path. it works correctly in browser.

now, i want to replicate script above in Windows environment (Windows 8.1) with run.bat below:

@echo OFF

set HTML_INDEX=site\index.html
set MAIN_JS=site\main.js
set TYPE=Content-Type: application/javascript

shell2http.exe -cgi / "type %HTML_INDEX%" ^
/main.js "echo %TYPE% & type %MAIN_JS%"

the problem is, the Content-Type not set in Response Header, but included in Response Body output of /main.js

Content-Type: application/javascript
alert('Hello World') <-- content inside `site\main.js`

i've tried adding \n\n and echo. to add newline

/main.js "echo %TYPE%\n\n & type %MAIN_JS%"
/main.js "echo %TYPE% & echo. & type %MAIN_JS%"

the Content-Type always included inside the Response Body.

but of course, if i only using this command below:

/main.js "type %MAIN_JS%"

the main.js script executed by the browser successfully, but with Content-Type: text/plain.

How to make the Response Header setting works in windows?

I think SSL option will be great. :-)

It seems that shell2http waits for any child process that is spawned to finish.
For example if your un script.py that will spawn another child process script2.py and exits, it will still wait for script2.py to finish. Even thou _script.py has exited. I have tried using & and & disown when starting the script it still seems to be attached to the process and every child process.

Repro steps:

• Download release version 1.12 on linux amd64
• untar
• ./shell2http --version
• print 1.11

Could you use this to turn https://github.com/fogleman/primitive into a web app easily?

Thanks

Hi,
Can this be achieved? Let me tell you what is going on here.
I have a script that stays on the screen after executing, i've tried sending it to >dev/null with success but http still waits for response until a timeout is displayed. What i want to do is access the "/command" and the website to reply with no content.
Thanks.

For example:

X-Shell2http-Exit-Code: 0