Code Monkey home page Code Monkey logo

subprocess's Introduction

subprocess

cppver build codecov issues GitHub online stars

A no nonsense library for writing shell commands in C++.

Writing shell commands in modern C++ is deceptively hard. There are many C++ subprocessing libraries out there, but none of them just work. The aim of this library is to allow you to write shell commands in C++ almost as if you were writing them in shell.

Full documentation for subprocess is available here.

Note: Windows is not currently supported.

TL;DR

It works exactly how you would expect it to work. Drop subprocess.hpp in your project, include it, and start working!

#include <subprocess/subprocess.hpp>

using namespace subprocess::literals;

int main()
{
    std::string cmd_output;
    ("ls -l"_cmd | "awk 'NR>1{print $3}'"_cmd | "sort"_cmd | "uniq"_cmd > cmd_output).run();
    // Use cmd_output in the program
}

Philosophy

Instead of trying to emulate the subprocess interface from libraries in other languages, this library aims to use (and abuse) C++ operator overloading to achieve a shell-like syntax for writing shell commands.

subprocess follows these design goals:

  • Intuitive Syntax: Common operations on subprocess commands should feel like shell. This should allow users to compose commands in a familiar manner. There should ideally be no gotchas or differences in the behavior of subprocess and unix shell. In case such differences arise, they should be clearly documented.
  • Trivial Integration: The whole code comprises of a single subprocess.hpp and requires no adjustments to compiler flags or project settings. It has no dependencies, subprojects or dependencies on any build system.
  • Serious Testing: A CI pipeline performs heavy integration-testing that covers more than 90% of the code. These tests are run on all the platforms the library supports. Additionally, address and memory sanitizers are run to detect any memory or resourse leaks.

Overview

You can write shell commands using the subprocess::command class and use the resulting object to pipe I/O from/to standard streams, files, and variables.

Examples:

// Running a command
subprocess::command cmd{"touch" + file_path}.run();

// Piping the output of a command to another command
cmd | "uniq"_cmd;

Redirecting stdin

You can use operator< to redirect stdin to the command object.

Examples:

// Reading input from a file
cmd < std::filesystem::path{file_name};
cmd < "file_name";

// Reading input from a variable
std::string input{"abc"};
cmd < input;

// Reading from an already created fd
cmd < subprocess::file_descriptor{fd_no};

Redirecting stdout

The following operators are available for redirecting stdout:

  • operator>: Truncates the file and writes output
  • operator>>: Appends output to file if it already exists, otherwise creates one.

Examples:

// Redirecting stdout to stderr
cmd > subprocess::err;

// Redirecting stdout to a file
cmd > std::filesystem::path{file_name};

// or appending to a file
cmd >> std::filesystem::path{file_name};

// Capturing stdout in a variable
std::string var_name;
cmd > var_name;

Redirecting stderr

The following operators are available for redirecting stdout:

  • operator>=: Truncates the file and writes stderr
  • operator>>=: Appends stderr to file if it already exists, otherwise creates one.

Examples:

// Redirecting stderr to stdout
cmd >= subprocess::out;

// Redirecting stderr to a file
cmd >= std::filesystem::path{file_name};

// or appending to a file
cmd >>= std::filesystem::path{file_name};

// Capturing stderr in a variable
std::string var_name;
cmd >= var_name;

CMake FetchContent

This is a header only library. So developer can just copy-paste its content into their project without any configuration.

For those who want to manage their dependencies using CMake FetchContent, this can be used:

include(FetchContent)
FetchContent_Declare(subprocess
  GIT_REPOSITORY https://github.com/rajatjain1997/subprocess
  GIT_TAG v0.1.1
  GIT_PROGRESS TRUE
)
target_link_libraries(my_application subprocess::subprocess)

Conclusion

I would love your feedback! If you find any issues, feel free to log them.

subprocess's People

Contributors

arthursonzogni avatar nlohmann avatar rajatjain1997 avatar shawnw avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

subprocess's Issues

File descriptors are opened during command creation

Currently, file descriptors are opened when the command is being built. i.e.

command{"ls"} | command{"cat"};

would actually open a pipe even though the command has not been run. If the command never runs, these pipes will never be closed. Change it so that pipes are only opened when the commands are being executed.

Redirection operators don't quite work like shell.

Currently, the redirection operations are not ordered. For example, if I want to redirect stdout and stderr to a file log_file, the shell command would be:

command >> log_file 2>&1

One would think that the same command in subprocess would look like:

command{"command"} >> "log_file" >= subprocess::out();

But that is not the case. This command redirects stdout to log_file and stderr to stdout. Since the operations are unordered and you are assigning file descriptors, the command should be:

command{"command"} >> "log_file" >= "log_file";

This also causes an additional problem that subprocess::command would treat the same file as two different objects. This should be rectified with immediate effect.

Error in the documention

Hello,

In the documentation, this sample doesn't work :

subprocess::command cmd("touch " + file_path.string()).run();
#include <iostream>
#include <filesystem>
#include <subprocess/subprocess.hpp>

using namespace subprocess::literals;

int main()
{
    std::filesystem::path file_path{"/tmp/file.txt"};
    
    // error
    //subprocess::command cmd("touch " + file_path.string()).run();

    // ok
    subprocess::command cmd2("touch " + file_path.string());
    cmd2.run();
}

When I try to compile, I get this errror on gcc 11 ๐Ÿ‘

main.cpp: In function 'int main()':
main.cpp:12:59: error: expected ',' or ';' before '.' token
   12 |     subprocess::command cmd("touch " + file_path.string()).run();
      |                                                         

Is this an error in the documentation?

Jean-Marc

Support for C++14/C++11

The library only uses std::filesystem::path from C++17. That is just used for differentiating overloads for variable/file redirection.

We can port the library to comply with C++14 standard as well.

Allow output capture into a stream

Currently, stdout/stderr can only be captured into std::string. This can be a problem for programs that dump a huge amount of output. There should be an alternate option to capture the output into an ofstream and read it whenever needed.

I am currently undecided on the syntax since std::ofstream doesn't support creation of streams from file descriptors.

Support shell-like tokenization

Currently, commands need to be written as delimited tokens:

auto cmd = command{"ls", "-l"} | command{"awk", "{print $1}"};

In addition to the above syntax, the following alternative should be available:

auto cmd = command{"ls -l"} | command{"awk '{print $1}'"};

posix_process reads stdout before stderr

A subtle issue with variable_descriptor is that it attempts to read all the data at once in the end. As opposed to whenever the data is available. Proposed design:

posix_process would poll the given output fds for output. Once that is done, the read method will be called on them directly.

Make the library header only

Look into organising the library such that it can be moved into a single header.
One problem with this is the hiding of C includes from the user space. Would need to figure out a way to "undefine" these headers at the end of the source file.

Add support for setting options and exporting environment variables

Something close to the following syntax can be supported to set bash-like options and export environment variables:

{
    subprocess::set(pipefail);
    subprocess::export("var_name", "var_value");
    return (subprocess::command{"random_cmd"} | subprocess::command{"another_cmd"}).run();
}
// pipefail is automatically unset
// var_name is automatically unset

Robust integration testing

Should probably stop testing with just ls | awk...

Need to write commands that can be run as tests across mac and linux.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.