http://bmizerany.github.com/roundup/roundup.1.html

It says tests are searched for in $PWD/t/*-test.sh. The "t/" should be removed.

I have a test plan that creates a GPG key in before. I had upgraded GPG since the last time I'd run GPG, so the running gpg-agent didn't match /bin/gpg. This caused an error whenever it tried to run GPG. That is to say, I have a before function that started erroring for reasons unrelated to my test suite or the code being tested.

roundup handled this error in before by aborting, pretending that any pending tests or test plans did not exist.

This meant that the test plan with the failing before showed up as having 0 tests. I, being an alert developer, knew that I had no test plans with 0 tests, and that roundup must be wrong.

Upon hacking roundup to deal with this in a better way, I discovered that I had several other tests that had been failing for much longer; I had about 20 tests that roundup had silently been skipping for many months (as a different before/after had been failing)!

This is a critical bug for a testing framework. Title is only a slight exaggeration (rm -rf on somewhere important would be worse).

This test scenario:

it_waits() {
    sleep 1
}

seems to break roundup:

  it_waits:                                        [PASS]
Terminated: 15
MacBook-Air-de-Alexandre:jarvis alex$ /usr/local/bin/roundup: line 307: 44176 Terminated: 15          for roundup_p in $roundup_plans;
do
    ( roundup_desc=$(basename "$roundup_p" -test.sh); function describe () 
    { 
        roundup_desc="$*"
    }; function before () 
    { 
        :
    }; function after () 
    { 
        :
    }; roundup_plan=$(
            grep "^it_.*()" $roundup_p           |
            sed "s/\(it_[a-zA-Z0-9_]*\).*$/\1/g"
        ); . ./$roundup_p; printf "d %s" "$roundup_desc" | tr "\n" " "; printf "\n"; for roundup_test_name in $roundup_plan;
    do
        ( { { set -x; before; } >&"$roundup_tmp/$roundup_test_name"; set +x; } >&/dev/null; set -E; trap 'rc=$?; set +x; set -o | grep "errexit.*on" >/dev/null && exit $rc' ERR; before; set +e; ( set -xe; $roundup_test_name ) > "$roundup_tmp/$roundup_test_name" 2>&1; roundup_result=$?; set -e; after; if [ "$roundup_result" -ne 0 ]; then
            printf "f";
        else
            printf "p";
        fi; printf " $roundup_test_name\n" );
    done );
done
     44177                       | roundup_summarize

Is it possible now, or conceivably possible, to run (& then re-run) just one of the "it" tests in a plan script?

Or is (only possible) the solution to put each test in a separate script?

Is there a way to test my script has a non-zero exit code i.e. it errors as expected under certain conditions? It seems that roundup will always exit?

The following test:

it_regexp() {
  output="Hello from your rack app is awesome"
  expected="Hello from your rack app"
  [[ ${output} =~ ${expected} ]]
}

returns this error:

    + output=Hello from your rack app is awesome
    + expected=Hello from your rack app
    + [[ Hello from your rack app is awesome =~ Hello from your rack app ]]
    /usr/local/bin/roundup: 1: [[: not found

I tried nesting [[...]] within an if/fi block but get the same error.

Yet, inside my bash terminal, the script works:

# output="Hello from your rack app is awesome"
# expected="Hello from your rack app"
# [[ ${output} =~ ${expected} ]] && echo 'yes'
yes

Before should setup a test case condition.
After should clean up that condition to eliminate side effects put in by "before". Therefore, it is typical to pass state from before to after which cleans up the mess. For instance, creating a scratch directory in before with "mktemp -d" should be deleted by after... This doesn't work when roundup tests the sanity of before/after and you end up with side effects.

before() {
    OUT=$(mktemp -d "/tmp/roundup.test.XXXXX")
}

after() {
    rm -rf ${OUT}
}

https://itsbonus.heroku.com/p/2010-11-01-roundup

The owner of itsbonus.heroku.com has configured their website improperly. To protect your information from being stolen, Firefox has not connected to this website.

This site uses HTTP Strict Transport Security (HSTS) to specify that Firefox may only connect to it securely. As a result, it is not possible to add an exception for this certificate.

The link to http://bmizerany.github.com/roundup is not working anymore.

All the install seemed very complicated so I tried if I could simply

curl https://raw.github.com/bmizerany/roundup/master/roundup.sh > /usr/local/bin/roundup
chmod +x /usr/local/bin/roundup

and it worked....
so please add this as "quick and dirty install" or whatever you would name this,
but I think its very simple and helps users that are not comfortable with running make/confugure just to try out a bash script...

Hi, I've been using roundup and it's been working great, but i'd like to consolidate/abstract a lot of similar tests using helper functions and common variables. Is roundup the right place for something like a 'test_helper', or is this outside the scope of roundup? I'd be willing to work on a patch for this (something like a --helper option), unless of course there's a better way that you recommend.

thanks.

When no plans are specified, the documentation says that roundup will run $PWD/t/*-test.sh when in fact it runs $PWD/*-test.sh. Should the documentation be fixed to match the behaviour, or the behaviour be fixed to match the documentation?

After running ./configure, running make on OpenBSD (tested on 6.5) results in the following syntax errors:

*** Parse error in /home/linetrace/Projects/roundup: Need an operator in 'SHOCCO' (Makefile:53)
*** Parse error: Need an operator in 'endif' (Makefile:55)
*** Parse error: Need an operator in 'RONN' (Makefile:58)
*** Parse error: Need an operator in 'else' (Makefile:60)
*** Parse error: Need an operator in 'endif' (Makefile:62)
*** Parse error: Need an operator in 'RONN' (Makefile:67)
*** Parse error: Need an operator in 'else' (Makefile:69)
*** Parse error: Need an operator in 'endif' (Makefile:73)

This is because, in OpenBSD's make (per the man page:

Makefile inclusion, conditional structures and for loops reminiscent of the C programming language are provided in make. All such structures are identified by a line beginning with a single dot (‘.’) character.

Adding the preceding dot (.) character to the conditional structures resolves the syntax errors. However, running make then fails with the following error (because GNU make's force targets don't seem to be supported by OpenBSD's make):

make: don't know how to make FORCE (prerequisite of: roundup)

Removing FORCE from roundup: roundup.sh FORCE resolves this issue.

On Debian based Linux systems - and basically every system, where /bin/sh links to dash - you will receive the above error message, since dash doesn't support the ERR SIGNAL_SPEC

A quick workaround would be, to replace the shebang of roundup.sh with the direct use of bash:

!/bin/bash

https://bmizerany.github.com/roundup in repo description is 404, it should be replaced with
https://bmizerany.github.io/roundup

And the link to this page can give a taste how it looked like
https://bmizerany.github.io/roundup/roundup.5.html

P.S. Found this repo from https://github.com/felipec/sharness#alternatives

When attempting to run roundup on OpenBSD (6.5-release, in my case), one receives the following error:

./roundup[307]: set: -E: unknown option

This is because /bin/sh on OpenBSD does not support the -E option, only the -e option.

None of the examples show how to source a file into the test file, say for testing a set of functions.

I tried simply using "source ../lib/functions.sh" at the start of the test script, but got "/usr/local/bin/roundup: 3: ./functions-test.sh: source: not found".

Changing the line to ". ../lib/functions.sh" got further, but errored with:

". ../lib/functions.sh

• function myfunc {
  /usr/local/bin/roundup: 3: ../lib/functions.sh: function: not found"

I also tried adding both of these to a before() call, but neither worked.

How are you supposed to source functions for testing when they're defined in other files?