Code Monkey home page Code Monkey logo

libtap's Introduction

NAME

libtap - Write tests in C

SYNOPSIS

#include <tap.h>

int main () {
    plan(5);
    int bronze = 1, silver = 2, gold = 3;
    ok(bronze < silver, "bronze is less than silver");
    ok(bronze > silver, "not quite");
    is("gold", "gold", "gold is gold");
    cmp_ok(silver, "<", gold, "%d <= %d", silver, gold);
    like("platinum", ".*inum", "platinum matches .*inum");
    done_testing();
}

results in:

1..5
ok 1 - bronze is less than silver
not ok 2 - not quite
#   Failed test 'not quite'
#   at t/synopsis.c line 7.
ok 3 - gold is gold
ok 4 - 2 <= 3
ok 5 - platinum matches .*inum
# Looks like you failed 1 test of 5 run.

DESCRIPTION

tap is an easy to read and easy to write way of creating tests for your software. This library creates functions that can be used to generate it for your C programs. It is implemented using macros that include file and line info automatically, and makes it so that the format message of each test is optional. It is mostly based on the Test::More Perl module.

INSTALL

On Unix systems:

$ make
$ make install

For more detailed installation instructions (eg, for Windows), see INSTALL.

FUNCTIONS

  • plan(tests)

  • plan(NO_PLAN)

  • plan(SKIP_ALL);

  • plan(SKIP_ALL, fmt, ...)

    Use this to start a series of tests. When you know how many tests there will be, you can put a number as a number of tests you expect to run. If you do not know how many tests there will be, you can use plan(NO_PLAN) or not call this function. When you pass it a number of tests to run, a message similar to the following will appear in the output:

    1..5
    

    If you pass it SKIP_ALL, the whole test will be skipped.

  • ok(test)

  • ok(test, fmt, ...)

    Specify a test. the test can be any statement returning a true or false value. You may optionally pass a format string describing the test.

    ok(r = reader_new("Of Mice and Men"), "create a new reader");
    ok(reader_go_to_page(r, 55), "can turn the page");
    ok(r->page == 55, "page turned to the right one");
    

    Should print out:

    ok 1 - create a new reader
    ok 2 - can turn the page
    ok 3 - page turned to the right one
    

    On failure, a diagnostic message will be printed out.

    not ok 3 - page turned to the right one
    #   Failed test 'page turned to the right one'
    #   at reader.c line 13.
    
  • is(got, expected)

  • is(got, expected, fmt, ...)

  • isnt(got, unexpected)

  • isnt(got, unexpected, fmt, ...)

    Tests that the string you got is what you expected. with isnt, it is the reverse.

    is("this", "that", "this is that");
    

    prints:

    not ok 1 - this is that
    #   Failed test 'this is that'
    #   at is.c line 6.
    #          got: 'this'
    #     expected: 'that'
    
  • cmp_ok(a, op, b)

  • cmp_ok(a, op, b, fmt, ...)

    Compares two ints with any binary operator that doesn't require an lvalue. This is nice to use since it provides a better error message than an equivalent ok.

    cmp_ok(420, ">", 666);
    

    prints:

    not ok 1
    #   Failed test at cmpok.c line 5.
    #     420
    #         >
    #     666
    
  • cmp_mem(got, expected, n)

  • cmp_mem(got, expected, n, fmt, ...)

    Tests that the first n bytes of the memory you got is what you expected. NULL pointers for got and expected are handled (if either is NULL, the test fails), but you need to ensure n is not too large.

    char *a = "foo";
    char *b = "bar";
    cmp_mem(a, b, 3)
    

    prints

    not ok 1
    #   Failed test at t/cmp_mem.c line 9.
    #     Difference starts at offset 0
    #          got: 0x66
    #     expected: 0x62
    
  • like(got, expected)

  • like(got, expected, fmt, ...)

  • unlike(got, unexpected)

  • unlike(got, unexpected, fmt, ...)

    Tests that the string you got matches the expected extended POSIX regex. unlike is the reverse. These macros are the equivalent of a skip on Windows.

    like("stranger", "^s.(r).*\\1$", "matches the regex");
    

    prints:

    ok 1 - matches the regex
    
  • pass()

  • pass(fmt, ...)

  • fail()

  • fail(fmt, ...)

    Speciy that a test succeeded or failed. Use these when the statement is longer than you can fit into the argument given to an ok() test.

  • dies_ok(code)

  • dies_ok(code, fmt, ...)

  • lives_ok(code)

  • lives_ok(code, fmt, ...)

    Tests whether the given code causes your program to exit. The code gets passed to a macro that will test it in a forked process. If the code succeeds it will be executed in the parent process. You can test things like passing a function a null pointer and make sure it doesnt dereference it and crash.

    dies_ok({abort();}, "abort does close your program");
    dies_ok({int x = 0/0;}, "divide by zero crash");
    lives_ok({pow(3.0, 5.0);}, "nothing wrong with taking 3**5");
    

    On Windows, these macros are the equivalent of a skip.

  • done_testing()

    Summarizes the tests that occurred and exits the main function. If there was no plan, it will print out the number of tests as.

    1..5
    

    It will also print a diagnostic message about how many failures there were.

    # Looks like you failed 2 tests of 3 run.
    

    If all planned tests were successful, it will return 0. If any test fails, it will return 1. If they all passed, but there were missing tests, it will return 2.

  • diag(fmt, ...)

    print out a message to the tap output on stdout. Each line is preceeded by a "# " so that you know its a diagnostic message.

    diag("This is\na diag\nto describe\nsomething.");
    

    prints:

    # This is
    # a diag
    # to describe
    # something
    

    ok() and this function return an int so you can use it like:

    ok(0) || diag("doh!");
    
  • skip(test, n)

  • skip(test, n, fmt, ...)

  • end_skip

    Skip a series of n tests if test is true. You may give a reason why you are skipping them or not. The (possibly) skipped tests must occur between the skip and end_skip macros.

    skip(TRUE, 2);
    ok(1);
    ok(0);
    end_skip;
    

    prints:

    ok 1 # skip
    ok 2 # skip
    
  • todo()

  • todo(fmt, ...)

  • end_todo

    Specifies a series of tests that you expect to fail because they are not yet implemented.

    todo()
    ok(0);
    end_todo;
    

    prints:

    not ok 1 # TODO
    #   Failed (TODO) test at todo.c line 7
    
  • BAIL_OUT()

  • BAIL_OUT(fmt, ...)

    Immediately stops all testing.

    BAIL_OUT("Can't go no further");
    

    prints

    Bail out!  Can't go no further
    

    and exits with 255.

libtap's People

Contributors

alyptik avatar autarch avatar beatgammit avatar bricef avatar fredmorcos avatar gonzus avatar horgh avatar jeffsw avatar mlafeldt avatar outergod avatar quentusrex avatar sevko avatar sloanebernstein avatar sni avatar sputtene avatar sunnythepooh avatar tekknolagi avatar travispaul avatar zorgnax avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

libtap's Issues

Modern compilers issue warnings when running make

In file included from t/diesok.c:1:0:
t/diesok.c: In function ‘main’:
t/diesok.c:7:19: warning: variable ‘x’ set but not used [-Wunused-but-set-variable]
     lives_ok({int x; x = 3/7;}, "this is a perfectly fine statement");
                   ^
./tap.h:96:13: note: in definition of macro ‘dies_ok_common’
             code                                            \
             ^
t/diesok.c:7:5: note: in expansion of macro ‘lives_ok’
     lives_ok({int x; x = 3/7;}, "this is a perfectly fine statement");
     ^
t/diesok.c:7:19: warning: variable ‘x’ set but not used [-Wunused-but-set-variable]
     lives_ok({int x; x = 3/7;}, "this is a perfectly fine statement");
                   ^
./tap.h:106:14: note: in definition of macro ‘dies_ok_common’
             {code}                                          \
              ^
t/diesok.c:7:5: note: in expansion of macro ‘lives_ok’
     lives_ok({int x; x = 3/7;}, "this is a perfectly fine statement");
     ^
gcc   t/diesok.o libtap.a  -o t/diesok

Even though all these warnings are completely harmless, they are pretty ugly.

Used compiler: gcc-5.real (Ubuntu 5.2.1-22ubuntu2) 5.2.1 20151010

make check fails without make install

The INSTALL file says to run make,make check, make install.

On a system without this library installed, try
$ sh -c 'git clone http://github.com/zorgnax/libtap && cd libtap && make && make check'

all tests fail because libtap.so can't be found

$ cd libtap && LD_LIBRARY_PATH=$PWD make check

all tests work

The easiest way imho is to run the tests using the static library

Enhancement: add "All tests passed!" output from exit_status()

I would like to suggest adding the following code just after line 752 in tap.c

if (retval == 0) {
printf("All tests passed!\n");
}

The effect of this would be to let the user know that no tests failed. This is helpful for console programs that produce a lot of output, so that the user doesn't have to scroll back through all of it to see what happened. In other cases, exit_status() prints informative information, so it would be good to have information printed about tests passing as well.

Please consider licensing as GPLv2+

The current license makes it incompatible with code under the Apache 2.0 license. Allowing version 2+ of the GPL would make it compatible.

use of plan() causes bool functions that don't return anything to work both as true and false

I have been using libtap in my Introduction to Programming with C class at UCF this semester. Overall we are very happy with it. Thanks!

However, students complained that for one of my test suites all the tests passed, even though they had no code in a predicate function (i.e., one that should return a _Bool). Of course, that is an incorrect implementation, but often students are omitting the return statement in a function, and so not detecting that is a serious problem when not detected. I was eventually able to find that the tests improperly passed in this manner only if the testing code called plan(). If the call to plan() is omitted, then the tests work properly, checking the return value. I have boiled this down to the attached files, which I thiink clearly show the problem. This happens under Windows 10 using gcc (GCC) 5.4.0 running under cygwin and also under MingW gcc.

The attached zip file has 3 files. bfun.c has a Boolean-valued function named bfun in it that doesn't return a value. bfun.h is a header file for that. test_bfun.c has a main() function, which shows the problem. If you comment out the call to plan() in test_bfun.c, you get the expected detection of incorrect behavior in bfun().

bfun-example.zip

Semicolon at end of cmp_mem macro

One of the features of this library is that it allows you to use the assertions as parts of if statements. e.g.

int rv = 1;
if (!cmp_ok(rv, "==", 0, "should return 0")) {
  diag("It didn't produce 0, that's bad");
}

This works great, except for the cmp_mem macro, because it is defined as:

#define cmp_mem(...)     cmp_mem_at_loc(__FILE__, __LINE__, __VA_ARGS__, NULL);

There's a semicolon at the end, which means that you get a compile-time error if you try to do:

if (!cmp_mem(actual, expected, length, "message")) {
  // Different
}

I believe this only needs that semicolon removed and it will work correctly.

Problem with ok macro

I was using the ok macro to compare two pointer type variables, like this:

ok(fff.call_history[0] == my_func, "blah_blah_msg");

and it was causing the following warning from gcc:

../../../helpers/tap/tap.h:48:56: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
#define ok(...) ok_at_loc(FILE, LINE, (int) VA_ARGS, NULL)
^
..//.build/test/tests_eppos_update_logic.c:79:2: note: in expansion of macro ‘ok’

Acording to the manual, you can use it to test anything that returns a true or false, but it is not the case.
The problem (I think) is the definition of the mentioned macro:

#define ok(...) ok_at_loc(__FILE__, __LINE__, (int) __VA_ARGS__, NULL)

When the preprocessor expands the macro, you get this:

ok_at_loc("file.c", 291, (int) fff.call_history[5] == EpposUpdate_InstallUpdate, "msg", ((void *)0));

So it is casting a function pointer to int, then compare it with a pointer to function, which causes the warning shown above. The only way to eliminate the warning is to sorround the test with parentheses:

ok( (fff.call_history[0] == my_func), "blah_blah_msg");

It is fine as long as you compare ints, but in any other case, you need the parentheses.
Although not an issue per se, probably it would be great to add that recommendation to the documentation.

Hope it helps,
Fernando

plan is no more

The synopsis says plan(5), however, that function appears to be called plan_tests().

Version?

I submitted libtap to the AUR, but I didn't know what the version of this library was so I made one up. It would help a lot if you would tag releases so they show up under "Releases" on github: git tag 0.1.0 or git tag v0.1.0. If you do that, I can make another PKGBUILD that targets released versions instead of development versions.

If my pull request (#17) is acceptable, I'll switch my PKGBUILD to point to this repository instead of my fork.

exit_status()

I had some time to waste today...

exit(failed_tests) & similar are not ok - C11 says you can exit(EXIT_SUCCESS) or exit(EXIT_FAILURE), unix&windows have been using only the low byte since forever (see posix/msdn), meaning that eg a program that calls "exit(256)" appears to have succeeded (tried just now on linux).

I thought that exit codes>128 mean "killed by signal", but apparently I misremembered.

Only affects testsuites with >255 failures (or a number of failures that's a multiple of 256, which is funnier), but anyway... return min(retval,255)

How can I use mock in libtap?

There are many complex functions in function testing, how can we use the piling method to give them specific return values.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.