freebsd / atf Goto Github PK

View Code? Open in Web Editor NEW

126.0 126.0 44.0 12.69 MB

Libraries to write tests in C, C++ and shell

License: Other

Shell 11.72% Awk 1.01% C++ 27.16% C 45.22% Makefile 3.49% M4 5.76% Roff 5.64%

atf's People

Contributors

Stargazers

Watchers

Forkers

s5unty cemeyer brd ashang stafwag brooksdavis shivansh wellsie ckhardin marka63 floppym gaybro8777 linruohan ngie-eign richiewoo realnedsanders saucec0de msgpo arichardson lxngoddess5321 asellappen kevans91 tubbz-alt jannick0 asomers chivvychen jkoshy nxtbsd enigbsd mmalewski antoniprzybylik dhgutteridge kahadoc rillig ludoplex korli seanpm2001 kaltest1 lwhsu fel1x-developer apteryks ncopa ihoro kastningbrandon

atf's Issues

atf-sh -h doesn't do anything

Originally from (misfiled) freebsd/kyua#89 by yaneurabeya.

The manpage says that atf-sh -h does the following:

-h Shows a short summary of all available options and their purpose.

However, atf-sh -h says the option's unknown:

% /usr/local/bin/atf-sh -h
atf-sh: ERROR: Unknown option -h.
atf-sh: See atf-sh(1) for usage details.
% man atf-sh
ATF-SH(1)               FreeBSD General Commands Manual              ATF-SH(1)

NAME
     atf-sh -- interpreter for shell-based test programs

SYNOPSIS
     atf-sh script
     atf-sh -h

DESCRIPTION
     atf-sh is an interpreter that runs the test program given in script after
     loading the atf-sh-api(3) library.

     atf-sh is not a real interpreter though: it is just a wrapper around the
     system-wide shell defined by the `atf_shell' configuration value in
     atf-config(1).  atf-sh executes the interpreter, loads the atf-sh-api(3)
     library and then runs the script.

     Scripts using atf-sh-api(3) should start with:

           #! /usr/bin/env atf-sh

     The following options are available:

     -h    Shows a short summary of all available options and their purpose.

SEE ALSO
     atf-config(1), atf-sh-api(3), atf(7)

FreeBSD 10.0                   October 15, 2013                   FreeBSD 10.0

% pkg info | grep atf
atf-0.20_1                     C, C++ and shell libraries to write ATF-compliant test programs
kyua-atf-compat-0.2_2          Kyua (automated testing framework) - ATF compatibility tools
% uname -a
FreeBSD freebsd-10-x64.localdomain 10.0-STABLE FreeBSD 10.0-STABLE #0 7905925(stable/10): Wed Jun 11 14:10:17 PDT 2014     [email protected]:/usr/obj/usr/src/sys/GENERIC-WITHOUT-WITNESS  amd64
%

atf-c++ shouldn't catch exceptions from test cases

atf-c++/tests.cpp:wrap_body() catches exceptions from the test case body. Assuming that the test framework executes each test in a child process, this should not be necessary. In fact, if any exception is thrown, the easiest way to debug what went wrong is if there is a crash dump. But the try..catch statements in this method preclude that from ever occurring.

Add require.diskspace

If a test wants to create a huge temporary file, it should be able to tell the runtime engine about that so that we bail out early if not enough space is available.

Suggestion from Martin Husemann over private email.

atf-0.22 has been released, but not tagged

Just noting that the 0.22 release was done over a year ago, but the tag hasn't been added in github.

atf-check_test.sh workarounds don't work for rebranded versions of Darwin/FreeBSD/Linux; \e should be replaced with a more standard spelling, e.g. \033

The code in atf-check_test.sh implement workarounds for OSes because...

# XXX Ugly hack to workaround the lack of \e in FreeBSD. Also, \e doesn't
# seem to work as expected in Linux. Look for a nicer solution.

"\e" is not in the POSIX standard, i.e. is only implemented on NetBSD AFAICT. It would be better to use the full spelling for "\e", i.e. "\033".

typo in shell functions

called form should be called from.

ATF macros should not reevaluate arguments

Some macros in the atf-c and atf-c++ libraries evaluate the arguments given to the more than once. This only happens when there is a test failure, but ideally we would avoid this in case the expressions have side-effects.

I'm not sure if this is doable in C nor in C++ without the use of the new auto keyword in C++11. But we should give this some more thought and add some tests.

request for release 0.22

the 0.21 was released almost 10 years ago. There are around 50 commits since then.

Would be nice to have a new release.

three performance issues ?

atf-sh/atf-check.cpp:592]: (performance) Function parameter 'oc' should be passed by const reference.

I couldn't find the source code for the next two:

tools/user.cpp:55]: (performance) Function parameter 'ids' should be passed by const reference.
tools/parser.cpp:43]: (performance) Function parameter 'msg' should be passed by const reference.

unused variable errors in lib/atf/libatf-c/macros_test:use

Under FreeBSD stable/10 and CURRENT, the lib/atf/libatf-c/macros_test:use test fails because of unused variable errors. The easiest solution is to add references to those variables, as shown in this patch:

--- contrib/atf/atf-c/macros_h_test.c 2013-10-17 15:43:33.000000000 -0600
+++ contrib/atf/atf-c/macros_h_test.c 2014-08-21 15:12:11.000000000 -0600
@@ -101,3 +101,12 @@
ATF_TC_BODY(TEST_MACRO_3, tc) { if (tc != NULL) {} }
atf_tc_t _test_name_3 = &ATF_TC_NAME(TEST_MACRO_3);
void (_body_3)(const atf_tc_t *) = ATF_TC_BODY_NAME(TEST_MACRO_3);
+
+ATF_TP_ADD_TCS(tp)
+{

```
   ATF_TP_ADD_TC(tp, TEST_MACRO_1);
```
```
   ATF_TP_ADD_TC(tp, TEST_MACRO_2);
```
```
   ATF_TP_ADD_TC(tp, TEST_MACRO_3);
```
```
   return atf_no_error();
```
+}

atf-c/utils.c:atf_utils_wait is not multiprocess safe

Filed from freebsd/kyua#87 by yaneurabeya:

This code is writing to a non-temporary file with a constant filename in order to capture the exec output. This is not multiprocess/multithreading safe:

381 void
382 atf_utils_wait(const pid_t pid, const int exitstatus, const char *expout,
383                const char *experr)
384 {
385     int status;
386     ATF_REQUIRE(waitpid(pid, &status, 0) != -1);
387
388     atf_utils_cat_file("atf_utils_fork_out.txt", "subprocess stdout: ");
389     atf_utils_cat_file("atf_utils_fork_err.txt", "subprocess stderr: ");
390
391     ATF_REQUIRE(WIFEXITED(status));
392     ATF_REQUIRE_EQ(exitstatus, WEXITSTATUS(status));
393
394     const char *save_prefix = "save:";
395     const size_t save_prefix_length = strlen(save_prefix);
396
397     if (strlen(expout) > save_prefix_length &&
398         strncmp(expout, save_prefix, save_prefix_length) == 0) {
399         atf_utils_copy_file("atf_utils_fork_out.txt",
400                             expout + save_prefix_length);
401     } else {
402         ATF_REQUIRE(atf_utils_compare_file("atf_utils_fork_out.txt", expout));
403     }
404
405     if (strlen(experr) > save_prefix_length &&
406         strncmp(experr, save_prefix, save_prefix_length) == 0) {
407         atf_utils_copy_file("atf_utils_fork_err.txt",
408                             experr + save_prefix_length);
409     } else {
410         ATF_REQUIRE(atf_utils_compare_file("atf_utils_fork_err.txt", experr));
411     }
412
413     ATF_REQUIRE(unlink("atf_utils_fork_out.txt") != -1);
414     ATF_REQUIRE(unlink("atf_utils_fork_err.txt") != -1);
415 }

Assessment of the difficulty in porting CPU architecture for atf

Hello everyone! I am working on implementing a tool to assess the complexity of CPU architecture porting. It primarily focuses on RISC-V architecture porting. In fact, the tool may have an average estimate of various architecture porting efforts.My focus is on the overall workload and difficulty of transplantation in the past and future,even if a project has already been ported.As part of my dataset, I have collected the atf project. I would like to gather community opinions to support my assessment. I appreciate your help and response! Based on scanning tools, the porting complexity is determined to be simple, with a small amount of code related to the CPU architecture in the project. Is this assessment accurate?I look forward to your help and response.

atf-run is still referenced in atf code

From rodrigc:

atf-run was removed in atf-0.20, but it is still referenced internally in the code:

atf-c/detail/tp_main.c
atf-c++/tests.cpp
atf-c++/detail/process.cpp
atf-sh/libatf-sh.subr
test-programs/result_test.sh

allow atf to be used in a C-only environment

Discussed in https://reviews.freebsd.org/D7597. It looks like FreeBSD will disable tests when building without C++, as some ATF components require a C++ compiler. I don't believe this is a high priority as FreeBSD does build a C++ toolchain by default, but if it's straightforward to migrate (a minimal set of) ATF components to C we could enable/disable tests on a more fine-grained basis.

check_test:build_cpp: test failure

 $ make installcheck
...
atf-c/check_test:build_cpp  ->  failed: 2 checks failed; see output for more details  [0.004s]
atf-c++/check_test:build_cpp  ->  failed: atf-c/utils.c:320: (fd = open(file, O_RDONLY | O_CLOEXEC)) != -1 not met  [0.004s]

Standard output:
Looking for '-o.*test.p' in '> -o test.p test.c'
Looking for 'test.c' in '> -o test.p test.c'

Standard error:
Files left in work directory after failure: result, stderr, stdout, test.c
===> atf-c/check_test:build_cpp
Result:     failed: atf-c/utils.c:320: (fd = open(file, O_RDONLY | O_CLOEXEC)) != -1 not met
Start time: 2024-03-11T17:06:21.610078Z
End time:   2024-03-11T17:06:21.613030Z
Duration:   0.003s

Standard output:
Looking for '-o.*test.p' in '> -o test.p test.c'
Looking for 'test.c' in '> -o test.p test.c'

Standard error:
Files left in work directory after failure: result, stderr, stdout, test.c
===> Failed tests
atf-c++/check_test:build_cpp  ->  failed: atf-c/utils.c:320: (fd = open(file, O_RDONLY | O_CLOEXEC)) != -1 not met  [0.004s]
atf-c/check_test:build_cpp  ->  failed: atf-c/utils.c:320: (fd = open(file, O_RDONLY | O_CLOEXEC)) != -1 not met  [0.003s]
===> Summary
Results read from /home/minsoochoo/.kyua/store/results.home_minsoochoo_local_tests_atf.20240311-170621-536675.db
Test cases: 383 total, 0 skipped, 0 expected failures, 0 broken, 2 failed
Start time: 2024-03-11T17:06:21.544862Z
End time:   2024-03-11T17:06:33.056275Z
Total time: 10.743s
*** Error code 1

atf-c-api example for ATF_TP_ADD_TCS/ATF_TP_ADD_TC is incorrect

It uses tcs for the second argument for ATF_TP_ADD_TC instead of tp. This commit fixed the problem on FreeBSD: https://svnweb.freebsd.org/base?view=revision&revision=314793 .

Fix references to atf-config(1)

Some manual pages still reference the removed atf-config(1) manual page. Should reinstantiate the documentation for the supported environment variables in some way.

atf-c/macros/use: fails with clang -Wunused on FreeBSD head

Both the base and port testcases for atf appear to be failing on FreeBSD CURRENT because of clang -Wunused ( http://kyua3.nyi.freebsd.org/head/data/0-LATEST/results/local_atf_atf-c_macros_test_use.html ):

/usr/tests/local/atf/atf-c/macros_h_test.c:86:1: error: unused variable 'atfu_TEST_MACRO_1_tc_pack' [-Werror,-Wunused-const-variable]
ATF_TC(TEST_MACRO_1);
^
/usr/local/include/atf-c/macros.h:61:26: note: expanded from macro 'ATF_TC'
    static atf_tc_pack_t atfu_ ## tc ## _tc_pack = { \
                         ^
:41:1: note: expanded from here
atfu_TEST_MACRO_1_tc_pack
^
/usr/tests/local/atf/atf-c/macros_h_test.c:92:1: error: unused variable 'atfu_TEST_MACRO_2_tc_pack' [-Werror,-Wunused-const-variable]
ATF_TC_WITH_CLEANUP(TEST_MACRO_2);
^
/usr/local/include/atf-c/macros.h:73:26: note: expanded from macro 'ATF_TC_WITH_CLEANUP'
    static atf_tc_pack_t atfu_ ## tc ## _tc_pack = { \
                         ^
:66:1: note: expanded from here
atfu_TEST_MACRO_2_tc_pack
^
/usr/tests/local/atf/atf-c/macros_h_test.c:100:1: error: unused variable 'atfu_TEST_MACRO_3_tc_pack' [-Werror,-Wunused-const-variable]
ATF_TC_WITHOUT_HEAD(TEST_MACRO_3);
^
/usr/local/include/atf-c/macros.h:50:26: note: expanded from macro 'ATF_TC_WITHOUT_HEAD'
    static atf_tc_pack_t atfu_ ## tc ## _tc_pack = { \
                         ^
:93:1: note: expanded from here
atfu_TEST_MACRO_3_tc_pack
^
3 errors generated.
cc failed with exit code 1

The code will probably need to be modified to use the variable, or to just add __attribute__((__unused__)) around the declaration.

The descr meta-data field is not required anymore

atf-test-case.4 says that descr is required for a test case, while in fact, it was made optional in version 0.8.

While here, atf-test-case.4 promises that descr "will be shown to the user in reports", but when I ran atf-run on NetBSD, none of the test descriptions were listed in the raw report file, which started with application/X-atf-tps; version="3".

Add option to dump core with gdb after a program signals unexpectedly

I had to manually execute an ATF test program in a loop with gdb in order reproduce the issue noted in https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=215715 . I shouldn't have to do this -- the framework should do this for me.

$ cat cond_timedwait_race.gdbscript
set args cond_timedwait_race
run
generate-core-file
$ while gdb -batch -x cond_timedwait_race.gdbscript /usr/tests/lib/libthr/cond_test cond_timedwait_race; do :; done

Double-free in atf-c/map.c in low memory scenarios

Per https://svnweb.freebsd.org/base?view=revision&revision=299496 :

atf map: Fix double-free in low memory error path

If atf_list_append(, X, ) fails, X is freed.  Don't free it again.

If anyone wants to walk this patch upstream, be my guest.  I literally cannot
upstream it myself due to Google's stupid CLA.

Reported by:    Coverity
CID:        979936
Sponsored by:   EMC / Isilon Storage Division

Add build-time checks to atf-sh

The 0.7 release introduced build-time tests to atf-c and atf-c++, but not to atf-sh. Expose the functionality to the shell interface.

This will probably require writing an atf-build utility that exposes the C code and can be called from the shell.

Add ATF_CHECK* versions to atf-c++ to support non-fatal tests

The atf-c library has ATF_CHECK* and ATF_REQUIRE* whereas atf-c++ doesn't. Implement the useful non-fatal versions for feature parity.

execute_with_shell() failed on FreeBSD/MIPS with GCC 4.2.1

execute_with_shell() failed to convert C++ std::string into C const char* here:

sh_argv[0] = atf::env::get("ATF_SHELL", ATF_SHELL).c_str();

In result we have "ZZZZZZZZ" instead of "/bin/sh".
E.g. https://people.freebsd.org/~br/kyua_mips/bin_cat_cat_test_align.html

this workaround helps:

const std::string shell = atf::env::get("ATF_SHELL", ATF_SHELL).c_str();
sh_argv[0] = shell.c_str();

This problem found on FreeBSD/MIPS64EB platform (GCC 4.2.1).

atf-sh-api(3) doesn't mention atf-check(1)

Originally filed by yaneurabeya at freebsd/kyua#98

atf-sh-api(3) mentions atf_check, but it doesn't reference atf-check(1), which could be helpful for folks getting started with atf.

Why are tests being installed?

Currently when using make install I'm finding a lot of tests are being installed to the DESTDIR/PREFIX/tests. Is this intended?

When using --prefix=/usr it results in files installed to /usr/tests which seems wrong.

If this is intentional where does upstream consider a good place to install these tests?

Build failure with --enable-developer and g++ 13.2.1

$ g++ --version
g++ (Alpine 13.2.1_git20231014) 13.2.1 20231014
Copyright (C) 2023 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

./configure --enable-developer && make
...
g++ -DHAVE_CONFIG_H -I.     -g -O2  -D_FORTIFY_SOURCE=2 -Wall -Wcast-qual -Wextra -Wpointer-arith -Wredundant-decls -Wreturn-type -Wshadow -Wsign-compare -Wswitch -Wwrite-strings -g -Werror -Wctor-dtor-privacy -Wno-deprecated -Wno-non-template-friend -Wno-pmf-conversions -Wnon-virtual-dtor -Woverloaded-virtual -Wreorder -Wsign-promo -Wsynth -MT atf-c++/detail/auto_array_test.o -MD -MP -MF $depbase.Tpo -c -o atf-c++/detail/auto_array_test.o atf-c++/detail/auto_array_test.cpp &&\
mv -f $depbase.Tpo $depbase.Po
In file included from atf-c++/detail/auto_array_test.cpp:26:
In destructor 'atf::auto_array<T>::~auto_array() [with T = test_array]',
    inlined from 'virtual void {anonymous}::atfu_tc_auto_array_scope::body() const' at atf-c++/detail/auto_array_test.cpp:102:5:
./atf-c++/detail/auto_array.hpp:98:9: error: 'static void test_array::operator delete [](void*)' called on pointer returned from a mismatched allocation function [-Werror=mismatched-new-delete]
   98 |         delete [] m_ptr;
      |         ^~~~~~~~~~~~~~~
In static member function 'static void* test_array::operator new [](size_t)',
    inlined from 'virtual void {anonymous}::atfu_tc_auto_array_scope::body() const' at atf-c++/detail/auto_array_test.cpp:100:53:
atf-c++/detail/auto_array_test.cpp:66:35: note: returned from 'void* operator new(std::size_t)'
   66 |         void* mem = ::operator new(size);
      |                     ~~~~~~~~~~~~~~^~~~~~
In destructor 'atf::auto_array<T>::~auto_array() [with T = test_array]',
    inlined from 'virtual void {anonymous}::atfu_tc_auto_array_copy::body() const' at atf-c++/detail/auto_array_test.cpp:126:5:
./atf-c++/detail/auto_array.hpp:98:9: error: 'static void test_array::operator delete [](void*)' called on pointer returned from a mismatched allocation function [-Werror=mismatched-new-delete]
   98 |         delete [] m_ptr;
      |         ^~~~~~~~~~~~~~~
In static member function 'static void* test_array::operator new [](size_t)',
    inlined from 'virtual void {anonymous}::atfu_tc_auto_array_copy::body() const' at atf-c++/detail/auto_array_test.cpp:118:54:
atf-c++/detail/auto_array_test.cpp:66:35: note: returned from 'void* operator new(std::size_t)'
   66 |         void* mem = ::operator new(size);
      |                     ~~~~~~~~~~~~~~^~~~~~
In destructor 'atf::auto_array<T>::~auto_array() [with T = test_array]',
    inlined from 'virtual void {anonymous}::atfu_tc_auto_array_copy_ref::body() const' at atf-c++/detail/auto_array_test.cpp:150:5:
./atf-c++/detail/auto_array.hpp:98:9: error: 'static void test_array::operator delete [](void*)' called on pointer returned from a mismatched allocation function [-Werror=mismatched-new-delete]
   98 |         delete [] m_ptr;
      |         ^~~~~~~~~~~~~~~
In static member function 'static void* test_array::operator new [](size_t)',
    inlined from 'virtual void {anonymous}::atfu_tc_auto_array_copy_ref::body() const' at atf-c++/detail/auto_array_test.cpp:142:54:
atf-c++/detail/auto_array_test.cpp:66:35: note: returned from 'void* operator new(std::size_t)'
   66 |         void* mem = ::operator new(size);
      |                     ~~~~~~~~~~~~~~^~~~~~
In destructor 'atf::auto_array<T>::~auto_array() [with T = test_array]',
    inlined from 'virtual void {anonymous}::atfu_tc_auto_array_release::body() const' at atf-c++/detail/auto_array_test.cpp:186:5:
./atf-c++/detail/auto_array.hpp:98:9: error: 'static void test_array::operator delete [](void*)' called on pointer returned from a mismatched allocation function [-Werror=mismatched-new-delete]
   98 |         delete [] m_ptr;
      |         ^~~~~~~~~~~~~~~
In static member function 'static void* test_array::operator new [](size_t)',
    inlined from 'virtual void {anonymous}::atfu_tc_auto_array_release::body() const' at atf-c++/detail/auto_array_test.cpp:179:40:
atf-c++/detail/auto_array_test.cpp:66:35: note: returned from 'void* operator new(std::size_t)'
   66 |         void* mem = ::operator new(size);
      |                     ~~~~~~~~~~~~~~^~~~~~
In destructor 'atf::auto_array<T>::~auto_array() [with T = test_array]',
    inlined from 'virtual void {anonymous}::atfu_tc_auto_array_reset::body() const' at atf-c++/detail/auto_array_test.cpp:212:5:
./atf-c++/detail/auto_array.hpp:98:9: error: 'static void test_array::operator delete [](void*)' called on pointer returned from a mismatched allocation function [-Werror=mismatched-new-delete]
   98 |         delete [] m_ptr;
      |         ^~~~~~~~~~~~~~~
In static member function 'static void* test_array::operator new [](size_t)',
    inlined from 'virtual void {anonymous}::atfu_tc_auto_array_reset::body() const' at atf-c++/detail/auto_array_test.cpp:202:40:
atf-c++/detail/auto_array_test.cpp:66:35: note: returned from 'void* operator new(std::size_t)'
   66 |         void* mem = ::operator new(size);
      |                     ~~~~~~~~~~~~~~^~~~~~
In destructor 'atf::auto_array<T>::~auto_array() [with T = test_array]',
    inlined from 'virtual void {anonymous}::atfu_tc_auto_array_assign::body() const' at atf-c++/detail/auto_array_test.cpp:237:5:
./atf-c++/detail/auto_array.hpp:98:9: error: 'static void test_array::operator delete [](void*)' called on pointer returned from a mismatched allocation function [-Werror=mismatched-new-delete]
   98 |         delete [] m_ptr;
      |         ^~~~~~~~~~~~~~~
In static member function 'static void* test_array::operator new [](size_t)',
    inlined from 'virtual void {anonymous}::atfu_tc_auto_array_assign::body() const' at atf-c++/detail/auto_array_test.cpp:228:54:
atf-c++/detail/auto_array_test.cpp:66:35: note: returned from 'void* operator new(std::size_t)'
   66 |         void* mem = ::operator new(size);
      |                     ~~~~~~~~~~~~~~^~~~~~
In destructor 'atf::auto_array<T>::~auto_array() [with T = test_array]',
    inlined from 'virtual void {anonymous}::atfu_tc_auto_array_assign_ref::body() const' at atf-c++/detail/auto_array_test.cpp:263:5:
./atf-c++/detail/auto_array.hpp:98:9: error: 'static void test_array::operator delete [](void*)' called on pointer returned from a mismatched allocation function [-Werror=mismatched-new-delete]
   98 |         delete [] m_ptr;
      |         ^~~~~~~~~~~~~~~
In static member function 'static void* test_array::operator new [](size_t)',
    inlined from 'virtual void {anonymous}::atfu_tc_auto_array_assign_ref::body() const' at atf-c++/detail/auto_array_test.cpp:254:54:
atf-c++/detail/auto_array_test.cpp:66:35: note: returned from 'void* operator new(std::size_t)'
   66 |         void* mem = ::operator new(size);
      |                     ~~~~~~~~~~~~~~^~~~~~
In destructor 'atf::auto_array<T>::~auto_array() [with T = test_array]',
    inlined from 'virtual void {anonymous}::atfu_tc_auto_array_access::body() const' at atf-c++/detail/auto_array_test.cpp:284:1:
./atf-c++/detail/auto_array.hpp:98:9: error: 'static void test_array::operator delete [](void*)' called on pointer returned from a mismatched allocation function [-Werror=mismatched-new-delete]
   98 |         delete [] m_ptr;
      |         ^~~~~~~~~~~~~~~
In static member function 'static void* test_array::operator new [](size_t)',
    inlined from 'virtual void {anonymous}::atfu_tc_auto_array_access::body() const' at atf-c++/detail/auto_array_test.cpp:277:49:
atf-c++/detail/auto_array_test.cpp:66:35: note: returned from 'void* operator new(std::size_t)'
   66 |         void* mem = ::operator new(size);
      |                     ~~~~~~~~~~~~~~^~~~~~
cc1plus: all warnings being treated as errors
make[1]: *** [Makefile:2673: atf-c++/detail/auto_array_test.o] Error 1
make[1]: Leaving directory '/var/ncopa/src/atf'
make: *** [Makefile:1382: all] Error 2

Update README

No offense but README does little for me (meaning a complete beginner on ATF) to understand basic structure of tests and the functionality of ATF.

Some further reference to information already existing would be great :)

Regards,
Harry

no way to wait for a process that exits on signal

If you fork a process that will exit on a signal, there is no helper function to allow you to check which signal the process exited w/. The _wait function will check if the child exited normally.

See attached test case.
waitsig_test.c.log

"set -e" causes atf-sh subshells to exit prematurely

commit 7d68b8a added "set -e" to libatf-sh.subr. I don't know what original problem it tried to fix, because atf seems to have been using a different bug tracker back then. But "set -e" has unexpected consequences. Among them, backgrounded subshells whose commands are considered to be unchecked by /bin/sh will terminate as soon as one of those commands exits nonzero. You don't notice the effect for foregrounded subshells because they are wrapped in a function (the ATF test case body) and that function's exit status is checked by libatf-sh.subr.

Simply disabling "set -e" does not cause any of atf's builtin testcases to fail. Probably, this is because the truly important changes in commit 7d68b8a were checking the exit status of ${_tcname}_body and ${_tcname}_cleanup.

The problem can be demonstrated by the atf test program below. Were it not for "set -e", all four of the background and foreground tests would timeout. But due to "set -e", the loop in background_false exits early and that test case passes.

#! /usr/libexec/atf-sh

atf_test_case background_false
background_false_head()
{
    atf_set "timeout" "1"
}
background_false_body()
{
    while true; do
        false
    done &
    wait $!
    echo "background loop exited $?"
}

atf_test_case background_true
background_true_head()
{
    atf_set "timeout" "1"
}
background_true_body()
{
    while true; do
        true
    done &
    wait $!
    echo "background loop exited $?"
}

atf_test_case foreground_false
foreground_false_head()
{
    atf_set "timeout" "1"
}
foreground_false_body()
{
    while true; do
        false
    done
    echo "foreground loop exited $?"
}

atf_test_case foreground_true
foreground_true_head()
{
    atf_set "timeout" "1"
}
foreground_true_body()
{
    while true; do
        true
    done
    echo "foreground loop exited $?"
}

atf_test_case nonloop
nonloop_head()
{
    atf_set "timeout" "1"
}
nonloop_body()
{
    false
    echo "false command exited $?"
}

atf_init_test_cases()
{
    atf_add_test_case background_false
    atf_add_test_case background_true
    atf_add_test_case foreground_false
    atf_add_test_case foreground_true
    atf_add_test_case nonloop
}

atf-c-api cannot expect both failure and timeout

From asomers on January 16, 2014 18:34:38

What steps will reproduce the problem? 1. Create an ATF testcase in C that will both fail and timeout
2. Add atf_tc_expect_fail() and atf_tc_expect_timeout() statements to the test
3. Run it What is the expected output? What do you see instead? The expected output is something like "expected failure: blah blah blah". Instead, this is the output:
timeout_and_fail:expect_both -> failed: Test case was expecting a failure but none were raised [0.008s] What version of the product are you using? On what operating system? kyua 0.8

uname -a

FreeBSD alans-fbsd-head 11.0-CURRENT FreeBSD 11.0-CURRENT #23 r260786M: Thu Jan 16 13:06:43 MST 2014 [email protected]:/vmpool/obj/usr/home/alans/freebsd/head/sys/GENERIC amd64 Please provide any additional information below. Attaching a test program that demonstrates the issues.

Attachment: timeout_and_fail.c Makefile Kyuafile

Original issue: http://code.google.com/p/kyua/issues/detail?id=76

Recommend (expected, actual) syntax

Change the manual pages to recommend the more standard (expected, actual) ordering to check/require arguments.

Missing required programs on some of ATFs own tests

In atf/libatf-c++/Kyuafile, the following tests should have:

atf_test_program{name="atf_c++_test", required_programs="c++"} atf_test_program{name="check_test", required_programs="cc"} atf_test_program{name="macros_test", required_programs="c++"}

Same in atf/libatf-c/Kyuafile:

atf_test_program{name="atf_c_test", required_programs="cc"} atf_test_program{name="check_test", required_programs="cc"} atf_test_program{name="macros_test", required_programs="cc"}

Thanks

Various ATF manpage fixes

igor/manlint noted various issues with the atf manpages. A full list of commits is here:

ATF is incompatible with capsicum

@jansucan discovered that if an ATF test case enters capability mode with cap_enter, then it won't be able to write the results file. It seems like this would be easy to fix, but simply opening up the results file at the top of atf_tc_run. Thoughts? cc @jmmv @ngie-eign .

0.21: compile time warning

gcc 11.0.1

atf-c/check.c: In function 'cleanup_tmpdir':
atf-c/check.c:100:21: warning: unused variable 'err' [-Wunused-variable]
  100 |         atf_error_t err = atf_fs_rmdir(dir);
      |                     ^~~
atf-c/check.c: In function 'atf_check_exec_array':
atf-c/check.c:467:21: warning: unused variable 'err2' [-Wunused-variable]
  467 |         atf_error_t err2 = atf_fs_rmdir(&dir);
      |                     ^~~~
atf-c/detail/process.c: In function 'do_exec':
atf-c/detail/process.c:603:15: warning: unused variable 'ret' [-Wunused-variable]
  603 |     const int ret = const_execvp(atf_fs_path_cstring(ea->m_prog), ea->m_argv);
      |               ^~~
At top level:
atf-c/detail/process.c:104:1: warning: 'stream_is_valid' defined but not used [-Wunused-function]
  104 | stream_is_valid(const atf_process_stream_t *sb)
      | ^~~~~~~~~~~~~~~
atf-c/detail/fs.c: In function 'replace_contents':
atf-c/detail/fs.c:299:17: warning: variable 'err' set but not used [-Wunused-but-set-variable]
  299 |     atf_error_t err;
      |                 ^~~
In file included from ./atf-c++/macros.hpp:33,
                 from ./atf-c++.hpp:29,
                 from atf-c++/atf_c++_test.cpp:26:
./atf-c++/tests.hpp:76:10: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
   76 |     std::auto_ptr< tc_impl > pimpl;
      |          ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/tests.hpp:30,
                 from ./atf-c++/macros.hpp:33,
                 from ./atf-c++.hpp:29,
                 from atf-c++/atf_c++_test.cpp:26:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
atf-sh/atf-sh.cpp: In member function 'virtual int atf_sh::main()':
atf-sh/atf-sh.cpp:174:15: warning: unused variable 'ret' [-Wunused-variable]
  174 |     const int ret = execv(m_shell.c_str(), const_cast< char** >(argv));
      |               ^~~
In file included from atf-c++/check.cpp:26:
./atf-c++/check.hpp:74:17: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
   74 |     friend std::auto_ptr< check_result > exec(const atf::process::argv_array&);
      |                 ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/check.hpp:34,
                 from atf-c++/check.cpp:26:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
In file included from atf-c++/check.cpp:26:
./atf-c++/check.hpp:123:6: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
  123 | std::auto_ptr< check_result > exec(const atf::process::argv_array&);
      |      ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/check.hpp:34,
                 from atf-c++/check.cpp:26:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
atf-c++/check.cpp:144:6: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
  144 | std::auto_ptr< impl::check_result >
      |      ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/check.hpp:34,
                 from atf-c++/check.cpp:26:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
atf-c++/check.cpp: In function 'std::auto_ptr<atf::check::check_result> atf::check::exec(const atf::process::argv_array&)':
atf-c++/check.cpp:153:17: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
  153 |     return std::auto_ptr< impl::check_result >(new impl::check_result(&result));
      |                 ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/check.hpp:34,
                 from atf-c++/check.cpp:26:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
In file included from ./atf-c++/macros.hpp:33,
                 from ./atf-c++.hpp:29,
                 from atf-c++/build_test.cpp:31:
./atf-c++/tests.hpp:76:10: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
   76 |     std::auto_ptr< tc_impl > pimpl;
      |          ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/tests.hpp:30,
                 from ./atf-c++/macros.hpp:33,
                 from ./atf-c++.hpp:29,
                 from atf-c++/build_test.cpp:31:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
atf-c++/detail/process.cpp: In destructor 'atf::process::child::~child()':
atf-c++/detail/process.cpp:295:21: warning: unused variable 'err' [-Wunused-variable]
  295 |         atf_error_t err = atf_process_child_wait(&m_child, &s);
      |                     ^~~
In file included from ./atf-c++/macros.hpp:33,
                 from ./atf-c++.hpp:29,
                 from atf-c++/detail/application_test.cpp:32:
./atf-c++/tests.hpp:76:10: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
   76 |     std::auto_ptr< tc_impl > pimpl;
      |          ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/tests.hpp:30,
                 from ./atf-c++/macros.hpp:33,
                 from ./atf-c++.hpp:29,
                 from atf-c++/detail/application_test.cpp:32:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
In file included from ./atf-c++/macros.hpp:33,
                 from ./atf-c++.hpp:29,
                 from atf-c++/detail/exceptions_test.cpp:35:
./atf-c++/tests.hpp:76:10: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
   76 |     std::auto_ptr< tc_impl > pimpl;
      |          ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/tests.hpp:30,
                 from ./atf-c++/macros.hpp:33,
                 from ./atf-c++.hpp:29,
                 from atf-c++/detail/exceptions_test.cpp:35:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
In file included from ./atf-c++/macros.hpp:33,
                 from ./atf-c++.hpp:29,
                 from ./atf-c++/detail/test_helpers.hpp:37,
                 from atf-c++/detail/test_helpers.cpp:26:
./atf-c++/tests.hpp:76:10: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
   76 |     std::auto_ptr< tc_impl > pimpl;
      |          ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/tests.hpp:30,
                 from ./atf-c++/macros.hpp:33,
                 from ./atf-c++.hpp:29,
                 from ./atf-c++/detail/test_helpers.hpp:37,
                 from atf-c++/detail/test_helpers.cpp:26:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
In file included from atf-c++/detail/test_helpers.cpp:35:
./atf-c++/check.hpp:74:17: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
   74 |     friend std::auto_ptr< check_result > exec(const atf::process::argv_array&);
      |                 ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/tests.hpp:30,
                 from ./atf-c++/macros.hpp:33,
                 from ./atf-c++.hpp:29,
                 from ./atf-c++/detail/test_helpers.hpp:37,
                 from atf-c++/detail/test_helpers.cpp:26:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
In file included from atf-c++/detail/test_helpers.cpp:35:
./atf-c++/check.hpp:123:6: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
  123 | std::auto_ptr< check_result > exec(const atf::process::argv_array&);
      |      ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/tests.hpp:30,
                 from ./atf-c++/macros.hpp:33,
                 from ./atf-c++.hpp:29,
                 from ./atf-c++/detail/test_helpers.hpp:37,
                 from atf-c++/detail/test_helpers.cpp:26:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
In file included from ./atf-c++/macros.hpp:33,
                 from ./atf-c++.hpp:29,
                 from atf-c++/detail/env_test.cpp:28:
./atf-c++/tests.hpp:76:10: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
   76 |     std::auto_ptr< tc_impl > pimpl;
      |          ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/tests.hpp:30,
                 from ./atf-c++/macros.hpp:33,
                 from ./atf-c++.hpp:29,
                 from atf-c++/detail/env_test.cpp:28:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
In file included from atf-c++/tests_test.cpp:26:
./atf-c++/tests.hpp:76:10: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
   76 |     std::auto_ptr< tc_impl > pimpl;
      |          ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/tests.hpp:30,
                 from atf-c++/tests_test.cpp:26:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
In file included from ./atf-c++/macros.hpp:33,
                 from ./atf-c++.hpp:29,
                 from atf-c++/detail/auto_array_test.cpp:34:
./atf-c++/tests.hpp:76:10: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
   76 |     std::auto_ptr< tc_impl > pimpl;
      |          ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/tests.hpp:30,
                 from ./atf-c++/macros.hpp:33,
                 from ./atf-c++.hpp:29,
                 from atf-c++/detail/auto_array_test.cpp:34:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
In file included from atf-c++/check_test.cpp:26:
./atf-c++/check.hpp:74:17: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
   74 |     friend std::auto_ptr< check_result > exec(const atf::process::argv_array&);
      |                 ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/check.hpp:34,
                 from atf-c++/check_test.cpp:26:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
In file included from atf-c++/check_test.cpp:26:
./atf-c++/check.hpp:123:6: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
  123 | std::auto_ptr< check_result > exec(const atf::process::argv_array&);
      |      ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/check.hpp:34,
                 from atf-c++/check_test.cpp:26:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
In file included from ./atf-c++/macros.hpp:33,
                 from ./atf-c++.hpp:29,
                 from atf-c++/check_test.cpp:42:
./atf-c++/tests.hpp:76:10: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
   76 |     std::auto_ptr< tc_impl > pimpl;
      |          ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/check.hpp:34,
                 from atf-c++/check_test.cpp:26:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
atf-c++/check_test.cpp:55:6: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
   55 | std::auto_ptr< atf::check::check_result >
      |      ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/check.hpp:34,
                 from atf-c++/check_test.cpp:26:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
atf-c++/check_test.cpp:68:6: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
   68 | std::auto_ptr< atf::check::check_result >
      |      ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/check.hpp:34,
                 from atf-c++/check_test.cpp:26:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
atf-c++/check_test.cpp: In member function 'virtual void {anonymous}::atfu_tc_exec_cleanup::body() const':
atf-c++/check_test.cpp:251:10: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
  251 |     std::auto_ptr< atf::fs::path > out;
      |          ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/check.hpp:34,
                 from atf-c++/check_test.cpp:26:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
atf-c++/check_test.cpp:252:10: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
  252 |     std::auto_ptr< atf::fs::path > err;
      |          ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/check.hpp:34,
                 from atf-c++/check_test.cpp:26:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
atf-c++/check_test.cpp:255:14: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
  255 |         std::auto_ptr< atf::check::check_result > r =
      |              ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/check.hpp:34,
                 from atf-c++/check_test.cpp:26:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
atf-c++/check_test.cpp: In member function 'virtual void {anonymous}::atfu_tc_exec_exitstatus::body() const':
atf-c++/check_test.cpp:275:14: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
  275 |         std::auto_ptr< atf::check::check_result > r =
      |              ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/check.hpp:34,
                 from atf-c++/check_test.cpp:26:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
atf-c++/check_test.cpp:283:14: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
  283 |         std::auto_ptr< atf::check::check_result > r =
      |              ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/check.hpp:34,
                 from atf-c++/check_test.cpp:26:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
atf-c++/check_test.cpp:291:14: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
  291 |         std::auto_ptr< atf::check::check_result > r =
      |              ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/check.hpp:34,
                 from atf-c++/check_test.cpp:26:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
atf-c++/check_test.cpp: In member function 'virtual void {anonymous}::atfu_tc_exec_stdout_stderr::body() const':
atf-c++/check_test.cpp:324:10: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
  324 |     std::auto_ptr< atf::check::check_result > r1 =
      |          ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/check.hpp:34,
                 from atf-c++/check_test.cpp:26:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
atf-c++/check_test.cpp:329:10: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
  329 |     std::auto_ptr< atf::check::check_result > r2 =
      |          ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/check.hpp:34,
                 from atf-c++/check_test.cpp:26:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
atf-c++/check_test.cpp: In member function 'virtual void {anonymous}::atfu_tc_exec_unknown::body() const':
atf-c++/check_test.cpp:375:10: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
  375 |     std::auto_ptr< atf::check::check_result > r = atf::check::exec(argva);
      |          ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/check.hpp:34,
                 from atf-c++/check_test.cpp:26:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
In file included from ./atf-c++/macros.hpp:33,
                 from ./atf-c++.hpp:29,
                 from atf-c++/detail/process_test.cpp:31:
./atf-c++/tests.hpp:76:10: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
   76 |     std::auto_ptr< tc_impl > pimpl;
      |          ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/detail/fs.hpp:34,
                 from ./atf-c++/detail/process.hpp:41,
                 from atf-c++/detail/process_test.cpp:26:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
atf-c++/detail/process_test.cpp: In member function 'virtual void {anonymous}::atfu_tc_argv_array_assign::body() const':
atf-c++/detail/process_test.cpp:199:10: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
  199 |     std::auto_ptr< argv_array > argv1(new argv_array(carray1));
      |          ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/detail/fs.hpp:34,
                 from ./atf-c++/detail/process.hpp:41,
                 from atf-c++/detail/process_test.cpp:26:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
atf-c++/detail/process_test.cpp:200:10: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
  200 |     std::auto_ptr< argv_array > argv2(new argv_array(carray2));
      |          ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/detail/fs.hpp:34,
                 from ./atf-c++/detail/process.hpp:41,
                 from atf-c++/detail/process_test.cpp:26:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
atf-c++/detail/process_test.cpp: In member function 'virtual void {anonymous}::atfu_tc_argv_array_copy::body() const':
atf-c++/detail/process_test.cpp:229:10: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
  229 |     std::auto_ptr< argv_array > argv1(new argv_array(carray));
      |          ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/detail/fs.hpp:34,
                 from ./atf-c++/detail/process.hpp:41,
                 from atf-c++/detail/process_test.cpp:26:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
atf-c++/detail/process_test.cpp:230:10: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
  230 |     std::auto_ptr< argv_array > argv2(new argv_array(*argv1));
      |          ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/detail/fs.hpp:34,
                 from ./atf-c++/detail/process.hpp:41,
                 from atf-c++/detail/process_test.cpp:26:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
In file included from ./atf-c++/macros.hpp:33,
                 from ./atf-c++.hpp:29,
                 from test-programs/cpp_helpers.cpp:35:
./atf-c++/tests.hpp:76:10: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
   76 |     std::auto_ptr< tc_impl > pimpl;
      |          ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/tests.hpp:30,
                 from ./atf-c++/macros.hpp:33,
                 from ./atf-c++.hpp:29,
                 from test-programs/cpp_helpers.cpp:35:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
In file included from ./atf-c++/macros.hpp:33,
                 from ./atf-c++.hpp:29,
                 from atf-c++/detail/text_test.cpp:32:
./atf-c++/tests.hpp:76:10: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
   76 |     std::auto_ptr< tc_impl > pimpl;
      |          ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/tests.hpp:30,
                 from ./atf-c++/macros.hpp:33,
                 from ./atf-c++.hpp:29,
                 from atf-c++/detail/text_test.cpp:32:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
In file included from ./atf-c++/macros.hpp:33,
                 from ./atf-c++.hpp:29,
                 from atf-c++/utils_test.cpp:43:
./atf-c++/tests.hpp:76:10: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
   76 |     std::auto_ptr< tc_impl > pimpl;
      |          ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/tests.hpp:30,
                 from ./atf-c++/macros.hpp:33,
                 from ./atf-c++.hpp:29,
                 from atf-c++/utils_test.cpp:43:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
In file included from atf-sh/atf-check.cpp:46:
./atf-c++/check.hpp:74:17: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
   74 |     friend std::auto_ptr< check_result > exec(const atf::process::argv_array&);
      |                 ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from atf-sh/atf-check.cpp:43:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
In file included from atf-sh/atf-check.cpp:46:
./atf-c++/check.hpp:123:6: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
  123 | std::auto_ptr< check_result > exec(const atf::process::argv_array&);
      |      ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from atf-sh/atf-check.cpp:43:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
atf-sh/atf-check.cpp:106:10: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
  106 |     std::auto_ptr< atf::fs::path > m_path;
      |          ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from atf-sh/atf-check.cpp:43:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
atf-sh/atf-check.cpp: In function 'int parse_signal(const string&)':
atf-sh/atf-check.cpp:219:23: warning: catching polymorphic type 'class std::runtime_error' by value [-Wcatch-value=]
  219 |         } catch (std::runtime_error) {
      |                       ^~~~~~~~~~~~~
atf-sh/atf-check.cpp: At global scope:
atf-sh/atf-check.cpp:329:6: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
  329 | std::auto_ptr< atf::check::check_result >
      |      ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from atf-sh/atf-check.cpp:43:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
atf-sh/atf-check.cpp:345:6: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
  345 | std::auto_ptr< atf::check::check_result >
      |      ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from atf-sh/atf-check.cpp:43:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
atf-sh/atf-check.cpp: In member function 'virtual int {anonymous}::atf_check::main()':
atf-sh/atf-check.cpp:805:10: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
  805 |     std::auto_ptr< atf::check::check_result > r =
      |          ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from atf-sh/atf-check.cpp:43:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
In file included from ./atf-c++/macros.hpp:33,
                 from ./atf-c++.hpp:29,
                 from atf-c++/detail/fs_test.cpp:37:
./atf-c++/tests.hpp:76:10: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
   76 |     std::auto_ptr< tc_impl > pimpl;
      |          ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/detail/fs.hpp:34,
                 from atf-c++/detail/fs_test.cpp:26:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
In file included from ./atf-c++/macros.hpp:33,
                 from atf-c++/macros_test.cpp:26:
./atf-c++/tests.hpp:76:10: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
   76 |     std::auto_ptr< tc_impl > pimpl;
      |          ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/tests.hpp:30,
                 from ./atf-c++/macros.hpp:33,
                 from atf-c++/macros_test.cpp:26:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
In file included from atf-c++/tests.cpp:26:
./atf-c++/tests.hpp:76:10: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
   76 |     std::auto_ptr< tc_impl > pimpl;
      |          ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/tests.hpp:30,
                 from atf-c++/tests.cpp:26:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~
In function 'copy',
    inlined from '_S_copy' at /usr/include/c++/11/bits/basic_string.h:351:21,
    inlined from '_S_copy' at /usr/include/c++/11/bits/basic_string.h:346:7,
    inlined from '_M_replace' at /usr/include/c++/11/bits/basic_string.tcc:481:20,
    inlined from 'replace' at /usr/include/c++/11/bits/basic_string.h:1946:19,
    inlined from 'insert' at /usr/include/c++/11/bits/basic_string.h:1714:22,
    inlined from 'operator+' at /usr/include/c++/11/bits/basic_string.h:6183:23,
    inlined from 'build_check_cxx_o' at atf-c++/detail/test_helpers.cpp:56:76,
    inlined from 'header_check.constprop' at atf-c++/detail/test_helpers.cpp:82:27:
/usr/include/c++/11/bits/char_traits.h:402:56: warning: '__builtin_memcpy' reading 2 bytes from a region of size 1 [-Wstringop-overread]
  402 |         return static_cast<char_type*>(__builtin_memcpy(__s1, __s2, __n));
      |                                                        ^

and for test suites:

In file included from ./atf-c++/macros.hpp:33,
                 from ./atf-c++.hpp:29,
                 from bootstrap/h_tp_basic_cpp.cpp:26:
./atf-c++/tests.hpp:76:10: warning: 'template<class> class std::auto_ptr' is deprecated: use 'std::unique_ptr' instead [-Wdeprecated-declarations]
   76 |     std::auto_ptr< tc_impl > pimpl;
      |          ^~~~~~~~
In file included from /usr/include/c++/11/memory:76,
                 from ./atf-c++/tests.hpp:30,
                 from ./atf-c++/macros.hpp:33,
                 from ./atf-c++.hpp:29,
                 from bootstrap/h_tp_basic_cpp.cpp:26:
/usr/include/c++/11/bits/unique_ptr.h:57:28: note: declared here
   57 |   template<typename> class auto_ptr;
      |                            ^~~~~~~~

Avoid the need for -Wno-error=unused-private-field in ATF for NetBSD

From [email protected] on December 07, 2012 13:18:31

The -Wno-error=unused-private-field flag was added to the ATF build in NetBSD because otherwise we were raising warnings when building with clang. Figure out what fields are the unused ones and either nuke them or silence the warning with an attribute, if possible.

Original issue: http://code.google.com/p/kyua/issues/detail?id=51

kyua report should have the option of including list of passed tests

kyua report and kyua report-html currently only show a total of passing tests, not a list of them. It would be very useful to have a list of passed tests like there is for failed/broken/skipped tests.