Comments (13)
chrisosaurus
commented on July 17, 2024
Strange, from your travis log I see 2 interesting sections
The first time coveralls only analyses one files, the second time it analyses 'a lot'.
https://travis-ci.org/GlPortal/RadixEngine/builds/212253813#L506
[100%] Built target tests
Test project /data
Start 1: XmlHelperTest
1/1 Test #1: XmlHelperTest .................... Passed 0.01 sec
100% tests passed, 0 tests failed out of 1
Total Test time (real) = 0.01 sec
CMakeCXXCompilerId.gcda:cannot open data file, assuming not executed
File 'CMakeCXXCompilerId.cpp'
Lines executed:0.00% of 7
Creating 'CMakeCXXCompilerId.gcno##CMakeCXXCompilerId.cpp.gcov'
# it then moves on to doing other things
https://travis-ci.org/GlPortal/RadixEngine/builds/212253813#L715
[100%] Built target tests
Test project /data
Start 1: XmlHelperTest
1/1 Test #1: XmlHelperTest .................... Passed 0.01 sec
100% tests passed, 0 tests failed out of 1
Total Test time (real) = 0.01 sec
/data/external/json11/CMakeFiles/json11.dir/json11.cpp.gcda:cannot open data file, assuming not executed
File '/usr/include/c++/6.3.1/bits/stl_algobase.h'
Lines executed:0.00% of 23
Creating '#data#external#json11#CMakeFiles#json11.dir#json11.cpp.o###usr#include#c++#6.3.1#bits#stl_algobase.h.gcov'
File '/usr/include/c++/6.3.1/bits/predefined_ops.h'
Lines executed:0.00% of 1
Creating '#data#external#json11#CMakeFiles#json11.dir#json11.cpp.o###usr#include#c++#6.3.1#bits#predefined_ops.h.gcov'
[...]
# lots more files analysed
Very strange.
from cpp-coveralls.
chrisosaurus
commented on July 17, 2024
Attempted reproduction steps roughly following https://github.com/GlPortal/RadixEngine/blob/master/COMPILE.md
(I tried to use a build dir at first but it didn't seem to add anything, so tried without)
$ sudo pacman -S cmake make gcc pkg-config assimp libepoxy sdl2{,_mixer} bullet mesa tinyxml2 freeimage
$ sudo pip install cpp-coveralls
$ git clone https://github.com/GlPortal/RadixEngine.git
$ cd RadixEngine
$ git submodule init
$ git submodule update
$ cmake .
$ make
$ make test
$ coveralls --dump HERE
$ grep -o '"name":[^,]*,' HERE | wc -l
890
$ grep -o '"name":[^,]*,' HERE | head -10
"name": "include/radix/Camera.hpp",
"name": "include/radix/SimulationManager.hpp",
"name": "include/radix/EntityManager.hpp",
"name": "include/radix/Entity.hpp",
"name": "include/radix/Console.hpp",
"name": "include/radix/SoundManager.hpp",
"name": "include/radix/BaseGame.hpp",
"name": "include/radix/World.hpp",
"name": "include/radix/GameWorld.hpp",
"name": "include/radix/Viewport.hpp",
so it seems to be correctly generating coverage report sfor 890 source files after a single run.
Are you still able to produce this? if so please provide reproduction steps.
from cpp-coveralls.
hhirsch
commented on July 17, 2024
The test target does not exist so make test should just fail.
make should build the tests though.
iirc coveralls not running wasn't the problem but it reporting 0% coverage when there is 3% coverage.
Variations can be easily tested on a branch and I'll be happy to change the line accordingly:
https://github.com/GlPortal/RadixEngine/blob/master/.travis.yml
I tried to tackle this problem multiple times and I always had to revert to the run twice method.
from cpp-coveralls.
chrisosaurus
commented on July 17, 2024
make test seems to run the tests for RadixEngine
[chris@niflheim RadixEngine]$ git remote -vv
origin https://github.com/GlPortal/RadixEngine.git (fetch)
origin https://github.com/GlPortal/RadixEngine.git (push)
[chris@niflheim RadixEngine]$ make test
Running tests...
Test project /home/chris/devel/RadixEngine
Start 1: XmlHelperTest
1/3 Test #1: XmlHelperTest .................... Passed 0.01 sec
Start 2: QuaternionTest
2/3 Test #2: QuaternionTest ................... Passed 0.01 sec
Start 3: MathTest
3/3 Test #3: MathTest ......................... Passed 0.01 sec
100% tests passed, 0 tests failed out of 3
Total Test time (real) = 0.03 sec
Am I misunderstanding something?
I guess I can try find time to setup my own coveralls and travis for a clone of RadixEngine and see what I can get out of it
from cpp-coveralls.
chrisosaurus
commented on July 17, 2024
So in RadixEngine it looks like:
make testswill build the testsmake testandctestare equialent - either one will run the tests
[chris@niflheim RadixEngine]$ make tests
[ 3%] Built target RadixEntity
[ 3%] Built target GLAD
[ 4%] Built target json11
[ 11%] Built target VHACD_LIB
[ 33%] Built target angelscript
[ 36%] Built target easy_profiler
[ 88%] Built target RadixEngine
[ 93%] Built target MathTest
[ 96%] Built target XmlHelperTest
[100%] Built target QuaternionTest
[100%] Built target tests
[chris@niflheim RadixEngine]$ make test
Running tests...
Test project /home/chris/devel/RadixEngine
Start 1: XmlHelperTest
1/3 Test #1: XmlHelperTest .................... Passed 0.01 sec
Start 2: QuaternionTest
2/3 Test #2: QuaternionTest ................... Passed 0.01 sec
Start 3: MathTest
3/3 Test #3: MathTest ......................... Passed 0.01 sec
100% tests passed, 0 tests failed out of 3
Total Test time (real) = 0.03 sec
[chris@niflheim RadixEngine]$ ctest
Test project /home/chris/devel/RadixEngine
Start 1: XmlHelperTest
1/3 Test #1: XmlHelperTest .................... Passed 0.01 sec
Start 2: QuaternionTest
2/3 Test #2: QuaternionTest ................... Passed 0.01 sec
Start 3: MathTest
3/3 Test #3: MathTest ......................... Passed 0.01 sec
100% tests passed, 0 tests failed out of 3
Total Test time (real) = 0.03 sec
I've setup travis and coveralls for my clone, will play and around and then report results back, might take a while
from cpp-coveralls.
chrisosaurus
commented on July 17, 2024
As a side effect of investigating this I noticed an issue in that you have a public .coveralls.yml (only needed for private travis, not needed for open source.)
I created an issue on your repo to describe this
GlPortal/RadixEngine#173
and then sent a PR to fix this
GlPortal/RadixEngine#174
That should just work as-is unless I am misunderstanding something.
from cpp-coveralls.
chrisosaurus
commented on July 17, 2024
Okay, I now have coveralls running outside the docker container and I'm able to reproduce the 0% coveralls report on my clone.
from cpp-coveralls.
chrisosaurus
commented on July 17, 2024
tests and output:
Running:
- ctest inside docker ONCE
- cpp-coveralls outside docker ONCE
gives 0% coverage
Log: https://api.travis-ci.org/v3/job/401344880/log.txt
before_install:
- docker pull glportal/whale-gcc:coverall
- pip install --user cpp-coveralls
script:
- docker run -it --rm -w /data -v $(pwd):/data glportal/whale-gcc:coverall bash -c "cmake -DCOVERAGE:BOOL=ON ./&& make && make tests && ctest;"
- coveralls -i include -i source -e tests --gcov-options \'-lp\'
Running:
- ctest inside docker ONCE
- cpp-coveralls outside docker TWICE
gives 0% coverage
Log: https://api.travis-ci.org/v3/job/401348185/log.txt
before_install:
- docker pull glportal/whale-gcc:coverall
- pip install --user cpp-coveralls
script:
- docker run -it --rm -w /data -v $(pwd):/data glportal/whale-gcc:coverall bash -c "cmake -DCOVERAGE:BOOL=ON ./&& make && make tests && ctest;"
- coveralls --dryrun -i include -i source -e tests --gcov-options \'-lp\'
- coveralls -i include -i source -e tests --gcov-options \'-lp\'
Running:
- ctest inside docker TWICE
- cpp-coveralls outside docker ONCE
gives 0% coverage
Log: https://api.travis-ci.org/v3/job/401348813/log.txt
before_install:
- docker pull glportal/whale-gcc:coverall
- pip install --user cpp-coveralls
script:
- docker run -it --rm -w /data -v $(pwd):/data glportal/whale-gcc:coverall bash -c "cmake -DCOVERAGE:BOOL=ON ./&& make && make tests && ctest && ctest;"
- coveralls -i include -i source -e tests --gcov-options \'-lp\'
Running:
- 'everything' inside docker TWICE
- cpp-coveralls outside docker ONCE
gives 0% coverage
Log: https://api.travis-ci.org/v3/job/401349052/log.txt
before_install:
- docker pull glportal/whale-gcc:coverall
- pip install --user cpp-coveralls
script:
- docker run -it --rm -w /data -v $(pwd):/data glportal/whale-gcc:coverall bash -c "cmake -DCOVERAGE:BOOL=ON ./&& make && make tests && ctest && cmake -DCOVERAGE:BOOL=ON ./&& make && make tests && ctest;"
- coveralls -i include -i source -e tests --gcov-options \'-lp\'
Very strangely, only this last one produced a huge log.
All previous versions produced
1.24s$ coveralls -i include -i source -e tests --gcov-options \'-lp\'
CMakeCXXCompilerId.gcno:version '603*', prefer '408*'
CMakeCXXCompilerId.gcda:cannot open data file, assuming not executed
File 'CMakeCXXCompilerId.cpp'
No executable lines
Removing 'CMakeCXXCompilerId.gcno##CMakeCXXCompilerId.cpp.gcov'
{u'url': u'https://coveralls.io/jobs/38190001', u'message': u'Job #11.1'}
only the last one (running cmake/make/ctest twice, coveralls once) produces the output we expect
[0K$ coveralls -i include -i source -e tests --gcov-options \'-lp\'
CMakeCXXCompilerId.gcno:version '603*', prefer '408*'
CMakeCXXCompilerId.gcda:cannot open data file, assuming not executed
File 'CMakeCXXCompilerId.cpp'
No executable lines
Removing 'CMakeCXXCompilerId.gcno##CMakeCXXCompilerId.cpp.gcov'
/home/travis/build/mkfifo/RadixEngine/CMakeFiles/RadixEngine.dir/source/SoundManager.cpp.gcno:version '603*', prefer '408*'
/home/travis/build/mkfifo/RadixEngine/CMakeFiles/RadixEngine.dir/source/SoundManager.cpp.gcda:cannot open data file, assuming not executed
File '/usr/include/c++/6.3.1/bits/stl_tree.h'
No executable lines
Removing '#home#travis#build#mkfifo#RadixEngine#CMakeFiles#RadixEngine.dir#source#SoundManager.cpp.o###usr#include#c++#6.3.1#bits#stl_tree.h.gcov'
File '/usr/include/c++/6.3.1/bits/alloc_traits.h'
No executable lines
Removing '#home#travis#build#mkfifo#RadixEngine#CMakeFiles#RadixEngine.dir#source#SoundManager.cpp.o###usr#include#c++#6.3.1#bits#alloc_traits.h.gcov'
[...]
File '/usr/include/c++/6.3.1/atomic'
No executable lines
Removing '#home#travis#build#mkfifo#RadixEngine#external#easy_profiler#sample#CMakeFiles#profiler_sample.dir#main.cpp.o###usr#include#c++#6.3.1#atomic.gcov'
File '/usr/include/c++/6.3.1/bits/atomic_base.h'
No executable lines
Removing '#home#travis#build#mkfifo#RadixEngine#external#easy_profiler#sample#CMakeFiles#profiler_sample.dir#main.cpp.o###usr#include#c++#6.3.1#bits#atomic_base.h.gcov'
File '/data/external/easy_profiler/easy_profiler_core/include/easy/profiler.h'
No executable lines
Removing '#home#travis#build#mkfifo#RadixEngine#external#easy_profiler#sample#CMakeFiles#profiler_sample.dir#main.cpp.o###data#external#easy_profiler#easy_profiler_core#include#easy#profiler.h.gcov'
File '/usr/include/c++/6.3.1/bits/locale_facets.h'
No executable lines
Removing '#home#travis#build#mkfifo#RadixEngine#external#easy_profiler#sample#CMakeFiles#profiler_sample.dir#main.cpp.o###usr#include#c++#6.3.1#bits#locale_facets.h.gcov'
{u'url': u'https://coveralls.io/jobs/38190033', u'message': u'Job #13.1'}
but it still seemed to generate a coverage of 0% unless something else went wrong.
I think the problem is at least in part due to your make process not producing coverage data the first time it is run, possibly due to cmake not correctly settings that variable you pass in.
I'll keep poking around.
from cpp-coveralls.
chrisosaurus
commented on July 17, 2024
Confirmed that the single build isn't producing the profile data
- docker pull glportal/whale-gcc:coverall
- pip install --user cpp-coveralls
script:
- docker run -it --rm -w /data -v $(pwd):/data glportal/whale-gcc:coverall bash -c "cmake -DCOVERAGE:BOOL=ON ./&& make && make tests && ctest;"
- coveralls -i include -i source -e tests --gcov-options \'-lp\'
- find . -iname '*.gcda' | wc -l
- find . -iname '*.gcno' | wc -l
Log: https://api.travis-ci.org/v3/job/401354549/log.txt
output
travis_time:end:014a2c6d:start=1531020384350424558,finish=1531020386367779703,duration=2017355145
�[0K
�[32;1mThe command "coveralls -i include -i source -e tests --gcov-options \'-lp\'" exited with 0.�[0m
travis_time:start:12bf641b
�[0K$ find . -iname '*.gcda' | wc -l
0
travis_time:end:12bf641b:start=1531020386374362869,finish=1531020386390520461,duration=16157592
�[0K
�[32;1mThe command "find . -iname '*.gcda' | wc -l" exited with 0.�[0m
travis_time:start:03a20972
�[0K$ find . -iname '*.gcno' | wc -l
1
running just ctest twice
before_install:
- docker pull glportal/whale-gcc:coverall
- pip install --user cpp-coveralls
script:
- docker run -it --rm -w /data -v $(pwd):/data glportal/whale-gcc:coverall bash -c "cmake -DCOVERAGE:BOOL=ON ./&& make && make tests && ctest && ctest;"
- coveralls -i include -i source -e tests --gcov-options \'-lp\'
- find . -iname '*.gcda' | wc -l
- find . -iname '*.gcno' | wc -l
Log: https://api.travis-ci.org/v3/job/401354666/log.txt
outputs
travis_time:end:0557cb6e:start=1531020363849677574,finish=1531020365103355512,duration=1253677938
�[0K
�[32;1mThe command "coveralls -i include -i source -e tests --gcov-options \'-lp\'" exited with 0.�[0m
travis_time:start:13f0336e
�[0K$ find . -iname '*.gcda' | wc -l
0
travis_time:end:13f0336e:start=1531020365110018399,finish=1531020365127668255,duration=17649856
�[0K
�[32;1mThe command "find . -iname '*.gcda' | wc -l" exited with 0.�[0m
travis_time:start:01fb3190
�[0K$ find . -iname '*.gcno' | wc -l
1
running cmake/make/ctest twice
before_install:
- docker pull glportal/whale-gcc:coverall
- pip install --user cpp-coveralls
script:
- docker run -it --rm -w /data -v $(pwd):/data glportal/whale-gcc:coverall bash -c "cmake -DCOVERAGE:BOOL=ON ./&& make && make tests && ctest && cmake -DCOVERAGE:BOOL=ON ./&& make && make tests && ctest;"
- coveralls -i include -i source -e tests --gcov-options \'-lp\'
- find . -iname '*.gcda' | wc -l
- find . -iname '*.gcno' | wc -l
Log: https://api.travis-ci.org/v3/job/401354576/log.txt
outputs
travis_time:end:0771da55:start=1531020637657257692,finish=1531020639721292131,duration=2064034439
�[0K
�[32;1mThe command "coveralls -i include -i source -e tests --gcov-options \'-lp\'" exited with 0.�[0m
travis_time:start:1eb72f18
�[0K$ find . -iname '*.gcda' | wc -l
54
travis_time:end:1eb72f18:start=1531020639731115174,finish=1531020639750849314,duration=19734140
�[0K
�[32;1mThe command "find . -iname '*.gcda' | wc -l" exited with 0.�[0m
travis_time:start:1a6234f4
�[0K$ find . -iname '*.gcno' | wc -l
172
So the problem looks to be that your build tooling requires running cmake/make/ctest twice before it produces the gcno and gcda files.
Cpp-coveralls doesn't appear to be the problem here.
from cpp-coveralls.
chrisosaurus
commented on July 17, 2024
So poking around in CMakeFiles/ with grep looking for 'coverage' we get
single run of everything
before_install:
- docker pull glportal/whale-gcc:coverall
- pip install --user cpp-coveralls
script:
- docker run -it --rm -w /data -v $(pwd):/data glportal/whale-gcc:coverall bash -c "cmake -DCOVERAGE:BOOL=ON ./&& make && make tests && ctest;"
- coveralls -i include -i source -e tests --gcov-options \'-lp\'
- find . -iname '*.gcda' | wc -l
- find . -iname '*.gcno' | wc -l
- grep -Rni 'coverage' CMakeFiles/* | wc -l
Log: https://api.travis-ci.org/v3/job/401359763/log.txt
output
travis_time:end:1060f428:start=1531023482629119561,finish=1531023484216188718,duration=1587069157
�[0K
�[32;1mThe command "coveralls -i include -i source -e tests --gcov-options \'-lp\'" exited with 0.�[0m
travis_time:start:076beae1
�[0K$ find . -iname '*.gcda' | wc -l
0
travis_time:end:076beae1:start=1531023484225581695,finish=1531023484243654484,duration=18072789
�[0K
�[32;1mThe command "find . -iname '*.gcda' | wc -l" exited with 0.�[0m
travis_time:start:015e15a6
�[0K$ find . -iname '*.gcno' | wc -l
1
travis_time:end:015e15a6:start=1531023484255969002,finish=1531023484273764591,duration=17795589
�[0K
�[32;1mThe command "find . -iname '*.gcno' | wc -l" exited with 0.�[0m
travis_time:start:308de1c4
�[0K$ grep -Rni 'coverage' CMakeFiles/* | wc -l
2
So no gcno, no gcda, and only 2 mentions of 'coverage'
running cmake twice
before_install:
- docker pull glportal/whale-gcc:coverall
- pip install --user cpp-coveralls
script:
- docker run -it --rm -w /data -v $(pwd):/data glportal/whale-gcc:coverall bash -c "cmake -DCOVERAGE:BOOL=ON && cmake -DCOVERAGE:BOOL=ON ./&& make && make tests && ctest;"
- coveralls -i include -i source -e tests --gcov-options \'-lp\'
- find . -iname '*.gcda' | wc -l
- find . -iname '*.gcno' | wc -l
- grep -Rni 'coverage' CMakeFiles/* | wc -l
Log: https://api.travis-ci.org/v3/job/401359814/log.txt
output
travis_time:end:0ce384e0:start=1531023535870351025,finish=1531023537691723347,duration=1821372322
�[0K
�[32;1mThe command "coveralls -i include -i source -e tests --gcov-options \'-lp\'" exited with 0.�[0m
travis_time:start:1e795360
�[0K$ find . -iname '*.gcda' | wc -l
54
travis_time:end:1e795360:start=1531023537698709357,finish=1531023537714924357,duration=16215000
�[0K
�[32;1mThe command "find . -iname '*.gcda' | wc -l" exited with 0.�[0m
travis_time:start:0f774228
�[0K$ find . -iname '*.gcno' | wc -l
172
travis_time:end:0f774228:start=1531023537721351873,finish=1531023537738222833,duration=16870960
�[0K
�[32;1mThe command "find . -iname '*.gcno' | wc -l" exited with 0.�[0m
travis_time:start:0dd3c07f
�[0K$ grep -Rni 'coverage' CMakeFiles/* | wc -l
3
3 mentions of coverage, 54 gcda files, 172 gcno files
running 'everything' twice inside docker
before_install:
- docker pull glportal/whale-gcc:coverall
- pip install --user cpp-coveralls
script:
- docker run -it --rm -w /data -v $(pwd):/data glportal/whale-gcc:coverall bash -c "cmake -DCOVERAGE:BOOL=ON ./&& make && make tests && ctest && cmake -DCOVERAGE:BOOL=ON ./&& make && make tests && ctest;"
- coveralls -i include -i source -e tests --gcov-options \'-lp\'
- find . -iname '*.gcda' | wc -l
- find . -iname '*.gcno' | wc -l
- grep -Rni 'coverage' CMakeFiles/* | wc -l
Log: https://api.travis-ci.org/v3/job/401359864/log.txt
travis_time:end:0599ce52:start=1531023759612471574,finish=1531023761327501590,duration=1715030016
�[0K
�[32;1mThe command "coveralls -i include -i source -e tests --gcov-options \'-lp\'" exited with 0.�[0m
travis_time:start:1f4625f9
�[0K$ find . -iname '*.gcda' | wc -l
54
travis_time:end:1f4625f9:start=1531023761335501133,finish=1531023761353031077,duration=17529944
�[0K
�[32;1mThe command "find . -iname '*.gcda' | wc -l" exited with 0.�[0m
travis_time:start:134c59f5
�[0K$ find . -iname '*.gcno' | wc -l
172
travis_time:end:134c59f5:start=1531023761360645133,finish=1531023761376400031,duration=15754898
�[0K
�[32;1mThe command "find . -iname '*.gcno' | wc -l" exited with 0.�[0m
travis_time:start:2519013c
�[0K$ grep -Rni 'coverage' CMakeFiles/* | wc -l
3
Again 3 mentions of coverage, 54 gcda, 172 gcno files.
from cpp-coveralls.
chrisosaurus
commented on July 17, 2024
So at the very least, your cmake config is buggy, as these two commands produce different outputs:
cmake -DCOVERAGE:BOOL=ON ./&& make && make tests && ctest
cmake -DCOVERAGE:BOOL=ON && cmake -DCOVERAGE:BOOL=ON ./ && make && make tests && ctest
from cpp-coveralls.
chrisosaurus
commented on July 17, 2024
I have filed GlPortal/RadixEngine#175 for the cmake config issue
from cpp-coveralls.
chrisosaurus
commented on July 17, 2024
I'm still seeing a lot of noise in the genereated output, lots of mentions for system included files which will throw your coverage stats way off.
from cpp-coveralls.
Related Issues (20)
- ValueError: invalid literal for int() with base 10: '917*' HOT 3
- Where is args.lcov_file being set?
- unable to generate report
- pip install failure on ubuntu/trusty64 HOT 5
- Best method for implementing with Boost Unit Test Cases
- Cannot find files due to odd path creation HOT 1
- 0 of 0 relevant lines covered: am I not using correct arguments? HOT 2
- Exclusion regexps do not appear to work HOT 2
- Variable interpolation into shell commands is unsafe HOT 1
- Is this project dead? HOT 1
- circleci not supported HOT 1
- --include or --exclude options doesn't seem to work at all HOT 3
- FIleNotFoundError with invalid path HOT 1
- YAMLLoadWarning HOT 1
- Cannot open source file
- Failed to see coverage results on Coveralls HOT 1
- Deprecation warning due to invalid escape sequences
- FileNotFoundError for not existing file
- github actions not showing the correct files on coveralls website
- feature request: carryforward flags
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cpp-coveralls.