csmacnz / coveralls.net Goto Github PK

As a user who wants to run coveralls.net I want to be able to provide the git data as input parameters on the command line so that I can have these set correctly when the results are sent to coveralls.io.

• Git commit hash is provided as an input parameter
• Git commit message is provided as an input parameter
• Git branch name is provided as an input parameter
• Git commit user name is provided as an input parameter
• Git commit user email is provided as an input parameter

As a user I want to trial run the app and look at the json that will be sent to coveralls.io so that I can verify what gets sent is what I expected.

Add a -o/--output parameter to match the input parameter, that saves the json payload to that file when provided. Expected to be used with a --dryrun paramater so the results are saved but not published.

https://coveralls.io/github/csMACnz/Coveralls.net-Samples?branch=core-xunit-coverlet-appveyor

files from different jobs show up against the same result. The code should be fetching and providing this:

https://github.com/csMACnz/coveralls.net/blob/release-1.0.0/src/csmacnz.Coveralls/CoverageMetadataResolver.cs#L51

but on coveralls the results are merged.

• is the value being output into the json?
• is it in the right place in the json?
• is the value valid?

Add an option to ignore ssl certificate errors when posting to coveralls api.

In Program.cs

var handler = new WebRequestHandler();
if (args.IsProvided("--ignoreCertErrors") && args.OptIgnoreCertErrors)
{
    handler.ServerCertificateValidationCallback += (sender, cert, chain, sslPolicyErrors) => true;
}
var uploadResult = new CoverallsService().Upload(fileData, handler);

In CoverallsService.cs

public Result<bool, string> Upload(string fileData, WebRequestHandler handler)
{
    using (HttpContent stringContent = new StringContent(fileData))
    {
        using (var client = new HttpClient(handler))
        using (var formData = new MultipartFormDataContent())
        {
            formData.Add(stringContent, "json_file", "coverage.json");

            var response = client.PostAsync(RequestUri, formData).Result;

            if (!response.IsSuccessStatusCode)
            {
                var content = response.Content.ReadAsStringAsync().Result;
                var message = TryGetJsonMessageFromResponse(content).ValueOr(content);

                if (message.Length > 200)
                {
                    message = message.Substring(0, 200);
                }

                return string.Format("{0} - {1}", response.StatusCode, message);
            }
            return true;
        }
    }
}

New Option in T4DocoptNet.cs

public bool OptIgnoreCertErrors { get { return _args["--ignoreCertErrors"].IsTrue; } }

(String) source
Optional (Required for Manual repos on Enterprise)
The contents of the source file.

Because this isn't required when the source is on GitHub, this should probably be configurable to not require the files (if on GitHub). I think the better option is to not send the files, unless send files is provided. This would be a breaking change though.

As a Travis-CI user, I want to set the service_name to "travis-ci" so that I can hook into extra supported features of coveralls.io

So something got lost along the way. I have to use that beta version to get coveralls feedback on pull requests.

Consider the following scenario: a user wants to send multiple coverage files to coveralls website from a single command line run.
Feature proposal: add --folder flag.
This flag will receive a folder as input.
The code will take all the files in that folder and parse them according to file extensions.
file.opencover.xml (or file.opencover haven't decided yet) will be parsed as open cover
file.chuzpah.json (or file.chutzpah) will be parsed as chutzpah
file.lcov will be parsed as lcov (pending implementation of lcov format).

I'm not entirely familiar with the other options but I think that there's an option now that uses a folder which may use the same mechanism.

As a part of this changes I really think it would be wise to add an IoC container and some folders to split the logic into business logic and data access.
The current state doesn't really support unit testing well as far as I could see.

Also I think that the parsers should receive a file path and they should all inherit from the same interface.

What do you think?

As a progress tracker of the project I want to see coverage of coveralls.net run by the visual studio coverage tool as well as opencover so that the app is truly dog-fooding.

As a coveralls user I want my windows fake linux paths to not include the : in the "C:" drive portion of my path so that it looks more linux-ey

input: c:\home\test
output /c/home/test

As a user with different build servers on different branches, I want source paths to be made relative to the repository root path so that I can see the source regardless of what the original path on the build server was.

Acceptance criteria:

• On a Travis CI branch, the source paths are relative to the root of the repository
• On an AppVeyor Branch, the source paths are relative to the root of the repository
• a flag is used to determine to use the relative path mode, full path mode will be the default.

There are now zip files as os-targeted standalone apps that can be downloaded and run on linux, mac(?) and windows. These complement the dotnet 2.1 tool format.

Document examples of how to use the zip file versions instead.

As a product owner i want to see ascii art when someone runs the console app so that people see it when reading the output

• add ascii art print on startup
• add --noascii option so not annoying

I can see that OpenCover support is not finished, but there is no backlog, so I am not sure what is missing. I see your reports on https://coveralls.io/r/csmacnz/coveralls.net, so it looks good. What is missing?
Also, why is the package unlisted on nuget?
https://www.nuget.org/packages/coveralls.net

As a repository owner, I want to see coveralls.net results from Pull Request builds so that I can check coverage stats have not decreased from a PR change.

Pull Requests do not currently build using the same calling code from AppVeyor. The application might need to be away of pull request builds to publish correctly.

Property coverage from auto-properties does not get mapped back to source code with opencover results.

As a user when I run coverage over my autoproperties, then properties get correctly highlighted in the coverage report so that I can accurately map my coverage percentages over these properties.

The value of the --basePath argument is never used because the private member field is checked for not being null instead of the constructor argument for PathProcessor. Because of this the specified value is always thought to be null and is consequently never used.

here is an example of this format, I saw it when looking into possible alternatives, I think it's in use in some cases, chutzpah is one of them and I think there are more, I'll see if I can send a pull request to add it.
coverage-chutzpah.txt

if anyone is using this with the webhook, I want to be able to support that use case.

Later, this could be able to post the webhook for you as well.

As a CI user using Git, I want to have the git commit information automatically populated so that It is easier to use coverall.net

As a SharpCover coverage user, I want to be able to publish my coverage results to coveralls.io so that I can monitor my coverage results over time.

As a user who wants to run coveralls.net I want to be able to provide the coveralls.io repo token and jobId data as input parameters on the command line so that I can have these set correctly when the results are sent to coveralls.io.

• coverall.io repo specific token is provided as an input parameter
• buildserver jobId is (optionally) provided as an input parameter

https://ci.appveyor.com/project/MarkClearwater/coveralls-net/build/1.1.0-PullRequest.76+7.build.1339/job/24oc5q09qulthb5c

This Linux build was missing the repo token, should investigate why, and fix if possible. Works for the windows pr...

We're using coveralls.net to upload our OpenCover coverage data to Coveralls via an AppVeyor build. However, sometimes this fails for reasons I do not understand. For instance, currently it does not work (most of the time, it worked "the next day", but it now consistently failed for a couple of days already). Here's some output of our AppVeyor build - invocation of coveralls.net:

& $coveralls --opencover -i OpenCover_CoverageData.xml --repoToken $env:COVERALLS_REPO_TOKEN --commitId $env:APPVEYOR_REPO_COMMIT --commitBranch $env:APPVEYOR_REPO_BRANCH --commitAuthor $env:APPVEYOR_REPO_COMMIT_AUTHOR --commitEmail $env:APPVEYOR_REPO_COMMIT_AUTHOR_EMAIL --commitMessage $env:APPVEYOR_REPO_COMMIT_MESSAGE --jobId $env:APPVEYOR_JOB_ID --serviceName AppVeyor

coveralls.net output:

Usage:
csmacnz.Coveralls (--opencover | --dynamiccodecoverage | --monocov) -i ./opencovertests.xml (--repoToken <repoToken> | [--repoTokenVariable <repoTokenVariable>]) [-o ./opencovertests.json] [--dryrun] [--useRelativePaths [--base Path <path>] ] [--commitId <commitId> --commitBranch <commitBranch> [--commitAuthor <commitAuthor> --commitEmail <commitEmail> --commitMessage <commitMessage>] ] [--jobId <jobId>] [--serviceName <Name>] [--treatUploadErrorsAsWarnings]
csmacnz.Coveralls --version
csmacnz.Coveralls --help

Since we have not changed invocation of coveralls.net (see e.g. this build), I'm pretty sure that the problem is not a "usage" problem.

Here's our latest (and failing) build and the appveyor.yml.

Is this a coveralls.io issue (e.g., is their web service not available at times)? Or is this even an AppVeyor issue (e.g., some env variables are not bound to their correct values, thus causing the "Usage" output of coveralls.net)? How can I debug the actual cause of the problem?

Thanks in advance!

Edited: Forgot to mention that we are using coveralls.net v0.6

As coveralls.io I want posted coverage reports to use a md5 of the source instead of the entire source so that posting coverage is more efficient.

https://coveralls.zendesk.com/hc/en-us/articles/201774865-API-Introduction

As a Jetbrains dotcover coverage user, I want to be able to publish my dotcover results to coveralls.io so that i can monitor my coverage results over time.

As an NCover coverage user, I want to be able to publish my NCover results to coveralls.io so that i can monitor my coverage results over time.

Would be nice to have a class TeamCityGitDataResolver : IGitDataResolver

For example, when the Codecov.io bash script is ran in a teamcity build step, the following output is given:

[19:08:41]	[Step 4/5] ==> TeamCity CI detected.
[19:08:41]	[Step 4/5]     Teamcity does not automatically make build parameters available as environment variables.
[19:08:41]	[Step 4/5]     Add the following environment parameters to the build configuration
[19:08:41]	[Step 4/5]     env.TEAMCITY_BUILD_BRANCH = %teamcity.build.branch%
[19:08:41]	[Step 4/5]     env.TEAMCITY_BUILD_ID = %teamcity.build.id%
[19:08:41]	[Step 4/5]     env.TEAMCITY_BUILD_URL = %teamcity.serverUrl%/viewLog.html?buildId=%teamcity.build.id%
[19:08:41]	[Step 4/5]     env.TEAMCITY_BUILD_COMMIT = %system.build.vcs.number%
[19:08:41]	[Step 4/5]     env.TEAMCITY_BUILD_REPOSITORY = %vcsroot.<YOUR TEAMCITY VCS NAME>.url%

Then after following the instructions, it proceeds to work as expected.

Could this project implement a similar way? Thoughts?

I may attempt to implement this if reasonable. Would appreciate any insight.

The error messaging from the argument parser isn't very good.
if you have --repoToken -o output.xml (ie missing token value) it tries to parse the -o as the value, then fails with a usage error when it tries to process output.xml as valid.

Lets replace with a new library
http://stackoverflow.com/questions/491595/best-way-to-parse-command-line-arguments-in-c
http://blog.simontimms.com/2014/07/09/parsing-command-line-arguments-in-c/

too many options:

• http://commandline.codeplex.com/
• http://www.ndesk.org/Options
• https://github.com/fschwiet/ManyConsole
• https://github.com/fclp/fluent-command-line-parser

As a nuget user I want to see that the package has its own icon so that I know it is more important than the others.

I'm using visual studio to write my c# code, I also write my web application code with typescript and Chutzpah.
Is it possible to add support for Chutzpah coverage report as well? I think it can write a few output format for coverage as far as I can understand from their wiki:
https://github.com/mmanela/chutzpah/wiki/command-line-options

command line syntax not clear:
installing version 0.6.0
command executed :

mono ./testrunner/coveralls.net.0.6.0/tools/csmacnz.Coveralls.exe --monocov -i ./monocovCoverage --commitId $TRAVIS_COMMIT --commitBranch $TRAVIS_BRANCH --commitAuthor "$REPO_COMMIT_AUTHOR" --commitEmail "$REPO_COMMIT_AUTHOR_EMAIL" --commitMessage "$REPO_COMMIT_MESSAGE" --jobId $TRAVIS_JOB_ID  --serviceName "travis-ci"  --useRelativePaths

No token found in Environment Variable 'COVERALLS_REPO_TOKEN'.

how do you go with a public repo without leaking the coveralls token?
Apparently Coveralls.io does not need repo token when submitting coverage for public repositories. reference : lemurheavy/coveralls-public#208

When the 500 error content is an html page then it is too big to print out.
https://ci.appveyor.com/project/MarkClearwater/coveralls-net/build/0.6.0-unstable.49+49%20(Build%20371)
fix this.

When all flag are in their correct place and all the data is present except --commitEmail, the application crashes (ran it from powershell) using latest stable Nuget package:

PS> Invoke-Expression $cmd
output file 'cov.json' already exists and will be overwritten.

Unhandled Exception: System.NullReferenceException: Object reference not set to an instance of an object.
   at csmacnz.Coveralls.MainArgs.get_OptCommitemail() in c:\projects\coveralls-net\src\csmacnz.Coveralls\T4DocoptNet.cs:
line 84
   at csmacnz.Coveralls.CommandLineGitDataResolver.GenerateData() in c:\projects\coveralls-net\src\csmacnz.Coveralls\Com
mandLineGitDataResolver.cs:line 26
   at csmacnz.Coveralls.Program.<ResolveGitData>b__d(IGitDataResolver p) in c:\projects\coveralls-net\src\csmacnz.Covera
lls\Program.cs:line 182
   at System.Linq.Enumerable.WhereSelectListIterator`2.MoveNext()
   at System.Linq.Enumerable.FirstOrDefault[TSource](IEnumerable`1 source)
   at csmacnz.Coveralls.Program.ResolveGitData(MainArgs args) in c:\projects\coveralls-net\src\csmacnz.Coveralls\Program
.cs:line 182
   at csmacnz.Coveralls.Program.Main(String[] argv) in c:\projects\coveralls-net\src\csmacnz.Coveralls\Program.cs:line 1
18

As a XR-Baboon coverage user, I want to be able to publish my coverage results to coveralls.io so that I can monitor my coverage results over time.

Automagically resolve service name, job id, service number etc when detected running on appveyor. So you don't have to declare them.

As a mono Code Coverage user, I want to be able to publish my coverage results when using mono to coveralls.io so that I can monitor my coverage results over time.

I would like the coveralls build number to match the one created by AppVeyor. I tried setting the --JobId, like this

packages\coveralls.net.0.6.0\tools\csmacnz.coveralls.exe --opencover -i ./coverage.xml 
   --jobId %APPVEYOR_BUILD_NUMBER%

but it did't work. Here's the yml.

I want to see the application's icon in the filesystem so that the app looks more complete.

Getting "Failed to upload to coveralls" error when calling csmacnz.Coveralls.exe.

What should I check?

As an integration test I want to run the app multiple times with various inputs without publishing to coveralls.io so that I can avoid having test data in my coveralls.io stats.

Add a --dryrun parameter to the console app, that when provided, skips the publishing of the results.

Hi,

Since days I've been trying to upload the results of coverage from opencover to coveralls.io using coveralls.net.
I've tried several configurations, and finally I ended it to this

$opencoverPath = "user"
 If ($platform -eq "x86") {
       $opencoverPath = "Path32"
   } ElseIf ($platform -eq "x64") {
       $opencoverPath = "Path64"
   }

     Write-Host "Producing Coverage"

    $opencover = (Resolve-Path "src/packages/OpenCover.*/tools/OpenCover.Console.exe").ToString()
    $nunit = (Resolve-Path "src/packages/NUnit.Runners.*/tools/$nunitapp").ToString()
    & $opencover -register:$opecoverPath -target:"$nunit" -targetargs:"$root\src\deveeldb-nunit\bin\$platform\$conf\deveeldb-nunit.dll /noshadow /domain:single" -output:"$root\dist\$platform\coverage.xml"

     Write-Host "Sending Coverage results to Coveralls.io"

     $coveralls = (Resolve-Path "src/packages/coveralls.net.*/tools/csmacnz.coveralls.exe").ToString()
     & $coveralls --opencover -i "$root\dist\$platform\coverage.xml" --repoToken $COVERALLS_REPO_TOKEN --jobId $env:APPVEYOR_BUILD_NUMBER --serviceName appveyor

While the other configurations were giving me errors when trying to invoke coveralls.net, this one only prints out the usage, such as if the parameters passed were wrong.

Can you please verify if anything there is wrong?

As a Visual Studio Code Coverage user, I want to be able to publish my coverage results to coveralls.io so that I can monitor my coverage results over time.

As a .Net developer with a github project who wants coverage on coveralls.io I want to be able to install the csmacnz.coveralls.exe from a nuget package so that I can have it available on my build servers, which will publish the results to coveralls.io

As an Appveyor user I want to not have to provide the default environment variables on my CI build so that coverall.net is easier to use.

As a user i want to be able to optionally leave out optional command line parameters so that I can be less verbose.