Code Monkey home page Code Monkey logo

trx-parser's Introduction

trx-parser 🧹

This Action was inspired by https://github.com/zyborg/dotnet-tests-report

Build Test GitHub release (latest by date)

This GitHub Action provides a way of parsing dotnet test results from trx files in a given directory. The action will find trx files specified in the TRX-PATH input variable. This path must be accessible to the action.

It will read each individual .trx file. Loads up its data and converts it to a typed json object to make it easier to traverse through the data. For each TRX, it will create a Github Status check and generate a markup report for each trx. The report name and title are derived from the trx file using the data.TestRun.TestDefinitions.UnitTest[0]._storage

Reports

The following reports show a failing and a passing check generated by this action.

Workflow Reports:

Usage

To make trx-parser a part of your workflow, just add the following to your existing workflow file in your .github/workflows/ directory in your GitHub repository.

name: Test
on: [pull_request]

jobs:
  Build:
    runs-on: ubuntu-latest
    steps:
      # Replace this whichever way you build your code
      - name: Build & Test dotnet code
        run: |
          dotnet restore
          dotnet build -c Release no-restore
          dotnet test -c Release --no-restore --no-build --logger trx --results-directory ./TestResults
      # Using the trx-parser action
      - name: Parse Trx files
        uses: NasAmin/[email protected]
        id: trx-parser
        with:
          TRX_PATH: ${{ github.workspace }}/TestResults #This should be the path to your TRX files
          REPO_TOKEN: ${{ secrets.GITHUB_TOKEN }}          

Inputs

Name Description Default Required
TRX_PATH Path to the directory container your test files ./TestResults yes
REPO_TOKEN GitHub token to create status checks nil yes
SHA The commit SHA associated with the checks. This is optional nil no
IGNORE_FAILURE If true, will not set status check as failure. false no
REPORT_PREFIX The prefix for the report name. Useful for matrix builds nil no

⚠️ GitHub Actions Limitations ⚠️

  • The GitHub Checks API has a limit of 65535 characters. So if the test report exceeds this limit, GitHub will fail to create a check and fail your workflow. This was mitigated on #103 and #138 to only report details about failing tests.
  • If you have multiple workflows triggered by the same event, currently GitHub Actions will randomly associate a check run to one of the workflows. This is because currently there is no Check Suite API. Only GitHub apps are allowed to create a Check Suite. There is also no way to associate a custom check run with an existing check suite. GitHub actions automatically creates a check suite for each workflow run. However, since check runs are associated with a commit and event, any custom check runs are randomly linked under one of the triggered workflows for the same commit.

Contributing

Anyone is welcome to contribute and make this action better. Please fork the repository and create a pull request with proposed changes.

Development

  • Clone this repository
  • Run npm run build and npm run test
  • Run npm run all to regenerate the dist directory.

trx-parser's People

Contributors

dependabot[bot] avatar flcdrg avatar nasamin avatar snyk-bot avatar wilka avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

trx-parser's Issues

Error: Cannot read property '_storage' of undefined

Getting the following error, any idea?

https://github.com/mocsharp/clara-dicom-adapter/runs/1663835318?check_suite_focus=true

Run NasAmin/[email protected]
  with:
    TRX_PATH: /home/runner/work/clara-dicom-adapter/clara-dicom-adapter/test-results
    REPO_TOKEN: ***
    IGNORE_FAILURE: false
Finding Trx files in: /home/runner/work/clara-dicom-adapter/clara-dicom-adapter/test-results
Files count: 10
Processing 7 trx files
Transforming file /home/runner/work/clara-dicom-adapter/clara-dicom-adapter/test-results/_3860a79a0dda_2021-01-07_16_58_06.trx
Error: Cannot read property '_storage' of undefined

Error: Cannot read property 'UnitTest' of undefined

when there's no test detected in the trx.

Error: Cannot read property 'UnitTest' of undefined

is thrown

example trx:

<?xml version="1.0" encoding="utf-8"?>
<TestRun id="42bf458b-ce32-46d6-8e27-0560a086d144" name="Cheng@CHENG-PC-SSD 2021-02-18 18:07:52" runUser="CHENG-PC-SSD\Cheng" xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
  <Times creation="2021-02-18T18:07:52.0946403-05:00" queuing="2021-02-18T18:07:52.0946418-05:00" start="2021-02-18T18:07:49.7128282-05:00" finish="2021-02-18T18:07:52.0960517-05:00" />
  <TestSettings name="default" id="df32bb72-e474-448b-8dae-bcc400803a90">
    <Deployment runDeploymentRoot="Cheng_CHENG-PC-SSD_2021-02-18_18_07_52" />
  </TestSettings>
  <TestLists>
    <TestList name="Results Not in a List" id="8c84fa94-04c1-424b-9868-57a2d4851a1d" />
    <TestList name="All Loaded Results" id="19431567-8539-422a-85d7-44ee4e166bda" />
  </TestLists>
  <ResultSummary outcome="Completed">
    <Counters total="0" executed="0" passed="0" failed="0" error="0" timeout="0" aborted="0" inconclusive="0" passedButRunAborted="0" notRunnable="0" notExecuted="0" disconnected="0" warning="0" completed="0" inProgress="0" pending="0" />
    <Output>
      <StdOut>blabla
</StdOut>
    </Output>
    <RunInfos>
      <RunInfo computerName="CHENG-PC-SSD" outcome="Warning" timestamp="2021-02-18T18:07:51.9559154-05:00">
        <Text>No test matches the given testcase filter `UnitTests` in C:\blabla\test\bin\Debug\net5.0\blabla.dll</Text>
      </RunInfo>
    </RunInfos>
  </ResultSummary>
</TestRun>

Workflow not failing if there are test failures

trx-parser is not configurable for whether to break the workflow or proceed further in case of test failures, rather all the following steps/jobs get executed regardless of test status.

It would give more control to a developer if trx-parser can be configured based on some input parameters such as BREAK_ON_TEST_FAILURE with boolean values or similar.

Support a 'sha' property to override which commit to associate with

If I wanted to use this action in a workflow triggered by workflow_run then the current sha defaults to main/master.

I'd like to be able to pass in github.event.workflow_run.head_sha instead (as that would be the sha from the original workflow who's completion caused the current workflow to run.

Add optional report prefix

Currently this Action takes the name of the underlying assembly for each trx parser and make that the title of the report and the name of the status check.

In some cases where you want to run the same tests in multiple environments e.g. Dev and staging. In such cases there is one status check created. To support some seggregation between different test runs in the same workflow, a prefix should be allowed e.g. dev- or staging

This action should create a separate check suit

Currently, if you have multiple workflows in your CI, this action randomly puts the check in one of those workflows.
This sometimes causing the PR to be blocked even if the check passes because it can't find the check (presumably).

What should really happen is this action should create a separate check suite (name TBD).

This issue is to explore some options and find out why the check is created in ramdom workflows and create a separate check suit outside of workflows.

But what should happen on pushes and non PR workflow. Can a separate check suit be created on pushes. If not then where should the check be? Because we still want to fail the pipeline if there is a failure.

Error: Invalid request - only 65535 characters are allowed

I am kicking the tires on this action using a project with about 1000 tests. Unfortunately it fails. Any suggestions?

Finding Trx files in: /home/runner/work/mit-yousee-odin-api/mit-yousee-odin-api/TestResults
Files count: 1
Processing 1 trx files
Transforming file /home/runner/work/mit-yousee-odin-api/mit-yousee-odin-api/TestResults/_fv-az186-920_2020-12-28_08_37_52.trx
Checking for failing tests
Creating PR check for ODINAPI SELFCARE.TESTS
Creating status check for GitSha: 97c859732655f4633222f0db3b6483c07ab21e70 on a push event
Check time is: Mon, 28 Dec 2020 08:38:34 GMT
Error: Invalid request.

Only 65535 characters are allowed; 1607778 were supplied.

Dependabot and GH API access

Been running into some issues with this action and Dependabot PR's, when uploading the test result to the GH API. I've got a organization level token specified ,(ORG_GITHUB_TOKEN) as the REPO_TOKEN parameter, since Dependabot does not have access to GITHUB-TOKEN but get the following errror

Syntax

    - name: Parse Trx files
      uses: NasAmin/[email protected]
      id: trx-parser
      if: always()
      with:
        TRX_PATH: ${{ github.workspace }}/TestResults
        REPO_TOKEN: ${{ secrets.ORG_GITHUB_TOKEN }}

Error

....
Creating PR check for ORBIT UNITTESTS
Creating status check for GitSha: c040653ee8bead3071fee97ac7d27f52[18](https://github.com/...api/runs/5380650173?check_suite_focus=true#step:7:18)6c681d on a pull request event
Check time is: Tue, 01 Mar 2022 [19](https://github.com/...api/runs/5380650173?check_suite_focus=true#step:7:19):09:12 GMT
Error: Resource not accessible by integration
....

Should Dependabot not be able to write to GH API with this?

Status check not created on push

The trx-parser only creates a status check on pull request event.
So when you merge your changes to the default branch, no checks are created on the build.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.