webdriverio-boneyard / wdio-sumologic-reporter Goto Github PK

The problem

Sumologic reporter is not consistently transactionizing all tests - some blocks within a given spec will get all three events, where others do not.

Environment

• WebdriverIO version: 4.2.8
• Node.js version: 6.3.0
• Standalone mode or wdio testrunner: testrunner
• if wdio testrunner, running synchronous or asynchronous tests: sync
• Additional wdio packages used (if applicable): mocha, spec/dot, sumologic

Details

each block within a spec is determined to have a test:start, test:(result), and test:end - not all blocks within a spec are consistently receiving this event tag. after running a spec several times, i see that some blocks are getting only 1 or 2 of the expected 3 test state events.

Link to Selenium/WebdriverIO logs

spec reporter output (consistently 5 tests reported)

[Chrome #0a] Session ID: 8d49a74edc2e4cbb0da6a8f88f862ad894282cfd
[Chrome #0a] Spec: /Users/jterry/huge.amc-webdriverio/test/specs/homeHeroCarouselLarge_Test.js
[Chrome #0a] Running: Chrome
[Chrome #0a]
[Chrome #0a] AMC Homepage Hero Carousel - Full
[Chrome #0a] ✓ Carousel Next is visible at >1260
[Chrome #0a] ✓ Carousel Previous is visible at >1260
[Chrome #0a] ✓ Carousel Next works
[Chrome #0a] ✓ Carousel Previous works
[Chrome #0a] ✓ Carousel Buttons work
[Chrome #0a]
[Chrome #0a]
[Chrome #0a] 5 passing (33s)
[Chrome #0a]

here is a gist that should hopefully just point out something silly i shouldn't do:
https://gist.github.com/jonyet/da3b9859a26de94ba48dd33ac5e93afd

gist with the csv output from sumo:
https://gist.github.com/jonyet/089eda1a0e2e00e69cc0a34b38d07b9f

you can see in the csv that groups 3, 4, and 5 are the examples of the issue.

relevant block from wdio.conf.js:

  framework: 'mocha',
  reporters: ['spec', 'sumologic'],
    reporterOptions: {
        allure: {
            outputDir: './test/reports/allure-results/'
        },
        sumologic: {
          syncInterval: 100,
          sourceAddress: process.env.SUMO_SOURCE_ADDRESS
        }
    },

Code To Reproduce Issue [ Good To Have ]

will work on reproducing this with an example you can use, yourself

Please remember that, with sample code; it's easier to reproduce bug and much faster to fix it.