Original: #20

Original: #18

Hi!
Trying to play with knapsack-pro/jest in the project that was bootstrapped by the create-react-app tool and encountered an issue that jest cannot parse my files. Seems like it cannot read jest config provided by CRA.
Would be nice if you can create an example project that uses clean CRA (ideally with typescript) + knapsack-pro. CRA has become a very convenient tool that react community use, it would be much appreciated to have at least a docs section regarding it.
Thanks in advance!

https://github.com/KnapsackPro/knapsack_pro-ruby has KNAPSACK_PRO_SALT, but here did not find anything and tried to set it but filenames/branches are not encrypted.

is there any support for test file names encryption?

This should help with edge case when test file cannot be executed by test runner and because of that we won't record timing of that test file.

Edge case could happen when:

• test file is empty
• test file has only pending tests
• test file has syntax error

In such case, we could assume the test file has some default timing like 0.1s of 0 seconds.

Solution idea

We should improve this part of code:

// src/knapsack-pro-core.ts
        onSuccess(queueTestFiles).then(({ recordedTestFiles, isTestSuiteGreen }) => {
          // here we should take all tests from queueTestFiles and assume 0s of execution time for them
          // and merge them with recordedTestFiles.
          // In case recordedTestFiles has less test files than queueTestFiles then 
          // we would have 0s execution time for not executed test files by test runner
          // recordedTestFiles could have less tests when the test runner like Cypress/Jest won't be able to run it and it won't report the test file back to us
          this.recordedTestFiles = this.recordedTestFiles.concat(recordedTestFiles);
          this.isTestSuiteGreen = this.isTestSuiteGreen && isTestSuiteGreen;

          this.fetchTestsFromQueue(false, onSuccess, onFailure);
        });

Related

A similar issue was solved here: KnapsackPro/knapsack_pro-ruby#71 (this is old PR, we should use 0s as default time for not executed tests).

Original: #21

It seems like we can't run Cypress tests simultaneously using the NPM package Concurrently v5.0.0 since as soon as the first test starts in both workers, the package exits with no error and the message is: [2]+ Stopped

We're trying to use the following command (and rest of the environment variables are in our .env file):

yarn concurrently "KNAPSACK_PRO_CI_NODE_TOTAL=2 KNAPSACK_PRO_CI_NODE_INDEX=0 knapsack-pro-cypress" "KNAPSACK_PRO_CI_NODE_TOTAL=2 KNAPSACK_PRO_CI_NODE_INDEX=1 knapsack-pro-cypress"

It works fine when using two shells for the tests and also with our Jenkins pipeline*.

Original: #19

Original: #22

We've noticed a slowdown in our Jest tests after upgrading to @knapsack-pro/jest v5.4.1 and using the KNAPSACK_PRO_TEST_FILE_LIST_SOURCE_FILE option. What we're seeing is that the execution behavior of batches is very different and not optimized, leading to our Jest steps taking about 9 minutes (up from about 6 minutes). I've tried downgrading just the @knapsack-pro/core library from v4.0.0 to both v3.3.1 and v3.3.0, but did not see any change in behavior. However, reverting back from using KNAPSACK_PRO_TEST_FILE_LIST_SOURCE_FILE to KNAPSACK_PRO_TEST_FILE_PATTERN is what resolves the problem.

Here's a comparison of the two:

KNAPSACK_PRO_TEST_FILE_PATTERN
With this approach, you can see that the batches start with the slowest ones and get faster, leading to better durations when parallelized.

KNAPSACK_PRO_TEST_FILE_LIST_SOURCE_FILE
With this approach, you can see that the batches are not organized and as a result lead to many more batches being executed and durations taking much longer.

I tried to look through the changes, but didn't find an explanation for why the batching behavior would be different between the two approaches so I'm creating this issue in the hopes that the context you have can provide an answer.

Hello,

I started using Knapsack cypress pro for parallel executing component testing. I observed that the total duration is almost double due to continuous webpack compilation and project build before running the spec files.

Basically, when I specified 2 nodes for cypress component testing, each node invoked knapsack pro queue API to get the test spec for running and the API always returns only 1 spec file for executing the test spec instead it should return a list of test spec files which then get's build and executed without redundant webpack build step.

Original: #26

Original: #23

I used to get knapsack-pro to work with my github actions. But recently I am changing to run github actions based on my custom docker image and I tried to configure the same things as before. But I get complaints about

app/node_modules/@knapsack-pro/core/lib/config/knapsack-pro-env.config.js:32
            throw new Error("Please set test suite API token in CI environment variables.\n      Please check README for the Knapsack Pro client library.");

my workflow.yml looks like this:

steps:
            - uses: actions/checkout@v2
            - name: Pull image
              run: docker pull <image> && docker-compose -f docker-compose.test.yml up -d
              # run: docker-compose up -d
            - name: Configure Knapsack Pro
              run: |
                  echo ::set-env name=KNAPSACK_PRO_TEST_SUITE_TOKEN_CYPRESS::${{ secrets.KNAPSACK_PRO_TEST_SUITE_TOKEN_CYPRESS }}
                  echo ::set-env name=KNAPSACK_PRO_TEST_FILE_PATTERN::"cypress/integration/**/*.spec.{js,ts}"
                  echo ::set-env name=KNAPSACK_PRO_CI_NODE_TOTAL::${{ matrix.ci_node_total }}
                  echo ::set-env name=KNAPSACK_PRO_CI_NODE_INDEX::${{ matrix.ci_node_index }}
            - name: run test
              run: docker exec server  npm run test:e2e:ci (this is just my script)

I enjoyed using knapsack a lot and spent quite a lot of time on setting this, I really appreciate your help.

Original: #24

Release minor version to test the monorepo and publishing steps.

Original: #25

about the feature

Allow user to enable encryption of sensitive data like:

• test file paths
• branch names

why we need this feature?

Some of the customers requested encryption for Jest.

idea how to implement it

Let's keep it simple.

Here is some info about the existing encryption feature for Ruby which might be too complex. We could do it simpler for Knapsack Pro in JavaScript (feature should be implemented in @knapsack-pro/core npm package so other packages like @knapsack-pro/jest or @knapsack-pro/cypress could use it).

In knapsack_pro ruby gem there is an encryption feature but it is a bit overcomplicated. It allows encrypting separately test file paths and branch names. It does not allow decryption so that's a downside for the development and performance of some of the features like RSpec split by examples because there is no simple way to decrypt test files on CI without encrypting all test files first to get SHA2 key (SHA2 value is used to compare SHA2 values from API response to identify test file paths).

In case of @knapsack-pro/core library I would recommend:

• use some JS library (TODO: research what library would be a good fit) that allows to encrypt and decrypt string based on a key. If env var likeKNAPSACK_PRO_ENCRYPTION_KEY has value then it could mean encryption is enabled and the key will be used to encrypt and decrypt string like test file path & branch names.
• There are some branch names that are exceptions and they should not be encrypted because Knapsack Pro API uses them to better split tests in some edge cases situation. Here is list of branch names that should never be encrypted.

story

https://trello.com/c/f494kFmB

We've been using Knapsack Pro with our Jest tests for several months now with no problems; however, when we bump our version of Jest to the latest (29.7.0), the runs in Github Actions go from ~10 minutes each to ~50. The initial load of the app has always been very slow; however, it's a one-time cost, as I believe Jest uses a local cache for subsequent runs. Instead of something like this:

// ...Knapsack API response...
PASS a.test.ts (3 s)
PASS b.test.ts (2 s)
PASS c.test.ts (3 s)
// ...Knapsack API response...
PASS d.test.ts (3 s)
PASS e.test.ts (2 s)
PASS f.test.ts (3 s)

...we're seeing this:

// ...Knapsack API response...
PASS a.test.ts (75 s)
PASS b.test.ts (2 s)
PASS c.test.ts (3 s)
// ...Knapsack API response...
PASS d.test.ts (74 s)
PASS e.test.ts (2 s)
PASS f.test.ts (3 s)

...with the first test of each Jest run taking longer than a minute.

Is our suspicion correct, that Jest is for some reason ignoring the cache and reloading all the modules each time now?

Cypress already has a config file that says where the test source files are located. We could use that. Thanks to that user would not have to set KNAPSACK_PRO_TEST_FILE_PATTERN if he keeps tests in non-standard directory.