lennym / choma Goto Github PK

If you use a test suite with, for instance, faker, you might want to use the same seed for choma and faker to be able to reproduce a test run. You should not only be able to set the seed by an environment variable, but also to read that seed to pass it to faker.seed(...).

On my CI/CD:

Then, locally:

I see the choma seed printed matches the one in my CI/CD system, but the tests don't run in the same order. I expected the ordering to be the same across different machines.

But!

I noticed when I created a small PoC to test this, that things consistently ran as expected:

CI/CD:

Locally:

The files containing tests are exactly the same locally & in the CI/CD system. They're run on different OSes (Debian Stretch vs Ubuntu 16.04) but otherwise things are the same (node version and all deps)

I don't know the choma code well. What might cause this difference between a simple few test files vs a complex bunch of tests?