Need a continuous development setup for this project about ovn-scale-test HOT 11 CLOSED

Just stating the obvious here but for the scalability tests to be meaningful we need the underlying CI system remain the same from test to test and have certain properties. Do the services provided by the sites mentioned above provide this?

from ovn-scale-test.

Good idea :) Do you have resources to host and setup the jenkins service? I have not consider this yet, if you have any plan, please let's me know.

from ovn-scale-test.

No. But I am thinking something like circleci [1]. Given the traffic volume as of now, we may only need free account from them. Once the gate is setup, it will ensure the new PR will not break the existing code. Do you have any other idea?

• Hui

[1] https://circleci.com/gh/docker/libnetwork/3055

from ovn-scale-test.

Thanks for bringing it up. Yes, this is one of the main purposes of this project, to see the control-plane scalability impact of each change. We can set up some baseline tests in the beginning, e.g. port creation & binding ... At least 1k sandboxes/containers (the more the better) need to be supported in this CI/CD env.

from ovn-scale-test.

Another public CI service is https://travis-ci.org

from ovn-scale-test.

If I understand correctly, what Hui mentioned is a smoke test procedure with CI/CD, which is used to ensure new PR not break existing 'ovn-scale-test' code; what Han mentioned is a scale test env with CI, which is used to check how ovs's commits affect its performance. The goals are different, right?

I don't know if these public CI services provide enough resource to do scale-test with 1k sandboxes. We can start from a smoke test CI/CD. I didn't use circleci and travis-ci before, not sure which one is better. I found this link Travis CI vs. Codeship vs. CircleCI, let's us find out which one is better for this repo.

from ovn-scale-test.

Lei, thanks for elaborating my idea. I agree that we should start from such smoke test and extend it to the scalability test.

from ovn-scale-test.

Travis CI provides a new VM for each build, I guess other CI sites work in a similar way. It makes sense for build and unit-test, but for smoke test and scale test it may not viable.

OVS is using Travis CI to build and/or run 'make distcheck', if necessary we can try to use Travis CI firstly.

from ovn-scale-test.

@l8huang ok, lets start with travis CI. I can create the functional/unit test script for docker ovn-scale-test. I believe @l8huang will do it in the sandbox way ; )

from ovn-scale-test.

@mestery has prepared the CI/CD script for travis-ci. Lets wait for that.

from ovn-scale-test.

Handled here: #50

from ovn-scale-test.