capability to get/match container logs about kubetest HOT 2 CLOSED

According to the k8s documentation, we can use CoreV1Api's read_namespaced_pod_log to get logs of the specified Pod/container.

Here is an example of blackbox's leader logs:

time="2018-09-14T13:53:31Z" level=debug msg="[poller] re-scanning synse server"
time="2018-09-14T13:53:31Z" level=debug msg="[poller] reading device: &{rack-1 vec 12835beffd3e6c603aa4dd92127707b5}"
time="2018-09-14T13:53:31Z" level=debug msg="[poller] reading device: &{rack-1 vec 34c226b1afadaae5f172a4e1763fd1a6}"
time="2018-09-14T13:53:31Z" level=debug msg="[poller] reading device: &{rack-1 vec 12ea5644d052c6bf1bca3c9864fd8a44}"
time="2018-09-14T13:53:31Z" level=debug msg="[discovery] found service endpoint address" addr="10.16.7.11:5000" portName=http targetRef=synse-server-5bbb476b66-42cr8
time="2018-09-14T13:53:31Z" level=info msg="[discovery] discovery complete" results="[10.16.7.11:5000]"
time="2018-09-14T13:53:31Z" level=debug msg="[discovery] updating addresses" addrs="[10.16.7.11:5000]"
time="2018-09-14T13:53:31Z" level=debug msg="[discovery] done update at 2018-09-14 13:53:31.713162134 +0000 UTC m=+35140.018947679"
time="2018-09-14T13:53:31Z" level=info msg="[client] updated current address" address="10.16.7.11:5000"
time="2018-09-14T13:53:31Z" level=debug msg="[poller] reading device: &{rack-1 vec bcf0618c50bff9121cb10d141d66f46f}"

It follows the format

time=<timestamp> level=<logging_level> msg=[<component>] <component's log>

Here are some matching strategies that I can think of:

• Filter out the log by component, parsing them properly then verify against our mocked data.
• Save/Parse the output to a file and compare/diff with our mocked one. If the diff is less than X, say 10%, then it would be considered successful.
• Have a set of possible keywords and only check against them. For example, isLeader=true, [poller] synse server reads complete, [postgres] adding new readings,...

from kubetest.

I think the first two parse strategies could be tricky, namely because we won't be able to reliably mock out log data (I don't think we can reliably determine the order that certain things will run). Even if we measure closeness of the diff to within some percentage, I'm not sure that would buy us a whole lot for the amount of work that would take.

The last solution is probably the best because its the simplest. We can expand on it later as we see a need, but I think just being able to check for various states (like, isLeader=true or synse server reads complete) would be enough to verify that certain components are behaving how we expect.

For the first case, simple existence should be fine, e.g. "is isLeader=true found anywhere in the logs?". Eventually, we may need to consider how to search for reoccurring log messages (e.g. "check that it had isLeader=false first, then after election it was isLeader=true). That would be much further down the line though.

from kubetest.