projectatomic / commissaire-service Goto Github PK

A default bus-uri was set pre-release for testing. It should be removed or updated to better default.

Example: https://github.com/mbarnes/commissaire-service/blob/8123d85562a7d72bdf204fd3f8330b83e3219b43/src/commissaire_service/clusterexec/__init__.py#L287

The command commissaire-storage-service depends on the requirement redis, but this is not part of the requirements.txt file.

    raise ImportError('Missing redis library (pip install redis)')
ImportError: Missing redis library (pip install redis)

Curious if there is a reason for this (because kombu allows different options). If not, I will add the dependency...

Example:

$ commissaire-containermgr-service 
Traceback (most recent call last):
  File "/home/steve/Tech/Envs/cenv/bin/commissaire-containermgr-service", line 11, in <module>
    load_entry_point('commissaire-service', 'console_scripts', 'commissaire-containermgr-service')()
  File "/home/steve/Tech/GITHUB/commissaire-service/src/commissaire_service/containermgr/__init__.py", line 239, in main
    config_file=args.config)
AttributeError: 'Namespace' object has no attribute 'config'

As a user of Commissaire
I would like the Container Management features to exist in it's own microservice
so that I am able to enable, disable, and scale container management like other components of Commissaire

Use the configurable logging mechanism in the base commissaire package.

Requires: projectatomic/commissaire#26

Commissaire should require versions of Ansible that have been patched for security bug CVE-2016-9587 [1], to whatever extent is possible. In particular, allow Ansible >= 2.1.4 but forbid (if possible) Ansible 2.2.0.

[1] http://www.mail-archive.com/[email protected]/msg29406.html

As a user of Commissaire
I would like the clusterexec host enumeration to be optimized
so that the calls to the store are only what is required.

• Discussion
• Code

See https://pagure.io/atomic-wg/issue/428

If no reply is received to this issue (or on the above ticket),
this repository will be archived.

Now that I can easily observe service logs, I'm noticing the storage log is filled with etcd.EtcdKeyNotFound tracebacks when running the end-to-end test suite.

I traced this to a few HTTP handlers that test whether a host or cluster is present by calling storage.get and then handling a RemoteProcedureCallError with the assumption that it just means the key wasn't found.

Let's handle this more gracefully by introducing a new storage primitive: peek()

peek() works like get() except when the key doesn't exist it returns an empty result rather than throwing a RemoteProcedureCallError. (The StorageClient.peek() method will therefore either return a model instance or None.)

As a future developer on Commissaire
I want the complexity of the service code to be smaller
so that it's easy to digest and understand what is happening within callables.

Services should all take in the, at a minimum, the following as CLI arguments:

• config
• bus-exchange
• bus-uri

Having a common parser which can be extended and used in services will help us keep DRY.

Example: https://github.com/projectatomic/commissaire-service/blob/master/src/commissaire_service/storage/__init__.py#L207-L237

Since we've settled on supporting only containerized Commissaire deployments, I'm wondering if we should remove ServiceManager or at least its multi-process capability (which we don't test and I've personally never used).

It seems to me a preferable strategy to only run one service instance per container, and let your container manager (Kubernetes, OpenShift, etc.) handle service replication if multiple, say, storage or investigator service instances are desired.

@ashcrow, do you agree? You wrote ServiceManager and I get the impression it was more for rpm-based deployments.

Some of our configuration items use underscores while others are using (dashes)[https://github.com/projectatomic/commissaire/blob/master/src/commissaire/storage/etcd.py#L82]. One should be chosen and used throughout commissaire-service.

Found by @intlabs

As a developer of commissaire
I would like services to be able to define their own config default paths
so I don't have to step through hoops when testing.

I'd like to investigate having the storage service push one-way notification messages for each save and delete request.

This would help other services be more autonomous by reacting to changes on their own instead of relying on HTTP request handlers to drive everything with explicit remote procedure calls. It would also help simplify the HTTP request handling logic in some places.

In particular, container-manager-service could be nearly self-sufficient by listening for host and cluster record changes and calling the appropriate plugin automatically. For example if the watcher-service discovers that a host is no longer accessible, it would update the host's status to "failed" and save it to storage. container-manager-service would detect this and automatically remove the node from its container manager.

The investigator-service could also kick off bootstrapping automatically in response to newly-added host records. I'm sure there will be other use cases as things progress.

I'm thinking of establishing a unique message queue for each model type (Host, Cluster, etc.), so services can effectively filter the types of change notifications received.