ibm / cloud-operators Goto Github PK

Provision and bind IBM Cloud services to your Kubernetes cluster in a Kubernetes-native way

License: Apache License 2.0

Dockerfile 0.21% Makefile 2.34% Go 94.55% Shell 2.90%

cloud-operators's Introduction

IBM Cloud Operator

With the IBM Cloud Operator, you can provision and bind IBM public cloud services to your Kubernetes cluster in a Kubernetes-native way. The IBM Cloud Operator is based on the Kubernetes custom resource definition (CRD) API so that your applications can create, update, and delete IBM Cloud services from within the cluster by calling Kubnernetes APIs, instead of needing to use several IBM Cloud APIs in addition to configuring your app for Kubernetes.

Features

Service provisioning: Create any service with any plan that is available in the IBM Cloud catalog.
Bindings management: Automatically create Kubernetes secrets in your Kubernetes cluster with the credentials to bind to any provisioned service to the cluster.

Upgrading the operator

Follow the instructions for the kind of installation you used: OperatorHub or the curl installation command.

`curl` installation command

To upgrade existing curl installs, you can reinstall the operator with the curl installation command. If your current installation is version 0.1.x or 0.2.x, then follow these steps to complete a successful upgrade.

Copy your secrets and configmaps to use the new names.
Re-run the curl installation command to install the latest version.

Verify the new operator is running.

kubectl get pods -n ibmcloud-operator-system

Verify ibmcloud.ibm.com/v1 Service versions provision correctly.
Delete the old version with kubectl delete ns ibmcloud-operators.
Delete the secrets and configmaps using the old names from step 1.

OperatorHub

To upgrade an OperatorHub installation, use the OpenShift web console to perform the upgrade. If upgrading from the alpha channel subscription to stable, follow the additional upgrade instructions below.

Upgrading to version v0.3 or v1.0

IMPORTANT NOTICE: v0.1 and v0.2 used a different naming scheme for secrets and configmaps. Before you update the IBM Cloud Operator, create secret and configmap resources with these names and copy the contents of your previous resources to the new resources. Then, the upgraded operator recognizes and continues to update the resources.

Existing Service and Binding resources do not need modification for the upgrade.

Previous names (v0.1 and v0.2)	Current names (v0.3 and v1.0)	Description
secret-ibm-cloud-operator	ibmcloud-operator-secret	Secret with the API key, scoped to the namespace.
config-ibm-cloud-operator	ibmcloud-operator-defaults	ConfigMap with the default values for new resources.
ibm-cloud-operator	ibmcloud-operator-config	ConfigMap with the management namespace configuration.
${namespace}-secret-ibm-cloud-operator	${namespace}-ibmcloud-operator-secret	Management namespace Secret with the API key for ${namespace}.
${namespace}-config-ibm-cloud-operator	${namespace}-ibmcloud-operator-defaults	Management namespace ConfigMap with default values for new resources in ${namespace}.

Tip: Forgot to update the secret and configmap names before upgrading? The operator will not take action on Service and Binding resources until the new secrets and configmaps have been created. Creating these after the upgrade will also work.

OperatorHub stable channel

Continuous updates and bug fixes are provided in the latest stable channel. Subscribing to the stable channel in OperatorHub means you automatically get the latest updates without breaking backward compatibility.

Changelog

For a list of recent changes, see the releases page.

Prerequisites

Have an IBM Cloud account.
Have a cluster that runs Kubernetes version 1.11 or later (OpenShift 3.11 or later).
Install the required command line tools.
- IBM Cloud CLI (ibmcloud)
- Kubernetes CLI (kubectl)
Log in to your IBM Cloud account from the CLI.
```
ibmcloud login
```
Target the appropriate resource group (-g) and default region (-r) for provisioning services:
```
ibmcloud target -g default -r us-south
```
If you also want to provision Cloud Foundry services, target the appropriate org and space (--cf).
```
ibmcloud target --cf
```
Set the Kubernetes context of your CLI to your cluster so that you can run kubectl commands. For example, if your cluster runs OpenShift, use the oc login command.

If your cluster is in IBM Cloud Kubernetes Service, run the following command.

ibmcloud ks cluster config -c <cluster_name_or_ID>

To check that your Kubernetes context is set to your cluster, run the following command.

kubectl config current-context

Setting up the operator

You can use an installation script to set up the IBM Cloud Operator. By default, the installation script stores an API key in a Kubernetes secret in your cluster that can be accessed by the IBM Cloud Operator. Next, the script sets default values that are used to provision IBM Cloud services, like the resource group and region to provision the services in. Later, you can override any default value in the Service custom resource. Finally, the script deploys the operator in your cluster.

To use your own API key, set the IBMCLOUD_API_KEY environment variable to the key before running the installation script:

export IBMCLOUD_API_KEY="CUSTOM_API_KEY"

If installed with Operator Hub, the operator will run in the openshift-operators namespace. Otherwise, it will run in the ibmcloud-operator-system namespace.

Prefer to create the secrets and defaults yourself? See the IBM Cloud Operator installation guide.

Using a service ID

By default, the installation script creates an IBM Cloud API key that impersonates your user credentials, to use to set up the IBM Cloud Operator. However, you might want to create a service ID in IBM Cloud Identity and Access Managment (IAM). By using a service ID, you can control access for the IBM Cloud Operator without having the permissions tied to a particular user, such as if that user leaves the company. For more information, see the IBM Cloud docs.

Create a service ID in IBM Cloud IAM. If possible, do not use spaces in the names for your IAM credentials. When you use the operator binding feature, any spaces are replaced with underscores.
```
ibmcloud iam service-id-create serviceid-ico -d service-ID-for-ibm-cloud-operator
```
Assign the service ID access to the required permissions to work with the IBM Cloud services that you want the operator to manage. The required permissions vary with each IBM Cloud service. You can also scope an access policy to different regions or resource groups. For example, the following command grants the service ID the Administrator platform role in the default resource group in the US South (Dallas) region. For more information, see the IBM Cloud docs.
```
ibmcloud iam service-policy-create serviceid-ico --roles Administrator --resource-group-name default --region us-south
```

Create an API key for the service ID.

ibmcloud iam service-api-key-create apikey-ico serviceid-ico -d api-key-for-ibm-cloud-operator

Set the API key of the service ID as your CLI environment variable. Now, when you run the installation script, the script uses the service ID's API key. The following command is an example for macOS.
```
export IBMCLOUD_API_KEY=<apikey-ico-value>
```
Confirm that the API key environment variable is set in your CLI.
```
echo $IBMCLOUD_API_KEY
```
Follow the prerequisite steps to log in to the IBM Cloud account that owns the service ID.

Installing the operator for OpenShift clusters

Before you begin, complete the prerequisite steps to log in to IBM Cloud and your cluster, and optionally set up a service ID API key.

To configure the latest release for OpenShift before installing via the OperatorHub, run the following script:

Latest release:

curl -sL https://raw.githubusercontent.com/IBM/cloud-operators/master/hack/configure-operator.sh | bash

Specific release: Replace -v 0.0.0 with the specific version that you want to install.

curl -sL https://raw.githubusercontent.com/IBM/cloud-operators/master/hack/configure-operator.sh | bash -s -- -v 0.0.0 store-creds

Installing the operator for Kubernetes clusters

Before you begin, complete the prerequisite steps to log in to IBM Cloud and your cluster, and optionally set up a service ID API key.

Latest release: To install the latest stable release of the operator, run the following script.

curl -sL https://raw.githubusercontent.com/IBM/cloud-operators/master/hack/configure-operator.sh | bash -s -- install

Specific release: Replace -v 0.0.0 with the specific version that you want to install.

curl -sL https://raw.githubusercontent.com/IBM/cloud-operators/master/hack/configure-operator.sh | bash -s -- -v 0.0.0 install

Uninstalling the operator

WARNING: This is a destructive operation. The uninstaller deletes the Service and Binding CRDs, which deletes the custom resources of those types. If you only want to upgrade to a newer version, only delete the operator's Deployment.

Before you begin, complete the prerequisite steps to log in to IBM Cloud and your cluster.

To remove the operator and all of the associated custom resources, run the following script:

curl -sL https://raw.githubusercontent.com/IBM/cloud-operators/master/hack/configure-operator.sh | bash -s -- remove

Using the IBM Cloud Operator

To use the IBM Cloud Operator, create a service instance and then bind the service to your cluster. For more information, see the sample configuration files, user guide, and reference documentation.

Step 1: Creating a service instance

To create an instance of an IBM public cloud service, first create a Service custom resource file. For more options, see the Service properties reference doc.
- <SERVICE_CLASS> is the IBM Cloud service that you want to create. To list IBM Cloud services, run ibmcloud catalog service-marketplace and use the Name value from the output.
- <PLAN> is the plan for the IBM Cloud service that you want to create, such as free or standard. To list available plans, run ibmcloud catalog service <SERVICE_CLASS> | grep plan.
```
apiVersion: ibmcloud.ibm.com/v1
kind: Service
metadata:
    name: myservice
spec:
    plan: <PLAN>
    serviceClass: <SERVICE_CLASS>
```
Create the service instance in your cluster.
```
kubectl apply -f filepath/myservice.yaml
```

Check that your service status is Online in your cluster.

kubectl get services.ibmcloud
NAME           STATUS   AGE
myservice      Online   12s

Verify that your service instance is created in IBM Cloud.
```
ibmcloud resource service-instances | grep myservice
```

Step 2: Binding the service instance

To bind your service to the cluster so that your apps can use the service, create a Binding custom resource, where the serviceName field is the name of the Service custom resource that you previously created. For more options, see Binding properties.
```
apiVersion: ibmcloud.ibm.com/v1
kind: Binding
metadata:
    name: mybinding
spec:
    serviceName: myservice
```

Create the binding in your cluster.

kubectl apply -f filepath/mybinding.yaml

Check that your service status is Online.

kubectl get bindings.ibmcloud
NAME         STATUS   AGE
mybinding    Online   25s

Check that a secret of the same name as your binding is created. The secret contains the service credentials that apps in your cluster can use to access the service.
```
kubectl get secrets
NAME         TYPE      DATA   AGE
mybinding    Opaque    6      102s
```

Using separate IBM Cloud accounts

You can provision IBM Cloud services in separate IBM Cloud accounts from the same cluster. To use separate accounts, update the secrets and configmap in the Kubernetes namespace where you want to create services and bindings.

Tip: Just want to use a different account one time and don't want to manage a bunch of namespaces? You can also specify a different account in the individual service configuration, by overriding the default account context.

Get the IBM Cloud account details, including account ID, Cloud Foundry org and space, resource group, region, and API key credentials.
Edit or replace the ibmcloud-operator-secret secret in the Kubernetes namespace that you want to use to create services in the account.
Edit or replace the ibmcloud-operator-defaults configmap in the Kubernetes namespace that you want to use to create services in the account.
Optional: Set up a management namespace so that cluster users with access across namespaces cannot see the API keys for the different IBM Cloud accounts.

Setting up a management namespace

By default, the API key credentials and other IBM Cloud account information are stored in a secret and a configmap within each namespace where you create IBM Cloud Operator service and binding custom resources. However, you might want to hide access to this information from cluster users in the namespace. For example, you might have multiple IBM Cloud accounts that you do not want cluster users in different namespaces to know about.

Create a management namespace that is named safe.
```
kubectl create namespace safe
```

In the namespace where the IBM Cloud Operator runs, create an ibmcloud-operator-config configmap that points to the safe namespace.

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
  name: ibmcloud-operator-config
  namespace: openshift-operators # Or update to the namespace where IBM Cloud Operator is running
  labels:
    app.kubernetes.io/name: ibmcloud-operator
data:
  namespace: safe
EOF

Copy the existing or create your own ibmcloud-operator-secret secrets and ibmcloud-operator-defaults configmaps from the other Kubernetes namespace into the safe namespace. Important: You must rename the secrets and configmaps with the following naming convention.
```
<namespace>-ibmcloud-operator-secret
<namespace>-ibmcloud-operator-defaults
```
For example, if you have a cluster with three namespaces default, test, and prod:
```
default-ibmcloud-operator-secret
default-ibmcloud-operator-defaults
test-ibmcloud-operator-secret
test-ibmcloud-operator-defaults
prod-ibmcloud-operator-secret
prod-ibmcloud-operator-defaults
```
Delete the ibmcloud-operator-secret secrets and ibmcloud-operator-defaults configmaps in the other Kubernetes namespaces.

Reference documentation

Service Properties

A Service custom resource includes the properties in the following table. Each parameter value is treated as a raw value by the operator and passed as-is. For more information, see the IBM Cloud Operator user guide.

Parameter	Required	Type	Comments
serviceClass `*`	Yes	`string`	The IBM Cloud service that you want to create. To list IBM Cloud services, run `ibmcloud catalog service-marketplace` and use the Name value from the output.
plan `*`	Yes	`string`	The plan to use for the service instance, such as `free` or `standard`. To use an existing service instance, set to `Alias`. To list available plans, run `ibmcloud catalog service <SERVICE_CLASS>
serviceClassType `*`	CF only	`string`	Set to `CF` for Cloud Foundry services. Otherwise, omit this field.
externalName `*`	No	`string`	The name for the service instance in IBM Cloud, such as in the console.
parameters	No	`[]Param`	Parameters that are passed in to create the service instance. These parameters vary by service, and can be anything, such as a number, string, or object.
tags	No	`[]string`	The IBM Cloud tag to assign the service instance, to help organize your cloud resources such as in the IBM Cloud console.
context	No	`Context`	The IBM Cloud account context to use instead of the default account context.

* Note: The serviceClass, plan, serviceClassType, and externalName parameters are immutable. After you set these parameters, you cannot later edit their values. If you do edit the values, the changes are overwritten back to the original values.

Binding Properties

A Binding custom resources includes the properties in the following table. For more information, see the IBM Cloud Operator user guide.

Field	Required	Type	Comments
serviceName	Yes	`string`	The name of the `Service` resource that corresponds to the service instance on which to create credentials for the binding.
serviceNamespace	No	`string`	The namespace of the `Service` resource.
alias	No	`string`	The name of existing IBM Cloud credentials to link this binding to. This binding creates a secret for these credentials in the cluster namespace, but cannot modify the existing credentials in IBM Cloud. Note that any spaces are replaced with underscores.
secretName	No	`string`	The name of the `Secret` to be created. If you do not specify a value, the secret is given the same name as the binding.
role	No	`string`	The IBM Cloud IAM role to create the credentials to the service instance. Review the each service's documentation for a description of the roles. If you do not specify a role, the IAM `Manager` service access role is used. If the service does not support the `Manager` role, the first returned role from the service is used.
parameters	No	`[]Any`	Parameters that are passed in to create the create the service credentials. These parameters vary by service, and can be anything, such as an integer, string, or object.

Account context in operator secret and configmap

The IBM Cloud Operator needs an account context, which indicates the API key and the details of the IBM Cloud account to be used for creating services. The API key is stored in a ibmcloud-operator-secret secret that is created when the IBM Cloud Operator is installed. Account details such as the account ID, Cloud Foundry org and space, resource group, and region are stored in a ibmcloud-operator-defaults configmap.

When you create an IBM Cloud Operator service or binding resource, the operator checks the namespace that you create the resource in for the secret and configmap. If the the operator does not find the secret and configmap, it checks its own namespace for a configmap that points to a management namespace. Then, the operator checks the management namespace for <namespace>-ibmcloud-operator-secret secrets and <namespace>-ibmcloud-operator-defaults configmaps. If no management namespace exists, the operator checks the default namespace for the ibmcloud-operator-secret secret and ibmcloud-operator-defaults configmap.

You can override the account context in the Service configuration file with the context field, as described in the following table. You might override the account context if you want to use a different IBM Cloud account to provision a service, but do not want to create separate secrets and configmaps for different namespaces.

Field	Required	Type	Description
org	No	`string`	The Cloud Foundry org. To list orgs, run `ibmcloud account orgs`.
space	No	`string`	The Cloud Foundry space. To list spaces, run `ibmcloud account spaces`.
region	No	`string`	The IBM Cloud region. To list regions, run `ibmcloud regions`.
resourceGroupID	No	`string`	The IBM Cloud resource group ID. To list resource groups, run `ibmcloud resource groups`.
resourceLocation	No	`string`	The location of the resource.

Versions

Review the supported Kubernetes API versions for the following IBM Cloud Operator versions.

Operator version	Kubernetes API version
`v0.3` or later	`v1`
`v0.2`	`v1beta1` or `v1alpha1`
`v0.1`	`v1alpha`

Contributing to the project

See Contributing to Cloud Operators

Troubleshooting

See the Troubleshooting guide.

cloud-operators's People

Contributors

Stargazers

Watchers

cloud-operators's Issues

DB2 sample needs to be updated, plan & service name have changed

Make parameters for Service updatable

Example COS service with binding not working

https://github.com/IBM/cloud-operators/blob/master/config/samples/cos.yaml

Message: Failed: No deployment found for service plan lite at location us-south. Valid location(s) are: ["global"].

Object description:

Name:         b2c-auth-dev-cos-dal-01-test
Namespace:    tpol
Labels:       <none>
Annotations:  kubectl.kubernetes.io/last-applied-configuration:
                {"apiVersion":"ibmcloud.ibm.com/v1alpha1","kind":"Service","metadata":{"annotations":{},"name":"b2c-auth-dev-cos-dal-01-test","namespace":...
API Version:  ibmcloud.ibm.com/v1alpha1
Kind:         Service
Metadata:
  Creation Timestamp:  2019-09-04T12:02:39Z
  Generation:          1
  Resource Version:    16278396
  Self Link:           /apis/ibmcloud.ibm.com/v1alpha1/namespaces/tpol/services/b2c-auth-dev-cos-dal-01-test
  UID:                 e4a0f379-cf0b-11e9-a426-e27ade6abd72
Spec:
  Plan:           lite
  Service Class:  cloud-object-storage
Status:
  Context:
    Org:
    Region:
    Resourcegroup:
    Resourcelocation:
    Space:
  Message:             Failed: No deployment found for service plan lite at location us-south. Valid location(s) are: ["global"].
Use service instance example if the service is a Cloud Foundry service
  Plan:                lite
  Service Class:       cloud-object-storage
  Service Class Type:
  State:               Failed
Events:                <none>

Deletion of Binding corresponding to a Failed service fails

Use UBI image as the base image?

Is that possible to publish ibm-cloud-operator with http://registry.access.redhat.com/ubi8-minimal image?

Immutability check in service controller failing

Create a translator
Change serviceClass to messagehub
Result is 3 services (2 messagehubs and 1 translator)

Generate an Activity Tracker event for service creation

When using the IBM Cloud operator, any service creation results in a service creation event is logged into Activity Tracker. Currently, because of the way IBM Cloud operator works, the initiator will be corresponding to the API key/service id used by AT instead of the actual user who initiate the service creation. The request here is to add a feature to Cloud operator to create an additional service creation event that shows the actual user who created the service.

It might be possible to some how create a service with the user's name, then we don't need another AT event.

Please contact me if you have any questions.

Deleting a service that has just been deleted manually from Console fails

{"level":"info","ts":1572487334.5800693,"logger":"service","msg":"Error deleting resource","mytranslator":"Request failed with status code: 400, ServerErrorResponse: {"message":"Instance is already in pending_reclamation state.","status_code":400}\n"}

The controller doesn't recover from this

Deletion of Service/Binding CRD instances failed to delete service instance

Context:

I was testing an example application example that uses an instance of the Watson Language Translator from the IBM cloud. I was using an IKS 1.14.x cluster with cloudoperators/ibmcloud-operator:0.1.0 installed.

At the start of a sequence of operations, I had no instances of binding or service CRDs in the cluster and no instances of the Watson Language Translator in my cloud account.

At the end of a sequence of operations, I had no instances of binding or service CRDs in the cluster, BUT there was still an instance of the Watson Language Translator in my cloud account (still there 24 hours later). Based on the name of the instance, it was the one created by the ibmcloud operator and not deleted when the CRDs were deleted.

Attached is the relevant section of logs from the operator.

log.txt

Make fields of binding spec immutable

Console UI fails to load for example `mystreams` service

I followed the instructions to create a service and binding for Streams (e.g., mystreams service instance). The service and binding are created. When I click on the mystreams service from the OCP cluster console I get an empty page which you cannot go back.

Looking in the web browser console I see the following error.

`WebSocket connection to 'wss://console.dan-rhos10-f0a5715bb2873122b708ede2bf765701-0001.us-east.containers.appdomain.cloud/api/kubernetes/apis/ibmcloud.ibm.com/v1alpha1/namespaces/default/services?watch=true&fieldSelector=metadata.name%3Dmystreams&x-csrf-token=XXXXXXXXXXXX' failed: WebSocket is closed before the connection is established.
o.destroy @ index.tsx:52'

Improve Documentation

Document the following:

When a binding is deleted the associated secret is deleted as well. One way to refresh credentials is to delete a binding and recreate
Document the meaning of all optional parameters of Service and Binding
Elaborate on how to obtain service and plan names
Explain global region in the context for some services
Explain why some services display an inactive or failed status until they become online (that we simply channel the status coming from Bluemix)
Add section on Credential aliases
Add section on immutability of some fields in Service spec

Bug for service provisioning with multiple bindings

I provision a cloudant service with multiple bindings according to the following yaml file:

apiVersion: ibmcloud.ibm.com/v1alpha1
kind: Service
metadata:
  name: my-cloudant
spec:
  plan: standard 
  serviceClass: cloudantnosqldb
---
apiVersion: ibmcloud.ibm.com/v1alpha1
kind: Binding
metadata:
  name: my-binding-cloudant-1
spec:
  serviceName: my-cloudant
---
apiVersion: ibmcloud.ibm.com/v1alpha1
kind: Binding
metadata:
  name: my-binding-cloudant-2
spec:
  serviceName: my-cloudant

However, I can see there are two service instances in IBM Cloud Portal resource list, one has two bindings, another one has none, showing as below:

Operator creates an excessive number of policy entries

Each time a binding CR is created, it creates a corresponding credential in the service even if a credential of the same name already exists on that service.

Each credential, in turn, creates an access policy against the account.

An account has a hard limit of 600 policies that can be created. In several accounts, using the IBM Cloud operator on a medium scale deployment has resulted in policy limit to be hit.

This may be compounded by some other issue in the Operator where the credentials are not cleaned up properly when the service is destroyed and/or the Operator creating orphaned policies when it removes/re-creates the binding.

Question: The secret in the binding, can it be revoked or rotated?

The usage docs describe:

A Binding generates a secret with the same name as the binding resource and contains service credentials that can be consumed by your application.

Is there anyway to revoke or rotate the secret?

Thinking of use cases where the secret has been compromised or if it needs to be rotated periodically due to some biz security policy.

If there is a better place to ask such questions please let me know.

Thank You

FEATURE: Leverage IAM identity of kubernetes user when applicable (IKS and Red Hat OpenShift on IBM Cloud)

When using IKS or Red Hat OpenShift on IBM Cloud users are authenticating with openshift/kubernetes with an IBM Cloud IAM identity. Any thoughts on if it would be possible to use this identity for authentication with IBM Cloud when trying to manage the services?

It may be a pipedream, but I do think it's worth investigation to see if there could be a method for leveraging this identity.

Long provision times for Binding should not Fail (expect Pending)

Example: postgresql

Document behavior of `self-healing` annotation

There is a closed issue (#11) that added an annotation to control the self-healing behavior and there is at least one example that includes the self-healing annotation.

However, there is no documentation anywhere else as to what exactly this is doing and how to apply it. I believe this annotation will address an issue on my project but I'm hesitant to use it without getting a better understanding of how it works.

install-operator.sh is failed due to I don't know how to create secret

$ curl -sL https://raw.githubusercontent.com/IBM/cloud-operators/master/hack/install-operator.sh | bash

Downloading install yaml...
error: error validating "STDIN": error validating data: [unknown object type "nil" in Secret.data.api-key, unknown object type "nil" in Secret.data.region]; if you choose to ignore these errors, turn validation off with --validate=false

bindings fail for sysdig

apiVersion: ibmcloud.ibm.com/v1alpha1
kind: Service
metadata:
    name: sysdiglite
spec:
    plan: lite
    serviceClass: sysdig-monitor
---
apiVersion: ibmcloud.ibm.com/v1alpha1
kind: Binding
metadata:
    name: binding-sysdiglite
spec:
    serviceName: sysdiglite

The result is that the service.ibmcloud is just fine, but i get the following for the binding.ibmcloud

Name:         binding-sysdiglite
Namespace:    default
Labels:       <none>
Annotations:  kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"ibmcloud.ibm.com/v1alpha1","kind":"Binding","metadata":{"annotations":{},"name":"binding-sysdiglite","namespace":"default"},"spec":{"ser...
API Version:  ibmcloud.ibm.com/v1alpha1
Kind:         Binding
Metadata:
  Creation Timestamp:  2019-07-11T15:08:01Z
  Finalizers:
    binding.ibmcloud.ibm.com
  Generation:  1
  Owner References:
    API Version:           ibmcloud.ibm.com/v1alpha1
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  Service
    Name:                  sysdiglite
    UID:                   acea4390-a3ed-11e9-a260-7a9326862861
  Resource Version:        6010884
  Self Link:               /apis/ibmcloud.ibm.com/v1alpha1/namespaces/default/bindings/binding-sysdiglite
  UID:                     acf32645-a3ed-11e9-a260-7a9326862861
Spec:
  Service Name:  sysdiglite
Status:
  Instance Id:      crn:v1:bluemix:public:sysdig-monitor:us-south:a/33c5711b8afbf7fd809a4529de613a08:0d1d43af-29e8-4098-a7ad-f1252f5ca684::
  Key Instance Id:  crn:v1:bluemix:public:sysdig-monitor:us-south:a/33c5711b8afbf7fd809a4529de613a08:0d1d43af-29e8-4098-a7ad-f1252f5ca684:resource-key:0e21964e-e3d2-4c63-82f4-c4ccdd42af04
  Message:          Secret "binding-sysdiglite" is invalid: [data[Sysdig Access Key]: Invalid value: "Sysdig Access Key": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name',  or 'KEY_NAME',  or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[Sysdig Collector Endpoint]: Invalid value: "Sysdig Collector Endpoint": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name',  or 'KEY_NAME',  or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[Sysdig Customer Id]: Invalid value: "Sysdig Customer Id": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name',  or 'KEY_NAME',  or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[Sysdig Endpoint]: Invalid value: "Sysdig Endpoint": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name',  or 'KEY_NAME',  or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+')]
  State:            Failed
Events:             <none>

Missing stable releases

Hi, is that possible to create any release tag to publish the stable version instead of always installing with the latest master.zip?

Add parameters to Binding and Service

Deletion fails for cloudant, if service is manually deleted first on bluemix

{"level":"info","ts":1583890921.6279922,"logger":"service","msg":"Error deleting resource","my-cloudant":"Request failed with status code: 400, ServerErrorResponse: {"message":"Instance is pending reclamation. Please restore the instance and retry.","status_code":400,"transaction_id":"bss-47d37a02da442b8d"}\n"}

Make default namespace configurable

operator should look for seed-secret in current namespace, and if not present, in the namespace specified in seed-defaults, using a naming convention (one seed-secret per namespace)

Put seed-default /seed-secrets into an `admin-only` accessible workspace

Per @paolo, Another one is to create secrets in an admin-only accessible workspace, with some naming convention, such as ico-secret- and have the operator look up there as well to get the API key. This way we can keep a even better separation and use kube best practices for management of secrets.

Let us use this issue to track this feature development.

Cloudant with Alias plan: existence checks sometimes fails, gets recreated

The controller should not recreate an Alias service.
Recreating an existing Cloudant service seems to crash it and causes its deletion.

Fix Travis build (TAG now required in env)

Problem when creating a service that can only have one instance

Some services such as DB2 can only have one instance created for the free plan.
So if there is already such a service instantiated, and another one is created with this operator then that fails.

However, if the existing service is deleted then the operator should recover and create a new instance (since the reason for failure no longer exists). This does not happen and we are left with an ill-formed object that has no Status.State.

The status of a service that takes a long time to provision is not updated appropriately

[Question] Where does the olm was published

I found that we have a folder for olm https://github.com/IBM/cloud-operators/tree/master/olm , the question is for on-line install, where I can get those olm files? From operatorhub or some other places?

Make self-healing controllable via annotation

Currently self-healing is enabled by default. Since that might not be always desirable, especially for stateful services, we should control self-healing with an annotation such as:

annotations:
    ibmcloud.ibm.com/self-healing: enabled

Label not present or any other value than enabled should disable the self-healing feature.

account information checks from config-operator.sh

Sometimes when using the IBM Cloud CLI, the region is not being set resulting in IC_REGION being empty. This results in rather obscure errors during secret creation like:

error: error validating “STDIN”: error validating data: unknown object type “nil” in Secret.data.region; if you choose to ignore these errors, turn validation off with --validate=false

It would be helpful to do a quick validation of the IC... fields in this script and putting up an error message to help users spot and fix the CLI session configuration issue.

secretName in Binding spec was not used for secret name

Tried the binding operator with the following yaml. It appears that the operator did not name the secret with secretName as specified in yaml. The secret was named binding-messagehub instead of laura-messagehub-secret.

apiVersion: ibmcloud.ibm.com/v1alpha1
kind: Binding
metadata:
  name: binding-messagehub
spec:
  serviceName: mymessagehub
  secretName: laura-messagehub-secret

seed-secret-tokens is recreated every 30 seconds

2019/09/04 11:10:30 Registering Components.
{"level":"info","ts":1567595430.383996,"logger":"kubebuilder.controller","msg":"Starting EventSource","controller":"binding-controller","source":"kind source: /, Kind="}
{"level":"info","ts":1567595430.384281,"logger":"kubebuilder.controller","msg":"Starting EventSource","controller":"binding-controller","source":"kind source: /, Kind="}
{"level":"info","ts":1567595430.384528,"logger":"kubebuilder.controller","msg":"Starting EventSource","controller":"service-controller","source":"kind source: /, Kind="}
{"level":"info","ts":1567595430.3847806,"logger":"kubebuilder.controller","msg":"Starting EventSource","controller":"token-controller","source":"kind source: /, Kind="}
2019/09/04 11:10:30 Starting the Cmd.
{"level":"info","ts":1567595430.4853036,"logger":"kubebuilder.controller","msg":"Starting Controller","controller":"token-controller"}
{"level":"info","ts":1567595430.4853926,"logger":"kubebuilder.controller","msg":"Starting Controller","controller":"binding-controller"}
{"level":"info","ts":1567595430.485429,"logger":"kubebuilder.controller","msg":"Starting Controller","controller":"service-controller"}
{"level":"info","ts":1567595430.585615,"logger":"kubebuilder.controller","msg":"Starting workers","controller":"token-controller","worker count":1}
{"level":"info","ts":1567595430.5858042,"logger":"kubebuilder.controller","msg":"Starting workers","controller":"service-controller","worker count":30}
{"level":"info","ts":1567595430.5857067,"logger":"kubebuilder.controller","msg":"Starting workers","controller":"binding-controller","worker count":33}
{"level":"info","ts":1567595430.5860016,"logger":"iam-token","msg":"reconciling IBM cloud IAM tokens","secretRef":"seed-secret"}
{"level":"info","ts":1567595430.5862157,"logger":"iam-token","msg":"authenticating..."}
{"level":"info","ts":1567595431.5611506,"logger":"iam-token","msg":"creating tokens secret","name":"seed-secret-tokens"}
{"level":"info","ts":1567595431.6002471,"logger":"iam-token","msg":"secret created","name":"seed-secret-tokens"}
{"level":"info","ts":1567595460.457385,"logger":"iam-token","msg":"reconciling IBM cloud IAM tokens","secretRef":"seed-secret"}
{"level":"info","ts":1567595460.457479,"logger":"iam-token","msg":"authenticating..."}
{"level":"info","ts":1567595461.023003,"logger":"iam-token","msg":"creating tokens secret","name":"seed-secret-tokens"}
{"level":"info","ts":1567595461.1133666,"logger":"iam-token","msg":"secret created","name":"seed-secret-tokens"}
{"level":"info","ts":1567595490.4581864,"logger":"iam-token","msg":"reconciling IBM cloud IAM tokens","secretRef":"seed-secret"}
{"level":"info","ts":1567595490.4582605,"logger":"iam-token","msg":"authenticating..."}
{"level":"info","ts":1567595490.6674898,"logger":"iam-token","msg":"creating tokens secret","name":"seed-secret-tokens"}
{"level":"info","ts":1567595490.7247064,"logger":"iam-token","msg":"secret created","name":"seed-secret-tokens"}
{"level":"info","ts":1567595520.459059,"logger":"iam-token","msg":"reconciling IBM cloud IAM tokens","secretRef":"seed-secret"}
{"level":"info","ts":1567595520.4591346,"logger":"iam-token","msg":"authenticating..."}
{"level":"info","ts":1567595520.7943714,"logger":"iam-token","msg":"creating tokens secret","name":"seed-secret-tokens"}
{"level":"info","ts":1567595520.8722453,"logger":"iam-token","msg":"secret created","name":"seed-secret-tokens"}
{"level":"info","ts":1567595550.459095,"logger":"iam-token","msg":"reconciling IBM cloud IAM tokens","secretRef":"seed-secret"}
{"level":"info","ts":1567595550.459168,"logger":"iam-token","msg":"authenticating..."}
{"level":"info","ts":1567595550.6792963,"logger":"iam-token","msg":"creating tokens secret","name":"seed-secret-tokens"}
{"level":"info","ts":1567595550.7460973,"logger":"iam-token","msg":"secret created","name":"seed-secret-tokens"}
{"level":"info","ts":1567595580.4596167,"logger":"iam-token","msg":"reconciling IBM cloud IAM tokens","secretRef":"seed-secret"}
{"level":"info","ts":1567595580.4597182,"logger":"iam-token","msg":"authenticating..."}
{"level":"info","ts":1567595581.3568347,"logger":"iam-token","msg":"creating tokens secret","name":"seed-secret-tokens"}
{"level":"info","ts":1567595581.4320064,"logger":"iam-token","msg":"secret created","name":"seed-secret-tokens"}
{"level":"info","ts":1567595610.4605396,"logger":"iam-token","msg":"reconciling IBM cloud IAM tokens","secretRef":"seed-secret"}
{"level":"info","ts":1567595610.4606142,"logger":"iam-token","msg":"authenticating..."}
{"level":"info","ts":1567595610.647946,"logger":"iam-token","msg":"creating tokens secret","name":"seed-secret-tokens"}
{"level":"info","ts":1567595610.7103074,"logger":"iam-token","msg":"secret created","name":"seed-secret-tokens"}
{"level":"info","ts":1567595640.4604962,"logger":"iam-token","msg":"reconciling IBM cloud IAM tokens","secretRef":"seed-secret"}
{"level":"info","ts":1567595640.4605722,"logger":"iam-token","msg":"authenticating..."}
{"level":"info","ts":1567595641.455251,"logger":"iam-token","msg":"creating tokens secret","name":"seed-secret-tokens"}
{"level":"info","ts":1567595641.5189216,"logger":"iam-token","msg":"secret created","name":"seed-secret-tokens"}
{"level":"info","ts":1567595670.4615536,"logger":"iam-token","msg":"reconciling IBM cloud IAM tokens","secretRef":"seed-secret"}
{"level":"info","ts":1567595670.4616308,"logger":"iam-token","msg":"authenticating..."}
{"level":"info","ts":1567595670.781947,"logger":"iam-token","msg":"creating tokens secret","name":"seed-secret-tokens"}
{"level":"info","ts":1567595670.8239803,"logger":"iam-token","msg":"secret created","name":"seed-secret-tokens"}
{"level":"info","ts":1567595700.4620593,"logger":"iam-token","msg":"reconciling IBM cloud IAM tokens","secretRef":"seed-secret"}
{"level":"info","ts":1567595700.46217,"logger":"iam-token","msg":"authenticating..."}
{"level":"info","ts":1567595701.2653816,"logger":"iam-token","msg":"creating tokens secret","name":"seed-secret-tokens"}
{"level":"info","ts":1567595701.4408503,"logger":"iam-token","msg":"secret created","name":"seed-secret-tokens"}
{"level":"info","ts":1567595730.4625704,"logger":"iam-token","msg":"reconciling IBM cloud IAM tokens","secretRef":"seed-secret"}
{"level":"info","ts":1567595730.4626958,"logger":"iam-token","msg":"authenticating..."}
{"level":"info","ts":1567595731.3687491,"logger":"iam-token","msg":"creating tokens secret","name":"seed-secret-tokens"}
{"level":"info","ts":1567595731.6761453,"logger":"iam-token","msg":"secret created","name":"seed-secret-tokens"}
{"level":"info","ts":1567595760.4628518,"logger":"iam-token","msg":"reconciling IBM cloud IAM tokens","secretRef":"seed-secret"}
{"level":"info","ts":1567595760.4629266,"logger":"iam-token","msg":"authenticating..."}
{"level":"info","ts":1567595762.2687404,"logger":"iam-token","msg":"creating tokens secret","name":"seed-secret-tokens"}
{"level":"info","ts":1567595762.9215274,"logger":"iam-token","msg":"secret created","name":"seed-secret-tokens"}
{"level":"info","ts":1567595790.4639907,"logger":"iam-token","msg":"reconciling IBM cloud IAM tokens","secretRef":"seed-secret"}
{"level":"info","ts":1567595790.4640663,"logger":"iam-token","msg":"authenticating..."}
{"level":"info","ts":1567595790.766511,"logger":"iam-token","msg":"creating tokens secret","name":"seed-secret-tokens"}
{"level":"info","ts":1567595790.893934,"logger":"iam-token","msg":"secret created","name":"seed-secret-tokens"}
{"level":"info","ts":1567595820.4639208,"logger":"iam-token","msg":"reconciling IBM cloud IAM tokens","secretRef":"seed-secret"}
{"level":"info","ts":1567595820.464038,"logger":"iam-token","msg":"authenticating..."}
{"level":"info","ts":1567595821.0999365,"logger":"iam-token","msg":"creating tokens secret","name":"seed-secret-tokens"}
{"level":"info","ts":1567595821.1949873,"logger":"iam-token","msg":"secret created","name":"seed-secret-tokens"}
{"level":"info","ts":1567595850.4647455,"logger":"iam-token","msg":"reconciling IBM cloud IAM tokens","secretRef":"seed-secret"}
{"level":"info","ts":1567595850.464858,"logger":"iam-token","msg":"authenticating..."}
{"level":"info","ts":1567595851.0882504,"logger":"iam-token","msg":"creating tokens secret","name":"seed-secret-tokens"}
{"level":"info","ts":1567595851.1474195,"logger":"iam-token","msg":"secret created","name":"seed-secret-tokens"}
{"level":"info","ts":1567595880.4655426,"logger":"iam-token","msg":"reconciling IBM cloud IAM tokens","secretRef":"seed-secret"}
{"level":"info","ts":1567595880.4656255,"logger":"iam-token","msg":"authenticating..."}
{"level":"info","ts":1567595881.6596067,"logger":"iam-token","msg":"creating tokens secret","name":"seed-secret-tokens"}
{"level":"info","ts":1567595881.7811937,"logger":"iam-token","msg":"secret created","name":"seed-secret-tokens"}
{"level":"info","ts":1567595910.4656918,"logger":"iam-token","msg":"reconciling IBM cloud IAM tokens","secretRef":"seed-secret"}
{"level":"info","ts":1567595910.4657555,"logger":"iam-token","msg":"authenticating..."}
{"level":"info","ts":1567595910.7067425,"logger":"iam-token","msg":"creating tokens secret","name":"seed-secret-tokens"}
{"level":"info","ts":1567595910.7580051,"logger":"iam-token","msg":"secret created","name":"seed-secret-tokens"}
{"level":"info","ts":1567595940.466467,"logger":"iam-token","msg":"reconciling IBM cloud IAM tokens","secretRef":"seed-secret"}
{"level":"info","ts":1567595940.4665887,"logger":"iam-token","msg":"authenticating..."}
{"level":"info","ts":1567595941.3466737,"logger":"iam-token","msg":"creating tokens secret","name":"seed-secret-tokens"}
{"level":"info","ts":1567595941.4124768,"logger":"iam-token","msg":"secret created","name":"seed-secret-tokens"}
{"level":"info","ts":1567595970.46677,"logger":"iam-token","msg":"reconciling IBM cloud IAM tokens","secretRef":"seed-secret"}
{"level":"info","ts":1567595970.4668615,"logger":"iam-token","msg":"authenticating..."}
{"level":"info","ts":1567595970.9634576,"logger":"iam-token","msg":"creating tokens secret","name":"seed-secret-tokens"}
{"level":"info","ts":1567595971.0092242,"logger":"iam-token","msg":"secret created","name":"seed-secret-tokens"}
{"level":"info","ts":1567596000.467724,"logger":"iam-token","msg":"reconciling IBM cloud IAM tokens","secretRef":"seed-secret"}
{"level":"info","ts":1567596000.4677951,"logger":"iam-token","msg":"authenticating..."}
{"level":"info","ts":1567596000.628489,"logger":"iam-token","msg":"creating tokens secret","name":"seed-secret-tokens"}
{"level":"info","ts":1567596000.7129726,"logger":"iam-token","msg":"secret created","name":"seed-secret-tokens"}
{"level":"info","ts":1567596030.4676917,"logger":"iam-token","msg":"reconciling IBM cloud IAM tokens","secretRef":"seed-secret"}
{"level":"info","ts":1567596030.4677715,"logger":"iam-token","msg":"authenticating..."}
{"level":"info","ts":1567596030.6161482,"logger":"iam-token","msg":"creating tokens secret","name":"seed-secret-tokens"}
{"level":"info","ts":1567596030.6956184,"logger":"iam-token","msg":"secret created","name":"seed-secret-tokens"}
{"level":"info","ts":1567596031.6005745,"logger":"iam-token","msg":"reconciling IBM cloud IAM tokens","secretRef":"seed-secret"}
{"level":"info","ts":1567596031.6007035,"logger":"iam-token","msg":"authenticating..."}
{"level":"info","ts":1567596032.3850238,"logger":"iam-token","msg":"creating tokens secret","name":"seed-secret-tokens"}
{"level":"info","ts":1567596032.4597018,"logger":"iam-token","msg":"secret created","name":"seed-secret-tokens"}

.travis.yml: The 'sudo' tag is now deprecated in Travis CI

Travis are now recommending removing the sudo tag.

"If you currently specify sudo: false in your .travis.yml, we recommend removing that configuration"

Resources with Alias plan should consider external name

Add docs for service context info

Document that binding secret keys will replace spaces with underscores

uninstall script did not remove ibmcloud-operator-manager-role

ibmcloud-operator-manager-role does not have labels and was not removed by running uninstall-operator.sh.

Request to create `release` tag and upload deployment yaml file as releases assets

The current installation approach in https://github.com/IBM/cloud-operators/blob/master/hack/install-operator.sh#L37 is not friendly enough to end-user.

Suggest to create releases for this version, and attach the yaml files as releases assets, so that we can use https://github.com/IBM/cloud-operators/releases/download/.....yaml to do the installation.

Also, suggest to create an overall yaml file for deployment as well .

Can not provision IBM service with a service id

I used a IBM cloud service id(Administrator role)'s apikey to provision a service by using cloud-operator the latest release, the error threw:
Request failed with status code: 400, BXNIM0106E: Validation of property 'response_type' with value '[uaa, cloud_iam]' failed. Valid values: 'cloud_iam, delegated_refresh_token'

I have verified that with the service id's api key, I can successfully provision service by using ibmcloud cli.
So does cloud-operator support service id?

Rename seed-default and seed-secrets

Per discussion in #91, I created a new issue to bring this up

===================

@pdettor I just curious, are the names seed-secret and seed-defaults configurable? If we put them into the end-user's namespace where the service/bind resource being created, I would like to use a more explicitly name , i.e. secret-ibm-cloud-operator and config-ibm-cloud-operator to avoid the deletion by end-user.

Answered by @vazirim
@cdlliuy Sure, we can rename.

===================

Bindging did not detect credential removal on remote service

When a credential (originally created by Binding operator) is deleted manually on IBM cloud console, Binding operator did not detect the missing credential and the old secret remained unchanged.

Document usage of `secretName` in Binding

We should document the usage of secretName in Binding.

Mitigate the denial of service from Bluemix

Cannot create a binding to existing Cloudant service

I'm trying to apply this yaml:

apiVersion: ibmcloud.ibm.com/v1alpha1
kind: Binding
metadata:
    name: cloudant
spec:
    plan: Alias
    serviceName: cloudant
    serviceClass: cloudantnosqldb

After applying and waiting 3-4mn I see:

Status:
  Message:  Processing Resource
  State:    Pending

It'd be nice if the status section reports what's wrong with either spec or my configuration.

Support existing services

Re-introduce the ability to reference and bind existing services via usage of the Alias plan.

Adopt the descriptors from the service binding specification

Once we have RC1 of the service binding specification and SBO supports it, all that IBM Cloud's operator would have to do to be considered compliant with the spec is update its x-descriptor to also use the spec's annotation (which should be very similar), and another simple update to claim that it is bindable.

Requires service.ibmcloud instances to be created in the default namespace

It's not finding the secret in the default namespace.

olm $ oc describe service.ibmcloud
Name:         mypersonality
Namespace:    ibmcloud
Labels:       <none>
Annotations:  kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"ibmcloud.ibm.com/v1alpha1","kind":"Service","metadata":{"annotations":{},"name":"mypersonality","namespace":"ibmcloud"},"spec":{"plan":"...
API Version:  ibmcloud.ibm.com/v1alpha1
Kind:         Service
Metadata:
  Creation Timestamp:  2019-07-10T19:29:16Z
  Generation:          1
  Resource Version:    16373
  Self Link:           /apis/ibmcloud.ibm.com/v1alpha1/namespaces/ibmcloud/services/mypersonality
  UID:                 017c240f-a349-11e9-8052-229d59c3b269
Spec:
  Plan:           lite
  Service Class:  personality-insights
Status:
  Context:
    Org:
    Region:
    Resourcegroup:
    Resourcelocation:
    Space:
  Message:             Secret "seed-secret" not found
  Plan:                lite
  Service Class:       personality-insights
  Service Class Type:
  State:               Failed
Events:                <none>