What would you like to be documented:

There are many templates currently provided in the repo. Some install the CCM while others do not and the docs also provide steps to explicitly install the CCM.

Why is this needed:

For clarification purposes.

What would you like to be added:
The work for this issue will be done in the cluster-api repo, but will impact this provider. We will need to modify https://github.com/kubernetes-sigs/cluster-api/blob/a52bb727d22f13042d9810a6bf6566204ea28345/cmd/clusterctl/client/config/providers_client.go to make sure OCI shows up in the list

Why is this needed:
This will make it easier for customers to find our provider.

What would you like to be added:
As part of #94 some investigation went into upgrading the machine pool's images (InstanceConfiguration). It turns out it was a bit more involved. We will need figure out how to cordon/drain each node and then delete and recreate. This is what machine deployments do.

As part of this we will need to do the following:

• If the image is changed create a new InstanceConfiguration
• Update the machine pool to use the new InstanceConfirguration (this won't update the existing instances)
• Grab a list of the existing instances and iterate
  • cordon/drain the node (wait until done)
  • terminate instance (wait for new instance to start instance pools should automatically do this)
• delete old InstanceConfiguration

Why is this needed:
If we want users using machine pools a major use case will be upgrading their cluster. We need to make sure they are able to upgrade without service interruption.

What would you like to be documented:

Add documentation for VCN Peering feature

Why is this needed:

Customers can refer to documentation for using VCN peering feature.

What would you like to be added:

Add the OracleClusterTemplate resource such that it can be used with Cluster API's ClusterClass construct as per https://github.com/kubernetes-sigs/cluster-api/blob/main/docs/proposals/20210526-cluster-class-and-managed-topologies.md

Why is this needed:

Compatibility with ClusterClass.

cc @sedefsavas, @yastij and @shysank if there's any guidance to be given from the other CAP* providers.

What would you like to be added:
We need to have the ability to run all the e2e tests using Instance Principal

Why is this needed:
Since we allow using Instance Principals we need to make sure we have tests around the feature.

We need to work on getting a provider channel in the Kubernetes slack so we can then support our users.

We will need to follow this doc on how to do that: https://www.kubernetes.dev/docs/comms/slack/#requesting-a-channel

What would you like to be added:
Now that #55 has been merged we need to work on adding more validation checks for things like the following:

• NetworkSpec (valid CIDRs)
• Region is valid (might need to call to OCI for validation or the OCI SDK might have pre-flight validation we could use)
• ClusterName (length?)
• For cluster update region is considered an immutable field. We will want to handle other immutable fields

Why is this needed:
Now that #55 is merged we should expand upon our validation. This issue isn't to 100% cover all possibilities, but rather cover what we can now that make sense.

What would you like to be documented:

We should use instance_principal when installing CCM instead of asking the user for credentials.

Why is this needed:
It's more secure

What happened:
kubectl logs in control plane nodes are not working. Default NSG should allow 10250 port to be open in Control plane Subnet for Control Plane CIDR

What you expected to happen:
kubectl logs for pods running on control plane nodes should work.

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

Environment:

• CAPOCI version:
• v0.1.0
• Cluster-API version (use clusterctl version):
• Kubernetes version (use kubectl version):
• Docker version (use docker info):
• OS (e.g. from /etc/os-release):

What happened:
My capoci-controller-manager pod was in CrashLoopBackOff. Looking at the logs for it I saw:

E0322 15:17:19.001678 1 logr.go:270] setup "msg"="unable to create OCI VCN Client" "error"="can not create client, bad configuration: x509: decryption password incorrect"

Looking at the secret itself, the passphrase was set to this:

% echo -n "T0NJX0NSRURFTlRJQUxTX1BBU1NQSFJBU0U=" | base64 -d
OCI_CREDENTIALS_PASSPHRASE

Going back to documentation, the Authentication Configuration section here is using the name of an environment variable instead of its value:

export OCI_CREDENTIALS_PASSPHRASE_B64="$(echo -n "OCI_CREDENTIALS_PASSPHRASE" | base64 | tr -d '\n')"

should be

export OCI_CREDENTIALS_PASSPHRASE_B64="$(echo -n "$OCI_CREDENTIALS_PASSPHRASE" | base64 | tr -d '\n')"

What would you like to be added:
CAPOCI must have ability to use external Cluster Infrastructure except for API Server Load Balancer

Why is this needed:
In some cases, customers would like to use externally provisioned network infrastructure except for API Server LoadBalancer. CAPOCI must have the ability to do that.

What happened: The links to installation instructions on the Getting Started on the Introduction section of the book are broken and error 404

What you expected to happen: The links to take me to the installation instructions

How to reproduce it (as minimally and precisely as possible): Click on the links

Anything else we need to know?:

Environment:

• CAPOCI version: 0.1.0
• Cluster-API version (use clusterctl version):
• Kubernetes version (use kubectl version):
• Docker version (use docker info):
• OS (e.g. from /etc/os-release):

What would you like to be added:
We should rename the identity.Client interface to something like IdentityClient

Why is this needed:
This will keep our client interfaces consistent like the ComputeClient

What would you like to be added:
When the user chooses a bare metal shape that requires the PV_TRANSIT_ENCRYPTION to be false we should fast-fail before trying to launch a cluster. As a part of this we should look at other checks we should preform.

Why is this needed:
The user experience with in-flight encryption is somewhat hostile in that they aren't informed the cluster will fail until the machines fail to start.

What would you like to be documented:
We need to document how machine pools work

Why is this needed:
#89 added support for machine pools, but now we need to document how it works for users.

What would you like to be added:
As a part of #89 MachinePool support was added. We should add more support for machine pools to use the template variables.

• instanceConfigurationId should be updatable.
• AssignPublicIp
• ProviderIDList
• Update cluster.go to use less ListFaultDomainsRequest calls

Why is this needed:
These will help make machine pools better for users.

What would you like to be added:
We need to test the new 1.2 tag release (probably cut 2022-06-20) to make sure CAPOCI won't have issues with the next version release.

Migration guide: https://main.cluster-api.sigs.k8s.io/developer/providers/v1.1-to-v1.2.html#required-api-changes-for-providers

Why is this needed:
The version 1.2 of Cluster-API will be released in 2 - 3 weeks so we need to test and make sure we won't have any issues with this.

What would you like to be added:
We want to remove the mandatory from CompartmentId string `mandatory:"true" json:"compartmentId"` in the OCIClusterSpec and setup the mandatory validation in ocicluster_webhook

Why is this needed:
This seems to be the preferred way to do validation and it will allow use to remove the compartmentId: REPLACE defined in templates/clusterclass-example.yaml

What would you like to be added:
We need to test the new 1.1.5 release (probably cut 2022-07-04) to make sure CAPOCI won't have issues with the release.

We will want to run e2e tests using 1.1.5 at the minimum.

Why is this needed:
Need to make sure things don't break.

What would you like to be added:
Now that #55 has been merged we need to work on adding more validation checks for things like the following:

• CompartmentId
• Shape is valid (might need to call to OCI for validation or the OCI SDK might have pre-flight validation we could use. This one might not be straight foreword so some thought should be used on this validation via a webhook or not.)
• NetworkDetails (valid CIDRs)
• IsPvEncryptionInTransitEnabled (some bare metal shapes require this to be false https://docs.oracle.com/en-us/iaas/releasenotes/changes/60d602f5-abb3-4639-aa19-292a5744a808/)

Why is this needed:
Now that #55 is merged we should expand upon our validation. This issue isn't to 100% cover all possibilities, but rather cover what we can now that make sense.

What would you like to be documented:
I noticed some typos here: https://oracle.github.io/cluster-api-provider-oci/

• “The Cluster API Provier for OCI (CAPOCI) brings declarative, Kubernetes-style APIs to cluster creation, configuration and management” Provider is missing the alphabet ‘d’.

• “Deployment process: Chosing your deployment path” -> Should be “Choosing”

Why is this needed:
Fix typos

What would you like to be documented:
Add a section on using instance principal in CAPOCI

Why is this needed:
If management cluster is in OCI, Instance Principal is the recommended and secure authentication mechanism to be used instead of user principal.

What would you like to be added:
Users should have the ability to peer management cluster VCN with workload cluster VCN. There are various techniques that can be used such as Local Peering Gateway or Dynamic Routing Gateway. Using Dynamic Routing Gateway will provide a single approach to peer VCNs in same as well as different regions and hence recommended as an initial implementation.

Why is this needed:
This is required for private cluster support.

What happened:
webhook validations failing for templates/cluster-template-arm-free-tier.yaml. Since the subnet CIDR's are set the validation fails.

What you expected to happen:
The blank CIDRs would still pass validation, but would be automatically populated by the provider

How to reproduce it (as minimally and precisely as possible):
Try to create a workload cluster using the templates/cluster-template-arm-free-tier.yaml template.

Anything else we need to know?:

Environment:

• CAPOCI version: 0.3.0
• Cluster-API version (use clusterctl version):
• Kubernetes version (use kubectl version):
• Docker version (use docker info):
• OS (e.g. from /etc/os-release):

What would you like to be added:
There is a new release of OCI CCM 1.23 oracle/oci-cloud-controller-manager#393
we need to test to make sure there aren't any issues and update the e2e tests to use the newer versions

Why is this needed:
We need to make sure we work with this new version.

What would you like to be added:
This issue is specifically focused on ListInstancePoolInstances for machine pools. There is an other issue for confirming all the other list api calls are paginated (#97)

Why is this needed:
Machine Pools will possibly have many instances. We want to land this change quickly as this could cause MachinePools issues.

What would you like to be added:
End to end tests for machine pools needs to be updated. At a minimum we should have test for:

• Launching a cluster with a pool of size X
• Scaling up a cluster pool
• upgrading pool image

Why is this needed:
#89 added support for MachinePool but now we need to add testing around it.

What would you like to be documented:
As a part of #44 we are adding multi-region support. Currently this functionality isn't exposed in our provided templates, but we should.

Why is this needed:
This wasn't documented as part of the PR since we wanted to land the functionality then come back and document how it will work and provide a good user experience.

What would you like to be documented:
Cover all available parameters in some examples

Why is this needed:
our examples don't cover all the options. It would be good if we can provide an example usage of all available parameters. It doesn't all have to be in a single example but rather all parameters must be used at least once across all the provided examples.

What would you like to be documented:
CAPOCI should publish CRD reference

Why is this needed:
This will help users to understand the CRD, which the cluster templates are based on. Any customisation to the template will be greatly helped by this documentation.

What would you like to be added:
Right now if a cluster is defined as a BM.Standard3.64 shape both the control plane and all worker nodes are that same shape when the user is using our defined templates. We would like the ability to allow users to specify the control plane as on shape (example: VM.Standard.E4.Flex ) while the worker nodes are a different shape (example: BM.Standard3.64).

Why is this needed:
Customers have asked for this feature.

What would you like to be added:
We need to go through and make sure we are not using ObjectMeta.ClusterName since there will be a change made in cluster-api to remove this

Why is this needed:
There is an upstream discussion about dropping ObjectMeta.ClusterName: kubernetes/kubernetes#108717. Related CAPI issue: kubernetes-sigs/cluster-api#6305

This field is not populated by Kubernetes and we should not use it as an CAPOCI Cluster Name.

This issue is about doing some research where we use it in CAPOCI and fix it if it is used.
cluster-api update: kubernetes/kubernetes#108717

What would you like to be added:
When a user runs clusterctl config repositories we should show information about the CAPOCI provider. We will also want to coordinate with #20 to make sure the docs are updated as well.

Why is this needed:
This will make it easier for users to find our provider.

What would you like to be added:
There should be the ability to launch a work load cluster in multiple regions.
Example: A user can launch a cluster in SYD and another cluster in IAD and have the same management cluster managing the lifecycle of both workload clusters

Why is this needed:
We have users asking for this.

What happened:
clusterctl move does not work with CAPOCI. The reconciliation is failing with the below error

cluster api tags have been modified out of context

What you expected to happen:

Reconciliation should still happen

How to reproduce it (as minimally and precisely as possible):

Use clusterctl move of an OCICluster
Anything else we need to know?:

CAPOCI is using the cluster UID as the resource identifier and tags all created resources with this tag. But during the move operation, the clusters gets a new UID which throws the modified out of context error.

Environment:

• CAPOCI version:
• Cluster-API version (use clusterctl version):
• Kubernetes version (use kubectl version):
• Docker version (use docker info):
• OS (e.g. from /etc/os-release):

What would you like to be documented:

CAPOCI supports externally managed infrastructure feature - https://cluster-api.sigs.k8s.io/developer/providers/v1alpha3-to-v1alpha4.html?highlight=external#required-change-to-support-externally-managed-infrastructure

Why is this needed:

This will enable customers to cluster infrastructure outside CAPOCI and then let CAPOCI only manage the instances(nodes).

What would you like to be added:
Add a mdbook processor to get the latest release version of OCI CCM while building the book

Why is this needed:
The version of CCM will quickly get out of date if it isn't automated in some fashion.

What would you like to be added:

Add test for MachinePool Kubernetes upgrade scenario

Why is this needed:

To test MachinePool upgrade during Kubernetes version upgrade

What would you like to be added:
Add validations via validation webhook.

Why is this needed:
Validation webhooks will make sure that mandatory parameters are set much before reconciliation so that users can catch errors much earlier. Currently all validations happen during reconciliation.

What would you like to be added:

Currently, clusters created by CAPOCI can all be publicly accessed via a public load balancer. This issue is a proposal to make it possible to have clusters whose API servers can only be accessed via a restricted CIDR range and not exposed to the Internet.

Why is this needed:
Restricting public access to Kubernetes clusters is recommended in order to improve the security level of a Kubernetes cluster.

What would you like to be documented:
Add cluster-api-provider-oci to list of providers: https://cluster-api.sigs.k8s.io/user/quick-start.html

Why is this needed:

What would you like to be added:
Add defaults via webhooks.

Why is this needed:

Currently all defaults are set during reconciliation. Webhooks are a better place to do that rather than reconciliation.

What would you like to be added:
We don't currently have anyone on the core team using windows. It would be nice to have someone test:

1. local management cluster processes work on Windows
2. local development processes work on Windows
3. Update docs (or create issues) to document any needed Windows tools

Why is this needed:
We want to be able to support the community on whatever tools they choose to use.

What happened:
When launching a new management cluster with the OCI provider the capoci-controller-manager continually gets the OOMKilled error

What you expected to happen:
I would expect the CAPOCI manager to launch.

How to reproduce it (as minimally and precisely as possible):
Use kind to create a cluster then run clusterctl init --infrastructure oci and when the capoci-controller-manager attempts to launch it will get killed.

Anything else we need to know?:

Environment:

• CAPOCI version: 0.2.0/0.3.0
• Rancher Desktop version: 1.3.0
• Cluster-API version (use clusterctl version):
• Kubernetes version (use kubectl version):
• Docker version (use docker info):
• OS (e.g. from /etc/os-release):

What would you like to be added:
We need this to run through all pages and return the full list of running instances https://github.com/oracle/cluster-api-provider-oci/blob/main/cloud/scope/machine_pool.go#L158

Also should check all other list calls are using pagination.

Why is this needed:
As clusters grow we want to make sure we don't miss running instances.

What would you like to be documented:
The documents should all be updated to comply with our style guide.

Why is this needed:
We want to make sure our docs are styled consistently

What would you like to be added:
The current way a machine's failure domain (FD) is selected is randomly from 1 to 3. This should be changed to be an "even spread" across FDs where possible.

Why is this needed:
This will help improve availability and make sure machines don't randomly end up in the same FDs.

What would you like to be added:
Support for multiple MachinePools. This would be implemented by OCI's InstancePool.

Why is this needed:

Currently, when we create a cluster, worker instances, they are created as individual instances. This means that failures in control plane/worker nodes have to be manually addressed.

We can address the above and make self-managed clusters created by CAPOCI more flexible and more resilient to failures.

What happened:
When setting control plane to 3 nodes, even though the desired state (3 provisioned control plane nodes) has not yet been met, the cluster object has already transitioned to provisioned. This may happen when a control plane node takes longer to provision but the cluster is available.

What you expected to happen:
The cluster object should only transition to provisioned after the desired state (i.e. 3 control plane nodes are ready) are met.

How to reproduce it (as minimally and precisely as possible):
export CONTROL_PLANE_MACHINE_COUNT=3

Anything else we need to know?:

Environment:

• CAPOCI version: v0.1.0
• Cluster-API version (use clusterctl version): v1.0
• Kubernetes version (use kubectl version): v1.22.5
• Docker version (use docker info):
• OS (e.g. from /etc/os-release):