stolostron / deploy Goto Github PK
View Code? Open in Web Editor NEWDeploy Development Builds of Open Cluster Management (OCM) on RedHat Openshift Container Platform
License: Apache License 2.0
Deploy Development Builds of Open Cluster Management (OCM) on RedHat Openshift Container Platform
License: Apache License 2.0
Describe the bug
The start.sh script uses same Downstream Builds SNAPSHOTS version while installing multiclusterengine operator.
The catlogsource of multiclusterengine uses following image instead of latest tag from https://quay.io/repository/acm-d/mce-custom-registry?tab=tags after execute start.sh script.
Example:
$ cat catalogsources/multicluster-engine.yaml |grep image
image: "quay.io:443/acm-d/mce-custom-registry:2.5.0-DOWNSTREAM-2022-01-24-16-34-41"
The below events reported by catlogsource pod of multiclusterengine
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 2m22s default-scheduler Successfully assigned openshift-marketplace/multiclusterengine-catalog-f6bbc to pdhamdhe25acm-m6n5l-worker-centralus3-k9zcs
Normal AddedInterface 2m20s multus Add eth0 [10.128.2.73/23] from openshift-sdn
Warning Failed 56s (x6 over 2m19s) kubelet Error: ImagePullBackOff
Normal Pulling 45s (x4 over 2m20s) kubelet Pulling image "quay.io:443/acm-d/mce-custom-registry:2.5.0-DOWNSTREAM-2022-01-24-16-34-41"
Warning Failed 44s (x4 over 2m19s) kubelet Failed to pull image "quay.io:443/acm-d/mce-custom-registry:2.5.0-DOWNSTREAM-2022-01-24-16-34-41": rpc error: code = Unknown desc = reading manifest 2.5.0-DOWNSTREAM-2022-01-24-16-34-41 in quay.io:443/acm-d/mce-custom-registry: manifest unknown: manifest unknown
Warning Failed 44s (x4 over 2m19s) kubelet Error: ErrImagePull
Normal BackOff 30s (x7 over 2m19s) kubelet Back-off pulling image "quay.io:443/acm-d/mce-custom-registry:2.5.0-DOWNSTREAM-2022-01-24-16-34-41"
To Reproduce
Steps to reproduce the behavior:
Error: ImagePullBackOff
Expected behavior
The start.sh script should use multiclusterengine tag from https://quay.io/repository/acm-d/mce-custom-registry?tab=tags
Need to update the start.sh script to use correct multiclusterengine tag
Describe the bug
Tried installing ACM using ./start.sh --watch
with snapshot 2.6.0-SNAPSHOT-2022-07-13-23-23-00
but it fails to install acm-custom-registry.
To Reproduce
Steps to reproduce the behavior:
./start.sh --watch
2.6.0-SNAPSHOT-2022-07-13-23-23-00
snapshotExpected behavior
acm-custom-registry pods to come up and installation to complete
Screenshots
./start.sh --watch --silent
* Testing connection
* Using baseDomain: vthapar-acmhub.subm.red-chesterfield.com
* oc CLI Client Version: 4.11.0-rc.2
* Silent mode
* Applying multicluster-hub-cr values
Error: open /tmp/temp2200282646: no such file or directory
Error: open /tmp/temp1294710551: no such file or directory
catalogsource.operators.coreos.com/acm-custom-registry created
#####
Wait for acm-custom-registry to reach running state (4min).
* STATUS: Waiting
--snip--
* STATUS: Waiting
Timeout waiting for the acm-custom-registry. Try cleaning up using the uninstall scripts before running again.
List of current pods:
NAME READY STATUS RESTARTS AGE
certified-operators-w74dj 1/1 Running 0 39m
community-operators-8nt2m 1/1 Running 0 39m
marketplace-operator-5789466757-wsvmd 1/1 Running 0 42m
redhat-marketplace-hf2kb 1/1 Running 0 39m
redhat-operators-b6bfz 1/1 Running 0 39m
Desktop (please complete the following information):
Additional context
OCP Version:
oc version
Client Version: 4.11.0-rc.2
Kustomize Version: v4.5.4
Server Version: 4.11.0-rc.1
Kubernetes Version: v1.24.0+2dd8bb1
acm-custom-registry
CatalogSource has following error in its status:
status:
message: 'image and address unset: at least one must be set for sourcetype: grpc'
reason: SpecInvalidError
Version: 4.4.0-0.nightly-2020-03-15-192859
open-cluster-management-registry pod is "ErrImagePull" status.
Trying to run the steps in https://github.com/open-cluster-management/deploy
Stuck on step #6.
[kni@provisionhost-0 multiclusterhub-operator]$ oc get pod -A|grep -v Run|grep -v Comple
NAMESPACE NAME READY STATUS RESTARTS AGE
open-cluster-management open-cluster-management-registry-7bf489f7fd-p2fzx 0/1 ErrImagePull 0 16s
oc describe -n open-cluster-management pod open-cluster-management-registry-7bf489f7fd-p2fzx
Name: open-cluster-management-registry-7bf489f7fd-p2fzx
Namespace: open-cluster-management
Priority: 0
Node: worker-0/192.168.123.126
Start Time: Mon, 16 Mar 2020 21:32:48 +0000
Labels: app=open-cluster-management-registry
pod-template-hash=7bf489f7fd
Annotations: k8s.v1.cni.cncf.io/networks-status:
[{
"name": "openshift-sdn",
"interface": "eth0",
"ips": [
"10.131.0.4"
],
"dns": {},
"default-route": [
"10.131.0.1"
]
}]
openshift.io/scc: restricted
Status: Pending
IP: 10.131.0.4
IPs:
IP: 10.131.0.4
Controlled By: ReplicaSet/open-cluster-management-registry-7bf489f7fd
Containers:
multiclusterhub-operator-index:
Container ID:
Image: quay.io/open-cluster-management/multiclusterhub-operator-index:1.0.0-SNAPSHOT-2020-03-12-18-56-25
Image ID:
Port: 50051/TCP
Host Port: 0/TCP
State: Waiting
Reason: ImagePullBackOff
Ready: False
Restart Count: 0
Environment:
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-2zgxr (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
default-token-2zgxr:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-2zgxr
Optional: false
QoS Class: BestEffort
Node-Selectors:
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
Normal Scheduled default-scheduler Successfully assigned open-cluster-management/open-cluster-management-registry-7bf489f7fd-p2fzx to worker-0
Normal BackOff 20s (x2 over 49s) kubelet, worker-0 Back-off pulling image "quay.io/open-cluster-management/multiclusterhub-operator-index:1.0.0-SNAPSHOT-2020-03-12-18-56-25"
Warning Failed 20s (x2 over 49s) kubelet, worker-0 Error: ImagePullBackOff
Normal Pulling 9s (x3 over 50s) kubelet, worker-0 Pulling image "quay.io/open-cluster-management/multiclusterhub-operator-index:1.0.0-SNAPSHOT-2020-03-12-18-56-25"
Warning Failed 8s (x3 over 50s) kubelet, worker-0 Failed to pull image "quay.io/open-cluster-management/multiclusterhub-operator-index:1.0.0-SNAPSHOT-2020-03-12-18-56-25": [rpc error: code = Unknown desc = Error reading manifest 1.0.0-SNAPSHOT-2020-03-12-18-56-25 in quay.io/open-cluster-management/multiclusterhub-operator-index: manifest unknown: manifest unknown, rpc error: code = Unknown desc = Error reading manifest 1.0.0-SNAPSHOT-2020-03-12-18-56-25 in quay.io/open-cluster-management/multiclusterhub-operator-index: unauthorized: access to the requested resource is not authorized]
Warning Failed 8s (x3 over 50s) kubelet, worker-0 Error: ErrImagePull
Observed on: RHEL, Travis Worker (UBU 18.04)
./start.sh (with or without optional tags) results in the following output Travis Link:
##### Applying prerequisites
Error: unknown shorthand flag: 'k' in -k
Usage:
oc apply -f FILENAME [flags]
Examples:
# Apply the configuration in pod.json to a pod.
oc apply -f ./pod.json
# Apply the JSON passed into stdin to a pod.
cat pod.json | oc apply -f -
# Note: --prune is still in Alpha
# Apply the configuration in manifest.yaml that matches label app=nginx and delete all the other resources that are not in the file and match label app=nginx.
oc apply --prune -f manifest.yaml -l app=nginx
# Apply the configuration in manifest.yaml and delete all the other configmaps that are not in the file.
oc apply --prune -f manifest.yaml --all --prune-whitelist=core/v1/ConfigMap
Available Commands:
edit-last-applied Edit latest last-applied-configuration annotations of a resource/object
set-last-applied Set the last-applied-configuration annotation on a live object to match the contents of a file.
view-last-applied View latest last-applied-configuration annotations of a resource/object
Options:
--all=false: Select all resources in the namespace of the specified resource types.
--allow-missing-template-keys=true: If true, ignore any errors in templates when a field or map key is missing in the template. Only applies to golang and jsonpath output formats.
--cascade=true: If true, cascade the deletion of the resources managed by this resource (e.g. Pods created by a ReplicationController). Default true.
--dry-run=false: If true, only print the object that would be sent, without sending it.
-f, --filename=[]: that contains the configuration to apply
--force=false: Only used when grace-period=0. If true, immediately remove resources from API and bypass graceful deletion. Note that immediate deletion of some resources may result in inconsistency or data loss and requires confirmation.
--grace-period=-1: Period of time in seconds given to the resource to terminate gracefully. Ignored if negative. Set to 1 for immediate shutdown. Can only be set to 0 when --force is true (force deletion).
--include-uninitialized=false: If true, the kubectl command applies to uninitialized objects. If explicitly set to false, this flag overrides other flags that make the kubectl commands apply to uninitialized objects, e.g., "--all". Objects with empty metadata.initializers are regarded as initialized.
--openapi-patch=true: If true, use openapi to calculate diff when the openapi presents and the resource can be found in the openapi spec. Otherwise, fall back to use baked-in types.
-o, --output='': Output format. One of: json|yaml|name|go-template|go-template-file|template|templatefile|jsonpath|jsonpath-file.
--overwrite=true: Automatically resolve conflicts between the modified and live configuration by using values from the modified configuration
--prune=false: Automatically delete resource objects, including the uninitialized ones, that do not appear in the configs and are created by either apply or create --save-config. Should be used with either -l or --all.
--prune-whitelist=[]: Overwrite the default whitelist with <group/version/kind> for --prune
--record=false: Record current kubectl command in the resource annotation. If set to false, do not record the command. If set to true, record the command. If not set, default to updating the existing annotation value only if one already exists.
-R, --recursive=false: Process the directory used in -f, --filename recursively. Useful when you want to manage related manifests organized within the same directory.
-l, --selector='': Selector (label query) to filter on, supports '=', '==', and '!='.(e.g. -l key1=value1,key2=value2)
--server-dry-run=false: If true, request will be sent to server with dry-run flag, which means the modifications won't be persisted. This is an alpha feature and flag.
--template='': Template string or path to template file to use when -o=go-template, -o=go-template-file. The template format is golang templates [http://golang.org/pkg/text/template/#pkg-overview].
--timeout=0s: The length of time to wait before giving up on a delete, zero means determine a timeout from the size of the object
--validate=false: If true, use a schema to validate the input before sending it
--wait=false: If true, wait for resources to be gone before returning. This waits for finalizers.
Use "oc <command> --help" for more information about a given command.
Use "oc options" for a list of global command-line options (applies to all commands).
Describe the bug
A clear and concise description of what the bug is.
After the execution of clean-clusters.sh, klusterlets.operator.open-cluster-management.io has still existed.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
A clear and concise description of what you expected to happen.
The cluster is successfully imported.
Screenshots
If applicable, add screenshots to help explain your problem.
Error from server (AlreadyExists): error when creating "STDIN": customresourcedefinitions.apiextensions.k8s.io "klusterlets.operator.open-cluster-management.io" already exists
The cluster cannot be imported because its Klusterlet CRD already exists.
Either the cluster was already imported, or it was not detached completely during a previous detach process.
Detach the existing cluster before trying the import again.%
Desktop (please complete the following information):
Additional context
Add any other context about the problem here.
Hi,
i am trying to install ingress-nginx into the AKS cluster using the kubernetes helm repository (https://kubernetes.github.io/ingress-nginx) and it gets successfully deployed.
and i also got the external ip of ingress but whenever i am trying to hit the external ip over the internet it is giving me gateway timeout error. even i have also checked my NSG rules.
what i have tried :
i thought the problem is in the cluster bt when i installed the nginx from the NGINX official helm repository (https://helm.nginx.com/stable) i got the external ip of nginx.
and when hit that external ip i got the outpit (404 nginx ) which is fine.
but the NGINX official helm repository doesnt comes with the admission controllers.
can you please tell me how to resolve this i want to use kubernetes nginx helm repository(https://kubernetes.github.io/ingress-nginx) but it seems not to be working
Describe the bug
A clear and concise description of what the bug is.
During deployment of ACM the managment-ingress pod cannot start with an ImagePullBackoff with an ErrImagePull for the image quay.io/openshift/origin-oauth-proxy.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
All pods are running ore completed with no errors.
Screenshots
$ oc get pods -n open-cluster-management
NAME READY STATUS RESTARTS AGE
acm-custom-registry-544ddf4f84-75997 1/1 Running 0 12h
application-chart-1e0db-applicationui-7466647bb6-v6xp2 1/1 Running 0 21m
b204aeeeb5e8e89542b70ae24092621f8a5d905bd698700ca6b9d41de0kxqqf 0/1 Completed 0 32m
cert-manager-1098a-54c7cd774-mjd27 1/1 Running 0 21m
cert-manager-webhook-c5a7a-cainjector-8484b449dd-zfcc6 1/1 Running 0 21m
cert-manager-webhook-f97bdb9fd-6jwhd 1/1 Running 1 21m
configmap-watcher-f9ec5-8647dc6946-h2mzg 1/1 Running 0 21m
console-chart-0f5e7-consoleapi-866c88ff58-pc5qx 1/1 Running 0 19m
console-chart-0f5e7-consoleui-5df747b8ff-fj556 1/1 Running 0 19m
console-header-856c64fc64-rbccd 1/1 Running 0 19m
etcd-cluster-4f4p6c6pwv 1/1 Running 0 21m
etcd-cluster-fhnhwdgxrq 1/1 Running 0 19m
etcd-cluster-zn5w9hnkh2 1/1 Running 0 21m
etcd-operator-558567f79d-fs9gn 3/3 Running 0 32m
grc-7fba5-grcui-697f87f79b-q98kk 1/1 Running 0 18m
grc-7fba5-grcuiapi-f55595748-ztwnh 1/1 Running 0 18m
grc-7fba5-policy-propogator-8cb9ff78d-s8cl5 1/1 Running 0 18m
hive-operator-fbcf446c5-bgl9n 1/1 Running 0 31m
kui-web-terminal-6fb99c7fbb-gm5mc 1/1 Running 0 21m
kui-web-terminal-75f6cb7f4d-69cw2 0/1 Pending 0 17m
management-ingress-48ccc-6648d5fbdf-rt78l 1/2 ImagePullBackOff 0 17m
mcm-apiserver-648989b9f4-lkdvt 0/1 Pending 0 17m
mcm-apiserver-9fdff5f69-bhppr 1/1 Running 0 20m
mcm-controller-57955b6c66-28p7k 1/1 Running 0 21m
mcm-webhook-794f8d575-9vpst 1/1 Running 0 21m
multicluster-mongodb-0 1/1 Running 0 18m
multicluster-operators-application-6d7bf76cd8-jfscx 4/4 Running 3 31m
multicluster-operators-hub-subscription-65b48f6bb-zfjr9 1/1 Running 0 31m
multicluster-operators-standalone-subscription-8496f67d7-sqlkg 1/1 Running 0 31m
multiclusterhub-operator-5474547c66-shjfh 1/1 Running 0 31m
multiclusterhub-repo-6cd66688cc-gvlwx 1/1 Running 0 22m
rcm-controller-544b7cdd6b-v9n4c 1/1 Running 3 21m
search-operator-958dd6d67-m6nz5 1/1 Running 0 19m
search-prod-73f12-redisgraph-6cbcf6c958-s8j5x 1/1 Running 0 17m
search-prod-73f12-search-aggregator-cbb6f96bc-78cxd 1/1 Running 0 17m
search-prod-73f12-search-api-5bd9d57669-wsbtg 1/1 Running 0 17m
search-prod-73f12-search-collector-6f7cfffcdb-dg7mj 1/1 Running 0 17m
topology-33ea9-topology-56d466bfc9-hgtb8 1/1 Running 0 19m
topology-33ea9-topologyapi-584d474477-bdlhr 1/1 Running 0 19m
Desktop (please complete the following information):
Additional context
The problem looks related to the wrong SHA256 of the image. I used the following temporary workaround:
skopeo inspect docker://quay.io/openshift/origin-oauth-proxy | grep Digest
oc edit deployment/managment-ingress-XXXXX
Current example-multiclusterhub-cr.yaml in multiclusterhub is incomplete. After I replace it with the content from README, the deployment of ACM became successful.
apiVersion: operators.open-cluster-management.io/v1beta1
kind: MultiClusterHub
metadata:
name: multiclusterhub
namespace: open-cluster-management
spec:
imageRepository: "quay.io/open-cluster-management"
imageTagSuffix: SNAPSHOT-2020-04-21-17-28-34
imagePullPolicy: Always
imagePullSecret: multiclusterhub-operator-pull-secret
Expected content is (from README):
apiVersion: operators.open-cluster-management.io/v1beta1
kind: MultiClusterHub
metadata:
name: example-multiclusterhub
namespace: open-cluster-management
spec:
version: latest
imageRepository: "quay.io/open-cluster-management"
imageTagSuffix: "SNAPSHOT-2020-03-17-21-24-18"
imagePullPolicy: Always
imagePullSecret: multiclusterhub-operator-pull-secret
foundation:
apiserver:
configuration:
http2-max-streams-per-connection: "1000"
replicas: 1
apiserverSecret: "mcm-apiserver-self-signed-secrets"
klusterletSecret: "mcm-klusterlet-self-signed-secrets"
controller:
configuration:
enable-rbac: "true"
enable-service-registry: "true"
replicas: 1
mongo:
endpoints: mongo-0.mongo.open-cluster-management
replicaSet: rs0
hive:
additionalCertificateAuthorities:
- name: letsencrypt-ca
managedDomains:
- s1.openshiftapps.com
globalPullSecret:
name: private-secret
failedProvisionConfig:
skipGatherLogs: true
Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Follow the readme at https://github.com/open-cluster-management/deploy#prepare-to-deploy-open-cluster-management-instance-only-do-once
I think the major reason is mongodb failed to start caused the mcm apiserver failed to start.
I was using snapshot 2.0.0-SNAPSHOT-2020-06-23-14-20-27
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
[root@ocp43-dev-inf prereqs]# oc get pods -n open-cluster-management
NAME READY STATUS RESTARTS AGE
acm-controller-9dd999dcc-c42jg 1/1 Running 0 14m
acm-proxyserver-7c45654bf6-bg9jg 1/1 Running 0 14m
application-chart-ae36f-applicationui-55f468c4f8-6hcnt 1/1 Running 0 13m
cert-manager-e61c1-6cf697d5df-rhhcc 1/1 Running 0 14m
cert-manager-webhook-03d4d-cainjector-8d76f6646-t5crm 1/1 Running 0 13m
cert-manager-webhook-74bdc8455d-jn6qf 1/1 Running 0 13m
cluster-manager-64964fdf4f-bmhld 1/1 Running 0 15m
cluster-manager-64964fdf4f-vfckd 1/1 Running 0 15m
cluster-manager-64964fdf4f-xwk9v 1/1 Running 0 15m
configmap-watcher-4b90e-66c867f4f5-7rj8z 1/1 Running 0 13m
console-chart-65124-consoleapi-56ff5dbdfb-5cslf 1/1 Running 0 10m
console-chart-65124-consoleui-7b4fd788c6-4zrqr 1/1 Running 0 10m
console-header-55ffb7666d-pnnp7 1/1 Running 0 10m
etcd-cluster-8mvj8pznpx 0/1 Init:0/1 0 14m
etcd-operator-558567f79d-wsczb 3/3 Running 0 15m
grc-7079f-grcui-7d9c8dd454-d8qf5 1/1 Running 0 13m
grc-7079f-grcuiapi-74896974c4-6pvj6 1/1 Running 0 13m
grc-7079f-policy-propagator-67c7546d77-6v628 1/1 Running 0 13m
hive-operator-6bf77bd558-v5nqd 1/1 Running 0 15m
klusterlet-addon-controller-5f47d9f99-dd8pw 1/1 Running 0 13m
managedcluster-import-controller-69b69bf967-kjmw9 1/1 Running 0 13m
management-ingress-2511c-6c7dff479c-9tzqr 2/2 Running 0 12m
mcm-apiserver-564cb96f8d-2v4gc 0/1 CrashLoopBackOff 7 14m
mcm-apiserver-6f794b6df-ggm44 0/1 CrashLoopBackOff 6 12m
mcm-controller-8676c9b6db-gqkw2 1/1 Running 0 14m
mcm-webhook-98957b97f-7sdjw 1/1 Running 0 14m
multicluster-hub-custom-registry-64cdb758bc-7d9g7 1/1 Running 0 16m
multicluster-mongodb-0 0/1 Init:1/2 0 13m
multicluster-operators-application-68445cbf88-5rxjr 4/4 Running 0 15m
multicluster-operators-hub-subscription-84c69bb5bf-h766b 1/1 Running 0 15m
multicluster-operators-standalone-subscription-55cc9d964d-c9p7q 1/1 Running 0 15m
multiclusterhub-operator-7cf7b55cc7-kh2cg 1/1 Running 0 6m50s
multiclusterhub-repo-fdd98b94f-nwrvh 1/1 Running 0 14m
search-operator-5c9f65c7c9-td78r 1/1 Running 0 10m
search-prod-798d3-redisgraph-58858bdb48-t5drs 1/1 Running 0 10m
search-prod-798d3-search-aggregator-65c8cbcd4f-j4qpr 1/1 Running 0 10m
search-prod-798d3-search-api-6df9bd58cd-qwzkf 1/1 Running 0 10m
search-prod-798d3-search-collector-774667b9f-68qg8 1/1 Running 0 10m
topology-30155-topology-5447cdd666-sq2vw 1/1 Running 0 10m
topology-30155-topologyapi-5fc4c96466-h6m45 1/1 Running 0 10m
[root@ocp43-dev-inf prereqs]# oc get pods -n open-cluster-management | grep -v Running
NAME READY STATUS RESTARTS AGE
etcd-cluster-8mvj8pznpx 0/1 Init:0/1 0 14m
mcm-apiserver-564cb96f8d-2v4gc 0/1 CrashLoopBackOff 7 14m
mcm-apiserver-6f794b6df-ggm44 0/1 CrashLoopBackOff 6 12m
multicluster-mongodb-0 0/1 Init:1/2 0 13m
[root@ocp43-dev-inf prereqs]# oc describe pods -n open-cluster-management etcd-cluster-8mvj8pznpx mcm-apiserver-564cb96f8d-2v4gc mcm-apiserver-6f794b6df-ggm44 multicluster-mongodb-0
Name: etcd-cluster-8mvj8pznpx
Namespace: open-cluster-management
Priority: 0
Node: worker1.ocp43-dev.os.fyre.ibm.com/10.16.100.29
Start Time: Tue, 23 Jun 2020 07:54:33 -0700
Labels: app=etcd
etcd_cluster=etcd-cluster
etcd_node=etcd-cluster-8mvj8pznpx
Annotations: etcd.version: 3.2.13
k8s.v1.cni.cncf.io/networks-status:
[{
"name": "openshift-sdn",
"interface": "eth0",
"ips": [
"10.254.8.26"
],
"dns": {},
"default-route": [
"10.254.8.1"
]
}]
openshift.io/scc: multicloud-scc
Status: Pending
IP: 10.254.8.26
IPs:
IP: 10.254.8.26
Controlled By: EtcdCluster/etcd-cluster
Init Containers:
check-dns:
Container ID: cri-o://0616c0d1d24a5d34578631732fbee767547736ec874f105423004c72b149d3c3
Image: busybox:1.28.0-glibc
Image ID: docker.io/library/busybox@sha256:0b55a30394294ab23b9afd58fab94e61a923f5834fba7ddbae7f8e0c11ba85e6
Port: <none>
Host Port: <none>
Command:
/bin/sh
-c
TIMEOUT_READY=0
while ( ! nslookup etcd-cluster-8mvj8pznpx.etcd-cluster.open-cluster-management.svc )
do
# If TIMEOUT_READY is 0 we should never time out and exit
TIMEOUT_READY=$(( TIMEOUT_READY-1 ))
if [ $TIMEOUT_READY -eq 0 ];
then
echo "Timed out waiting for DNS entry"
exit 1
fi
sleep 1
done
State: Running
Started: Tue, 23 Jun 2020 07:55:01 -0700
Ready: False
Restart Count: 0
Environment: <none>
Mounts: <none>
Containers:
etcd:
Container ID:
Image: quay.io/coreos/etcd:v3.2.13
Image ID:
Ports: 2380/TCP, 2379/TCP
Host Ports: 0/TCP, 0/TCP
Command:
/usr/local/bin/etcd
--data-dir=/var/etcd/data
--name=etcd-cluster-8mvj8pznpx
--initial-advertise-peer-urls=http://etcd-cluster-8mvj8pznpx.etcd-cluster.open-cluster-management.svc:2380
--listen-peer-urls=http://0.0.0.0:2380
--listen-client-urls=http://0.0.0.0:2379
--advertise-client-urls=http://etcd-cluster-8mvj8pznpx.etcd-cluster.open-cluster-management.svc:2379
--initial-cluster=etcd-cluster-8mvj8pznpx=http://etcd-cluster-8mvj8pznpx.etcd-cluster.open-cluster-management.svc:2380
--initial-cluster-state=new
--initial-cluster-token=2341bb1f-5975-444c-8aef-4c6f8ee82d83
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Liveness: exec [/bin/sh -ec ETCDCTL_API=3 etcdctl endpoint status] delay=10s timeout=10s period=60s #success=1 #failure=3
Readiness: exec [/bin/sh -ec ETCDCTL_API=3 etcdctl endpoint status] delay=1s timeout=5s period=5s #success=1 #failure=3
Environment: <none>
Mounts:
/var/etcd from etcd-data (rw)
Conditions:
Type Status
Initialized False
Ready False
ContainersReady False
PodScheduled True
Volumes:
etcd-data:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: etcd-cluster-8mvj8pznpx
ReadOnly: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling <unknown> default-scheduler pod has unbound immediate PersistentVolumeClaims (repeated 3 times)
Warning FailedScheduling <unknown> default-scheduler pod has unbound immediate PersistentVolumeClaims (repeated 3 times)
Normal Scheduled <unknown> default-scheduler Successfully assigned open-cluster-management/etcd-cluster-8mvj8pznpx to worker1.ocp43-dev.os.fyre.ibm.com
Normal SuccessfulAttachVolume 15m attachdetach-controller AttachVolume.Attach succeeded for volume "pvc-b8bdd641-d4fa-437a-b0bd-2923c1234880"
Normal Pulling 14m kubelet, worker1.ocp43-dev.os.fyre.ibm.com Pulling image "busybox:1.28.0-glibc"
Normal Pulled 14m kubelet, worker1.ocp43-dev.os.fyre.ibm.com Successfully pulled image "busybox:1.28.0-glibc"
Normal Created 14m kubelet, worker1.ocp43-dev.os.fyre.ibm.com Created container check-dns
Normal Started 14m kubelet, worker1.ocp43-dev.os.fyre.ibm.com Started container check-dns
Name: mcm-apiserver-564cb96f8d-2v4gc
Namespace: open-cluster-management
Priority: 0
Node: worker1.ocp43-dev.os.fyre.ibm.com/10.16.100.29
Start Time: Tue, 23 Jun 2020 07:54:30 -0700
Labels: app=mcm-apiserver
pod-template-hash=564cb96f8d
Annotations: k8s.v1.cni.cncf.io/networks-status:
[{
"name": "openshift-sdn",
"interface": "eth0",
"ips": [
"10.254.8.32"
],
"dns": {},
"default-route": [
"10.254.8.1"
]
}]
openshift.io/scc: restricted
Status: Running
IP: 10.254.8.32
IPs:
IP: 10.254.8.32
Controlled By: ReplicaSet/mcm-apiserver-564cb96f8d
Containers:
mcm-apiserver:
Container ID: cri-o://ff500f77e94a41b9971e0a039ada8ede86bb4168f55044ef01b961f76c400f68
Image: quay.io/open-cluster-management/multicloud-manager@sha256:7e6fa2399ac53feda232bff542feadc4861ec03a1548c36973ccadc9f7e14259
Image ID: quay.io/open-cluster-management/multicloud-manager@sha256:7e6fa2399ac53feda232bff542feadc4861ec03a1548c36973ccadc9f7e14259
Port: <none>
Host Port: <none>
Args:
/mcm-apiserver
--mongo-database=mcm
--enable-admission-plugins=HCMUserIdentity,KlusterletCA,NamespaceLifecycle
--secure-port=6443
--tls-cert-file=/var/run/apiserver/tls.crt
--tls-private-key-file=/var/run/apiserver/tls.key
--klusterlet-cafile=/var/run/klusterlet/ca.crt
--klusterlet-certfile=/var/run/klusterlet/tls.crt
--klusterlet-keyfile=/var/run/klusterlet/tls.key
--http2-max-streams-per-connection=1000
--etcd-servers=http://etcd-cluster.open-cluster-management.svc.cluster.local:2379
--mongo-host=multicluster-mongodb
--mongo-replicaset=rs0
State: Waiting
Reason: CrashLoopBackOff
Last State: Terminated
Reason: Error
Exit Code: 2
Started: Tue, 23 Jun 2020 08:04:50 -0700
Finished: Tue, 23 Jun 2020 08:05:11 -0700
Ready: False
Restart Count: 7
Limits:
memory: 2Gi
Requests:
cpu: 200m
memory: 256Mi
Liveness: http-get https://:6443/healthz delay=2s timeout=1s period=10s #success=1 #failure=3
Readiness: http-get https://:6443/healthz delay=2s timeout=1s period=10s #success=1 #failure=3
Environment:
MONGO_USERNAME: <set to the key 'user' in secret 'mongodb-admin'> Optional: false
MONGO_PASSWORD: <set to the key 'password' in secret 'mongodb-admin'> Optional: false
MONGO_SSLCA: /certs/mongodb-ca/tls.crt
MONGO_SSLCERT: /certs/mongodb-client/tls.crt
MONGO_SSLKEY: /certs/mongodb-client/tls.key
Mounts:
/certs/mongodb-ca from mongodb-ca-cert (rw)
/certs/mongodb-client from mongodb-client-cert (rw)
/var/run/apiserver from apiserver-certs (rw)
/var/run/klusterlet from klusterlet-certs (rw)
/var/run/secrets/kubernetes.io/serviceaccount from acm-foundation-sa-token-jqbzg (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
apiserver-certs:
Type: Secret (a volume populated by a Secret)
SecretName: mcm-apiserver-self-signed-secrets
Optional: false
klusterlet-certs:
Type: Secret (a volume populated by a Secret)
SecretName: mcm-klusterlet-self-signed-secrets
Optional: false
mongodb-ca-cert:
Type: Secret (a volume populated by a Secret)
SecretName: multicloud-ca-cert
Optional: false
mongodb-client-cert:
Type: Secret (a volume populated by a Secret)
SecretName: multicluster-mongodb-client-cert
Optional: false
acm-foundation-sa-token-jqbzg:
Type: Secret (a volume populated by a Secret)
SecretName: acm-foundation-sa-token-jqbzg
Optional: false
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: node.kubernetes.io/memory-pressure:NoSchedule
node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled <unknown> default-scheduler Successfully assigned open-cluster-management/mcm-apiserver-564cb96f8d-2v4gc to worker1.ocp43-dev.os.fyre.ibm.com
Warning FailedMount 14m (x8 over 15m) kubelet, worker1.ocp43-dev.os.fyre.ibm.com MountVolume.SetUp failed for volume "mongodb-ca-cert" : secret "multicloud-ca-cert" not found
Warning FailedMount 14m (x8 over 15m) kubelet, worker1.ocp43-dev.os.fyre.ibm.com MountVolume.SetUp failed for volume "mongodb-client-cert" : secret "multicluster-mongodb-client-cert" not found
Warning FailedMount 13m kubelet, worker1.ocp43-dev.os.fyre.ibm.com Unable to attach or mount volumes: unmounted volumes=[mongodb-ca-cert mongodb-client-cert], unattached volumes=[acm-foundation-sa-token-jqbzg apiserver-certs klusterlet-certs mongodb-ca-cert mongodb-client-cert]: timed out waiting for the condition
Normal Pulling 12m kubelet, worker1.ocp43-dev.os.fyre.ibm.com Pulling image "quay.io/open-cluster-management/multicloud-manager@sha256:7e6fa2399ac53feda232bff542feadc4861ec03a1548c36973ccadc9f7e14259"
Normal Pulled 12m kubelet, worker1.ocp43-dev.os.fyre.ibm.com Successfully pulled image "quay.io/open-cluster-management/multicloud-manager@sha256:7e6fa2399ac53feda232bff542feadc4861ec03a1548c36973ccadc9f7e14259"
Normal Created 12m kubelet, worker1.ocp43-dev.os.fyre.ibm.com Created container mcm-apiserver
Normal Started 12m kubelet, worker1.ocp43-dev.os.fyre.ibm.com Started container mcm-apiserver
Warning Unhealthy 12m (x2 over 12m) kubelet, worker1.ocp43-dev.os.fyre.ibm.com Liveness probe failed: Get https://10.254.8.32:6443/healthz: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
Warning Unhealthy 12m (x2 over 12m) kubelet, worker1.ocp43-dev.os.fyre.ibm.com Readiness probe failed: Get https://10.254.8.32:6443/healthz: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
Warning BackOff 5m2s (x31 over 11m) kubelet, worker1.ocp43-dev.os.fyre.ibm.com Back-off restarting failed container
Name: mcm-apiserver-6f794b6df-ggm44
Namespace: open-cluster-management
Priority: 0
Node: worker2.ocp43-dev.os.fyre.ibm.com/10.16.100.30
Start Time: Tue, 23 Jun 2020 07:56:02 -0700
Labels: app=mcm-apiserver
certmanager.k8s.io/time-restarted=2020-6-23.1456
pod-template-hash=6f794b6df
Annotations: k8s.v1.cni.cncf.io/networks-status:
[{
"name": "openshift-sdn",
"interface": "eth0",
"ips": [
"10.254.0.38"
],
"dns": {},
"default-route": [
"10.254.0.1"
]
}]
openshift.io/scc: restricted
Status: Running
IP: 10.254.0.38
IPs:
IP: 10.254.0.38
Controlled By: ReplicaSet/mcm-apiserver-6f794b6df
Containers:
mcm-apiserver:
Container ID: cri-o://b4b4cc14454bd0b3b7d43ac339dabbdcc9000d4aa2644004b0febae4d58bce70
Image: quay.io/open-cluster-management/multicloud-manager@sha256:7e6fa2399ac53feda232bff542feadc4861ec03a1548c36973ccadc9f7e14259
Image ID: quay.io/open-cluster-management/multicloud-manager@sha256:7e6fa2399ac53feda232bff542feadc4861ec03a1548c36973ccadc9f7e14259
Port: <none>
Host Port: <none>
Args:
/mcm-apiserver
--mongo-database=mcm
--enable-admission-plugins=HCMUserIdentity,KlusterletCA,NamespaceLifecycle
--secure-port=6443
--tls-cert-file=/var/run/apiserver/tls.crt
--tls-private-key-file=/var/run/apiserver/tls.key
--klusterlet-cafile=/var/run/klusterlet/ca.crt
--klusterlet-certfile=/var/run/klusterlet/tls.crt
--klusterlet-keyfile=/var/run/klusterlet/tls.key
--http2-max-streams-per-connection=1000
--etcd-servers=http://etcd-cluster.open-cluster-management.svc.cluster.local:2379
--mongo-host=multicluster-mongodb
--mongo-replicaset=rs0
State: Running
Started: Tue, 23 Jun 2020 08:09:26 -0700
Last State: Terminated
Reason: Error
Exit Code: 2
Started: Tue, 23 Jun 2020 08:09:14 -0700
Finished: Tue, 23 Jun 2020 08:09:24 -0700
Ready: False
Restart Count: 8
Limits:
memory: 2Gi
Requests:
cpu: 200m
memory: 256Mi
Liveness: http-get https://:6443/healthz delay=2s timeout=1s period=10s #success=1 #failure=3
Readiness: http-get https://:6443/healthz delay=2s timeout=1s period=10s #success=1 #failure=3
Environment:
MONGO_USERNAME: <set to the key 'user' in secret 'mongodb-admin'> Optional: false
MONGO_PASSWORD: <set to the key 'password' in secret 'mongodb-admin'> Optional: false
MONGO_SSLCA: /certs/mongodb-ca/tls.crt
MONGO_SSLCERT: /certs/mongodb-client/tls.crt
MONGO_SSLKEY: /certs/mongodb-client/tls.key
Mounts:
/certs/mongodb-ca from mongodb-ca-cert (rw)
/certs/mongodb-client from mongodb-client-cert (rw)
/var/run/apiserver from apiserver-certs (rw)
/var/run/klusterlet from klusterlet-certs (rw)
/var/run/secrets/kubernetes.io/serviceaccount from acm-foundation-sa-token-jqbzg (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
apiserver-certs:
Type: Secret (a volume populated by a Secret)
SecretName: mcm-apiserver-self-signed-secrets
Optional: false
klusterlet-certs:
Type: Secret (a volume populated by a Secret)
SecretName: mcm-klusterlet-self-signed-secrets
Optional: false
mongodb-ca-cert:
Type: Secret (a volume populated by a Secret)
SecretName: multicloud-ca-cert
Optional: false
mongodb-client-cert:
Type: Secret (a volume populated by a Secret)
SecretName: multicluster-mongodb-client-cert
Optional: false
acm-foundation-sa-token-jqbzg:
Type: Secret (a volume populated by a Secret)
SecretName: acm-foundation-sa-token-jqbzg
Optional: false
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: node.kubernetes.io/memory-pressure:NoSchedule
node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled <unknown> default-scheduler Successfully assigned open-cluster-management/mcm-apiserver-6f794b6df-ggm44 to worker2.ocp43-dev.os.fyre.ibm.com
Warning FailedMount 13m (x6 over 13m) kubelet, worker2.ocp43-dev.os.fyre.ibm.com MountVolume.SetUp failed for volume "mongodb-client-cert" : secret "multicluster-mongodb-client-cert" not found
Warning Unhealthy 12m (x2 over 12m) kubelet, worker2.ocp43-dev.os.fyre.ibm.com Readiness probe failed: Get https://10.254.0.38:6443/healthz: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
Normal Killing 12m kubelet, worker2.ocp43-dev.os.fyre.ibm.com Container mcm-apiserver failed liveness probe, will be restarted
Normal Pulling 12m (x3 over 13m) kubelet, worker2.ocp43-dev.os.fyre.ibm.com Pulling image "quay.io/open-cluster-management/multicloud-manager@sha256:7e6fa2399ac53feda232bff542feadc4861ec03a1548c36973ccadc9f7e14259"
Normal Pulled 12m (x3 over 12m) kubelet, worker2.ocp43-dev.os.fyre.ibm.com Successfully pulled image "quay.io/open-cluster-management/multicloud-manager@sha256:7e6fa2399ac53feda232bff542feadc4861ec03a1548c36973ccadc9f7e14259"
Normal Created 12m (x3 over 12m) kubelet, worker2.ocp43-dev.os.fyre.ibm.com Created container mcm-apiserver
Normal Started 12m (x3 over 12m) kubelet, worker2.ocp43-dev.os.fyre.ibm.com Started container mcm-apiserver
Warning Unhealthy 11m (x4 over 12m) kubelet, worker2.ocp43-dev.os.fyre.ibm.com Liveness probe failed: Get https://10.254.0.38:6443/healthz: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
Warning Unhealthy 8m29s (x2 over 8m39s) kubelet, worker2.ocp43-dev.os.fyre.ibm.com Readiness probe failed: Get https://10.254.0.38:6443/healthz: dial tcp 10.254.0.38:6443: connect: connection refused
Warning BackOff 3m30s (x33 over 11m) kubelet, worker2.ocp43-dev.os.fyre.ibm.com Back-off restarting failed container
Name: multicluster-mongodb-0
Namespace: open-cluster-management
Priority: 0
Node: worker0.ocp43-dev.os.fyre.ibm.com/10.16.100.28
Start Time: Tue, 23 Jun 2020 07:55:22 -0700
Labels: app=multicluster-mongodb
controller-revision-hash=multicluster-mongodb-557c8b465f
release=multicluster-mongodb-62daa
statefulset.kubernetes.io/pod-name=multicluster-mongodb-0
Annotations: k8s.v1.cni.cncf.io/networks-status:
[{
"name": "openshift-sdn",
"interface": "eth0",
"ips": [
"10.254.12.37"
],
"dns": {},
"default-route": [
"10.254.12.1"
]
}]
openshift.io/scc: anyuid
Status: Pending
IP: 10.254.12.37
IPs:
IP: 10.254.12.37
Controlled By: StatefulSet/multicluster-mongodb
Init Containers:
install:
Container ID: cri-o://9e4d0cfc80e3cebdb12181b8499fff387559e54552d72f3c8b3368481eda1daa
Image: quay.io/open-cluster-management/multicluster-mongodb-init@sha256:904ebd15cf4074dca8d8f980433501af7037335ecaf06c79c90b3fda9a99b7e3
Image ID: quay.io/open-cluster-management/multicluster-mongodb-init@sha256:904ebd15cf4074dca8d8f980433501af7037335ecaf06c79c90b3fda9a99b7e3
Port: <none>
Host Port: <none>
Command:
/install/install.sh
Args:
--work-dir=/var/lib/mongodb/work-dir
--config-dir=/var/lib/mongodb/data/configdb
State: Terminated
Reason: Completed
Exit Code: 0
Started: Tue, 23 Jun 2020 07:56:41 -0700
Finished: Tue, 23 Jun 2020 07:56:41 -0700
Ready: True
Restart Count: 0
Limits:
memory: 5Gi
Requests:
memory: 2Gi
Environment: <none>
Mounts:
/ca-readonly from ca (rw)
/configdb-readonly from config (rw)
/install from install (rw)
/keydir-readonly from keydir (rw)
/tmp from tmp-mongodb (rw)
/var/lib/mongodb/data/configdb from configdir (rw)
/var/lib/mongodb/data/db from mongodbdir (rw,path="datadir")
/var/lib/mongodb/work-dir from mongodbdir (rw,path="workdir")
/var/run/secrets/kubernetes.io/serviceaccount from multicluster-mongodb-token-4g4x5 (ro)
bootstrap:
Container ID: cri-o://ac85c09ab3dc962da8a3e3b2abb263717d5fb787adf1a7b872bcad43e8d5fbd0
Image: quay.io/open-cluster-management/multicluster-mongodb@sha256:9320e0acc578efd94b6056b8be344b3e742fd0597568013187ef69ecbd077866
Image ID: quay.io/open-cluster-management/multicluster-mongodb@sha256:9320e0acc578efd94b6056b8be344b3e742fd0597568013187ef69ecbd077866
Port: <none>
Host Port: <none>
Command:
/var/lib/mongodb/work-dir/peer-finder
Args:
-on-start=/init/on-start.sh
-service=multicluster-mongodb
State: Running
Started: Tue, 23 Jun 2020 07:57:03 -0700
Ready: False
Restart Count: 0
Limits:
memory: 5Gi
Requests:
memory: 2Gi
Environment:
POD_NAMESPACE: open-cluster-management (v1:metadata.namespace)
REPLICA_SET: rs0
AUTH: true
ADMIN_USER: <set to the key 'user' in secret 'mongodb-admin'> Optional: false
ADMIN_PASSWORD: <set to the key 'password' in secret 'mongodb-admin'> Optional: false
NETWORK_IP_VERSION: ipv4
Mounts:
/init from init (rw)
/tmp from tmp-mongodb (rw)
/var/lib/mongodb/data/configdb from configdir (rw)
/var/lib/mongodb/data/db from mongodbdir (rw,path="datadir")
/var/lib/mongodb/work-dir from mongodbdir (rw,path="workdir")
/var/run/secrets/kubernetes.io/serviceaccount from multicluster-mongodb-token-4g4x5 (ro)
Containers:
multicluster-mongodb:
Container ID:
Image: quay.io/open-cluster-management/multicluster-mongodb@sha256:9320e0acc578efd94b6056b8be344b3e742fd0597568013187ef69ecbd077866
Image ID:
Port: 27017/TCP
Host Port: 0/TCP
Command:
mongod
--config=/var/lib/mongodb/data/configdb/mongod.conf
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Limits:
memory: 5Gi
Requests:
memory: 2Gi
Liveness: exec [mongo --ssl --sslCAFile=/var/lib/mongodb/data/configdb/tls.crt --sslPEMKeyFile=/var/lib/mongodb/work-dir/mongo.pem --eval db.adminCommand('ping')] delay=30s timeout=5s period=10s #success=1 #failure=3
Readiness: exec [mongo --ssl --sslCAFile=/var/lib/mongodb/data/configdb/tls.crt --sslPEMKeyFile=/var/lib/mongodb/work-dir/mongo.pem --eval db.adminCommand('ping')] delay=5s timeout=1s period=10s #success=1 #failure=3
Environment:
AUTH: true
ADMIN_USER: <set to the key 'user' in secret 'mongodb-admin'> Optional: false
ADMIN_PASSWORD: <set to the key 'password' in secret 'mongodb-admin'> Optional: false
Mounts:
/tmp from tmp-mongodb (rw)
/var/lib/mongodb/data/configdb from configdir (rw)
/var/lib/mongodb/data/db from mongodbdir (rw,path="datadir")
/var/lib/mongodb/work-dir from mongodbdir (rw,path="workdir")
/var/run/secrets/kubernetes.io/serviceaccount from multicluster-mongodb-token-4g4x5 (ro)
Conditions:
Type Status
Initialized False
Ready False
ContainersReady False
PodScheduled True
Volumes:
mongodbdir:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: mongodbdir-multicluster-mongodb-0
ReadOnly: false
config:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: multicluster-mongodb
Optional: false
init:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: multicluster-mongodb-init
Optional: false
install:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: multicluster-mongodb-install
Optional: false
ca:
Type: Secret (a volume populated by a Secret)
SecretName: multicloud-ca-cert
Optional: false
keydir:
Type: Secret (a volume populated by a Secret)
SecretName: multicluster-mongodb-keyfile
Optional: false
configdir:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
tmp-mongodb:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
tmp-metrics:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
multicluster-mongodb-token-4g4x5:
Type: Secret (a volume populated by a Secret)
SecretName: multicluster-mongodb-token-4g4x5
Optional: false
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: node.kubernetes.io/memory-pressure:NoSchedule
node.kubernetes.io/not-ready:NoExecute
node.kubernetes.io/unreachable:NoExecute
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling <unknown> default-scheduler pod has unbound immediate PersistentVolumeClaims (repeated 3 times)
Warning FailedScheduling <unknown> default-scheduler pod has unbound immediate PersistentVolumeClaims (repeated 3 times)
Normal Scheduled <unknown> default-scheduler Successfully assigned open-cluster-management/multicluster-mongodb-0 to worker0.ocp43-dev.os.fyre.ibm.com
Normal SuccessfulAttachVolume 14m attachdetach-controller AttachVolume.Attach succeeded for volume "pvc-891a979a-c1ce-4234-a016-98fde691c76f"
Warning FailedMount 13m (x7 over 14m) kubelet, worker0.ocp43-dev.os.fyre.ibm.com MountVolume.SetUp failed for volume "ca" : secret "multicloud-ca-cert" not found
Normal Pulling 13m kubelet, worker0.ocp43-dev.os.fyre.ibm.com Pulling image "quay.io/open-cluster-management/multicluster-mongodb-init@sha256:904ebd15cf4074dca8d8f980433501af7037335ecaf06c79c90b3fda9a99b7e3"
Normal Pulled 12m kubelet, worker0.ocp43-dev.os.fyre.ibm.com Successfully pulled image "quay.io/open-cluster-management/multicluster-mongodb-init@sha256:904ebd15cf4074dca8d8f980433501af7037335ecaf06c79c90b3fda9a99b7e3"
Normal Created 12m kubelet, worker0.ocp43-dev.os.fyre.ibm.com Created container install
Normal Started 12m kubelet, worker0.ocp43-dev.os.fyre.ibm.com Started container install
Normal Pulling 12m kubelet, worker0.ocp43-dev.os.fyre.ibm.com Pulling image "quay.io/open-cluster-management/multicluster-mongodb@sha256:9320e0acc578efd94b6056b8be344b3e742fd0597568013187ef69ecbd077866"
Normal Pulled 12m kubelet, worker0.ocp43-dev.os.fyre.ibm.com Successfully pulled image "quay.io/open-cluster-management/multicluster-mongodb@sha256:9320e0acc578efd94b6056b8be344b3e742fd0597568013187ef69ecbd077866"
Normal Created 12m kubelet, worker0.ocp43-dev.os.fyre.ibm.com Created container bootstrap
Normal Started 12m kubelet, worker0.ocp43-dev.os.fyre.ibm.com Started container bootstrap
Desktop (please complete the following information):
[root@ocp43-dev-inf deploy]# oc get nodes -owide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
master0.ocp43-dev.os.fyre.ibm.com Ready master 10h v1.16.2 10.16.96.192 <none> Red Hat Enterprise Linux CoreOS 43.81.202004130853.0 (Ootpa) 4.18.0-147.8.1.el8_1.x86_64 cri-o://1.16.5-1.dev.rhaos4.3.git91157c1.el8
worker0.ocp43-dev.os.fyre.ibm.com Ready worker 10h v1.16.2 10.16.100.28 <none> Red Hat Enterprise Linux CoreOS 43.81.202004130853.0 (Ootpa) 4.18.0-147.8.1.el8_1.x86_64 cri-o://1.16.5-1.dev.rhaos4.3.git91157c1.el8
worker1.ocp43-dev.os.fyre.ibm.com Ready worker 10h v1.16.2 10.16.100.29 <none> Red Hat Enterprise Linux CoreOS 43.81.202004130853.0 (Ootpa) 4.18.0-147.8.1.el8_1.x86_64 cri-o://1.16.5-1.dev.rhaos4.3.git91157c1.el8
worker2.ocp43-dev.os.fyre.ibm.com Ready worker 11h v1.16.2 10.16.100.30 <none> Red Hat Enterprise Linux CoreOS 43.81.202004130853.0 (Ootpa) 4.18.0-147.8.1.el8_1.x86_64 cri-o://1.16.5-1.dev.rhaos4.3.git91157c1.el8
2.0.0-SNAPSHOT-2020-06-23-14-20-27
Additional context
Add any other context about the problem here.
Using https://github.com/open-cluster-management/deploy/blob/master/multiclusterhub/uninstall.sh
to uninstall ACM, completed with:
$ ./uninstall.sh
Now using project "ocm" on server "https://api.nmanos-aws-devcluster-a.devcluster.openshift.com:6443".
Uninstall stuck... Striping out finalizers from helm releases...
The request is invalid
The request is invalid
The request is invalid
The request is invalid
The request is invalid
The request is invalid
The request is invalid
The request is invalid
The request is invalid
Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "certificates.certmanager.k8s.io" not found
Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "certificaterequests.certmanager.k8s.io" not found
Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "challenges.certmanager.k8s.io" not found
Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "clusterissuers.certmanager.k8s.io" not found
Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "issuers.certmanager.k8s.io" not found
Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "orders.certmanager.k8s.io" not found
Error from server (NotFound): clusterroles.rbac.authorization.k8s.io "cert-manager-webhook-requester" not found
Error from server (NotFound): clusterrolebindings.rbac.authorization.k8s.io "cert-manager-webhook-auth-delegator" not found
Error from server (NotFound): consolelinks.console.openshift.io "acm-console-link" not found
customresourcedefinition.apiextensions.k8s.io "userpreferences.console.open-cluster-management.io" deleted
Error from server (NotFound): oauthclients.oauth.openshift.io "multicloudingress" not found
Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "endpointconfigs.multicloud.ibm.com" not found
Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "klusterletconfigs.agent.open-cluster-management.io" not found
Error from server (NotFound): clusterroles.rbac.authorization.k8s.io "rcm-controller" not found
Error from server (NotFound): clusterrolebindings.rbac.authorization.k8s.io "rcm-controller" not found
error: the server doesn't have a resource type "policies"
error: the server doesn't have a resource type "policies"
error: the server doesn't have a resource type "policies"
error: the server doesn't have a resource type "policies"
configmap "ingress-controller-leader-ingress-open-cluster-management" deleted
customresourcedefinition.apiextensions.k8s.io "agentclusterinstalls.extensions.hive.openshift.io" deleted
error: resource(s) were provided, but no name was specified
No resources found in hive namespace.
No resources found in hive namespace.
No resources found in hive namespace.
error: the server doesn't have a resource type "hiveconfig"
But later when looking at the ACM namespace I still see several ACM services, replicasets, deployments and olms, which are not in termination process:
$ oc get all,olm -n ocm
NAME READY STATUS RESTARTS AGE
pod/console-chart-d59f8-console-v2-797dff8b66-66d29 1/1 Terminating 0 7h
pod/console-chart-d59f8-console-v2-797dff8b66-tndpb 1/1 Terminating 0 7h
pod/multicluster-operators-hub-subscription-84bbd555ff-jb8xb 1/1 Terminating 0 7h2m
pod/multicluster-operators-standalone-subscription-64fc6cfdc4-qvkw8 1/1 Terminating 0 7h2m
pod/multiclusterhub-repo-5b66558845-tlbgg 1/1 Running 0 7h1m
pod/my-catalog-source-kscwk 1/1 Running 0 157m
pod/ocm-controller-8485654c75-46mt2 1/1 Running 0 7h
pod/ocm-controller-8485654c75-8qppv 1/1 Running 0 7h
pod/ocm-proxyserver-856f674b8b-5b5lh 1/1 Running 0 7h
pod/ocm-proxyserver-856f674b8b-fx8gs 1/1 Running 0 7h
pod/ocm-webhook-67fc798c69-p5r5g 1/1 Running 0 7h1m
pod/ocm-webhook-67fc798c69-smdmb 1/1 Running 0 7h1m
pod/search-prod-19fb4-search-api-65c887c77-5qtcn 1/1 Terminating 0 6h59m
pod/search-prod-19fb4-search-api-65c887c77-q4c22 1/1 Terminating 0 6h59m
pod/search-redisgraph-0 1/1 Terminating 0 6h58m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/multiclusterhub-operator-webhook ClusterIP 172.40.160.186 <none> 443/TCP 7h1m
service/multiclusterhub-repo ClusterIP 172.40.195.78 <none> 3000/TCP 7h1m
service/my-catalog-source ClusterIP 172.40.164.109 <none> 50051/TCP 157m
service/ocm-proxyserver ClusterIP 172.40.15.120 <none> 443/TCP 7h
service/ocm-webhook ClusterIP 172.40.80.99 <none> 443/TCP 7h1m
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/multiclusterhub-repo 1/1 1 1 7h1m
deployment.apps/ocm-controller 2/2 2 2 7h
deployment.apps/ocm-proxyserver 2/2 2 2 7h
deployment.apps/ocm-webhook 2/2 2 2 7h1m
NAME DESIRED CURRENT READY AGE
replicaset.apps/multiclusterhub-repo-5b66558845 1 1 1 7h1m
replicaset.apps/ocm-controller-8485654c75 2 2 2 7h
replicaset.apps/ocm-proxyserver-856f674b8b 2 2 2 7h
replicaset.apps/ocm-webhook-67fc798c69 2 2 2 7h1m
NAME IMAGE REPOSITORY TAGS
imagestream.image.openshift.io/acm-operator-bundle-index image-registry.openshift-image-registry.svc:5000/ocm/acm-operator-bundle-index v2.4.0 3
NAME AGE
operatorgroup.operators.coreos.com/my-group 7h2m
NAME AGE
operator.operators.coreos.com/advanced-cluster-management.ocm 7h2m
operator.operators.coreos.com/submariner.submariner-operator 6h51m
NAME DISPLAY TYPE PUBLISHER AGE
catalogsource.operators.coreos.com/my-catalog-source Testing Catalog Source grpc Red Hat Partner (Test) 157m
To Reproduce
Install ACM + managed clusters.
Use the following script to uninstall ACM:
https://github.com/open-cluster-management/deploy/blob/master/multiclusterhub/uninstall.sh
Run "oc get all,olm -n ocm" (on the ACM namespace).
Expected behavior
All ACM resources should be deleted or in termination process.
Describe the bug
Cert manager is trying to issue certificates, but it appears to get errors when updating the certificate resource.
Error Detail.
E0731 22:10:10.006286 1 controller.go:131] cert-manager/controller/certificates "msg"="re-queuing item due to error processing" "error"="Certificate.certmanager.k8s.io \"multicloud-ca-cert\" is invalid: metadata.managedFields.fieldsType: Invalid value: \"\": must be `FieldsV1`" "key"="open-cluster-management/multicloud-ca-cert"
I think #99 is caused by this.
To Reproduce
Deploy 4.5 on vsphere (behind firewall)
Install ACM
Expected behavior
A clear and concise description of what you expected to happen.
Additional context
$ k describe po mcm-apiserver-c66554bf6-rdsmw
Warning FailedMount 7m30s (x13 over 17m) kubelet, mgmt-lzlbb-worker-7dvtj MountVolume.SetUp failed for volume "mongodb-ca-cert" : secret "multicloud-ca-cert" not found
Warning FailedMount 2m4s (x2 over 6m35s) kubelet, mgmt-lzlbb-worker-7dvtj Unable to attach or mount volumes: unmounted volumes=[mongodb-ca-cert mongodb-client-cert], unattached volumes=[hub-sa-token-tl2j7 apiserver-certs klusterlet-certs mongodb-ca-cert mongodb-client-cert]: timed out waiting for the condition
root $ k get certificate
NAME READY SECRET AGE EXPIRATION
grc-2f6b9-ca-cert False grc-2f6b9-grc-secrets 16m
kui-proxy False kui-proxy-secret 16m
management-ingress-31ccb-cert False management-ingress-31ccb-tls-secret 16m
multicloud-ca-cert False multicloud-ca-cert 16m
Error on step 4:
kubectl apply -k .
namespace/hive created
namespace/open-cluster-management created
secret/multiclusterhub-operator-pull-secret created
Error from server (AlreadyExists): error when creating ".": serviceaccounts "default" already exists
Describe the bug
When importing an existing cluster, although the cluster imports successfully and reports "ready", I get the following error:
$ oc logs endpoint-appmgr-b77bd7cc6-9v298 -n multicluster-endpoint
I0318 19:07:11.008195 1 manager.go:53] Go Version: go1.13
I0318 19:07:11.008294 1 manager.go:54] Go OS/Arch: linux/amd64
I0318 19:07:11.008305 1 manager.go:55] Version of operator-sdk: v0.15.0+git
I0318 19:07:13.315393 1 manager.go:103] Starting ... Registering Components for cluster: mcm-spoke3/mcm-spoke3
I0318 19:07:18.750275 1 discovery.go:137] Discovering cluster resources
E0318 19:07:21.194216 1 namespace_subscriber.go:121] failed to initialize default channel to cluster namespace
E0318 19:07:21.194244 1 manager.go:191] Failed to initialize subscriber with error:failed to get informer for deployable from cache: no matches for kind "Deployable" in version "apps.open-cluster-management.io/v1"
E0318 19:07:21.194270 1 manager.go:130] Failed to setup standalone subscription, error:failed to get informer for deployable from cache: no matches for kind "Deployable" in version "apps.open-cluster-management.io/v1"
To Reproduce
Steps to reproduce the behavior:
Expected behavior
All pods in the multicluster-endpoint namespace are up and running.
Screenshots
none, see output above.
Both hub and spoke are OCP 4.3.5 IPv4.
Desktop (please complete the following information):
Additional context
Add any other context about the problem here.
Hi,
I'm trying to get pull secret following the docs, but it seems Slack channel is not open for non-Redhat. Is Open Cluster Management open for external testing ?
Regards
Attempted to follow the steps in https://github.com/open-cluster-management/deploy
After Step 6 checking the pods status:
open-cluster-management 15b2552bda98af9ca6f85d34f2bfb89e5e18d86f7c7267f112126cfd278g9hc 0/1 Init:ImagePullBackOff 0 21m
Checking what's wrong with that pod:
Failed to pull image "quay.io/open-cluster-management/multicloudhub-operator-bundle@sha256:ba919b34aa7c7c7135f4474791defb6240ff1af2491b876c25ba960af81c5267": rpc error: code = Unknown desc = Error reading manifest sha256:ba919b34aa7c7c7135f4474791defb6240ff1af2491b876c25ba960af81c5267 in quay.io/open-cluster-management/multicloudhub-operator-bundle: unauthorized: access to the requested resource is not authorized
Tried several users with access to quay. Same error.
Hi.
Is it possible to add as managed cluster the same cluster where the MultiClusterHub instance is?
Thank you.
I take that we should not copy a file and replace it back in the same dir.
copy the pull-secret.yaml file from the prereqs dir and paste it back into the prereqs directory with the name quay-secret.yaml
Should it be:
copy the pull-secret.yaml file from the prereqs dir and paste it back into the multicloudhub-operator directory with the name quay-secret.yaml
Describe the bug
In nuke.sh, there should be a prompt(like the one in uninstall.sh) added to give users a chance to confirm before proceeding to clean up...esp nuke is a destructive script like uninstall.sh
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Added a prompt (from uninstall.sh)
echo "***"
printf "\n"
echo "This script will destroy Open Cluster Management from the current OpenShift target cluster:"
printf "\n"
oc cluster-info | head -n 1 | awk '{print $NF}'
printf "\n"
echo "If you would like to proceed with cleanup, type: DESTROY"
read -r DESTROY_YES
if [ "${DESTROY_YES}" != "DESTROY" ]; then
echo "You must type DESTROY to clean up the Hive deployed clusters"
exit 1
fi
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):
Additional context
Add any other context about the problem here.
Describe the bug
apiVersion: operators.open-cluster-management.io/v1beta1
kind: MultiClusterHub
metadata:
name: multiclusterhub
namespace: open-cluster-management
spec:
imagePullSecret: pull-secret
multiclusterhub-operator
deployes the repo.{
"level":"info",
"ts":1595934620.980875,
"logger":"controller_multiclusterhub",
"msg":"Waiting for API group to be available",
"API group":"certmanager.k8s.io/v1alpha1"
}
To Reproduce
Steps to reproduce the behavior:
oc create -f ...
of the above yamloc -n open-cluster-management logs multiclusterhub-operator-<POD>
Expected behavior
I expect a route to be able to login to ACM Web console
Desktop (please complete the following information):
Describe the bug
multicloudhub-operator/uninstall.sh hangs
To Reproduce
Steps to reproduce the behavior:
% cd multicloudhub-operator
% kubectl delete -k .
service "open-cluster-management-registry" deleted
deployment.apps "open-cluster-management-registry" deleted
operatorgroup.operators.coreos.com "default" deleted
catalogsource.operators.coreos.com "open-cluster-management" deleted
subscription.operators.coreos.com "multicloudhub-operator-bundle" deleted
% ./uninstall.sh
Already on project "open-cluster-management" on server "https://api.sberens-demo.acm-demo.azure.devcluster.openshift.com:6443".
clusterserviceversion.operators.coreos.com "multicloudhub-operator.v0.0.1" deleted
clusterserviceversion.operators.coreos.com "etcdoperator.v0.9.4" deleted
clusterserviceversion.operators.coreos.com "multicloud-operators-subscription.v0.1.2" deleted
customresourcedefinition.apiextensions.k8s.io "multicloudhubs.operators.multicloud.ibm.com" deleted
customresourcedefinition.apiextensions.k8s.io "channels.app.ibm.com" deleted
customresourcedefinition.apiextensions.k8s.io "deployables.app.ibm.com" deleted
customresourcedefinition.apiextensions.k8s.io "helmreleases.app.ibm.com" deleted
^C
% oc get customresourcedefinition.apiextensions.k8s.io/helmreleases.app.ibm.com
NAME CREATED AT
helmreleases.app.ibm.com 2020-03-11T19:49:21Z
% oc describe customresourcedefinition.apiextensions.k8s.io/helmreleases.app.ibm.com
Name: helmreleases.app.ibm.com
Namespace:
Labels: <none>
Annotations: <none>
API Version: apiextensions.k8s.io/v1
Kind: CustomResourceDefinition
Metadata:
Creation Timestamp: 2020-03-11T19:49:21Z
Deletion Grace Period Seconds: 0
Deletion Timestamp: 2020-03-12T05:12:49Z
Finalizers:
customresourcecleanup.apiextensions.k8s.io
Generation: 1
Resource Version: 2964709
Self Link: /apis/apiextensions.k8s.io/v1/customresourcedefinitions/helmreleases.app.ibm.com
UID: 3f28d79f-a0f3-403c-90d5-eb6c7db104d7
Spec:
Conversion:
Strategy: None
Group: app.ibm.com
Names:
Kind: HelmRelease
List Kind: HelmReleaseList
Plural: helmreleases
Singular: helmrelease
Preserve Unknown Fields: true
Scope: Namespaced
Versions:
Name: v1alpha1
Schema:
openAPIV3Schema:
Description: HelmRelease is the Schema for the subscriptionreleases API
Properties:
API Version:
Description: APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources
Type: string
Kind:
Description: Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds
Type: string
Metadata:
Type: object
Spec:
Description: HelmReleaseSpec defines the desired state of HelmRelease
Properties:
Chart Name:
Description: ChartName is the name of the chart within the repo
Type: string
Config Map Ref:
Description: Configuration parameters to access the helm-repo defined in the CatalogSource
Properties:
API Version:
Description: API version of the referent.
Type: string
Field Path:
Description: If referring to a piece of an object instead of an entire object, this string should contain a valid JSON/Go field access statement, such as desiredState.manifest.containers[2]. For example, if the object reference is to a container within a pod, this would take on a value like: "spec.containers{name}" (where "name" refers to the name of the container that triggered the event) or if no container name is specified "spec.containers[2]" (container with index 2 in this pod). This syntax is chosen only to have some well-defined way of referencing a part of an object. TODO: this design is not final and this field is subject to change in the future.
Type: string
Kind:
Description: Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds
Type: string
Name:
Description: Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
Type: string
Namespace:
Description: Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/
Type: string
Resource Version:
Description: Specific resourceVersion to which this reference is made, if any. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#concurrency-control-and-consistency
Type: string
UID:
Description: UID of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids
Type: string
Type: object
Secret Ref:
Description: Secret to use to access the helm-repo defined in the CatalogSource.
Properties:
API Version:
Description: API version of the referent.
Type: string
Field Path:
Description: If referring to a piece of an object instead of an entire object, this string should contain a valid JSON/Go field access statement, such as desiredState.manifest.containers[2]. For example, if the object reference is to a container within a pod, this would take on a value like: "spec.containers{name}" (where "name" refers to the name of the container that triggered the event) or if no container name is specified "spec.containers[2]" (container with index 2 in this pod). This syntax is chosen only to have some well-defined way of referencing a part of an object. TODO: this design is not final and this field is subject to change in the future.
Type: string
Kind:
Description: Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds
Type: string
Name:
Description: Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
Type: string
Namespace:
Description: Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/
Type: string
Resource Version:
Description: Specific resourceVersion to which this reference is made, if any. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#concurrency-control-and-consistency
Type: string
UID:
Description: UID of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids
Type: string
Type: object
Source:
Description: INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "operator-sdk generate k8s" to regenerate code after modifying this file Add custom validation using kubebuilder tags: https://book-v1.book.kubebuilder.io/beyond_basics/generating_crd.html Source holds the url toward the helm-chart
Properties:
Github:
Description: GitHub provides the parameters to access the helm-chart located in a github repo
Properties:
Branch:
Type: string
Chart Path:
Type: string
Urls:
Items:
Type: string
Type: array
Type: object
Helm Repo:
Description: HelmRepo provides the urls to retrieve the helm-chart
Properties:
Urls:
Items:
Type: string
Type: array
Type: object
Type:
Description: SourceTypeEnum types of sources
Type: string
Type: object
Values:
Description: Values is a string containing (unparsed) YAML values
Type: string
Version:
Description: Version is the chart version
Type: string
Type: object
Status:
Description: HelmReleaseStatus struct containing the status
Properties:
Last Update:
Format: date-time
Type: string
Message:
Type: string
Phase:
Description: HelmReleaseStatusEnum defines the status of a Subscription release
Type: string
Reason:
Type: string
Required:
lastUpdate
Type: object
Type: object
Served: true
Storage: true
Subresources:
Status:
Status:
Accepted Names:
Kind: HelmRelease
List Kind: HelmReleaseList
Plural: helmreleases
Singular: helmrelease
Conditions:
Last Transition Time: 2020-03-11T19:49:21Z
Message: no conflicts found
Reason: NoConflicts
Status: True
Type: NamesAccepted
Last Transition Time: 2020-03-11T19:49:16Z
Message: the initial names have been accepted
Reason: InitialNamesAccepted
Status: True
Type: Established
Last Transition Time: 2020-03-12T05:12:49Z
Message: CustomResource deletion is in progress
Reason: InstanceDeletionInProgress
Status: True
Type: Terminating
Stored Versions:
v1alpha1
Events: <none>
Expected behavior
I expect the deploy uninstall.sh to clear up any related items cleanly.
Screenshots
NA
Desktop (please complete the following information):
Additional context
The documentation link in the project's README seems to be broken.
See: https://github.com/open-cluster-management/deploy/blame/master/README.md#L32
(Needed to use the blame version as it is otherwise not possible to get the actual markdown source in github)
The links inside the linked documentation always refer to outdated sites. Also the commit history also does not look up-to-date.
Describe the bug
The uninstall script assumes the clustername = namespace
https://github.com/open-cluster-management/deploy/blob/master/clean-clusters.sh#L41
oc delete namespace ${clusterName} --wait=false
But if it doesn't, the cluster is not deleted (the namespace is deleted)
Detaching imported clusters
Detaching cluster kni1-virtualmasters-baremetalworkers
Error from server (NotFound): clusters.clusterregistry.k8s.io "kni1-virtualmasters-baremetalworkers" not found
namespace "kni1-virtualmasters-baremetalworkers" deleted
Second pass cleaning, by endpointConfig
Detaching cluster kni1-virtualmasters-baremetalworkers
Error from server (NotFound): clusters.clusterregistry.k8s.io "kni1-virtualmasters-baremetalworkers" not found
namespace "kni1-virtualmasters-baremetalworkers" deleted
To Reproduce
Steps to reproduce the behavior:
Expected behavior
The proper namespace should be deleted
Screenshots
N/A
Desktop (please complete the following information):
Additional context
Add any other context about the problem here.
Hi,
How can I configure the storage class that the etc instance uses?
After installting the operator and creating a MultiClusterHub instance I see an etcd-cluster instance also, it by default uses the defautl storage class.
Is it possible to configure what storage class to use by the etcd instead of the default one?
Thank you.
Describe the bug
We are now in April, can this be updated "here(private link... to be opened 03/27/20)."
https://github.com/open-cluster-management/deploy/blob/master/README.md
To Reproduce
Steps to reproduce the behavior:
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):
Additional context
Really, the readme is charming! Thanks!!
Describe the bug
Using the current default snapshot version 1.0.0-SNAPSHOT-2020-03-30-18-55-25
the deployment fails with error on topology-aggregator
pods due unavailable tag: Failed to pull image "quay.io/open-cluster-management/topology-aggregator:0.0.1-SNAPSHOT-2020-03-30-17-01-26": (...) Error reading manifest 0.0.1-SNAPSHOT-2020-03-30-17-01-26 in quay.io/open-cluster-management/topology-aggregator: unauthorized: access to the requested resource is not authorized
To Reproduce
./start.sh --watch
with default values.Expected behavior
Deployment finishes successfully.
Describe the bug
The start script shows an incorrect console URL:
Pods still NOT running : 14
Detected ACM Console URL: https://multicloud-console.apps.acm-dev-pmorie.devcluster.openshift.com
The correct URL is https://multicluster-console.apps.acm-dev-pmorie.devcluster.openshift.com/
To Reproduce
start.sh --watch
Expected behavior
The right URL
Describe the bug
Hints/Notes: If the correct snapshot format (1.0.0-SNAPSHOT-2020-04-01-02-43-31) is not used then the start.sh script used to install ACM will timeout and fail at Wait for multiclusterhub-operator to reach running state (4min). The default snapshot will also be replaced with the mistyped snapshot. To resolve use the correct snapshot format when prompted and the script will run successfully.
Also the script will also fail at Wait for multiclusterhub-operator to reach running state (4min). if the correct pull-secret is not used. Make sure when generating the pull secret from Quay, you select Kubernetes Secret as it will generate the wrong format. You'll see in the script an error at apply per-requisites with the pull secret if the wrong one is used. "Failed to read Resources YAML file [pull-secret.yaml] encounters a format error.
Are we handling the above in deploy doc or improvements to the start.sh
? eg, script can verify format of the user-provided snapshot. script can verify quay credentials prior to beginning the pulls.
Is your feature request related to a problem? Please describe.
When no default StorageClass is defined, the deployment fails
Describe the solution you'd like
Specify in the Doc that a default StorageClass must be set.
Annotation:
storageclass.kubernetes.io/is-default-class
And if possible check in the deployment script:
SC_RESOLVE=$(oc get sc 2>&1)
if [[ $SC_RESOLVE =~ (default) ]];
then
echo " OK: Default Storage Class defined"
else
echo " ERROR: No default Storage Class defined."
echo " Define Annotation: storageclass.kubernetes.io/is-default-class=true"
echo " Aborting."
exit 1
fi
Describe the bug
uninstall.sh
fails to detect installed oc
4.10 versions. Code can be found here.
To Reproduce
Steps to reproduce the behavior:
Install oc version 4.10+
./uninstall.sh
Is your feature request related to a problem? Please describe.
We demonstrated RHACM to a Customer showing ETCD encryption and ETCD backup.
Customer would like to monitor both. This Enhancement is about how to monitor ETCD backup is working properly.
This policy currently does the backup:
https://github.com/ch-stark/policies-demo/blob/master/policies/backup-etcd-policy-v2.yaml
Generated yaml-objects
kind: ServiceAccount
kind: ClusterRoleBinding
kind: PersistentVolumeClaim
metadata:
name: etcd-backup
kind: ConfigMap
metadata:
name: etcd-backup-script
kind: CronJob
it works so far
This is an example how to monitor that:
https://github.com/ch-stark/policies-demo/blob/master/policies/policy-checkjob.yaml
it checks if the job that does the backup has succeeded but this is not ideal
Describe the solution you'd like
Support for such a policy
Describe alternatives you've considered
We check manually:
oc logs cronjob-etcd-backup-1603974060-jj7bt
found latest kube-apiserver-pod: /etc/kubernetes/static-pod-resources/kube-apiserver-pod-9
found latest kube-controller-manager-pod: /etc/kubernetes/static-pod-resources/kube-controller-manager-pod-8
found latest kube-scheduler-pod: /etc/kubernetes/static-pod-resources/kube-scheduler-pod-8
found latest etcd-pod: /etc/kubernetes/static-pod-resources/etcd-pod-3
Snapshot saved at /assets/backup/snapshot_2020-10-29_122123.db
snapshot db and kube resources are successfully saved to /assets/backup
Copied backup files to PVC mount point.
We can monitor the timestamp of the backup
Additional context
We discussed that a CustomPolicyController might be needed
Thanks for any help!!!
Describe the bug
It appears that the deploy process for the multi-cluster engine expects a file that doesn't exist. I followed the directions in the README.md
To Reproduce
Steps to reproduce the behavior:
oc login
and token obtained from the console./start.sh
2.0-latest
as the SNAPSHOT TAGExpected behavior
I'd expect the multi-cluster operator to be installed.
Screenshots
$ ./start.sh
* Testing connection
* Using baseDomain: router-default.lbragstad-dev.devcluster.openshift.com
* oc CLI Client Version: 4.10.6
Find snapshot tags @ https://quay.io/stolostron/cmb-custom-registry?tab=tags
Enter SNAPSHOT TAG:
2.0-latest
SNAPSHOT_CHOICE is set to 2.0-latest
Error: stat catalogsources/multicluster-engine.yaml: no such file or directory
Desktop (please complete the following information):
When deploying the upstream (mid-stream?) stolostron operator on OKD 4.12, everything deploys as expected, except for the hypershift-addon-agent pod. I get this pull failure for the kube-rbac-proxy-mce container.
Failed to pull image "quay.io/stolostron/kube-rbac-proxy-mce@sha256:d0cd6580603bcdf2ebef1f9a836f91c024ce8113dca480962b515bc0efbda523": rpc error: code = Unknown desc = quay.io/stolostron/kube-rbac-proxy-mce: unauthorized: access to the requested resource is not authorized
Note, the other container in the pod deploys successfully (hypershift-addon-agent)
Describe the bug
Need to specify that the minimum oc cli version is 4.3. I had 4.1 and the start.sh script threw an error since the -k flag wasn't valid for 4.1.
Expected behavior
start script can check the version level before running. splat an error if below 4.3.
I want to run a demo of OCM, but as per the documentation, I must prepare an OpenShift Container Platform.
So, I wonder if I can run OCM on my original Kubernetes clusters, such as a Kind cluster?
Describe the bug
This is required maintenance
Describe the bug
When running the start.sh
script, the following error happens:
Error from server (AlreadyExists): error when creating "prereqs/": serviceaccounts "default" already exists
To Reproduce
Steps to reproduce the behavior:
start.sh
Expected behavior
kubectl apply should just update the default service account that already exists with the imagePullSecret
Desktop (please complete the following information):
Client Version: version.Info{Major:"", Minor:"", GitVersion:"v0.0.0-master+$Format:%h$", GitCommit:"$Format:%H$", GitTreeState:"", BuildDate:"1970-01-01T00:00:00Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"16+", GitVersion:"v1.16.2", GitCommit:"aa10b5b", GitTreeState:"clean", BuildDate:"2020-03-16T18:11:23Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"}
Additional context
This is present in my locally cloned start.sh
https://github.com/open-cluster-management/deploy/blob/master/start.sh#L160
[root@bastion deploy]# oc get sa default -oyaml
apiVersion: v1
imagePullSecrets:
- name: default-dockercfg-r4kf9
kind: ServiceAccount
metadata:
creationTimestamp: "2020-04-07T15:33:46Z"
name: default
namespace: open-cluster-management
resourceVersion: "258550"
selfLink: /api/v1/namespaces/open-cluster-management/serviceaccounts/default
uid: 067cdba1-e067-4904-8d4b-47ff1c375eae
secrets:
- name: default-token-672pk
- name: default-dockercfg-r4kf9
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.