Comments (3)
tmshort
commented on July 22, 2024
Took 8 attempts to fail; 12.5% occurrance.
from operator-lifecycle-manager.
tmshort
commented on July 22, 2024
This is likely due to an error updating the subscription, possibly at a critical time. But, I do see similar errors during a success.
This is from the catalog-operator pod:
2023-11-07T19:43:34.379154025Z stderr F time="2023-11-07T19:43:34Z" level=debug msg="resolving subscriptions in namespace" id=TjnDL namespace=fail-forward-e2e-t25w6
2023-11-07T19:43:34.387130717Z stderr F time="2023-11-07T19:43:34Z" level=warning msg="an error was encountered during reconciliation" error="Operation cannot be fulfilled on subscriptions.operators.coreos.com \"mc-csv-failed-qn8mm-sub\": the object has been modified; please apply your changes to the latest version and try again" event=update reconciling="*v1alpha1.Subscription" selflink=
2023-11-07T19:43:34.388610715Z stderr F E1107 19:43:34.388510 1 queueinformer_operator.go:319] sync {"update" "fail-forward-e2e-t25w6/mc-csv-failed-qn8mm-sub"} failed: Operation cannot be fulfilled on subscriptions.operators.coreos.com "mc-csv-failed-qn8mm-sub": the object has been modified; please apply your changes to the latest version and try again
2023-11-07T19:43:34.388813015Z stderr F time="2023-11-07T19:43:34Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=
From the e2e log:
19:44:38.6016: subscription fail-forward-e2e-t25w6/mc-csv-failed-qn8mm-sub state: AtLatestKnown (csv example-operator.v0.2.0): installPlanRef: &v1.ObjectReference{Kind:"InstallPlan", Namespace:"fail-forward-e2e-t25w6", Name:"install-kktrz", UID:"0666549d-ad6f-4d65-abd7-2ab374236509", APIVersion:"operators.coreos.com/v1alpha1", ResourceVersion:"1122", FieldPath:""}
19:49:38.5972: subscription fail-forward-e2e-t25w6/mc-csv-failed-qn8mm-sub never got correct status: v1alpha1.SubscriptionStatus{CurrentCSV:"example-operator.v0.2.0", InstalledCSV:"example-operator.v0.2.0", Install:(*v1alpha1.InstallPlanReference)(0xc000fc3f40), State:"AtLatestKnown", Reason:"", InstallPlanGeneration:2, InstallPlanRef:(*v1.ObjectReference)(0xc0005fc690), CatalogHealth:[]v1alpha1.SubscriptionCatalogHealth{v1alpha1.SubscriptionCatalogHealth{CatalogSourceRef:(*v1.ObjectReference)(0xc0005fc5b0), LastUpdated:time.Date(2023, time.November, 7, 19, 44, 17, 0, time.Local), Healthy:true}, v1alpha1.SubscriptionCatalogHealth{CatalogSourceRef:(*v1.ObjectReference)(0xc0005fc620), LastUpdated:time.Date(2023, time.November, 7, 19, 44, 17, 0, time.Local), Healthy:true}}, Conditions:[]v1alpha1.SubscriptionCondition{v1alpha1.SubscriptionCondition{Type:"CatalogSourcesUnhealthy", Status:"False", Reason:"AllCatalogSourcesHealthy", Message:"all available catalogsources are healthy", LastHeartbeatTime:<nil>, LastTransitionTime:time.Date(2023, time.November, 7, 19, 44, 17, 0, time.Local)}, v1alpha1.SubscriptionCondition{Type:"BundleUnpacking", Status:"False", Reason:"", Message:"", LastHeartbeatTime:<nil>, LastTransitionTime:<nil>}, v1alpha1.SubscriptionCondition{Type:"ResolutionFailed", Status:"True", Reason:"ConstraintsNotSatisfiable", Message:"constraints not satisfiable: clusterserviceversion example-operator.v0.1.0 exists and is not referenced by a subscription, @existing/fail-forward-e2e-t25w6//example-operator.v0.2.0, @existing/fail-forward-e2e-t25w6//example-operator.v0.1.0 and mc-csv-failed-qn8mm/fail-forward-e2e-t25w6/stable/example-operator.v0.3.0 originate from package packageA, subscription mc-csv-failed-qn8mm-sub exists, subscription mc-csv-failed-qn8mm-sub requires at least one of mc-csv-failed-qn8mm/fail-forward-e2e-t25w6/stable/example-operator.v0.3.0 or @existing/fail-forward-e2e-t25w6//example-operator.v0.2.0", LastHeartbeatTime:<nil>, LastTransitionTime:<nil>}}, LastUpdated:time.Date(2023, time.November, 7, 19, 44, 38, 0, time.Local)}
from operator-lifecycle-manager.
stevekuznetsov
commented on July 22, 2024
Fail Forward Upgrades when a CSV resource is in a failed state eventually reports a successful state when using skips
/go/src/github.com/openshift/operator-framework-olm/staging/operator-lifecycle-manager/test/e2e/fail_forward_e2e_test.go:402
STEP: creating the testing namespace with an OG that enabled fail forward behavior @ 11/14/23 02:22:51.908
created the fail-forward-e2e-hm8m2 testing namespace
STEP: deploying the testing catalog @ 11/14/23 02:22:51.999
Creating catalog source mc-csv-failed-b47k7 in namespace fail-forward-e2e-hm8m2...
Catalog source mc-csv-failed-b47k7 created
waiting 148.179373ms for catalog pod fail-forward-e2e-hm8m2/mc-csv-failed-b47k7 to be available (for sync) - NO_CONNECTION
waiting 590.956103ms for catalog pod fail-forward-e2e-hm8m2/mc-csv-failed-b47k7 to be available (for sync) - IDLE
waiting 200.019269ms for catalog pod fail-forward-e2e-hm8m2/mc-csv-failed-b47k7 to be available (for sync) - TRANSIENT_FAILURE
waiting 602.234668ms for catalog pod fail-forward-e2e-hm8m2/mc-csv-failed-b47k7 to be available (for sync) - CONNECTING
waiting 197.393456ms for catalog pod fail-forward-e2e-hm8m2/mc-csv-failed-b47k7 to be available (for sync) - TRANSIENT_FAILURE
waiting 15.656604807s for catalog pod fail-forward-e2e-hm8m2/mc-csv-failed-b47k7 to be available (for sync) - READY
probing catalog mc-csv-failed-b47k7 pod with address mc-csv-failed-b47k7.fail-forward-e2e-hm8m2.svc:50051
skipping health check
STEP: creating the testing subscription @ 11/14/23 02:23:09.484
STEP: waiting until the subscription has an IP reference @ 11/14/23 02:23:09.526
02:23:10.8861: subscription fail-forward-e2e-hm8m2/mc-csv-failed-b47k7-sub state: UpgradePending (csv example-operator.v0.1.0): installPlanRef: &v1.ObjectReference{Kind:"InstallPlan", Namespace:"fail-forward-e2e-hm8m2", Name:"install-sl4wl", UID:"1df970d6-3803-406d-8dc4-214c36af1b22", APIVersion:"operators.coreos.com/v1alpha1", ResourceVersion:"128538", FieldPath:""}
waiting 1.359020737s for subscription fail-forward-e2e-hm8m2/mc-csv-failed-b47k7-sub to have installplan ref: has ref &v1.ObjectReference{Kind:"InstallPlan", Namespace:"fail-forward-e2e-hm8m2", Name:"install-sl4wl", UID:"1df970d6-3803-406d-8dc4-214c36af1b22", APIVersion:"operators.coreos.com/v1alpha1", ResourceVersion:"128538", FieldPath:""}
STEP: waiting for the v0.1.0 CSV to report a succeeded phase @ 11/14/23 02:23:10.886
waiting for CSV fail-forward-e2e-hm8m2/example-operator.v0.1.0 to reach condition
waited 199.608397ms for csv fail-forward-e2e-hm8m2/example-operator.v0.1.0 - Installing (InstallWaiting): installing: waiting for deployment example-operator-controller-manager to become ready: deployment "example-operator-controller-manager" not available: Deployment does not have minimum availability.
waited 199.697425ms for CSV fail-forward-e2e-hm8m2/example-operator.v0.1.0: to be in phases [Succeeded], in phase Installing (InstallWaiting): installing: waiting for deployment example-operator-controller-manager to become ready: deployment "example-operator-controller-manager" not available: Deployment does not have minimum availability.
waited 1.601659075s for csv fail-forward-e2e-hm8m2/example-operator.v0.1.0 - Succeeded (InstallSucceeded): install strategy completed with no errors
waited 1.601733023s for CSV fail-forward-e2e-hm8m2/example-operator.v0.1.0: to be in phases [Succeeded], in phase Succeeded (InstallSucceeded): install strategy completed with no errors
STEP: updating the catalog with a broken v0.2.0 csv @ 11/14/23 02:23:12.688
STEP: removing the existing catalog source @ 11/14/23 02:23:12.688
STEP: removing the previous catalog source pod(s) @ 11/14/23 02:23:12.892
waiting for the catalog source mc-csv-failed-b47k7-8ckq6 pod to be deleted...
STEP: updating the catalog with a new bundle images @ 11/14/23 02:23:13.976
Creating catalog source mc-csv-failed-b47k7 in namespace fail-forward-e2e-hm8m2...
Catalog source mc-csv-failed-b47k7 created
waiting 138.410284ms for catalog pod fail-forward-e2e-hm8m2/mc-csv-failed-b47k7 to be available (for sync) - NO_CONNECTION
waiting 2.26910633s for catalog pod fail-forward-e2e-hm8m2/mc-csv-failed-b47k7 to be available (for sync) - TRANSIENT_FAILURE
waiting 2.598714655s for catalog pod fail-forward-e2e-hm8m2/mc-csv-failed-b47k7 to be available (for sync) - CONNECTING
waiting 200.99772ms for catalog pod fail-forward-e2e-hm8m2/mc-csv-failed-b47k7 to be available (for sync) - TRANSIENT_FAILURE
waiting 12.400568791s for catalog pod fail-forward-e2e-hm8m2/mc-csv-failed-b47k7 to be available (for sync) - CONNECTING
waiting 199.285203ms for catalog pod fail-forward-e2e-hm8m2/mc-csv-failed-b47k7 to be available (for sync) - READY
probing catalog mc-csv-failed-b47k7 pod with address mc-csv-failed-b47k7.fail-forward-e2e-hm8m2.svc:50051
skipping health check
STEP: verifying the subscription has installed the current csv @ 11/14/23 02:23:31.885
02:23:32.0872: subscription fail-forward-e2e-hm8m2/mc-csv-failed-b47k7-sub state: AtLatestKnown (csv example-operator.v0.1.0): installPlanRef: &v1.ObjectReference{Kind:"InstallPlan", Namespace:"fail-forward-e2e-hm8m2", Name:"install-sl4wl", UID:"1df970d6-3803-406d-8dc4-214c36af1b22", APIVersion:"operators.coreos.com/v1alpha1", ResourceVersion:"128538", FieldPath:""}
02:23:32.2868: subscription fail-forward-e2e-hm8m2/mc-csv-failed-b47k7-sub state: UpgradePending (csv example-operator.v0.2.0): installPlanRef: &v1.ObjectReference{Kind:"InstallPlan", Namespace:"fail-forward-e2e-hm8m2", Name:"install-m4jh9", UID:"4d3d9467-23cd-4055-884c-34ffc0c08b28", APIVersion:"operators.coreos.com/v1alpha1", ResourceVersion:"128987", FieldPath:""}
STEP: waiting for the bad CSV to report a failed state @ 11/14/23 02:23:32.286
waiting for CSV fail-forward-e2e-hm8m2/example-operator.v0.2.0 to reach condition
waited 207.233924ms for csv fail-forward-e2e-hm8m2/example-operator.v0.2.0 - Pending (RequirementsUnknown): requirements not yet checked
waited 1m12.23119933s for CSV fail-forward-e2e-hm8m2/example-operator.v0.2.0: to be in phases [Failed], in phase Pending (RequirementsUnknown): requirements not yet checked
waited 593.392252ms for csv fail-forward-e2e-hm8m2/example-operator.v0.2.0 - InstallReady (AllRequirementsMet): all requirements found, attempting install
waited 593.428936ms for CSV fail-forward-e2e-hm8m2/example-operator.v0.2.0: to be in phases [Failed], in phase InstallReady (AllRequirementsMet): all requirements found, attempting install
waited 398.113032ms for csv fail-forward-e2e-hm8m2/example-operator.v0.2.0 - Failed (InstallComponentFailed): install strategy failed: Deployment.apps "example-operator-controller-manager&invalid" is invalid: metadata.name: Invalid value: "example-operator-controller-manager&invalid": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')
waited 398.154075ms for CSV fail-forward-e2e-hm8m2/example-operator.v0.2.0: to be in phases [Failed], in phase Failed (InstallComponentFailed): install strategy failed: Deployment.apps "example-operator-controller-manager&invalid" is invalid: metadata.name: Invalid value: "example-operator-controller-manager&invalid": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')
STEP: patching the catalog with a fixed version @ 11/14/23 02:23:33.486
STEP: removing the existing catalog source @ 11/14/23 02:23:33.486
STEP: removing the previous catalog source pod(s) @ 11/14/23 02:23:33.698
waiting for the catalog source mc-csv-failed-b47k7-297mr pod to be deleted...
STEP: updating the catalog with a new bundle images @ 11/14/23 02:23:34.793
Creating catalog source mc-csv-failed-b47k7 in namespace fail-forward-e2e-hm8m2...
Catalog source mc-csv-failed-b47k7 created
waiting 138.763895ms for catalog pod fail-forward-e2e-hm8m2/mc-csv-failed-b47k7 to be available (for sync) - NO_CONNECTION
waiting 4.050054318s for catalog pod fail-forward-e2e-hm8m2/mc-csv-failed-b47k7 to be available (for sync) - CONNECTING
waiting 19.996456256s for catalog pod fail-forward-e2e-hm8m2/mc-csv-failed-b47k7 to be available (for sync) - TRANSIENT_FAILURE
waiting 1.000994366s for catalog pod fail-forward-e2e-hm8m2/mc-csv-failed-b47k7 to be available (for sync) - READY
probing catalog mc-csv-failed-b47k7 pod with address mc-csv-failed-b47k7.fail-forward-e2e-hm8m2.svc:50051
skipping health check
STEP: waiting for the subscription to have the example-operator.v0.3.0 status.currentCSV @ 11/14/23 02:24:00.085
02:24:00.2865: subscription fail-forward-e2e-hm8m2/mc-csv-failed-b47k7-sub state: AtLatestKnown (csv example-operator.v0.2.0): installPlanRef: &v1.ObjectReference{Kind:"InstallPlan", Namespace:"fail-forward-e2e-hm8m2", Name:"install-m4jh9", UID:"4d3d9467-23cd-4055-884c-34ffc0c08b28", APIVersion:"operators.coreos.com/v1alpha1", ResourceVersion:"128987", FieldPath:""}
02:29:00.4869: subscription fail-forward-e2e-hm8m2/mc-csv-failed-b47k7-sub never got correct status: v1alpha1.SubscriptionStatus{CurrentCSV:"example-operator.v0.2.0", InstalledCSV:"example-operator.v0.2.0", Install:(*v1alpha1.InstallPlanReference)(0xc001b13e40), State:"AtLatestKnown", Reason:"", InstallPlanGeneration:2, InstallPlanRef:(*v1.ObjectReference)(0xc00087ec40), CatalogHealth:[]v1alpha1.SubscriptionCatalogHealth{v1alpha1.SubscriptionCatalogHealth{CatalogSourceRef:(*v1.ObjectReference)(0xc00087e930), LastUpdated:time.Date(2023, time.November, 14, 2, 23, 41, 0, time.Local), Healthy:true}, v1alpha1.SubscriptionCatalogHealth{CatalogSourceRef:(*v1.ObjectReference)(0xc00087e9a0), LastUpdated:time.Date(2023, time.November, 14, 2, 23, 41, 0, time.Local), Healthy:true}, v1alpha1.SubscriptionCatalogHealth{CatalogSourceRef:(*v1.ObjectReference)(0xc00087ea80), LastUpdated:time.Date(2023, time.November, 14, 2, 23, 41, 0, time.Local), Healthy:true}, v1alpha1.SubscriptionCatalogHealth{CatalogSourceRef:(*v1.ObjectReference)(0xc00087eaf0), LastUpdated:time.Date(2023, time.November, 14, 2, 23, 41, 0, time.Local), Healthy:true}, v1alpha1.SubscriptionCatalogHealth{CatalogSourceRef:(*v1.ObjectReference)(0xc00087eb60), LastUpdated:time.Date(2023, time.November, 14, 2, 23, 41, 0, time.Local), Healthy:true}}, Conditions:[]v1alpha1.SubscriptionCondition{v1alpha1.SubscriptionCondition{Type:"CatalogSourcesUnhealthy", Status:"False", Reason:"AllCatalogSourcesHealthy", Message:"all available catalogsources are healthy", LastHeartbeatTime:<nil>, LastTransitionTime:time.Date(2023, time.November, 14, 2, 23, 41, 0, time.Local)}, v1alpha1.SubscriptionCondition{Type:"BundleUnpacking", Status:"False", Reason:"", Message:"", LastHeartbeatTime:<nil>, LastTransitionTime:<nil>}, v1alpha1.SubscriptionCondition{Type:"ResolutionFailed", Status:"True", Reason:"ConstraintsNotSatisfiable", Message:"constraints not satisfiable: @existing/fail-forward-e2e-hm8m2//example-operator.v0.2.0, @existing/fail-forward-e2e-hm8m2//example-operator.v0.1.0 and mc-csv-failed-b47k7/fail-forward-e2e-hm8m2/stable/example-operator.v0.3.0 originate from package packageA, clusterserviceversion example-operator.v0.1.0 exists and is not referenced by a subscription, subscription mc-csv-failed-b47k7-sub exists, subscription mc-csv-failed-b47k7-sub requires at least one of mc-csv-failed-b47k7/fail-forward-e2e-hm8m2/stable/example-operator.v0.3.0 or @existing/fail-forward-e2e-hm8m2//example-operator.v0.2.0", LastHeartbeatTime:<nil>, LastTransitionTime:<nil>}}, LastUpdated:time.Date(2023, time.November, 14, 2, 28, 34, 0, time.Local)}
02:29:00.4871: subscription fail-forward-e2e-hm8m2/mc-csv-failed-b47k7-sub spec: &v1alpha1.SubscriptionSpec{CatalogSource:"mc-csv-failed-b47k7", CatalogSourceNamespace:"fail-forward-e2e-hm8m2", Package:"packageA", Channel:"stable", StartingCSV:"", InstallPlanApproval:"", Config:(*v1alpha1.SubscriptionConfig)(nil)}
[FAILED] in [It] - /go/src/github.com/openshift/operator-framework-olm/staging/operator-lifecycle-manager/test/e2e/fail_forward_e2e_test.go:412 @ 11/14/23 02:29:00.488
STEP: removing the testing catalog resources @ 11/14/23 02:29:00.489
Deleting config map mc-csv-failed-mkq2q-configmap...
Deleting catalog source mc-csv-failed-mkq2q...
Deleting config map mc-csv-failed-mkq2q-configmap...
Deleting catalog source mc-csv-failed-mkq2q...
Deleting config map mc-csv-failed-mkq2q-configmap...
Deleting catalog source mc-csv-failed-mkq2q...
Deleting config map mc-csv-failed-b47k7-configmap...
Deleting catalog source mc-csv-failed-b47k7...
waiting for the catalog source mc-csv-failed-b47k7-29zm8 pod to be deleted...
Deleting config map mc-csv-failed-b47k7-configmap...
Deleting catalog source mc-csv-failed-b47k7...
Deleting config map mc-csv-failed-b47k7-configmap...
Deleting catalog source mc-csv-failed-b47k7...
STEP: deleting the testing namespace @ 11/14/23 02:29:02.588
collecting the fail-forward-e2e-hm8m2 namespace artifacts as the 'eventually reports a successful state when using skips' test case failed
collecting logs in the /logs/artifacts artifacts directory
unable to determine kubeconfig path so defaulting to the $KUBECONFIG value
Using the oc kubectl binary
Using the /logs/artifacts/fail-forward-e2e-hm8m2 output directory
Storing the test artifact output in the /logs/artifacts/fail-forward-e2e-hm8m2 directory
Collecting get catalogsources -o yaml output...
Collecting get subscriptions -o yaml output...
Collecting get operatorgroups -o yaml output...
Collecting get clusterserviceversions -o yaml output...
Collecting get installplans -o yaml output...
Collecting get pods -o wide output...
Collecting get events --sort-by .lastTimestamp output...
tearing down the fail-forward-e2e-hm8m2 namespace
resetting e2e kube client
deleting fail-forward-e2e-hm8m2/mc-csv-failed-b47k7-sub
deleting fail-forward-e2e-hm8m2/fail-forward-e2e-hm8m2-operatorgroup
deleting <global>/fail-forward-e2e-hm8m2
garbage collecting CRDs
• [FAILED] [375.072 seconds]
Fail Forward Upgrades when a CSV resource is in a failed state [It] eventually reports a successful state when using skips
/go/src/github.com/openshift/operator-framework-olm/staging/operator-lifecycle-manager/test/e2e/fail_forward_e2e_test.go:402
[FAILED] Expected
<wait.errInterrupted>:
timed out waiting for the condition
{
cause: <*errors.errorString | 0xc000250190>{
s: "timed out waiting for the condition",
},
}
to be nil
In [It] at: /go/src/github.com/openshift/operator-framework-olm/staging/operator-lifecycle-manager/test/e2e/fail_forward_e2e_test.go:412 @ 11/14/23 02:29:00.488
likely similar culprit in different test
from operator-lifecycle-manager.
Related Issues (20)
- Unit test Flake: TestRegistryProviderList/OneCatalog/ManyPackages/OneMissingBundle/Elided
- [FAIL] Install Plan [It] creation with permissions
- GORELEASER GitHub Action resulted in two Draft Releases HOT 2
- [FLAKE e2e (0)] ClusterServiceVersion AllNamespaces OperatorGroup [It] update deployment spec in an existing CSV for a hotfix
- [FLAKE e2e (2)] Operator Group [It] e2e functionality HOT 6
- [FLAKE e2e (3)] Subscription when unpacking bundle with bundle which OLM will fail to unpack [It] should expose a condition indicating failure to unpack
- [FLAKE e2e (2)] Operator Group [It] insufficient permissions resolve via RBAC HOT 2
- We should investigate why #3093 is necessary
- Create 'deprecated' condition for Subscription
- Add 'deprecated' condition to Subscription status HOT 1
- Update PackageManifest API with deprecation information HOT 1
- Augment OLM e2e to ensure Subscription API contains deprecation info HOT 1
- Augment OLM e2e to ensure PackageManifests API contains deprecation info HOT 1
- Operator OLM installation failing with error message "Failed to install OLM version "latest": deployment/olm-operator failed to rollout: timed out waiting for the condition " HOT 2
- Install Plan creation with permissions HOT 7
- Operator Group [It] insufficient permissions resolve via RBAC HOT 1
- Channel missing from installed/dangling CSV; subscription complaining about csv that it doesn't own
- Readiness probe errored HOT 4
- Establish behavior for installed content no longer present in the catalog
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from operator-lifecycle-manager.