Comments (14)
@DarkBlaez @resouer Tried again with a new allinone cluster, rook.io works properly after changing the frakti's default memory to 256MB. The full step is here: https://gist.github.com/feiskyer/b023d703355753878053cedf1fa4b538.
from frakti.
I've resolved this by editing rook-operator.yaml and adding a securityContext of privileged to the container section. Am I correct that in doing so this then has Frakti run with dockerd instead of hyperd?
from frakti.
@DarkBlaez Yep, priviledged containers are running in docker. For Rook.io problem, will figure out what's wrong.
from frakti.
For Rook.io I was able to get the rook-operator to run by setting a securityContext to privileged: true. Now I'm just trying to solve getting the next step, rook-cluster to run
from frakti.
rook-operator failed because of OOM, after increasing memory, it could start normally:
# cat rook-operator.yaml
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: rook-operator
rules:
- apiGroups:
- ""
resources:
- namespaces
- serviceaccounts
- secrets
- pods
- services
- nodes
- configmaps
- events
- persistentvolumes
- persistentvolumeclaims
verbs:
- get
- list
- watch
- create
- update
- delete
- apiGroups:
- extensions
resources:
- thirdpartyresources
- deployments
- daemonsets
- replicasets
verbs:
- get
- list
- watch
- create
- delete
- apiGroups:
- rbac.authorization.k8s.io
resources:
- clusterroles
- clusterrolebindings
verbs:
- get
- list
- watch
- create
- update
- delete
- apiGroups:
- storage.k8s.io
resources:
- storageclasses
verbs:
- get
- list
- watch
- delete
- apiGroups:
- rook.io
resources:
- "*"
verbs:
- "*"
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: rook-operator
namespace: default
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: rook-operator
namespace: default
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: rook-operator
subjects:
- kind: ServiceAccount
name: rook-operator
namespace: default
---
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: rook-operator
namespace: default
spec:
replicas: 1
template:
metadata:
labels:
name: rook-operator
spec:
serviceAccountName: rook-operator
containers:
- name: rook-operator
image: quay.io/rook/rookd:master-latest
resources:
limits:
memory: 512Mi
args: ["operator"]
env:
- name: ROOKD_REPO_PREFIX
value: quay.io/rook
But there are more problems, you can't change the memory for pods mon0, mon1, mon2
provisioned by rook operator. Kubernetes doesn't support this because of many reasons.
from frakti.
Added #152 which could help with the issue. After setting default memory to 512MB, rook.io works properly.
from frakti.
How would this be manifested once the change is in to allow the default memory and cpu to be configurable. This could vary based on POD or at the container level. Is this something that would be picked up from the yaml or would this have to be configured at the service level for configuring hyperd?
Along with this does hyperd/frakti report metrics back to insure kubernetes is aware of allocations. I think under docker since the pool of resources being utilized comes from the host this is known. If each hyperd spun up grabs memory is this conveyed to Kubernetes This comes in to play with autoscaling or placement of pods, etc. Just curious
Thanks
DB
from frakti.
@DarkBlaez frakti will always pick up information from Pod level cgroup, so the resource boundary of your pod VM is consistent with Kubernetes's resource model, nothing changed.
What #152 solves is: when Pod does not have resource limit set, then this Pod is classified as BestEffort (BE) in k8s. (some Burstable Pods without any limit set also drop into this case)
And in Hyper case, BE pod is not allowed (VM should always have resource limit), that's why we need to set a default resource for BE in frakti.
Please note BE pod will be evicted immediately when system resource is low and is not recommended to run your core business workloads.
btw, that's why we recommend user to run BE pod in Docker runtime by using mixed mode of frakti.
from frakti.
Thank you for explaining. That makes sense to me now. Thanks again
from frakti.
Is this issue something that can be resolved in frakti or should this be run as privileged?
from frakti.
It's already been fixed in frakti by #152 I think
from frakti.
Ok thanks. I will pull latest code and build so I can test further.
from frakti.
Even with the change in frakti defining defaults and allowing overrides with cpu/mem, rook-cluster fails (mon0, mon1, mon2, rook-api). rook osd runs fine. I can get this to run under kubernetes without frakti integration. I just wonder what the constraints are using frakti since in most cases one would just want to launch pods and let the backend deploy, manage, in a secure manner. Thoughts on what would be causing this not to run in a hyperd/frakti cluster?
from frakti.
The issue should be already fixed, closing it.
from frakti.
Related Issues (20)
- Implement ImageStats API HOT 7
- hyperd api field update
- Upgrade cri to v1alpha2 HOT 1
- finished with errors: map[addInterface:allocate device failed] HOT 1
- kubelet node register has failed HOT 4
- kubelet Failed create pod sandbox got wiretype 0, want 2 HOT 2
- k8s can't exec to pod, unknown runtime: remote HOT 8
- Network in POD doesn't work fine HOT 8
- Upgrade dockershim to 1.10 HOT 9
- [GSoC] Give more technical inside about containerd plugin for kata-container runtime HOT 6
- [GSoC] Some questions about "containerd-kata" HOT 4
- [GSoC] Which Category to choose for this GSoC project HOT 1
- Deprecate dockershim in vendor and change to use new containerd based architecture HOT 10
- Create a SECURITY_CONTACTS file. HOT 2
- Hyperd: got wiretype 0, want 2 HOT 11
- Update frakti to 1.11 HOT 1
- Use tide for PR merging HOT 1
- Upgrade frakti to Kubernetes v1.12 HOT 5
- frakti permissions cleanup HOT 13
- Snap package suppport HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from frakti.