Code Monkey home page Code Monkey logo

poolboy's Introduction

Poolboy

Poolboy Icon

Poolboy provides a structure to manage OpenShift resources by allowing users to request resources which are then satisfied by resource providers and from pre-provisioned resource pools. This allows for users to manage resources in a cluster for which they do not have direct access through Kubernetes role-based access controls.

Poolboy Operation

Poolboy Diagram

ResourceClaim Creation

  1. User creates a ResourceClaim which contain a list of resource templates to defines desired resources.

  2. The ResourceClaim is matched to a ResourceProvider.

  3. The ResourceClaim’s resource templates are updated with defaults as defined by the ResourceProvider.

  4. Each resource template in the ResourceClaim is checked for validity against the OpenAPIv3 schema in the ResourceProvider.

  5. A ResourceHandle from a ResourcePool is matched to the ResourceClaim or a new ResourceHandle is created to satisfy the claim:

    1. To match an existing ResourceHandle each resource template in the ResourceClaim must match the ResourceHandle, excepting any fields specified in the ResourceProvider’s spec.matchIgnore.

    2. If created automatically, the ResourceHandle resource templates will be copied from the ResourceClaim.

ResourceClaim Update

  1. User updates an existing ResourceClaim

  2. Each resource template in the ResourceClaim is checked for validity against the OpenAPIv3 schema in the ResourceProvider.

  3. A JSON Patch for each resource template in the ResourceHandle is generated and then the JSON Patch is filtered according to each ResourceProvider’s spec.updateFilters.

  4. Updates are applied to the ResourceHandle.

ResourceHandle Creation or Update

  1. For each resource template in the ResourceHandle a resource definition is generated.

    1. ResourceProvider spec.override is applied to each resource template.

    2. The metadata.name is determined for each templated resource to match the random suffix appended to ResourceHandle.

  2. Create or update the resource

    1. If the resource does not exist, it is created.

    2. If the resource exists then it is updated.

    3. The state of the resource is copied into the ResourceClaim status if a ResourceClaim is bound to the ResourceHandle.

Templating

Templates are supported for ResourceProviders to supply defaults for ResourceClaims and for overrides for resources. Templates are enable for a ResourceProvider by specifying spec.template.enable on the ResourceProvider. Templates use Jinja2.

For ResourceClaim defaults templates may reference:

  • resource_claim - ResourceClaim object

  • resource_index - Resource index in the ResourceClaim spec.resources

  • resource_provider - ResourceProvider object

For resource overrides templates may reference:

  • resource_claim - ResourceClaim object if a ResourceClaim is bound

  • resource_handle - ResourceHandle object

  • resource_index - Resource index in the ResourceHandle spec.resources

  • resource_provider - ResourceProvider object

  • resource_template - Resource template from ResourceHandle

  • requester_identity - OpenShift identity of requester determined from ResourceClaim namespace’s openshift.io/requester annotation

  • requester_user - OpenShift user of requester determined from ResourceClaim namespace’s openshift.io/requester annotation

Normally Poolboy creates resources from ResourcePools as soon as the ResourceHandle is created. If the resource needs to reference resource_claim then the ResourceProvider may specify spec.resourceRequiresClaim to delay resource creation until the ResourceHandle is bound by a ResourceClaim.

Additionally templates may make use of variables timestamp and timedelta for date and time calculations. Examples:

  • timestamp("2021-05-01T12:30:00Z") - Parsed timestamp

  • timestamp("2021-05-01T12:30:00Z").add("3h") - Interval from parsed timestamp

  • timestamp.utcnow - Current UTC timestamp in %Y-%m-%dT%H:%M:%SZ format

  • timestamp.utcnow.add("1d") - UTC timestamp for this time tomorrow

  • timestamp.utcnow.datetime - Python datetime object for UTC now

  • timedelta("10m") - Representation of time delta for ten minutes.

  • timedelta("10m").timedelta - Python datetime timedelta

Lifespan

By default no lifespan policy is applied to Poolboy resources.

Lifespan is configured in ResourceHandles by specifying:

  • default - Default lifespan to apply to ResourceHandle when it is claimed.

  • maximum - Maximum lifespan which may be requested in the ResourceClaim calculated from the ResourceClaim the creation timestamp.

  • relativeMaximum - Maximum lifespan which can be requested in the ResourceClaim relative to the present datetime.

If both maximum and relativeMaximum are specified then the effective maximum is whichever is earlier. The calculated lifespan end set in the ResourceHandle lifespan.

The ResourceClaim can specify a lifespan end which will propagate to the ResourceHandle so long as it is within the maximum limits.

ResourcePools can specify lifespan configuration for resource handles they create. In addition to default, maximum, and relativeMaximum the ResourcePool can specify unclaimed to specify the lifespan of unclaimed ResourceHandles in the pool.

ResourceHandles created dynamically for ResourceClaims get their lifespan configuration from the ResourceProviders. If multiple ResourceProviders are used for a ResourceClaim then the minimum of each of the lifespan configuration options is applied to the ResourceHandle.

Use Case - Project Babylon Anarchy Operator

Poolboy was designed to manage custom resource types for the Anarchy operator framework. The Anarchy Operator orchestrates API calls, tracking the state of remote resources in custom resource kinds. As part of Project Babylon, Anarchy is configured to manage cloud resources and environments. Poolboy allows for these cloud environments to be pre-provisioned and then claimed by users.

In Anarchy a cloud environment is represented by the AnarchySubject custom resource kind.

Pre-Provisioned Environments

ResourcePools allow AnarychSubjects to be pre-created. The scale of a ResourcePool can be adjusted to add or reduce capacity based on expected demand. ResourceProvider validation and overrides allow for environments to be created on demand while retaining control on what environments can be requested.

User Management of Environments

Because the resource status is monitored and synced to the ResourceClaim status, users are able to track the state of their provisioned environments.

Each AnarchySubject has a spec.desiredState. The ResourceProvider spec.updateFilters allow this field to be updated while the spec.validation OpenAPIv3 check enforces that it can only be set to explicitly permitted values.

Installation

Poolboy Install

Install from helm template:

helm template poolboy helm/ | oc apply -f -

Build

OpenShift Build

  1. Process OpenShift build template to create BuildConfig and ImageStream

    oc process --local -f build-template.yaml | oc apply -n poolboy -f -
  2. Build poolboy image

    oc start-build poolboy -n poolboy --from-dir=. --follow
  3. Deploy Poolboy from build image

    helm template poolboy helm/ \
    --set=image.tagOverride=- \
    --set=image.repository=$(oc get imagestream poolboy -o jsonpath='{.status.tags[?(@.tag=="latest")].items[0].dockerImageReference}') \
    | oc apply -f -

Credits

Poolboy logo is original art by Lara Ditkoff

poolboy's People

Contributors

dependabot[bot] avatar fridim avatar jkupferer avatar pabrahamsson avatar tylerauerbeck avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

poolboy's Issues

Patch calculation bug in checking allowed operations

The python jsonpatch module calculates diffs can produce move operations while the poolboy design expects only add, remove, and replace to be used when processing allowed update filters.

Poolboy should stop using the jsonpatch module in favor of calculating json patches directly so that we can be sure the patches match expectations.

Resource names respect long unique suffix

The unique id, "guid", is considered the generated portion of the ResourceHandle name. Currently poolboy expects that the generated portion is always five characters long.

Poolboy should use metadata.generateName to determine the unique suffix rather than have a length of five characters hard-coded.

Anarchy Runner denied accessing Poolboy resources

Overview

Anarchy Runner is logging permission denied events when attempting to access Poolboy ResourceClaims and ResourceHandles.

I had (possibly incorrectly) assumed this might have been part of the Poolboy to Anarchy integration which is enabled with the following helm chart values when deploying Poolboy.

anarchy:
  create: true

Example errors

# Example read of ResourceHandle
lodestar-babylon-operators/anarchy-runner-default-hfpvf[runner]: fatal: [localhost]: FAILED! => {"changed": false, "error": 403, "msg": "Failed to retrieve requested object: b'{\"kind\":\"Status\",\"apiVersion\":\"v1\",\"metadata\":{},\"status\":\"Failure\",\"message\":\"resourcehandles.poolboy.gpte.redhat.com \\\\\"guid-wxd2j\\\\\" is forbidden: User \\\\\"system:serviceaccount:lodestar-babylon-operators:anarchy-runner-default\\\\\" cannot get resource \\\\\"resourcehandles\\\\\" in API group \\\\\"poolboy.gpte.redhat.com\\\\\" in the namespace \\\\\"lodestar-babylon-operators\\\\\"\",\"reason\":\"Forbidden\",\"details\":{\"name\":\"guid-wxd2j\",\"group\":\"poolboy.gpte.redhat.com\",\"kind\":\"resourcehandles\"},\"code\":403}\\n'", "reason": "Forbidden", "status": 403}

# Example delete of ResourceClaim
lodestar-babylon-operators/anarchy-runner-default-hfpvf[runner]: ...ignoring
lodestar-babylon-operators/anarchy-runner-default-hfpvf[runner]:
lodestar-babylon-operators/anarchy-runner-default-hfpvf[runner]: TASK [babylon_anarchy_governor : Delete resource claim] ************************
lodestar-babylon-operators/anarchy-runner-default-hfpvf[runner]: fatal: [localhost]: FAILED! => {"changed": false, "error": 403, "msg": "Failed to retrieve requested object: b'{\"kind\":\"Status\",\"apiVersion\":\"v1\",\"metadata\":{},\"status\":\"Failure\",\"message\":\"resourceclaims.poolboy.gpte.redhat.com \\\\\"do500.day2.idm-22155\\\\\" is forbidden: User \\\\\"system:serviceaccount:lodestar-babylon-operators:anarchy-runner-default\\\\\" cannot get resource \\\\\"resourceclaims\\\\\" in API group \\\\\"poolboy.gpte.redhat.com\\\\\" in the namespace \\\\\"lodestar-babylon-operators\\\\\"\",\"reason\":\"Forbidden\",\"details\":{\"name\":\"do500.day2.idm-22155\",\"group\":\"poolboy.gpte.redhat.com\",\"kind\":\"resourceclaims\"},\"code\":403}\\n'", "reason": "Forbidden", "status": 403}

Versions

The current versions of the Babylon Operators are being used in this test.

  • Babylon Operators v0.11.5
  • AgnosticV Operator v0.15.4
  • AgnosticD v1.0.9
  • Anarchy v0.16.25
  • Anarchy Governor v0.9.3
  • Poolboy v0.10.8

Questions

  • Am I missing these permissions already being deployed from another Helm chart and a fix is not required?

  • What is the minimum verbs Anarchy Runner requires for the Runner to access Poolboy CRDs to function? (Currently I have granted all in tests)

  • Is there an existing Group which contains all Runner service accounts that can be used in place of anarchy-runner-default in case additional runners are used in future?

PR

Please see PR #63 for the current status of testing a fix.

Use load_incluster_config

The config setup when running containerized in kubernetes would be much simpler with load_incluster_config.

Poolboy watch loop error

Traceback (most recent call last):
  File "/operator/gpte/kubeoperative.py", line 100, in watch_loop
    self.watch()
  File "/operator/gpte/kubeoperative.py", line 120, in watch
    raise Exception("Watch failure: " + event_obj['message'])
Exception: Watch failure: too old resource version: 97758 (474398)
[2020-01-09 18:19:03,239] operator             [ERROR   ] Error in watch_loop: Watch failure: too old resource version: 97758 (474398)
Traceback (most recent call last):
  File "/operator/gpte/kubeoperative.py", line 100, in watch_loop
    self.watch()
  File "/operator/gpte/kubeoperative.py", line 120, in watch
    raise Exception("Watch failure: " + event_obj['message'])
Exception: Watch failure: too old resource version: 97758 (474398)

Use labels to identify ResourceClaim/Handle/Pool on created resources

There are currently annotations on created resources, but not labels. The values that are annotations that should likely be labels are:

      poolboy.gpte.redhat.com/resource-claim-name: ...
      poolboy.gpte.redhat.com/resource-claim-namespace: ...
      poolboy.gpte.redhat.com/resource-handle-name: ...
      poolboy.gpte.redhat.com/resource-handle-namespace: ...
      poolboy.gpte.redhat.com/resource-provider-name: ...
      poolboy.gpte.redhat.com/resource-provider-namespace: ...

We would need to continue to set both labels and annotations for several releases when the labels are added.

Implement custom validation checks

Custom validation checks should process using jinja2 to validate check conditions:

spec:
  validation:
    customValidation:
    - name: destroy schedule must be no more than 14 days from creation
      check: spec.actionSchedule.destroy < timestamp(resource_claim.metadata.creationTimestamp).add('14d')
    - name: stop schedule must be no more than 8 hours from now
      check: spec.actionSchedule.stop < timestamp.utcnow.add('8h')

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.