Right now we have “internal” events that are fired when a project or a service is created or deleted, e.g., project.create.triggered, project.create.started and project.create.finished.

Observation 1: We never fire a project.create.triggered, but shipyard-controller sends a project.create.started and .finished (I guess the same is true for services).
This means that it is not possible for any other service to react on a project.create.triggered.

Observation 2: It seems that to counteract my first observation, some services listen on the *.create.finished events. This is suboptimal, as they sometimes event send another *.create.finished event.
For instance, helm-service sends a second service.create.finished (the other one is from shipyard-controller)

Suggested fix:
Provide post-*-create cloud-events, e.g.:
post-project-create.triggered
post-service-create.triggered

Question: Considering all our CloudEvent naming (e.g., test, deployment, evaluation), should get-sli be renamed to something that aligns with the other names, e.g., sli-retrieval?

In 0.1.7 we still had evaluation.invalidated: https://github.com/keptn/spec/blob/0.1.7/cloudevents.md#evaluation-invalidated

This is missing in the 0.2.0-alpha spec.

We should introduce this CloudEvent again, but using our new CloudEvents spec, e.g.:

sh.keptn.event.invalidate-evaluation.(triggered/started/finished)

with a similar payload than before.

add "event type/severity" field to sh.keptn.event.problem.open and sh.keptn.event.problem cloud events

In helm-service of Keptn 0.8.0-rc1 I noticed that we are sending deploymentNames within the deployment.finished event:

{
  "data": {
    "deployment": {
      "deploymentNames": [
        "canary"
      ],
      "deploymentURIsLocal": [
        "http://tempberry-backend.tempberry-hardening:80"
      ],
      "deploymentURIsPublic": [
        "http://tempberry-backend.tempberry-hardening.1-2-3-4.nip.io:80"
      ],
      "deploymentstrategy": "duplicate",
      "gitCommit": "bb31a5c6a65a0ca6f42081ce1fe1099dadba697d"
    },
    "message": "Successfully deployed",
    "project": "tempberry",
    "result": "pass",
    "service": "tempberry-backend",
    "stage": "hardening",
    "status": "succeeded"
  },
  "source": "helm-service",
  "type": "sh.keptn.event.deployment.finished",
}

The information provided by helm-service in this Array is either "canary" or "direct" and is consumed by SLI providers to determine whether the SLI is fetched for the primary or canary deployment (e.g., in case of blue-green deployments).
In addition, we refer to deployment in helm-charts (for dynatrace integration):
https://github.com/keptn/examples/blob/519680b56e270e9a59b09d864415df3d45643a09/onboarding-carts/carts/templates/deployment.yaml#L35-L36

In Keptn 0.7.x this field was called "deployment", e.g.:

{
  "contenttype": "application/json",
  "data": {
    "customFilters": [],
    "deployment": "canary",
    "deploymentstrategy": "blue_green_service",
    "end": "2020-12-04T07:00:54Z",
    "indicators": [
      "response_time_p95"
    ],
    "project": "ck-sockshop",
    "service": "carts",
    "sliProvider": "dynatrace",
    "stage": "staging",
    "start": "2020-12-04T06:59:00Z",
    "teststrategy": "performance"
  },
  "source": "lighthouse-service",
  "type": "sh.keptn.internal.event.get-sli",
}

See also helm-service#deployment-finished.

Questions / Problems

• I did not expect "direct" nor "canary" for DeploymentNames. Coming from helm-service, I would have expected the actual name of the release, helm-chart or Kubernetes Deployment (e.g., tempberry-primary).
• Why is it an array? Do we consume it as an array anywhere (e.g., sli provider) or do we always consume the value at index 0?
• The respective attribute is not visible/document in keptn/spec, therefore writing an integration that consumes this attribute is impossible unless you have profound knowledge of helm-service.

task properties are denotes as:

- name: "task1"
  properties:
    foo: bar

A task can be "triggered after" a certain amount of time like this:

- name: "task1"
  triggeredAfter: 10m

Why the difference? Why not streamline and list as just another property:

- name: "task1"
  properties:
    foo: bar
    triggeredAfter: 10m

Same idea for sequences being "triggered on" other events. Currently the syntax is:

- name: sequence1
  triggeredOn:
    - event: "dev.delivery.finished"

Why not:

- name: sequence1
  properties:
    triggeredOn:
      event: dev.delivery.finished

When reading CloudEvents spec, it is not clear what gitCommit is and why I need it.

gitCommit appears in multiple cloudevents, e.g., deployment (here it actually makes sense, if we consider our helm-service), but also evaluation, as well as also action (e.g., action.finished).

To be clear, I know that gitCommit means it should be a commit hash of a git commit, but of "what" (and "why")?
The Git config repo? What if I don't have one (e.g., unleash-service is an action-provider and does not need to have access to a Git repository at all)?

Also, what is the added benefit of having a gitCommit in certain cloud-events (but not all)?

In Keptn 0.7.x and spec 0.1.7 as well as Keptn 0.8.0-alpha and spec 0.2.0-alpha we are still using the old Problem / Problem.Open CloudEvents.

Those are most certainly not compatible with our new .triggered/.started/.finished structure. In addition, in our spec we only list problem.open, but no longer “just” problem (which is sent by Dynatrace to our API).

Suggested fix (to be discussed):
Right now the workflow is a little bit weird. Dynatrace sends a Problem Notification to Keptn API, which is processed as if it was a CloudEvent, and then Service parses this and sends a Problem.Open CloudEvent.

For one, Dynatrace could try to communicate directly via Dynatrace-Service (and not via Keptn API) to send the Problem Event (though this would require Dynatrace-Service to be reachable from the outside, which it currently is not).

Second, Problem.Open CloudEvents could be streamlined to use .triggered/.started/.finished - or maybe it should even just be .problem.triggered (without the .open)?

As a tradeoff, we could do the following:
Dynatrace keeps sending a sh.keptn.events.problem to stay backwards compatible (we can think of introducing sh.keptn.events.problem.triggered). We need to define this event in our CloudEvents spec.
Dynatrace-Service reacts on this event, and sends a problem.open.triggered (we need to introduce problem.open.triggered).
Remediation-Service listens on problem.open.triggered and reacts on it with sending appropriate .started and .finished events
To be discussed: How do we handle problem close/resolve events from Dynatrace?

In v0.15.0, I'm getting breaking validation errors.

name: invalid response
shortmsg: |-
  properties: /data/events/0/data/evaluation/indicatorResults/0/value/comparedValue: must NOT have additional properties
  properties: /data/events/0/data/evaluation/indicatorResults/1/value/comparedValue: must NOT have additional properties
  properties: /data/events/0/data/evaluation: must have property 'gitCommit'
info:
  request:
    baseURL: https://[REDACTED]/api/
    headers:
      x-token: ***
    method: GET
    url: mongodb-datastore/event
    params:
      type: sh.keptn.event.evaluation.finished
      keptnContext: b204cf07-5973-470a-8f63-7e44e1bd4d35
  response:
    status: 200
    statusText: OK
    headers:
      server: istio-envoy
      date: Sat, 14 May 2022 17:34:30 GMT
      content-type: application/json
      x-envoy-upstream-service-time: '11'
      transfer-encoding: chunked
    data:
      events:
      - data:
          evaluation:
            indicatorResults:
            - displayName: ''
              keySli: true
              passTargets:
              - criteria: <2000
                targetValue: 2000
                violated: false
              score: 20
              status: pass
              value:
                comparedValue: 0
                metric: avg_responsetime
                success: true
                value: 1.5889433962264152
              warningTargets:
              - criteria: <1000
                targetValue: 1000
                violated: false
            - displayName: ''
              keySli: true
              passTargets:
              - criteria: <1
                targetValue: 1
                violated: false
              score: 20
              status: pass
              value:
                comparedValue: 0
                metric: error_rate
                success: true
                value: 0
              warningTargets:
              - criteria: <2
                targetValue: 2
                violated: false
            result: pass
            score: 100
            sloFileContent: c3BlY192ZXJzaW9uOiAnMS4wJwpvYmplY3RpdmVzOgotIHNsaTogYXZnX3Jlc3BvbnNldGltZQogIHBhc3M6CiAgLSBjcml0ZXJpYToKICAgIC0gPDIwMDAKICB3YXJuaW5nOgogIC0gY3JpdGVyaWE6CiAgICAtIDwxMDAwCiAgd2VpZ2h0OiAyMAogIGtleV9zbGk6IHRydWUKLSBzbGk6IGVycm9yX3JhdGUKICBwYXNzOgogIC0gY3JpdGVyaWE6CiAgICAtIDwxCiAgd2FybmluZzoKICAtIGNyaXRlcmlhOgogICAgLSA8MgogIHdlaWdodDogMjAKICBrZXlfc2xpOiB0cnVlCnRvdGFsX3Njb3JlOgogIHBhc3M6IDkwJQogIHdhcm5pbmc6IDYwJQpjb21wYXJpc29uOgogIGNvbXBhcmVfd2l0aDogc2V2ZXJhbF9yZXN1bHRzCiAgaW5jbHVkZV9yZXN1bHRfd2l0aF9zY29yZTogcGFzc19vcl93YXJuCiAgbnVtYmVyX29mX2NvbXBhcmlzb25fcmVzdWx0czogMQo=
            timeEnd: '2022-05-14T17:31:04.360Z'
            timeStart: '2022-05-14T17:30:04.360Z'
          labels:
            DtCreds: dynatrace-np
            appServiceId: appsvc0
            buildId: '20115'
            definition: EMF.Keptn.Sandbox
            dtEntityName0: EMFSandbox_Blue
            dtEntityWebUrl0: https://[REDACTED]/e/54d139b9-23ee-466d-ad7a-f98e0bd8bdf4/ui/nav/SERVICE-A5D15D718DF71792
            pipeline: https://dev.azure.com/[REDACTED]/6b945a9d-1349-4d3c-9858-0b57fcad3e9e/_build/results?buildId=20115
            runby: Microsoft.VisualStudio.Services.TFS
          project: devops
          result: pass
          service: emf.keptn.sandbox
          stage: stage
          status: succeeded
        id: 6c3b309f-91b0-4ae0-92b8-c789537b7101
        shkeptncontext: b204cf07-5973-470a-8f63-7e44e1bd4d35
        shkeptnspecversion: 0.2.4
        source: lighthouse-service
        specversion: '1.0'
        time: '2022-05-14T17:33:04.769Z'
        triggeredid: 1126c26a-d70d-48ee-be37-c04df27c6016
        type: sh.keptn.event.evaluation.finished
      pageSize: 20
      totalCount: 1
  errors:
  - instancePath: /data/events/0/data/evaluation/indicatorResults/0/value/comparedValue
    schemaPath: /definitions/indicatorResult/properties/value
    keyword: properties
    params:
      error: additional
      additionalProperty: comparedValue
    message: must NOT have additional properties
  - instancePath: /data/events/0/data/evaluation/indicatorResults/1/value/comparedValue
    schemaPath: /definitions/indicatorResult/properties/value
    keyword: properties
    params:
      error: additional
      additionalProperty: comparedValue
    message: must NOT have additional properties
  - instancePath: /data/events/0/data/evaluation
    schemaPath: /definitions/evaluation/properties/gitCommit
    keyword: properties
    params:
      error: missing
      missingProperty: gitCommit
    message: must have property 'gitCommit'

These PRs seem to conflict with schema lines.

• keptn/go-utils#430
  

  spec/cloudevents.md

  Line 30 in f6f1e1f

  "gitcommitid",

• keptn/go-utils#358
  

  spec/cloudevents.md

  Line 2769 in f6f1e1f

  "additionalProperties": false,

Could CI/CD check for compatibility breaks between design & generated schemas?
There are various tools to do that:

• https://github.com/tufin/oasdiff
• https://github.com/OpenAPITools/openapi-diff
• https://bitbucket.org/atlassian/openapi-diff

It is possible to interpret the cloudevents spec file and build a payload for a mysequence.triggered event, but it's not quick.

Add a new subsection for custom events.

During this, make it clear which parts the shipyard-controller will automatically add on your behalf and which are mandatory:

For example, AFAIK this is a minimal working example of a cloudevent that can be ingested and used:

{
  "specversion": "1.0",
  "contenttype": "application/json",
  "data": {
      "project": "<projectNameHere>",
      "service": "<serviceNameHere>",
      "stage": "<stageNameHere>"
  },
  "type": "sh.keptn.event.<stageNameHere>.<sequenceNameHere>.triggered",
  "source": "<yourSource>"
}

This spec change is based on the KEP 0005
https://github.com/keptn/enhancement-proposals/blob/master/text/0005-deployment-finished-with-deployed-endpointurl.md

The goal is to avoid having jmeter guestimate the URI of the deployed service.

Follow-up of keptn/keptn#2449

we need to add the cloud-event evaluation.invalidated to the spec

NOTE: This was changed from spec 0.2.1 to 0.2.2 and the issue description might be outdated

In the 0.2.0-alpha spec, the CloudEvent action.triggered contains name, action, description and value.

Based on the spec for remediation.yaml I have to say that action and value make perfect sense to me:
https://github.com/keptn/spec/blob/master/remediation.md#example-of-a-remediation-in-yaml

Though why do we need to send all 4 attributes with an action.triggered event?

Also, what happens if I send something else in there? Is it validated? Will it fail?

Counter-proposal:

• Only send action and value

Reasoning:

• If I ever send an action.triggered cloud-event (e.g., through Keptn Bridge or via CLI), I don't want to need to specify name and description - implementing this properly would actually require reading remediation.yaml and processing name and description before sending the CloudEvent
• remediation.yaml should be independent from the action-provider that reacts on the action.triggered event. The action-provider should only need to know what action it performs, and what values it needs for this action.
• name and description of a remediation action are specific to the problem/remediation workflow, not the actual action performed.

In keptn/keptn#2388 (PR keptn/keptn#2394 ) we have introduced a new field in the evaluation-done cloud-event that contains all evaluation-done events that we compare with.

This needs to be reflected in the spec: https://github.com/keptn/spec/blob/master/cloudevents.md#evaluation-done

When just reading the CloudEvents spec, it is not clear what the difference between result and status is.

This needs to be written down.

Currently, the Keptn Cloudevent Spec page is created manually.
Due to the error-prone nature of this process, it is very hard to keep the JSON examples and JSON schema descriptions in sync and up to date.

Task:

The go structs representing the cloud events (mainly in keptn/go-utils) shall be used to auto-generate JSON schema files and JSON examples which can then be used in the keptn spec for our cloud events. Note that it might be necessary to add additional metadata to the existing go structs.

Initial work on generating the cloudevents.md file has been done here There are some Tasks left:

• Add jsonschema tags to the golang structs to make the generated JSON schema more descriptive (add type information, enums, quantities, etc...)
• Fix keptn/keptn#2830 which causes the generated schema generation to output weird $ref fields (see json schema for EvaluationTriggeredEventData
• Move code to one of the keptn repositories
• Nice to have: trigger the generation of the cloudevents.md file automatically via a GH action.

My proposal:

If we want to trigger artifact delivery, we should only deploy some data, e.g.:

keptn trigger artifact-delivery --deployment.image=docker.io/keptnexamples/carts --deployment.tag=0.11.3

which should result in a cloud-event that looks like this (simplified cloud-event):

{
    "type": "artifact-delivery",
    "data": {
        "deployment": {
               "image": "docker.io/keptnexamples/carts",
               "tag": "0.11.3"
         }
    }
}

And then shipcard-controller will send a deploy.triggered with the same data.

@bacherfl could you please add the resulting cloud-events of our current implementation (master branch) as a comment?

Before we release Keptn 0.8.0 we need to double-check every cloud-event and every payload attribute

• whether it is needed
• whether it is used (currently)
• whether it is documented properly within the spec

The remediation config uses the property apiVersion to declare the version of this spec file, has "kind", "metadata", and "spec": https://github.com/keptn/spec/blob/master/sre.md#remediation

When modifying the spec of shipyard, sli, or slo please change the property spec_version to apiVersion with the value: apiVersion: spec.keptn.sh/x.y.z, and add kind (= Shipyard, ServiceLevelIndicator, ServiceLevelObjective), metadata, and spec according to the example of the remedation spec.

Definition of Done:

• The specifications of shipyard, sli, or slo are aligned with the remedation spec

As a developer (that develops a Keptn integration) I want to be able to read the CloudEvents spec and understand what CloudEvents and Payloads I need to deal with.

Also, as a developer, I want to make sure that my Keptn-service validates the input based on my requirements, not based on what Keptn spec specifies (e.g., lighthouse-service requires a test and deployment payload for evaluation.triggered - but this should not be part of the evaluation.triggered event).

Current situation

In spec 0.2.0, several of our CloudEvents specify payloads of previous CloudEvents, e.g.:

• sh.keptn.event.deployment.triggered references configurationChange
• sh.keptn.event.test.triggered references deployment, but not configurationChange
• sh.keptn.event.evaluation.triggered references test and deployment, but not configurationChange
• sh.keptn.event.release.triggered references deployment, but nothing else
• sh.keptn.event.get-sli.triggered references nothing, but our implementation would actually require deployment and test and ...

On a site node, sh.keptn.event.get-sli.triggered is absolutely not Keptn CloudEvents 0.2.0 compatible... we are breaking current implementations of SLI providers, and we will need to break implementations again once we send get-sli.triggered from shipyard-controller - I'll leave that for another issue.

Desired situation

• We have a basic definition of how a Keptn Cloud Event looks like (I think we already have that: cloudevents.md#type, cloudevents.md#task-events, cloudevents.md#data
• For each task we have a concrete definition of the payload of the task, e.g., sh.keptn.evaluation.triggered should specify the actual evaluation payload (start and end)
• Every Keptn-service defines the desired .triggered CloudEvent (see example below)

Example: helm-service specifies deployment.triggered

go-utils

type DeploymentTriggeredEventData struct {
	EventData

	ConfigurationChange ConfigurationChange    `json:"configurationChange"`
	Deployment          DeploymentWithStrategy `json:"deployment"`
}

becomes

type DeploymentTriggeredEventData struct {
	EventData
	Deployment          DeploymentWithStrategy `json:"deployment"`
}

helm-service

Needs to either use the go-utils DeploymentTriggeredEventData or specify their own using an imported struct, e.g.:

type DeploymentTriggeredEventData struct {
	keptnv2.DeploymentTriggeredEventData
	ConfigurationChange ConfigurationChange    `json:"configurationChange"`
}

Example: lighthouse-service specifies evaluation.triggered

go-utils

type EvaluationTriggeredEventData struct {
	EventData
	Test       Test       `json:"test"`
	Evaluation Evaluation `json:"evaluation"`
	Deployment Deployment `json:"deployment"`
}

becomes

type EvaluationTriggeredEventData struct {
	EventData
	Evaluation Evaluation `json:"evaluation"`
}

lighthouse-service

Needs to either use the go-utils EvaluationTriggeredEventData or specify their own using an imported struct, e.g.:

type EvaluationTriggeredEventData struct {
	keptnv2.EvaluationTriggeredEventData
	Test       Test       `json:"test"`
	Deployment Deployment `json:"deployment"`
}

From discussion in #68

Goal: We need to add a deployment field to get-sli.triggered

Tasks

• Add deployment to get-sli.triggered in go-utils
• Merge go-utils in keptn core
• Generate spec using CLI keptn generate-spec and create a PR on the keptn/spec repo

The specification documentation still uses the property strategy for the deployment- and test-strategy. It should be deploymentstrategy and teststrategy.

Failing example:

- name: deployment
  properties:
    strategy: direct

should be

- name: deployment
  properties:
    deploymentstrategy: direct

In the https://github.com/keptn/spec/blob/master/cloudevents.md a section should be provided to explain the stage events fired by the shipyard-controller. Given the below example of a shipyard, the following stage events are fired:

• sh.keptn.event.dev.artifact-delivery.triggered
• sh.keptn.event.dev.artifact-delivery.finished
• sh.keptn.event.staging.artifact-delivery.triggered
• sh.keptn.event.staging.artifact-delivery.finished

apiVersion: "spec.keptn.sh/0.2.0"
kind: "Shipyard"
metadata:
  name: "shipyard-sockshop"
spec:
  stages:
    - name: "dev"
      sequences:
        - name: "artifact-delivery"
          tasks:
            - name: "deployment"
            - name: "test"
            - name: "evaluation"
            - name: "release"

    - name: "staging"
      sequences:
        - name: "artifact-delivery"
          triggers:
            - "dev.artifact-delivery.finished"
          tasks:
            - name: "deployment"
            - name: "test"
            - name: "evaluation"
            - name: "release"

Task: Provide documentation to explain:

• The pattern of a stage event follows this format: sh.keptn.event.{stage}.{seqeunece}.{triggered | finished}
• The result, status, and message properties represent the properties from the last successfully executed task in the sequence.

Is your feature request related to a problem? Please describe.
Ketpn JSON schemas are defined in the .md files. It would be nice if we create a folder and aggregate schema definition there.

Describe the solution you'd like
I am working on the issue #9452 related to publishing Keptn schemas in JSON Schemastore.
It is possible to publish self-hosted schemas in the Schemastore. If we host the file in keptn/spec repo then it will be easy to refer to the schemas in the schema catalog of the Schemastore without needing to create the files in the schemastore repo.

Describe alternatives you've considered
We can directly create JSON schemas in the schemastore repository. However In my opinion, it would be more maintainable if we define the schemas in keptn/spec repo and refer them in the catalog.

The ProblemDetails field in the cloudevents.md example shows "ProblemDetails": "Pod name", whereas it's actually passed like this:

"ProblemDetails":{"displayName":"641","endTime":-1,"hasRootCause":false,"id":"1234_5678V2","impactLevel":"SERVICE","severityLevel":"PERFORMANCE","startTime":1587624420000,"status":"OPEN"}

Does this SLO example contradict itself?

https://github.com/keptn/spec/blob/0.1.2/sre.md#service-level-indicators-sli-configuration

comparison:
  compare_with: "single_result"
  include_result_with_score: "pass"
  number_of_comparison_results: 3
  aggregate_function: avg

This comparison configuration means that the current result is only compared to the last result that passed.

Doesn't the compare_with: "single_result" actually negate the number_of_comparison_results?

Perhaps the example should be:

comparison:
  compare_with: "single_result"
  include_result_with_score: "pass"
  aggregate_function: avg

The time field is a mandatory field. however the example cloud events listed here do not contain this field, hence the examples are invalid.

Notes from a cloud event spec pov, time is optional, however, we are checking for existence of the field during validation: https://github.com/keptn/go-utils/blob/349cd94a73287f7151281c7da5cea973217e2491/pkg/api/models/keptn_context_extended_c_e.go#L65