dapr / dapr-shared Goto Github PK

View Code? Open in Web Editor NEW

47.0 47.0 9.0 1.2 MB

Dapr Shared enables users to deploy Dapr as a DaemonSet or a Deployment instead of a sidecar

Home Page: htttps://dapr.io

License: Apache License 2.0

Smarty 28.38% Go 66.08% Dockerfile 1.29% Makefile 4.24%

apis cloud-native dapr deployment strategy

dapr-shared's Introduction

Any language, any framework, anywhere

Dapr is a portable, serverless, event-driven runtime that makes it easy for developers to build resilient, stateless and stateful microservices that run on the cloud and edge and embraces the diversity of languages and developer frameworks.

Dapr codifies the best practices for building microservice applications into open, independent, building blocks that enable you to build portable applications with the language and framework of your choice. Each building block is independent and you can use one, some, or all of them in your application.

We are a Cloud Native Computing Foundation (CNCF) incubation project.

Goals

Enable developers using any language or framework to write distributed applications
Solve the hard problems developers face building microservice applications by providing best practice building blocks
Be community driven, open and vendor neutral
Gain new contributors
Provide consistency and portability through open APIs
Be platform agnostic across cloud and edge
Embrace extensibility and provide pluggable components without vendor lock-in
Enable IoT and edge scenarios by being highly performant and lightweight
Be incrementally adoptable from existing code, with no runtime dependency

How it works

Dapr injects a side-car (container or process) to each compute unit. The side-car interacts with event triggers and communicates with the compute unit via standard HTTP or gRPC protocols. This enables Dapr to support all existing and future programming languages without requiring you to import frameworks or libraries.

Dapr offers built-in state management, reliable messaging (at least once delivery), triggers and bindings through standard HTTP verbs or gRPC interfaces. This allows you to write stateless, stateful and actor-like services following the same programming paradigm. You can freely choose consistency model, threading model and message delivery patterns.

Dapr runs natively on Kubernetes, as a self hosted binary on your machine, on an IoT device, or as a container that can be injected into any system, in the cloud or on-premises.

Dapr uses pluggable component state stores and message buses such as Redis as well as gRPC to offer a wide range of communication methods, including direct dapr-to-dapr using gRPC and async Pub-Sub with guaranteed delivery and at-least-once semantics.

Why Dapr?

Writing highly performant, scalable and reliable distributed application is hard. Dapr brings proven patterns and practices to you. It unifies event-driven and actors semantics into a simple, consistent programming model. It supports all programming languages without framework lock-in. You are not exposed to low-level primitives such as threading, concurrency control, partitioning and scaling. Instead, you can write your code by implementing a simple web server using familiar web frameworks of your choice.

Dapr is flexible in threading and state consistency models. You can leverage multi-threading if you choose to, and you can choose among different consistency models. This flexibility enables you to implement advanced scenarios without artificial constraints. Dapr is unique because you can transition seamlessly between platforms and underlying implementations without rewriting your code.

Features

Event-driven Pub-Sub system with pluggable providers and at-least-once semantics
Input and output bindings with pluggable providers
State management with pluggable data stores
Consistent service-to-service discovery and invocation
Opt-in stateful models: Strong/Eventual consistency, First-write/Last-write wins
Cross platform virtual actors
Secret management to retrieve secrets from secure key vaults
Rate limiting
Built-in Observability support
Runs natively on Kubernetes using a dedicated Operator and CRDs
Supports all programming languages via HTTP and gRPC
Multi-Cloud, open components (bindings, pub-sub, state) from Azure, AWS, GCP
Runs anywhere, as a process or containerized
Lightweight (58MB binary, 4MB physical memory)
Runs as a sidecar - removes the need for special SDKs or libraries
Dedicated CLI - developer friendly experience with easy debugging
Clients for Java, .NET Core, Go, Javascript, Python, Rust and C++

Get Started using Dapr

See our Getting Started guide over in our docs.

Quickstarts and Samples

See the quickstarts repository for code examples that can help you get started with Dapr.
Explore additional samples in the Dapr samples repository.

Community

We want your contributions and suggestions! One of the easiest ways to contribute is to participate in discussions on the mailing list, chat on IM or the bi-weekly community calls. For more information on the community engagement, developer and contributing guidelines and more, head over to the Dapr community repo.

Contact Us

Reach out with any questions you may have and we'll make sure to answer them as soon as possible!

Platform	Link
💬 Instant Message Chat (preferred)
📧 Mailing List	https://groups.google.com/forum/#!forum/dapr-dev
🐤 Twitter	@daprdev

Community Call

Every two weeks we host a community call to showcase new features, review upcoming milestones, and engage in a Q&A. All are welcome!

📞 Visit https://aka.ms/dapr-community-call for upcoming dates and the meeting link.

Videos and Podcasts

We have a variety of keynotes, podcasts, and presentations available to reference and learn from.

📺 Visit https://docs.dapr.io/contributing/presentations/ for previous talks and slide decks.

Contributing to Dapr

See the Development Guide to get started with building and developing.

Repositories

Repo	Description
Dapr	The main repository that you are currently in. Contains the Dapr runtime code and overview documentation.
CLI	The Dapr CLI allows you to setup Dapr on your local dev machine or on a Kubernetes cluster, provides debugging support, launches and manages Dapr instances.
Docs	The documentation for Dapr.
Quickstarts	This repository contains a series of simple code samples that highlight the main Dapr capabilities.
Samples	This repository holds community maintained samples for various Dapr use cases.
Components-contrib	The purpose of components contrib is to provide open, community driven reusable components for building distributed applications.
Dashboard	General purpose dashboard for Dapr
Go-sdk	Dapr SDK for Go
Java-sdk	Dapr SDK for Java
JS-sdk	Dapr SDK for JavaScript
Python-sdk	Dapr SDK for Python
Dotnet-sdk	Dapr SDK for .NET
Rust-sdk	Dapr SDK for Rust
Cpp-sdk	Dapr SDK for C++
PHP-sdk	Dapr SDK for PHP

Code of Conduct

Please refer to our Dapr Community Code of Conduct

dapr-shared's People

Contributors

Stargazers

Watchers

Forkers

mcruzdev asarenkansah zeiss bibryam mikeee tpmccallum salaboy boularak-techs santode

dapr-shared's Issues

Remove Proxy Code and use new daprd app-channel flag

The daprd container now has a flag to proxy requests to external apps, so we need to use that instead of the proxy code that we have here.

We need to remove the app channel proxy code located here:

https://github.com/dapr-sandbox/dapr-ambient/blob/main/main.go#L91

Instead, we need to set up this new flag in the template:

https://github.com/search?q=org%3Adapr+app-channel-address&type=code

--app-channel-address

Tranform container with ambient-proxy to initContainer

The init container should only get the trust bundles (https://github.com/dapr-sandbox/dapr-ambient/blob/main/main.go#L50) and create a configMap with the variables.

The daprd container should load these variables from the configMap created by the initContainer, so we avoid packaging and extending the daprd container.

Change the namespace dapr-shared runs in?

I tried to install the helm chart using a different namespace. Everything installed, but it was not able to come up because it could not find the configmap.

'Error: configmap "dapr-shared-cm" not found'

I investigated and found that the configmap is in the default namespace and not with the deployment so it cannot come up.

Dapr pods do not initialize after re-deployment

Given a running configuration

NAME                                READY   STATUS    RESTARTS   AGE
distributor-7f7769c8f6-7wxhx        1/1     Running   0          35m
distributor-dapr-w968l              1/1     Running   0          35m
receiver-express-dapr-vlbp9         1/1     Running   0          20m
receiver-express-fb667dd79-tnn9g    1/1     Running   0          10m
receiver-standard-c8b8577b4-49c8j   1/1     Running   0          10m
receiver-standard-dapr-vrv5v        1/1     Running   0          20m

when trying to restart ambient Dapr deployment

kubectl rollout restart daemonset/receiver-express-dapr

pod does not initialize

NAME                                READY   STATUS       RESTARTS     AGE
distributor-7f7769c8f6-7wxhx        1/1     Running      0            37m
distributor-dapr-w968l              1/1     Running      0            37m
receiver-express-dapr-z7nhs         0/1     Init:Error   1 (4s ago)   5s
receiver-express-fb667dd79-tnn9g    1/1     Running      0            11m
receiver-standard-c8b8577b4-49c8j   1/1     Running      0            11m
receiver-standard-dapr-vrv5v        1/1     Running      0            22m

later

NAME                                READY   STATUS                  RESTARTS      AGE
distributor-7f7769c8f6-7wxhx        1/1     Running                 0             42m
distributor-dapr-w968l              1/1     Running                 0             42m
receiver-express-dapr-z7nhs         0/1     Init:CrashLoopBackOff   5 (98s ago)   4m37s
receiver-express-fb667dd79-tnn9g    1/1     Running                 0             16m
receiver-standard-c8b8577b4-49c8j   1/1     Running                 0             16m
receiver-standard-dapr-vrv5v        1/1     Running                 0             26m

pod events show

  Warning  BackOff    2m52s (x6 over 4m)    kubelet            Back-off restarting failed container ambient-init-container in pod receiver-express-dapr-z7nhs_default(4f4ae10b-c02a-43a1-9b0b-aab3b546ae64)

Rename dapr-ambient to dapr-shared

The name Ambient confuse people because Service Meshes have been using the term too, like Istio Ambient.
Rename to Dapr Shared

Update `daprd` mechanisms to get certs

I need to investigate more about how certs are set into daprd so I can remove the daprd/Dockefile needed for ambient work right now.

This needs to be removed:

export DAPR_TRUST_ANCHORS=`cat /shared/DAPR_TRUST_ANCHORS`;
            export DAPR_CERT_CHAIN=`cat /shared/DAPR_CERT_CHAIN`;
            export DAPR_CERT_KEY=`cat /shared/DAPR_CERT_KEY`;

From here: https://github.com/dapr-sandbox/dapr-ambient/blob/main/chart/dapr-ambient/templates/daemonset.yaml#L63

Variables that are written in here:
https://github.com/dapr-sandbox/dapr-ambient/blob/main/main.go#L55

Update the daprd version to v1.11.0

I think that we can change the release canditate version to v1.11.0

Package and distribute Dapr-Ambient resoruces

Create a multi-platform Docker container for the init-container
Helm chart package and host in dapr helm repository
Add section into the documentation

FIx issues with ConfigMap for Trust Bundles

There are two issues with the Trust Bundles configMap today:

Today the configMap name is hardcoded, making multiple installations of ambient to clash due the configMap name.
Use the Helm Release name to customize the name of the configmap. Here: https://github.com/dapr-sandbox/dapr-ambient/blob/main/chart/dapr-ambient/templates/_daemondeployshared.yaml#L23 . Use the args option to set the release name. Inside the init-container read the argument and use for the configmap name
The configMap is not deleted if the helm chart is deleted

Add support for strategy using deployments instead of daemonsets

Use a helm value to switch between Kubernetes Daemonsets and Kubernetes Deployments.

https://github.com/dapr-sandbox/dapr-ambient/blob/main/chart/dapr-ambient/templates/daemonset.yaml

Use Deployment with 1 replica by default

Add application health check settings

Add application health check settings to configuration options

I suggest adding this block to daprd section in configuration values

daprd:
    app: ...
    appHealth:
      # -- Enables the app health checks
      enabled: false
      # -- Path that Dapr invokes for health probes when the app channel is HTTP (this value is ignored if the app channel is using gRPC)
      checkPath: "/healthz"
      # -- Number of seconds between each health probe
      probeInterval: 5
      # -- Timeout in milliseconds for health probe requests
      probeTimeout: 500
      # -- Max number of consecutive failures before the app is considered unhealthy
      threshold: 3

please assign to me

Add Helm-docs to the chart

Use something like: https://github.com/norwoodj/helm-docs to generate docs

Add dapr-ambient sample with Knative

We have a tutorial on @salaboy's dapr-ambient repo.

The idea is to move the tutorial for this repo.

dapr-hook failed when uninstalling multiple releases at once

When uninstalling multiple releases at once, this error is logged

$ helm uninstall receiver-standard-dapr receiver-express-dapr distributor-dapr

release "receiver-standard-dapr" uninstalled
release "receiver-express-dapr" uninstalled
Error: 1 error occurred:
        * pod distributor-dapr-hook failed

looking into logs of the related -dapr-hook pod/container, it seems like a conflict in deletion of shared objects

$ kubectl logs pods/distributor-dapr-hook
2024/01/10 15:59:30 executing dapr-shared-init
panic: configmaps "distributor-dapr-cm" is forbidden: User "system:serviceaccount:default:distributor" cannot delete resource "configmaps" in API group "" in the namespace "default": RBAC: [clusterrole.rbac.authorization.k8s.io "dapr-shared-secret-reader" not found, clusterrole.rbac.authorization.k8s.io "dapr-shared-configmap-creator" not found]
Azure does not have opinion for this user.

goroutine 1 [running]:
main.RemoveHandler()
        /home/kai/src/dapr-shared/cmd/shared-init/main.go:148 +0xac
main.NewRemoveCmd.func1(0xc0003aa100?, {0x18d8cb1?, 0x4?, 0x18d8cb5?})
        /home/kai/src/dapr-shared/cmd/shared-init/main.go:96 +0xf
github.com/spf13/cobra.(*Command).execute(0xc00030ac00, {0xc0003842a0, 0x1, 0x1})
        /home/kai/.go/pkg/mod/github.com/spf13/[email protected]/command.go:944 +0x863
github.com/spf13/cobra.(*Command).ExecuteC(0xc00030a600)
        /home/kai/.go/pkg/mod/github.com/spf13/[email protected]/command.go:1068 +0x3a5
github.com/spf13/cobra.(*Command).Execute(...)
        /home/kai/.go/pkg/mod/github.com/spf13/[email protected]/command.go:992
main.main()
        /home/kai/src/dapr-shared/cmd/shared-init/main.go:31 +0x8b

be aware, that I already had to "trick" release installation into multiple instances by overriding fullnameOverride and especially shared.serviceAccount.name to already avoid conflicts while installing

  apps=("distributor" "receiver-express" "receiver-standard")

  for app in "${apps[@]}"
  do
    echo "$app"

    # fork/branch: https://github.com/ZEISS/dapr-shared/tree/recycle-configmap
    helm upgrade --install $app-dapr $REPO_ROOT/../dapr-shared/chart/dapr-shared/ \
      --set fullnameOverride=$app-dapr \
      --set shared.initContainer.image.registry=$AZURE_CONTAINER_REGISTRY_ENDPOINT \
      --set shared.strategy=deployment \
      --set shared.scheduling.nodeSelector.agentpool=backend \
      --set shared.deployment.replicas=3 \
      --set shared.daprd.image.tag=1.11.6 \
      --set shared.appId=$app \
      --set shared.remoteURL=$app-svc \
      --set shared.remotePort=80 \
      --set shared.serviceAccount.name=$app \
      --set shared.daprd.listenAddresses=0.0.0.0

  done

Add nodeSelector to chart values

To have a better control on which node pool etc. Dapr shared is placed, I'd like to have a setting for nodeSelector

Add gRPC support for app channel

Comment from @yaron2

"we have this issue here which might be resolved for 1.11: dapr/dapr#6177. this will enable us to set the app channel in daprd and remove the need to proxy the response back to the app.
if this doesn't get implemented, I suggest we use a standard Go net/http reverse proxy director to forward all traffic from daprd to the app endpoint. this should give us easy HTTP1.1/HTTP2 (gRPC) support"

Helm chart deployment with strategy: deployment not working

When changing strategy to deployment like

  helm upgrade --install $app-dapr $REPO_ROOT/../dapr-ambient/chart/dapr-ambient/ \
      --set fullnameOverride=$app-dapr \
      --set ambient.strategy=deployment \
      --set ambient.daprd.image.tag=1.11.6 \
      --set ambient.appId=$app \
      --set ambient.remoteURL=$app-svc \
      --set ambient.remotePort=80 \
      --set ambient.serviceAccount.name=$app \
      --set ambient.daprd.listenAddresses=0.0.0.0

this error is shown

Error: UPGRADE FAILED: failed to create resource: Deployment.apps "distributor-dapr" is invalid: [spec.selector: Required value, spec.template.metadata.labels: Invalid value: map[string]string{"app.kubernetes.io/instance":"distributor-dapr", "app.kubernetes.io/name":"dapr-ambient-chart"}: `selector` does not match template `labels`]

Use new flag to set remote app channel

Based on this PR, now we don't need to proxy the app channel for remote URLs

dapr/dapr#6281

We need to test and refactor ambient to not do that anymore

"--app-channel-address", "",

Do we want to change the name `dapr-ambient` to something else to avoid confusions?

Dapr Ambient might cause confusion, as there are other projects using that name, such as Istio Ambient: https://istio.io/latest/blog/2022/introducing-ambient-mesh/

If we want to change the name, we need to gather a list of proposed names and then vote on them.

Originally, I proposed "ambient" because of its definition; "relating to the immediate (surroundings) of something." And because it does follow the same architectural pattern as Istio Ambient, I do understand that the name might confuse people.

Maintain identity in ambient mode

There is currently an effort in dapr to have strong identity. Ideally, ambient mode should maintain this security posture.

In ambient mode, daprd is installed as a daemon set where each dapr app connects to its node's daprd. That daprd must either 1. only allow apps which share the same app ID to connect, or 2. a single daprd should be able to serve any number of app IDs which are not necessarily known beforehand.

Create tutorial for Functions as a Service use case

Throw Error: configmap "dapr-ambient-configmap" not found

I followed the guidance at https://github.com/dapr-sandbox/dapr-ambient/blob/8ed4f260f44249aba275a54404506f3080deed61/tutorial/README.md

The only difference is I changed the version of daprd as

helm install product-api-dapr-ambient ambient/dapr-ambient --set ambient.appId=product-api --set ambient.remoteURL=product-api --set daprd.image.tag=1.11.0-rc.7

But when I run the helm install above, it always threw the exception

15s         Warning   Failed                    pod/product-api-dapr-ambient-mgspf                     Error: configmap "dapr-ambient-configmap" not found

I checked all the helm charts in this repo, but couldn't find any clue to fix it.

Could anyone show me how can I overcome this issue? Thank you very much.

Add Helm tests to chart

Add Helm tests to the chart to validate parameters: https://helm.sh/docs/topics/chart_tests/