Error while deploying sensor-data-scala example application about cloudflow HOT 6 CLOSED

I am using Cloudflow Managed cluster.
Yeah strimzi pods are healthy and are up and running.

I deployed after removing file-ingress streamlet. It works fine.
Logs are:
[WARN] [11/24/2019 11:16:07.056] [akka_streamlet-akka.actor.default-dispatcher-3] [akka.actor.ActorSystemImpl(akka_streamlet)] Invalid metric detected! {"metric": {"deviceId": "c75cb448-df0e-4692-8e06-0321b7703992", "timestamp": 1495545346279, "name": "power", "value": -1.7}, "error": "All measurements must be positive numbers!"}
Thank you for the help. It means a lot.

from cloudflow.

@dhillonr hi! Thank you for reporting this!
Currently regarding :

For applications requiring volume mounts are we supposed to manually create pvc's in respective namespaces ?

...the answer is yes. However, the other examples dont require a PVC being available, so you could start with the other ones (avoiding the sensor-data-xxx examples for now), if you havent done so already. I am planning to update the documentation for this example ASAP (the GKE deployment is in TODO status). Btw the declaration of volume mount point is done by the Cloudflow runtime at deployment time, check for example here. The latter helps to understand how naming is done and what PVC you are required to create. In the app the volume mount point used is declared here.
To provide a bit more context when you run buildAndPublish the sbt-cloudflow plugin is used and a label is added to the docker image published in the GKE cluster. That label has the application specification encoded in base64. The latter contains the info of the volume mounts. Then the kubectl-cloudflow go plugin decodes that label and creates a CR to be picked-up by the Cloudflow Operator, then the real deployment happens.

from cloudflow.

@dhillonr if you would like to proceed with the example you will have to point the mount point to an existing pvc in the namespace the app is going to be created. The specific app is created in the
sensor-data-scala ns so given this limitation that the PVC needs to pre-exist you need to do:
claim.yaml:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: claim1
  namespace: sensor-data-scala
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: nfs-client
  resources:
    requests:
      storage: 1Gi

Note: Assuming you picked nfs for the ReadWriteMany storage class during the Cloudflow installation process.

kubectl create ns sensor-data-scala
kubectl apply -f claim.yaml
kubectl cloudflow  deploy -u oauth2accesstoken --volume-mount file-ingress.source-data-mount=claim1   <app-docker-image> -p "$(gcloud auth print-access-token)"

Alternatively you could remove the file inrgess from the example and test the rest of the logic.

from cloudflow.

Thank you I was able to successfully deploy the application.
But in verification step when I sent a POST request to http-ingress I received an internal server error STATUS CODE 500.
I checked logs:
2019-11-22 15:51:03,642 WARN [NetworkClient] - [Consumer clientId=consumer-1, groupId=sensor-data-scala.valid-logger.in] Error while fetching metadata with correlation id 2 : {sensor-data-scala.validation.valid=UNKNOWN_TOPIC_OR_PARTITION}

Seems like streamlets are not able access kafka strimzi cluster.

Also I have a question that for different applications. Since a new namespace get created for every application (e.g.-sensor-data-scala in this case), whether their is only one kafka strimzi cluster with a predefined namespace (e.g.- lightbend in this case) .

Also if you can help us understand the rationale behind mapping multiple namespace(one for each application) to a single different kafka strimzi namespace ?

from cloudflow.

@dhillonr the Cloudflow runtime (Cloudflow, Spark, Flink, Kafka, Akka) is installed under lightbend namespace but the plan is to make the targeted namespace flexible.
So yes that is right there is only one instance by default that serves apps. The pattern here is to have a K8s operator for each framework which monitors for CR (Custom Resource) objects being created in the K8s cluster. The operators implementation allows to monitor either the whole cluster "all" namespaces or only a specific one. In the future the target is to have multi-namespace support, but currently there are some restrictions on how K8s operators are implemented.
That means you can setup multiple Cloudflow runtimes but you will have to restrict them monitoring one namespace each or you could have one Cloudflow runtime for the whole cluster.
This is also in compliance with current status of multi-tenancy on K8s (except for the multi-namespace support), so in practice you could have a cluster dedicated to an app or use Cloudflow as a service for the whole cluster.
The cloudflow go cli creates a Cloudflow CR and then the Cloudflow Operator creates the required Spark or Flink or Akka deployments. Apps are by default installed in their own namespace (apps currently cannot be installed in an existing arbitrary namespace unless it matches the one it is generated for them, plan is to remove the restriction).
I will check the error you are getting and get back to you.

from cloudflow.

@dhillonr I tried the app on my GKE cluster which I created using the cloudflow-installer:

$ ./gke-create-cluster.sh awesome-cluster
$ ./install-gke.sh awesome-cluster

$ kubectl cloudflow status sensor-data-scala
Name:             sensor-data-scala
Namespace:        sensor-data-scala
Version:          8-2a0f65d-dirty
Created:          2019-11-22 19:06:18 +0200 EET
Status:           Running

STREAMLET         POD                                                 STATUS            RESTARTS          READY             
validation        sensor-data-scala-validation-565978dffd-pdgcg       Running           0                 True
invalid-logger    sensor-data-scala-invalid-logger-7469c8fbd4-z9rxk   Running           0                 True
rotorizer         sensor-data-scala-rotorizer-775984d4db-xwpqg        Running           0                 True
http-ingress      sensor-data-scala-http-ingress-5bd67c9b76-5zr4h     Running           0                 True
rotor-avg-logger  sensor-data-scala-rotor-avg-logger-5ffb76d54d-wwd5k Running           0                 True
valid-logger      sensor-data-scala-valid-logger-69cbbb8654-m2tvh     Running           0                 True
merge             sensor-data-scala-merge-79944d79f8-s227z            Running           0                 True
file-ingress      sensor-data-scala-file-ingress-57456d9c94-cpw4d     Running           0                 True
metrics           sensor-data-scala-metrics-64b7d555dd-chk56          Running           0                 True

Then used port-forwarding as there no external url to access the http ingress by default:

$ kubectl port-forward sensor-data-scala-http-ingress-5bd67c9b76-5zr4h -n sensor-data-scala 3003:3003
$ curl -i -X POST localhost:3003 -H "Content-Type: application/json" --data '@test-data/invalid-metric.json'
HTTP/1.1 202 Accepted
Server: akka-http/10.1.10
Date: Fri, 22 Nov 2019 19:37:41 GMT
Content-Type: text/plain; charset=UTF-8
Content-Length: 88

The request has been accepted for processing, but the processing has not been completed.

kubectl logs sensor-data-scala-invalid-logger-7469c8fbd4-z9rxk   -n sensor-data-scala
...
[WARN] [11/22/2019 19:37:41.620] [akka_streamlet-akka.actor.default-dispatcher-5] [akka.actor.ActorSystemImpl(akka_streamlet)] Invalid metric detected! {"metric": {"deviceId": "c75cb448-df0e-4692-8e06-0321b7703992", "timestamp": 1495545346279, "name": "power", "value": -1.7}, "error": "All measurements must be positive numbers!"}

Note: Still this app docs are wip.

Are you using an existing cluster? Do you impose any security policies? Could you check first that strimzi pods are healthy and are up and running? Anything weird in the logs there?

from cloudflow.