Is your feature request related to a problem? Please describe.
I would like to use RabbitMQ virtual host for connection as I have restricted access to RabbitMQ.

Describe the solution you'd like

1. Add environment variable RMQ_VHOST, eg: RMQ_VHOST:openfaas
2. Same can be utilized in getRabbitMQConnectionURL() in config.go for enabling virtual host
   fmt.Sprintf("amqp://%s:%s@%s:%s/%s", user, pass, host, port, vhost)

Is your feature request related to a problem? Please describe.

I have multiple k8s namespaces holding openfaas functions. In my k8s cluster, I am deploying my software for three different environments: staging, pre-production, and production. Each of those environments is deployed in a different k8s namespace. In each of those namespaces, I have potentially the same openfaas function names, but with different routing keys (e.g. in the staging environment the routing keys all start with staging.). The functions' definition is the same up to the k8s namespace they should land into.

In that setup, I have deployed openfaas in its own namespace, as well as rabbitmq. Both of them are used by my three environments. In each environment, I have deployed a rabbitmq-connector. So if I have my three environments up and running, I have overall 1 openfaas, 1 rabbitmq, and 3 rabbitmq-connector deployments.

Now, when rabbitmq-connector consumes the messages from the rabbitmq queue, it seems like it looks for the corresponding functions in the openfaas-fn namespace, i.e. the default function namespace I've defined in my openfaas setup.

I would like that rabbitmq-connector routes the messages to the corresponding function in the right namespace.

Describe the solution you'd like

Probably a new environment variable could do the trick. I do not know exactly how rabbitmq-connector fetches information about the function it should deliver the consumes messages to. I can imagine that rabbitmq-connector uses the openfaas api and could probably get the function's namespace based on the routing key, but I have too little knowledge of the go language or the openfaas api to be really able to evaluate what solution would be best.

Describe alternatives you've considered

I could install openfaas and rabbitmq in each namespace but it would cost me more resources.

I have the problem that my pod it's restart every time.

2018/12/07 15:26:15 [*] Waiting for messages. To exit press CTRL+C
2018/12/07 15:26:45 invalid character 'i' looking for beginning of value

Any suggestion?

I have set up openfaas, rabbitmq on a kubernetes cluster (set up cluster with kind) on a windows machine but linux container. Both openfaas and rabbitmq seem to be working as expected.

However I'm facing an issue when deploying the rabbitmq-connector code. I have the Path_to_topology variable as below in the yaml file

•     name: PATH_TO_TOPOLOGY
      value: "./csharp/hack/test/artifacts/example_topology.yaml"
  

But whatever path I give I get the following error for the container "During Config validation provided topology is either non existing or does not end with .yaml occurred.".

Could you please suggest how I should define the path? I have tried the following values as well

a. Given absolute path:- C:/Users/jdhawan/csharp/hack/test/artifacts/example_topology.yaml
b. Given the same like :- C:\Users\jdhawan\csharp\hack\test\artifacts\example_topology.yaml
c. Given relative path like :- (since i have changed my directory to C:/Users/jdhawan/csharp/hack/test).- /artifacts/example_topology.yaml

I have also used the below
apiVersion: apps/v1
kind: Deployment
metadata:
name: rabbitmq-connector
namespace: openfaas
spec:
selector:
matchLabels:
app: rabbitmq-connector
replicas: 1
template:
metadata:
labels:
app: rabbitmq-connector
spec:
containers:
- name: connector
image: templum/rabbitmq-connector:develop
imagePullPolicy: Always
env:
- name: basic_auth
value: "true"
- name: secret_mount_path
value: "/etc/openfaas"
- name: OPEN_FAAS_GW_URL
value: "http://127.0.0.1:8080"
- name: RMQ_TOPICS
value: "demo"
- name: RMQ_HOST
value: "http://127.0.0.1"
- name: RMQ_PORT
value: "5672"
- name: RMQ_USER
value: "user"
- name: RMQ_PASS
value: "pass"
- name: REQ_TIMEOUT
value: "30s"
- name: TOPIC_MAP_REFRESH_TIME
value: "30s"
resources: # Adjust them to suite your needs
requests:
memory: 512Mi
cpu: 1
limits:
memory: 1024Mi
cpu: 2
volumeMounts:
- name: basic-auth
readOnly: true
mountPath: "/etc/openfaas"
- name: config-volume
mountPath: /artifacts/cfg/example-topology.yaml
subPath: example-topology.yaml
volumes:
- name: basic-auth
secret:
secretName: basic-auth
- name: config-volume
configMap:
name: topology-test

Any help is greatly appreciated. Also is there a way to know as to what path is it trying to form and cant get to it?

General Information

Environment

Docker: v20.10.5
Kubernetes: v1.19.7
Other: []

Connector Version
[Version: dev Commit: aa10f5d]

Bug Details

Describe the bug
Openfaas functions are not being triggered with rabbitmq connector. It was working before this new commit was made yesterday

Expected behaviour
The openfaas function should trigger

Steps to Reproduce

This is using Gitbash on windows

1. $ arkade install openfaas --basic-auth=false
2. $ kubectl rollout status -n openfaas deploy/gateway
3. $ kubectl port-forward -n openfaas svc/gateway 8080:8080 &
4. opened another git bash terminal and confirmed faas-cli. Not i already had faas-cli installed so didnt need to install it here
5. faas-cli list - this lists the functions if they are running
6. faas-cli store list
7. faas-cli store deploy figlet - also deployed function
8. helm install rabbit -n openfaas -f ./rabbit-values.yaml bitnami/rabbitmq - i deployed this in the openfaas namespace
9. kubectl port-forward --namespace openfaas svc/rabbit-rabbitmq 5672:5672 - run this for port forward so that with code u can send the data to something like "amqp://user:[email protected]:5672"
10. opened another terminal
11. kubectl port-forward --namespace openfaas svc/rabbit-rabbitmq 15672:15672 - run this so that u can access rabbitmq interface from browser
12. kubectl get po -n openfaas - wanted to check all the pods in openfaas namespace
13. kubectl get svc -n openfaas - this will help us get the ClusterIP which will be used in the rabbitmq-connector.yaml file. Note the ClusterIP of gateway and rabbit-rabbitmq
14. kubectl get cm -n openfaas - gets all configmaps in openfaas namespace
15. kubectl create cm top-test --namespace=openfaas --from-file=/artifacts/cfg/topology.yaml - created configmap for topology.

Contents of topology.yaml

• name: DEx
  topics: [abc]
  declare: true
  type: "direct"
  durable: false
  auto-deleted: false

16. kubectl apply -f ./artifacts/rabbitmq-connector.yaml

Contents of rabbitmq-connector yaml file

apiVersion: apps/v1
kind: Deployment
metadata:
name: rabbitmq-connector
namespace: openfaas
spec:
selector:
matchLabels:
app: rabbitmq-connector
replicas: 1
template:
metadata:
labels:
app: rabbitmq-connector
spec:
containers:
- name: connector
image: templum/rabbitmq-connector:develop
imagePullPolicy: Always
env:
- name: basic_auth
value: "false"
- name: OPEN_FAAS_GW_URL
value: "http://10.96.107.112:8080" # this was the gateway clusterIP
- name: RMQ_TOPICS
value: "demo"
- name: RMQ_HOST
value: "10.96.194.90" # this was the rabbitmq clusterIP
- name: RMQ_PORT
value: "5672"
- name: RMQ_USER
value: "user"
- name: RMQ_PASS
value: "pass"
- name: REQ_TIMEOUT
value: "30s"
- name: TOPIC_MAP_REFRESH_TIME
value: "30s"
- name: PATH_TO_TOPOLOGY
value: "/artifacts/cfg/topology.yaml"
resources: # Adjust them to suite your needs subPath: topology.yaml
requests:
memory: 512Mi
cpu: 1
limits:
memory: 1024Mi
cpu: 2
volumeMounts:
- name: top-test
mountPath: "/artifacts/cfg/topology.yaml"
subPath: topology.yaml
volumes:
- name: top-test
configMap:
name: top-test

17. kubectl get po -n openfaas - make sure all pods are running
18. kubectl logs rabbitmq-connector-5b59f648c-kvb5t -n openfaas -f - then check logs for connector to ensure it is able to connect
19. deployed openfaas function - faas-cli store deploy figlet --annotation topic="demo" --name integration
20. run rabbitmq producer to generate topic data. note i have .net rabbitmq producer application for this
21. open openfaas portal at http://127.0.0.1:8080/ui/ and confirm the invocation count for the function after generating the data
22. faas-cli deploy -f csharp.yml --annotation topic="demo" --name csharp - this is runnign from local Git location

Screenshots/Logs
If applicable, add screenshots/logs to help explain your problem.

Logs of connector
2021/04/08 23:03:13 maxprocs: Updating GOMAXPROCS=2: determined from CPU quota
2021/04/08 23:03:13 OpenFaaS RabbitMQ Connector [Version: dev Commit: aa10f5d]
2021/04/08 23:03:13 Started Cache Task which populates the topic map
2021/04/08 23:03:13 Started RabbitMQ <=> OpenFaaS Connector
2021/04/08 23:03:13 Will now establish connection to amqp://10.96.72.115:5672
2021/04/08 23:03:13 Crawling namespaces for functions
2021/04/08 23:03:13 Crawling for functions
2021/04/08 23:03:13 Crawling finished will now refresh the cache
2021/04/08 23:03:13 Update cache with 2 entries
2021/04/08 23:03:13 Successfully established connection to Rabbit MQ Cluster
2021/04/08 23:03:13 Factory is configured for exchange DEx
2021/04/08 23:03:13 Successfully declared exchange DEx of type direct { Durable: false Auto-Delete: false }
2021/04/08 23:03:13 Successfully declared Queue OpenFaaS_DEx_abc
2021/04/08 23:03:13 Successfully bound Queue OpenFaaS_DEx_abc to exchange DEx
2021/04/08 23:03:13 [*] Waiting for messages. To exit press CTRL+C
2021/04/08 23:03:43 Crawling namespaces for functions
2021/04/08 23:03:43 Crawling for functions
2021/04/08 23:03:43 Crawling finished will now refresh the cache
2021/04/08 23:03:43 Update cache with 2 entries
2021/04/08 23:04:13 Crawling namespaces for functions
2021/04/08 23:04:13 Crawling for functions
2021/04/08 23:04:13 Crawling finished will now refresh the cache
2021/04/08 23:04:13 Update cache with 2 entries
2021/04/08 23:04:43 Crawling namespaces for functions
2021/04/08 23:04:43 Crawling for functions
2021/04/08 23:04:43 Crawling finished will now refresh the cache
2021/04/08 23:04:43 Update cache with 2 entries
2021/04/08 23:05:13 Crawling namespaces for functions
2021/04/08 23:05:13 Crawling for functions
2021/04/08 23:05:13 Crawling finished will now refresh the cache
2021/04/08 23:05:13 Update cache with 2 entries
2021/04/08 23:05:43 Received message for topic OpenFaaS_DEx_abc that did not match subscribed topic abc will reject it
2021/04/08 23:05:43 Crawling namespaces for functions
2021/04/08 23:05:43 Crawling for functions
2021/04/08 23:05:43 Crawling finished will now refresh the cache
2021/04/08 23:05:43 Update cache with 2 entries
2021/04/08 23:06:13 Crawling namespaces for functions
2021/04/08 23:06:13 Crawling for functions
2021/04/08 23:06:13 Crawling finished will now refresh the cache
2021/04/08 23:06:13 Update cache with 2 entries
2021/04/08 23:06:43 Crawling namespaces for functions
2021/04/08 23:06:43 Crawling for functions
2021/04/08 23:06:43 Crawling finished will now refresh the cache
2021/04/08 23:06:43 Update cache with 2 entries
2021/04/08 23:07:13 Crawling namespaces for functions
2021/04/08 23:07:13 Crawling for functions
2021/04/08 23:07:13 Crawling finished will now refresh the cache
2021/04/08 23:07:13 Update cache with 2 entries
2021/04/08 23:07:43 Crawling namespaces for functions
2021/04/08 23:07:43 Crawling for functions
2021/04/08 23:07:43 Crawling finished will now refresh the cache
2021/04/08 23:07:43 Update cache with 2 entries
2021/04/08 23:08:13 Crawling namespaces for functions
2021/04/08 23:08:13 Crawling for functions
2021/04/08 23:08:13 Crawling finished will now refresh the cache
2021/04/08 23:08:13 Update cache with 2 entries
2021/04/08 23:08:43 Crawling namespaces for functions
2021/04/08 23:08:43 Crawling for functions
2021/04/08 23:08:43 Crawling finished will now refresh the cache
2021/04/08 23:08:43 Update cache with 2 entries
2021/04/08 23:09:13 Crawling namespaces for functions
2021/04/08 23:09:13 Crawling for functions
2021/04/08 23:09:13 Crawling finished will now refresh the cache
2021/04/08 23:09:13 Update cache with 2 entries
2021/04/08 23:09:43 Crawling namespaces for functions
2021/04/08 23:09:43 Crawling for functions
2021/04/08 23:09:43 Crawling finished will now refresh the cache
2021/04/08 23:09:43 Update cache with 2 entries
2021/04/08 23:10:13 Crawling namespaces for functions
2021/04/08 23:10:13 Crawling for functions
2021/04/08 23:10:13 Crawling finished will now refresh the cache
2021/04/08 23:10:13 Update cache with 2 entries
2021/04/08 23:10:43 Crawling namespaces for functions
2021/04/08 23:10:43 Crawling for functions
2021/04/08 23:10:43 Crawling finished will now refresh the cache
2021/04/08 23:10:43 Update cache with 2 entries
2021/04/08 23:11:13 Crawling namespaces for functions
2021/04/08 23:11:13 Crawling for functions
2021/04/08 23:11:13 Crawling finished will now refresh the cache
2021/04/08 23:11:13 Update cache with 2 entries
2021/04/08 23:11:43 Crawling namespaces for functions
2021/04/08 23:11:43 Crawling for functions
2021/04/08 23:11:43 Crawling finished will now refresh the cache
2021/04/08 23:11:43 Update cache with 2 entries
2021/04/08 23:12:13 Crawling namespaces for functions
2021/04/08 23:12:13 Crawling for functions
2021/04/08 23:12:13 Crawling finished will now refresh the cache
2021/04/08 23:12:13 Update cache with 2 entries
2021/04/08 23:12:43 Crawling namespaces for functions
2021/04/08 23:12:43 Crawling for functions
2021/04/08 23:12:43 Crawling finished will now refresh the cache
2021/04/08 23:12:43 Update cache with 2 entries
2021/04/08 23:13:13 Crawling namespaces for functions
2021/04/08 23:13:13 Crawling for functions
2021/04/08 23:13:13 Crawling finished will now refresh the cache
2021/04/08 23:13:13 Update cache with 2 entries
2021/04/08 23:13:43 Crawling namespaces for functions
2021/04/08 23:13:43 Crawling for functions
2021/04/08 23:13:43 Crawling finished will now refresh the cache
2021/04/08 23:13:43 Update cache with 2 entries
2021/04/08 23:14:13 Crawling namespaces for functions
2021/04/08 23:14:13 Crawling for functions
2021/04/08 23:14:13 Crawling finished will now refresh the cache
2021/04/08 23:14:13 Update cache with 2 entries
2021/04/08 23:14:43 Crawling namespaces for functions
2021/04/08 23:14:43 Crawling for functions
2021/04/08 23:14:43 Crawling finished will now refresh the cache
2021/04/08 23:14:43 Update cache with 2 entries
2021/04/08 23:15:13 Crawling namespaces for functions
2021/04/08 23:15:13 Crawling for functions
2021/04/08 23:15:13 Crawling finished will now refresh the cache
2021/04/08 23:15:13 Update cache with 2 entries
2021/04/08 23:15:43 Crawling namespaces for functions
2021/04/08 23:15:43 Crawling for functions
2021/04/08 23:15:43 Crawling finished will now refresh the cache
2021/04/08 23:15:43 Update cache with 2 entries
2021/04/08 23:16:13 Crawling namespaces for functions
2021/04/08 23:16:13 Crawling for functions
2021/04/08 23:16:13 Crawling finished will now refresh the cache
2021/04/08 23:16:13 Update cache with 2 entries
2021/04/08 23:16:43 Crawling namespaces for functions
2021/04/08 23:16:43 Crawling for functions
2021/04/08 23:16:43 Crawling finished will now refresh the cache
2021/04/08 23:16:43 Update cache with 2 entries
2021/04/08 23:17:13 Crawling namespaces for functions
2021/04/08 23:17:13 Crawling for functions
2021/04/08 23:17:13 Crawling finished will now refresh the cache
2021/04/08 23:17:13 Update cache with 2 entries
2021/04/08 23:17:43 Crawling namespaces for functions
2021/04/08 23:17:43 Crawling for functions
2021/04/08 23:17:43 Crawling finished will now refresh the cache
2021/04/08 23:17:43 Update cache with 2 entries
2021/04/08 23:18:13 Crawling namespaces for functions
2021/04/08 23:18:13 Crawling for functions
2021/04/08 23:18:13 Crawling finished will now refresh the cache
2021/04/08 23:18:13 Update cache with 2 entries
2021/04/08 23:18:43 Crawling namespaces for functions
2021/04/08 23:18:43 Crawling for functions
2021/04/08 23:18:43 Crawling finished will now refresh the cache
2021/04/08 23:18:43 Update cache with 2 entries
2021/04/08 23:19:13 Crawling namespaces for functions
2021/04/08 23:19:13 Crawling for functions
2021/04/08 23:19:13 Crawling finished will now refresh the cache
2021/04/08 23:19:13 Update cache with 2 entries

Using a .net application to publish message to RabbitMQ. Sample code below

static void Main(string[] args)
{

        var factory = new ConnectionFactory
        {
            Uri = new Uri("amqp://user:[email protected]:5672")
        };

        //amqp://127.0.0.1:5672/
        using var connection = factory.CreateConnection();
        using var channel = connection.CreateModel();
      

      
        channel.ExchangeDeclare(exchange: "DEx", type: "direct");
       
        var message = new { Name = "Producer", Message = "Hello" };
    
        channel.BasicPublish(exchange: "DEx",
                routingKey: "abc",
                basicProperties: null,
                body: body);


        Console.WriteLine(" [x] Sent {0}", message);
    }

General Information

Environment
OpenFaas is deployed locally on kubernetes(using Minikube)
Rabbitmq and rabbitmq connector is deployed on on kubernetes(using Minikube)

Connector Version
Latest version

Bug Details

Describe the bug
When I send messages to different topics, the OpenFaas function is not getting invoked every time.
Expected behaviour
I expect OpenFaas function to get invoked every time there is a message for that topic to which openfaas function is subscribed.

Steps to Reproduce
Set RMQ_TOPICS: "input,output"
Using faas-cli to deploy OpenFaas function
faas-cli store deploy Colorization --annotation topic="output"
faas-cli store deploy fillet --annotation topic="input"

Checking whether function is invoked or not with faas-cli list

Sending messages to the OpenFaaSQueue and topics(input, output) with a python client.

Please let me know if you need any more information.

General Information

Environment

Docker: []
Kubernetes: [x]
Other: []

Connector Version
Tag:
Sha:

Bug Details

Describe the bug
I try to configure the following setup:

-> OpenFaas Function send a message to RabbitMQ
-> RabbitMQ shall trigger another OpenFaas Function

In my RabbitMQ I have configured this:

Exchange: OpenFaasEx
Type | direct
durable:true 

Queue OpenFaaS_default
durable:true

Looking in RabbitMQ I can see the RabbitMQ Connector Consumers on the OpenFaas_default Queue

In my function I use this to send the message:

var exchange = "OpenFaasEx";
var routingKey = "default";
channel.assertExchange(exchange, "direct", {
        durable: true,
      });

channel.publish(exchange, routingKey, Buffer.from(event.body));

I can see the messages appear on the exchange.

My "to be called" function is deployed like this:

faas deploy --replace=true --update=false -f default-message-function.yml --annotation topic="default"

However I don't get any traffic on the queue nor a function call. I feel like I'm missing one link/configuration here...

Expected behaviour
default-message-function should be called

General Information

Environment

Docker: []
Kubernetes: [X]
Other: []

Connector Version
Tag:develop

Bug Details

Describe the bug
When I try to run the connector, I get crashes just after the connector succeed to connect to the RabbitMQ. I'm not sure if I missconfigure something, but eitherway the error message is not meaningfull at all invalid character 'i' looking for beginning of value. On the web, there is several mentions of this kind of error, related with a bad parsing of json.

Environment
My RabbitMQ is in version 3.6.9 and seems to be functionnal as it is a preprod of another program, and as I saw the connector successfully pop elements of the OpenFaas_MyTopic queue, but the function annotated with topic: MyTopic is never called.

Full stack

2019/07/23 12:25:03 OpenFaaS RabbitMQ Connector [Version: dev Commit: a76329e]
2019/07/23 12:25:03 Connector is in local mode will use the debug credentials
2019/07/23 12:25:03 Started Map Building. Be Aware it will take 30s until the first map is avaliable.
2019/07/23 12:25:03 1 Topic(s) are registered. Will be spawning 8 Worker(s) per Topic. 
2019/07/23 12:25:03 Binding Queue OpenFaaS_MyTopic to Exchange OpenFaasEx for Topic: MyTopic
2019/07/23 12:25:03 Binding Queue OpenFaaS_MyTopic to Exchange OpenFaasEx for Topic: MyTopic
2019/07/23 12:25:03 Binding Queue OpenFaaS_MyTopic to Exchange OpenFaasEx for Topic: MyTopic
2019/07/23 12:25:03 Binding Queue OpenFaaS_MyTopic to Exchange OpenFaasEx for Topic: MyTopic
2019/07/23 12:25:03 Binding Queue OpenFaaS_MyTopic to Exchange OpenFaasEx for Topic: MyTopic
2019/07/23 12:25:03 Binding Queue OpenFaaS_MyTopic to Exchange OpenFaasEx for Topic: MyTopic
2019/07/23 12:25:03 Binding Queue OpenFaaS_MyTopic to Exchange OpenFaasEx for Topic: MyTopic
2019/07/23 12:25:03 Binding Queue OpenFaaS_MyTopic to Exchange OpenFaasEx for Topic: MyTopic
2019/07/23 12:25:03 Subscriber agitated_leakey4 is now listening for messages on MyTopic
2019/07/23 12:25:03 Subscriber adoring_euclid0 is now listening for messages on MyTopic
2019/07/23 12:25:03 Subscriber sharp_stonebraker0 is now listening for messages on MyTopic
2019/07/23 12:25:03 Subscriber dreamy_davinci6 is now listening for messages on MyTopic
2019/07/23 12:25:03 Subscriber vibrant_curie9 is now listening for messages on MyTopic
2019/07/23 12:25:03 Subscriber kind_haibt9 is now listening for messages on MyTopic
2019/07/23 12:25:03 Subscriber gracious_edison3 is now listening for messages on MyTopic
2019/07/23 12:25:03 Subscriber tender_mcclintock9 is now listening for messages on MyTopic
2019/07/23 12:25:03  [*] Waiting for messages. To exit press CTRL+C
2019/07/23 12:25:33 invalid character 'i' looking for beginning of value

Expected behaviour
Either the program should be working if this is a bug, or the error message should be more meaningfull to help user to troubleshoot.

I checked out the old revision c9c0edd, with the aim of building the old image but I encountered an error:

ubuntu@ip-10-128-2-235:~/async-code/connectrabbit/rabbitmq-connector$ docker build -t connect2rabbit .
Sending build context to Docker daemon  895.5kB
Step 1/13 : FROM golang:1.9.2
1.9.2: Pulling from library/golang
723254a2c089: Pull complete
abe15a44e12f: Pull complete
409a28e3cc3d: Pull complete
503166935590: Pull complete
abe52c89597f: Pull complete
ce145c5cf4da: Pull complete
96e333289084: Pull complete
39cd5f38ffb8: Pull complete
Digest: sha256:947826b5b6bc42f6f25c8fae9fe89d5e2bf18ac87264856742c3baebb150ea9f
Status: Downloaded newer image for golang:1.9.2
 ---> 138bd936fa29
Step 2/13 : RUN mkdir -p /go/src/github.com/Templum/rabbitmq-connector
 ---> Running in 3e4f82df7917
Removing intermediate container 3e4f82df7917
 ---> 4c24f4bb5823
Step 3/13 : WORKDIR /go/src/github.com/Templum/rabbitmq-connector
 ---> Running in 876b915ba449
Removing intermediate container 876b915ba449
 ---> 8b3e85334186
Step 4/13 : LABEL maintainer = "Simon Pelczer <[email protected]>"
 ---> Running in 61082f0c4c74
Removing intermediate container 61082f0c4c74
 ---> a450bec820d0
Step 5/13 : LABEL version = "0.1.0"
 ---> Running in 7ba79db60227
Removing intermediate container 7ba79db60227
 ---> d6c036d5877b
Step 6/13 : COPY vendor     vendor
 ---> 43a433188dac
Step 7/13 : COPY sdk        sdk
 ---> dfca939cf862
Step 8/13 : COPY connector  connector
 ---> dd1dcd48bc1d
Step 9/13 : COPY main.go    .
 ---> b22f8fdbbbb2
Step 10/13 : RUN test -z "$(gofmt -l $(find . -type f -name '*.go' -not -path "./vendor/*"))"
 ---> Running in f064558ca327
The command '/bin/sh -c test -z "$(gofmt -l $(find . -type f -name '*.go' -not -path "./vendor/*"))"' returned a non-zero code: 1
ubuntu@ip-10-128-2-235:~/async-code/connectrabbit/rabbitmq-connector$ /bin/sh -c test -z "$(gofmt -l $(find . -type f -name '*.go' -not -path "./vendor/*"))"' returned a non-zero code: 1

General Information

Environment

Irrelevant.

Connector Version
Tag: v0.4.1
Sha: 43976f8

Bug Details

Describe the bug

It is no big deal. It seems like the documentation is not in sync with the code. In the configuration options section, there is the mention of the environment variable RMQ_QUEUE. I was curious to see why this variable would be used and how. I made a full search in the codebase and found a use in the config_test.go only. And there it is only set to some value, it is then never used anywhere in the code.

Expected behaviour
I would expect that the environment variable RMQ_QUEUE be not part of the list of environment variables that is provided in the documentation. Instead, I would expect that it is somewhere mentioned how the queue name is generated. It is defined in queue_consumer_factory.go like this:

func generateQueueName(topic string) string {
	const PreFix = "OpenFaaS"
	return fmt.Sprintf("%s_%s", PreFix, topic)
}

I would therefore expect a comment of the kind "the queue name is generated automatically and reads OpenFaaS_".

Steps to Reproduce

Irrelevant.

Provide simple instruction to add the component to existing OpenFaaS deployment.
And general modifications

Is your feature request related to a problem? Please describe.

When I want to deploy the connector on my kubernetes cluster, I would like to deploy it by means of a helm chart, as I do for all my other microservices. Such a helm chart would obviously make the setup of the connector a bit simpler than hacking into the yaml configurations stored in the artifacts folder.

Describe the solution you'd like

A helm chart.

Describe alternatives you've considered

If I based purely on the code published to this repo, then I'd need to modify the yaml configuration files located in the artifacts folder and use kubectl to install them.

Additional context

I have written a helm chart that I can use for my own purposes. If you want, I'm happy to give it to you.

A user reported that in the current version of the connector, the incoming messages are processed sequentially instead of parallel. As it is basically waiting for the ACK before sending the next message.

• Investigate if there was an accidently code change on the connector part. As with the old version it was parallel.
• Generally improve throughput by spawning multiple consumer based on the topics and the available CPU's.
• Ensure that an ACK is send to Rabbit MQ only once the invocation was performed either successfully or with an failure.

• Add golint ./... during CI
• Fix existing issues

Is your feature request related to a problem? Please describe.
As a user, I would like to receive detailed feedback when calls to OpenFaaS faile, due to incorrect setup.

Describe the solution you'd like
Switch over from the Connector SDK to an own implementation, which provides better feedback and is better suited for usage.

Describe alternatives you've considered

Additional context
Example of users experiencing issues with Auth Setup:

• #37 (comment)
• #32

Is your feature request related to a problem? Please describe.
Right now the number of max hosts for an individual function is hardcoded to 256. Which might be a reasonable default value, but does not leave any room for configurations. Especially if bursts of deliveries can be expected

Describe the solution you'd like
Provide an env that allows setting a value if needed.

Describe alternatives you've considered

Is your feature request related to a problem? Please describe.
In my workflow, we'd like to react to existing events, pushed by different microservices (let say A and B) which commit their actions in different Exchange (AEx and BEx).

Describe the solution you'd like
I would nice think to have a list of echanges, instead of only one :

RMQ_EXCHANGES : 
  - AEx
  - BEx

Maybe more precisely, each exchange could have their topics to listen ?

RMQ_EXCHANGES:
- name: AEx
  topics:
  - Foo
  - Bar
- name: BEx
  topics:
  - Dead
  - Beef

Which can also be writen

RMQ_EXCHANGES:
- name: AEx
  topics: [Foo, Bar]
- name: BEx
  topics: [Dead, Beef]

Describe alternatives you've considered
Having some kind of worker to copy-paste events from AEx and BEx to OpenFaasEx, but this is neither convenient nor a good practice.

In the same way functions in OpenFaaS Cloud access a secret to access the API Gateway's function list, we should enable this for the connector.

General Information

Bug Details

Describe the bug
Currently the connector does not always use the correct HTTP Method. Which may cause issues in newer version of OpenFaaS.

Expected behaviour
Correct use of HTTP Methods

Steps to Reproduce

Screenshots/Logs

Hi
I am deploying the rabbitmq-connector on K8s cluster running on VMs with AMD processors, but getting following error:
2023/02/15 13:18:45 maxprocs: Updating GOMAXPROCS=2: determined from CPU quota
2023/02/15 13:18:45 OpenFaaS RabbitMQ Connector [Version: v1.0.8 Commit: a9e3668]
2023/02/15 13:18:45 Will read basic64 secret from path /etc/openfaas which was set via 'secret_mount_path'
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0x6e2535]

goroutine 1 [running]:
github.com/Templum/rabbitmq-connector/pkg/config.getTopology({0x7fa540, 0xa7d7e8})
/go/src/github.com/Templum/rabbitmq-connector/pkg/config/config.go:212 +0x95
github.com/Templum/rabbitmq-connector/pkg/config.NewConfig({0x7fa540, 0xa7d7e8})
/go/src/github.com/Templum/rabbitmq-connector/pkg/config/config.go:75 +0x2ee
main.main()
/go/src/github.com/Templum/rabbitmq-connector/main.go:40 +0x270

OpenFaaS now has support for annotations meaning we can use a rich value for the topic list.

faas-swarm 0.4.0
faas-netes 0.6.0
openfaas-operator (awaiting review & merge)
This task means changing the code to look at the function's topic value in an annotation instead of a label.

It should be a straight-forward change plus basic smoke testing on Kubernetes and Swarm.

Is your feature request related to a problem? Please describe.
We would like to use the RabbitMQ-Connector on company internal k8s cluster, which uses certificates signed by our company CA. This is not supported by the connector resulting in error accessing the gateway URL:
"Get https://dev.function.our-company-name.com/system/functions: x509: certificate signed by unknown authority"

Describe the solution you'd like

• Add environment bool variable, e.g.: INSECURE_SKIP_VERIFY: true.

• Assuming that in the code this environment variable will be expressed as: insecureSkipVerify, the parameter could be apssed to the RabbitMQ connection or the definition of the HTTP client could be, e.g.:

client := &http.Client{ Transport: &http.Transport{ TLSClientConfig: &tls.Config{ InsecureSkipVerify: insecureSkipVerify, }, }, }

Describe alternatives you've considered
Directly configuring RabbitMQ instead of the connector.

Additional context
None.

Can somebody provide an example using the topology? updating the artifact with an example would be helpful. I get the error "During Config validation provided topology is either non existing or does not end with .yaml occurred"

Is your feature request related to a problem? Please describe.
Currently, code cov will report on patches, resulting in the PR being flagged red

Describe the solution you'd like
Follow the guidance provided here and here

Describe alternatives you've considered

Additional context
Only affects the building process

As part of go 1.11, they introduced go modules. Make sure to set up a module for the connector and remove files associated with go dep.

The way I understand your code, the rabbitmq-connector is a rabbitmq consumer. It consumes messages out of the rabbitmq, maps the message with the relevant function, waits until the function returns; upon function result reception, the rabbitmq-connector sends back an acknowledgment (or a failure) to rabbitmq.

In order to be able to send the right acknowledgement of the right function to rabbitmq, the rabbitmq-connector consumer needs to maintain a state. Currently, it is an in-memory map.

My question is: what will happen if the rabbitmq-connector consumer has just called a function and suddenly crashes, and the function's return value has not been obtained yet? Is that resolved in this way that rabbitmq never gets an acknowledgment / failure confirmation and takes the consumed message back in the queue?

I'm sorry for that noob question, I am not very familiar with the rabbitmq works.

Is your feature request related to a problem? Please describe.
Version 0.1.0 is bugged because of #26 and the develop tag cannot be used productive because of #32 (credentials are hard-coded when build as dev version).

Describe the solution you'd like
Publish a new 0.1.1 release to Docker Hub build from the current develop branch.
I build a Docker image locally for testing and now everything works as expected.

General Information

I found that hasNamespaceSupport is true but the namespaces array returned from getNamespaces can be empty, leading to zero functions being found and added to the cache (as crawlFunctions iterates over the namespaces array)

Environment

Docker: [x]
Kubernetes: []
Other: []

Connector Version
Tag:0.1.2
Sha:

Bug Details

I found that hasNamespaceSupport is true but the namespaces array returned from getNamespaces can be empty, leading to zero functions being found and added tot he cache as crawlFunctions iterates over the namespaces array.

I fixed this locally by adding a length check and default value to the namespaces array if it is returned empty from client.GetNamespaces() in cacher.go;

`
func (c *Controller) refreshTick(ctx context.Context, hasNamespaceSupport bool) {
builder := NewFunctionMapBuilder()
var namespaces []string
var err error

if hasNamespaceSupport {
	log.Println("Crawling namespaces for functions")
	namespaces, err = c.client.GetNamespaces(ctx)
	if err != nil {
		log.Printf("Received the following error during fetching namespaces %s", err)
		namespaces = []string{}
	}

	if(len(namespaces) < 1) {
		namespaces = []string{""}
	}
} else {
	namespaces = []string{""}
}

log.Println("Crawling for functions")
c.crawlFunctions(ctx, namespaces, builder)

log.Println("Crawling finished will now refresh the cache")
c.cache.Refresh(builder.Build())

}
`

Expected behaviour

I would expect any functions that contain the annotation "topic" to be found and added to the cache.

Steps to Reproduce

Using the Docker compose file and Local instructions, the figlet function was not added to the cache.

Since both ActiveMQ and RabbitMQ support AMQP 1.0, is at all possible to use this connector with ActiveMQ? (I have to stick to ActiveMQ, due to legacy code.)

General Information

Environment

Docker: []
Kubernetes: [x]
Other: []

Connector Version
Tag: master
Sha:

Bug Details

Describe the bug
The http request to open-faas does not provide a content-type header. This causes the lose of the request body when calling a nodejs based open-faas function (based on https://github.com/openfaas/templates/tree/master/template/node12). This default template uses express to interpret the call and renders any body that comes without a content-type as invalid.

Expected behaviour
the http request should come with a content-type header. Idealy one should be able to set the header through the message properties on the queue.

Upgrade to apps/v1 for the Deployment for K8s 1.16

https://github.com/Templum/rabbitmq-connector/blob/develop/artifacts/connector-dep.yaml

https://goreportcard.com/report/github.com/Templum/rabbitmq-connector

General Information

Environment

Docker: [X]
Kubernetes: [X]
Other: []

Bug Details

Describe the bug
In the kubernetes template (as well as in the docker compose), you use the docker image templum/rabbitmq-connector:dev. But this one is nowhere to be found, in fact on your registry there is a develop tag but no dev tag.

Expected behaviour
The image should either be tagged dev instead of develop (I find that better) or at least, the correct tag should be used inside of the docker-compose and kubernetes templates.

Steps to Reproduce

docker pull templum/rabbitmq-connector:dev                                      
# > Error response from daemon: manifest for templum/rabbitmq-connector:dev not found
docker pull templum/rabbitmq-connector:develop
#develop: Pulling from templum/rabbitmq-connector
#169185f82c45: Downloading [=======>                                           ]  342.8kB/2.207MB
#27ccad71c6e2: Downloading [============================>                      ]  179.8kB/310.3kB
#616561da8f52: Downloading [>                                                  ]  20.98kB/2.028MB
#1b94b286c8b4: Waiting 

Add the following things:

• Badges (Coverage, Build, Go Repot)
• Give Information on Topologie for RabbitMQ
• Link to RabbitMQ Production Checklist
• Create PR Template

Bug Details

Describe the bug
When using TLS it is not possible at all to provide credentials. Meaning if TLS is enabled and the user requires Auth it is not possible to leverage the connector.

Expected behavior
Leverage a user when present

Steps to Reproduce

Screenshots/Logs

Feature request (I don't think it's currently possible?)
All connections to the Rabbitmq I'm using are required to use TLS for both the connection and client authentication. Would it be possible to add some configuration to allow a certificate, key and CA certificate chain to be provided to the underlying rabbit client?

I've noticed the library being used for the connection has a DialTLS method which I think could be used.

Happy to discuss and implement.

General Information

Environment

Docker: [20.10.5]
Kubernetes: []
Other: []

Connector Version
Tag: templum/rabbitmq-connector:latest
Sha: 8db570831cf6b595f2beb4481b2f121283053b34502ff0adca257b3426eb3260

Bug Details

Describe the bug

The connection between the rabbitmq-connector and rabbitmq is working well (thank you for this project!) however, my messages never get acknowledged, they stay in the queue if they are successfully processed or not.

I can't seem to find what should be the message returned by the function to trigger that, I am currently returning that

    return {
        statusCode, // the HTTP status code
         headers: {
           'content-type': 'application/json' // I also tried with the default content-type without any success
        },
        Body: content, // can be anything
    };

(Also, I don't seem to be able to set a different status code)

Expected behaviour
Please give a clear and concise description of what you expected to happen.

Manually or automatically acknowledge the message on response

Steps to Reproduce

If possible provide a step by step guide on how to reproduce the issue.
Please make sure to also highlight environmental factors.

Screenshots/Logs
If applicable, add screenshots/logs to help explain your problem.

Is your feature request related to a problem? Please describe.
In case openfaas is busy (timeout), or there is an error (response !=2xx), it seems required that the connector pushes the message back to the queue in order to avoid data to be lost.

Describe the solution you'd like
As mentioned, I think it would be nice if the message is pushed back to the queue, it may seem also a good idea to throttle execution in general (or in this case) to avoid highload loops, in case openfaas is not available.

Describe alternatives you've considered

Additional context
100.000 messages in a queue that shall be pushed through one single, not scaleable function (due to system limits and cost contrains - but not time constrains)

How it works?
Hi, Can you help me, how to this connector work?
Do you have any blog or video tutorial for this?
Thank so much.