Expected Behaviour
It should be possible to deploy functions independently into two separate openfaas stacks in the same swarm while avoiding name collisions.
Current Behaviour
I have two openfaas gateways in two different docker swarm openfaas stacks in the same swarm. When I deploy a function into the gateway of the first stack, a function service gets created for that gateway and works fine. But when I deploy the same function (same name) into the other gateway, then the first gateway loses the connection to that function. Redeploying the function into the first gateway does not help. In the gateway logs I see:
2019/04/16 09:20:10 error with upstream request to: , Post http://my-function:8080: dial tcp: lookup my-function on 127.0.0.11:53: no such host
bemk_gateway.1.2jy84fkyzxz9@m1rrzsbem01t | 2019/04/16 09:20:10 Forwarded [POST] to /function/my-function - [502] - 0.004523 seconds
Ultimately I had to remove all functions and all openfaas stacks and recreate everything from scratch to get it working again.
My understanding is that function services do not become part of a stack. The gateway finds its function services by means of a label. Here is my function in faas-cli describe:
Name: my-function
Status: Ready
Replicas: 1
Available replicas: 1
Invocations: 0
Image: bem-docker-registry.kvnurs.intra:443/my-function:latest
Function process:
URL: http://bemk.kvnurs.intra/function/my-function
Async URL: http://bemk.kvnurs.intra/async-function/my-function
Labels: function : true
com.openfaas.function : my-function
I assume the gateway finds its functions using the com.openfaas.function
label. On deployment the gateway apparently attaches the function to its own network.
Now, when the gateway in the second stack deploys the same function, it seems to disconnect the function from its current network and attaches it to its own network (and probably does other things which cause more confusion).
Possible Solution
A solution must allow the function to be a separate entity for docker if it is deployed into a different stack. The function must become a separate service for each stack, which scales independently etc. The only way to achieve that is a different service name, I am not aware that service labeling alone can help us here.
As long as the service has the same name in both stacks, it is probably not possible to achieve that.
Suggestion:
The openfaas yml format could introduce a name
field below each function key whose value supports env variable substitution. The gateway uses the service key to build the service url and the name
field to create the service name in docker.
Docker has done similar things with name fields for configs and volumes, so maybe that is not so far-fetched after all.
Workaround:
Create separate stack.yml files with separate service names prefixed for each stack and let callers use the appropriate url for each stack, i.e. http://localhost:8080/functions/develop_my-function.
Steps to Reproduce
- deploy the openfaas stack twice under two different stack names. (To make this possible, let the gateway of the second stack run under a different port)
- use faas-cli to deploy a function into the gateway of the first stack. Check that it works properly
- deploy the same function into the gateway of the second stack
Result:
- faas-cli list shows that both stacks know the function, fass-cli describe even claims that the function is ready in both stacks
- the function does not work, rather you get a 502
- redeploying the function into the gateway of the first stack does not fix it. I had to delete all functions and all stacks and deploy a new stack from scratch to get it working again with a single stack.
Context
In my scenario we have used swarm stacks for staging. Each swarm stack is a different stage (stages as in development - acceptance test etc. where placement constraints are used to distribute the containers to separate nodes depending on the stage).
Your Environment
- FaaS-CLI version ( Full output from:
faas-cli version
):
commit: 87ca614cfe27c4cf2975f7629992c1351b18c2bc
version: 0.8.8
- Docker version
docker version
(e.g. Docker 17.0.05 ):
Client:
Version: 18.06.1-ce
API version: 1.38
Go version: go1.10.7
Git commit: e68fc7a215d7
Built: Wed Dec 19 10:23:04 2018
OS/Arch: linux/amd64
Experimental: false
Server:
Engine:
Version: 18.06.1-ce
API version: 1.38 (minimum version 1.12)
Go version: go1.10.7
Git commit: e68fc7a215d7
Built: Tue Aug 21 17:16:31 2018
OS/Arch: linux/amd64
Experimental: false
-
Are you using Docker Swarm or Kubernetes (FaaS-netes)?
Swarm
-
Operating System and version (e.g. Linux, Windows, MacOS):
Linux Suse SLE12 sp 4
-
Link to your project or a code example to reproduce issue:
n/a yet