Comments (10)
Looking at more logs, I'm also seeing a few:
undefined method `map' for #<Consul::Async::ConsulTemplateNodes:0x000055996a095ed0>
I assume it's the same issue.
from consul-templaterb.
Hello @jeromegn,
Well, there is a reason (which sucks) for this behavior.
If you list nodes using nodes()
and then try to call node(my_node_name)
, there might be a race condition in such code, because it is possible that the call to nodes()
would return node1, node2, node3, but just after the value is returned, node3 is deregistered in the Consul cluster. In such case, the value for node(node3)
returned by Consul API would be null
(not even {}
).
In such case, the value read by consul-templaterb would be nil, which does not contains []
, hence, the error you have.
This can be reproduced trivially with this ERB code:
<%
node('a_nodes_that_does_not_exists')['Services']
%>
This sucks a bit, I agree, in theory, we should have an helper to check this.
How to avoid this?
There is no way the root cause could be prevented easily because the operations inside the clusters would be hard to get in a "transactional way".
However, it is possible to do the following:
<%=
res = node('a_nodes_that_does_not_exists')
unless res.result.is_a?(Hash)
{} # alternative, return nil if you want to decide between node without service and node that does not exists
else
res['Services']
end
%>
if you want to use this in many places, you can also have a function to do that:
<%
def get_services_for_node(node_name)
res = node(node_name)
return res['Services'] if res.result.is_a?(Hash)
{} # services is actually an Hash
end
%>
<%=
get_services_for_node("a_node_that_does_not_exists")
%>
Other Considerations
To be fair, we did not used that much this function on our clusters as it gives poor performance with many nodes since you have to interact with every node (and this creates lots of calls).
I also found that actually, there is a little bug on init time where node() -> is initialized with []
by default instead of {}
, I'll try to fix this soon
Regards
from consul-templaterb.
from consul-templaterb.
Our nodes are never deregistered really, they never change. Unless a short restart might cause this? We very rarely restart them, but I'm seeing this across the board from time to time.
Could it also be caused by a transient network issue?
from consul-templaterb.
Yes, restart of a node might explain it maybe, but not a network transient issue (Consul is supposed to be protected against that)
from consul-templaterb.
To be fair, we did not used that much this function on our clusters as it gives poor performance with many nodes since you have to interact with every node (and this creates lots of calls).
Is there a better way to get all services (with all their data)? Using /catalog/services
only gives service names and their tags. We have tens of thousands of different services. Querying per node seemed more efficient since we don't add as many nodes as we add services.
from consul-templaterb.
Perhaps we should've named all our services with the same name (or the same few names) and used the service(name)
method?
from consul-templaterb.
We I used to work at Criteo, that's what we did:
we had 4000+ services, with around 260k+ instances, we did list all services to have an aggregated view of all services (this is exactly what is done in https://github.com/criteo/consul-templaterb/blob/master/samples/consul-ui/consul_services.json.erb). The big bonus is that you also have the status of all the services at the same time
from consul-templaterb.
Thank you @kamaradclimber for merging this!
Do you think you might be able to do a release as well?
from consul-templaterb.
FWIW: we're going to be slowly transitioning to non-unique service names and this should help a lot with Consul issues we've been having (including this one).
from consul-templaterb.
Related Issues (20)
- Createing multiple files from one template
- Throttle EM requests to max/sec or max parallel? HOT 3
- Cannot see nodes/services/any in the UI HOT 4
- Improve Docker image builld HOT 2
- Getting timestamp in templates HOT 5
- Executing command on each change HOT 2
- Get checks by state HOT 6
- Trying to auth to vault with certs HOT 3
- Signal Handling from Orchestrator HOT 9
- Expose list of templates objects being generated HOT 3
- inactivity_timeout for the vault endpoint isn't configurable/too short HOT 2
- [FEATURE] Prometheus Endpoint for Template Rendering Times HOT 2
- `ready?` method for remote_resource.as_json is always true HOT 2
- Eventmachine Seg Fault When Using Vault HOT 5
- [Q] AIX 7.1 Support HOT 1
- Potential Memory Leak HOT 13
- consul_templaterb uses deprecated methods HOT 3
- Correct usage of --wait-signal option HOT 2
- Question: How can we dynamically generate different templates from a consul K/V JSON object ? HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from consul-templaterb.