Don't seem to be able to change the version about elasticsearch HOT 24 CLOSED

This is strange. What if you use a specific role for Logstash clusters and override it in the role? I don't think there's a Ruby test for version override, but I did tested it manually for #61, #68 and #69.

from elasticsearch.

Sorry, I can't use roles as they are not versioned (hence the wrapper cookbook).

from elasticsearch.

Understood -- which Chef version is this? I'll try to replicate it, but it will take some time, I'm afraid, since I'm tied to another project at the moment.

from elasticsearch.

I run my tests currently in Vagrant 1.2.2 with the Berkshelf plugin. The VM is Ubuntu 12.04.2 and chef-solo 11.4.2 (omnibus). The wrapper cookbook recipe looks like this:

node.override["elasticsearch"]["version"] = "0.20.6"

include_recipe "elasticsearch"
include_recipe "elasticsearch::plugins"
include_recipe "elasticsearch::proxy"

Fwiw, I also tried to set the attribute before including elasticsearch with no effect, e.g. with the attribute file:

override["elasticsearch"]["version"] = "0.20.6"

include_attribute "elasticsearch"

and the recipe

include_recipe "elasticsearch"
include_recipe "elasticsearch::plugins"
include_recipe "elasticsearch::proxy"

I can stay on commit 9b20d84 for now (which is the last one before you changed the default version).

from elasticsearch.

This is frustrating, I understand that... I'll try to debug the issue, and see whether it's with the cookbook or Ark...

from elasticsearch.

I had the wrapper cookbook log the configuration (with Chef::Log which happens in the compile phase):

[2013-06-17T22:06:22+00:00] INFO: elasticsearch config: {"version"=>"0.20.6",
"host"=>"http://download.elasticsearch.org", "repository"=>"elasticsearch/elasticsearch",
"filename"=>"elasticsearch-0.90.1.tar.gz",
"download_url"=>"http://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.90.1.tar.gz",
"cluster"=>{"name"=>"elasticsearch"}, "node"=>{"name"=>"logstash-server", "max_local_storage_nodes"=>1},
"dir"=>"/usr/local", "user"=>"elasticsearch", "path"=>{"conf"=>"/usr/local/etc/elasticsearch",
"data"=>"/usr/local/var/data/elasticsearch", "logs"=>"/usr/local/var/log/elasticsearch"},
"pid_path"=>"/usr/local/var/run/elasticsearch", "pid_file"=>"/usr/local/var/run/elasticsearch/logstash_server.pid",
"allocated_memory"=>"512m", "bootstrap"=>{"mlockall"=>false}, "limits"=>{"memlock"=>"unlimited",
"nofile"=>"64000"}, "index"=>{"mapper"=>{"dynamic"=>true}}, "action"=>{"auto_create_index"=>true,
"disable_delete_all_indices"=>true}, "discovery"=>{"zen"=>{"ping"=>{"multicast"=>{"enabled"=>false},
"unicast"=>{"hosts"=>["192.168.27.3"]}}, "minimum_master_nodes"=>1}, "type"=>nil, "ec2"=>{"groups"=>nil,
"tag"=>{}}}, "gateway"=>{"type"=>nil, "expected_nodes"=>1}, "thread_stack_size"=>"256k", "custom_config"=>{},
"plugins"=>{"elasticsearch-cloud-aws"=>{"version"=>"1.12.0"}, "karmi/elasticsearch-paramedic"=>{}},
"plugin"=>{"mandatory"=>[]}, "cloud"=>{"aws"=>{"access_key"=>nil, "secret_key"=>nil, "region"=>nil},
"ec2"=>{"endpoint"=>nil}, "node"=>{"auto_attributes"=>true}}, "data"=>{"devices"=>{}},
"deb_url"=>"https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.90.1.deb",
"deb_sha"=>"3ea1045b037912d9e2ebfbce5b6fc3aaffb77e5d", "logging"=>{"action"=>"DEBUG",
"com.amazonaws"=>"WARN", "index.search.slowlog"=>"TRACE, index_search_slow_log_file",
"index.indexing.slowlog"=>"TRACE, index_indexing_slow_log_file"}, "nginx"=>{"port"=>"8080",
"dir"=>"/etc/nginx", "user"=>"www-data", "log_dir"=>"/var/log/nginx", "users"=>[{"username"=>"admin",
"password"=>"password"}], "passwords_file"=>"/usr/local/etc/elasticsearch/passwords",
"allow_cluster_api"=>true, "allow_status"=>false, "client_max_body_size"=>"50M"},
"rpm_url"=>"https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.90.1.noarch.rpm",
"rpm_sha"=>"a56eb8a88a15adad9ab8e59168c6cd51a1bd9ba3", "network"=>{"host"=>"192.168.27.3"},
"master"=>true}

from elasticsearch.

Thanks, this seems like the version is set correctly, but is not propagated further in https://github.com/elasticsearch/cookbook-elasticsearch/blob/master/attributes/default.rb#L18-L19

from elasticsearch.

So in most cases where attributes end up being a composite of other attributes, I'll either set them (or reset them) in the recipe where they are in use. The order things are loaded means that attributes composed within the attribute files will never see the overrides provided either from wrapper cookbooks or roles.

One direct workaround for this now is to explicitly set the required url attribute to the correct location for the required version, and the wrapper cookbook could do this by updating the individual attributes and then composing them for the url.

from elasticsearch.

It seems like a problematic idea to blanket merge the default scope of elasticsearch attributes with the normal scope into the normal scope:
https://github.com/elasticsearch/cookbook-elasticsearch/blob/master/attributes/default.rb#L10

The behavior I'm seeing with this is that the version number is being ignored because the download_url ends up set as a normal variable. So default setting of download_url no longer takes precedence in that attribute file in future runs and will never be updated unless overridden explicitly in a wrapper cookbook, role, etc.

I put some debug output and tested locally. The default attributes file is evaluating twice for me. So the first time the default values are not present and not merged into normal scope. On the second evaluation though the default values are all present and put into normal scope.

I'm still not sure why the second evaluation is happening. Regardless though since normal scope attributes are persisted it seems like the approach in the line linked above will be problematic.

from elasticsearch.

It looks like double evaluation of attributes/default.rb is due to the include_attribute "elasticsearch::default" statement in attributes/aws.rb and attributes/proxy.rb

It looks like Chef will eval each attributes file once per cookbook and once per include_attribute setting. So on the second eval, everything in default scope under elasticsearch key goes into normal scope.

from elasticsearch.

Have been experiencing this issue as well. It seems like changing the version attribute does not change the deb_url or rpm_url attributes. We're running Chef 11 with 0.3.3 of this cookbook on an Ubuntu server and it ends up downloading the 0.9.1 .deb file and installing it into a folder named with 0.90.3.

Overriding the deb_url attribute to point to the 0.90.3 download link seemed to fix the problem.

from elasticsearch.

I noticed the wip tag applied to this ticket.

Has any work been done on this yet? If not I can take a shot at a pull request.

from elasticsearch.

I don't have a test setup right now to try myself, but can you force a re-resolve of the elasticsearch attributes file after setting the version (so the attributes with interpolated values get changed), i.e.:

node.override["elasticsearch"]["version"] = "0.20.6"

node.from_file(run_context.resolve_attribute("elasticsearch", "default.rb"))

include_recipe "elasticsearch"
include_recipe "elasticsearch::plugins"
include_recipe "elasticsearch::proxy"

(Ref: http://lists.opscode.com/sympa/arc/chef/2013-08/msg00210.html)

from elasticsearch.

I have the same problem.

What I see is that /var/chef/cache/elasticsearch.tar.gz is never overwritten even though a new version of the cookbook is deployed.

I solve it temporarily by adding

name  "elasticsearch-#{node.elasticsearch[:version]}"

to https://github.com/elasticsearch/cookbook-elasticsearch/blob/master/recipes/default.rb#L73

from elasticsearch.

@dsennerlov that leaves you with the install unpacked into /usr/local/elasticsearch-x.xx.x and a symlink elasticsearch-x.xx.x-x.xx.x pointing to it. Probably not what you want.

from elasticsearch.

I'd like to move away from Ark entirely, and manage all the download/extract/symlink/etc actions ourselves, so we have some kind of control over it.

@davekonopka Nobody is working on this specifically right now, if you'd have a fix, please submit a pull request.

from elasticsearch.

+1 for getting rid of ark

from elasticsearch.

Run into the same issue.
I create a fix which worked for me and added it as a pull request.
#165

from elasticsearch.

👍 seeing this here too in a wrapper cookbook. Temporarily solved with:

node.override["elasticsearch"]["download_url"] = "http://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.90.8.tar.gz"

from elasticsearch.

Hello.

Is there any ETA to a real fix ?

from elasticsearch.

See #178 and #180 -- they seem like duplicates of this issue. Until overriding attributes works differently, derived attributes aren't updated when the first attribute is overridden. Until Chef changes something, this is the way chef works -- you must override any calculated values if you override the variables they refer to. We can either (a) remove all calculated values or (b) convert to LWRPs that wrapper cookbooks can use directly or (b) wait for chef to change how this works.

Until then, there's a definite workaround though -- just copy the three lines and override all three values (example).

from elasticsearch.

Thanks. I overridden the value, but it's ... chef ... ;)

from elasticsearch.

Hi, I've merged #242, so this bug should be fixed now. I'm sorry for the inconvenience the bad behaviour caused you. Closing this issue for now, please ping me if you would like to discuss it further.

from elasticsearch.

Hi, I've merged #242, so this bug should be fixed now. I'm sorry for the inconvenience the bad behaviour caused you. Closing this issue for now, please ping me if you would like to discuss it further.

from elasticsearch.