wffls / waffles Goto Github PK

Hello,

I'm starting to use waffles to set up a personal server. Thank you for sharing it, I really enjoy its simplicity.

The issue I'm having is that file and directory resources are always considered to be out-of-date when the permissions are prepended a zero (as usual in C to denote octals, and accepted by chmod).

For example:

stdlib.file /tmp/abc --mode 0755

The issue is caused by stat which returns unprefixed octals, and string instead of arithmetic comparison. The following patch seems to help:

diff --git i/lib/stdlib/resources/directory.sh w/lib/stdlib/resources/directory.sh
index 7d10a51..8f1fb91 100644
--- i/lib/stdlib/resources/directory.sh
+++ w/lib/stdlib/resources/directory.sh
@@ -68,7 +68,7 @@ function stdlib.directory.read {
   stdlib.split "$_stats" ':'
   _owner="${__split[0]}"
   _group="${__split[1]}"
-  _mode="${__split[2]}"
+  _mode="0${__split[2]}"
   _type="${__split[3]}"

   if [[ $_type != "directory" ]]; then
@@ -87,7 +87,7 @@ function stdlib.directory.read {
     return
   fi

-  if [[ ${options[mode]} != $_mode ]]; then
+  if [[ ${options[mode]} -ne $_mode ]]; then
     stdlib_current_state="update"
     return
   fi
diff --git i/lib/stdlib/resources/file.sh w/lib/stdlib/resources/file.sh
index 22145f6..3d40223 100644
--- i/lib/stdlib/resources/file.sh
+++ w/lib/stdlib/resources/file.sh
@@ -74,7 +74,7 @@ function stdlib.file.read {
   stdlib.split "$_stats" ':'
   _owner="${__split[0]}"
   _group="${__split[1]}"
-  _mode="${__split[2]}"
+  _mode="0${__split[2]}"
   _type="${__split[3]}"
   _md5=$(md5sum "${options[name]}" | cut -d' ' -f1)

@@ -102,7 +102,7 @@ function stdlib.file.read {
     return
   fi

-  if [[ ${options[mode]} != $_mode ]]; then
+  if [[ ${options[mode]} -ne $_mode ]]; then
     stdlib_current_state="update"
     return
   fi

I think it would be better if users immediately saw how waffles is used by it's developers.

I'd like Waffles to be able to do a pull-based deployment. In this scenario, a Waffles "client" or "agent" contacts a central Waffles server to receive its configuration.

The benefits of this are that nodes behind a firewall or NAT can easily access a publicly accessible Waffles server.

I feel the biggest hurdle to this feature is going to be implementing a simple yet secure authentication system. It'd be best if authentication could somehow be done outside of Waffles.

Create a way to easily manage sudo configuration.

stdlib.useradd is able to configure a user to have full passwordless sudo access, but that can be dangerous. There should be an easy way to manage individual commands that a user can run via sudo.

It might be useful or interesting to have Waffles be able to generate the raw underlying shell commands. The final output could be a single executable script with no outside dependencies.

Thoughts:

• How to handle static files / assets?
• Should the output be base64 (or similar) encoded?
• If a profile contains conditionals, should the entire conditional be included? Or just the path taken when the script was generated?

Each resource has a large amount of repeated boilerplate code. This should be reduced in order to:

1. Reduce bugs (833c72a)
2. Enable more robust features by only modifying a single area.

The git repo resource would be able to manage... git repositories.

Currently, Waffles does not support modules, or the ability to make configuration scripts so generalized that they can be used by a large audience. However, I've always said that profiles and scripts can be re-used internally. This includes common scripts such as setting an apt proxy, adding standard users, etc.

Having Waffles be able to pull a script from a git repo has a few benefits:

• The script can be versioned and locked down to a single version
• Variations of the script could be branched for dev, prod, etc
• Basically all the benefits of storing code in a repository

However, scripts are part of profiles and profiles can also include static files. How could Waffles easily handle this?

The iptables rule resources work, but do not provide any type of persistence.

An augeas-based httpd resource would be very useful for managing Apache configuration files.

A cursory search on Google shows that the httpd lens may be a bit awkward to work with, so the Waffles resource might have to add some assistance -- similar to the JSON resources.

https://github.com/hercules-team/augeas/blob/master/lenses/httpd.aug

The main reason I began exploring using Bash and core utils as a configuration management tool is because I admire the Unix philosophy. I've assumed that by building atomic resources, they could automatically be used to compose more complex resources. A simple example of this is with the sudoers.cmd resource.

However, this difficult to achieve with the way resources are currently designed. Discussions in #175 and #180 give details as to where this begins to cause problems.

This issue will serve as an area for discussion about how resources could be redesigned to enable them to be completely independent and atomic, thus allowing them to be called by other resources.

I'm not a fan of using templates to build configuration files. The main reason is that I feel templates provide only a "one-way street" for data: you can use data to build templates but you cannot use the same system to pull data out of a template.

However, I wonder if there are good uses for templates that I am overlooking.

I've written a mysql_user and mysql_grant resource for Terraform to go along with the existing mysql_database resource. If those get merge, I might remove the MySQL resources from Waffles.

On a RedHat system apt.pkg, waffles fails with:

.../waffles/resources/apt_pkg.sh: line 57: apt-cache: command not found
apt.pkg error: No such package: sudo

I would expect something like

apt.pkg error: resource dependency not installed: apt-cache

I would like to create/manage a symbolic link.

An option could be to allow a special state in the stdlib.file, as in Puppet:

stdlib.file --name /tmp/aaa --state link --target /tmp/bbb

though a special resource stdlib.link might be simpler. What do you think?

A yum resource for RPM-based distributions would be nice.

A config option to specify the site directory exists, but it's mostly ignored right now. This is mainly due to the rsync push-based deployment. It would be nice to be able to have the site directory separate from the main Waffles directory/repo and still be able to do a simple/clean rsync bundle.

I'd prefer not to build a staged Waffles destination directory which would be used as the source for the rsync directory because large static files may exist and copying would use up disk space and/or slow down the push. But I'm open to it if it's the only way.

The stdlib.array_* functions that modify an array are acting strange in the following context:

x=(foo bar baz)
foo=$(stdlib.array_pop x) # foo
bar=$(stdlib.array_pop x) # foo
baz=$(stdlib.array_pop x) # foo

However, the following works:

x=(foo bar baz)
foo=$(stdlib.array_pop x) # foo
stdlib.array_pop x
bar=$(stdlib.array_pop x) # bar
stdlib.array_pop x
baz=$(stdlib.array_pop x) # baz

Augeas provides a RabbitMQ lens to help parse the Erlang-based RabbitMQ configuration files.

http://augeas.net/docs/references/lenses/files/rabbitmq-aug.html

To me, it seems these are better named os.group and os.user.

os.groupadd  --name "ftp" --state absent

is considerably less readable than

os.group  --name "ftp" --state absent

Similar to #1, an Augeas-based nginx resource would be very useful.

http://augeas.net/docs/references/lenses/files/nginx-aug.html

A user asked about the missing Augeas resources in the v.30 release.

v.30 stripped a lot of functionality away from Waffles in an attempt to make it a very small "core". Though (incorrectly) not documented, the Augeas resources were also removed. This was due to the ambiguity that the Augeas resources brought: since it's technically possible to write an Augeas resource for any type of configuration, there was no clear line between what Augeas should control and what another tool should.

However, bringing back the augeas.generic resource sounds like a good idea. It can be used to manipulate configuration files that Augeas supports.

I'm opening this issue as a placeholder until I make the commit that reintroduces the resource.

Rabbit Hole is a complete Go library for managing RabbitMQ resources. I'm going to look into making them Terraform resources and then possibly remove the existing RabbitMQ Waffles resources.

In some situations, it might be useful to have access to the using profile's name from within a resource implementation. For example, one might combine the resource name with the using profile to set the default path for some file in the profile's files:

# This is the wordpress profile.  We have a backported .deb we want
# to install via a dpkg.deb resource.

dpkg.deb --name waffles --version 1.0 --deb "${WAFFLES_SITE_DIR}/profiles/wordpress/files/wordpress_1.0_amd64.deb";

# It would be easier this way, where the resource derives the package path from the profile, the package name and version.
dpkg.deb --name waffles --version 1.0;

The example might be not too clear; please, ask for clarification if needed.

Puppet offers that as the $caller_module_name variable. It looks to me that caller (http://wiki.bash-hackers.org/commands/builtin/caller) could be used to retrieve the name.

As discussed in #175, file_ini.sh and dnf_repo.sh both call waffles.noop. This is would be better done in functions/ini_file.sh.

Create a resource that will manage packages installed via pip.

Currently, the --components option must be non-empty, as it is a __required__ option. This prevents specifying componentless sources such as the following (please, see https://wiki.debian.org/RepositoryFormat#Flat_Repository_Format):

   deb uri file:///some/directory/

The --components option should probably be made optional.

For some reason, I haven't yet gotten this correct -- or at least in a way I'm happy with.

Previously, quoting in conditionals was an inconsistent mess. This was fixed in cf28cb3 and 3ffae1e. However, I think things need changed around.

Bash's [[ does not require either side of the conditional to be quoted, but not quoting things in Bash is a known hazard. To keep things safe and consistent with everything else outside of [[, I figured everything will be quoted.

But this has begun to look a little strange to me. Having:

[[ "$foo" == "bar" ]]

Feels like

if "#{foo}" == "bar"

So I am thinking to apply the following rules:

1. Variables in [[ will not be quoted.
2. Strings in [[ will be quoted.
3. Regex's in [[ will not be quoted.

I want to include a sites-example directory that contains various example deployments using Waffles.

Should Waffles have meta-resources that abstract underlying resources?

For example, right now Waffles only has the apt resource to install packages. If a yum resource is created, should the user have to create separate roles for apt and yum-based nodes? Or should they be able to have a single node that uses a package resource and Waffles will intelligently determine whether to use apt or yum?

A lot of the resources utilize stdlib.mute to check the state of the given resource. The problem is that stdlib.mute prints with info so it's visible during every run of Waffles. It would be cleaner / quieter if nothing was printed if no changes were needed. Therefore, stdlib.mute should be modified to use stdlib.debug.

The --destination parameter should not be required when specifying --state absent.

This is a debugging effort for the waffles code.

• "errexit" is equivalent to set -e and makes bash exit on any unexpected error.
• "nounset" is equivalent to set -u and makes bash exit on usage of an unset variable.

These settings would make it harder for bugs to sneak into the scripts. The actual setting may be left to the user.

I guess the nounset option doesn't require any work, and if it does it will be mostly bugs. The errexit requires more work, but this workaround seems very manageable.

The meaning of --source and --destination seems non-intuitive. In my mind, source ~= origin and destination ~= target, but the resource is implemented in the opposite manner (--destination is the link file and --source is the target). Of course, this is all very subjective.

Would it be possible/make sense to switch them to --name for the link file, and --target (or --source, for consistency with os.file)?

Apologies for bringing this so late.

The standard library is a collection of core components in Waffles. Naming them stdlib makes sense from a development and internal point of view, but not a lot of sense externally. Therefore, the components of stdlib should be renamed.

For example:

• stdlib.file => os.file
• stdlib.apt => pkg.apt
• stdlib.debconf = > pkg.debconf