kemra102 / puppet-cloudwatchlogs Goto Github PK
View Code? Open in Web Editor NEWPuppet module for configuring AWS Cloudwatch Logs on Amazon Linux, Ubuntu, Red Hat & CentOS EC2 instances.
License: BSD 2-Clause "Simplified" License
Puppet module for configuring AWS Cloudwatch Logs on Amazon Linux, Ubuntu, Red Hat & CentOS EC2 instances.
License: BSD 2-Clause "Simplified" License
file { '/var/awslogs/etc/config':
ensure => 'link',
target => '/etc/awslogs/config',
}
result in this error when updating from a previous version of this module:
Notice: /Stage[main]/Cloudwatchlogs/File[/var/awslogs/etc/config]: Not removing directory; use 'force' to override
Notice: /Stage[main]/Cloudwatchlogs/File[/var/awslogs/etc/config]: Not removing directory; use 'force' to override
Error: Could not remove existing file
Error: /Stage[main]/Cloudwatchlogs/File[/var/awslogs/etc/config]/ensure: change from directory to link failed: Could not remove existing file
After fixing that, I then had another error:
This require and this one are only valid on amazon linux, as on other OS's, there is no Package['awslogs']. I've replaced it with a variable that's set based on OS.
#20
I see on the documentation that the streamname
parameter can be used to specify the stream name to use. But how do I specify the log group name?
I believe "datetime_format" is not required and in cases where it's not given, the current system time will be applied to the log.
This template generates a configuration that always expects a datetime_format from a log (even when none is given):
https://github.com/kemra102/puppet-cloudwatchlogs/blob/master/templates/awslogs_log.erb#L2
when it should have an if condition like the "multi_line_start_pattern"
Hello,
I can not understand how the issue of IAM user, wish I could set up a specific User for this service, and that he was not the default of cli aws, is this possible?
Att
We have servers where this module worked fine, and we have servers where the installer fails. Unfortunately, there are two issues here..
If the installer fails, the puppet module doesn't detect this properly. It just moves on merrily trying to start a non-installed Cloud Watch Logs agent every puppet run.
If you have make (and gcc) installed, apparently you need to have "python-devel" (or python-dev in Ubuntu) installed. I realize this is actually a problem with the AWS installer, but I figured I would post it here in case someone else ran into this.
NOTE: If you have make and no gcc, you just get an error that gcc could not be executed (or something like that). Unfortunately "make" is required for some other packages, such as shibboleth :( .
The error that the installer pukes out is completely useless:
Step 2 of 5: Downloading the latest CloudWatch Logs agent bits ... Traceback (most recent call last):
File "awslogs-agent-setup.py", line 1272, in <module>
main()
File "awslogs-agent-setup.py", line 1268, in main
setup.setup_artifacts()
File "awslogs-agent-setup.py", line 813, in setup_artifacts
self.install_awslogs_cli()
File "awslogs-agent-setup.py", line 552, in install_awslogs_cli
subprocess.call([AWSCLI_CMD, 'configure', 'set', 'plugins.cwlogs', 'cwlogs'], env=DEFAULT_ENV)
File "/usr/lib64/python2.7/subprocess.py", line 524, in call
return Popen(*popenargs, **kwargs).wait()
File "/usr/lib64/python2.7/subprocess.py", line 711, in __init__
errread, errwrite)
File "/usr/lib64/python2.7/subprocess.py", line 1327, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
The real error is in the setup log (/var/log/awslogs-agent-setup.log):
Successfully installed virtualenv-15.1.0
/usr/bin/virtualenv
New python executable in /var/awslogs/bin/python
"awslogs-agent-setup.log" [readonly] 195L, 14474C
copying lib/yaml/resolver.py -> build/lib.linux-x86_64-2.7/yaml
copying lib/yaml/scanner.py -> build/lib.linux-x86_64-2.7/yaml
copying lib/yaml/events.py -> build/lib.linux-x86_64-2.7/yaml
copying lib/yaml/emitter.py -> build/lib.linux-x86_64-2.7/yaml
copying lib/yaml/serializer.py -> build/lib.linux-x86_64-2.7/yaml
copying lib/yaml/representer.py -> build/lib.linux-x86_64-2.7/yaml
copying lib/yaml/constructor.py -> build/lib.linux-x86_64-2.7/yaml
copying lib/yaml/cyaml.py -> build/lib.linux-x86_64-2.7/yaml
copying lib/yaml/reader.py -> build/lib.linux-x86_64-2.7/yaml
copying lib/yaml/parser.py -> build/lib.linux-x86_64-2.7/yaml
copying lib/yaml/loader.py -> build/lib.linux-x86_64-2.7/yaml
copying lib/yaml/nodes.py -> build/lib.linux-x86_64-2.7/yaml
copying lib/yaml/composer.py -> build/lib.linux-x86_64-2.7/yaml
copying lib/yaml/tokens.py -> build/lib.linux-x86_64-2.7/yaml
running build_ext
creating build/temp.linux-x86_64-2.7
checking if libyaml is compilable
gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.7 -c build/temp.linux-x86_64-2.7/check_libyaml.c -o build/temp.linux-x86_64-2.7/check_libyaml.o
checking if libyaml is linkable
gcc -pthread build/temp.linux-x86_64-2.7/check_libyaml.o -L/usr/lib64 -lyaml -o build/temp.linux-x86_64-2.7/check_libyaml
building '_yaml' extension
creating build/temp.linux-x86_64-2.7/ext
gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.7 -c ext/_yaml.c -o build/temp.linux-x86_64-2.7/ext/_yaml.o
ext/_yaml.c:4:20: fatal error: Python.h: No such file or directory
#include "Python.h"
^
compilation terminated.
error: command 'gcc' failed with exit status 1
----------------------------------------
Command "/var/awslogs/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip-build-A2hNfz/PyYAML/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-8oCjAK-record/install-record.txt --single-version-externally-managed --compile --install-headers /var/awslogs/include/site/python2.7/PyYAML" failed with error code 1 in /tmp/pip-build-A2hNfz/PyYAML
... it is, for some reason, compiling PyYAML instead of using the binary distribution, and failing because Python.h is not available. Python.h is available in the python development package. (python-devel in CentOS/RHEL or python-dev in Ubuntu) Good times!
~tommy
Hi, there.
When package is installed, directory '/var/awslogs/etc/config' is created.
But when your 'file' (init.pp) runs, it ensures this is a symlink, but generates an error:
Error: Could not remove existing file
Error: /Stage[main]/Cloudwatchlogs/File[/var/awslogs/etc/config]/ensure: change from directory to link failed: Could not remove existing file
I needed to edit your code to run like below.
FROM:
file { '/var/awslogs/etc/config':
ensure => 'link',
target => '/etc/awslogs/config',
}
TO:
file { '/var/awslogs/etc/config':
ensure => 'link',
target => '/etc/awslogs/config',
force => true,
}
My environment is small and that change did not affect anything.
Thanks!
When i try to run your module on Xenial, this error occur:
Error: Could not start Service[awslogs]: Execution of '/usr/sbin/service awslogs start' returned 5: Failed to start awslogs.service: Unit awslogs.service not found.
Error: /Stage[main]/Cloudwatchlogs/Service[awslogs]/ensure: change from stopped to running failed: Could not start Service[awslogs]: Execution of '/usr/sbin/service awslogs start' returned 5: Failed to start awslogs.service: Unit awslogs.service not found.
The plugin fails to correctly install the init script on Centos 7.2. It also fails to correctly install the awslogs-agent-launcher.sh.
This seems more to do with the python installer provided by AWS than with the puppet module.
My workaround was to puppet these two files on after the install, but moving forward this is probably not the best solution.
AmazonLinux2 requires a minor patch in order to use a proper service name.
Hey,
Not exactly sure how this happened, but I installed this module via the forge, got v2.2.1, but this is the declaration for init.pp:
class cloudwatchlogs (
$state_file = $::cloudwatchlogs::params::state_file,
$region = $::cloudwatchlogs::params::region,
$logs = {}
) inherits cloudwatchlogs::params {
This is the top of it in github:
class cloudwatchlogs (
$state_file = $::cloudwatchlogs::params::state_file,
$logging_config_file = $::cloudwatchlogs::params::logging_config_file,
$region = $::cloudwatchlogs::params::region,
$log_level = $::cloudwatchlogs::params::log_level,
$logs = {}
) inherits cloudwatchlogs::params {
Not sure how things get published or how this disconnect happened... any chance for a fix? I'd rather not have to install it via git.
Thanks,
-Dan
hi,
I'm getting invalid parameter errors when running the latest 2.0.0 version.
I had to comment the the following ensure and ensure_newline parameters in the init manifest:
concat { '/etc/awslogs/awslogs.conf':
# ensure => 'present',
owner => 'root',
group => 'root',
mode => '0644',
# ensure_newline => true,
warn => true,
require => File['/etc/awslogs'],
}
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.