"Start the CloudWatch Agent on an Amazon EC2 Instance Using the Command Line" section is wrong about amazon-cloudwatch-user-guide HOT 13 CLOSED

You can to run:

# mkdir -p /usr/share/collectd/
# touch /usr/share/collectd/types.db

after you need to run:
# /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c ssm:configuration-parameter-store-name -s

For check the status, run:
/opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -m ec2 -a status
Output:

{
  "status": "running",
  "starttime": "2018-10-12T02:18:16+0000",
  "version": "1.203420.0"
}

from amazon-cloudwatch-user-guide.

I second what @jeffreyctang said. The docs didn't say it clearly it is in fact not using the config.json to read the configuration. It is in fact using the toml file. If you not able to run the agent correctly, the error message will tell you there is error reading the toml file, so it makes us thinking we should put in the toml file in the '-c' option but it is not.

The command sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c file:configuration-file-path is correct where the configuration-file-path actually refer to the config.json got generated from the wizard. But what the docs didn't tell is, the config.json is not being used, it in fact generated another toml file, and the agent will read the toml file. You will end up thinking file:configuration-file-path is not referring to config.json, and troubleshoot it in a completely wrong direction. Due to the script will assume you have collectd, you will get error message like this:

/opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent -schematest -config /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.toml
Configuration validation second phase failed
======== Error Log ========
2020-10-21T08:43:43Z E! [telegraf] Error running agent: Error parsing /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.toml, open /usr/share/collectd/types.db: no such file or directory

You will notice suddenly there is a toml file and not config.json which is supposed to be used. You will then waste hours to troubleshoot that. If the docs could just mention config.json is used to generate toml file, it will be just a few minutes thing to find out the collectd checking is wrong. It wasted me a few hours to finally find this thread. One will wonder since it's aws then it must be our own set up something wrong. No, it's not.

It's 2 years now AWS still haven't solved the collectd checking problem and this misleading docs. (to solve the collectd checking problem, you could either create an empty /usr/share/collectd/types.db or remove collectd section in your config.json )

from amazon-cloudwatch-user-guide.

I am really amazed that this issue has not been fixed for so long? We have to hack our way to a successful install?

from amazon-cloudwatch-user-guide.

@chrisdr I still find this very unclear - after all, it's not like amazon-cloudwatch-agent-ctl is used to start the agent under 'ordinary' circumstances (e.g. building an AMI, it will get started by systemd/upstart/whatever, and that does NOT use amazon-cloudwatch-agent-ctl). If you're going to have such a non-standard/obscured deployment process, it would help to document it.

from amazon-cloudwatch-user-guide.

This cloudwatch agent thing is crap. I really thought that Amazon would make it easier for us to collect logs, and thus charge us all for storage. I am going to destroy another instance in order to try to get detailed monitoring on my app. Terrible UX.

from amazon-cloudwatch-user-guide.

@BruceLampson that doesn't work. It ouptuts a message that says: use sudo amazon-linux-extras install collectd

Interestingly, in the documentation page https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/install-CloudWatch-Agent-commandline-fleet.html there's no mention of collectd, but I found that installing it was a prerequisite to successfully starting the agent.

from amazon-cloudwatch-user-guide.

Hmm, I will double-check with the product team to see what is going on. Thanks for taking time to make the comment, Jeffrey.

from amazon-cloudwatch-user-guide.

I have confirmed that the docs are correct, the amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c ssm:configuration-parameter-store-name -s command does start the agent, provided you include that -s at the end. Creating the TOML file is a by-product of the command. If this command is failing to start the agent for you, something else is wrong. In this case, you should see some error messages in the log files, such as configuration-validation.log

from amazon-cloudwatch-user-guide.

/opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl is an ordinary script (not a binary file), so I was able to open it and find what I was after: service amazon-cloudwatch-agent restart.

I suppose it does no harm to start it via /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c ssm:configuration-parameter-store-name -s but this is pretty unfortunate UX. Tools that need to do start-up configuration should do so via the normal systemd/upstart/systemv unit or service files. Configuration files in /etc should hold the parameters needed for start-up.

None of these deviations were obvious from the docs, hence how I ended up on this ticket. :)

from amazon-cloudwatch-user-guide.

We are trying to install on Windows via Chef cookbooks and it is failing on trying to start the agent. No logs are generated at C:\ProgramData\Amazon\AmazonCloudWatchAgent\Logs, just stack dumps from chef run. If I log on to the box and manually run .\amazon-cloudwatch-agent-ctl.ps1 -a fetch-config -m ec2 -c file:config.json -s the service starts.
Linux instance works perfectly. Any ideas?

Code is:
# Start CloudWatch agent powershell_script 'Start Amazon CloudWatch Agent' do code '.\amazon-cloudwatch-agent-ctl.ps1 -a fetch-config -m ec2 -c file:config.json -s' cwd agent_install_path action :nothing end

Error:
`STDOUT:
STDERR: C:\Users\Administrator\AppData\Local\Temp\chef-script20191022-2880-ej7wfm.ps1

: The running command stopped because the preference variable

"ErrorActionPreference" or common parameter is set to Stop: 2019/10/22

09:44:37 Reading json config file path:

C:\ProgramData\Amazon\AmazonCloudWatchAgent\Configs\file_config.json.tmp`

from amazon-cloudwatch-user-guide.

sudo yum -y install collectd

from amazon-cloudwatch-user-guide.

I've wasted an incredible amount of time troubleshooting this issue. Thanks everyone for the helpful advice in this thread. Really disappointing that Amazon's documentation for the cloudwatch agent is so obtuse...

from amazon-cloudwatch-user-guide.

Not to mention there's a deceptively named /opt/aws/amazon-cloudwatch-agent/bin/start-amazon-cloudwatch-agent which doesn't seem to do absolutely nothing (not output whatsoever).

from amazon-cloudwatch-user-guide.