Code Monkey home page Code Monkey logo

fluent-plugin-cloudwatch-put's Introduction

fluent-plugin-cloudwatch-put

Build Status

Fluentd output plugin to put metric data to AWS CloudWatch.

This plugin for fluentd-0.14.x or later.

Installation

RubyGems

$ gem install fluent-plugin-cloudwatch-put

Bundler

Add following line to your Gemfile:

gem "fluent-plugin-cloudwatch-put"

And then execute:

$ bundle

Plugin helpers

  • inject

  • See also: Fluent::Plugin::Output

Configuration

<match cloudwatch.metric_name>
  @type cloudwatch_put

  <buffer tag, key1>
    path cloudwatch.*.buffer

    flush_interval 1m
  </buffer>

  aws_key_id "#{ENV["AWS_ACCESS_KEY_ID"]}"
  aws_sec_key "#{ENV["AWS_SECRET_ACCESS_KEY"]}"

  region ap-northeast-1

  namespace "Dummy/Namespace"
  metric_name ${tag[1]}
  unit Count
  value_key value

  use_statistic_sets

  <dimensions>
    name method
    value ${key1}
  </dimensions>
</match>

namespace (string) (required)

CloudWatch metric namespace (support placeholder)

metric_name (string) (required)

CloudWatch metric name (support placeholder)

key_as_metric_name (bool) (optional)

Use record key as metric name

Default value: false

unit (string) (required)

CloudWatch metric unit (support placeholder)

value_key (array<string>) (required)

Use this key as metric value

storage_resolution (integer) (optional)

Cloudwatch storage resolution

Default value: 60.

use_statistic_sets (bool) (optional)

If this is true, aggregates record chunk before put metric

<dimensions> section (required) (multiple)

name (string) (required)

Dimension name (support placeholder)

key (string) (optional)

Use this key as dimension value. If use_statistic_sets is true, this param is not supported. Use value

value (string) (optional)

Use static value as dimension value (support placeholder)

<buffer> section (optional) (multiple)

chunk_limit_size (optional)

Default value: 30720.

chunk_limit_records (optional)

Default value: 20.

Configuration for Authentication

aws_key_id (string) (optional)

AWS access key id

aws_sec_key (string) (optional)

AWS secret key.

region (string) (optional)

region name

Default value: us-east-1.

proxy_uri (string) (optional)

URI of proxy environment

<assume_role_credentials> section (optional) (single)

role_arn (string) (required)

The Amazon Resource Name (ARN) of the role to assume

role_session_name (string) (required)

An identifier for the assumed role session

policy (string) (optional)

An IAM policy in JSON format

duration_seconds (integer) (optional)

The duration, in seconds, of the role session (900-3600)

external_id (string) (optional)

A unique identifier that is used by third parties when assuming roles in their customers' accounts.

<instance_profile_credentials> section (optional) (single)

retries (integer) (optional)

Number of times to retry when retrieving credentials

ip_address (string) (optional)

IP address (default:169.254.169.254)

port (integer) (optional)

Port number (default:80)

http_open_timeout (float) (optional)

Number of seconds to wait for the connection to open

http_read_timeout (float) (optional)

Number of seconds to wait for one block to be read

<shared_credentials> section (optional) (single)

path (string) (optional)

Path to the shared file. (default: $HOME/.aws/credentials)

profile_name (string) (optional)

Profile name. Default to 'default' or ENV['AWS_PROFILE']

Copyright

  • Copyright(c) 2017- joker1007
  • License
    • MIT License

fluent-plugin-cloudwatch-put's People

Contributors

abicky avatar joker1007 avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

Forkers

abicky jmmgr

fluent-plugin-cloudwatch-put's Issues

Storage resolution error

Whenever I try using this plugin I get

2017-11-13 11:23:31 +0300 [warn]: #0 failed to flush the buffer. retry_time=5 next_retry_seconds=2017-11-13 11:23:31 +0300 chunk="55dd8f6e3f7d891e1482293cbf73b7db" error_class=ArgumentError error="unexpected value at params[:metric_data][0][:storage_resolution]"

Any help would be appreciated

Use statistic_value

I'm getting the following error while running the plugin

"parameter validator found 3 errors:\n - missing required parameter params[:metric_data][0][:statistic_values][:sum]\n - missing required parameter params[:metric_data][0][:statistic_values][:minimum]\n - missing required parameter params[:metric_data][0][:statistic_values][:maximum]"

getting "The parameter MetricData is required."

I've been getting the following error when using the config below, would you be able to help me out? I'm confused on what value_key should be as it doesnt look to map to anything related to the aws cli command.

I am running this in a container and doing gem install of your plugin, my versions are

2018-07-26 00:40:03 +0000 [info]: starting fluentd-1.2.3 pid=7 ruby="2.4.4"
2018-07-26 00:40:04 +0000 [info]: gem 'fluent-plugin-cloudwatch-put' version '0.2.1'
2018-07-26 00:40:04 +0000 [info]: gem 'fluentd' version '1.2.3'
    @type cloudwatch_put

    <buffer tag>
      flush_interval 20s
    </buffer>

    region us-east-1

    namespace "test-namesapce"
    metric_name ${tag[0]}
    unit Count
    value_key "test"

    storage_resolution 1

    <dimensions>
      name "test"
      value ${tag[0]}
    </dimensions>
  </match>```

wrong timestamp

Current timestamps are sent in local timezone.
In case local time - UTC time > 2 hours, AWS returns error that the timestamp is not within allowed boundaries.
In order to fix this, timestamps must be sent in UTC is8601 format, this way it will fit any timezone.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.