Code Monkey home page Code Monkey logo

elixometer's Introduction

Elixometer

A light wrapper around exometer.

Elixometer allows you to define metrics and subscribe them automatically to the default reporter for your environment.

Installation

Add the following to your dependencies in mix.exs:

{:elixometer, "~> 1.5"}

Or to track the master development branch:

{:elixometer, github: "pinterest/elixometer"}

Then, add :elixometer to your applications. That's it!

Configuration

In one of your config files, set up an exometer reporter, and then register it to elixometer like this:

config(:exometer_core, report: [reporters: [{:exometer_report_tty, []}]])
config(:elixometer,
  reporter: :exometer_report_tty,
  env: Mix.env,
  metric_prefix: "myapp")

Metrics are prepended with the metric_prefix, the type of metric and the environment name.

The optional update_frequency key of the :elixometer config controls the interval between reports. By default this is set to 1000 ms in the dev environment and 20 ms in the test environment.

You can use an environment variable to set the env.

config :elixometer, env: {:system, "ELIXOMETER_ENV"}

Some reporters know how to handle multiple metrics reported at the same time. Or perhaps you want to write a custom reporter, and would like to receive all data points for a single metric all at once. This can be achieved by adding the report_bulk: true option to the reporter configuration, and adding bulk_subscribe: true to the elixometer configuration. Like so:

config :exometer_core,
  report: [
    reporters: [
      {My.Custom.Reporter, [report_bulk: true]}
    ]
  ]

config :elixometer,
  # ...
  bulk_subscribe: true

By default, metrics are formatted using Elixometer.Utils.format/2. This function takes care of composing metric names with prefix, environment and the metric type (e.g. myapp_prefix.dev.timers.request_time).

This behaviour can be overridden with a custom formatter module (implementing the Elixometer.Formatter behaviour) by adding the following configuration entry:

config :elixometer, Elixometer.Updater,
  formatter: MyApp.Formatter

A formatting module implements the Elixometer.Formatter behaviour and implements a single function, format as such:

defmodule MyApp.Formatter do
  @behaviour Elixometer.Formatter

  # If you prefer to hyphen-separate your strings, perhaps
  def format(metric_type, name) do
    String.split("#{metric_type}-#{name}", "-")
  end
end

A formatting function can also be used as the configuration entry, provided it follows the same signature as a formatting module:

config :elixometer, Elixometer.Updater,
  formatter: &MyApp.Formatter.format/2

Elixometer uses pobox to prevent overload. A maximum size of message buffer, defaulting to 1000, can be configured with:

config :elixometer, Elixometer.Updater,
  max_messages: 5000

Excluding datapoints subscriptions

By default, adding a histogram adds for example 11 subscriptions ([:n, :mean, :min, :max, :median, 50, 75, 90, 95, 99, 999]). If you would like to restrict which of these you care about, you can exclude some like so:

config :elixometer, excluded_datapoints: [:median, 999]

Metrics

Defining metrics in elixometer is substantially easier than in exometer. Instead of defining and then updating a metric, just update it. Also, instead of providing a list of terms, a metric is named with a period separated bitstring. Presently, Elixometer supports timers, histograms, gauges, counters, and spirals.

Timings may also be defined by annotating a function with a @timed annotation. This annotation takes a key argument, which tells elixometer what key to use. You can specify :auto and a key will be generated from the module name and method name.

Updating a metric is similarly easy:

defmodule ParentModule.MetricsTest do
  use Elixometer

  # Updating a counter
  def counter_test(thingie) do
    update_counter("metrics_test.\#{thingie}.count", 1)
  end

  # Updating a spiral
  def spiral_test(thingie) do
    update_spiral("metrics_test.\#{thingie}.qps", 1)
  end

  # Timing a block of code in a function
  def timer_test do
    timed("metrics_test.timer_test.timings") do
      OtherModule.slow_method
    end
  end

  # Timing a function. The metric name will be [:timed, :function]
  @timed(key: "timed.function") # key will be: prefix.dev.timers.timed.function
  def function_that_is_timed do
    OtherModule.slow_method
  end

  # Timing a function with an auto generated key
  # The key will be "<prefix>.<env>.timers.parent_module.metrics_test.another_timed_function"
  # If the env is prod, the environment is omitted from the key
  @timed(key: :auto)
  def another_timed_function do
    OtherModule.slow_method
  end
end

Additional Reporters

By default, Elixometer only requires the exometer_core package. However, some reporters (namely OpenTSDB and Statsd) are only available by installing the full exometer package. If you need the full package, all you need to do is update your mix.exs to include exometer as a dependency and start it as an application. For example:

def application do
  [
    applications: [:exometer,
    ... other applications go here
    ],
    ...
  ]
end

defp deps do
  [{:exometer_core, github: "Feuerlabs/exometer_core"}]
end

In case a reporter allows for extra configuration options on subscribe, you can configure them in your elixometer config like so:

config(:elixometer,
  ...
  subscribe_options: [{:tag, :value1}])

Custom Reporters

The documentation on custom reporters in exometer is fairly difficult to find, and isn't exactly correct. It is still quite useful though, and can be found here.

Your best bet is to define a module that uses the :exometer_report behaviour, and use @impl on the functions that you add to conform to that behaviour. That will make sure that each function aligns with the behaviour, and that you haven't missed any required ones.

There is one function that is not included as part of the erlang behaviour, but that you may want, which is exometer_report_bulk/3. If and only if you have defined that function in your reporter and you passed report_bulk: true as an option to the reporter, it will pass a list of datapoint/value pairs to your exometer_report_bulk/3.

elixometer's People

Contributors

aerosol avatar alco avatar andyleclair avatar argonus avatar asummers avatar ctreatma avatar cwc avatar dantame avatar dependabot-preview[bot] avatar dependabot[bot] avatar edwardt avatar entone avatar ephe-meral avatar fishcakez avatar jparise avatar lucaspolonio avatar nichochar avatar obmarg avatar pguillory avatar scohen avatar seleem1337 avatar thecodeboss avatar timbuchwaldt avatar zachdaniel avatar zorbash avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

elixometer's Issues

@timed macro reports 1 microsecond

The following code using @timed reports 1 microsecond time. The timed/1 macro seems to work correctly.

defmodule Repro do
  use Elixometer

  def hello do
    :dbg.tracer()
    :dbg.p(:all, :c)
    :dbg.tpl(Elixometer.Updater, :timer, :x)
    Repro.time_me()
  end

  @timed(key: "timedkey")
  def time_me do
    :timer.sleep(1500)
  end
end
iex(1)> Repro.hello
(<0.245.0>) call 'Elixir.Elixometer.Updater':timer(<<"timedkey">>,microsecond,1)
(<0.245.0>) returned from 'Elixir.Elixometer.Updater':timer/3 -> ok
:ok

Accessing predefined exometer Reports, e.g. erlang memory

Hi!

I'm just wondering how I might access predefined exomter reports. I tried to add a predefined section to the config but that didn't work:

# Configure Elixometer Instrumentation
config(:exometer_core,
report: [
  reporters: [
    {:exometer_report_tty, []}
  ]
],
predefined: [
  {
    ~w(erlang memory)a,
    {:function, :erlang, :memory, [], :proplist, ~w(atom binary ets processes total)a},
    []
  },
  {
    ~w(erlang statistics)a,
    {:function, :erlang, :statistics, [:'$dp'], :value, [:run_queue]},
    []
  }
])
config(:elixometer, reporter: :exometer_report_tty,
    env: Mix.env,
    metric_prefix: "bms")

Making private functions overridable is deprecated

We use Module.make_overridable/2 as part module compilation step. This is deprecated for private functions, as show here from our unit test run:

warning: making private functions (secret_timed/0 in this case) overridable is deprecated
  (elixir) lib/module.ex:815: anonymous fn/2 in Module.make_overridable/2
  (stdlib) lists.erl:1338: :lists.foreach/2
  (elixometer) lib/elixometer.ex:135: Elixometer."-MACRO-__before_compile__/2-fun-0-"/2
  (elixir) lib/enum.ex:1229: Enum."-map/2-lists^map/1-0-"/2
  (elixir) lib/enum.ex:1229: Enum."-map/2-lists^map/1-0-"/2

We can easily avoid this warning in our tests by making secret_timed/0 non-private, but we should revisit this approach in general and do one of the following:

  1. Document the fact that we can't decorate private functions.
  2. Change the implementation to something that also support private functions.

i have some issue while runing the phoenix server

** (Mix) Could not compile dependency :meck, "/home/ashu/.mix/rebar compile skip_deps=true deps_dir="/home/ashu/Documents/learn/elixir/elixir_todo_app/_build/dev/lib"" command failed. You can recompile this dependency with "mix deps.compile meck", update it with "mix deps.update meck" or clean it with "mix deps.clean meck"

Dialyzer reports "Function has no local return"

For any function in my application, if I make a call to, say, update_counter or clear_counter, dialyzer will report that my application's function has no local return. Removing the call to update_counter or clear_counter causes the dialyzer warning to go away. Why is this?

Check if we support spirals

The README current states:

Presently, Elixometer supports timers, histograms, gauges, and counters.

But @dantame reports the code may also support spirals.

Unable to install dependencies when adding exometer to deps

Following the docs and trying to add

{:exometer, github: "PSPDFKit-labs/exometer_core"}

to my deps. However, I get "(Mix) Command "git --git-dir=.git checkout --quiet HEAD" failed" when I run mix deps.get:

$ mix deps.get
* Getting elixometer (https://github.com/pinterest/elixometer.git)
remote: Counting objects: 339, done.
remote: Compressing objects: 100% (12/12), done.
remote: Total 339 (delta 0), reused 0 (delta 0), pack-reused 326
Receiving objects: 100% (339/339), 82.48 KiB | 0 bytes/s, done.
Resolving deltas: 100% (176/176), done.
* Getting exometer (https://github.com/PSPDFKit-labs/exometer_core.git)
remote: Counting objects: 628, done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 628 (delta 0), reused 2 (delta 0), pack-reused 626
Receiving objects: 100% (628/628), 585.82 KiB | 0 bytes/s, done.
Resolving deltas: 100% (385/385), done.
* Getting lager (git://github.com/basho/lager.git)
remote: Counting objects: 3496, done.
remote: Total 3496 (delta 0), reused 0 (delta 0), pack-reused 3496
* Getting parse_trans (git://github.com/uwiger/parse_trans.git)
remote: Counting objects: 797, done.
remote: Total 797 (delta 0), reused 0 (delta 0), pack-reused 797
* Getting setup (git://github.com/uwiger/setup.git)
remote: Counting objects: 582, done.
remote: Total 582 (delta 0), reused 0 (delta 0), pack-reused 582
* Getting edown (git://github.com/uwiger/edown.git)
remote: Counting objects: 965, done.
remote: Compressing objects: 100% (16/16), done.
remote: Total 965 (delta 0), reused 16 (delta 0), pack-reused 949
error: pathspec 'HEAD' did not match any file(s) known to git.
** (Mix) Command "git --git-dir=.git checkout --quiet HEAD" failed

Did some spelunking and I see that the setup dependency pulls in edown like so:

{deps, [{edown, ".*", {git, "git://github.com/uwiger/edown.git", "HEAD"}}]}

which I believe mix is interpreting as a ref (hence the error message above). However, I am not really sure what to do about it.

Elixometer.clear_counter/1 is supposed to use configured Formatter

A default formatter Elixometer.Utils is called instead of a configured one:

clear_counter(format(:counters, metric_name))

To fix, Elixometer.Utils should either expose the configured formatter, or implement clear_counter/1.
Exposing formatter is preferable, because there are often situations where an application may need to translate metric names between "Elixometer view" and "exometer view".

Question: is there any way to set tags?

We are planning to use elixometer to send metrics to datadog (with exometer_datadog). I know it's possible with some of datadog's libraries to set tags on metrics. For example, the my_app_api_endpoint metric could have a status_code tag to record what code the method returned, a method tag to record what the HTTP method was, and an endpoint tag to record the endpoint.

Is there a way to record tags when updating a metric, maybe with optional parameters in a call like update_spiral("metrics_test.\#{thingie}.qps", 1)? I looked through the source code, but it was not obvious whether this is supported.

Release version 1.3.0

This issue tracks progress toward tagging version 1.3.0, including a hex.pm release.

  • Remove lager dependency override. We override exometer's lager dependency for OTP 19+ compatibility, and we can't build a hex.pm package until we remove the override. However ...
  • Upgrade exometer_core. exometer_core no longer uses lager (Feuerlabs/exometer_core#66), but that change hasn't made it into a hex.pm release yet.

ArgumentError in runtime

Trying to configure the elixometer
Everything complies without any problem but then trying to submit anything i'm getting this error:

iex(10)>  Elixometer.update_counter "foo", 1
** (ArgumentError) argument error
    (stdlib) gen_fsm.erl:215: :gen_fsm.send_event/2

Details:
OSX 10.11.6
Erlang/OTP 18
Elixir 1.2.6
Phoenix 1.2

# mix.exs
     {:elixometer, "~> 1.2"}, # same result with latest module from master
     {:exometer_core, "~> 1.4", override: true},
     {:lager, "3.0.2", override: true},
     {:hackney, "~> 1.4.4", override: true},

Config:

# Copied from readme, other reporters fail as well
config(:exometer_core, report: [reporters: [{:exometer_report_tty, []}]])
config(:elixometer, reporter: :exometer_report_tty,
  env: Mix.env,
  metric_prefix: "myapp")

Potentially unnecessary dependencies

It would be nice to have only exometer_core as a dependency (instead of the whole exometer) and also get rid of netlink.

P.S.: awesome library in every other aspect, thanks!

Duplicate updates with weird data

Bear with me because I'm not sure if this is due to elixometer, exometer_core, exometer_influxdb, or some combination of the three.

Here's what my config looks like:

config :exometer_core,
  report: [
    reporters: [
      exometer_report_influxdb: [
        protocol: :https,
        host: "influxdb.appcues.net",
        username: System.get_env("INFLUXDB_USER"),
        password: System.get_env("INFLUXDB_PASS"),
        port: 8086,
        db: "metrics",
        timestamping: true
      ]
    ]
  ]

config :elixometer,
    reporter: :exometer_report_influxdb,
    env: Mix.env,
    metric_prefix: "event_ingester"

I then call this once:

update_spiral("web.ping", 1)

When I query the measurement in InfluxDB, it looks like this:

SELECT * FROM event_ingester_spirals_web_ping
time count host one
2016-11-03T20:06:18.995495995Z 1 "65b56de2e2b2"
2016-11-03T20:06:19.76431866Z 1 "65b56de2e2b2"
2016-11-03T20:06:19.768817958Z "65b56de2e2b2" 1
2016-11-03T20:06:20.764425275Z "65b56de2e2b2" 1
2016-11-03T20:06:20.768760131Z 1 "65b56de2e2b2"
2016-11-03T20:06:21.764176139Z 1 "65b56de2e2b2"
2016-11-03T20:06:21.775893812Z "65b56de2e2b2" 1
2016-11-03T20:06:22.763868246Z "65b56de2e2b2" 1
2016-11-03T20:06:22.768669734Z 1 "65b56de2e2b2"
2016-11-03T20:06:23.764132114Z 1 "65b56de2e2b2"

It looks like there are two measurements sent every second and for some reason the value is under count for some and one for others. If I change the measurements to a count, something similar happens with the count being sent in one instance and ms_since_reset being sent in the other instance.

Erlang 19

Anyone have luck getting this to compile with Erlang 19?

While runing "mix ecto.create" or "mix phoenix.server" getting below error

While runing "mix ecto.create" or "mix phoenix.server" getting below error

** (Mix) Could not compile dependency :ranch, "/home/cloud-user/.mix/rebar3 bare compile --paths "/home/cloud-user/myapp/_build/dev/lib/*/ebin"" command failed. You can recompile this dependency with "mix deps.compile ranch", update it with "mix deps.update ranch" or clean it with "mix deps.clean ranch"

And i am using below versions:
[cloud-user@msd-postgresql ~]$ elixir -v
Erlang/OTP 21 [erts-10.0] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]

Elixir 1.7.0 (compiled with Erlang/OTP 19)
[cloud-user@msd-postgresql ~]$

Can you please provide any solution , how to fix it...

Using statsd over a network

I’m having trouble getting Elixometer to work with statsd when statsd is running on a server other than localhost.

Everything works fine when running it locally, but when I have the app running on a different machine than statsd, it doesn’t work. My firewall does allow UDP requests from the app server to port 8125 of the statsd server, so I don't think it's a networking problem.

I can successfully update statsd from the app server with a bash command. I can even successfully update statsd from the app console on my server using :os.cmd. It is only when I go through Elixometer/exometer_core/exometer_report_statsd that nothing happens.

Here is the config that works on my local machine:

config :elixometer,
  reporter: :exometer_report_statsd,
  update_frequency: 5_000,
  env: Mix.env,
  metric_prefix: "myapp"

config :exometer_core, report: [
  reporters: [
    exometer_report_statsd: [
      host: "localhost",
      port: 8125
    ]
  ]
]

And here is the config on the app server:

config :elixometer,
  reporter: :exometer_report_statsd,
  update_frequency: 5000,
  env: Mix.env,
  metric_prefix: "hydrogen"

config :exometer_core, report: [
  reporters: [
    exometer_report_statsd: [
      host: "10.0.0.6",
      port: 8125
    ]
  ]
]

I can use the following bash command to update statsd successfully (I run this from the same server as the app):

echo "deploys.test.myservice:1|c" | nc -w 1 -u 10.0.0.6 8125

I know this may not specifically be Elixometer's fault, but I'm running out of ideas to get this working πŸ˜„

[Question] No env and prod converges to the same value. Invitation to human error?

When people (or mistake in the script) forget to put value for env, it will output the same
value as if "prod" has been put in. It looks like it hides a mistake, and makes is a hard to find error.

Suggest to make env a required settings, not optional, during init for least surprises.

File: elixometer/lib/elixometer.ex

def name_to_exometer(metric_type, name) when is_bitstring(name) do
    config = Application.get_all_env(:elixometer)
    prefix = config[:metric_prefix] || "elixometer"
    base_name = case config[:env] do
                  nil -> "#{prefix}.#{metric_type}.#{name}"    <<<<<<<<<<<<<<<<<<<<<<<
                  :prod -> "#{prefix}.#{metric_type}.#{name}"   <<<<<<<<<<<<<<<<<<<<<<
                  env -> "#{prefix}.#{env}.#{metric_type}.#{name}"
                end

Question: why does elixometer need a GenServer in front of exometer?

What is the rationale behind a process gateway in front of exometer?

I'm scratching my head and trying to figure out why is everything proxied through a single gen server instance.

I can see an ETS table that keeps track of subscriptions/metrics defined and a periodic tick that resets counters. It somewhat justifies the need of having a dedicated process, however am I wrong in thinking that exometer_core does that on its own anyway? πŸ€”

[Question] Each handle cal calls get_all_env for two settings can be moved in init phase and stored in state..

For every subscribe call it is calling application:gen_all_env..., though I suspect subscribe call is not that frequent.

Would moving get_all_env to init phase and store in state be better?
How about getting the config settings before starting the worker. If getting settings error-ed out(I assume that is essential info for worker to work), supervisor then does not need to start worker then restart etc.

File: elixometer/lib/elixometer.ex

 def handle_call({:subscribe, name}, _caller, state) do
      cfg = Application.get_all_env(:elixometer) <-----
      reporter = cfg[:reporter]   # <--- looks like this is relatively constant
      interval = cfg[:update_frequency]  # <--  looks like this is relatively constant

Can't limit the datapoints elixometer will subscribe to.

I've been looking into integrating elixometer with one of our applications that has quite a few histograms in it. Because elixometer subscribes to all the datapoints of a metric, this means I get 11 datapoint outputs for each histogram, when I'm usually only interested in 2 or 3 of them. It's not a massive problem, but it'd be nice if elixometer had some way of customising which datapoints of a metric are subscribed to.

A configurable blacklist of datapoints seems like it'd be the simplest option, if not very flexible.

Happy to put together a PR for this if it sounds like something that would be accepted.

@timed functions return [do: result]

Under Elixir 1.6.0-dev, as well as Elixir 1.5.2 the following code:

defmodule Mod do
  use Elixometer

  @timed(key: "key")
  def my_fun do
    :ok
  end

  def hello do
    IO.puts(inspect my_fun())
  end
end

returns [do: :ok]

iex(1)> Mod.hello
[do: :ok]
:ok

Looks like this line might be to blame.

Timeout on registering a metric

Seeing a genserver timeout via ensure_registered. What's weird is that genserver message ({:subscribe,...}) should only be called when the metric hasn't been registered before, but the metric should be registered because we have plenty of data points for the metric in question within the same session. This starts to happen when the system is under heavy load (and after the system has been running for several hours).

Trace:

GenServer Elixometer.Updater terminating
** (stop) exited in: GenServer.call(Elixometer, {:subscribe, ["my_app_prefix", "timers", "namespace", "of", "my", "module", "function"]}, 5000)
** (EXIT) time out
(elixir) lib/gen_server.ex:924: GenServer.call/3
(elixometer) lib/elixometer.ex:372: Elixometer.ensure_registered/2
(elixometer) lib/updater.ex:108: Elixometer.Updater.do_update/2
(elixir) lib/enum.ex:765: Enum.-each/2-lists^foreach/1-0-/2
(elixir) lib/enum.ex:765: Enum.each/2
(elixometer) lib/updater.ex:45: Elixometer.Updater.handle_info/2
(stdlib) gen_server.erl:637: :gen_server.try_dispatch/4
(stdlib) gen_server.erl:711: :gen_server.handle_msg/6

Any thoughts on what could cause this?

Viewing the Metrics

I have configured the mix.exs and config.exs as per the doc and my application is up and running. I want to know how to view the metrics measured or captured through Elixometer. Please provide me a flow on how to approach on viewing the metrics or configuring them.

CRASH REPORT Process exometer_report_tty with 0 neighbours crashed

I am attempting to use elixometer in a Phoenix app. However, when I try to use update_histogram and the exometer_report_tty, I get the following crash:

10:34:31.086 [error] CRASH REPORT Process exometer_report_tty with 0 neighbours crashed with reason: bad argument in call to erlang:'++'(<<"histograms">>, [95|<<"test">>]) in exometer_report_tty:metric_to_string/1 line 116

I am simply doing Elixometer.update_histogram("test", 1) in a test controller.

I created a sample project.

Environment:

Mac OS X (El Capitain)
Erlang/OTP 19 [erts-8.2.2] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]
Elixir 1.4.2

start with :lager instead of :logger

When application is started, it error on :logger. exometer and other dependency projects uses :lager. Application starts with the built in logger instead.

Supervisor: {local,'Elixir.Logger.Supervisor'}
Context: child_terminated
Reason: normal
Offender: [{pid,<0.183.0>},{id,'Elixir.Logger.ErrorHandler'},{mfargs,{'Elixir.Logger.Watcher',watcher,[error_logger,'Elixir.Logger.ErrorHandler',{true,false,500},link]}},{restart_type,permanent},{shutdown,5000},{child_type,worker}]

#28 (comment)

Elixometer default config does not seem to be pulled in.

The README states:

The optional update_frequency key of the :elixometer config controls the interval between reports. By default this is set to 1000 ms in the dev environment and 20 ms in the test environment.

But update_frequency doesn't seem to have a default for me. If I remove all my config code for elixometer, I get this:

Application.get_all_env(:elixometer)
[included_applications: []]

It does not look like the config of Elixometer is getting pulled into my app.

[Question] Why is there two different function names in `DeclarativeTest` module for testing `@timed` with a guarded function?

πŸ‘‹ I'm relatively new to Elixir and was wondering how the @timed annotation worked, so I investigated the codebase a bit.

I found that the function used to test the @timed annotation with a guard condition, had a very slightly different function name to the one above it - my_other_timed_method2 instead of my_other_timed_method.

When I ran the tests after removing the 2 from the function name, I got this error, but I had expected it to work:

** (ArgumentError) cannot make function my_other_timed_method/1 overridable because it was not defined
    (elixir) lib/module.ex:1006: anonymous fn/2 in Module.make_overridable/2

Is this a limitation of elixometer or of Elixir annotations?

https://github.com/pinterest/elixometer/blob/master/test/elixometer_test.exs#L24-L31

Get metric value by wildcard key

Let's say we have metrics app.dev.spirals.registered.user.us and app.dev.spirals.registered.user.ca - correct me if I'm wrong, but as of now there's no way to get accumulated value for app.dev.spirals.registered.user._ with Elixometer. It is possible to either introduce a new key like app.dev.spirals.registered.user.total or use :exometer.get_values/1 function, which allows wildcards, directly (:exometer.get_values([:app, :dev, :spirals, :registered, :user, :_]). Both are possible to go with, however I think it would be best to implement call to Exometer.get_values/1 via Elixometer to simplify dealing with keys (which should be list of atoms for Exometer) and results parsing (which is a list of all results found for wildcard).

Disable debug output

Each time the tests run I get the following output. Is there a way to disable it? It is very annoying.

Running tests...
11:14:25.246 [error] Supervisor 'Elixir.Logger.Supervisor' had child 'Elixir.Logger.ErrorHandler' started with 'Elixir.Logger.Watcher':watcher(error_logger, 'Elixir.Logger.ErrorHandler', {true,false,500}, link) at <0.270.0> exit with reason normal in context child_terminated
11:14:25.285 [info] Application lager started on node nonode@nohost
11:14:25.317 [info] Setup running ...
11:14:25.318 [info] Directories verified. Res = ok
11:14:25.318 [info] Setup finished processing hooks (Mode=normal)...
11:14:25.318 [info] Application setup started on node nonode@nohost
11:14:25.358 [info] Starting reporters with [{reporters,[{exometer_report_statsd,[{hostname,"localhost"},{port,8125}]}]},{subscribers,[{exometer_report_statsd,[my_app,webapp,resp_time],[min,max,mean,'95','90'],1000,true},{exometer_report_statsd,[my_app,ecto,query_exec_time],[min,max,mean,'95','90'],1000,true},{exometer_report_statsd,[erlang,memory],[atom,binary,ets,processes,total],1000,true},{exometer_report_statsd,[my_app,webapp,resp_count],one,1000,true},{exometer_report_statsd,[my_app,ecto,query_count],one,1000,true},{exometer_report_statsd,[erlang,statistics],run_queue,1000,true}]}]
11:14:25.359 [info] Application exometer_core started on node nonode@nohost
11:14:25.361 [info] exometer_report_statsd([{hostname,"localhost"},{port,8125}]): Starting
11:14:25.370 [info] Application exometer started on node nonode@nohost
11:14:25.378 [info] Application pobox started on node nonode@nohost
11:14:25.398 [info] Application elixometer started on node nonode@nohost
11:14:25.510 [info] Application my_app started on node nonode@nohost
11:14:25.835 [info] Application ex_unit started on node nonode@nohost
11:14:25.847 [info] Application ex_machina started on node nonode@nohost

"Enumerable not implemented for nil" on subscription creation

Following the docs and trying to set up elixometer with influxDB, I get the following error on the first call to Elixometer.update_counter/2. I've not found any configuration changes that get the subscription step to succeed.

I've set up a small example repo displaying the problem: https://bitbucket.org/kmeehl/exotest

** (Protocol.UndefinedError) protocol Enumerable not implemented for nil. This protocol is implemented for: Date.Range, File.Stream, Function, GenEvent.Stream, HashDict, HashSet, IO.Stream, List, Map, MapSet, Range, Stream
            (elixir) lib/enum.ex:1: Enumerable.impl_for!/1
            (elixir) lib/enum.ex:116: Enumerable.reduce/3
            (elixir) lib/enum.ex:1832: Enum.map/2
            (elixometer) lib/elixometer.ex:352: Elixometer.create_subscription/1
            (elixometer) lib/elixometer.ex:309: Elixometer.handle_call/3
            (stdlib) gen_server.erl:636: :gen_server.try_handle_call/4
            (stdlib) gen_server.erl:665: :gen_server.handle_msg/6
            (stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3
    (elixir) lib/gen_server.ex:774: GenServer.call/3
    (elixometer) lib/elixometer.ex:293: Elixometer.ensure_registered/2
    (elixometer) lib/updater.ex:79: Elixometer.Updater.do_update/1
    (elixir) lib/enum.ex:675: Enum."-each/2-lists^foreach/1-0-"/2
    (elixir) lib/enum.ex:675: Enum.each/2
    (elixometer) lib/updater.ex:50: Elixometer.Updater.handle_info/2
    (stdlib) gen_server.erl:616: :gen_server.try_dispatch/4
    (stdlib) gen_server.erl:686: :gen_server.handle_msg/6
Last message: {:mail, #PID<0.279.0>, [{:counter, "Exoteststartcount", 1, nil}], 1, 0}

Elixometer and Logstash

I'm looking for a pragmatic way to send metrics (via UDP) to Logstash with elixometer.
We want to use Logstash to transform logs and store them in Elasticsearch for analysis in Kibana.
As I don't come from an Erlang background, I have a hard time understanding how to utilize reporters and what the relation between elixometer, exometer and lager is.

So far, I've identified the following options:

  1. Make a elixometer reporter that sends metrics formatted as JSON via UDP to Logstash directly. This is the preferred way, but I couldn't figure out how to make the reporter.
  2. Have elixometer output metrics to the Elixir logger, which in turn sends them (like all the logs) to Logstash using logger_logstash_backend. I've made the second part work, but don't know how to achieve the first (tried :exometer_report_tty and lager_logger without success).
  3. Send metrics with elixometer to statsd (as described in this blog article), and then from statsd to Logstash. I haven't tried that yet, as I'd prefer not involving yet another intermediary (as Logstash already is one).

I'd appreciate any hints, especially on making a custom UDP JSON reporter. I think it would be a great addition to elixometer to have that option included, so if I find a way to make it, I'd be happy to send a PR!

Could not compile dependency :setup, "/Users/alan/.mix/rebar3 bare compile --paths

$ iex -S mix phx.server
Erlang/OTP 20 [erts-9.0.4] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

===> Compiling setup
src/setup.erl:166: Warning: export_all flag enabled - all functions will be exported

./rebar skip_deps=true escriptize
make: ./rebar: No such file or directory
make: *** [escriptize] Error 1
===> Hook for compile failed!

** (Mix) Could not compile dependency :setup, "/Users/alan/.mix/rebar3 bare compile --paths "/Users/alan/Code/example-phx-app/_build/dev/lib/*/ebin"" command failed. You can recompile this dependency with "mix deps.compile setup", update it with "mix deps.update setup" or clean it with "mix deps.clean setup"

I had to install the old version of rebar, not rebar3, to make it work:

$ brew install rebar

This might be worth adding to your setup instructions.

[Question] some calls query for the pid other does not..

Other calls is querying for the pid using the name, add_counter is not.

File: elixometer/lib/elixometer.ex

defp add_counter(metric_name, ttl_millis) do
    GenServer.cast(__MODULE__, {:add_counter, metric_name, ttl_millis})   <---
 end
def ensure_subscribed(name) do
    if not metric_subscribed?(name) do
      GenServer.call(Process.whereis(__MODULE__), {:subscribe, name})  <---
    end
 end

Could not compile dependecy :exometer_core

I'm following the instructions in the readme and this happens

Elixir 1.3.2
Phoenix 1.2.1

** (Mix) Could not compile dependency :exometer_core, "/Users/romariolopezc/.mix/rebar3 bare compile --paths "/Users/romariolopezc/Repos/Phoenix/soranus/_build/dev/lib/*/ebin"" command failed. You can recompile this dependency with "mix deps.compile exometer_core", update it with "mix deps.update exometer_core" or clean it with "mix deps.clean exometer_core"

mix deps.compile exometer_core

===> Compiling exometer_core
===> Compiling /Users/romariolopez/Repos/Phoenix/soranus/deps/exometer_core/src/exometer_report.erl failed
/Users/romariolopez/Repos/Phoenix/soranus/deps/exometer_core/src/exometer_report.erl:none: error in parse transform 'lager_transform': {function_clause,
                                             [{lager_transform,
                                               '-walk_ast/2-fun-0-',
                                               [{typed_record_field,
                                                 {record_field,251,
                                                  {atom,251,reporter}},
                                                 {type,251,union,
                                                  [{type,251,module,[]},
                                                   {atom,251,'_'}]}}],
                                               [{file,
                                                 "/Users/romariolopez/Repos/Phoenix/soranus/deps/lager/src/lager_transform.erl"},
                                                {line,62}]},
                                              {lists,map,2,
                                               [{file,"lists.erl"},
                                                {line,1239}]},
                                              {lager_transform,walk_ast,2,
                                               [{file,
                                                 "/Users/romariolopez/Repos/Phoenix/soranus/deps/lager/src/lager_transform.erl"},
                                                {line,62}]},
                                              {compile,
                                               '-foldl_transform/2-anonymous-2-',
                                               2,
                                               [{file,"compile.erl"},
                                                {line,958}]},
                                              {compile,foldl_transform,2,
                                               [{file,"compile.erl"},
                                                {line,960}]},
                                              {compile,
                                               '-internal_comp/4-anonymous-1-',
                                               2,
                                               [{file,"compile.erl"},
                                                {line,315}]},
                                              {compile,fold_comp,3,
                                               [{file,"compile.erl"},
                                                {line,341}]},
                                              {compile,internal_comp,4,
                                               [{file,"compile.erl"},
                                                {line,325}]}]}

** (Mix) Could not compile dependency :exometer_core, "/Users/romariolopez/.mix/rebar3 bare compile --paths "/Users/romariolopez/Repos/Phoenix/soranus/_build/dev/lib/*/ebin"" command failed. You can recompile this dependency with "mix deps.compile exometer_core", update it with "mix deps.update exometer_core" or clean it with "mix deps.clean exometer_core"

Dependencies

defp deps do
    [
        {:meck, github: "eproxus/meck", tag: "0.8.2", override: true},
        {:edown, github: "uwiger/edown", tag: "0.7", override: true},
        {:lager, github: "basho/lager", tag: "2.1.0", override: true},
        {:exometer, github: "pspdfkit-labs/exometer"},
        {:netlink, github: "Feuerlabs/netlink", ref: "d6e7188e", override: true},
    ]
end

The overrides gives me some problem when compiling with other packages (like erlcloud for example).
I get "different specs were given for the meck app".

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.