Code Monkey home page Code Monkey logo

prometheus_client_php's Introduction

A prometheus client library written in PHP

Tests

This library uses Redis or APCu to do the client side aggregation. If using Redis, we recommend running a local Redis instance next to your PHP workers.

How does it work?

Usually PHP worker processes don't share any state. You can pick from four adapters. Redis, APC, APCng, or an in-memory adapter. While the first needs a separate binary running, the second and third just need the APC extension to be installed. If you don't need persistent metrics between requests (e.g. a long running cron job or script) the in-memory adapter might be suitable to use.

Installation

Add as Composer dependency:

composer require promphp/prometheus_client_php

Usage

A simple counter:

\Prometheus\CollectorRegistry::getDefault()
    ->getOrRegisterCounter('', 'some_quick_counter', 'just a quick measurement')
    ->inc();

Write some enhanced metrics:

$registry = \Prometheus\CollectorRegistry::getDefault();

$counter = $registry->getOrRegisterCounter('test', 'some_counter', 'it increases', ['type']);
$counter->incBy(3, ['blue']);

$gauge = $registry->getOrRegisterGauge('test', 'some_gauge', 'it sets', ['type']);
$gauge->set(2.5, ['blue']);

$histogram = $registry->getOrRegisterHistogram('test', 'some_histogram', 'it observes', ['type'], [0.1, 1, 2, 3.5, 4, 5, 6, 7, 8, 9]);
$histogram->observe(3.5, ['blue']);

$summary = $registry->getOrRegisterSummary('test', 'some_summary', 'it observes a sliding window', ['type'], 84600, [0.01, 0.05, 0.5, 0.95, 0.99]);
$summary->observe(5, ['blue']);

Manually register and retrieve metrics (these steps are combined in the getOrRegister... methods):

$registry = \Prometheus\CollectorRegistry::getDefault();

$counterA = $registry->registerCounter('test', 'some_counter', 'it increases', ['type']);
$counterA->incBy(3, ['blue']);

// once a metric is registered, it can be retrieved using e.g. getCounter:
$counterB = $registry->getCounter('test', 'some_counter')
$counterB->incBy(2, ['red']);

Expose the metrics:

$registry = \Prometheus\CollectorRegistry::getDefault();

$renderer = new RenderTextFormat();
$result = $renderer->render($registry->getMetricFamilySamples());

header('Content-type: ' . RenderTextFormat::MIME_TYPE);
echo $result;

Change the Redis options (the example shows the defaults):

\Prometheus\Storage\Redis::setDefaultOptions(
    [
        'host' => '127.0.0.1',
        'port' => 6379,
        'password' => null,
        'timeout' => 0.1, // in seconds
        'read_timeout' => '10', // in seconds
        'persistent_connections' => false
    ]
);

Using the InMemory storage:

$registry = new CollectorRegistry(new InMemory());

$counter = $registry->registerCounter('test', 'some_counter', 'it increases', ['type']);
$counter->incBy(3, ['blue']);

$renderer = new RenderTextFormat();
$result = $renderer->render($registry->getMetricFamilySamples());

Using the APC or APCng storage:

$registry = new CollectorRegistry(new APCng());
 or
$registry = new CollectorRegistry(new APC());

(see the README.APCng.md file for more details)

Using the PDO storage:

$registry = new CollectorRegistry(new \PDO('mysql:host=localhost;dbname=prometheus', 'username', 'password'));
 or
$registry = new CollectorRegistry(new \PDO('sqlite::memory:'));

Advanced Usage

Advanced Histogram Usage

On passing an empty array for the bucket parameter on instantiation, a set of default buckets will be used instead. Whilst this is a good base for a typical web application, there is named constructor to assist in the generation of exponential / geometric buckets.

Eg:

Histogram::exponentialBuckets(0.05, 1.5, 10);

This will start your buckets with a value of 0.05, grow them by a factor of 1.5 per bucket across a set of 10 buckets.

Also look at the examples.

PushGateway Support

As of Version 2.0.0 this library doesn't support the Prometheus PushGateway anymore because we want to have this package as small als possible. If you need Prometheus PushGateway support, you could use the companion library: https://github.com/PromPHP/prometheus_push_gateway_php

composer require promphp/prometheus_push_gateway_php

Development

Dependencies

  • PHP ^7.2 | ^8.0
  • PHP Redis extension
  • PHP APCu extension
  • Composer
  • Redis

Start a Redis instance:

docker-compose up redis

Run the tests:

composer install

# when Redis is not listening on localhost:
# export REDIS_HOST=192.168.59.100
./vendor/bin/phpunit

Black box testing

Just start the nginx, fpm & Redis setup with docker-compose:

docker-compose up

Pick the adapter you want to test.

docker-compose run phpunit env ADAPTER=apc vendor/bin/phpunit tests/Test/
docker-compose run phpunit env ADAPTER=apcng vendor/bin/phpunit tests/Test/
docker-compose run phpunit env ADAPTER=redis vendor/bin/phpunit tests/Test/

Performance testing

This currently tests the APC and APCng adapters head-to-head and reports if the APCng adapter is slower for any actions.

phpunit vendor/bin/phpunit tests/Test/ --group Performance

The test can also be run inside a container.

docker-compose up
docker-compose run phpunit vendor/bin/phpunit tests/Test/ --group Performance

prometheus_client_php's People

Contributors

1player avatar bracki avatar buffcode avatar carusogabriel avatar crazycodr avatar daneiserman avatar dependabot-preview[bot] avatar deyv avatar encero avatar fbhdk avatar hamed-fo avatar ikhabitoff avatar kswzr avatar lkaemmerling avatar mario-f avatar martinssipenko avatar mp3000mp avatar mps-sepetrov avatar ngrewe avatar noeldavies avatar oraoto avatar pluk77 avatar rdohms avatar schnipseljagd avatar seiffert avatar simpod avatar thedeacon avatar tobiasbengtsson avatar ttk avatar vlahanas avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

prometheus_client_php's Issues

Implement URL/DSN parsing for redis connection string (or incorporate predis/predis bundle to handle redis connection)

Currently there is no way to use a URL/DSN to define redis connection parameters, which is problematic in applications that use bundles such as predis or SNC Redis, which allow redis connection parameter definition via a dsn.

This request is to implement an option to define redis connection parameters as either separate values (as it is now in this bundle) OR as a single DSN.

Example of URL parsing used by predis/predis: https://github.com/predis/predis/blob/main/src/Connection/Parameters.php#L69

Allow setting an expiration time for metrics

Correct me if I'm wrong, but as far as I can tell there's no expiration time set on metrics stored in redis. Would it be possible to add this as an option, to prevent the storage gradually filling up?

Very slow usage with large REDIS KEYS count

Hello,

We are using version 2.3 of this client and its been working great until the Redis DB started to balloon. We are writing metrics to REDIS in PHP threads which is causing lots of duplicates and total number of keys. We can see anywhere from 100-200k KEYS. This causes the RenderTextFormat to never return.

We are looking into maybe just flushing the DB every render to keep it small (every 10 sec). Is there a better way to handle this or some way to set generic/non-changing key names instead of:

PROMETHEUS_summary_METRIC_KEYS:panda_requests_trackerall:W10=:value:63bdd2f417f0e4.82673862
PROMETHEUS_summary_METRIC_KEYS:panda_requests_trackerall:W10=:value:63bdd38696a042.04345791
PROMETHEUS_summary_METRIC_KEYS:panda_requests_trackerall:W10=:value:63bdd33908d485.81089573
PROMETHEUS_summary_METRIC_KEYS:panda_requests_trackerall:W10=:value:63bdd488b3f4b7.30254979

So they just get overwritten?

Thanks for help!

Were is the connection to prometheus?

Hi,

Sorry I do not get it, what is this library for?
I assumed it is a client to the prometheus db, but I can not find any configuration to setup the connection to the database server, nor could I find any code with requests to a database.

Is there any documentation to use this library with an actual database?

My requirement is to connect to our PromQL time series database (currently testing VictoriaMetrics ) and get some data / insert new data / delete old data.

missing array type

Fatal error: Declaration of Prometheus\CollectorRegistry::registerGauge(string $namespace, string $name, string $help, $labels = Array): Prometheus\Gauge must be compatible with Prometheus\RegistryInterface::registerGauge(string $namespace, string $name, string $help, array $labels = Array): Prometheus\Gauge in ***/vendor/promphp/prometheus_client_php/src/Prometheus/CollectorRegistry.php on line 12

APC Adapter hangs if APCu is disabled

Hello, i want to discuss possible solutions for "issue" with APC Adapter.

If CollectorResgistry is instantiated with APC adapter and apcu is not enabled, any call to observe metrics will hang in endless loop. For instance in this snippet from updateHistogram.

apcu_fetch and apcu_cas returns false when apcu is disabled, so the !$done condition is never met.

public function updateHistogram(array $data): void {
...
        while (!$done) {
            $old = apcu_fetch($sumKey);
            if ($old !== false) {
                $done = apcu_cas($sumKey, $old, $this->toBinaryRepresentationAsInteger($this->fromBinaryRepresentationAsInteger($old) + $data['value']));
            }
        }
}

Possible solutions from my standpoint:

  1. check apcu_enabled in APC Adapter constructor and throw exception
  2. limit max interation count

I can make the changes myself, just want to make agreement with maintainers on right approach

Story time: our CI pipelines was stuck on tests because of this ( disabled apcu in CLI )

Redis Persistenet Connection Default Value

Hello,
In Redis class the default value for persistent_connections is set to False while in connectToServer method it is compared against null. So persistent connections are made by default which probably is not the desired behavior (according to the default value of persistent_connections).

Add Summary Metric Type

Similar to a histogram, a summary samples observations (usually things like request durations and response sizes). While it also provides a total count of observations and a sum of all observed values, it calculates configurable quantiles over a sliding time window.

A summary with a base metric name of exposes multiple time series during a scrape:

streaming φ-quantiles (0 ≤ φ ≤ 1) of observed events, exposed as {quantile="<φ>"}
the total sum of all observed values, exposed as _sum
the count of events that have been observed, exposed as _count
See histograms and summaries for detailed explanations of φ-quantiles, summary usage, and differences to histograms.

https://prometheus.io/docs/concepts/metric_types/#summary

Validation values

private function encodeLabelValues(array $values): string

if some of $values will be numeric, like status code for example, then it will not work with "false !== ($value = apcu_fetch($key))", because json_encode will return different strings to base64encoder.

i tried this and it works, but i guess it's bad solution. Hope i helped with something.

    //status code to int from string
    if(isset($values[1])){
        $mayBeStatusCode = $values[1];
        $values[1] = is_numeric($mayBeStatusCode) ? (int) $mayBeStatusCode : $mayBeStatusCode;
    }

Make polyfill-apcu a dev dependency

Hi,

thanks for sharing this library!

I am seeing an error PHP message: PHP Warning: Can't preload already declared class APCuIterator in app/vendor/symfony/polyfill-apcu/bootstrap80.php with php 8 and apcu extension installed.

Would it be possible to make symfony/polyfill-apcu a dev-dependency and maybe a composer suggestion?

Best
h

Lock composer development dependecies

It is a general practice to not commit composer.lock files for libraries. It is unfortunate to have broken PR check when some analytics dependency like phpstan is upgraded and has new checks.

Redis missing/invalid meta data triggers fatal error

Hello,
we encounter some issues when updating from the old jimdo/prometheus_client_php to promphp/prometheus_client_php.
For any (so far) unknown reasons, there seem to be some invalid entries existing in redis which will result in a fatal error when trying to parse the non existing __meta data. In our case, collectCounters() is affected, but in general it could affect the other collect...() methods as well.

In https://github.com/PromPHP/prometheus_client_php/blob/main/src/Prometheus/Storage/Redis.php#L608 the meta data is parsed to the json_decode method, in case of non existing meta data this will result in a fatal error due to the strict_types and null being passed as the first string parameter:

$raw = $this->redis->hGetAll(str_replace($this->redis->_prefix(''), '', $key));
$counter = json_decode($raw['__meta'], true);

triggering

PHP Fatal error:  Uncaught TypeError: json_decode() expects parameter 1 to be string, null given in ...

To make the code less error prone i would suggest to ignore entries with invalid __meta data, e.g. like that, maybe with some error logging as well:

foreach ($keys as $key) {
    $raw = $this->redis->hGetAll(str_replace($this->redis->_prefix(''), '', $key));
    if (!isset($raw['__meta'])) {
        // maybe add some error reporting as well?!
        trigger_error(sprintf('prometheus_client_php got invalid raw data from redis for key %s', $key), E_USER_NOTICE);
        continue;
    }
    $counter = json_decode($raw['__meta'], true);
    // ....
}

(similar json_decode($raw['__meta'], true); code exists in collectHistograms() and collectGauges() as well)

Before creating a PR it would be really nice to get some feedback and opinions, regarding this topic as I'm currently not that much into the details of prometheus and redis.

Counters, gauges and histograms init with zero values

Atm u can't create histogram with zero value upfront (for correct grafana increase function to work), because it will increment _count and mess with averages in future. It would be nice to have initialization mechanics (for example, on register method), as discussed in other clients, like here or here

I'm not that good at redis and lua, i've looked into current updateSomething realizations for redis and couln't come up with solution, but if u'll point me in right direction, i think i can try to help

Do not collect default metrics in constructor

Currently, the collector registry can be instructed to collect the default metrics by passing the $registerDefaultMetrics flag to the constructor:

$registry = new CollectorRegistry($adapter, registerDefaultMetrics: true);

The registry will register the default metrics right away by calling a private member method.

Looks like a good idea at first, but producing side-effects in a constructor is considered an anti-pattern for a reason :) (Edit: Sounds kind of condescending, isn't meant that way!) Case in point, when using dependency injection, it may be necessary to type-hint the collector registry as a constructor dependency for, say, a Symfony Console Command. Commands will generally be constructed very early in the application lifecycle, as the console component requires actual instances to parse CLI input.
This means that an application will create instances of all registered commands, inject the collector registry somewhere, which attempts to collect the default metrics... which breaks down if the connection to, say, Redis, can't be established at that point.

This may seem fine for production environments, but it also breaks stuff like artisan discover in Laravel applications if any command requires the Prometheus client as a dependency, both locally or in a CI (where Redis isn't available).


What I'd like to ask for would be to open that registerDefaultMetrics method to the outside, and make it public; that way, implementations could choose the point at which they desire to collect those metrics.

Don't use colon in metric names

All the metrics are namespaced in the library :

https://github.com/PromPHP/prometheus_client_php/blob/master/src/Prometheus/CollectorRegistry.php#L273

However these are for the use of a person writing recording rules on the Prometheus server : https://prometheus.io/docs/practices/rules/ as stated in the documentation https://prometheus.io/docs/practices/rules/, Note: The colons are reserved for user defined recording rules. They should not be used by exporters or direct instrumentation.

prometheus_client_php v2.1.1 RenderTextFormat fails with an array_combine warning

Issue:
When rendering metrics, I encountered this error:
Warning: array_combine(): Both parameters should have an equal number of elements in /var/www/vendor/promphp/prometheus_client_php/src/Prometheus/RenderTextFormat.php:43

Code Excerpt:

$adapter           = new Redis();
$collectorRegistry = new CollectorRegistry($adapter);

$renderer = new RenderTextFormat();
$result   = $renderer->render($collectorRegistry->getMetricFamilySamples());

return new Response($result, 200, ['Content-Type' => RenderTextFormat::MIME_TYPE]);

Stack:

Symfony\Component\Debug\Exception\ContextErrorException: Warning: array_combine(): Both parameters should have an equal number of elements in /var/www/vendor/promphp/prometheus_client_php/src/Prometheus/RenderTextFormat.php:43
Stack trace:
#0 /var/www/vendor/promphp/prometheus_client_php/src/Prometheus/RenderTextFormat.php(26): Prometheus\RenderTextFormat->renderSample(Object(Prometheus\MetricFamilySamples), Object(Prometheus\Sample)) #1 /var/www/src/Controller/Metrics.php(39): Prometheus\RenderTextFormat->render(Array)
#2 /var/www/vendor/symfony/symfony/src/Symfony/Component/HttpKernel/HttpKernel.php(151): BounceX\Core\Controller\Metrics->getMetrics(Object(Symfony\Component\HttpFoundation\Request))
#3 /var/www/vendor/symfony/symfony/src/Symfony/Component/HttpKernel/HttpKernel.php(68): Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object(Symfony/src/Symfony/Component/HttpKernel/Kernel.php(200): Symfony\Component\HttpKernel\HttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#5 /var/www/web/index.php(15): Symfony\Component\HttpKernel\Kernel->handle(Object(Symfony\Component\HttpFoundation\Request))
#6 {main} [] []

Steps to reproduce:
After encountering this error, I restarted both my php-fpm and redis docker instances, and unfortunately, no longer able to reproduce. I don't have any STR.

Stack:

  • prometheus_client_php v2.1.1
  • php-fpm
  • symfony v3.4.2
  • php v7.2

APCU flushing data

When we try to get the data it is showing data upto some level and then starting form the beginning or flushing the data.

Redis storage uses KEYS for metrics collection

KEYS is very expensive command, according to manual:

Warning: consider KEYS as a command that should only be used in production environments with extreme care. It may ruin performance when it is executed against large databases. This command is intended for debugging and special operations, such as changing your keyspace layout. Don't use KEYS in your regular application code. If you're looking for a way to find keys in a subset of your keyspace, consider using SCAN or sets.

I saw that there was an attempt to change KEYS to SCAN and performance dropped.

IMO redis storage should use some kind of list/set to keep track of used metrics.

In our deployment, keys command its responsible for 90%+ load of whole redis instance and we use redis storage ONLY for jobs (APCu for http requests).

Exemplars support

Prometheus have added support for Exemplars, which are a technique to allow linking metrics (like histograms buckets) with tracing.
The golang client library supports them already
https://github.com/prometheus/client_golang/blob/75560f1d20c5e1fd435c0d26b5601843a18dc22b/prometheus/histogram.go#L278

Not really sure if it will go by design here
https://github.com/PromPHP/prometheus_client_php/blob/master/src/Prometheus/Histogram.php

But the idea is to be able to observe and to add the exemplar (which usually contains a traceID value) so you can render it with grafana afterwards and open the trace/span interface related with it.

Prometheus exemplar source code
https://github.com/prometheus/prometheus/blob/main/pkg/exemplar/exemplar.go

Breaking Changes along the way

Hello, I came by this fork migrating a project from using endclothing/prometheus_client_php to latest php 8.1 version. Thanks for your hard work

Along the way, #31 & #53 were merged and considered as adding features and so, bumping minor of the package.

Troubles come from adding method to public interfaces, namely Prometheus\Storage\Adapter. Each of this addition breaks an homemade Storage adapter (with a backend similar to redis but with different limitations).

Bad performance after updating to v2.6.1

After updating to v2.6.1, which introduced #91, we've seen a enormous drop in performance for our collection script. This is interesting, because the merge request is meant to increase performance.

Here's a screenshot from the average response time of our prometheus collection script:
Screenshot 2022-06-17 at 11 50 11

APC engine very slow when APCu contains many keys - O(N^2) performance

I am running the prometheus_client_php code on servers with several million entries in APCu. The use of nested calls to APCUIterator in collectCounters(), collectGauges(), and collectHistograms() is O(N^2), with N being the number of keys stored in APCu. When N is large -- several millions of keys -- this slows the code to a crawl and makes the APC storage engine unusable.

In my staging environment with approximately 2.5 million APCu keys, I tested a call to getMetricFamilySamples() taking approximately 9 seconds to return, due to the above issue, for example. During that time, one CPU core is pegged by the code running this nested pair of loops across all the APCu keys!

I have forked the code and re-implemented portions of the APC storage engine, to use a tree-based storage pattern, essentially a directed acyclic graph, where starting at the root of the tree leads to discovery of all APCu keys that the storage engine is using. This presents a massive speed-up in our environment, since performance is now O(N), where N is the number of (metrics x labels x buckets) being tracked, and is unrelated to the number of keys stored in APCu. Usually N in this case is a couple dozen keys to maybe a few hundred max, so relatively small.

Performance scales MUCH better with this change. In my same staging environment with 2.5M APCu keys, a test call to getMetricFamilySamples() decreased from 9 seconds to 33ms(!). This is fast enough that I can have Prometheus poll metrics every second, if I want.

Before dropping a PR here, I figured I'd open this issue and see if there is any interest in discussing the solution, or if you prefer to see the code right away. The current code passes all phpstan and phpunit tests. Feel free to reply, or if you just want to see the code, let me know and I'll drop a PR.

Add default metrics about the gathering process

ATM all official Prometheus client libraries expose default metrics about the scraping process, like how many RAM was used for the gathering of the metrics, how long did it take?

https://github.com/prometheus/client_golang/blob/master/prometheus/process_collector.go#L63
https://github.com/prometheus/client_python/blob/master/prometheus_client/process_collector.py#L62

Some of the "default" metrics does not make sense in the PHP world, like https://github.com/prometheus/client_golang/blob/master/prometheus/process_collector.go#L101 but the others might make sense: https://github.com/prometheus/client_golang/blob/master/prometheus/process_collector.go#L69

@rdohms what do you think about it?

Support floats in Counters

Hey,

is there a specific reason, that the counter does not support floats? The Gauge for example seems to support floats including the increment methods.

Thanks!

Notice: Undefined index: sum

For some reason, occasionally one of the histograms throws a PHP notice when displaying the metrics.

Not 100% sure this is a bug in your bundle, or the metrics bundle that wraps your bundle in Symfony.

The historgram in question:

pas_request_durations_histogram_seconds_bucket{action="POST-saml_acs",le="0.005"} 0
pas_request_durations_histogram_seconds_bucket{action="POST-saml_acs",le="0.01"} 0
pas_request_durations_histogram_seconds_bucket{action="POST-saml_acs",le="0.025"} 0
pas_request_durations_histogram_seconds_bucket{action="POST-saml_acs",le="0.05"} 0
pas_request_durations_histogram_seconds_bucket{action="POST-saml_acs",le="0.075"} 0
pas_request_durations_histogram_seconds_bucket{action="POST-saml_acs",le="0.1"} 0
pas_request_durations_histogram_seconds_bucket{action="POST-saml_acs",le="0.25"} 2
pas_request_durations_histogram_seconds_bucket{action="POST-saml_acs",le="0.5"} 4
pas_request_durations_histogram_seconds_bucket{action="POST-saml_acs",le="0.75"} 4
pas_request_durations_histogram_seconds_bucket{action="POST-saml_acs",le="1"} 4
pas_request_durations_histogram_seconds_bucket{action="POST-saml_acs",le="2.5"} 4
pas_request_durations_histogram_seconds_bucket{action="POST-saml_acs",le="5"} 4
pas_request_durations_histogram_seconds_bucket{action="POST-saml_acs",le="7.5"} 4
pas_request_durations_histogram_seconds_bucket{action="POST-saml_acs",le="10"} 4
pas_request_durations_histogram_seconds_bucket{action="POST-saml_acs",le="+Inf"} 4

storage: APCu
promphp/prometheus_client_php: v2.6.2
artprima/prometheus-metrics-bundle: v1.14.0

I can not replicate why this happened, just that it happened and resolves itself. However, the fix is small and easy, as per my PR.

BC break in v2.2.2

Hi Folks!

The version v2.2.2 adds a new method wipeStorage() to the Adapter interface.
This breaks all implementations in private libraries, which is hard move for a patch release.

Maybe this method should be removed for all v2 releases and added again in a major v3 release.

greetings!

Redis::ensureOpenConnection should only connect once

Hi,

looks like $this->connectionInitialized = true; is not set at the end of method Redis::ensureOpenConnection after successful calling$this->connectToServer()and setting options. This leads to new connects on each call.

Redis Sentinel?

Is there any guidance on how to configure this library to work with Redis Sentinel?

Our infrastructure makes use of this for high availability reasons, we have 3 separate nodes for this.

Counters not tracked in Redis when using floats

The Lua script for incrementing and tracking counters uses either HINCRBY or HINCRBYFLOAT depending on whether the value to increment by is an integer or a float.

The first time a counter is incremented the key is also added to a set of counters, which is then later used to find all counters.

To check whether a counter was newly created the Lua script compares the new counter value with the given value it was incremented by and if they are equal, considers the counter as new.

For this check the script converts the value to increment by to a number (using Luas tonumber function) but does not do the same for the result of the HINCRBY / HINCRBYFLOAT command.

In case of HINCRYBY the result will already be a number and the comparison will work, but HINCRBYFLOAT returns a string which causes the comparison to return false eventhough the numerical values are the same.

As a result the counter will not be added to the set of counters and can't be retrieved or rendered.

Can I reset a counter on purpose?

I'm counting a metric that resets to zero every month (coinciding with billing for our app). It seems like I don't have a way to purposely reset a counter to zero - is that correct? (Maybe this is a hint that a Gauge makes more sense?).

For context, we have usage based billing and I'm interested in graphing that (resets to zero at the beginning of each month).

Additional Maintainers

Hello everyone whoever reads this,

I'm searching for additional Maintainers for this important project. If you use the client within production and want to help, just write under this issue and will chat a bit about it.

  • Lukas

Wipe storage from registry itself

I have the necessity to be able to wipe either 1) a metric (preferable, as in Golang's client) or 2) the entire storage but I see it's not possible since the only way to wipe data is through the storage.
Unfortunately, though, there is no storage getter either in the registry.

Which solution would you prefer? I can send an MR, if you find one of these features is acceptable.

To give more context about my use case, I have written an exporter that exposes AWS EC2 data written in Go that I am currently porting to PHP.
It periodically fetches data using AWS SDK and each time it has to flush the old data since you cannot just reset new values, as the labels combination might be new, leaving behind obsolete data. Only wiping the entire old data makes sense.

Summary is calculated wrong?

I have the following samples [1.33, 2.33, 2.33].
Summary metric is returned like this:

some_duration_seconds{quantile="0.01"} 1.35
some_duration_seconds{quantile="0.05"} 1.43
some_duration_seconds{quantile="0.5"} 2.33
some_duration_seconds{quantile="0.95"} 2.33
some_duration_seconds{quantile="0.99"} 2.33
some_duration_seconds_sum 5.99
some_duration_seconds_count 3

Compared results with Go prometheus client and it return different result

some_duration_seconds{quantile="0.01"} 1.33
some_duration_seconds{quantile="0.05"} 1.33
some_duration_seconds{quantile="0.5"} 2.33
some_duration_seconds{quantile="0.95"} 2.33
some_duration_seconds{quantile="0.99"} 2.33
some_duration_seconds_sum 5.99
some_duration_seconds_count 3

Also tried some online percentile calculators:

So, I assume there is something wrong on PHP client

Type issue on bucket values

Hey,

I think there is an issue, if you try this :

$registry = new \Prometheus\CollectorRegistry(new Prometheus\Storage\Redis());

$histogram = $registry->getOrRegisterHistogram('test', 'some_histogram', 'it observes', [], [0, 1, 2, 3, 4, 5]);
$histogram->observe(5);

$renderer = new \Prometheus\RenderTextFormat();
$result = $renderer->render($registry->getMetricFamilySamples());

echo $result;

You get PHP Fatal error: Uncaught TypeError: str_replace(): Argument #3 ($subject) must be of type array|string, int given
Here src/Prometheus/RenderTextFormat.php(59)
Because bucket values are "labelized" and labels values are "sanitized".

I found a workaround :

$histogram = $registry->getOrRegisterHistogram('test', 'some_histogram', 'it observes', [], ['0', '1', '2', '3', '4', '5']);

But I think it is not the way it should work, am I right ?

Histogram labels are dropped for redis

Hey there,

not sure if thats how histograms work but with 19ff5d1, all labels were dropped from histogram metrics.

Is that intended and if so, could someone lead me to the prometheus docs where it says that histograms must not have labels?

Thanks in advance.

PS: If thats not intended, I'd be happy to contribute to re-add the labels again.

Redis histogram metadata and metric keys not saved

When observing a histogram, there's a chance the metric metadata and keys are not saved set in Redis.

On https://github.com/PromPHP/prometheus_client_php/blob/master/src/Prometheus/Storage/Redis.php#L188-L191, there's a conditional checking the return value of hIncrByFloat with the value passed in. The only time this check can be true is the first time the metric is observed. Subsequent calls will always return false since increment would always be greater.

Per https://redis.io/commands/incrbyfloat, the precision of the output of hIncrByFloat is fixed at 17. Floats that are being observed with a precision of > 17 (e.g. 0.005) will cause the check to fail.

Since the metadata and metric keys are never saved, the metric never gets rendered.

Code To Reproduce:

        $redis = new Redis([
            'host' => 'localhost',
            'port' => 6379,
            'timeout' => 0.1,
            'read_timeout' => 10,
            'persistent_connections' => false,
            'prefix' => 'PROMETHEUS_' . gethostname(),
        ]);
        $collectorRegistry = new CollectorRegistry($redis);
        $renderer = new RenderTextFormat();

        $histogram = $collectorRegistry->getOrRegisterHistogram(
            'bug_report',
            'not_rendered',
            'Bug report for histograms not rendering'
        );

        $histogram->observe(0.01);
        $metrics = $collectorRegistry->getMetricFamilySamples();

        header('Content-Type: ' . RenderTextFormat::MIME_TYPE);
        echo $renderer->render($metrics);

Implement better interfaces

I had originally submitted this PR to Jimdo library: Jimdo/prometheus_client_php#61

In a general gist it adds better interfacing to the boundaries to allow projects to better handle injection and clean their dependencies.

This one adds an interface to the Registry and the Renderer.

@LKaemmerling is this something you would be open to improving? i would be willing to invest time in this with you.

[Gauge] decBy() should be able to receive a float value

Hi,

Looks like the Prometheus\Gauge::decBy method is forcing the caller to send an int value. Here's the method signature:

/**
 * @param int $value
 * @param string[] $labels
 */
public function decBy(int $value, array $labels = []): void
{
    $this->incBy(-$value, $labels);
}

I think this a mistake, because set() and incBy() accept float values:

/**
 * @param double $value e.g. 123
 * @param string[] $labels e.g. ['status', 'opcode']
 */
public function set(float $value, array $labels = []): void {}

/**
 * @param int|float $value
 * @param string[] $labels
 */
public function incBy($value, array $labels = []): void {}

Am I missing something?
I can create a PR if you want to fix the issue, but I wasn't sure if there was any kind of issues preventing the usage of a flat specifically here.

Thx

BC on RegistryInterface in getMetricFamilySamples() method from 2.7.0

Hello,

I have a custom implementation for Prometheus\RegistryInterface class which since 2.7.0 requires me to implement the method getMetricFamilySamples() with an argument:

PHP Fatal error:  Declaration of Javespi\MyCollectorRegistry::getMetricFamilySamples(): array
must be compatible with Prometheus\RegistryInterface::getMetricFamilySamples(bool $sortMetrics = true): array

It looks like the BC was introduced in this PR:

I am going to try to create a pull-request to solve it without losing the new feature.

Cheers

Question: adding values to a histogram appears to add to the bucket the value belongs to and all larger buckets

I created the following from the example code:

$h = rand(0, 10);
$histogram = $registry->getOrRegisterHistogram(
    'example',
    'the_histogram',
    'A histogram puts your observations in predefined buckets (+Inf is registered when no bucket found)',
    ['type'],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9,]
);
$histogram->observe($h, ['blue']);

When I repeatedly call this code and $h is e.g. 6, the buckets for 6, 7, 8 and 9 are increased

Is this the expected behaviour?

Invalid metrics name are stored to backend but cannot be read by prometheus

Hey there,

I've recently added a metric name containing a - in it. Sadly, the client stores it to the backend and adds it to the meta and now I cannot easily remove the metric from the backend because there is that PROMETHEUS_histogram_METRIC_KEYS key containing all available keys...

So I'd like to feature request some kind of metric name validation so it is impossible to store keys not matching the regex from prometheus:

[a-zA-Z_:][a-zA-Z0-9_:]

https://prometheus.io/docs/concepts/data_model/#metric-names-and-labels

Call to undefined method Illuminate\Support\Facades\Redis::connect()

So installed the package in a Laravel 8.x application and I tried testing by running this part of the example

$adapter = new Redis(config('prometheus.redis'));
$registry = new CollectorRegistry($adapter, false);
$renderer = new RenderTextFormat();
$result = $renderer->render($registry->getMetricFamilySamples());
return response()->json(['data' => $result]);

And I am getting the error
Call to undefined method Illuminate\Support\Facades\Redis::connect()
Which occurs in the package
vendor/promphp/prometheus_client_php/src/Prometheus/Storage/Redis.php:225

How do I fix this?
Thanks

Label values proposal

Agenda

As a developer, I want to send correct label values to prometheus.

Example:

metric name : engine_cpu
label: instance = n1
label: size = "10 inches"
increment: 1

How can I send this metric:

\Prometheus\CollectorRegistry::getDefault()
    ->getOrRegisterCounter('factory', 'engine_cpu', 'factory engine state', ['instance' => 'n1', 'size' => '10 inches']);
    ->inc();

But, I got an error:

Invalid label name: '10 inches'

Because assertValidLabel function validates label values, not label names

public static function assertValidLabel(string $label): void

Proposal

  • accept associative array in labels parameter
  • validate label values, not label names

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.