go-gorm / prometheus Goto Github PK

View Code? Open in Web Editor NEW

132.0 132.0 19.0 236 KB

Collect DB Status or user-defined metrics with Prometheus

Home Page: http://gorm.io/docs/prometheus.html

License: MIT License

Go 100.00%

gorm prometheus

prometheus's People

Contributors

Stargazers

Watchers

Forkers

sdm2345 harunnryd rluisr bohaosun gostool harrywang29 mjudeikis hixio-mh yan234280533 omi-lab duanjiong daihu inginjadipro ross-bryan jangocheng wanzirong leozhantw k4leung4 haifeiwu

prometheus's Issues

Race condition while using dockerized database, gorm and prometheus

Description

I am using prometheus plugin and came accross this problem. Version used is: gorm.io/plugin/prometheus v0.0.0-20210507023802-dc84a49b85d1
In project our testing environment is set up in following way:

creating new gorm connection, on new dockerized database. Each test case creates new container
attach prometheus plugin using db.Use()
Execute test case
Run next test

This connection is fine for 3-5 tests where SELECT/CREATE/UPDATE/DELETE queries are sent. Then the environment is somehow corrupted (either by using global variables) or by not using tear down functions for database connection (defer Close()).
Usage in out project:

        timeZoneLocation := url.QueryEscape(location.String()) // location is time.Location
        dsn := fmt.Sprintf("%s:%s@tcp(%s:%d)/%s?charset=utf8mb4,utf8&sql_mode=ANSI_QUOTES&parseTime=True&loc=%s", user, password, host, port, name, timeZoneLocation)
        db, err = gorm.Open(mysql.Open(dsn), &gorm.Config{
                Logger:         logger.Default.LogMode(loglevel),
                NamingStrategy: &modernTvNamingStrategy{},
        })  
...
        gormPrometheus := prometheus.New(prometheus.Config{
                DBName:          name,
                RefreshInterval: 25, 
                MetricsCollector: []prometheus.MetricsCollector{
                        &prometheus.MySQL{
                                VariableNames: []string{"Threads_running"},
                        },  
                },  
        })  
        err = db.Use(gormPrometheus)
...

Here is a part of stack trace, where race condition occured. Two tests cases are included:

[mysql] 2021/05/22 16:28:00 packets.go:123: closing bad idle connection: EOF

2021/05/22 16:28:00 gorm.io/plugin/[email protected]/mysql.go:48 dial tcp [::1]:50111: connect: connection refused
[0.000ms] [rows:-] SHOW STATUS

2021/05/22 16:28:00 gorm.io/plugin/[email protected]/mysql.go:51
[error] gorm:prometheus query error: dial tcp [::1]:50111: connect: connection refused
--- PASS: TestCreateRecordingTvShowEpisode (62348400.00s)
=== RUN   TestCreateRecordingTvShowEpisodeWithoutSeason
2021/05/08 16:28:00 connecting to db. host = localhost; port = 50118
==================
WARNING: DATA RACE
Read at 0x00c0012aaed0 by goroutine 33:
  time.Time.abs()
      /usr/lib/go/src/time/time.go:447 +0x106
  time.Time.date()
      /usr/lib/go/src/time/time.go:926 +0x4e
  time.Time.Date()
      /usr/lib/go/src/time/time.go:483 +0x409
  log.(*Logger).formatHeader()
      /usr/lib/go/src/log/log.go:112 +0x3ed
  log.(*Logger).Output()
      /usr/lib/go/src/log/log.go:179 +0x1fe
  log.(*Logger).Printf()
      /usr/lib/go/src/log/log.go:191 +0x6e
  gorm.io/gorm/logger.logger.Trace()
      /home/martinv/go/pkg/mod/gorm.io/[email protected]/logger/logger.go:152 +0xd97
  gorm.io/gorm/logger.(*logger).Trace()
      <autogenerated>:1 +0x164
  gorm.io/gorm.(*processor).Execute()
      /home/martinv/go/pkg/mod/gorm.io/[email protected]/callbacks.go:133 +0xd61
  gorm.io/gorm.(*DB).Rows()
      /home/martinv/go/pkg/mod/gorm.io/[email protected]/finisher_api.go:440 +0xc7
  gorm.io/plugin/prometheus.(*MySQL).collect()
      /home/martinv/go/pkg/mod/gorm.io/plugin/[email protected]/mysql.go:48 +0x7c
  gorm.io/plugin/prometheus.(*MySQL).Metrics.func1()
      /home/martinv/go/pkg/mod/gorm.io/plugin/[email protected]/mysql.go:33 +0xb4

Previous write at 0x00c0012aaed0 by goroutine 125:
  [failed to restore the stack]

Goroutine 33 (running) created at:
  gorm.io/plugin/prometheus.(*MySQL).Metrics()
      /home/martinv/go/pkg/mod/gorm.io/plugin/[email protected]/mysql.go:31 +0x267
  gorm.io/plugin/prometheus.(*Prometheus).Initialize.func1()
      /home/martinv/go/pkg/mod/gorm.io/plugin/[email protected]/prometheus.go:76 +0x151
  sync.(*Once).doSlow()
      /usr/lib/go/src/sync/once.go:68 +0x127
  sync.(*Once).Do()
      /usr/lib/go/src/sync/once.go:59 +0x46
  gorm.io/plugin/prometheus.(*Prometheus).Initialize()
      /home/martinv/go/pkg/mod/gorm.io/plugin/[email protected]/prometheus.go:74 +0x206
  gorm.io/gorm.(*DB).Use()
      /home/martinv/go/pkg/mod/gorm.io/[email protected]/gorm.go:436 +0xc1
  git.moderntv.eu/websystem/app-api/internal/database.NewConnection()
      /home/martinv/moderntv/app-api/internal/database/connection.go:66 +0xa0f
  git.moderntv.eu/websystem/app-api/internal/database/db_test_utils.(*DBTestCase).attachDB.func1()
      /home/martinv/moderntv/app-api/internal/database/db_test_utils/connection.go:246 +0x16d
  github.com/cenkalti/backoff/v4.RetryNotifyWithTimer()
      /home/martinv/go/pkg/mod/github.com/cenkalti/backoff/[email protected]/retry.go:55 +0x1b1
  github.com/cenkalti/backoff/v4.RetryNotify()
      /home/martinv/go/pkg/mod/github.com/cenkalti/backoff/[email protected]/retry.go:34 +0xfd
  github.com/cenkalti/backoff/v4.Retry()
      /home/martinv/go/pkg/mod/github.com/cenkalti/backoff/[email protected]/retry.go:28 +0xe5
  github.com/ory/dockertest/v3.(*Pool).Retry()
      /home/martinv/go/pkg/mod/github.com/ory/dockertest/[email protected]/dockertest.go:550 +0x100
  git.moderntv.eu/websystem/app-api/internal/database/db_test_utils.(*DBTestCase).attachDB()
      /home/martinv/moderntv/app-api/internal/database/db_test_utils/connection.go:242 +0x344
  git.moderntv.eu/websystem/app-api/internal/database/db_test_utils.(*DBTestCase).Initialize()
      /home/martinv/moderntv/app-api/internal/database/db_test_utils/connection.go:111 +0x195
  git.moderntv.eu/websystem/app-api/internal/database/db_test_utils.NewDBTestCase()
      /home/martinv/moderntv/app-api/internal/database/db_test_utils/connection.go:65 +0x399
  git.moderntv.eu/websystem/app-api/internal/test_utils.createDbTestCase()
      /home/martinv/moderntv/app-api/internal/test_utils/db.go:129 +0x77
  git.moderntv.eu/websystem/app-api/internal/test_utils.dbTestCaseWithoutLocking()
      /home/martinv/moderntv/app-api/internal/test_utils/db.go:119 +0xca
  git.moderntv.eu/websystem/app-api/internal/test_utils.DbTestCase()
      /home/martinv/moderntv/app-api/internal/test_utils/db.go:112 +0x10b
  git.moderntv.eu/websystem/app-api/internal/content/pvr_manager.TestCreateExistingBundle()
      /home/martinv/moderntv/app-api/internal/content/pvr_manager/bundle_create_test.go:21 +0x88
  testing.tRunner()
      /usr/lib/go/src/testing/testing.go:1259 +0x22f
  testing.(*T).Run·dwrap·21()
      /usr/lib/go/src/testing/testing.go:1306 +0x47

Goroutine 125 (running) created at:
  testing.(*T).Run()
      /usr/lib/go/src/testing/testing.go:1306 +0x726
  testing.runTests.func1()
      /usr/lib/go/src/testing/testing.go:1598 +0x99
  testing.tRunner()
      /usr/lib/go/src/testing/testing.go:1259 +0x22f
  testing.runTests()
      /usr/lib/go/src/testing/testing.go:1596 +0x7ca
  testing.(*M).Run()
      /usr/lib/go/src/testing/testing.go:1504 +0x9d1
  main.main()
      _testmain.go:73 +0x22b
==================

As it can be seen part of the writing stack is missing, but I dont know why it is stripping it off.
By stripping the MetricsCollector part from prometheus config, the race condition is not causing race condition. The problem could either lay in sending EOF by the database to the prometheus plugin or by dockerized setup of gorm database tear down functions.

I'll try to replicate it in playground but atm. I am not able to do it

Add additional labels to the metrics

Describe the feature

Hi,

As far as I know there is no way to define additional labels to attach to the exported metrics. It would be nice to allow the user to explicitly define custom labels to identify where the client metrics come from (eg. DBStats). Please tell me if I'm just missing the configuration that allows to do so. Otherwise, I would be very pleased to contribute in case my request seems relevant to you.

Motivation

Basically the limitation I'm facing at the moment with the current implementation is that when exporting metrics from multiple replicas of the same service, db_name is not enough to identify which metrics belong to which replica. Each replica should export metrics labeled with an identifier to make DBStats replica scoped.

Related Issues

N/A

Thanks in advance

Support postgresql

I hope this library support postgresql soon :)

Repo is missing a license

You can use #10 if it applies. Thanks!

is there a whole example incluing func main ? Many thanks

Describe the feature

is there a whole example incluing func main ? Many thanks

Motivation

Related Issues

Plugin running into errors with postgres

GORM Playground Link

go-gorm/playground#1

Description

Getting this error in the console when trying the plugin with postgres

Continuoiusly

2022/10/17 10:16:47 /home/yashkumarverma/go/pkg/mod/gorm.io/plugin/[email protected]/mysql.go:48 ERROR: unrecognized configuration parameter "status" (SQLSTATE 42704)
[0.449ms] [rows:-] SHOW STATUS

2022/10/17 10:16:47 /home/yashkumarverma/go/pkg/mod/gorm.io/plugin/[email protected]/mysql.go:51
[error] gorm:prometheus query error: ERROR: unrecognized configuration parameter "status" (SQLSTATE 42704)

2022/10/17 10:16:52 /home/yashkumarverma/go/pkg/mod/gorm.io/plugin/[email protected]/mysql.go:48 ERROR: unrecognized configuration parameter "status" (SQLSTATE 42704)
[0.422ms] [rows:-] SHOW STATUS

2022/10/17 10:16:52 /home/yashkumarverma/go/pkg/mod/gorm.io/plugin/[email protected]/mysql.go:51
[error] gorm:prometheus query error: ERROR: unrecognized configuration parameter "status" (SQLSTATE 42704)

Authentication feature of Prometheus pusher

Describe the feature

Support authenticated Prometheus endpoints

Motivation

Prometheus Pusher support authenticatio mechanisms we need to use this gorm feature
https://pkg.go.dev/github.com/prometheus/[email protected]/prometheus/push#Pusher.BasicAuth

Related Issues

实现无关的 metrics

Describe the feature

prometheus golang mysql 实现看到只用到了 gauge，是否可以提供更实现无关的 metrics 支持呢？比如我们已经在使用的 metrics wrapper,几乎无法使用 go-gorm/prometheus 此项目。

当然这样此项目会更名为 go-gorm/metrics 会比较好一些

Motivation

Related Issues

Query duration metrics with user specified label

Describe the feature

A metric likegorm_query_duration_seconds_bucket which takes a label say query_name, now we can have a chain method on each query and specify this label
This metric collects the time taken for a query execution.

Motivation

I was looking at enabling prometheus metrics for my postgres DB, and wanted to capture how long each of my queries are taking to execute.
I can currently see that in postgres gorm logs print how long a query took to run, so i am guessing we have this information and that this can be emitted as a prometheus metric with user specified label.
So instead of measuring the duration myself and emitting prometheus metrics in the application, i thought gorm can do that easily by adding a chain method for this.

I don't have much idea about the architecture of code and so not sure if this would be simple change or something that does not fit well with the structure, but wanted to see if this is something that would be a good addition

Related Issues

Why not make goroutine stoppable / cancellable?

Your Question

This library start multiple process without a way to stop them using the context.Context and/or a Finalize() / Close() function(s).
Are there any reasons? Could this be added?

The document you expected this should be explained

I would expect every loop with goroutine to support context done, like:

select {
	case <-time.After(time.Duration(p.Config.RefreshInterval) * time.Second):
		// LOGIC
	case <-ctx.Done():
		return
}

And for the server to store it and expose the Close function.

Expected answer

Hopefully yes and I or someone else can do it.

Promtool check metrics reports error in metric name

Description

Promtool is complaining about non-conformity with OpenMetrics standards.

curl -sS http://api:8081/metrics | promtool check metrics
gorm_dbstats_wait_count non-histogram and non-summary metrics should not have "_count" suffix

Meanwhile, I used the following workaround to disable this collector:

// workaround for
// prom tool is complaining about this particular field:
dbStatsValue := reflect.ValueOf(*dbStatsPrometheus.DBStats)
field := dbStatsValue.FieldByName("WaitCount")
fmt.Println(field, field.String())
prometheus.Unregister(field.Interface().(prometheus.Gauge))

Please Create a Semver Release :D

^ what's on the title.

Motivation is dependency management.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.