retzkek / chia_exporter Goto Github PK
View Code? Open in Web Editor NEWPrometheus exporter for Chia node metrics
Home Page: https://chia.net
License: GNU Affero General Public License v3.0
Prometheus exporter for Chia node metrics
Home Page: https://chia.net
License: GNU Affero General Public License v3.0
Tip of the tree commit hash f8fd48d
When I run the exporter it doesn't like the JSON response from my node (which is running localhost)
2021/07/15 13:29:33 chia_exporter version 0.5.0
2021/07/15 13:29:33 Connected to node at https://localhost:8555 on mainnet
2021/07/15 13:29:35 error decoding get_plots response: json: cannot unmarshal string into Go struct field PlotFiles.failed_to_open_filenames of type main.PlotData
2021/07/15 13:29:35 Listening on :9133. Serving metrics on /metrics.
using a debugger the json reply
(dlv) p r
*net/http.Response {
Status: "200 OK",
StatusCode: 200,
Proto: "HTTP/1.1",
ProtoMajor: 1,
ProtoMinor: 1,
Header: net/http.Header [
"Content-Type": [
"application/json",
],
"Content-Length": ["187912"],
"Date": [
"Thu, 15 Jul 2021 13:56:40 GMT",
],
"Server": [
"Python/3.8 aiohttp/3.7.4",
],
],
Body: io.ReadCloser(*net/http.cancelTimerBody) *{
stop: net/http.setRequestCancel.func2,
rc: io.ReadCloser(*net/http.bodyEOFSignal) ...,
reqDidTimeout: net/http.(*atomicBool).isSet-fm,},
ContentLength: 187912,
TransferEncoding: []string len: 0, cap: 0, nil,
Close: false,
Uncompressed: false,
Trailer: net/http.Header nil,
Request: *net/http.Request {
Method: "POST",
URL: *(*"net/url.URL")(0xc0002c8600),
Proto: "HTTP/1.1",
ProtoMajor: 1,
ProtoMinor: 1,
Header: net/http.Header [...],
Body: io.ReadCloser(io/ioutil.nopCloser) *(*io.ReadCloser)(0xc0002a8a40),
GetBody: net/http.NewRequestWithContext.func3,
ContentLength: 7,
TransferEncoding: []string len: 0, cap: 0, nil,
Close: false,
Host: "localhost:8560",
Form: net/url.Values nil,
PostForm: net/url.Values nil,
MultipartForm: *mime/multipart.Form nil,
Trailer: net/http.Header nil,
RemoteAddr: "",
RequestURI: "",
TLS: *crypto/tls.ConnectionState nil,
Cancel: <-chan struct {} {
qcount: 0,
dataqsiz: 0,
buf: *[0]struct struct {} [],
elemsize: 0,
closed: 1,
elemtype: *runtime._type {
size: 0,
ptrdata: 0,
hash: 670477339,
tflag: tflagExtraStar (2),
align: 1,
fieldalign: 1,
kind: 25,
alg: *(*runtime.typeAlg)(0xd97270),
gcdata: *1,
str: 38913,
ptrToThis: 273984,},
sendx: 0,
recvx: 0,
recvq: waitq<struct {}> {
first: *sudog<struct {}> nil,
last: *sudog<struct {}> nil,},
sendq: waitq<struct {}> {
first: *sudog<struct {}> nil,
last: *sudog<struct {}> nil,},
lock: runtime.mutex {key: 0},},
Response: *net/http.Response nil,
ctx: context.Context(*context.emptyCtx) ...,},
TLS: *crypto/tls.ConnectionState {
Version: 772,
HandshakeComplete: true,
DidResume: false,
CipherSuite: 4866,
NegotiatedProtocol: "",
NegotiatedProtocolIsMutual: true,
ServerName: "",
PeerCertificates: []*crypto/x509.Certificate len: 2, cap: 2, [
*(*"crypto/x509.Certificate")(0xc0001d0b00),
*(*"crypto/x509.Certificate")(0xc0001d1080),
],
VerifiedChains: [][]*crypto/x509.Certificate len: 0, cap: 0, nil,
SignedCertificateTimestamps: [][]uint8 len: 0, cap: 0, nil,
OCSPResponse: []uint8 len: 0, cap: 0, nil,
ekm: crypto/tls.(*cipherSuiteTLS13).exportKeyingMaterial.func1,
TLSUnique: []uint8 len: 0, cap: 0, nil,},}
Basically the r.Body is empty
118: if err != nil {
119: return fmt.Errorf("error calling %s: %w", endpoint, err)
120: }
121: t := io.TeeReader(r.Body, os.Stdout)
122: t = io.TeeReader(r.Body, ioutil.Discard)
=> 123: if err := json.NewDecoder(t).Decode(result); err != nil {
124: if err != nil {
125: return fmt.Errorf("error decoding %s response: %w", endpoint, err)
126: }
127: }
128: return nil
(dlv) p t
io.Reader (unreadable unknown or unsupported kind: "invalid")
(dlv) p result
interface {}(*main.PlotFiles) *{
FailedToOpen: []main.PlotData len: 0, cap: 0, nil,
NotFound: []main.PlotData len: 0, cap: 0, nil,
Plots: []main.PlotData len: 0, cap: 0, nil,
Success: false,}
The server isn't returning any plot file data, I'm running 1.2.0 on chia.
(Sadly) the chia netspace has gone up a lot the last 12 hours and now doesn't fit into int64 anymore:
2021/05/22 04:51:25 error decoding get_blockchain_state response: json: cannot unmarshal number 9566905519282954240 into Go struct field .blockchain_state.Space of type int64
Also the blockchain_sync_status doesn't get populated but I don't see any error in the exporter log lines..
During testing for #14 I noticed that even if I don't provide a -wallet
flag and so on, the endpoint is still queried since it has a default localhost value. Even if I set the param to an empty value, i.e.: chia_exporter -wallet=''
the query is still attempted:
2021/11/11 14:10:26 error calling get_wallets: Post "/get_wallets": unsupported protocol scheme ""
This means that every endpoint that the user doesn't actually care about for that exporter, is producing an error on every run, which fills up logs quick and makes it harder to debug any real issues.
Ergo, there is no way to enable or disable which endpoints are queried. While on a monolithic setup where all pieces live on the same host this is not a problem, the moment you separate services you'll want to run multiple instances of the exporter with different scopes. For example, my setup right now is one full_node and two harvesters. For the full_node exporter I would want to disable the harvester requests and on the harvester exporters only request the local harvester.
Since it's more user-friendly to default to pulling everything from the localhost I understand wanting to set the default value to match that. What I'd recommend is that setting the string to 'false' or empty would be interpreted as disabling that portion.
TL;DR; for any setup that's more complex than a naive monolithic one, the ability to selectively enable endpoints would be grand.
error decoding get_blockchain_state response: json: cannot unmarshal number 10555548272510144512 into Go struct field .blockchain_state.Space of type int64
I am receiving this message in the logs of image docker for chia_exporter.
Any ideas how to resolve it.
When I query the /metrics endpoint I see all my wallet and balances, but the fingerprint values are missing. I had the issue with chia 1.2.3 and 1.2.5.
It also seems to miss all other wallets (only finding wallet_id=1 and I have many wallets)
chia_wallet_farmed_amount{wallet_fingerprint="",wallet_id="1"} 6.000006000021e+12
# HELP chia_wallet_fee_amount Fee amount amount
# TYPE chia_wallet_fee_amount gauge
chia_wallet_fee_amount{wallet_fingerprint="",wallet_id="1"} 6.000021e+06
# HELP chia_wallet_height Wallet synced height.
# TYPE chia_wallet_height gauge
chia_wallet_height{wallet_fingerprint="",wallet_id="1"} 825447
# HELP chia_wallet_last_height_farmed Last height farmed
# TYPE chia_wallet_last_height_farmed gauge
chia_wallet_last_height_farmed{wallet_fingerprint="",wallet_id="1"} 816184
# HELP chia_wallet_pool_reward_amount Pool Reward amount
# TYPE chia_wallet_pool_reward_amount gauge
chia_wallet_pool_reward_amount{wallet_fingerprint="",wallet_id="1"} 5.25e+12
# HELP chia_wallet_reward_amount Reward amount
# TYPE chia_wallet_reward_amount gauge
chia_wallet_reward_amount{wallet_fingerprint="",wallet_id="1"} 7.5e+11
# HELP chia_wallet_sync_status Sync status, 0=not synced, 1=syncing, 2=synced
# TYPE chia_wallet_sync_status gauge
chia_wallet_sync_status{wallet_fingerprint="",wallet_id="1"} 2
# HELP go_gc_duration_seconds A summary of the pause duration of garbage collection cycles.
Also it is timing out on:
2021/09/07 12:09:42 error calling get_wallet_balance: Post "https://localhost:9256/get_wallet_balance": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
In chia my wallets take some time to answer, I think that timeout is causing me problems. Can I increase it somewhere?
Dear @retzkek ,
thanks for putting together your docker-compose stack chiamon which includes this chia_exporter.
I am currently evaluating this setup which looks promising, but if I am not mistaken, there is no option yet to monitor multiple (n) harvesters.
Any plans to do so in the near future?
Thanks,
Tobias
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.