Unable to store more than 2 weeks of history about homeassistant-addon-victoriametrics HOT 4 CLOSED

First I wanted to thank the maintainer and original inspiration for creating this addon.

I don't know or understand VM very well and are convinced that the issue is user error as I'm unable to find anyone with similar issues. My issue is retention, I cannot store data for more than 2 weeks before the data disappears. I went through the documentation on VM's site, but I don't know if I just need to increase the retention (currently 99y) or if I also need to add some sort of setting to move/process short term data into long term? This may be best suited for VM support? Any help is appreciated.

Addon settings:

retention: 99y additionalArguments: "" enableHTTPAuth: false username: "" password: "" enablePrometheusScrape: false prometheusScrapeHTTPS: false prometheusScrapeInterval: 20s prometheusScrapeTimeout: 15s longelivedToken: "" homeassistantUrl: ""

HA settings:

influxdb: #token: abc1234 api_version: 1 host: 8f49de54-victoria-metrics.local.hass.io port: 8428 max_retries: 3 password: homeassistant username: homeassistant database: homeassistant ssl: false verify_ssl: false tags: source: HA measurement_attr: entity_id default_measurement: state tags_attributes: - friendly_name - unit_of_measurement

Hey @borrelan ,

There is nothing to do from VictoriaMetrics side if you have set retention: 99y. If this value is passed by the addon incorrectly or if it is overridden somewhere else - then there may be such a problem.

You can try to check retention period via curl query to VictoriaMetrics host, for example

curl <victoriametrics-ip-here>:8428/flags
-graphiteListenAddr=":2003"
-httpListenAddr=":8428"
-influxListenAddr=":8089"
-opentsdbListenAddr=":4242"
-retentionPeriod="99y"
-storageDataPath="/storage"
-vmalert.proxyURL="http://vmalert:8880"

or by checking VictoriaMetrics logs:

docker compose logs victoriametrics
victoriametrics  | 2024-01-22T17:00:03.614Z	info	VictoriaMetrics/lib/logger/flag.go:12	build version: victoria-metrics-20231212-224619-tags-v1.96.0-0-g304fe0565
victoriametrics  | 2024-01-22T17:00:03.614Z	info	VictoriaMetrics/lib/logger/flag.go:13	command-line flags
victoriametrics  | 2024-01-22T17:00:03.614Z	info	VictoriaMetrics/lib/logger/flag.go:20	  -graphiteListenAddr=":2003"
victoriametrics  | 2024-01-22T17:00:03.614Z	info	VictoriaMetrics/lib/logger/flag.go:20	  -httpListenAddr=":8428"
victoriametrics  | 2024-01-22T17:00:03.614Z	info	VictoriaMetrics/lib/logger/flag.go:20	  -influxListenAddr=":8089"
victoriametrics  | 2024-01-22T17:00:03.614Z	info	VictoriaMetrics/lib/logger/flag.go:20	  -opentsdbListenAddr=":4242"
victoriametrics  | 2024-01-22T17:00:03.614Z	info	VictoriaMetrics/lib/logger/flag.go:20	  -retentionPeriod="99y"
victoriametrics  | 2024-01-22T17:00:03.614Z	info	VictoriaMetrics/lib/logger/flag.go:20	  -storageDataPath="/storage"

from homeassistant-addon-victoriametrics.

First I wanted to thank the maintainer and original inspiration for creating this addon.
I don't know or understand VM very well and are convinced that the issue is user error as I'm unable to find anyone with similar issues. My issue is retention, I cannot store data for more than 2 weeks before the data disappears. I went through the documentation on VM's site, but I don't know if I just need to increase the retention (currently 99y) or if I also need to add some sort of setting to move/process short term data into long term? This may be best suited for VM support? Any help is appreciated.
Addon settings:
retention: 99y additionalArguments: "" enableHTTPAuth: false username: "" password: "" enablePrometheusScrape: false prometheusScrapeHTTPS: false prometheusScrapeInterval: 20s prometheusScrapeTimeout: 15s longelivedToken: "" homeassistantUrl: ""
HA settings:
influxdb: #token: abc1234 api_version: 1 host: 8f49de54-victoria-metrics.local.hass.io port: 8428 max_retries: 3 password: homeassistant username: homeassistant database: homeassistant ssl: false verify_ssl: false tags: source: HA measurement_attr: entity_id default_measurement: state tags_attributes: - friendly_name - unit_of_measurement

Hey @borrelan ,

There is nothing to do from VictoriaMetrics side if you have set retention: 99y. If this value is passed by the addon incorrectly or if it is overridden somewhere else - then there may be such a problem.

You can try to check retention period via curl query to VictoriaMetrics host, for example

curl <victoriametrics-ip-here>:8428/flags
-graphiteListenAddr=":2003"
-httpListenAddr=":8428"
-influxListenAddr=":8089"
-opentsdbListenAddr=":4242"
-retentionPeriod="99y"
-storageDataPath="/storage"
-vmalert.proxyURL="http://vmalert:8880"

or by checking VictoriaMetrics logs:

docker compose logs victoriametrics
victoriametrics  | 2024-01-22T17:00:03.614Z	info	VictoriaMetrics/lib/logger/flag.go:12	build version: victoria-metrics-20231212-224619-tags-v1.96.0-0-g304fe0565
victoriametrics  | 2024-01-22T17:00:03.614Z	info	VictoriaMetrics/lib/logger/flag.go:13	command-line flags
victoriametrics  | 2024-01-22T17:00:03.614Z	info	VictoriaMetrics/lib/logger/flag.go:20	  -graphiteListenAddr=":2003"
victoriametrics  | 2024-01-22T17:00:03.614Z	info	VictoriaMetrics/lib/logger/flag.go:20	  -httpListenAddr=":8428"
victoriametrics  | 2024-01-22T17:00:03.614Z	info	VictoriaMetrics/lib/logger/flag.go:20	  -influxListenAddr=":8089"
victoriametrics  | 2024-01-22T17:00:03.614Z	info	VictoriaMetrics/lib/logger/flag.go:20	  -opentsdbListenAddr=":4242"
victoriametrics  | 2024-01-22T17:00:03.614Z	info	VictoriaMetrics/lib/logger/flag.go:20	  -retentionPeriod="99y"
victoriametrics  | 2024-01-22T17:00:03.614Z	info	VictoriaMetrics/lib/logger/flag.go:20	  -storageDataPath="/storage"

@denisgolius
Here is what the HA addon I have installed is doing.
I believe I'm going to switch to a separate docker container and move on as the data folder is hard coded and is within the container and not stored externally.
Advice on path forward is of course welcomed

curl 172.30.32.1:8428/flags
-retentionPeriod="99y"
-storageDataPath="/share/victoria-metrics-data"

Addon Logs

s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service legacy-services: starting
s6-rc: info: service legacy-services successfully started
[13:05:24] INFO: Starting Victoria Metrics with retention set to 99y
[13:05:24] INFO: Starting Victoria Metrics with args set to 
2024-01-24T18:05:24.266Z	info	VictoriaMetrics/lib/logger/flag.go:12	build version: victoria-metrics-20231002-214434-tags-v1.94.0-0-gf13a96f42
2024-01-24T18:05:24.266Z	info	VictoriaMetrics/lib/logger/flag.go:13	command-line flags
2024-01-24T18:05:24.266Z	info	VictoriaMetrics/lib/logger/flag.go:20	  -retentionPeriod="99y"
2024-01-24T18:05:24.266Z	info	VictoriaMetrics/lib/logger/flag.go:20	  -storageDataPath="/share/victoria-metrics-data"
2024-01-24T18:05:24.266Z	info	VictoriaMetrics/app/victoria-metrics/main.go:70	starting VictoriaMetrics at ":8428"...
2024-01-24T18:05:24.266Z	info	VictoriaMetrics/app/vmstorage/main.go:108	opening storage at "/share/victoria-metrics-data" with -retentionPeriod=99y
2024-01-24T18:05:24.408Z	info	VictoriaMetrics/lib/memory/memory.go:42	limiting caches to 40341331968 bytes, leaving 26894221312 bytes to the OS according to -memory.allowedPercent=60
2024-01-24T18:05:24.474Z	info	VictoriaMetrics/lib/storage/storage.go:933	discarding /share/victoria-metrics-data/cache/curr_hour_metric_ids, since it contains outdated hour; got 473734; want 473922
2024-01-24T18:05:24.474Z	info	VictoriaMetrics/lib/storage/storage.go:933	discarding /share/victoria-metrics-data/cache/prev_hour_metric_ids, since it contains outdated hour; got 473733; want 473921
2024-01-24T18:05:25.242Z	info	VictoriaMetrics/lib/storage/storage.go:894	discarding /share/victoria-metrics-data/cache/next_day_metric_ids_v2, since it contains data for stale date; got 19738; want 19746
2024-01-24T18:05:28.218Z	info	VictoriaMetrics/app/vmstorage/main.go:122	successfully opened storage "/share/victoria-metrics-data" in 3.952 seconds; partsCount: 124; blocksCount: 235196; rowsCount: 279501699; sizeBytes: 304777947
2024-01-24T18:05:28.316Z	info	VictoriaMetrics/app/vmselect/promql/rollup_result_cache.go:114	loading rollupResult cache from "/share/victoria-metrics-data/cache/rollupResult"...
2024-01-24T18:05:28.341Z	info	VictoriaMetrics/app/vmselect/promql/rollup_result_cache.go:142	loaded rollupResult cache from "/share/victoria-metrics-data/cache/rollupResult" in 0.024 seconds; entriesCount: 0, sizeBytes: 0
2024-01-24T18:05:28.341Z	info	VictoriaMetrics/app/victoria-metrics/main.go:80	started VictoriaMetrics in 4.075 seconds
2024-01-24T18:05:28.341Z	info	VictoriaMetrics/lib/httpserver/httpserver.go:96	starting http server at http://127.0.0.1:8428/
2024-01-24T18:05:28.341Z	info	VictoriaMetrics/lib/httpserver/httpserver.go:97	pprof handlers are exposed at http://127.0.0.1:8428/debug/pprof/

from homeassistant-addon-victoriametrics.

Storing VM data outside from the docker container is great idea and will work.

But could you please elaborate on your issue: I cannot store data for more than 2 weeks before the data disappears ?

from homeassistant-addon-victoriametrics.

FYI: we have a different communities where you can ask anything regarding VictoriaMetrics

from homeassistant-addon-victoriametrics.