Comments (4)
First I wanted to thank the maintainer and original inspiration for creating this addon.
I don't know or understand VM very well and are convinced that the issue is user error as I'm unable to find anyone with similar issues. My issue is retention, I cannot store data for more than 2 weeks before the data disappears. I went through the documentation on VM's site, but I don't know if I just need to increase the retention (currently 99y) or if I also need to add some sort of setting to move/process short term data into long term? This may be best suited for VM support? Any help is appreciated.
Addon settings:
retention: 99y additionalArguments: "" enableHTTPAuth: false username: "" password: "" enablePrometheusScrape: false prometheusScrapeHTTPS: false prometheusScrapeInterval: 20s prometheusScrapeTimeout: 15s longelivedToken: "" homeassistantUrl: ""
HA settings:
influxdb: #token: abc1234 api_version: 1 host: 8f49de54-victoria-metrics.local.hass.io port: 8428 max_retries: 3 password: homeassistant username: homeassistant database: homeassistant ssl: false verify_ssl: false tags: source: HA measurement_attr: entity_id default_measurement: state tags_attributes: - friendly_name - unit_of_measurement
Hey @borrelan ,
There is nothing to do from VictoriaMetrics side if you have set retention: 99y
. If this value is passed by the addon incorrectly or if it is overridden somewhere else - then there may be such a problem.
You can try to check retention period via curl query to VictoriaMetrics host, for example
curl <victoriametrics-ip-here>:8428/flags
-graphiteListenAddr=":2003"
-httpListenAddr=":8428"
-influxListenAddr=":8089"
-opentsdbListenAddr=":4242"
-retentionPeriod="99y"
-storageDataPath="/storage"
-vmalert.proxyURL="http://vmalert:8880"
or by checking VictoriaMetrics logs:
docker compose logs victoriametrics
victoriametrics | 2024-01-22T17:00:03.614Z info VictoriaMetrics/lib/logger/flag.go:12 build version: victoria-metrics-20231212-224619-tags-v1.96.0-0-g304fe0565
victoriametrics | 2024-01-22T17:00:03.614Z info VictoriaMetrics/lib/logger/flag.go:13 command-line flags
victoriametrics | 2024-01-22T17:00:03.614Z info VictoriaMetrics/lib/logger/flag.go:20 -graphiteListenAddr=":2003"
victoriametrics | 2024-01-22T17:00:03.614Z info VictoriaMetrics/lib/logger/flag.go:20 -httpListenAddr=":8428"
victoriametrics | 2024-01-22T17:00:03.614Z info VictoriaMetrics/lib/logger/flag.go:20 -influxListenAddr=":8089"
victoriametrics | 2024-01-22T17:00:03.614Z info VictoriaMetrics/lib/logger/flag.go:20 -opentsdbListenAddr=":4242"
victoriametrics | 2024-01-22T17:00:03.614Z info VictoriaMetrics/lib/logger/flag.go:20 -retentionPeriod="99y"
victoriametrics | 2024-01-22T17:00:03.614Z info VictoriaMetrics/lib/logger/flag.go:20 -storageDataPath="/storage"
from homeassistant-addon-victoriametrics.
First I wanted to thank the maintainer and original inspiration for creating this addon.
I don't know or understand VM very well and are convinced that the issue is user error as I'm unable to find anyone with similar issues. My issue is retention, I cannot store data for more than 2 weeks before the data disappears. I went through the documentation on VM's site, but I don't know if I just need to increase the retention (currently 99y) or if I also need to add some sort of setting to move/process short term data into long term? This may be best suited for VM support? Any help is appreciated.
Addon settings:
retention: 99y additionalArguments: "" enableHTTPAuth: false username: "" password: "" enablePrometheusScrape: false prometheusScrapeHTTPS: false prometheusScrapeInterval: 20s prometheusScrapeTimeout: 15s longelivedToken: "" homeassistantUrl: ""
HA settings:
influxdb: #token: abc1234 api_version: 1 host: 8f49de54-victoria-metrics.local.hass.io port: 8428 max_retries: 3 password: homeassistant username: homeassistant database: homeassistant ssl: false verify_ssl: false tags: source: HA measurement_attr: entity_id default_measurement: state tags_attributes: - friendly_name - unit_of_measurementHey @borrelan ,
There is nothing to do from VictoriaMetrics side if you have set
retention: 99y
. If this value is passed by the addon incorrectly or if it is overridden somewhere else - then there may be such a problem.You can try to check retention period via curl query to VictoriaMetrics host, for example
curl <victoriametrics-ip-here>:8428/flags -graphiteListenAddr=":2003" -httpListenAddr=":8428" -influxListenAddr=":8089" -opentsdbListenAddr=":4242" -retentionPeriod="99y" -storageDataPath="/storage" -vmalert.proxyURL="http://vmalert:8880"or by checking VictoriaMetrics logs:
docker compose logs victoriametrics victoriametrics | 2024-01-22T17:00:03.614Z info VictoriaMetrics/lib/logger/flag.go:12 build version: victoria-metrics-20231212-224619-tags-v1.96.0-0-g304fe0565 victoriametrics | 2024-01-22T17:00:03.614Z info VictoriaMetrics/lib/logger/flag.go:13 command-line flags victoriametrics | 2024-01-22T17:00:03.614Z info VictoriaMetrics/lib/logger/flag.go:20 -graphiteListenAddr=":2003" victoriametrics | 2024-01-22T17:00:03.614Z info VictoriaMetrics/lib/logger/flag.go:20 -httpListenAddr=":8428" victoriametrics | 2024-01-22T17:00:03.614Z info VictoriaMetrics/lib/logger/flag.go:20 -influxListenAddr=":8089" victoriametrics | 2024-01-22T17:00:03.614Z info VictoriaMetrics/lib/logger/flag.go:20 -opentsdbListenAddr=":4242" victoriametrics | 2024-01-22T17:00:03.614Z info VictoriaMetrics/lib/logger/flag.go:20 -retentionPeriod="99y" victoriametrics | 2024-01-22T17:00:03.614Z info VictoriaMetrics/lib/logger/flag.go:20 -storageDataPath="/storage"
@denisgolius
Here is what the HA addon I have installed is doing.
I believe I'm going to switch to a separate docker container and move on as the data folder is hard coded and is within the container and not stored externally.
Advice on path forward is of course welcomed
curl 172.30.32.1:8428/flags
-retentionPeriod="99y"
-storageDataPath="/share/victoria-metrics-data"
Addon Logs
s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service legacy-services: starting
s6-rc: info: service legacy-services successfully started
[13:05:24] INFO: Starting Victoria Metrics with retention set to 99y
[13:05:24] INFO: Starting Victoria Metrics with args set to
2024-01-24T18:05:24.266Z info VictoriaMetrics/lib/logger/flag.go:12 build version: victoria-metrics-20231002-214434-tags-v1.94.0-0-gf13a96f42
2024-01-24T18:05:24.266Z info VictoriaMetrics/lib/logger/flag.go:13 command-line flags
2024-01-24T18:05:24.266Z info VictoriaMetrics/lib/logger/flag.go:20 -retentionPeriod="99y"
2024-01-24T18:05:24.266Z info VictoriaMetrics/lib/logger/flag.go:20 -storageDataPath="/share/victoria-metrics-data"
2024-01-24T18:05:24.266Z info VictoriaMetrics/app/victoria-metrics/main.go:70 starting VictoriaMetrics at ":8428"...
2024-01-24T18:05:24.266Z info VictoriaMetrics/app/vmstorage/main.go:108 opening storage at "/share/victoria-metrics-data" with -retentionPeriod=99y
2024-01-24T18:05:24.408Z info VictoriaMetrics/lib/memory/memory.go:42 limiting caches to 40341331968 bytes, leaving 26894221312 bytes to the OS according to -memory.allowedPercent=60
2024-01-24T18:05:24.474Z info VictoriaMetrics/lib/storage/storage.go:933 discarding /share/victoria-metrics-data/cache/curr_hour_metric_ids, since it contains outdated hour; got 473734; want 473922
2024-01-24T18:05:24.474Z info VictoriaMetrics/lib/storage/storage.go:933 discarding /share/victoria-metrics-data/cache/prev_hour_metric_ids, since it contains outdated hour; got 473733; want 473921
2024-01-24T18:05:25.242Z info VictoriaMetrics/lib/storage/storage.go:894 discarding /share/victoria-metrics-data/cache/next_day_metric_ids_v2, since it contains data for stale date; got 19738; want 19746
2024-01-24T18:05:28.218Z info VictoriaMetrics/app/vmstorage/main.go:122 successfully opened storage "/share/victoria-metrics-data" in 3.952 seconds; partsCount: 124; blocksCount: 235196; rowsCount: 279501699; sizeBytes: 304777947
2024-01-24T18:05:28.316Z info VictoriaMetrics/app/vmselect/promql/rollup_result_cache.go:114 loading rollupResult cache from "/share/victoria-metrics-data/cache/rollupResult"...
2024-01-24T18:05:28.341Z info VictoriaMetrics/app/vmselect/promql/rollup_result_cache.go:142 loaded rollupResult cache from "/share/victoria-metrics-data/cache/rollupResult" in 0.024 seconds; entriesCount: 0, sizeBytes: 0
2024-01-24T18:05:28.341Z info VictoriaMetrics/app/victoria-metrics/main.go:80 started VictoriaMetrics in 4.075 seconds
2024-01-24T18:05:28.341Z info VictoriaMetrics/lib/httpserver/httpserver.go:96 starting http server at http://127.0.0.1:8428/
2024-01-24T18:05:28.341Z info VictoriaMetrics/lib/httpserver/httpserver.go:97 pprof handlers are exposed at http://127.0.0.1:8428/debug/pprof/
from homeassistant-addon-victoriametrics.
Storing VM data outside from the docker container is great idea and will work.
But could you please elaborate on your issue: I cannot store data for more than 2 weeks before the data disappears
?
from homeassistant-addon-victoriametrics.
FYI: we have a different communities where you can ask anything regarding VictoriaMetrics
from homeassistant-addon-victoriametrics.
Related Issues (17)
- home assistant database HOT 3
- home assistant core does not have addons HOT 1
- DB size
- Question: migrate from influx DB HOT 3
- Use vmagent instead of full victoria metrics HOT 1
- Addon victoriametrics did not start
- Better instruction for configuring InfluxDB interface HOT 4
- victoriametrics disk usage HOT 1
- Map the Home Assistant config directory to the add-on container HOT 4
- How to rewrite labels HOT 3
- Since 4th of December victoriametrics does not start anymore HOT 2
- Update to victoriametrics >= 1.96.0 to enable use of new flag to skip tls cert verification
- SSL configs HOT 3
- [feature] Add support for graphite HOT 3
- Fail to update addon HOT 4
- Some incorrect assumptions in timeseries processing. HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from homeassistant-addon-victoriametrics.