Code Monkey home page Code Monkey logo

fuslwusl / homeassistant-addon-victoriametrics Goto Github PK

View Code? Open in Web Editor NEW
90.0 3.0 20.0 26.15 MB

VictoriaMetrics Add-on for Home Assistant OS is the perfect solution for long term data storage of your smart home sensor data and visualization with Grafana.

License: MIT License

Dockerfile 69.86% Shell 4.38% Smarty 25.75%
grafana grafana-datasource grafana-influxdb grafana-prometheus home-assistant homeassistant homeassistant-addons influxdb monitoring homeassistant-database

homeassistant-addon-victoriametrics's Introduction

Home Assistant Add-on VictoriaMetrics Time Series Database

If you are looking for an efficient and easy to use way for long term storage of your Home Assistant data - just use VictoriaMetrics.

This add-on makes it easy to run a VictoriaMetrics Time Series Database on Home Assistant OS on ARM64 systems like Raspberry Pi 4 or AMD64.

Replacement for InfluxDB and Prometheus

VictoriaMetrics Time Series Database is 100% compatible with the Prometheus and InfluxDB line protocol. So Homeassistant is able to connect to VictoriaMetrics with the InfluxDB integration. In Grafana you can use the Prometheus or InfluxDB data source to use your data in your VictoriaMetrics database.

Much better long term storage

VictoriaMetrics offers an ultra efficient compression for data storage, is highly optimized for time series data and has low RAM usage. So this is the perfect solution for long term storage of high amounts of data like all sensor values of your smart home.

There are many reasons to switch to VictoriaMetrics. Compared to other products VictoriaMetrics offers:

  • better compression
  • better performance
  • lower RAM usage
  • fully open source
  • outperfomes other time series databases
  • ease of use

VictoriaMetrics on GitHub

VictoriaMetrics wins InfluxDB and Timescale in all the queries by a margin of up to 20x. It especially excels at heavy queries, which scan many millions of datapoints across thousands of distinct timeseries. More info here VictoriaMetrics vs. InfluxDB vs. Timescale

This add-on continues the work of Andreas Habel whose hassio-victoriametrics add-on was not updated since 2021. I developed this mainly for my purpose running VictoriaMetrics on my Raspberry Pi 4 running Home Assistant Operating System. The direction of InfluxDB 2.0 was not what I wanted in combination with Grafana so VictoriaMetrics Time Series Database was the best choice.

Installation and configuration

  1. Add the reposity. (Quick link: Open your Home Assistant instance and show the Supervisor add-on store. )

  2. Install the add-on:

    • Find, and install the VictoriaMetrics add-on
  3. Configure VictoriaMetrics

    Read the addon documentation which can also be found on the Documentation tab of the VictoriaMetrics addon in the Home Assistant settings.

  4. Start the VictoriaMetrics add-on

  5. Add the influxdb integration to your Home Assistant configuration

    Instructions for adding and configuring the influxdb integration are in the addon documentation which can also be found on the Documentation tab of the VictoriaMetrics addon configuration.

    Don't forget to restart Home Assistant!

Well done! You can install and configure Grafana or similar to check data is being logged.

homeassistant-addon-victoriametrics's People

Contributors

denisgolius avatar eckad158 avatar falzm avatar fuslwusl avatar googanhiem avatar hermanbanken avatar mjmeierhoefer avatar scriptengine avatar waroen avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

homeassistant-addon-victoriametrics's Issues

Better instruction for configuring InfluxDB interface

The README.md doesn't include detailed instructions on how to configure InfluxDB for the newly installed VictoriaMetrics addon.

This is a problem for less experienced Home Assistant and InfluxDB users (like me).

Detailed instructions would allow the effective installation/configuration of this plugin.

As it is, people may just get stuck at configuring InfluxDB for VM and finally give up on the whole thing and remove the VictoriaMetrics addon.

It would also be important to explain how to validate the setup so that people can be sure that data is collected properly and can be queried as well (e.g.: via Grafana)

home assistant database

Hello, first of all thank you for your work.
I have installed your addon and it works very well in grafana, but grafana not so much on home assistant, especially when you connect successively from outside and from within your own network.
I wanted to know if there is any way for the home assistant to read the sensors directly from the victoria metrics database, thanks again, best regards.

Since 4th of December victoriametrics does not start anymore

I was on the home assistant beta channel and received an update on the 4th of December. Since then unfortunately, my victoriametrics add-on does not start anymore with the configuration that used to work for months. I can't really find out what's wrong as the VM log does not show anything particular (see further down). However, I can find entries in the log file that show login issues. How can I fix this with my plain vanilla configuration taken over from the documentation?

homeassistant.components.influxdb] Cannot connect to InfluxDB due to 'HTTPConnectionPool(host='homeassistant.local', port=8428): Max retries exceeded with url: /write?db=home_assistant (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5f135e50>: Failed to establish a new connection: [Errno 22] Invalid argument'))'. Please check that the provided connection details (host, port, etc.) are correct and that your InfluxDB server is running and accessible. Retrying in 60 seconds.

Log file:

s6-rc: info: service s6rc-oneshot-runner: starting s6-rc: info: service s6rc-oneshot-runner successfully started s6-rc: info: service fix-attrs: starting s6-rc: info: service fix-attrs successfully started s6-rc: info: service legacy-cont-init: starting s6-rc: info: service legacy-cont-init successfully started s6-rc: info: service legacy-services: starting s6-rc: info: service legacy-services successfully started [06:52:32] INFO: Starting Victoria Metrics with retention set to 99y [06:52:32] INFO: Starting Victoria Metrics with args set to 2023-10-10T04:52:32.929Z info VictoriaMetrics/lib/logger/flag.go:12 build version: victoria-metrics-20231002-214442-tags-v1.94.0-0-gf13a96f42 2023-10-10T04:52:32.932Z info VictoriaMetrics/lib/logger/flag.go:13 command-line flags 2023-10-10T04:52:32.933Z info VictoriaMetrics/lib/logger/flag.go:20 -retentionPeriod="99y" 2023-10-10T04:52:32.933Z info VictoriaMetrics/lib/logger/flag.go:20 -storageDataPath="/share/victoria-metrics-data" 2023-10-10T04:52:32.933Z info VictoriaMetrics/app/victoria-metrics/main.go:70 starting VictoriaMetrics at ":8428"... 2023-10-10T04:52:32.936Z info VictoriaMetrics/app/vmstorage/main.go:108 opening storage at "/share/victoria-metrics-data" with -retentionPeriod=99y 2023-10-10T04:52:32.953Z info VictoriaMetrics/lib/memory/memory.go:42 limiting caches to 2386204262 bytes, leaving 1590802842 bytes to the OS according to -memory.allowedPercent=60 2023-10-10T04:52:33.265Z info VictoriaMetrics/app/vmstorage/main.go:122 successfully opened storage "/share/victoria-metrics-data" in 0.328 seconds; partsCount: 211; blocksCount: 61659; rowsCount: 13829967; sizeBytes: 31961533 2023-10-10T04:52:33.278Z info VictoriaMetrics/app/vmselect/promql/rollup_result_cache.go:114 loading rollupResult cache from "/share/victoria-metrics-data/cache/rollupResult"... 2023-10-10T04:52:33.292Z info VictoriaMetrics/app/vmselect/promql/rollup_result_cache.go:142 loaded rollupResult cache from "/share/victoria-metrics-data/cache/rollupResult" in 0.014 seconds; entriesCount: 0, sizeBytes: 0 2023-10-10T04:52:33.296Z info VictoriaMetrics/app/victoria-metrics/main.go:80 started VictoriaMetrics in 0.359 seconds 2023-10-10T04:52:33.296Z info VictoriaMetrics/lib/httpserver/httpserver.go:96 starting http server at http://127.0.0.1:8428/ 2023-10-10T04:52:33.296Z info VictoriaMetrics/lib/httpserver/httpserver.go:97 pprof handlers are exposed at http://127.0.0.1:8428/debug/pprof/

configuration.yaml

Prometheus

prometheus:
namespace: hass
component_config_glob:
sensor._bat:
override_metric: battery_percent
filter:
include_domains:
- sensor
exclude_entity_globs:
- sensor.waqi

- climate.refrigerator*

Victoria Metrics InfluxDB

influxdb:
api_version: 1
host: homeassistant.local
port: 8428
max_retries: 3
measurement_attr: entity_id
tags_attributes:
- friendly_name
- unit_of_measurement
ignore_attributes:
- icon
- source
- options
- editable
- min
- max
- step
- mode
- marker_type
- preset_modes
- supported_features
- supported_color_modes
- effect_list
- attribution
- assumed_state
- state_open
- state_closed
- writable
- stateExtra
- event
- friendly_name
- device_class
- state_class
- ip_address
- device_file
- unit_of_measurement
- unitOfMeasure
include:
domains:
- sensor
- binary_sensor
- light
- switch
- cover
- climate
- input_boolean
- input_select
- number
- lock
- weather
exclude:
entity_globs:
- sensor.clock*
- sensor.date*
- sensor.glances*
- sensor.time*
- sensor.uptime*
- sensor.dwd_weather_warnings_*
- weather.weatherstation
- binary_sensor.smartphone
- sensor.smartphone
- sensor.adguard_home_*
- binary_sensor.*_internet_access

How to rewrite labels

Hi,

I want to change some labels and the name of metrics. I followed the VictoriaDB guide and setup a relabel config. Now I'm struggling with where to place the relabel config.

Does this work at all, when using the influx integration?

For completeness, I want to apply following config:

  • source_labels: [name]
    regex: 'sensor..plant_sensor(.)_value'
    replacement: '$1'
    target_label: plant_id
  • source_labels: [name]
    regex: 'sensor.(.)plant_sensor._value'
    replacement: '$1'
    target_label: sensor_type
  • source_labels: [name]
    regex: 'sensor..plant_sensor._value'
    replacement: sensor.plant_sensor
    target_label: name

victoriametrics disk usage

Hi,
If i do a backup of HA with Victoriametrics, it has a compressed size of > 50MB.
Without Victoriametrics the size is 10MB.
This sounds not to be so much, but if i i do the backup it takes 5 times longer.
I saw that Victoria metrics brings along many libraries whitch have an uncompressed size of 120Mb.
Is it possible to shrink the Add-On e.g. without all the libraries? I don't want to build it by myself.
Regards, Walter

Unable to store more than 2 weeks of history

First I wanted to thank the maintainer and original inspiration for creating this addon.

I don't know or understand VM very well and are convinced that the issue is user error as I'm unable to find anyone with similar issues.
My issue is retention, I cannot store data for more than 2 weeks before the data disappears. I went through the documentation on VM's site, but I don't know if I just need to increase the retention (currently 99y) or if I also need to add some sort of setting to move/process short term data into long term? This may be best suited for VM support? Any help is appreciated.

Addon settings:

retention: 99y
additionalArguments: ""
enableHTTPAuth: false
username: ""
password: ""
enablePrometheusScrape: false
prometheusScrapeHTTPS: false
prometheusScrapeInterval: 20s
prometheusScrapeTimeout: 15s
longelivedToken: ""
homeassistantUrl: ""

HA settings:

influxdb:
#token: abc1234
api_version: 1
host: 8f49de54-victoria-metrics.local.hass.io
port: 8428
max_retries: 3
password: homeassistant
username: homeassistant
database: homeassistant
ssl: false
verify_ssl: false
tags:
source: HA
measurement_attr: entity_id
default_measurement: state
tags_attributes:
- friendly_name
- unit_of_measurement

Question: migrate from influx DB

Hi
Any guidance on migrating my existing influx data towards this great alternative?
Does fully compatible mean I can even import a backup from influx DB?
Best Eric

Some incorrect assumptions in timeseries processing.

I wish people writing timeseries databases would read and understand all the rrdtool documentation first. That tool contains a huge amount of accumulated knowledge/wisdom about how to handle timeseries efficiently and accurately.

In https://docs.victoriametrics.com/keyconcepts/#instant-query the documentation talks about how queries for timeseries values at a particular time look for the datapoint before that time.

However, in general timeseries datapoints better represent the time period before, so it's more accurate to use the datapoint after the requested time. For rate timeseries with points at regular intervals, each point represents the average rate for the period between that point and the previous point. Using the rate datapoint after the requested time gives you the best possible instantaneous rate estimate for that time. However, if you are querying a rate1m timeseries (points every minute that represent the average rate over the previous minute) and want the best rate1m at a particular time (average rate over the minute before the particular time), then the most accurate estimate you can get is by linearly interpolating between the point after and before the requested time.

This "datapoints represent the time before the datapoint, not after" behaviour is definitely true for rates, but is also generally true for gauges and counters too. In general linearly interpolating between points gives the most accurate estimate.

When there are missing datapoints, it becomes even more crucial to do this right, as you are approximating over longer time intervals, so using the wrong approximation makes the answer even more wrong. Just returning the value of points before the requested time as if it was at the requested time is time-shifting the value to an arbitrary, up to your step interval, later time. It would be better to provide the actual timestamp in the answer, but I'm not sure frontends would be happy with that.

If the timeseries lookups are time-shifting data-points like this by arbitrary times, then any queries that combine different timeseries will be using values taken at different times. This can be the source of many problems, like error rates greater than 100%.

Doing this correctly would answer many of the problems trying to be addressed with https://docs.victoriametrics.com/keyconcepts/#query-latency.

SSL configs

Is there a way to configure this addon to use Home Assistants SSL certificates like you can for other addons?

DB size

Tell me how to make a database size sensor in home assistant?

[feature] Add support for graphite

I'd like to enable graphite listening server, as documented here

I can add the argument -graphiteListenAddr=8867 , but get an error. It returns error fatal VictoriaMetrics/lib/ingestserver/graphite/server.go:48 cannot start TCP Graphite server at "8867": listen tcp4: address 8867: missing port in address

If you could add the ability to specify another port I imagine that would solve the issue.

Map the Home Assistant config directory to the add-on container

Hi ๐Ÿ‘‹
Would you consider adding the Home Assistant global config directory to the mapping of your add-on container? In this PR I've introduced the ability to pass extra command line arguments such as -promscrape.config=FILE, which allows using alternative Prometheus server configuration โ€“ however at the moment it isn't possible to pass a file via the global /config Home Assistant directory. Even a read-only mapping would suffice in this case.

Basically, you'd have to add - config to the map: attribute of your config.yaml add-on file: https://github.com/fuslwusl/homeassistant-addon-victoriametrics/blob/main/victoria-metrics/config.yaml#L19

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.