Need help with expressions. about loki-enhance-middleware HOT 8 CLOSED

The container logs for promtail show this error:

Unable to parse config: /etc/promtail/promtail-config.yaml: yaml: unmarshal errors:
field pipeline_stages not found in type struct { Targets []string "yaml:"targets""; Labels model.LabelSet "yaml:"labels"" }

from loki-enhance-middleware.

To use this middleware, just include GeoIP_Source=XX in output template, and your template is fine.

The real issue is that the indent level of pipeline_stages field is wrong, so promtail could not parse it.

Please look into
https://grafana.com/docs/loki/latest/clients/promtail/pipelines/
https://spacelift.io/blog/yaml

from loki-enhance-middleware.

Thanks for the info. I think I've formatted my yaml correct so I can see stuff importing into loki. I see GeoIP_source, but It doesnt seem to be working.

server:
  http_listen_port: 9080
  grpc_listen_port: 0

positions:
  filename: /tmp/positions.yaml

clients:
  - url: http://lokimiddleware:3100/loki/api/v1/push


scrape_configs:
  - job_name: netify
    static_configs:
    - targets:
        - localhost
      labels:
        job: netifylogs
        __path__: /var/log/netify/*log
    pipeline_stages:
      - json:
          expressions:
            detected_protocol_name: detected_protocol_name
            host_server_name: host_server_name
            local_ip: local_ip
            local_mac: local_mac
            local_port: local_port
            other_ip: other_ip
            other_mac: other_mac
            other_port: other_port
      - labels:
          host_server_name:
          local_ip:
          local_mac:
          local_port:
          other_ip:
          other_mac:
          other_port:
      - template:
          source: output_msg
          template: 'detected_protocol_name:"{{ .detected_protocol_name }}" local_ip:"{{ .local_ip }}" local_mac:"{{ .local_mac }}" local_port:"{{ .local_port }}"  GeoIP_Source:"{{.other_ip }}" other_mac:"{{ .other_mac }}" other_port:"{{ .other_port }}"' 

      - output:
          source: output_msg

Middleware is giving this error, which I guess makes sense as it's not a public ip

error: target reject {"name":"server"}

Here is what gets imported into loki:

2023-02-02 21:31:00 | detected_protocol_name:"MDNS" local_ip:"10.0.5.170" local_mac:"78:64:c0:34:7a:ee" local_port:"5353" GeoIP_Source:"224.0.0.251" other_mac:"01:00:5e:00:00:fb" other_port:"5353"

from loki-enhance-middleware.

ok, so I can see the public IP in loki and I see the GeoIP, but its not replacing with the location/city. Can you send me a snippet of your log file?

from loki-enhance-middleware.

Try to use = instead of : in the template

from loki-enhance-middleware.

it works!!!! I needed the = instead of : in my template.

from loki-enhance-middleware.

Do you have a grafana dashboard that I can use as an example?

from loki-enhance-middleware.

Does the data fields need to match naming conventions that geomap is looking for? I can't make worldmap or geomap work :)

From Grafana:
The Geomap panel needs a source of geographical data. This data comes from a database query, and there are four mapping options for your data.

Auto automatically searches for location data. Use this option when your query is based on one of the following names for data fields.
geohash: “geohash”
latitude: “latitude”, “lat”
longitude: “longitude”, “lng”, “lon”
lookup: “lookup”

from loki-enhance-middleware.