dutchcoders / marija Goto Github PK

Data exploration and visualisation for Elasticsearch and Splunk.

License: GNU Affero General Public License v3.0

Go 99.23% Shell 0.21% Dockerfile 0.56%

dataviz elasticsearch golang visualisation

marija's Introduction

Marija

Marija is a data exploration and visualisation tool for (un)structured Elasticsearch data. Using Marija you'll be able to see relations between data of different datasources without any modifications to your data or index.

Demo

Screenshot

Features

work on multiple servers and indexes at the same time
different fields can be used as node identifier
identifiers can be normalized through normalization regular expressions
each field will have its own icon
query indexes natively to the datasource
histogram view to identify nodes in time
select and delete nodes
select related nodes, deselect all but selected nodes
zoom and move nodes
navigate through selected data using the tableview
multiple datasources supported like Elasticsearch, Splunk, Blockchain.info, Twitter
submit nodes in realtime

Install

Using Docker

$ docker pull marija/marija
$ vim config-docker.toml # update elasticsearch configuration
$ docker run -d -p 8080:8080 -v $(pwd)/config-docker.toml:/config/config.toml --name marija marija/marija

Installation from source

Install Golang

If you do not have a working Golang environment setup please follow Golang Installation Guide.

Install Marija

Installation of Marija is easy.

$ go get github.com/dutchcoders/marija
$ marija

Installation using Homebrew (macOS)

$ brew tap dutchcoders/homebrew-marija
$ brew install marija

Configuration

Elasticsearch

[datasource]

[datasource.elasticsearch]
type="elasticsearch"
url="http://127.0.0.1:9200/demo_index"
#username=
#password=

[[logging]]
output = "stdout"
level = "debug"

Splunk

[datasource]

[datasource.splunk]
type="splunk"
url="https://localhost:8089"
username="admin"
password="admin"

Contribute to Marija

Please follow Marija Contributor's Guide

Community

Join the marija-users mailing list to discuss all things Marija.

Copyright and license

Code released under AGPL-3.0.

marija's People

Contributors

Stargazers

Watchers

marija's Issues

Composite keys

for example {schema}{user_id}

Unable to use marija with elasticsearch 5.0.2 over ssh tunnel

I'm connecting to a remote Elasticsearch 5.0.2 instance over an SSH tunnel; the tunnel works because:

connecting to http://localhost:9200/ in a browser returns the proper version information
marija detects my indexes

Attempting to "use the refresh icon to refresh the list of available fields" or "add the fields [...] to use as nodes" gives an error

Error executing query: The connection was closed abnormally, e.g., without sending or receiving a Close control frame

In my terminal the following messages are visible:

$ ./marija 
Marija server started, listening on address 127.0.0.1:8080.
2016/11/30 13:46:47 Connection upgraded.
2016/11/30 13:46:53 Connection closed
2016/11/30 13:46:56 Connection upgraded.
2016/11/30 13:47:04 Connection closed
2016/11/30 13:47:07 Connection upgraded.

Any suggestions on what to do next?

Autocomplete for query history

Removing a query should remove nodes and links as well.

Show path and links between selected nodes.

Allow size to be configured, now it is a hardcoded value.

Load initial state on application startup.

This will contain for example the supported datasources (es3 / es5), a default workspace configuration, etc.

Show total hits for query

Make workspaces exportable as json

Auto complete for adding fields to configuration.

Configuration pane for normalization.

Normalization will normalize the key fields to be consequent.

Import workspaces from json

Use fields from configuration

now we need to add fields manually, those should be defined in configuration

Date field being used by Histogram should be configurable.

Embeddable Marija view

An embeddable Marija view will allow users to embed the view within Kibana, website or applications. The view should contain all configuration already, and show the graph immediately.