This will build 5 containers:
- 1 x Elasticsearch Master / Ingest Node
- 2 x Elasticsearch Data Nodes
- 1 x Kibana
- 1 x Logstash
Versions of Elasticsearch and cluster password can be set using the env.sh.
NOTE: if you change the password you will also need to update ./pipeline/logstash.conf too
- docker
- docker-compose
source ./env.sh
docker-compose up -d
docker-compose logs -f
curl -X GET "localhost:9200/_cat/nodes?v&pretty" -u elastic:changeme
Open browser to http://localhost:5601/
NOW=$(($(date +'%s * 1000 + %-N / 1000000')))
curl -H "Content-Type: application/json" -X POST "http://localhost:9600" -d'
{
"timestamp": "'"${NOW}"'",
"message": "This is a sample event...."
}
'
If you update the ./pipeline/logstash.conf file then changes will be reflected inside the Docker image. Using SIGHUP provides a convenient way to test Logstash configurations by forcing a reload of the Logstash service.
docker exec -u root logstash-01 bash -c 'kill -HUP 1'
On your test environment (use following link for detailed instructions)
MAC OS
cp filebeat.yml /usr/local/etc/metricbeat/filebeat.yml
Linux
cp filebeat.yml /etc/metricbeat/filebeat.yml
filebeat setup --index-management -E output.logstash.enabled=false -E 'output.elasticsearch.hosts=["elastic:changeme@localhost:9200"]'
filebeat modules enable system
filebeat setup --dashboards
filebeat -e
On your test environment (use following link for detailed instructions)
MAC OS
cp metricbeat.yml /usr/local/etc/metricbeat/metricbeat.yml
Linux
cp metricbeat.yml /etc/metricbeat/metricbeat.yml
metricbeat setup --index-management -E output.logstash.enabled=false -E 'output.elasticsearch.hosts=["elastic:changeme@localhost:9200"]'
metricbeat setup --dashboards
metricbeat -e
docker-compose down
docker container prune
docker network prune
docker image prune
docker volume prune
# or do this....
docker system prune --volumes
- Notes for testing Beats
Elasticsearch require vm.max_map_count to be set as 262144. To fix the problem with vm.max_map_count too small run to increase it on the box
sudo sysctl -w vm.max_map_count=262144