- Node.js
- Docker
-
Clone this repository 1-1. Create the rabbitMQ log directory:
mkdir -p /tmp/rabbitmq/logs/
-
From the newly cloned repo directory, run
docker-compose up --build
NOTE: If you get an error like this:
"Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /tmp/rabbitmq/etc/"
You can fix it by adding the directory in the terminal:
mkdir /tmp/rabbitmq/etc
- Run
docker-compose ps
to see containers runnning. - Install dependencies with
npm i
- (optional: Revert all current migrations -
npx sequelize-cli db:migrate:undo:all
) Run migration withnpx sequelize-cli db:migrate
- Make a copy of
config/local.sample.js
under the name ofconfig/local.js
- We can use default values for most fields except the Jira section.
For how to set up basic authorization with Jira, and many more things, please see CONFIGURATION.md
Grafana Connection For Data Visualization (https://localhost:3002)
Connect to the Grafana database:
Inside docker-compose.yml
edit the environment variables as needed to connect to your local postgres instance, specifically:
GF_DATABASE_NAME
GF_DATABASE_USER
GF_DATABASE_PASSWORD
Connect the Grafana data source:
Additionally to use the postgres database as data source inside grafana, ensure postgres config options are correct in ./grafana/datasources/datasource.yml
, specifically:
database
user
secureJsonData/password
- From the terminal, execute
npm run all
- From Postman (or similar), send a request like...
POST http://localhost:3001/
{
"jira": {
"boardId": 8
},
"gitlab": {
"projectId": 8967944
}
}
Or, by using `curl`
# ee
curl -X POST "http://localhost:3001/" -H 'content-type: application/json' \
-d '{"jira":{"boardId": 8}, "gitlab": {"projectId": 8967944}}'
# small data set for test
curl -X POST "http://localhost:3001/" -H 'content-type: application/json' \
-d '{"jira":{"boardId": 29}, "gitlab": {"projectId": 24547305}}'
- See that the collection job was published, jira collection ran, the enrichment job was published, and jira enrichment ran
To run only the enrichment job on existing collections: POST http://localhost:3000/
Login Credentials
- Visit:
http://localhost:3002
- Username:
admin
- Password:
admin
Provisioning a Grafana Dashboard
To save a dashboard in the lake
repo and load it:
- Create a dashboard in browser (visit
/dashboard/new
, or use sidebar) - Save dashboard (in top right of screen)
- Go to dashboard settings (in top right of screen)
- Click on JSON Model in sidebar
- Copy code into a new
.json
file in/grafana/dashboards