This repository contains a set of scripts that you can use to deploy a Neo4j cluster on Kubernetes. It will work for Neo4j versions 3.1 and higher.
You can either setup Kubernetes locally using minikube or on one of the cloud providers e.g. Google Compute Engine
Once you’ve got that setup you can run the following command to see where Kubernetes is running. e.g. on my machine
$ kubectl cluster-info
Kubernetes master is running at https://192.168.99.100:8443
KubeDNS is running at https://192.168.99.100:8443/api/v1/proxy/namespaces/kube-system/services/kube-dns
kubernetes-dashboard is running at https://192.168.99.100:8443/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
All good so far. Now we’re ready to install Neo4j.
First let’s install the Core servers. We’d typically have a small number of Core servers in a cluster and they can handle both read and write traffic.
Run the following commands to create the Core servers:
$ kubectl apply -f cores
service "neo4j" configured
statefulset "neo4j-core" created
We can check that those servers have spun up by running:
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
neo4j-core-0 1/1 Running 0 23s
neo4j-core-1 1/1 Running 0 19s
neo4j-core-2 1/1 Running 0 16s
And we can check that Neo4j is up and running by tailing the logs of one of the servers until we see the following:
$ kubectl logs -f neo4j-core-2
Starting Neo4j.
2017-09-13 09:39:31.406+0000 INFO ======== Neo4j 3.2.3 ========
2017-09-13 09:39:31.824+0000 INFO Starting...
2017-09-13 09:39:55.741+0000 INFO Bolt enabled on 0.0.0.0:7687.
2017-09-13 09:39:56.050+0000 INFO Initiating metrics...
2017-09-13 09:39:58.842+0000 INFO Resolved initial host 'neo4j.default.svc.cluster.local:5000' to [172.17.0.4:5000, 172.17.0.5:5000, 172.17.0.6:5000]
2017-09-13 09:39:59.976+0000 INFO My connection info: [
Discovery: listen=0.0.0.0:5000, advertised=neo4j-core-0.neo4j.default.svc.cluster.local:5000,
Transaction: listen=0.0.0.0:6000, advertised=neo4j-core-0.neo4j.default.svc.cluster.local:6000,
Raft: listen=0.0.0.0:7000, advertised=neo4j-core-0.neo4j.default.svc.cluster.local:7000,
Client Connector Addresses: bolt://neo4j-core-0.neo4j.default.svc.cluster.local:7687,http://neo4j-core-0.neo4j.default.svc.cluster.local:7474,https://neo4j-core-0.neo4j.default.svc.cluster.local:7473
]
2017-09-13 09:39:59.977+0000 INFO Discovering cluster with initial members: [neo4j.default.svc.cluster.local:5000]
2017-09-13 09:39:59.977+0000 INFO Attempting to connect to the other cluster members before continuing...
2017-09-13 09:41:28.430+0000 INFO Started.
2017-09-13 09:41:29.911+0000 INFO Mounted REST API at: /db/manage
2017-09-13 09:41:39.562+0000 INFO Remote interface available at http://neo4j-core-2.neo4j.default.svc.cluster.local:7474/
Neo4j also exposes the cluster topology via a procedure, which we can call like this:
$ kubectl exec neo4j-core-0 -- bin/cypher-shell --format verbose "CALL dbms.cluster.overview()"
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| id | addresses | role | groups |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| "719fa587-68e4-4194-bc61-8a35476a0af5" | ["bolt://neo4j-core-0.neo4j.default.svc.cluster.local:7687", "http://neo4j-core-0.neo4j.default.svc.cluster.local:7474", "https://neo4j-core-0.neo4j.default.svc.cluster.local:7473"] | "LEADER" | [] |
| "bb057924-f304-4f6d-b726-b6368c8ac0f1" | ["bolt://neo4j-core-1.neo4j.default.svc.cluster.local:7687", "http://neo4j-core-1.neo4j.default.svc.cluster.local:7474", "https://neo4j-core-1.neo4j.default.svc.cluster.local:7473"] | "FOLLOWER" | [] |
| "f84e7e0d-de6c-480e-8981-dad114de08cf" | ["bolt://neo4j-core-2.neo4j.default.svc.cluster.local:7687", "http://neo4j-core-2.neo4j.default.svc.cluster.local:7474", "https://neo4j-core-2.neo4j.default.svc.cluster.local:7473"] | "FOLLOWER" | [] |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
3 rows available after 3 ms, consumed after another 4 ms
Now that we’ve got the Core Servers up and running let’s add some Read Replicas. Read Replicas are used to scale out reads and they don’t accept any write operations.
Run the following command to create a single Read Replica:
$ kubectl apply -f read-replicas
deployment "neo4j-replica" created
If we query for the list of pods now we’ll see that it’s been added to the list:
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
neo4j-core-0 1/1 Running 0 5m
neo4j-core-1 1/1 Running 0 5m
neo4j-core-2 1/1 Running 0 5m
neo4j-replica-3562563584-sckr8 1/1 Running 0 3s
Let’s check if the replica has joined the cluster and is ready to go:
$ kubectl logs -f neo4j-replica-3562563584-sckr8
2017-09-13 09:44:31.804+0000 INFO ======== Neo4j 3.2.3 ========
2017-09-13 09:44:32.042+0000 INFO Starting...
2017-09-13 09:44:41.555+0000 INFO Bolt enabled on 0.0.0.0:7687.
2017-09-13 09:44:41.675+0000 INFO Initiating metrics...
2017-09-13 09:44:42.402+0000 INFO Resolved initial host 'neo4j.default.svc.cluster.local:5000' to [172.17.0.4:5000, 172.17.0.5:5000, 172.17.0.6:5000]
2017-09-13 09:45:23.073+0000 INFO Started.
2017-09-13 09:45:24.440+0000 INFO Mounted REST API at: /db/manage
2017-09-13 09:45:30.631+0000 INFO Remote interface available at http://neo4j-replica-3562563584-sckr8:7474/
Yep, looks good!
Let’s check that Neo4j knows about our new server:
$ kubectl exec neo4j-core-0 -- bin/cypher-shell --format verbose "CALL dbms.cluster.overview()"
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| id | addresses | role | groups |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| "719fa587-68e4-4194-bc61-8a35476a0af5" | ["bolt://neo4j-core-0.neo4j.default.svc.cluster.local:7687", "http://neo4j-core-0.neo4j.default.svc.cluster.local:7474", "https://neo4j-core-0.neo4j.default.svc.cluster.local:7473"] | "LEADER" | [] |
| "bb057924-f304-4f6d-b726-b6368c8ac0f1" | ["bolt://neo4j-core-1.neo4j.default.svc.cluster.local:7687", "http://neo4j-core-1.neo4j.default.svc.cluster.local:7474", "https://neo4j-core-1.neo4j.default.svc.cluster.local:7473"] | "FOLLOWER" | [] |
| "f84e7e0d-de6c-480e-8981-dad114de08cf" | ["bolt://neo4j-core-2.neo4j.default.svc.cluster.local:7687", "http://neo4j-core-2.neo4j.default.svc.cluster.local:7474", "https://neo4j-core-2.neo4j.default.svc.cluster.local:7473"] | "FOLLOWER" | [] |
| "df1f9885-3197-4546-a163-0e6a357df0c5" | ["bolt://neo4j-replica-3562563584-sckr8:7687", "http://neo4j-replica-3562563584-sckr8:7474", "https://neo4j-replica-3562563584-sckr8:7473"] | "READ_REPLICA" | [] |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
4 rows available after 2 ms, consumed after another 4 ms
It does indeed.
Now let’s scale up to 3 read replicas.
Run the following command:
$ kubectl scale deployment neo4j-replica --replicas=3
deployment "neo4j-replica" scaled
And give it a few seconds and Neo4j will know about those servers as well:
$ kubectl exec neo4j-core-0 -- bin/cypher-shell --format verbose "CALL dbms.cluster.overview()"
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| id | addresses | role | groups |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| "719fa587-68e4-4194-bc61-8a35476a0af5" | ["bolt://neo4j-core-0.neo4j.default.svc.cluster.local:7687", "http://neo4j-core-0.neo4j.default.svc.cluster.local:7474", "https://neo4j-core-0.neo4j.default.svc.cluster.local:7473"] | "LEADER" | [] |
| "bb057924-f304-4f6d-b726-b6368c8ac0f1" | ["bolt://neo4j-core-1.neo4j.default.svc.cluster.local:7687", "http://neo4j-core-1.neo4j.default.svc.cluster.local:7474", "https://neo4j-core-1.neo4j.default.svc.cluster.local:7473"] | "FOLLOWER" | [] |
| "f84e7e0d-de6c-480e-8981-dad114de08cf" | ["bolt://neo4j-core-2.neo4j.default.svc.cluster.local:7687", "http://neo4j-core-2.neo4j.default.svc.cluster.local:7474", "https://neo4j-core-2.neo4j.default.svc.cluster.local:7473"] | "FOLLOWER" | [] |
| "d8c92a76-82c9-4146-9742-ce422e66cd08" | ["bolt://neo4j-replica-3562563584-7qs3x:7687", "http://neo4j-replica-3562563584-7qs3x:7474", "https://neo4j-replica-3562563584-7qs3x:7473"] | "READ_REPLICA" | [] |
| "dac490ed-3ad7-4c86-b22f-41e0a429dce2" | ["bolt://neo4j-replica-3562563584-c8sqd:7687", "http://neo4j-replica-3562563584-c8sqd:7474", "https://neo4j-replica-3562563584-c8sqd:7473"] | "READ_REPLICA" | [] |
| "df1f9885-3197-4546-a163-0e6a357df0c5" | ["bolt://neo4j-replica-3562563584-sckr8:7687", "http://neo4j-replica-3562563584-sckr8:7474", "https://neo4j-replica-3562563584-sckr8:7473"] | "READ_REPLICA" | [] |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
6 rows available after 19 ms, consumed after another 2 ms
Great, that worked.
We can also scale the core servers by running the following command:
$ kubectl scale statefulsets neo4j-core --replicas 5
statefulset "neo4j-core" scaled
And we can check that those were added:
$ kubectl exec neo4j-core-0 -- bin/cypher-shell --format verbose "CALL dbms.cluster.overview()"
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| id | addresses | role | groups |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| "719fa587-68e4-4194-bc61-8a35476a0af5" | ["bolt://neo4j-core-0.neo4j.default.svc.cluster.local:7687", "http://neo4j-core-0.neo4j.default.svc.cluster.local:7474", "https://neo4j-core-0.neo4j.default.svc.cluster.local:7473"] | "LEADER" | [] |
| "bb057924-f304-4f6d-b726-b6368c8ac0f1" | ["bolt://neo4j-core-1.neo4j.default.svc.cluster.local:7687", "http://neo4j-core-1.neo4j.default.svc.cluster.local:7474", "https://neo4j-core-1.neo4j.default.svc.cluster.local:7473"] | "FOLLOWER" | [] |
| "f84e7e0d-de6c-480e-8981-dad114de08cf" | ["bolt://neo4j-core-2.neo4j.default.svc.cluster.local:7687", "http://neo4j-core-2.neo4j.default.svc.cluster.local:7474", "https://neo4j-core-2.neo4j.default.svc.cluster.local:7473"] | "FOLLOWER" | [] |
| "c4252e52-b4a4-4e88-8e1f-4773989c555a" | ["bolt://neo4j-core-3.neo4j.default.svc.cluster.local:7687", "http://neo4j-core-3.neo4j.default.svc.cluster.local:7474", "https://neo4j-core-3.neo4j.default.svc.cluster.local:7473"] | "FOLLOWER" | [] |
| "fca4057a-bd8b-4b16-a439-0cbce12ffe53" | ["bolt://neo4j-core-4.neo4j.default.svc.cluster.local:7687", "http://neo4j-core-4.neo4j.default.svc.cluster.local:7474", "https://neo4j-core-4.neo4j.default.svc.cluster.local:7473"] | "FOLLOWER" | [] |
| "d8c92a76-82c9-4146-9742-ce422e66cd08" | ["bolt://neo4j-replica-3562563584-7qs3x:7687", "http://neo4j-replica-3562563584-7qs3x:7474", "https://neo4j-replica-3562563584-7qs3x:7473"] | "READ_REPLICA" | [] |
| "dac490ed-3ad7-4c86-b22f-41e0a429dce2" | ["bolt://neo4j-replica-3562563584-c8sqd:7687", "http://neo4j-replica-3562563584-c8sqd:7474", "https://neo4j-replica-3562563584-c8sqd:7473"] | "READ_REPLICA" | [] |
| "df1f9885-3197-4546-a163-0e6a357df0c5" | ["bolt://neo4j-replica-3562563584-sckr8:7687", "http://neo4j-replica-3562563584-sckr8:7474", "https://neo4j-replica-3562563584-sckr8:7473"] | "READ_REPLICA" | [] |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
8 rows available after 6 ms, consumed after another 3 ms
Success!
By default we’ve disabled security to make it easier to play around with the cluster.
Once you’re ready to deploy to production you’ll want to protect your servers, which you can do by removing the NEO4J_dbms_security_auth__enabled
environment variable in cores/statefulset.yaml and read-replicas/deployment.yaml
Helm is a tool that streamlines installing and managing Kubernetes applications. You can think of it as an App Store for Kubernetes.
We created a Neo4j Helm package and it was recently accepted into the Helm incubator.
You can deploy Neo4j on Kubernetes using the Helm package by running the following commands:
helm repo add incubator https://kubernetes-charts-incubator.storage.googleapis.com/
helm install incubator/neo4j --name neo-helm
This will create a 3 core cluster. You can read more about this in our blog post.