GlusterFS is an open source scale-out filesystem. This solution references installation guide for Centos and installation guide for RHEL to provision a GlusterFS server cluster on Azure. It also provides examples to how to mount GlusterFS volumes locally and how to use GlusterFS volumes from Kubernetes containers.
Deploy the Gluster cluster on Azure using the Deploy button ☝️ or using the Azure CLI.
From Azure CLI, populate azuredeploy.parameters.json with the following parameters before starting the deployment.
az group deployment create --resource-group <RESOURCEGROUP> --template-file azuredeploy.json --parameters azuredeploy.parameters.json
Below is the list of template parameters:
| Name | Required | Description | Default |
|---|---|---|---|
| adminUsername | ✔️ | SSH user name | |
| adminPassword | ✔️ | SSH user password | |
| hostOs | ✔️ | OS to install on the host system. Allowed values: Centos, Ubuntu, and RHEL |
Centos |
| nodeNumber | ✔️ | Number of nodes in the gluster file system. Allowed values: 2, 4, 6, and 8 |
2 |
| diskNumber | ✔️ | Number of disks per node. Allowed values: 2, 3 and 4 |
2 |
| rhsmUsername | [RHEL only] Name of the virtual network to be used | ||
| rhsmPassword | [RHEL only] Name of the virtual network to be used | ||
| rhsmPoolId | [RHEL only] Name of the virtual network to be used | ||
| virtualNetworkName | ✔️ | Name of the virtual network to be used | |
| virtualNetworkNewOrExisting | ✔️ | Create a new or use existing Virtual Network. Allowed values: new and existing |
new |
| vmNamePrefix | ✔️ | VM name prefix, a number will be appended for each node | |
| volumeName | ✔️ | Gluster file system volume name | gfsvol |
To verify the new gluster volume is created and running, run the following on one of the gluster nodes:
# gluster volume info
Volume Name: gfsvol
Type: Replicate
Volume ID: 2bff68ee-xxxx-xxxx-xxxx-bbd6b3fcfa51
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: <NODE1>:/datadrive/brick
Brick2: <NODE2>:/datadrive/brick
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
To mount the new gluster volume to a new host, install the GlusterFS native client from here.
Use the private ip or node name of one of the gluster hosts to mount to a local mount directory
# mount -t glusterfs HOSTNAME-OR-IPADDRESS:/VOLNAME MOUNTDIR
# mount -t glusterfs 10.0.0.10:/gfsvol /mnttest/
To verify the size of the new gluster volume, run df -h you should see the local mount directory with the same size as the total size of all the managed disks you have created.
# df -h
...
10.0.0.10:/gfsvol 2.0T 81M 1.9T 1% /mnttest
..
If you want access gluster volumes from Kubernetes pods, one way is to provision the gluster cluster in the same vnet as your Kubernetes cluster.
Assuming you already have a K8s cluster running, update the azuredeploy.parameters.json file to provision a gluster cluster in the same vnet and same resource group as your k8s cluster.
"virtualNetworkName": {
"value": "k8s-vnet-xxxxxxxx"
},
"virtualNetworkNewOrExisting": {
"value": "existing"
},
From Azure CLI,
az group deployment create --resource-group <KUBERNETES-CLUSTER-RESOURCEGROUP> --template-file azuredeploy.json --parameters azuredeploy.parameters.json
Now that you have a gluster cluster and a kubernetes cluster running in the same vnet, use this documentation to allow containers to use GlusterFS volumes.
$ kubectl create -f _output/glusterfs-endpoints.json
endpoints "glusterfs-cluster" created
$ kubectl create -f _output/glusterfs-service.json
service "glusterfs-cluster" created
$ kubectl create -f _output/glusterfs-pod.json
Once the pod is running, let's verify the gluster volume has been mounted:
$ kubectl exec -it glusterfs -- /bin/bash
root@glusterfs:/# df -h
...
10.0.0.10:gfsvol 2.0T 81M 1.9T 1% /mnt/gluster
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent