Hammerspace CSI Volume Plugin

This plugin uses Hammerspace backend as distributed data storage for containers.

Supports CSI Spec 1.1.0

Implements the Identity, Node, and Controller interfaces as single Golang binary.

Supported Capabilities

CREATE_DELETE_VOLUME
GET_CAPACITY
CREATE_DELETE_SNAPSHOT
STAGE_UNSTAGE_VOLUME
GET_VOLUME_STATS

Unsupported Capabilities

LIST_VOLUMES
LIST_SNAPSHOTS
EXPAND_VOLUME
CLONE_VOLUME

Volume Types

File-backed Block Volume (raw device)

Storage is exposed to the container as a raw device
Exists as a special device file on a Hammerspace share (backing share)

File-backed Mounted volume (filesystem)

Storage is exposed to the container as a directory
Exists as a special device file on a Hammerspace share (backing share) which contains a filesystem

Share-backed Mounted volume (shared filesystem)

Storage is exposed to the container as a directory
Exists as a Hammerspace share
Mounted via NFS

Plugin Dependencies

Ensure that nfs-utils is installed on the Kubernetes hosts

Ubuntu

$ apt install nfs-common

CentOS

$ yum install nfs-utils

The plugin container(s) must run as privileged containers

Installation

Kubernetes specific deployment instructions are located at here

Configuration

Configuration parameters for the driver (passed as environment variables to plugin container):

* Required

Variable	Default	Description
*`CSI_ENDPOINT`		Location on host for gRPC socket (Ex: /tmp/csi.sock)
*`CSI_NODE_NAME`		Identifier for the host the plugin is running on
`CSI_USE_ANVIL_FOR_DATA`	`true`	Whether to try mount shares as connections to the Anvil server over pNFS (NFS v4.2). If false, data portals are used.
*`HS_ENDPOINT`		Hammerspace API gateway
*`HS_USERNAME`		Hammerspace username (admin role credentials)
*`HS_PASSWORD`		Hammerspace password
`HS_TLS_VERIFY`	`false`	Whether to validate the Hammerspace API gateway certificates
`HS_DATA_PORTAL_MOUNT_PREFIX`		Override the prefix for data portal mounts. Ex `/mnt/data-portal`
`CSI_MAJOR_VERSION`	`"1"`	The major version of the CSI interface used to communicate with the plugin. Valid values are "1" and "0"

Usage

Supported volume parameters for CreateVolume requests (maps to Kubernetes storage class params):

Name	Default	Description
`exportOptions`		Export options applied to shares created by plugin. Format is ';' seperated list of subnet,access,rootSquash. Ex `*,RW,false; 172.168.0.0/20,RO,true`
`deleteDelay`	`-1`	The value of the delete delay parameter passed to Hammerspace when the share is deleted. '-1' implies Hammerspace cluster defaults.
`volumeNameFormat`	`%s`	The name format to use when creating shares or files on the backend. Must contain a single '%s' that will be replaced with unique volume id information. Ex: `csi-volume-%s-us-east`
`objectives`	`""`	Comma separated list of objectives to set on created shares and files in addition to default objectives.
`blockBackingShareName`		The share in which to store Block Volume files. If it does not exist, the plugin will create it. Alternatively, a preexisting share can be used. Must be specified if provisioning Block Volumes.
`mountBackingShareName`		The share in which to store File-backed Mount Volume files. If it does not exist, the plugin will create it. Alternatively, a preexisting share can be used. Must be specified if provisioning Filesystem Volumes other than 'nfs'.
`fsType`	`nfs`	The file system type to place on created mount volumes. If a value other than "nfs", then a file-backed volume is created instead of an NFS share.
`additionalMetadataTags`		Comma separated list of tags to set on files and shares created by the plugin. Format is ',' separated list of key=value pairs. Ex `storageClassName=hs-storage,fsType=nfs`

Topology support

Currently, only the topology.csi.hammerspace.com/is-data-portal key is supported. Values are 'true' and 'false'

Development

Requirements

Docker
Golang 1.12+
nfs-utils

Building

Build a new docker image from local source:

sudo make build

Build a new release:

Update VERSION file, then

make build-release

Publish a new release

docker push hammerspaceinc/csi-plugin:$(cat VERSION)

Testing

Manual tests

Manual tests can be facilitated by using the Dev Image. Local files can be exposed to the container to facilitate iterative development and testing.

Example Usage:

Building the image -

make build-dev

Create ENV file for plugin and csi-sanity configuration.

echo "
CSI_ENDPOINT=/tmp/csi.sock
HS_ENDPOINT=https://anvil.example.com
HS_USERNAME=admin
HS_PASSWORD=admin
HS_TLS_VERIFY=false
CSI_NODE_NAME=test
CSI_USE_ANVIL_FOR_DATA=true
SANITY_PARAMS_FILE=/tmp/csi_sanity_params.yaml
 " >  ~/csi-env

Create params file for csi-sanity (defines the parameters passed to CreateVolume)

echo "
blockBackingShareName: test-csi-block
deleteDelay: 0
objectives: "test-objective"
" > ~/csi_sanity_params.yaml

Running the image -

docker run --privileged=true \
-v /tmp/:/tmp/:shared \
-v /dev/:/dev/ \
--env-file ~/csi-env \
-it \
-v ~/csi_sanity_params.yaml:/tmp/csi_sanity_params.yaml \
-v ~/csi-plugin:/csi-plugin/:shared \
--name=csi-dev \
hammerspaceinc/csi-plugin-dev

Running CSI plugin in dev image

make compile # Recompile
./bin/hs-csi-plugin

Using csc to call the plugin -

# open additional shell into dev container
docker exec -it csi-dev /bin/sh

# use csc tool
## Call GetPluginInfo 
CSI_DEBUG=true CSI_ENDPOINT=/tmp/csi.sock csc identity plugin-info

## Make a 1GB file-backed mount volume
CSI_DEBUG=true CSI_ENDPOINT=/tmp/csi.sock csc controller create --cap 5,mount,ext4 --req-bytes 1073741824 --params mountBackingShareName=file-backed test-filesystem

## Delete volume
CSI_DEBUG=true CSI_ENDPOINT=/tmp/csi.sock csc controller delete  /file-backed/test-filesystem

## Explore additional commands
csc -h

Running unit tests

make unittest

Running Sanity tests

These tests are functional and will create and delete volumes on the backend.

Must have connections from the host to the HS_ENDPOINT. This can be run from within the Dev image. Uses the CSI sanity package

Make parameters

echo "
fsType: nfs
blockBackingShareName: test-csi-block
deleteDelay: 0
objectives: "test-objective"
" > ~/csi_sanity_params.yaml

Run sanity tests

export CSI_ENDPOINT=/tmp/csi.sock
export HS_ENDPOINT="https://anvil.example.com"
export HS_USERNAME=admin
export HS_PASSWORD=admin
export HS_TLS_VERIFY=false
export CSI_USE_ANVIL_FOR_DATA=true
export CSI_NODE_NAME=test
export SANITY_PARAMS_FILE=~/csi_sanity_params.yaml
make sanity

ameade / csi-plugin Goto Github PK

csi-plugin's Introduction