Gemini is an efficient GPU resource sharing system with fine-grained control for Linux platforms.

It shares a NVIDIA GPU among multiple clients with specified resource constraint, and works seamlessly with any CUDA-based GPU programs. Besides, it is also work-conserving and with low overhead, so nearly no compute resource waste will happen.

Gemini consists of two components: scheduler (back-end) and hook library (front-end).

• scheduler (GPU device manager) (gem-schd): A daemon process managing token. Based on information provided in resource configuration file (resource.conf), scheduler determines whom to give token. Clients can launch CUDA kernels only when holding a valid token.
• hook library (libgemhook.so.1): A library intercepting CUDA-related function calls. It utilizes the mechanism of LD_PRELOAD, which forces our hook library being loaded before any other dynamic linked libraries.

Currently we use Unix domain socket as the communication interface between components.

• libzmq
• glib-2.0
• gio-2.0

Basically all components can be built with the following command:

make [CUDA_PATH=/path/to/cuda/installation] [PREFIX=/place/to/install] [DEBUG=1]

This command will install the built binaries in $(PREFIX)/bin and $(PREFIX)/lib. Default value for PREFIX is $(pwd)/...

Adding DEBUG=1 in above command will make hook library and scheduler output more scheduling details.

We follow the syntax of XDG Desktop Entry Specification for configuration file.

Settings for a client group is described by a section like below:

[client_group_name]
MinUtil = 0.1  # minimum required ratio of GPU usage time (between 0 and 1); default is 0
MaxUtil = 0.5  # maximum allowed ratio of GPU usage time (between 0 and 1); default is 1
MemoryLimit = 2GiB  # maximum allowed GPU memory usage (in bytes); default is 1GiB

Suffixes like M, GB will be interpreted as power of 10 (e.g. 1 000 000 and 1 000 000 000), and suffixes like Ki, MiB will be interpreted as power of 2 (e.g. 1 024 and 1 048 576).

Default values will be used for unspecified parameters.

Changes to this file will be monitored by gem-schd. After each change, scheduler will read this file again and update settings.

Specify a directory in environment variable GEMINI_IPC_DIR for keeping unix domain socket files. Default value is /tmp/gemini/ipc.

When launching applications, set environment variable GEMINI_GROUP_NAME to name of client group and LD_PRELOAD to location of libgemhook.so.1.

For convenience, we provide a Python script tools/launch-command.py for launching applications. Refer to scripts and source codes for more details.

jim90247 eee4017 ncy9371