intel / gpgmm Goto Github PK

GPGMM, a General-Purpose GPU Memory Management Library.

License: Apache License 2.0

Python 1.63% C++ 95.18% C 0.98% Shell 0.02% CMake 2.18%

d3d12 vulkan

gpgmm's Introduction

Build Config	Build Status
GN/Clang
CMake/MSVC

GPGMM

GPGMM is a General-Purpose GPU Memory Management C++ library used by GPU applications/runtimes that use modern graphics and compute APIs (D3D12 or Vulkan) that allow low-level memory management. GPGMM is a fast, multi-threaded, full-fledged GPU Memory Manager (GMM) implementation that replaces what older graphics and compute APIs (D3D11 or OpenGL) had accomplished through the GPU driver.

How do I use it?

Prerequisites

Error handing uses GPU API error codes (HRESULT and VkResult for D3D12 and Vulkan, respectively).

First create an allocator then create allocations from it:

D3D12

#include <gpgmm_d3d12.h>

gpgmm::d3d12::RESOURCE_ALLOCATOR_DESC allocatorDesc = {};

ComPtr<gpgmm::d3d12::IResidencyManager> residencyManager; // Optional
ComPtr<gpgmm::d3d12::IResourceAllocator> allocator;
gpgmm::d3d12::CreateResourceAllocator(desc, device, adapter, &allocator, &residencyManager);

D3D12_RESOURCE_DESC& resourceDesc = {};
D3D12_RESOURCE_STATES initialState = {}
gpgmm::d3d12::ALLOCATION_DESC allocationDesc = {};

ComPtr<gpgmm::d3d12::IResourceAllocation> allocation;
allocator->CreateResource(allocationDesc, resourceDesc, initialState, nullptr, &allocation);

GPUs do not support page-faulting, so it's up the GPU application to avoid using too much physical GPU memory. GPGMM integrates residency into the resource allocators to simplify and optimize allocation:

ComPtr<gpgmm::d3d12::IResidencyList> list;
gpgmm::d3d12::CreateResidencyList(&list);
list->Add(allocation->GetMemory());

residencyManager->ExecuteCommandList(&queue, &commandList, &list, 1);

// Prepare for next frame.
list->Reset();

Residency also works for non-resources too:

// Shader-visible heaps should be locked or always resident.
gpgmm::d3d12::RESIDENCY_HEAP_DESC shaderVisibleHeapDesc = {};
shaderVisibleHeapDesc.Flags |= RESIDENCY_HEAP_FLAG_CREATE_LOCKED;
shaderVisibleHeapDesc.HeapSegment = DXGI_MEMORY_SEGMENT_GROUP_LOCAL;

ComPtr<gpgmm::d3d12::IResidencyHeap> descriptorHeap;
gpgmm::d3d12::CreateResidencyHeap(shaderVisibleHeapDesc, residency.Get(),
      d3d12DescriptorHeap.Get(),
      &descriptorHeap);

To clean-up, simply call Release() once the is GPU is finished.

Vulkan

#include <gpgmm_vk.h>

gpgmm::vk::GpAllocatorCreateInfo allocatorInfo = {};

gpgmm::vk::GpResourceAllocator resourceAllocator;
gpgmm::vk::gpCreateResourceAllocator(allocatorInfo, &resourceAllocator)

VkBufferCreateInfo bufferInfo = {};
VkBuffer buffer;

gpgmm::vk::GpResourceAllocation allocation;
gpgmm::vk::GpResourceAllocationCreateInfo allocationInfo = {};

gpgmm::vk::gpCreateBuffer(resourceAllocator, &bufferInfo, &buffer, &allocationInfo, &allocation)

Then to clean-up:

gpgmm::vk::gpDestroyBuffer(resourceAllocator, buffer, allocation); // Make sure GPU is finished!
gpgmm::vk::gpDestroyResourceAllocator(resourceAllocator);

Project integration

Aside from DirectX or Vulkan libraries, GPGMM has no additional dependencies when deployed. GPGMM is distributed for use as source-only. It is recommended to use the built-in GN or CMake build targets. Alternatively, a shared library can be built to enable other build systems.

Source based builds

Allows GPGMM to be added as a source-package dependency to an existing build system.

GN

BUILD.gn

source_set("proj") {
  deps = [ "${gpgmm_dir}:src/gpgmm" ]
}

Create build_overrides/gpgmm.gni file in root directory.

To build with a backend, add the corresponding argument from following table in ``build_overrides/gpgmm.gni`:

Backend	Build override argument
DirectX 12	`gpgmm_enable_d3d12=true`
Vulkan	`gpgmm_enable_vk=true`

CMake

CMakeLists.txt

Using CMake 3.14 or newer:

include(FetchContent)
FetchContent_Declare(gpgmm
  GIT_REPOSITORY https://github.com/intel/gpgmm.git
  GIT_TAG main
)

# Specify GPGMM build options (see below).
...

FetchContent_MakeAvailable(gpgmm)

Or alternatively, manually fetch:

git clone https://github.com/intel/gpgmm.git third_party/gpgmm

And make available:

add_subdirectory(third_party/gpgmm)
...

# Specify GPGMM build options (see below).
...

Then add:

target_link_libraries(proj PRIVATE gpgmm)

Linked-library builds

Allows GPGMM to be built as a library-binary then linked as build-dependency.

vcpkg (package manager)

First install vcpkg then run install gpgmm.

vcpkg install gpgmm

GN

Use is_clang=false gpgmm_shared_library=true when Setting up the build. Then use ninja -C out/Release gpgmm or ninja -C out/Debug gpgmm to build the shared library.

CMake

cmake -DBUILD_SHARED_LIBS=ON -DCMAKE_BUILD_TYPE=Debug .
cmake --build . --config Debug

Visual Studio configuration

Copy the DLL into the $(OutputPath) folder and configure the VS build:
Highlight project in the Solution Explorer, and then select Project > Properties.
Under Configuration Properties > C/C++ > General, add gpgmm\include to Additional Include Directories.
Under Configuration Properties > Linker > Input, add gpgmm.dll.lib to Additional Dependencies.
Under Configuration Properties > Linker > General, add the folder path to out\Release to Additional Library Directories.

Build and Run

Building GPGMM for development.

GN

Install depot_tools and visual studio

GPGMM uses the Chromium build system and dependency management so you need to install depot_tools and add it to the PATH. Then install visual studio with the SDK and “Debugging Tools For Windows” package.

Notes:

On Windows, you'll need to set the environment variable DEPOT_TOOLS_WIN_TOOLCHAIN=0. This tells depot_tools to use your locally installed version of Visual Studio (by default, depot_tools will try to download a Google-internal version).

Get the code

Get the source code as follows:

# Clone the repo as "GPGMM"
> git clone https://github.com/intel/GPGMM.git GPGMM && cd GPGMM

# Bootstrap the gclient configuration
> cp scripts/standalone.gclient .gclient

# Fetch external dependencies and toolchains with gclient
> gclient sync

Configure the build

Generate build files using gn args out/Debug or gn args out/Release.

A text editor will appear asking build arguments, the most common argument is is_debug=true/false; otherwise gn args out/Release --list shows all the possible options.

Build

Then use ninja -C out/Release or ninja -C out/Debug to build.

CMake

Install `CMake`

GPGMM uses CMake 3.14 or higher for the build system and dependency management, so you need to install cmake and add it to the PATH.

Configure the build

Generate build files using cmake . -DCMAKE_BUILD_TYPE=Debug or cmake . -DCMAKE_BUILD_TYPE=Release.

To build with a backend, please specify the corresponding argument from following table.

Backend	Build argument
DirectX 12	`GPGMM_ENABLE_D3D12=ON`
Vulkan	`GPGMM_ENABLE_VK=ON`

For example, adding -DGPGMM_ENABLE_VK=OFF builds without Vulkan.

Additional build options further configure GPGMM:

Build option	Build argument
Disable compilation of tests	`GPGMM_ENABLE_TESTS=OFF`
GPGMM is NOT being built in GPGMM repo	`GPGMM_STANDALONE=OFF`
Compile GPGMM as shared library	`BUILD_SHARED_LIBS`

For example, adding -dGPGMM_STANDALONE=OFF, and -dBUILD_SHARED_LIBS builds GPGMM as a shared DLL.

Notes: CMake 3.14 or higher is required for GPGMM_ENABLE_TESTS or GPGMM_ENABLE_VK.

Vulkan-specific

The following build options are only available when GPGMM_ENABLE_VK=ON:

Build option	Build argument
Import Vulkan functions statically	`GPGMM_ENABLE_VK_STATIC_FUNCTIONS=ON`

For example, adding -dGPGMM_ENABLE_VK_STATIC_FUNCTIONS=ON will use Vulkan by statically linking functions (from the built-in Vulkan Loader).

Build

cmake --build .

Testing

Unit tests check the front-end code in isolation or without a GPU.
End2End tests check both the front AND backend code with a GPU.
Capture replay checks using pre-recorded memory patterns with a GPU.
Fuzzer checks using random memory patterns with a GPU.

GN

> cd out/Debug # or out/Release

Run unit tests:

> gpgmm_unittests

Run end2end tests:

> gpgmm_end2end_tests

Run capture replay tests:

> gpgmm_capture_replay_tests

To re-generate the capture:

gpgmm_capture_replay_tests.exe --gtest_filter=*Replay/* --event-mask=0x3 --disable-memory-playback

Run fuzzing tests:

> gpgmm_*_fuzzer  -max_total_time=<seconds_to_run>

CMake

> cd Debug # or Release

Run ALL tests (after build)

> cmake --build . --target RUN_TESTS

License

Apache 2.0 Public License, please see LICENSE.

gpgmm's People

Contributors

Stargazers

Watchers

Forkers

bbernhar fujunwei greenbaum isabella232 bjjones python3kgae chencha3

gpgmm's Issues

NoLeak tests appear to be flaky

Examples,

https://github.com/intel/GPGMM/actions/runs/2778014818
https://github.com/intel/GPGMM/actions/runs/2778014810

Add callback API to register backgrounds threads

Need a call-back API so spawned threads accessed by capture tools can be exempt.

Sync cmake DEPS versions

See https://github.com/intel/GPGMM/actions/runs/3491420677/jobs/5844031949

Add detection for resource alignment rejection

ID3D12Device::GetResourceAllocationInfo will sometimes force a higher alignment for small textures (4KB vs 64KB or 64KB vs 4MB for non-msaa and msaa respectively). Detect this so we can inform the driver which formats should be most favorably small aligned.

[Meta] Support for building through CMakelist

Remove Usage of CreateHeapFn in MVI

Chromium doesn't allow usage of std::function, which is what the CreateHeapFn is defined as.

I see two options:

Keep this as a callback and take a dependency on chromium base::OnceCallback.
Get rid of the callback param and inline the code in some manner, probably will add a ResourceAllocator::CreateCommittedResource function and call it.

2 is my preferred option since we don't have any other possible values for CreateHeapFn right now. This will probably mean some code duplication in the future when adding other methods of heap creation (I think this is what's in Dawn).

@egalli
@bbernhar - I'm not sure how much autonomy you're giving me over GPGMM issues while you're on loan. I'm assuming you'll want to have high level input until you tell me otherwise.

Support pooled dynamic memory allocation for CPU allocations.

Replace new and delete to allocate and de-allocate, respectively, through a allocator that re-cycles CPU heaps.

Eviction on budget change events.

Budget change events occur less frequent resulting into larger budget changes and thus more heaps to evict at once. If some new heaps must be made resident, when over budget, there might be beneifit evicting more heaps sooner, on budget change, vs waiting until ExecuteCommandList(), evicting fewer heaps more frequently.

Enable GPGMM with WebNN

Enable WebNN's DML backend to use GPGMM for resource allocation and residency management.

Add support for page-aligned sub-allocation.

Add support to sub-allocate using page (vs power-of-two)-alignment to avoid internal fragmentation when placing resources in large heaps (ex. ML tensor data).

Avoid attemping to sub-allocate large resources.

Support GPU memory preallocation.

Right now, GPGMM only supports on-demand allocation. Meaning, GPGMM allocates only when the request is made then later, de-allocates iff the allocation is no longer in use. This is very slow. Instead of allocating memory "as needed", GPGMM could support pre-allocation so a pool of memory is always available or ready-to-allocate. Unfortunately, this could cause OOM frequently so the behavior must be optional.

Reduce memory usage using Trim

GPGMM pre-allocates internal memory to speed-up subsequent allocation requests. These allocations count against the app's memory usage. Trim()
would allow the app discard these internal buffers to reduce memory footprint.

Design and enable Trim for existing memory allocators.

Update budget using OS events or non-polling based.

Replaces use of QueryVideoMemoryInfo with RegisterVideoMemoryBudgetChangeNotificationEvent.

GPGMM question

I have a question. Is the library related to SYCL or RMM (https://github.com/rapidsai/rmm) ? Thanks.

De-couple memory management from Dawn

Eliminate dependencies on WGPUDevice.
- Require allocations to be explicitly freed rather then implicitly deferred deleted.
- Control which heaps are evicted with a separate submission fence.
- Put Dawn specific functionality behind flags (ALLOCATOR_FLAGS or ALLOCATION_FLAGS).
Define per backend API for allocation and residency operations.
- Allow limits (ex. sub-allocated heap size) to be client specified rather then hard coded.
- Conditionally enable pool-allocated resources.

Support for priority based residency

Uses ID3D12Device1::SetResidencyPriority to manage residency per resource/heap.

Implement and decide if/how this will work with ID3D12Device::[MakeResident|Evict] based residency via gpgmm::d3d12::ResidencySet.

Support for a minimum viable implementation

Allows users to leverage GPGMM by only including a couple source/headers files instead of the entire GPGMM project. This will be "lite" version of GPGMM which helps D3D developers use the full GPGMM interfaces but with a simple implementation.

Support zero initialized allocations

Heaps and committed resources already zero memory by default unless D3D12_HEAP_FLAG_CREATE_NOT_ZEROED gets specified.

Applications may require resources to be always initialized to zero (for security reasons) but GPGMM lacks a way to determine what is zeroed on a per allocation basis.

Meta issue to track progress to open source

This issue tracks the work to make GPGMM a public repository (or OSS). This means getting legal approval, figuring out licensing, deciding on the contribution model, and releases.

Fixes for Coverity scans

Simplify residency management API

Simplify the use of residency manager API by eliminating ResidencyManager::Evict/MakeResident and ResidencyManager::InsertHeap in favor of using higher-level APIs like ResidencyManager::LockHeap and ResidencyManager::ExecuteCommandLists` via a heap creation call-back.

Enable precise memory debugging

Report detailed information about leaked allocations. This is similar to ReportLiveDeviceObjects but at the allocation (or block) level.

Support testing other end2end integrations

Basically we want to integrate GPGMM in each client so we can use their existing end2end tests to test (new) GPGMM changes rather then rely on GPGMM end2end for coverage. We can limit the feature to chromium based clients (WebNN, Skia, and Dawn) for now.

Current status:

Project	Backend	Test
Dawn	D3D12: done, Vulkan: TBD	dawn_end2end_tests
Skia	D3D12: TODO, Vulkan: TBD
WebNN	DML: done	webnn_end2end_tests

intel / gpgmm Goto Github PK

gpgmm's Introduction

GPGMM

How do I use it?

Prerequisites

D3D12

Vulkan

Project integration

Source based builds

GN

CMake

Linked-library builds

vcpkg (package manager)

GN

CMake

Visual Studio configuration

Build and Run

GN

Install depot_tools and visual studio

Get the code

Configure the build

Build

CMake

Install CMake

Configure the build

Vulkan-specific

Build

Testing

GN

Run unit tests:

Run end2end tests:

Run capture replay tests:

Run fuzzing tests:

CMake

Run ALL tests (after build)

License

gpgmm's People

Contributors

Stargazers

Watchers

Forkers

gpgmm's Issues

Recommend Projects

Recommend Topics

Recommend Org

Install `CMake`