ITK filters accelerated with OpenCL via [clEsperanto](https://clesperanto.github.io/).
License: Apache License 2.0
ITK filters accelerated with OpenCL via clEsperanto.
ITK is an open-source, cross-platform library that provides developers with an extensive suite of software tools for image analysis. Developed through extreme programming methodologies, ITK employs leading-edge algorithms for registering and segmenting multidimensional scientific images.
From https://github.com/InsightSoftwareConsortium/ITKCLEsperanto/runs/8164285776?check_suite_focus=true:
[66/113] Building CXX object _deps/clci-build/clic/CMakeFiles/CLIc.dir/src/tier2/cleDetectMaximaKernel.cpp.o
FAILED: _deps/clci-build/clic/CMakeFiles/CLIc.dir/src/tier2/cleDetectMaximaKernel.cpp.o
/Applications/Xcode_11.7.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -DCLIC_DEBUG=0 -I/Users/runner/work/ITKCLEsperanto/ITKCLEsperanto/_skbuild/macosx-10.9-universal2-3.10/cmake-build/_deps/clci-src/clic/include/core -I/Users/runner/work/ITKCLEsperanto/ITKCLEsperanto/_skbuild/macosx-10.9-universal2-3.10/cmake-build/_deps/clci-src/clic/include/tier1 -I/Users/runner/work/ITKCLEsperanto/ITKCLEsperanto/_skbuild/macosx-10.9-universal2-3.10/cmake-build/_deps/clci-src/clic/include/tier2 -I/Users/runner/work/ITKCLEsperanto/ITKCLEsperanto/_skbuild/macosx-10.9-universal2-3.10/cmake-build/_deps/clci-src/clic/include/tier3 -I/Users/runner/work/ITKCLEsperanto/ITKCLEsperanto/_skbuild/macosx-10.9-universal2-3.10/cmake-build/_deps/clci-src/clic/include/tier4 -I/Users/runner/work/ITKCLEsperanto/ITKCLEsperanto/_skbuild/macosx-10.9-universal2-3.10/cmake-build/_deps/clci-src/clic/include/tier5 -I/Users/runner/work/ITKCLEsperanto/ITKCLEsperanto/_skbuild/macosx-10.9-universal2-3.10/cmake-build/_deps/clci-src/clic/include/tier6 -I/Users/runner/work/ITKCLEsperanto/ITKCLEsperanto/_skbuild/macosx-10.9-universal2-3.10/cmake-build/_deps/clci-src/clic/kernels -I/Users/runner/work/ITKCLEsperanto/ITKCLEsperanto/_skbuild/macosx-10.9-universal2-3.10/cmake-build/_deps/clci-src/thirdparty/opencl/ocl-headers -I/Users/runner/work/ITKCLEsperanto/ITKCLEsperanto/_skbuild/macosx-10.9-universal2-3.10/cmake-build/_deps/clci-src/thirdparty/opencl/ocl-clhpp/include -O3 -DNDEBUG -arch x86_64 -isysroot /Applications/Xcode_11.7.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk -mmacosx-version-min=10.9 -fPIC -std=c++11 -MD -MT _deps/clci-build/clic/CMakeFiles/CLIc.dir/src/tier2/cleDetectMaximaKernel.cpp.o -MF _deps/clci-build/clic/CMakeFiles/CLIc.dir/src/tier2/cleDetectMaximaKernel.cpp.o.d -o _deps/clci-build/clic/CMakeFiles/CLIc.dir/src/tier2/cleDetectMaximaKernel.cpp.o -c /Users/runner/work/ITKCLEsperanto/ITKCLEsperanto/_skbuild/macosx-10.9-universal2-3.10/cmake-build/_deps/clci-src/clic/src/tier2/cleDetectMaximaKernel.cpp
In file included from /Users/runner/work/ITKCLEsperanto/ITKCLEsperanto/_skbuild/macosx-10.9-universal2-3.10/cmake-build/_deps/clci-src/clic/src/tier2/cleDetectMaximaKernel.cpp:6:
/Users/runner/work/ITKCLEsperanto/ITKCLEsperanto/_skbuild/macosx-10.9-universal2-3.10/cmake-build/_deps/clci-src/clic/include/core/utils.hpp:24:23: error: call to 'abs' is ambiguous
difference += std::abs( roundf(static_cast<float>(*it_output) * 10000)/10000 - roundf(static_cast<float>(*it_valid) * 10000)/10000 );
^~~~~~~~
/Applications/Xcode_11.7.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include/stdlib.h:132:6: note: candidate function
int abs(int) __pure2;
^
/Applications/Xcode_11.7.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/stdlib.h:110:44: note: candidate function
inline _LIBCPP_INLINE_VISIBILITY long abs( long __x) _NOEXCEPT {return labs(__x);}
^
/Applications/Xcode_11.7.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/stdlib.h:112:44: note: candidate function
inline _LIBCPP_INLINE_VISIBILITY long long abs(long long __x) _NOEXCEPT {return llabs(__x);}
^
1 error generated.
Test project /home/tom/builds/ITKCLEsperanto
Start 5: itkCLEAddImageTestPython
1/1 Test #5: itkCLEAddImageTestPython .........***Failed 2.96 sec
Loading ITKPyBase... done
Loading ITKCommon... done
Loading ITKStatistics... done
Loading ITKImageSources... done
Loading ITKImageFilterBase... done
Loading ITKTransform... done
Loading ITKImageFunction... done
Loading ITKPath... done
Loading ITKMesh... done
Loading ITKSpatialObjects... done
Loading ITKImageCompose... done
Loading ITKImageStatistics... done
Loading ITKImageGrid... done
Loading ITKImageIntensity... done
Loading CLEsperanto... done
Loading ITKPyUtils... done
Traceback (most recent call last):
File "itkCLEAddImageTestPython.py", line 42, in <module>
raise ValueError(f'{output_image.GetPixel([i,j])} != {expected_sum} at [{i},{j}]')
ValueError: 1.1205306214998533e-17 != 3.0 at [0,0]
itkTestDriver: Process exited with return value: 1
0% tests passed, 1 tests failed out of 1
Label Time Summary:
CLEsperanto = 2.96 sec*proc (1 test)
Python = 2.96 sec*proc (1 test)
Total Test time (real) = 2.96 sec
The following tests FAILED:
5 - itkCLEAddImageTestPython (Failed)
Errors while running CTest
Test Graft.
Error in B == Expected
In /home/runnerx/actions-runner/_work/ITKCLEsperanto/ITKCLEsperanto/test/itkCLEImageTest.cxx, line 151
lh: 7
rh: 22
Expression is not equal
There should be few, if any, changes required. We will have to enable CiRun -- I already enabled on the dashboard, but @tbirdso you should now have admin on this repository to make any additional changes.
As a follow-up to #42
Explicitly setting -DOpenCL_INCLUDE_DIR at config results in that include directory being used for OpenCL headers
CMakeCache.txt has OpenCL_INCLUDE_DIR=<build_dir>/_deps/clci-src/thirdparty/opencl/ocl-headers;<build_dir>/_deps/clci-src/thirdparty/opencl/ocl-clhpp/include after config.
Windows 10
CMake 3.22.3
ITKCLEsperanto@067723f0a0bf77414ce6289c414f5508d2185dc5
Tested with OpenCL-ICD-Loader include directory per CI steps.
OpenCL execution crashes for high computational kernel execution codes. If a triple nested loop is implemented and each loop needs to iterate over a large number of values, the execution time increases exponentially and even crashes if a watchdog timer for the graphic card is activated (this is my hypothesis).
This graphic card limitation is not an ITK bug itself, but I think that ITK should warn about possible execution crashing for big images and long execution times per kernel.
This bug was tested with a synthetic image and with itkGPUMeanImageFilter class. This class has (for 3D images) 3 nested loops per kernel. In each pixel, surrounding pixel values are measured and the mean is calculated. These surrounding pixels are determined by the radius specified. The higher the radius is, the more pixels are analysed for mean calculation. High radius values lead to a three nested loop execution for many iterations per loop. If high radius value is combined with high image buffer size (big image), the graphic card crashes and OpenCL cannot be accessed until a new context is created (stop execution and rerun).
The error and exception thrown by OpenCL is CL_INVALID_COMMAND_QUEUE.
After this error is raised, any call to OpenCL leads to CL_OUT_OF_RESOURCES error.
This error is probably caused by a watchdog timer that stops kernel execution due to high execution time. In those computers where this watchdog is not activated, the execution finishes successfully, but after a very long time.
The tests performed in watchdog timer deactivated computers with dummy calculations, led to these execution times:
In order to reproduce this, a big image has to be created and itkGPUMeanImageFilter should be applied with a high radius value. The code posted below works for radius values less or equal four, but it crashes (or lasts too much) for values higher or equal five.
#include <iostream>
#include <itkGPUImage.h>
#include <itkGPUMeanImageFilter.h>
#include <itkImageFileReader.h>
#include <itkTimeProbe.h>
using PixelType = float;
using ImageType = itk::GPUImage<PixelType, 3>;
using MeanFilterType = itk::GPUMeanImageFilter<ImageType, ImageType>;
int
main()
{
// Create synthetic image
typename ImageType::RegionType imageRegion;
typename ImageType::IndexType imageIndex;
imageIndex[0] = 0;
imageIndex[1] = 0;
imageIndex[2] = 0;
typename ImageType::SizeType imageSize;
imageSize[0] = 500;
imageSize[1] = 500;
imageSize[2] = 200;
imageRegion.SetIndex(imageIndex);
imageRegion.SetSize(imageSize);
typename ImageType::Pointer volume = ImageType::New();
volume->SetRegions(imageRegion);
volume->Allocate();
using VolumeIteratorType = itk::ImageRegionIteratorWithIndex<ImageType>;
VolumeIteratorType it(volume, imageRegion);
for (it.GoToBegin(); !it.IsAtEnd(); ++it)
{
it.Set(1000);
}
// Test the execution time of the filter. Execution time is the indicative of the GPU limitation
// in cases where the watchdog timer does not crash the program.
MeanFilterType::Pointer mean = MeanFilterType::New();
mean->SetInput(volume);
mean->SetRadius(3); // normal execution
// mean->SetRadius(6); // abnormal execution
itk::TimeProbe timer;
timer.Start();
mean->Update();
timer.Stop();
std::cout << "Elapsed time in mean computation: " << timer.GetMean() << '\n';
}I would expect this code to perform successfully for any radius value (ideally). I would expect higher execution times for higher radius values, but those times should increase coherently with the radius value, the image size and hardware characteristics.
Additionally, I would not expect such exponential rises in terms of execution time due to small kernel radius increments.
It should not have a radius limit, above which the execution times increase in more than 100% with minor radius changes (e.g. radius 32 lasts 0.4s while radius 34 lasts 257s).
It may be useful to provide a user warning with respect to the maximum kernel radius allowed in case of convolutional filters, based on the characteristics of images and hardware detected. At this purpose, a clarification on how to compute the algorithm limits may be useful.
For every image size, there is a radius limit. Below this limit, the execution is fast. Above this limit, the execution is extremely slow and graphic card crashes if watchdog timer is activated.
The radius limit depending on the image size:
This error happens every time.
This test was executed using ITK 4.13 and 5.0. The problem is not ITK itself, the error is a bad graphic card management, but may be useful to warn the user about it.
This test was performed in Windows with a Quadro P1000 graphic card and the test crashed for radius values above the limit.
It was also tested in Windows with a Quadro P2000 graphic card and the limits were remaining but the execution didn´t crash. It lasted more than it should, but it did not crash as we believe the watchdog timer was not activated.
The test was executed in Linux with a Quadro P2000 graphic card and the same behavior as in the second case was obtained.
Traceback (most recent call last):
repairing D:\a\im\dist\itk_clesperanto-0.2.0-cp37-cp37m-win_amd64.whl
File "C:\Python37-x64\lib\runpy.py", line 193, in _run_module_as_main
finding DLL dependencies
"__main__", mod_spec)
File "C:\Python37-x64\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "C:\P\IPP\venv-37-x64\Scripts\delvewheel.exe\__main__.py", line 7, in <module>
File "C:\P\IPP\venv-37-x64\lib\site-packages\delvewheel\__main__.py", line 67, in main
wr.repair(args.target, no_mangles, args.no_mangle_all, args.lib_sdir)
File "C:\P\IPP\venv-37-x64\lib\site-packages\delvewheel\wheel_repair.py", line 436, in repair
discovered, ignored = patch_dll.get_all_needed(extension_module_path, self._no_dlls, self._wheel_dirs, 'raise', self._verbose)[:2]
File "C:\P\IPP\venv-37-x64\lib\site-packages\delvewheel\patch_dll.py", line 294, in get_all_needed
raise FileNotFoundError(f'Unable to find library: {dll_name}')
FileNotFoundError: Unable to find library: opencl.dll
SCRIPT_DIR: C:\P\IPP\scripts
ROOT_DIR: D:\a\im
Python3_EXECUTABLE: C:\P\IPP\venv-37-x64\Scripts\python.exe
Python3_INCLUDE_DIR: C:/Python37-x64/include
Python3_LIBRARY: C:/Python37-x64/libs/python37.lib
NINJA_EXECUTABLE:D:\a\_temp\-1645359103\ninja.exe
ITKDIR: C:\P\IPP\ITK-win_37-x64
Library paths for fixup: C:/P/IPP/oneTBB-prefix/bin
Traceback (most recent call last):
File "C:\P\IPP\scripts\windows_build_module_wheels.py", line 142, in <module>
fixup_wheels(args.py_envs, ';'.join(args.lib_paths), ';'.join(args.exclude_libs))
File "C:\P\IPP\scripts\windows_build_module_wheels.py", line 129, in fixup_wheels
fixup_wheel(py_envs, wheel, lib_paths, exclude_libs)
File "C:\P\IPP\scripts\windows_build_module_wheels.py", line 119, in fixup_wheel
os.path.join(ROOT_DIR, "dist"), filepath])
File "C:\Python37-x64\lib\subprocess.py", line 347, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['C:/P/IPP\\venv-37-x64\\Scripts\\delvewheel.exe', 'repair', '--no-mangle-all', '--add-path', 'C:/P/IPP/oneTBB-prefix/bin', '--no-dll', '', '--ignore-in-wheel', '-w', 'D:\\a\\im\\dist', 'D:\\a\\im\\dist\\itk_clesperanto-0.2.0-cp37-cp37m-win_amd64.whl']' returned non-zero exit status 1.
Failure to compile for Linux Python CI:
-- Found OpenCL: /usr/lib64/libOpenCL.so (found version "2.2")
CMake Error at /usr/share/cmake-3.23/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
-- Configuring incomplete, errors occurred!
Could NOT find Python (missing: Python_INCLUDE_DIRS Python_LIBRARIES
See also "/work/_skbuild/linux-x86_64-3.8/cmake-build/CMakeFiles/CMakeOutput.log".
Development Development.Module Development.Embed) (found version "2.7.5")
See also "/work/_skbuild/linux-x86_64-3.8/cmake-build/CMakeFiles/CMakeError.log".
Call Stack (most recent call first):
/usr/share/cmake-3.23/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
/usr/share/cmake-3.23/Modules/FindPython.cmake:561 (find_package_handle_standard_args)
_skbuild/linux-x86_64-3.8/cmake-build/_deps/clci-src/CMakeLists.txt:16 (find_package)
File "/opt/python/cp38-cp38/lib/python3.8/site-packages/skbuild/setuptools_wrap.py", line 544, in setup
env = cmkr.configure(cmake_args,
File "/opt/python/cp38-cp38/lib/python3.8/site-packages/skbuild/cmaker.py", line 221, in configure
raise SKBuildError(
An error occurred while configuring with CMake.
Command:
"cmake" "/work" "-G" "Ninja" "-DCMAKE_INSTALL_PREFIX:PATH=/work/_skbuild/linux-x86_64-3.8/cmake-install" "-DPYTHON_EXECUTABLE:FILEPATH=/opt/python/cp38-cp38/bin/python" "-DPYTHON_VERSION_STRING:STRING=3.8.13" "-DPYTHON_INCLUDE_DIR:PATH=/opt/_internal/cpython-3.8.13/include/python3.8" "-DPYTHON_LIBRARY:FILEPATH=libpython3.8.a" "-DSKBUILD:BOOL=TRUE" "-DCMAKE_MODULE_PATH:PATH=/opt/python/cp38-cp38/lib/python3.8/site-packages/skbuild/resources/cmake" "-DITK_DIR:PATH=/work/ITK-cp38-cp38-manylinux_2_28_x64" "-DITK_USE_SYSTEM_SWIG:BOOL=ON" "-DWRAP_ITK_INSTALL_COMPONENT_IDENTIFIER:STRING=PythonWheel" "-DSWIG_EXECUTABLE:FILEPATH=/work/ITK-cp38-cp38-manylinux_2_28_x64/Wrapping/Generators/SwigInterface/swig/bin/swig" "-DCMAKE_CXX_COMPILER_TARGET:STRING=x86_64-linux-gnu" "-DBUILD_TESTING:BOOL=OFF" "-DPython3_EXECUTABLE:FILEPATH=/opt/python/cp38-cp38/bin/python" "-DPython3_INCLUDE_DIR:PATH=/opt/python/cp38-cp38/bin/../include/python3.8" "-DCMAKE_BUILD_TYPE:STRING=Release"
2022-09-02T18:53:20.4583802Z -- Generating done
2022-09-02T18:53:20.4685154Z CMake Warning:
2022-09-02T18:53:20.4685758Z Manually-specified variables were not used by the project:
2022-09-02T18:53:20.4685963Z
2022-09-02T18:53:20.4686046Z PYTHON_EXECUTABLE
2022-09-02T18:53:20.4686268Z PYTHON_INCLUDE_DIR
2022-09-02T18:53:20.4686498Z PYTHON_VERSION_STRING
2022-09-02T18:53:20.4686662Z
itk-wheel-tagGPU test failure resolved (depends on #31)
I am using GPU Demons for myself project. I find a strange bug: the size of input (fix and moving image) is (174, 256, 256), but the size of output is (87, 128, 128).
The main code is:
RegistrationFilterType::Pointer DemonsFilter = RegistrationFilterType::New();
CommandIterationUpdate::Pointer observer = CommandIterationUpdate::New();
DemonsFilter->AddObserver(itk::IterationEvent(), observer);
DemonsFilter->SetFixedImage(fixImageCPUToGPUFilter->GetOutput());
DemonsFilter->SetMovingImage(movingImageCPUToGPUFilter->GetOutput());
DemonsFilter->GetOutput()->SetRequestedRegionToLargestPossibleRegion();
auto sizexxx = gpuDisplacementField->GetLargestPossibleRegion().GetSize();
if (i != 0)
{
DemonsFilter->SetInitialDisplacementField(gpuDisplacementField);
}
else
{
DemonsFilter->SetInitialDisplacementField(nullptr);
}
DemonsFilter->SetNumberOfIterations(nIterations[i]);
DemonsFilter->SetStandardDeviations(1.0);
DemonsFilter->Update();
auto fixImageSize = DemonsFilter->GetFixedImage()->GetLargestPossibleRegion().GetSize();
std::cout << "fix image size: " << fixImageSize << std::endl;
auto movingImageSize = DemonsFilter->GetMovingImage()->GetLargestPossibleRegion().GetSize();
std::cout << "moving image size: " << movingImageSize << std::endl;
auto displacementSize = DemonsFilter->GetOutput()->GetLargestPossibleRegion().GetSize();
std::cout << "displacement size: " << displacementSize << std::endl;
And the information in the console is:
fix image size: [174, 256, 256]
moving image size: [174, 256, 256]
displacement size: [87, 128, 128]
Why the size of displacement is not (174, 256, 256) ??
Moreover, I have visualized the displacement field, and the result is just random noise. Also, I have checked the input fix and moving image, and they are correct.
How can I make the demons registration correct?
From https://github.com/InsightSoftwareConsortium/ITKCLEsperanto/actions/runs/3054684986:
The ubuntu-18.04 environment is deprecated, consider switching to ubuntu-20.04(ubuntu-latest), or ubuntu-22.04 instead. For more details see actions/runner-images#6002
The CLIc target rebuilds every time a consuming project is built regardless of whether there have been changes in CLIc sources. This is a nonblocking issue but does add about a minute to every build iteration.
Initially logging here but may need to be addressed in the CLEsperanto dependency.
Windows 10
Visual Studio 2022
CMake 3.22.3
Cloning into '/home/runnerx/actions-runner/_work/ITKCLEsperanto/build/_deps/clci-src/thirdparty/opencl/ocl-clhpp/external/CMock/vendor/c_exception/vendor/unity'...
fatal: sha1 file '/home/runnerx/actions-runner/_work/ITKCLEsperanto/build/_deps/clci-src/.git/modules/thirdparty/opencl/ocl-clhpp/modules/external/CMock/modules/vendor/c_exception/modules/vendor/unity/objects/pack/tmp_idx_Y2sM2e' write error: No space left on device
fatal: index-pack failed
fatal: clone of 'https://github.com/throwtheswitch/unity.git' into submodule path '/home/runnerx/actions-runner/_work/ITKCLEsperanto/build/_deps/clci-src/thirdparty/opencl/ocl-clhpp/external/CMock/vendor/c_exception/vendor/unity' failed
Update build-test-package.yml with Python 3.10 build config for Windows, Linux
The CLE object we are using as a buffer does not yet have print support, but it seems like they have the intention to have one
https://github.com/clEsperanto/CLIc_prototype/blob/master/clic/include/core/cleObject.hpp#L164
Warning in PyPI upload:
Warning: You are using "pypa/gh-action-pypi-publish@master". The "master" branch of this project has been sunset and will not receive any updates, not even security bug fixes. Please, make sure to use a supported version. If you want to pin to v1 major version, use "pypa/gh-action-pypi-publish@release/v1". If you feel adventurous, you may opt to use use "pypa/gh-action-pypi-publish@unstable/v1" instead. A more general recommendation is to pin to exact tags or commit shas.
See https://github.com/InsightSoftwareConsortium/ITKCLEsperanto/actions/runs/3054684986/jobs/4956033503
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.