llnl / care Goto Github PK

CHAI and RAJA provide an excellent base on which to build portable codes. CARE expands that functionality, adding new features such as loop fusion capability and a portable interface for many numerical algorithms. It provides all the basics for anyone wanting to write portable code.

License: BSD 3-Clause "New" or "Revised" License

CMake 4.35% C 19.31% C++ 74.65% Python 0.12% Shell 1.58%

gpu gpu-computing gpu-programming gpu-acceleration portable portable-library portable-apps portableapps portable-executable portable-class-library

care's Introduction

CARE v0.13.0

CARE: CHAI and RAJA Extensions

CHAI and RAJA provide an excellent base on which to build portable code. CARE expands that functionality, adding new features such as loop fusion capability, a portable interface for many numerical algorithms, and additional data structures. It provides all the basics for anyone wanting to write portable code.

Getting Started

git submodule update --init --recursive
mkdir build
cd build
cmake ../   # May need to pass -DCMAKE_INSTALL_PREFIX=/path/to/install/in if the next instruction fails
make -j install

If desired, external libraries can be used instead of submodules. For example, an external CHAI can be specified with -DCHAI_DIR=<path to CHAI install directory or directory containing chai-config.cmake>. Note that if using an external CHAI, it must be configured with -DENABLE_PICK=ON -DENABLE_PINNED=ON.

To build with CUDA support, use -DENABLE_CUDA=ON -DCUDA_TOOLKIT_ROOT_DIR=/path/to/cuda/toolkit. If using external libraries, note that Umpire, RAJA, and CHAI must also be configured with those options.

To build with HIP support, set -DENABLE_HIP=ON -DHIP_ROOT_DIR=/path/to/rocm/hip/ -DHIP_CLANG_PATH=/path/to/rocm/clang. If using external libraries, note that Umpire, RAJA, and CHAI must also be configured with those options. If using an external BLT, note that version 0.6.1 or later is required. Other compilers besides the hip vendor-supplied clang compiler have not yet been tried for HIP builds.

To build with OpenMP support, use -DENABLE_OPENMP=ON. If using external libraries, Umpire, RAJA, and CHAI must also be configured with that option.

By default, only the tests are built. Documentation, benchmarks, and examples can be turned on with -DENABLE_DOCS=ON, -DENABLE_BENCHMARKS=ON, and -DENABLE_EXAMPLES=ON.

License

CARE is release under the BSD-3-Clause License. See the LICENSE and NOTICE files for more details.

SPDX-License-Identifier: BSD-3-Clause

LLNL-CODE-809741

care's People

Contributors

Stargazers

Watchers

Forkers

pierfied scchan bmhan12 mrowan137

care's Issues

File and line number or macro layer in care/util.h?

Some of the functions in care/util.h take a file and line number so if there is a GPU API error code returned, we can print out the file and line number of the calling location. Should all the functions take a file and line number? Do we want to provide a macro layer that injects the file and line number for users? Do we want to wait until c++ 20 when source_location becomes a thing (https://en.cppreference.com/w/cpp/utility/source_location)?

Universal CARE free function (both host and device memory)

Write a function free to replace both host free and gpuFree (cudaFree / hipFree )

A function like
cudaPointerGetAttributes (and similar function for HIP)

can query whether a pointer is allocated on the host or device or managed.

Add minmax_element

ArrayMin and ArrayMax start index

ArrayMin has an input parameter for start index but ArrayMax does not. Using slicing or pointer arithmetic instead and make the two consistent.

Fix TestLoopFuser and Benckmarks tests for CPU-only builds

Some tests in LoopFuserTest.cxx and Benchmarks.cpp have been ifdefed out for CPU-only builds to avoid failures; fix this.

Benchmarks test fails in GPU SIM mode

get warning about pinned memory, and a test fails

Add template parameter for Key in KeyValueSorter

Right now the KeyValueSorter just uses an int for the key type. Add another template parameter to allow using other types for the key.

Add Build and Test for Clang to Gitlab CI

Setting external dependencies does not work correctly

How to make it work correctly?

--->  Configuring CARE
Executing:  cd "/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_CARE/CARE/work/build" && /opt/local/bin/cmake -G "CodeBlocks - Unix Makefiles" -DCMAKE_BUILD_TYPE=MacPorts -DCMAKE_INSTALL_PREFIX="/opt/local" -DCMAKE_INSTALL_NAME_DIR="/opt/local/lib" -DCMAKE_SYSTEM_PREFIX_PATH="/opt/local;/usr" -DCMAKE_C_COMPILER="$CC" -DCMAKE_CXX_COMPILER="$CXX" -DCMAKE_OBJC_COMPILER="$CC" -DCMAKE_OBJCXX_COMPILER="$CXX" -DCMAKE_POLICY_DEFAULT_CMP0025=NEW -DCMAKE_POLICY_DEFAULT_CMP0060=NEW -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_COLOR_MAKEFILE=ON -DCMAKE_FIND_FRAMEWORK=LAST -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DCMAKE_MAKE_PROGRAM=/usr/bin/make -DCMAKE_MODULE_PATH="/opt/local/share/cmake/Modules" -DCMAKE_PREFIX_PATH="/opt/local/share/cmake/Modules" -DCMAKE_BUILD_WITH_INSTALL_RPATH:BOOL=ON -DCMAKE_INSTALL_RPATH="/opt/local/lib" -Wno-dev -DCAMP_DIR=/opt/local -DCARE_DISABLE_RAJAPLUGIN=OFF -DCARE_ENABLE_DOCS=OFF -DCARE_ENABLE_EXTERN_INSTANTIATE=OFF -DCARE_ENABLE_IMPLICIT_CONVERSIONS=ON -DCARE_ENABLE_MANAGED_PTR=ON -DCARE_ENABLE_REPRODUCERS=ON -DCARE_ENABLE_TESTS=ON -DCARE_LEGACY_COMPATIBILITY_MODE=ON -DCARE_NEVER_USE_RAJA_PARALLEL_SCAN=OFF -DCHAI_DIR=/opt/local -DENABLE_ASTYLE=OFF -DENABLE_CPPCHECK=ON -DENABLE_CLANGTIDY=OFF -DENABLE_BENCHMARKS=OFF -DENABLE_CUDA=OFF -DENABLE_DOCS=OFF -DENABLE_EXAMPLES=OFF -DENABLE_HIP=OFF -DENABLE_MPI=ON -DENABLE_PICK=ON -DENABLE_PINNED=ON -DENABLE_TESTS=OFF -DENABLE_UNCRUSTIFY=ON -DENABLE_VALGRIND=OFF -DENABLE_YAPF=OFF -DRAJA_DIR=/opt/local -DUMPIRE_DIR=/opt/local -DCMAKE_OSX_ARCHITECTURES="ppc" -DCMAKE_OSX_DEPLOYMENT_TARGET="10.6" -DCMAKE_OSX_SYSROOT="/" /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_CARE/CARE/work/CARE-0.11.0 
-- The CXX compiler identification is GNU 12.3.0
-- Checking whether CXX compiler has -isysroot
-- Checking whether CXX compiler has -isysroot - no
-- Checking whether CXX compiler supports OSX deployment target flag
-- Checking whether CXX compiler supports OSX deployment target flag - yes
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /opt/local/bin/mpicxx-mpich-gcc12 - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- CARE: Using BLT submodule
-- BLT Version: 0.5.2
-- CMake Version: 3.26.0-rc6
-- CMake Build Type: MacPorts
-- CMake Executable: /opt/local/bin/cmake
-- Setting CMAKE_CXX_EXTENSIONS to OFF
-- Standard C++14 selected
-- Found Git: /opt/local/bin/git (found version "2.41.0") 
-- Git Support is ON
-- Git Executable: /opt/local/bin/git
-- Git Version: 2.41.0
-- MPI Support is ON
-- FindMPI Enabled  (ENABLE_FIND_MPI == ON)
-- Found MPI_CXX: /opt/local/bin/mpicxx-mpich-gcc12 (found version "4.0") 
-- Found MPI: TRUE (found version "4.0")  
-- BLT MPI Compile Flags:  
-- BLT MPI Include Paths:  
-- BLT MPI Libraries:      
-- BLT MPI Link Flags:     SHELL:
-- MPI Executable:       /opt/local/bin/mpiexec
-- MPI Num Proc Flag:    -n
-- MPI Command Append:   
-- OpenMP Support is OFF
-- CUDA Support is OFF
-- HIP Support is OFF
-- Sphinx support is OFF
-- Valgrind support is OFF
-- AStyle support is OFF
-- ClangFormat support is OFF
-- Uncrustify support is ON
-- Found Uncrustify: /opt/local/bin/uncrustify  
-- Yapf support is OFF
-- CMakeFormat support is OFF
-- Cppcheck support is ON
-- Found Cppcheck: /opt/local/bin/cppcheck  
-- ClangQuery support is OFF
-- ClangTidy support is OFF
-- C Compiler family is GNU
-- Adding optional BLT definitions and compiler flags
-- Enabling all compiler warnings on all targets.
-- Fortran support disabled.
-- CMAKE_C_FLAGS flags are:   -Wall -Wextra  
-- CMAKE_CXX_FLAGS flags are:   -Wall -Wextra  -pipe -Os -DNDEBUG -I/opt/local/include -D_GLIBCXX_USE_CXX11_ABI=0   
-- CMAKE_EXE_LINKER_FLAGS flags are:  -L/opt/local/lib -Wl,-headerpad_max_install_names -Wl,-rpath,/opt/local/lib/libgcc
-- CARE: Using CAMP submodule
-- The C compiler identification is GNU 12.3.0
-- Checking whether C compiler has -isysroot
-- Checking whether C compiler has -isysroot - no
-- Checking whether C compiler supports OSX deployment target flag
-- Checking whether C compiler supports OSX deployment target flag - yes
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /opt/local/bin/mpicc-mpich-gcc12 - skipped
-- Detecting C compile features
-- Detecting C compile features - done
CMake Error at blt/cmake/BLTMacros.cmake:648 (add_library):
  add_library cannot create target "camp" because an imported target with the
  same name already exists.
Call Stack (most recent call first):
  tpl/camp/CMakeLists.txt:122 (blt_add_library)


CMake Error at tpl/camp/CMakeLists.txt:140 (target_include_directories):
  target_include_directories may only set INTERFACE properties on IMPORTED
  targets


CMake Error at tpl/camp/CMakeLists.txt:157 (install):
  install TARGETS given target "camp" which does not exist.


-- CARE: Using external Umpire
CMake Error at /opt/local/lib/cmake/raja/bltTargets.cmake:42 (message):
  Some (but not all) targets in this export set were already defined.

  Targets Defined: blt::blt_stub

  Targets not yet defined: blt::openmp, blt::mpi

Call Stack (most recent call first):
  /opt/local/lib/cmake/raja/raja-config.cmake:74 (include)
  cmake/SetupLibraries.cmake:81 (find_package)
  CMakeLists.txt:218 (include)


-- Configuring incomplete, errors occurred!
Command failed:  cd "/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_CARE/CARE/work/build" && /opt/local/bin/cmake -G "CodeBlocks - Unix Makefiles" -DCMAKE_BUILD_TYPE=MacPorts -DCMAKE_INSTALL_PREFIX="/opt/local" -DCMAKE_INSTALL_NAME_DIR="/opt/local/lib" -DCMAKE_SYSTEM_PREFIX_PATH="/opt/local;/usr" -DCMAKE_C_COMPILER="$CC" -DCMAKE_CXX_COMPILER="$CXX" -DCMAKE_OBJC_COMPILER="$CC" -DCMAKE_OBJCXX_COMPILER="$CXX" -DCMAKE_POLICY_DEFAULT_CMP0025=NEW -DCMAKE_POLICY_DEFAULT_CMP0060=NEW -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_COLOR_MAKEFILE=ON -DCMAKE_FIND_FRAMEWORK=LAST -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DCMAKE_MAKE_PROGRAM=/usr/bin/make -DCMAKE_MODULE_PATH="/opt/local/share/cmake/Modules" -DCMAKE_PREFIX_PATH="/opt/local/share/cmake/Modules" -DCMAKE_BUILD_WITH_INSTALL_RPATH:BOOL=ON -DCMAKE_INSTALL_RPATH="/opt/local/lib" -Wno-dev -DCAMP_DIR=/opt/local -DCARE_DISABLE_RAJAPLUGIN=OFF -DCARE_ENABLE_DOCS=OFF -DCARE_ENABLE_EXTERN_INSTANTIATE=OFF -DCARE_ENABLE_IMPLICIT_CONVERSIONS=ON -DCARE_ENABLE_MANAGED_PTR=ON -DCARE_ENABLE_REPRODUCERS=ON -DCARE_ENABLE_TESTS=ON -DCARE_LEGACY_COMPATIBILITY_MODE=ON -DCARE_NEVER_USE_RAJA_PARALLEL_SCAN=OFF -DCHAI_DIR=/opt/local -DENABLE_ASTYLE=OFF -DENABLE_CPPCHECK=ON -DENABLE_CLANGTIDY=OFF -DENABLE_BENCHMARKS=OFF -DENABLE_CUDA=OFF -DENABLE_DOCS=OFF -DENABLE_EXAMPLES=OFF -DENABLE_HIP=OFF -DENABLE_MPI=ON -DENABLE_PICK=ON -DENABLE_PINNED=ON -DENABLE_TESTS=OFF -DENABLE_UNCRUSTIFY=ON -DENABLE_VALGRIND=OFF -DENABLE_YAPF=OFF -DRAJA_DIR=/opt/local -DUMPIRE_DIR=/opt/local -DCMAKE_OSX_ARCHITECTURES="ppc" -DCMAKE_OSX_DEPLOYMENT_TARGET="10.6" -DCMAKE_OSX_SYSROOT="/" /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_CARE/CARE/work/CARE-0.11.0 
Exit code: 1

GPU failures with one care::host_device_ptr<T> constructor

If you attempt to construct an host_device_ptr like this:
int tempb[1] = {1};
care::host_device_ptr b(tempb);

Subsequent calls to array_utils functions will fail. Instead, you have to do this (constructor that includes the length and a name):

int tempb[1] = {1};
care::host_device_ptr b(tempb, 1, "fillhostdev");

We should disable the constructor that leads to failure, or have a different fix

Add min_element and deprecate ArrayMin

Add care::count

Add a care equivalent of std::count (see https://www.geeksforgeeks.org/std-count-cpp-stl/ ) to care

TestArrayUtils - inconsistent results on different builds

The test TestArrayUtils.cpp:886 (code snippet below) gives different results on the rzansel debug vs relwithdebinfo builds. Perhaps there is a memory issue? We should investigate

// change the cutoff
// CURRENTLY TEST GIVES DIFFERENT RESULTS WHEN COMPILED RELEASE VS DEBUG!!!
cutoff = 5.0;
result = care_utils::PickAndPerformFindMinIndex(a, nullptr, subsetrev, 7, threshold7, cutoff, threshIdx);

Add algorithm copy_n to match std::copy_n

We have ArrayCopy, but we should add copy_n and deprecate ArrayCopy.

ArrayUtils - reorganize and more tests

Two issues

Most functions in array_utils.h are declared at the top of the file, but some are declared somewhere in the middle. Move all function declarations to the top for consistency.
Many of the functions declared in the middle of the file don't have tests (I missed them). Add tests to TestArrayUtils.cpp

Add Missing Copy on freeDeviceMemory when resoruce manager is enabled on host only build.

Add Build and Test for Intel to GitlabCI

Add algorithm is_sorted to match std::is_sorted

There is a checkSorted method, but it writes an error message specific to an application. We should have a method that just returns a Boolean so it is more usable.

ArrayUtils min and max - initialValue

For initial values when doing max or min, we sometimes use user-provided initial values, sometimes explicitly assume a double type (like -DBL_MAX), and sometimes use std::numeric_limits host only, standard library not available on the device). We should make this all consistent. Perhaps we could make something like std::numeric_limits that works on the GPU?

Add max_element and deprecate ArrayMax

Benchmarks test segfaults on blueos

Building CARE with commit 4fded0b on rzansel with the blueos config file provided in the repo resulted in a segfault on the Benchmarks test for both a debug and release build.

If it helps here's the exact cmake invocation I ran within the build directory as well:

cmake ../ -C ../configs/lc/blueos/nvcc10.1.243_clang8.0.1.cmake -DCMAKE_INSTALL_PREFIX=../install_dir/ -DCMAKE_BUILD_TYPE=Debug

I've also provided the gdb backtrace full for this problem, and it appears to at least be crashing on the TestFuser.OneMillionSmallKernels problem.

#0  0x0000000010013c8c in LoopFuser::registerAction<__nv_dl_wrapper_t<__nv_dl_tag<void (*)(), &(cuda_test_X_OneMillionSmallFusedKernels()), 2u>, int, int, int, care::host_device_ptr<int>, int, care::host_device_ptr<int> >, __nv_dl_wrapper_t<__nv_dl_tag<void (*)(), &(cuda_test_X_OneMillionSmallFusedKernels()), 1u>> >(int, int, int&, __nv_dl_wrapper_t<__nv_dl_tag<void (*)(), &(cuda_test_X_OneMillionSmallFusedKernels()), 1u>>&&, __nv_dl_wrapper_t<__nv_dl_tag<void (*)(), &(cuda_test_X_OneMillionSmallFusedKernels()), 2u>, int, int, int, care::host_device_ptr<int>, int, care::host_device_ptr<int> >&&, int, int&, care::host_device_ptr<int>) (this=0x132df850, 
    start=0, end=32, start_pos=@0x7fffffffc0bc: 0, 
    conditional=<unknown type in CARE/build/bin/Benchmarks, CU 0x0, DIE 0x3b682>, 
    action=<unknown type in CARE/build/bin/Benchmarks, CU 0x0, DIE 0x3b692>, scan_type=0, pos_store=@0x10431358: 0, counts_to_offsets_scanvar=...)
    at CARE/src/care/LoopFuser.h:955
        lambda_size = 136
        conditional_size = 8
#1  0x000000001001024c in cuda_test_X_OneMillionSmallFusedKernels ()
    at CARE/test/Benchmarks.cpp:101
        __fuser__ = 0x132df850
        __fusible_offset__ = 327648
        __fusible_end_index__ = 32
        __fusible_scan_pos__ = 0
        __fusible_start_index__ = 0
        i = 10239
        __phase_observer = 0x132eae10
        numLoops = 1000000
        loopLength = 32
        a = {<chai::ManagedArray<int>> = {<chai::CHAICopyable> = {<No data fields>}, 
            m_active_pointer = 0x130f44c0, m_active_base_pointer = 0x130f44c0, 
            m_resource_manager = 0x10432778 <chai::ArrayManager::getInstance()::s_resource_manager_instance>, 
            m_elems = 32, m_offset = 0, m_pointer_record = 0x130f4570, m_is_slice = false}, <No data fields>}
        b = {<chai::ManagedArray<int>> = {<chai::CHAICopyable> = {<No data fields>}, 
            m_active_pointer = 0x130f4620, m_active_base_pointer = 0x130f4620, 
            m_resource_manager = 0x10432778 <chai::ArrayManager::getInstance()::s_resource_manager_instance>, 
            m_elems = 32, m_offset = 0, m_pointer_record = 0x130b4670, m_is_slice = false}, <No data fields>}
#2  0x000000001000fe8c in TestFuser_OneMillionSmallFusedKernels_Test::TestBody (this=0x10443060)
    at CARE/test/Benchmarks.cpp:92
No locals.
#3  0x0000000010095174 in testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void> (
    object=0x10443060, method=&virtual testing::Test::TestBody(), location=0x101f96e8 "the test body")
    at CARE/blt/thirdparty_builtin/googletest-master-2020-01-07/googletest/src/gtest.cc:2555
No locals.
#4  0x0000000010071330 in testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void> (
    object=0x10443060, method=&virtual testing::Test::TestBody(), location=0x101f96e8 "the test body")
    at CARE/blt/thirdparty_builtin/googletest-master-2020-01-07/googletest/src/gtest.cc:2591
        e = @0x7fffffffc380: warning: can't find linker symbol for virtual table for `std::exception' value
warning:   found `testing::TestResult::Skipped() const' instead
{_vptr.exception = 0x1004f2bc <testing::TestResult::Skipped() const+92>}
#5  0x000000001005054c in testing::Test::Run (this=0x10443060)
    at CARE/blt/thirdparty_builtin/googletest-master-2020-01-07/googletest/src/gtest.cc:2630
        impl = 0x10440230
#6  0x0000000010051510 in testing::TestInfo::Run (this=0x10440970)
    at CARE/blt/thirdparty_builtin/googletest-master-2020-01-07/googletest/src/gtest.cc:2807
        impl = 0x10440230
        repeater = 0x10440460
        start = 1604098798174
        test = 0x10443060
#7  0x0000000010051ecc in testing::TestSuite::Run (this=0x10440620)
    at CARE/blt/thirdparty_builtin/googletest-master-2020-01-07/googletest/src/gtest.cc:2939
        i = 2
        impl = 0x10440230
        repeater = 0x10440460
#8  0x00000000100606d0 in testing::internal::UnitTestImpl::RunAllTests (this=0x10440230)
    at CARE/blt/thirdparty_builtin/googletest-master-2020-01-07/googletest/src/gtest.cc:5480
test_index = 0
        start = 1604098778744
        i = 0
        gtest_is_initialized_before_run_all_tests = true
        in_subprocess_for_death_test = false
        should_shard = false
        has_tests_to_run = true
        failed = false
        repeater = 0x10440460
        repeat = 1
        gtest_repeat_forever = false
#9  0x000000001009b614 in testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> (object=0x10440230, 
    method=(bool (testing::internal::UnitTestImpl::*)(testing::internal::UnitTestImpl * const)) 0x10060190 <testing::internal::UnitTestImpl::RunAllTests()>, 
    location=0x101f9f6b "auxiliary test code (environments or event listeners)")
    at CARE/blt/thirdparty_builtin/googletest-master-2020-01-07/googletest/src/gtest.cc:2555
No locals.
#10 0x0000000010074ec0 in testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> (object=0x10440230, 
    method=(bool (testing::internal::UnitTestImpl::*)(testing::internal::UnitTestImpl * const)) 0x10060190 <testing::internal::UnitTestImpl::RunAllTests()>, 
    location=0x101f9f6b "auxiliary test code (environments or event listeners)")
    at CARE/blt/thirdparty_builtin/googletest-master-2020-01-07/googletest/src/gtest.cc:2591
        e = @0x7fffffffc8e0: {_vptr.exception = 0x7fffffffc9c0}
#11 0x0000000010060114 in testing::UnitTest::Run (
    this=0x104311e0 <testing::UnitTest::GetInstance()::instance>)
    at CARE/blt/thirdparty_builtin/googletest-master-2020-01-07/googletest/src/gtest.cc:5067
        in_death_test_child_process = false
        premature_exit_file = {premature_exit_filepath_ = {static npos = <optimized out>, 
            _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x2000002e7c80 <std::string::_Rep::_S_empty_rep_storage+24> ""}}}
#12 0x0000000010042628 in RUN_ALL_TESTS ()
    at CARE/blt/thirdparty_builtin/googletest-master-2020-01-07/googletest/include/gtest/gtest.h:2470
No locals.
#13 0x00000000100425cc in main (argc=1, argv=0x7fffffffceb8)
    at CARE/blt/thirdparty_builtin/googletest-master-2020-01-07/googletest/src/gtest_main.cc:52

Add Build and Test for GCC to Gitlab CI

Add mask to care::IntersectArrays

Add an input argument for care::IntersectArrays that allows the algorithm to skip array elements in the intersection

Use RAJADeviceExec consistently

RAJADeviceExec is equal to RAJA::cuda_exec or RAJA::hip_exec or RAJA::omp_parallel_for_exec depending on how we compile. Using RAJADeviceExec everywhere would clean things up and make things more robust for different architectures

Add support for CARE building RAJA with rocPRIM.

Fix block size for Hip

RAJA::hip_exec<CARE_CUDA_BLOCK_SIZE, CARE_CUDA_ASYNC>{}

We should either have CARE_GPU_BLOCK_SIZE and CARE_GPU_ASYNC, or have Hip specific macros.

Add an overload of sortArray that takes a comparison functor

Avoid using CARE_CUDA_BLOCK_SIZE for hip contexts

Either redefine CARE_CUDA_BLOCK_SIZE to be CARE_GPU_BLOCK_SIZE, or make separate hip and cuda variables

Full code examples?

Are there any examples (other than BenchmarkNumeric, which is pretty bare) of using CARE?
It's possible that these are in the docs, but currently users have to download and build it themselves.

Execute blueos tests in Gitlab CI

Add macro guards for function overloads only needed when implicit conversions are enabled

See checkSorted in array_utils.h for an example.

Build errors on develop branch with OpenMP Enabled

I'm currently testing CARE out on an linux machine with GCC v7.5.0 compiler suite and a toss3 machine with GCC v7.3.0.

I'm finding that the build is failing on the TestSortFuser example with a number of errors that appear related to the RAJA execution policy for example here's one of the hard errors:

CARE/src/care/SortFuser.h:265:25: error: no matching function for call to ‘sortArray(RAJA::PolicyBaseT<(RAJA::Policy)4, (RAJA::Pattern)1, (RAJA::Launch)0, (camp::resources::v1::Platform)1, RAJA::policy::omp::Parallel, RAJA::wrapper<RAJA::policy::omp::omp_for_schedule_exec<RAJA::policy::omp::Auto> > >, care::host_device_ptr<int>&, int&)’
          care::sortArray(RAJAExec{}, m_concatenated_result, m_total_length);

It should be pretty easy to reproduce on an LC machine with the following cmake invocation on commit 654b9fa:

cmake ../ -DENABLE_OPENMP=ON -DCMAKE_INSTALL_PREFIX=../install_dir/

Replace checkSorted with an implementation of is_sorted

Rename checkSorted to is_sorted to match the std library
Return true or false rather than printing an error message
Remove the function name argument

Scan tests failing on blueos

The fix is adding the following jsrun argument: --smpiargs="-disable_gpu_hooks"

jsrun -n 1 -a 1 -g 1 ./ScanLoopExample
CPU count: 21
CUDA Hook Library: Failed to find symbol mem_find_dreg_entries, ./ScanLoopExample: undefined symbol: __PAMI_Invalidate_region

jsrun -n 1 -a 1 -g 1 ./TestScan
Running main() from /g/g17/dayton8/ale3d/github_care/blt/thirdparty_builtin/googletest-master-2020-01-07/googletest/src/gtest_main.cc
[==========] Running 5 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 5 tests from Scan
[ RUN ] Scan.test_scan_offset
[ OK ] Scan.test_scan_offset (172 ms)
[ RUN ] Scan.test_scan_zero_length
[ OK ] Scan.test_scan_zero_length (0 ms)
[ RUN ] Scan.test_scan_offset_index
[ OK ] Scan.test_scan_offset_index (0 ms)
[ RUN ] Scan.test_scan_offset_index_half
[ OK ] Scan.test_scan_offset_index_half (0 ms)
[ RUN ] Scan.test_scan_everywhere
[ OK ] Scan.test_scan_everywhere (0 ms)
[----------] 5 tests from Scan (172 ms total)

[----------] Global test environment tear-down
[==========] 5 tests from 1 test suite ran. (172 ms total)
[ PASSED ] 5 tests.
CUDA Hook Library: Failed to find symbol mem_find_dreg_entries, ./TestScan: undefined symbol: __PAMI_Invalidate_region