goddard-fortran-ecosystem / gftl Goto Github PK
View Code? Open in Web Editor NEWSoftware containers for Fortran
License: Apache License 2.0
Software containers for Fortran
License: Apache License 2.0
There is a .travis.yml
on develop, but no checks are triggered.
Hmm...
This is mostly for debugging purposes - would be useful to examine the state of the container. Should be two cases. One that just prints metadata and the other that prints contained values as well. (If the container is of derived type, another switch will be needed to indicate whether he type includes PDT methods.
The library looks very interesting (and sorely needed in the Fortran world!)
Is V2 ready to play with? The examples all seem to be for V1.
Is there a need to install to consume? As a template library, I was expecting to be able to just include the appropriate files from the include directory into my source.
The challenges I have are cloud build machines (not directly under my control), distributed team (would prefer not to require installs on everyone's machines) and builds for both Linux and Windows. We are using Intel OneAPI Fortan 2022.1.0 on both Windows and Linux. We are not using cmake as our build orchestrator.
Thanks!
I suggest adding the topics generic-programming
, fortran
, modern-fortran
in the About section.
Just need to add a simple gftl.mk file that can be included by make. The file will specify just a few paths and such.
BSD m4
behaves differently from GNU m4
. If m4
is hardcoded, the results of build are wrong (e.g. __type_MOVE
is generated instead of __TYPE_MOVE
). The proposed fix is in #102
There are lots of these warnings, so much that it's unfortunately significantly slowing down single-threaded compilation. Here's an extract:
[200/1384] Building Fortran preprocessed _deps/pfunit-build/extern/fArgParse/extern/gFTL-shared/src/v2/CMakeFiles/gftl-shared-v2.dir/map/IntegerReal128Map.F90-pp.f90
/home/peter/Codes/neasy-f/build/_deps/pfunit-src/extern/fArgParse/extern/gFTL-shared/extern/gFTL/include/v2/set/procedures.inc:1033:0:
1033 | #define __algorithm_iterator __set_iterator
|
Warning: "__algorithm_iterator" redefined
/home/peter/Codes/neasy-f/build/_deps/pfunit-src/extern/fArgParse/extern/gFTL-shared/extern/gFTL/include/v2/map/specification.inc:144:0:
144 | #define __algorithm_iterator __map_iterator
|
note: this is the location of the previous definition
[222/1384] Building Fortran preprocessed _deps/pfunit-build/extern/fArgParse/extern/gFTL-shared/src/v2/CMakeFiles/gftl-shared-v2.dir/ordered_map/IntegerDoubleOrderedMap.F90-pp.f90
/home/peter/Codes/neasy-f/build/_deps/pfunit-src/extern/fArgParse/extern/gFTL-shared/extern/gFTL/include/v2/vector/specification.inc:212:0:
212 | #define __algorithm_iterator __vector_iterator
|
Warning: "__algorithm_iterator" redefined
/home/peter/Codes/neasy-f/build/_deps/pfunit-src/extern/fArgParse/extern/gFTL-shared/extern/gFTL/include/v2/map/specification.inc:144:0:
144 | #define __algorithm_iterator __map_iterator
|
note: this is the location of the previous definition
/home/peter/Codes/neasy-f/build/_deps/pfunit-src/extern/fArgParse/extern/gFTL-shared/extern/gFTL/include/v2/vector/procedures.inc:1013:0:
1013 | #define __algorithm_iterator __vector_iterator
|
Warning: "__algorithm_iterator" redefined
/home/peter/Codes/neasy-f/build/_deps/pfunit-src/extern/fArgParse/extern/gFTL-shared/extern/gFTL/include/v2/map/procedures.inc:428:0:
428 | #define __algorithm_iterator __map_iterator
|
note: this is the location of the previous definition
It looks like there's only three unique sources of the warnings:
I think the above snippet demonstrates all of them.
This is GFTL commit dc85913, ultimately from pfunit 47b3b0ab1.
see discussion here
set_property(
TARGET myLibrary
APPEND
PROPERTY STATIC_LIBRARY_FLAGS "-no_warning_for_no_symbols"
)
PURE would allow more contexts where containers could be used.
The attached reproducer from @bena-nasa shows that the recent workaround for a gfortran memory leak unfortunately introduced a bug in the vector set() method. Previously the scenario in this reproducer relied on intrinsic assignment which was implicitly deallocating in this scenario. The memory workaround used explicit allocation and it was not obvious that a check needed to be done for deallocation.
I think the fix will be easy, but a bit ugly.
use pfio
implicit NONE
type(Variable) :: v, v1, v2
type(StringVariableMap) :: vars
v = variable(pfio_real32,dimensions='time')
call v%add_attribute('long_name','time')
v1 = variable(pfio_real32,dimensions='time')
call v%add_attribute('long_name','time1')
v2 = variable(pfio_real32,dimensions='lon')
call v%add_attribute('long_name','longitude')
call vars%insert('time',v1)
call vars%insert('lon',v2)
call vars%set('time',v)
End Program test_modify
(Note the reproducer is using code from GEOS-ESM/MAPL.
One of our builds using pFUnit is failing at runtime with the following back trace:
At line 19 of file /builds/gyrokinetics/utils/build/_deps/pfunit-src/extern/fArgParse/src/StringActionMap.F90
Fortran runtime error: Recursive call to nonrecursive procedure '__copy_fp_stringactionmap_Map_s_node'
Error termination. Backtrace:
#0 0x7f0fa5f95d51 in ???
#1 0x7f0fa5f968a9 in ???
#2 0x7f0fa5f96d8d in ???
#3 0x51747d in __fp_stringactionmap_MOD___copy_fp_stringactionmap_Map_s_node
at /builds/gyrokinetics/utils/build/_deps/pfunit-src/extern/fArgParse/src/StringActionMap.F90:19
#4 0x517553 in __fp_stringactionmap_MOD___copy_fp_stringactionmap_Map_s_node
at /builds/gyrokinetics/utils/build/_deps/pfunit-src/extern/fArgParse/src/StringActionMap.F90:19
#5 0x49bf22 in set_command_line_options
at /builds/gyrokinetics/utils/build/_deps/pfunit-src/src/funit/FUnit.F90:125
#6 0x49eda2 in __funit_MOD_generic_run
at /builds/gyrokinetics/utils/build/_deps/pfunit-src/src/funit/FUnit.F90:60
#7 0x406914 in __pfunit_MOD_run
at /builds/gyrokinetics/utils/build/_deps/pfunit-src/src/pfunit/pFUnit.F90:108
#8 0x4065df in funit_main_
at /builds/gyrokinetics/utils/build/_deps/pfunit-src/src/pfunit/pfunit_main.F90:16
#9 0x40618a in MAIN__
at /builds/gyrokinetics/utils/build/_deps/pfunit-src/include/driver.F90:81
#10 0x4061e0 in main
at /builds/gyrokinetics/utils/build/_deps/pfunit-src/include/driver.F90:52
It says it's from fArgParse, but I guess it's really from gFTL, though I couldn't work out where exactly.
This build is using gfortran 9.3.1, and the following compiler flags:
-ffpe-trap=invalid,zero -fcheck=all -finit-real=snan -finit-integer=-12345 -finit-logical=true -finit-character=95 -finit-derived -g
-fcheck=all
does include recursion
, so as a workaround we could probably turn that off.
The GFTL repository itself can be copied to an airgapped system with
.. on connected system
git clone https://github.com/Goddard-Fortran-Ecosystem/gFTL-shared
then copy the gFTL-shared directory to the airgapped system and attempt further build.
Upon doing so ON THE AIRGAPPED SYSTEM though the build generates
cd gFTL-shared/
How do we do this build on an airgapped system (NOAA jet tjet node in this example)?
A larger use case in which 10^6 integers are inserted in a set and then deleted one at a time fails at run time. First failure is not finding element 11.
Strongly suspect this is a corner case not properly covered by recent changes to use ALLOCATABLE rather than POINTER for the child nodes.
Reproducer:
program main
use, intrinsic :: iso_fortran_env, only: INT64, REAL64
use gFTL2_Integer32Set
implicit none
type(Integer32Set) :: s
type(Integer32SetIterator) :: iter
integer :: i, j, n
real(kind=REAL64) :: x
integer(kind=INT64) :: c0, c1, cr
n = 10**5
s = Integer32Set()
call system_clock(c0, cr)
do i = 1, N
!!$ call random_number(x)
!!$ j = 1 + floor(N*x)/N + i
!!$ call s%insert(j)
call s%insert(i)
end do
call system_clock(c1)
print*,'construction time: ', real(c1-c0)/cr, s%size()
call system_clock(c0)
!!$ call s%clear()
do i = 11, N
!!$ iter = s%find(i)
!!$ print*, i, iter == s%end()
j = s%erase(i)
if (j /= 1) then
print*, i, j
associate (b => s%begin(), e => s%end())
iter = b
do while (iter /= e)
print*,'iter: ', iter%of()
call iter%next()
end do
end associate
end if
end do
call system_clock(c1)
print*,'deconstruction time: ', real(c1-c0)/cr
end program main
This might break some existing applications, but if the key does not exist, this interface should "throw" an exception.
The existing iterator approach in gFTL mimics C++ behavior, but with a significant wrinkle. Namely, in C++ the loop itself can do the increment, but Fortran requires an explicit call to the next()
method at the end of the loop. For simple loops this is not a problem, but users can easily go astray if they attempt to use CYCLE
which will not on its own trigger the increment to the iterator.
So rather than
associate (e => container%end())
iter = container%begin()
do while (iter /= e)
if (cond) cycle ! infinite loop
call iter%next()
end do
end associate
I would like to have:
associate (e => container%f_end())
iter = container%f_begin()
do while (iter /= e)
call iter%next() ! Top of loop - always triggered
if (cond) cycle ! works just fine
end do
end associate
Corner cases:
next(f_begin())
should.I copied cmake/Intel.cmake to cmake/IntelLLVM.cmake in order to get gftl-shared-v1.5.0 to compile with the LLVM based compilers from Intel.
Note that the LLVM based compilers are very different than the Classic compilers in their support for compiler options. See https://www.intel.com/content/www/us/en/developer/articles/guide/porting-guide-for-ifort-to-ifx.html for further details.
This is a simple one. I found out today as I was trying to build gFTL with PGI 18.5 that it doesn't support PGI. I see two issues:
cmake_utils/PGI.cmake
fileThe use case is to have a container that is part of a family of subclasses. The type definition would start:
TYPE, EXTENDS(AbstractX) :: XVector
The current need is for both Vector and Map, but should treat Set and altSet to be consistent.
I have a use case where I need pointers into a container to remain valid as the container grows. For polymorphic cases, the wrapper type with ALLOCATABLE
enables this as gFTL uses MOVE_ALLOC()
under the hood.
gFTL v1 also had the ability to use a wrapper type for non-polymorphic types, and it appears that v2 almost has the capability baked in. Just a minor tweak to an FPP conditional is required and then one can use #define T_deferred
.
There is a test (see [1]) for vector and deque that calls erase_range(iterator, iterator)
(empty range). This currently ends-up calling MOVE_ALLOC(X, X)
(at [2] and [3]) which is not legal Fortran (see [4]).
While most compilers do not emit runtime errors, they silently deallocate X, which in the case of erase_range ends-up destroying the vector/deque elements after the iterator. I expect the intention for erase_range(iterator, iterator)
is to be a no-op instead, the vector/deque code should probably do nothing in this case.
You can observe the issue if you add an ASSERT to test the element values (like ASSERT(v%at(2), two)
) after the erase_range
call in the test_erase_empty_range
test.
gfortran builds will fail with:
[Test_deferred_stringVector.pf:851]
String assertion failed:
expected: <"">
but found: <"two">
first diff:
[1] :
gFTL/tests/vector/Test_Vector.m4
Line 409 in dc93a5f
gFTL/include/v2/deque/procedures.inc
Line 743 in dc93a5f
gFTL/include/v2/vector/procedures.inc
Line 777 in dc93a5f
I found this issue because flang generate runtime errors for such MOVE_ALLOC.
currently if the key is not found, a null pointer is returned, but the underlying set has grown.
Polymorphism gets interesting in this case.
Testing with the FORD documenter found there were some v1 macros that were being redefined. As this might have bad effects (see Fortran-FOSS-Programmers/ford#421 (comment)), we aim to clean those up. PR incoming.
Sorry for the intrusion, but my employer (NASA) is requesting that all external contributors to my open source projects sign a contributors license agreement. There are two flavors - individual or corporate, depending on whether your contributions are a hobby or part of your day job.
Please download the appropriate CLA at https://github.com/Goddard-Fortran-Ecosystem/cla/tree/main/gFTL and follow the instructions there.
Thank you in advance.
Consider a case when a user wants to install headers into ${prefix}/include/GFTL
(as opposed to dumping them into a common dir and promoting a chaos there). And then, inside GFTL
there are V1 and V2 (without an unnecessary another include
in between). There seems to be no trivial way to do that, and even patching CMake files requires multiple iterations, because settings in one CMakeLists file are ignored due to the same being redefined elsewhere.
If one does not notice what the build system does, everything gets coerced into ${prefix}/GFTL-${version}
.
It looks like Travis is removing oclint
from osx images:
https://changelog.travis-ci.com/oclint-is-removed-from-mac-builds-79270
As we currently ask to uninstall it, this triggers faults in Travis tests. Remove it.
When T1_EQ is not defined by T2_EQ is defined, the template generates compile errors because local variables lt
and gt
are used but not declared.
If multiple (v2) containers are used in a single program unit, some of the procedure names conflict. E.g., before begin()
was only a type-bound procedure on containers that support iterators. But it is now also a PUBLIC function that takes an iterator as an argument.
I took the easy path and did not use a generic interface block to expose the method which results in unnecessary name conflicts. The correction is to use interface blocks in each case. Tedious, but straightforward.
The attached reproducer demonstrates a memory leak under Intel 18 compilers. (18.0.3 and 18.0.5 were tested.) This appears to be fixed under Intel 19, but a workaround is desired by some users.
If intrinsic assignment of PAIR objects is replaced with component-wise assignment, the memory leak goes away. (And hints at the necessary workaround.)
module foo_mod
implicit none
private
public :: pair
public :: container
type :: pair
private
character(:), allocatable :: key
integer :: value
end type pair
type :: container
type(pair), allocatable :: elements(:)
contains
procedure :: run
end type container
interface pair
module procedure new_pair
end interface pair
contains
function new_pair(key, value) result(p)
type (Pair) :: p
character(*), intent(in) :: key
integer, intent(in) :: value
p%key = key
p%value = value
end function new_pair
subroutine run(this, obj, n)
class(container), intent(inout) :: this
type(Pair), intent(in) :: obj
integer, intent(in) :: n
integer :: j
logical :: flag
flag = .true.
allocate(this%elements(n))
do j = 1, n
if (flag) then
this%elements(j) = obj
!!$ this%elements(j)%key = obj%key
!!$ this%elements(j)%value = obj%value
end if
end do
end subroutine run
end module foo_mod
program main
use foo_mod
type(Pair) :: obj
type(container) :: c
obj = pair('dog', 3)
!!$ obj%key = 'dog'
!!$ obj%value = 3
do i = 1, 1000
call c%run(obj, 100)
deallocate(c%elements)
end do
end program main
gs6101-midas:tmp tclune$
rc should be set to 0 for in bounds references if present.
Just discovered that _value_allocatable is not used in the map template. This is technically not a bug, as without it the Fortran assignment "=" operator is used instead which is legal.
In the early days move_alloc() was more robust. But even now it is preferable both for efficiency and to allow pointer references to persist when the container is modified. (Without documentation, I can hedge on whether that is guaranteed. But it would be nice ...)
In trying to build MAPL with GCC9 recently, I saw quite a few of these:
[ 54%] Building Fortran object MAPL_Base/CMakeFiles/MAPL_Base.dir/MAPL_ESMFTimeVectorMod.F90.o
/discover/swdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.4-SLES11/x86_64-unknown-linux-gnu/gfortran_9.1.0-openmpi_4.0.1/Linux/GFTL-1.2/include/templates/unused.inc:10:0:
10 | #define _UNUSED_DUMMY(dummy) if (.false.) print*,shape(dummy)
|
Warning: "_UNUSED_DUMMY" redefined
/discover/swdev/mathomp4/Models/MAPL-MAPL20-Cleanup-GCC9/MAPL/MAPL_Base/unused_dummy.H:13:0:
13 | #define _UNUSED_DUMMY(x) if (.false.) print*,shape(x)
|
note: this is the location of the previous definition
I believe this is due to unused.inc
being just a #define
without the usual quartet:
#ifdef FOO
#undef FOO
#endif
#define FOO
Thus in some files in MAPL that used templates/vector.inc
, say, they pick up this new (nigh-identical) macro.
Note: something perhaps should/could be added to all_macros_undef.inc
as well? I see it exists but gFTL is more complex than I'm used to.
In include/types
, gFTL installs the files Foo.inc
, FooPoly.inc
, FooPolyPtr.inc
, and FooPtr.inc
.
These require modules which are not available:
use Foo_mod, only: Foo
This looks like example files; shouldn't they be installed as examples?
The issue is not detectable with most compilers but with aggressive debugging flags NAG will give weird runtime issues if the TO argument of move_alloc() does not have the TARGET attribute. (Standard then allows pointers associated with FROM to become undefined.)
Probably would have been easier to diagnose, but my application causes a NAG internal panic if I use the -C=dangling flag that ought to have detected the issue.
This is a compiler defect, but the workaround is trivial. Replaces ASSOCIATE block with wordier version of one statement.
On Macports buildbots we got a failure: https://trac.macports.org/ticket/68505
CMake Error at CMakeLists.txt:38 (find_package):
By not providing "FindgFTL.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "gFTL", but
CMake did not find one.
Could not find a package configuration file provided by "gFTL" with any of
the following names:
gFTLConfig.cmake
gftl-config.cmake
Add the installation prefix of "gFTL" to CMAKE_PREFIX_PATH or set
"gFTL_DIR" to a directory containing one of the above files. If "gFTL"
provides a separate development package or SDK, be sure it has been
installed.
-- Configuring incomplete, errors occurred!
Presumably, it is due to case-sensitivity, since both locally and on GH IC (which are set to be non-case-sensitive) the build is all good.
Recent development work is breaking with Intel 2021.10.0 in some obscure map scenario. Troubleshooting shows that it is related to the generic ASSIGNMENT(=)
rather than relying on the compiler to do a deep copy.
A fix would be to add a CPP switch that disables the redefinition of assignment. For backward compatibility, "enabled" must remain the default. (Even though it is itself a workaround for possibly obsolete use cases/compiler versions.)
2 different (but hopefully related) issues have shown up when migrating GEOS to ifort 17. I suspect these are Intel compiler issues, possibly ones for which bug reports have already been submitted.
In the near term, a workaround is desired to allow GEOS to work.
1/18 Testing: test_derived_logical
1/18 Test: test_derived_logical
Command: "/usr/bin/diff" "/opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_gFTL/gFTL/work/gFTL-1.8.1/include/v2/parameters/tests/expected_logical.txt" "actual_logical.txt"
Directory: /opt/local/var/macports/build/_opt_PPCSnowLeopardPorts_devel_gFTL/gFTL/work/gFTL-1.8.1/include/v2/parameters/tests
"test_derived_logical" start time: Jan 22 11:05 CST
Output:
----------------------------------------------------------
/usr/bin/diff: actual_logical.txt: No such file or directory
<end of output>
Test time = 0.01 sec
----------------------------------------------------------
Test Failed.
"test_derived_logical" end time: Jan 22 11:05 CST
"test_derived_logical" time elapsed: 00:00:00
There is still an issue here
gFTL/include/v2/vector/procedures.inc
Line 139 in 6e2adb7
As seen here:
Line 12 in 9b8ea8f
the version number in gFTL is wrong for the release version (should be 1.1.1)
Thanks for the project.
Hi,
for the Debian package, I would need to put the files to places similar to the existing ones, which would be like this:
/usr/include/gftl/
(with subdirs templates
and types
)/usr/share/cmake/gftl/
/usr/share/gftl/gftl.mk
I tried to change the file paths via a patch, but when then building gftl-shared, the gftl include path is not set, and therefore the gftl include files are not found. That has been working with 1.2.7, but now stopped.
But maybe there is a simple way to put the install paths as parameters during the cmake run? Unfortunately, I am not good in CMAKE; could you give me some advise?
Thank you
Best regards, Ole
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.