Code Monkey home page Code Monkey logo

gfort2py's Introduction

Continuous Integration Coverage Status PyPI version DOI Python versions gfortran versions PyPI - Downloads

gfort2py

Library to allow calling Fortran code from Python. Requires gfortran>=8.0, Works with python >= 3.7

Build

Installing locally:

python -m pip install .

or install via pypi

python -m pip install --upgrade --user gfort2py

For a full list of supportetd platforms see the support documentation.

Why use this over other Fortran to Python translators?

gfort2py has three main aims:

  1. Make it trivially easy to call Fortran code from Python
  2. Minimise the number of changes needed in the Fortran code to make this work.
  3. Support as many Fortran features as possible.

We achieve this by tightly coupling the code to the gfortran compiler, by doing so we can easily embed assumptions about how advanced Fortran features work which makes development easier and minimises the number of changes needed on the Fortran side.

gfort2py use the gfortran mod files to translate your Fortran code's ABI to Python-compatible types using Python's ctype library. By using the mod file we can determine the call signature of all procedures, components of derived types, and the size and shapes of all module-level variables. As long as your code is inside a Fortran module, no other changes are needed to your Fortran code.

The downside to this approach is that we are tightly tied to gfortran's ABI, which means we can not support other non-gfortran compilers and we do not support all versions of gfortran. When gfortran next breaks its ABI (which happens rarely, the last break was gfortran 8) we will re-evaluate our supported gfortran versions.

Using

There are two ways to load Fortran code into Python. Either fFort or compile. The recommended way is via fFort for interfacing with existing code, while compile is more suitable for wrapping short snippets of Fortran code

fFort

Your Fortran code must be inside a module and then compiled as a shared library.

On linux:

gfortran -fPIC -shared -c file.f90
gfortran -fPIC -shared -o libfile.so file.f90

On MacOS:

gfortran -dynamiclib -c file.f90
gfortran -dynamiclib -o libfile.dylib file.f90

On Windows:

gfortran -shared -c file.f90
gfortran -shared -o libfile.dll file.f90

If the shared library needs other shared libraries you may need to set the LD_LIBRARY_PATH environment variable, and it is also recommended to run chrpath on the shared libraries so you can access them from anywhere.

Python side

import gfort2py as gf

SHARED_LIB_NAME=f'./test_mod.{gf.lib_ext()}' # Handle whether on Linux, Mac, or Windows
MOD_FILE_NAME='tester.mod'

x=gf.fFort(SHARED_LIB_NAME,MOD_FILE_NAME) 

NOTE: The mod data is cached to speed up re-reading the data. To control this pass cache_folder to fFort. A value of False disables caching, a string sets the folder location, while leaving the argument as None defaults to platformdirs user_cache_dir

compile

import gfort2py as gf


fstr = """
            integer function myfunc(x,y)
                integer :: x,y
                myfunc = x+y
            end function myfunc
"""

x  = gf.compile(string=fstr)

The Fortran code can also be in a file in which case:

import gfort2py as gf

x  = gf.compile(file='my_fortran_file.f90')

In either casee the code will be compilied into a Fortran module and then into a shared library. Any Fortran code is valid as long as it can be inserted into a Fortran Module (Its optional whether you need to wrap things in module/end module, if you do not then that is done automatically for you).

Additional options available for compile:

  • FC: str Path to gfortran compilier
  • FFLAGS: str Additional Fortran compile options. This defaults to -O2.
  • LDLIBS: str Any additional libraries needed to be linked in (-l)
  • LDFLAGS: str Locations of addtional libraries (-L)
  • ouput: str Location to save intermediate files to. Defaults to None which saves files in a temporary location. Otherwise save to the location specified.

NOTE: The interface to compile is currently considered unstable and may change.

Interface

x now contains all variables, parameters and procedures from the module (tab completable), and is independant on how the Fortran code was loaded.

Functions

y = x.func_name(a,b,c)

Will call the Fortran function with variables a,b,c and returns the result in y.

y will be named tuple which contains (result, args). Where result is a python object for the return value (0 if a subroutine) and where args is a dict containing all arguments passed to the procedure (both those with intent (in) which will be unchanged and intent(inout/out) which may have changed).

Variables

x.some_var = 1

Sets a module variable to 1, will attempt to coerce it to the Fortran type

x.some_var

Will return a Python object

Optional arguments that are not present should be passed as a Python None.

Arrays

Arrays should be passed as a NumPy array of the correct size and shape.

Remember that Fortran by default has 1-based array numbering while Numpy is 0-based.

If a procedure expects an unallocated array, then pass None as the argument, otherwise pass an array of the correct shape.

Derived types

Derived types can be set with a dict

x.my_dt={'x':1,'y':'abc'}
y=x.my_dt
y['x']
y['y']

If the derived type contains another derived type then you can set a dict in a dict

x.my_dt={'x':1,'y':{'a':1}}

When setting the components of a derived type you do not need to specify all of them at the same time.

If you have an array of derived types

type(my_type), dimension(5) :: my_dt
type(my_type), dimension(5,5) :: my_dt2

Elements can be accessed via an index:

x.my_dt[0]['x']
x.my_dt2[0,0]['x']

You can only access one component at a time (i.e no striding [:]). Allocatable derived types are not yet supported.

Derived types that are dummy arguments to a procedure are returned as a fDT type. This is a dict-like object where the components can only be accessed via the item interface ['x'] and not as attributes .x. This was done so that we do not have a name collision between Python functions (keys, items etc) and any Fortran-derived type components.

You can pass a fDT as an argument to a procedure.

Quad precision variables

Quad precision (REAL128) variables are not natively supported by Python thus we need a different way to handle them. For now that is the pyQuadp library which can be installed from PyPi with:

python -m pip install pyquadp

or from a git checkout:

python -m pip install .[qaud]

For more details see pyQuadp's documentation, but briefly you can create a quad precision variable from an int, float, or string. On return you will receive a qfloat type. This qfloat type acts like a Python Number, so you can do things like add, multiply, subtract etc this Number with other Numbers (including non-qfloat types).

We currently only support scalar Quad's and scalar complex Quad's. Arrays of quad precision values is planned but not yet supported. Quad values can also not be returned as a function result (this is a limitation in ctypes which we have no control over). Thus a quad precision value can only occur in:

  • Module variables
  • Parameters
  • Procedure arguments

pyQuadp is currently an optional requirement, you must manually install it, it does not get auto-installed when gfort2py is installed. If you try to access a quad precision variable without pyQuadp you should get a TypeError.

Callback arguments

To pass a Fortran function as a callback argument to another function then pass the function directly:

y = x.callback_function(1)

y = x.another_function(x.callback_function)

Currently only Fortran functions can be passed. No checking is done to ensure that the callback function has the correct signature to be a callback to the second function.

The callback and also be created in Python at runtime (but must be valid Fortran):

fstr = """
        integer function callback(x)
            integer :: x
            write(*,*) x
            callback = 3*x
        end function callback

        """

f = gf.compile(fstr)


y = x.another_function(f.callback)

Testing

python -m pip install .[test]
pytest -v

To run unit tests

Things that work

Module variables

  • Scalars
  • Parameters
  • Characters
  • Explicit size arrays
  • Complex numbers (Scalar and parameters)
  • Getting a pointer
  • Getting the value of a pointer
  • Allocatable arrays
  • Derived types
  • Nested derived types
  • Explicit Arrays of derived types
  • Allocatable Arrays of derived types
  • Procedure pointers inside derived types
  • Derived types with dimension(:) array components (pointer, allocatable, target)
  • Allocatable strings (partial)
  • Explicit Arrays of strings
  • Allocatable arrays of strings
  • Classes
  • Abstract interfaces
  • Common blocks
  • Equivalences
  • Namelists
  • Quad precision variables
  • function overloading

Procedures

  • Basic calling (no arguments)
  • Argument passing (scalars)
  • Argument passing (strings)
  • Argument passing (explicit arrays)
  • Argument passing (assumed size arrays)
  • Argument passing (assumed shape arrays)
  • Argument passing (allocatable arrays)
  • Argument passing (derived types)
  • Argument intents (in, out, inout and none)
  • Passing characters of fixed size (len=10 or len=* etc)
  • Functions that return a character as their result
  • Allocatable strings (Only for things that do not get altered inside the procedure)
  • Explicit arrays of strings
  • Allocatable arrays of strings
  • Pointer arguments
  • Optional arguments
  • Value arguments
  • Keyword arguments
  • Generic/Elemental functions
  • Functions as an argument
  • Unary operations (arguments that involve an expression to evaluate like dimension(n+1) or dimension((2*n)+1))
  • Functions returning an explicit array as their result

Common block elements

There is no way currently to access components of a common block.

Accessing module file data

For those wanting to explore the module file format, there is a routine mod_info available from the top-level gfort2py module:

module = gf.mod_info('file.mod')

That will parse the mod file and convert it into an intermediate format inside module.

Variables or procedures can be looked up via the item interface (I also recommend using pprint for easier viewing):

from pprint import pprint

pprint(module['a_variable'])

Accessing the list of all available components can be had via module.keys().

You can also do:

module = gf.mod_info('file.mod',json=True)
module['a_variable']

Then when you access each component the return value will be JSON-formatted. Note you can currently only access each component as JSON not the whole module file as JSON at the moment.

Contributing

Bug reports are of course welcome and PR's should target the main branch.

For those wanting to get more involved, adding Fortran examples to the test suite of currently untested or unsupported features would be helpful. Bonus points if you also provide a Python test case (that can be marked @pytest.mark.skip if it does not work) that demonstrates the proposed interface to the new Fortran feature. Features with test cases will move higher in the order of things I add to the code.

See how to write a test case for details on how to write test cases.

For those wanting to go further and add the new feature themselves open a bug report and we can chat about what needs doing.

Debugging

Debugging instructions are here

gfort2py's People

Contributors

certik avatar dependabot[bot] avatar rjfarmer avatar vincint-skywalker avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

gfort2py's Issues

Segmentation fault (core dumped)

This function

function mesh_exp(r_min, r_max, a, N) result(mesh)
real(dp), intent(in) :: r_min, r_max
real(dp), intent(in) :: a
integer, intent(in) :: N
real(dp) :: mesh(N+1)

gives segfault when used:

In [1]: %load a.py                                                              

In [2]: # %load a.py 
   ...: from gfort2py import fFort 
   ...:  
   ...: x = fFort("./libhfsolver.so", "utils.mod") 
   ...: #x = fFort("./libhfsolver.so", "types.mod") 
   ...: #x = fFort("./libhfsolver.so", "random.mod") 
   ...:                                                                         

In [3]: x.mesh_exp?                                                             
Call signature: x.mesh_exp(*args)
Type:           fFunc
String form:    Function: mesh_exp
Length:         1
Docstring:     
Function: mesh_exp(r_min <float>,r_max <float>,a <float>,n <int>)
Args In: r_min <float>, r_max <float>, a <float>, n <int> 
Args Out:  
Returns: None

In [4]: x.mesh_exp(0, 10, 2, 10)                                                
Segmentation fault (core dumped)

Is such a signature supported? I would need to Debug more to see what is going on.

arrays of fixed dimension in derived type

I'm testing gfort2py using a derived type which includes an array of a fixed dimension, but am getting the error

'File "/home/chrispr/.local/lib/python3.7/site-packages/gfort2py/types.py", line 169, in py_to_ctype
setattr(self._value,name,value[name])
TypeError: expected LP_c_float instance, got numpy.ndarray'

when passing a dictionary including a numpy array from python to the fortran derived type.

I can succesfully pass the same numpy array if it's not in the derived type. If I set the array in the derived type inside the fortran function using this other array, the returned array in the derived type is indexed fortran-style instead of python style, while the array modified outside the derived type is correct:

{'arr_test': array([1., 2., 3.], dtype=float32), 'xlc_p': OrderedDict([('simtype', 2), ('l0', 0.0), ('cdims', array([6., 8., 0.])), ('field_int_on_currently', True)])}

Sample python and fortran is attached - any advice would be greatly appreciated. I am using python 3.7, numpy 1.17, and gcc 9.2.

#!/usr/bin/python

"""
4/13/2020

Test gfort2py integration and timing speed

"""
import numpy as np
import gfort2py as gf

SHARED_LIB_NAME='./test.so'
MOD_FILE_NAME='params.mod'

x = gf.fFort(SHARED_LIB_NAME, MOD_FILE_NAME)

arr_test = np.array([2.,3.,4.],dtype=np.float32)
print ('python test array')
print(arr_test.dtype)

#don't pass cdims
print('python, no array passing')
y1 = x.set_param_defaults(arr_test, {'simtype':2,'l0':0,'field_int_on_currently':True})

print(y1)
# pass cdims directly; error
print('python, passing array to derived type')
y2 = x.set_param_defaults(arr_test, {'simtype':2,'l0':0,'cdims':arr_test,'field_int_on_currently':True})

print(y2)

##############################

module params

    use, intrinsic :: iso_fortran_env
!    use, intrinsic :: IEEE_ARITHMETIC

    implicit none

    public

    type d_params

        integer simType
        real l0
        real, dimension(3) :: cdims
        logical field_int_on_currently

    end type

contains

        subroutine set_param_defaults(arr_test, xlc_p)
                implicit none

                integer :: I
                real, dimension(3), intent(inout) :: arr_test
                type(d_params), intent(inout) :: xlc_p

                print *, 'array outside derived type'
                print *, arr_test

                xlc_p%cdims = arr_test

                print *, 'setting array inside derived type'
                print *, xlc_p%cdims

                print *, 'modifying arrays'
                do I = 1,3
                        print *, arr_test(I)
                        xlc_p%cdims(I) = xlc_p%cdims(I)*2
                        arr_test(I) = arr_test(I) - 1
                enddo

                print *, 'modified derived type array'
                print *, xlc_p%cdims

        end subroutine set_param_defaults

end module params

Encountering Problems Testing gfort2py on MacOSX 10.14.4 (Mojave)

I've installed the latest version of gfort2py (version 1.1.3) on my iMacPro running Mac OSX version 10.14.4 (Mojave) and the Anaconda distribution of Python version 3.6.8. In reviewing the output from attempting to run the tests (see the attached file), several error messages are issued from the linker having to do with "... unsupported file formats".

It is not clear to me how problems with the linker are occurring and yet the tests proceed.

Any thoughts as to what is going on here, and what I need to do get the tests to work?

Terminal Saved Output.txt

Wrong output for real*8 parameter of negative value

I use just a simple .f file,

      module test
      implicit none
      real*8,parameter::pi=-4.d0*datan(1.d0)
      real*8,parameter::a=-1.0
      end module test

but when I print a and pi in python, I get only the absolute value for them,

3.141592653589793
1.0

Add gfortran module version 15

Doesn't work with newer GNU Fortran versions (tested here with 8.1.1) as they generate / use GFORTRAN module version '15', but parseMod/parseMod.py checks for exactly (==) version 14 and throws exception if not matched.

Thanks !
Mike

[BUG] Mixed case function names are lower cased

import gfort2py as gf

fstr =  """

integer function MixedCase()

    MixedCase = 1

end function MixedCase

"""


x = gf.compile(fstr)

res = x.MixedCase()

print(res.result)

AttributeError: /tmp/gfort2pyu99peb66/a2dd937f3e65c4ebf8a88852a44e2765b.mod has no attribute MixedCase

Derived type allocatable arrays leak memory and have suprising behavior [BUG]

Describe the bug
When using allocatable arrays in derived types, the data in the array is shared between different python variables.
This causes the old memory to leak (confirmed with valgrind), as well as causing some very surprising behavior.

Fortran code

module test
implicit none 

public :: get_array, print3, my_t

type my_t
	real, allocatable :: d(:)
	integer :: n
end type

contains 

function get_array(N) result(retval)
	type(my_t) :: retval
	integer, intent(in) :: N
	
	allocate( retval%d(N) )
	retval%n = N
	
	! substitute with real data 
	call random_number( retval%d )
end function

subroutine print3(x)
	type(my_t), intent(in) :: x
	print *, x%d(:3)
end subroutine

end module

Python code

#! /usr/bin/env python3

import numpy as np
import gfort2py as gf 
libmy = gf.fFort("./libmy.so", "./test.mod")

def get_array(N):
	return libmy.get_array(N).result
	
def print3(x):
	return libmy.print3(x)

x = get_array(10)
print3(x)

y = get_array(10)
print3(x) # should print the same numbers, but doesn't

Software

  • OS: OpenSuSe Linux 6.7.5-1-default
  • gfort2py version: 2.4.2
  • gfortran version: 13.2.1 20240206
  • gfortran compile options: gfortran -fPIC -shared -o libmy.so test.f90
  • Python version: 3.11.8
  • If this involves arrays numpy version: 1.26.2

utils_cpython.c:218:41: warning: division by zero

gfort2py/parseMod/utils_cpython.c:218:41: warning: division by zero [-Wdiv-by-ze
ro]
enum { __pyx_check_sizeof_voidp = 1 / (int)(SIZEOF_VOID_P == sizeof(void*))
};
^
gfort2py/parseMod/utils_cpython.c:218:12: error: enumerator value for '__pyx_che
ck_sizeof_voidp' is not an integer constant
enum { __pyx_check_sizeof_voidp = 1 / (int)(SIZEOF_VOID_P == sizeof(void*))
};

Move github testcases to thier own file

As part of the test suite clean up the test cases that reference a Github issue should be moved into their own github.f90 and github_test.py files. This will help clear up which test cases that must be kept as things get reorganised.

memory leak for allocatable arrays

Hi Robert,

I am working with a version of the dev branch from april / may (commit hash: dc68ee6), and numpy 1.19.1. The function I'm using passes and returns an array to a fortran allocatable array in a python loop. Everything works well, except that the memory consumed grows by the size of this array with each iteration of the loop. Garbage collection or deleting the fortran module at the end of each loop does not help, so I'm guessing that there are some unaccounted references in the python part of gfort2py. tracemalloc() points to the _save_value method in arrays.py, specifically np.asfortranarray, which grows by the size of the array being passed in each loop. Any suggestions on how to release the allocatable array memory? Working example is below:

gfortran -fPIC -shared -c params_test.f95
gfortran -fPIC -shared -o test.so params_test.f95

Python:


#!/usr/bin/python

import numpy as np
import gfort2py as gf

import tracemalloc
import os
import linecache

def display_top(snapshot, key_type='lineno', limit=3):
    snapshot = snapshot.filter_traces((
        tracemalloc.Filter(False, ""),
        tracemalloc.Filter(False, ""),
    ))
    top_stats = snapshot.statistics(key_type)
    print("Top %s lines" % limit)
    for index, stat in enumerate(top_stats[:limit], 1):
        frame = stat.traceback[0]
        # replace "/path/to/module/file.py" with "module/file.py"
        filename = os.sep.join(frame.filename.split(os.sep)[-2:])
        print("#%s: %s:%s: %.1f KiB"
              % (index, filename, frame.lineno, stat.size / 1024))
        line = linecache.getline(frame.filename, frame.lineno).strip()
        if line:
            print('    %s' % line)

    other = top_stats[limit:]
    if other:
        size = sum(stat.size for stat in other)
        print("%s other: %.1f KiB" % (len(other), size / 1024))
    total = sum(stat.size for stat in top_stats)
    print("Total allocated size: %.1f KiB" % (total / 1024))

SHARED_LIB_NAME='./test.so'
MOD_FILE_NAME='params.mod'

x = gf.fFort(SHARED_LIB_NAME, MOD_FILE_NAME)

numloops = 10
dim = 3000
arr_test = np.random.rand(dim, dim)
print(arr_test[0,0])

tracemalloc.start()
for ii in np.arange(numloops):
    
    print('python %d' % ii)
    y1 = x.set_param_defaults(arr_test, {'simtype':2,'l0':0,'field_int_on_currently':True})

    snapshot = tracemalloc.take_snapshot()
    display_top(snapshot)

    # print('returned allocatable array')
    # print(y1.args['alloc_test'])

    arr_test = y1.args['alloc_test']
    print('python kb: %f' % (arr_test.nbytes/1024))
    print(arr_test[0,0])
    print('\n')

fortran:


module params

    use, intrinsic :: iso_fortran_env
    
    implicit none

    public

    type d_params
        
        integer simType           
        real l0
        real, dimension(3, 4) :: cdims
        logical field_int_on_currently

    end type

contains

        subroutine set_param_defaults(alloc_test, xlc_p)
                
               implicit none
        
                integer :: I,J, flag
                real, intent(inout), allocatable :: alloc_test(:,:)
                type(d_params), intent(inout) :: xlc_p
       
		alloc_test(1,1) = alloc_test(1,1) + 1

        end subroutine set_param_defaults

end module params

sample output:
python 0
Top 3 lines
#1: core/_asarray.py:218: 35156.5 KiB
return array(a, dtype, copy=False, order='F', ndmin=1)
#2: gfort2py/types.py:42: 3.0 KiB
class ctypesStruct(ctypes.Structure):
#3: gfort2py/descriptors.py:80: 2.8 KiB
class _fAllocArray(ctypes.Structure):
86 other: 44.8 KiB
Total allocated size: 35207.2 KiB
python kb: 35156.250000
1.2496924

python 1
Top 3 lines
#1: core/_asarray.py:218: 35156.5 KiB
return array(a, dtype, copy=False, order='F', ndmin=1)
#2: gfort2py/arrays.py:332: 35156.3 KiB
self._value = np.asfortranarray(value.astype(self.dtype))
#3: python3.8/linecache.py:137: 63.1 KiB
lines = fp.readlines()
155 other: 97.1 KiB
Total allocated size: 70473.0 KiB
python kb: 35156.250000
2.2496924

python 2
Top 3 lines
#1: gfort2py/arrays.py:332: 70312.6 KiB
self._value = np.asfortranarray(value.astype(self.dtype))
#2: core/_asarray.py:218: 35156.5 KiB
return array(a, dtype, copy=False, order='F', ndmin=1)
#3: python3.8/linecache.py:137: 107.8 KiB
lines = fp.readlines()
157 other: 111.8 KiB
Total allocated size: 105688.7 KiB
python kb: 35156.250000
3.2496924

python 3
Top 3 lines
#1: gfort2py/arrays.py:332: 105468.8 KiB
self._value = np.asfortranarray(value.astype(self.dtype))
#2: core/_asarray.py:218: 35156.5 KiB
return array(a, dtype, copy=False, order='F', ndmin=1)
#3: python3.8/linecache.py:137: 107.8 KiB
lines = fp.readlines()
159 other: 123.0 KiB
Total allocated size: 140856.2 KiB
python kb: 35156.250000
4.2496924

OSError When user modules call gfortran intrinsic procedures

I use gfortran to create .so and .mod files for my Fortran 77 module file which contains some subroutines and functions. Some of gfortran intrinsic procedures, such as Time() and Len_trim(), are called. When I use gfort2py to load .so and .mod in my python program, it failed and reply :

Traceback (most recent call last):
File "test.py", line 8, in
x=gf.fFort(SHARED_LIB_NAME,MOD_FILE_NAME)
File "/home/holton_rum/.local/lib/python3.6/site-packages/gfort2py/gfort2py.py", line 33, in init
self._lib = ctypes.CDLL(libname)
File "/home/holton_rum/anaconda3/lib/python3.6/ctypes/init.py", line 348, in init
self._handle = _dlopen(self.name, mode)
OSError: ./utils.so: undefined symbol: time

What should I do to help gfort2py support gfortran intrinsic procedures? I sincerely hope that you can provide me some help. Thank you !

Fortran derived type with arrays

Hi, I have been able with gfort2py to represent a Fortran derived type as a Python OrderedDict, except when the derived type contains an array. I have not been able either to represent an array of derived types. Are these two requirements possible?

Sample test code fails

Hi,

I'm just starting to experiment with gfort2py and having some trouble. My Fortran is a hello world:

subroutine hello(n)
integer :: n
write(,) "hello world. The secret number is ",n
return
end

And i named this test1.f90. I then compiled it and created a library using the commands on the github README. Everything seems fine. I then wrote some simple Python following the example on the README:

import gfort2py as gf

SHARED_LIB_NAME = "./test1"
MOD_FILE_NAME="test1"

x = gf.fFORT(SHARED_LIN_NAME, MOD_FILE_NAME)

When I do this I get an error from gfort2py:

$ python3 test1.py
Traceback (most recent call last):
File "test1.py", line 6, in
x = gf.fFORT(SHARED_LIN_NAME, MOD_FILE_NAME)
AttributeError: module 'gfort2py' has no attribute 'fFORT'

I was trying to follow the example and I seemed to have screwed up somewhere. This is an Ubuntu 18.04 system I grabbed the last release of gfort2py (1.1.4). I built the code using Conda 4.7.5 (fairly close to the latest).

Any help is really appreciated.

Thanks!

Jeff

[BUG] calling Fortran subroutine from Python, but the wrong output

Describe the bug
calling Fortran subroutine from Python, but the wrong output is returned

Fortran code

module module_data
   implicit none
   public :: datamu
contains
   subroutine datamu(y, xp, yp)
	  use iso_fortran_env, only: dp => real64

	  real(dp), intent(inout) :: y
	  real(dp), intent(inout) :: xp(:)
	  real(dp), intent(inout) :: yp(:)

	  y = -1000.0_dp
	  xp = [13.0_dp, -2.0_dp]
	  yp = [1.0_dp, -42.014_dp]
	  print *, "inside datamu sub: y, xp, yp", y, xp, yp
   end subroutine datamu
end module module_data

Python code

import numpy as np
import gfort2py as gf

SHARED_LIB_NAME="./module_data.dll"
MOD_FILE_NAME="./module_data.mod"

pes = gf.fFort(SHARED_LIB_NAME,MOD_FILE_NAME)


y = 0.
x = np.array([-0.6, 1.25]) 
yp = np.array([0.,0.])   


output = pes.datamu(y, x, yp) 
print('output: {}'.format(output))

print("NOTE: output[1]  also different from the output part (see output above)")
print(output[0])
print(output[1])

Software (please complete the following information):

  • OS: Linux

  • gfort2py version:: '2.2.1'

  • gfortran version: 9.4.0

  • gfortran compile options: gfortran -fPIC -shared -o module_data.dll module_data.f90

  • Python version: Python 3.9.12

  • If this involves arrays numpy version: '1.22.3'

Additional context
output from the above program:
inside datamu sub: y, xp, yp -1000.0000000000000 13.000000000000000 -2.0000000000000000 1.0000000000000000 -42.014000000000003
output: Result(result=None, args={'y': -1000.0, 'xp': array([13., -2.]), 'yp': array([13., 2.])})
None
{'y': -1000.0, 'xp': array([13., 2.]), 'yp': array([13., -2.])}
So, I expect the output similar to what is inside the datamu subroutine (output precision truncated), i.e. y = -1000.00000, xp = [13.00, -2.00], and yp= [1.0000, -42.014].

**In a nutshell, the output yp is not correct.**

[BUG] Add support for returning array of characters

import gfort2py as gf


fstr =  """

recursive function return_char(x) result(str)
  implicit none
  integer, intent(in) :: x
  character(len=20), dimension((2 ** x - 1) ) :: str

  str = ''

end function return_char

"""


x = gf.compile(fstr)


res = x.return_char(2)

print(res.result)

Returns:

TypeError: Unknown type of character array not currently supported

any plans

Are there any plans for adding the missing bits?

and make it a proper modern fortran to py

Slow startup when the wrapper is first called

Describe the bug
We have been successfully using gfort2py to wrap Fortran modules. However, we find a longish time interval (as long as 15-20 s) between the moment the Python wrapper is invoked and the beginning of the Fortran excecution. Why is this so?

Fortran code
Minimal Fortran needed to reproduce this bug

Python code
Minimal Python needed to reproduce this bug

Software (please complete the following information):

  • OS: [e.g. Linux]
  • gfort2py version:
  • gfortran version:
  • gfortran compile options:
  • Python version:
  • If this involves arrays numpy version:

Additional context
Add any other context about the problem here.

Add procedure pointers

need to see how the pointers are handled and set, first stage towards adding CLASS maybe?

[ENH] Add `fAssumedShapeDT` to support the allocatable array of derived types

Description of the new feature
Allows fortran function to return an allocatable array of derived types.

Proposed Fortran code
An example of what the Fortran code looks like

    function func_return_allocatable_s_struct_nested_2() result(s)
        type(s_struct_nested_2),allocatable::s(:)
        allocate(s(2))
        s(1)%a_int = 123
        s(1)%f_nested%a_int = 234
        s(1)%f_nested%f_struct%a_int = 345
        s(2)%a_int = 456
        s(2)%f_nested%a_int = 567
        s(2)%f_nested%f_struct%a_int = 678
    end function func_return_allocatable_s_struct_nested_2

Proposed Python code
Sorry, I am not familiar.

[BUG] repr on functions raises AttributeError

Describe the bug
Calling repr on a function breaks when it calls repr on the arguments.

Fortran code
Minimal Fortran needed to reproduce this bug

Python code

import os, sys
import ctypes

os.environ["_GFORT2PY_TEST_FLAG"] = "1"

import numpy as np
import gfort2py as gf

import pytest

SO = f"./tests/basic.{gf.lib_ext()}"
MOD = "./tests/basic.mod"

x = gf.fFort(SO, MOD)

repr(x.func_int_in_multi)
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[4], line 1
----> 1 repr(x.func_int_in_multi)

File ~/src/gfort2py/gfort2py/fProc.py:187, in fProc.__repr__(self)
    186 def __repr__(self):
--> 187     return self.__doc__

File ~/src/gfort2py/gfort2py/fProc.py:198, in fProc.__doc__(self)
    196 args = []
    197 for fval in self.obj.args():
--> 198     args.append(fVar(fval.ref, allobjs=self._allobjs).__doc__)
    200 args = ", ".join(args)
    201 return f"{ftype} ({args})"

File ~/src/gfort2py/gfort2py/fVar.py:16, in fVar.__new__(cls, obj, *args, **kwargs)
     15 def __new__(cls, obj, *args, **kwargs):
---> 16     if obj.is_derived():
     17         if obj.is_array():
     18             if obj.is_explicit():

AttributeError: 'int' object has no attribute 'is_derived'

String arrays in derived type structure

I have been using successfully the new arrangement in the main branch to wrap in Python Fortran derived types containing arrays and arrays of derived types. Thank you for the prompt fix. It works OK with derived types containing integer and floating-point arrays but not with character arrays. Will it be too difficult to also consider the latter? I have just noticed that string arrays are not included in the list of things that work.

Is there any way to call a Fortran function/subroutine which not in any module?

symbols exported from such subroutines and functions don't have any prefix like __moduleName_MOD but has _ as the suffix. an example is given below:

root@localhost# ldd libdll.so
        linux-vdso.so.1 (0x00007fffdbdb0000)
        libgfortran.so.3 => /usr/lib/x86_64-linux-gnu/libgfortran.so.3 (0x00007f030c920000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f030c610000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f030c3f0000)
        libquadmath.so.0 => /usr/lib/x86_64-linux-gnu/libquadmath.so.0 (0x00007f030c1b0000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f030be00000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f030d000000)
root@localhost# nm -D libdll.so
0000000000202064 B __bss_start
                 w __cxa_finalize
0000000000202120 B __data_def_MOD_i
00000000002020a0 B __data_def_MOD_string
0000000000202064 D _edata
0000000000202128 B _end
0000000000000b2c T _fini
00000000000009c0 T func2_
0000000000000960 T __funcmod_MOD_func1
                 U _gfortran_concat_string
                 U _gfortran_st_write
                 U _gfortran_st_write_done
                 U _gfortran_transfer_character_write
                 w __gmon_start__
0000000000000780 T _init
                 w _ITM_deregisterTMCloneTable
                 w _ITM_registerTMCloneTable
                 w _Jv_RegisterClasses
0000000000000900 T sub1_
0000000000000a20 T sub2_

where func2, sub1 and sub2 are defiend without a module, but func1 is defined in module "funcmod".

pip install fails due to absence of Cython

I am unable to pip install gfort2py and get the error below.
This is odd, because I do have Cython.
It is also intriguing because the last commit message of pyproject.toml is "remove last bits of cython".
Any hints on how to solve the issue?

(.venv) PS C:\Code\fortran-in-python\src> pip list
Package        Version
-------------- -------
autopep8       2.0.2
ConfigArgParse 1.5.3
Cython         0.29.35
f90wrap        0.2.13
flake8         6.0.0
fprettify      0.3.7
llvmlite       0.40.0
mccabe         0.7.0
numba          0.57.0
numpy          1.24.3
pip            23.1.2
pycodestyle    2.10.0
pyflakes       3.0.1
setuptools     65.5.0
(.venv) PS C:\Code\fortran-in-python\src> pip install gfort2py
Collecting gfort2py
  Using cached gfort2py-1.1.7.tar.gz (100 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [21 lines of output]
      Traceback (most recent call last):
        File "C:\Code\fortran-in-python\.venv\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
          main()
        File "C:\Code\fortran-in-python\.venv\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Code\fortran-in-python\.venv\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 118, in get_requires_for_build_wheel        
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\hugom\AppData\Local\Temp\pip-build-env-zdq9znd6\overlay\Lib\site-packages\setuptools\build_meta.py", line 341, in get_requires_for_build_wheel     
          return self._get_build_requires(config_settings, requirements=['wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\hugom\AppData\Local\Temp\pip-build-env-zdq9znd6\overlay\Lib\site-packages\setuptools\build_meta.py", line 323, in _get_build_requires
          self.run_setup()
        File "C:\Users\hugom\AppData\Local\Temp\pip-build-env-zdq9znd6\overlay\Lib\site-packages\setuptools\build_meta.py", line 488, in run_setup
          self).run_setup(setup_script=setup_script)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\hugom\AppData\Local\Temp\pip-build-env-zdq9znd6\overlay\Lib\site-packages\setuptools\build_meta.py", line 338, in run_setup
          exec(code, locals())
        File "<string>", line 7, in <module>
      ModuleNotFoundError: No module named 'Cython'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

pip fails to install (ModuleNotFoundError: No module named 'Cython')

Steps to reproduce:

$ conda create -n gfort python
$ conda activate gfort
$ pip install gfort2py
Collecting gfort2py
  Downloading https://files.pythonhosted.org/packages/71/50/6f27804fc3642ccc132c13ff03d999c7cde957e1aa04102b58ec0140234f/gfort2py-1.1.2.tar.gz (97kB)
    100% |████████████████████████████████| 102kB 1.7MB/s 
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-c0ux4c0l/gfort2py/setup.py", line 7, in <module>
        from Cython.Distutils import build_ext
    ModuleNotFoundError: No module named 'Cython'
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-c0ux4c0l/gfort2py/

ValueError: invalid literal for int() with base 10: ''

$ cat a.py 
from gfort2py import fFort

x = fFort("./libhfsolver.so", "utils.mod")
$ python a.py 
Traceback (most recent call last):
  File "/home/ondrej/miniconda3/envs/gfort/lib/python3.7/site-packages/gfort2py/gfort2py.py", line 44, in _load_data
    f = open(self._fpy, 'rb')
FileNotFoundError: [Errno 2] No such file or directory: 'utils.fpy'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "a.py", line 3, in <module>
    x = fFort("./libhfsolver.so", "utils.mod")
  File "/home/ondrej/miniconda3/envs/gfort/lib/python3.7/site-packages/gfort2py/gfort2py.py", line 38, in __init__
    self._load_data(ffile, rerun)
  File "/home/ondrej/miniconda3/envs/gfort/lib/python3.7/site-packages/gfort2py/gfort2py.py", line 49, in _load_data
    pm.run(ffile, save=True)
  File "/home/ondrej/miniconda3/envs/gfort/lib/python3.7/site-packages/gfort2py/parseMod/parseMod.py", line 57, in run
    x.processData()
  File "/home/ondrej/miniconda3/envs/gfort/lib/python3.7/site-packages/gfort2py/parseMod/parseModCommon.py", line 54, in processData
    self.parseAllSymbols()
  File "/home/ondrej/miniconda3/envs/gfort/lib/python3.7/site-packages/gfort2py/parseMod/parseModCommon.py", line 185, in parseAllSymbols
    x = self.parseSymbol(i)
  File "/home/ondrej/miniconda3/envs/gfort/lib/python3.7/site-packages/gfort2py/parseMod/parseModCommon.py", line 238, in parseSymbol
    r['proc'] = self.parseProc(info)
  File "/home/ondrej/miniconda3/envs/gfort/lib/python3.7/site-packages/gfort2py/parseMod/parseModCommon.py", line 321, in parseProc
    res['ret'] = self.parseVar(info)
  File "/home/ondrej/miniconda3/envs/gfort/lib/python3.7/site-packages/gfort2py/parseMod/parseModCommon.py", line 271, in parseVar
    res['length'] = self.getStrLen(info)
  File "/home/ondrej/miniconda3/envs/gfort/lib/python3.7/site-packages/gfort2py/parseMod/parseModCommon.py", line 467, in getStrLen
    return int(y)
ValueError: invalid literal for int() with base 10: ''

Fix filename handling with dots in paths

parsemod.py we need to replace

def fpyname(filename):
return filename.split('.')[0] + '.fpy'

with

def fpyname(filename):
return os.path.splitext(filename)[0]+'.fpy'

Derived types

I am trying to get familiar with the Python-Fortran transcription of arrays within derived types. I have the following Fortran module:

module expose3
implicit none
public :: point, derived_structure
type :: point
integer, dimension(4) :: iq
end type point
contains
subroutine derived_structure(p)
type(point), intent(out) :: p
p = point([10,20,30,40])
print '("Fortran: p = ",5i5)', p
end subroutine derived_structure
end module expose3

which I compile with the commands

gfortran -fPIC -shared -c expose3.f90
gfortran -fPIC -shared -o expose3.so expose3.o

and run with the Python script

import gfort2py as gf
import numpy as np
SHARED_LIB_NAME='./expose3.so'
MOD_FILE_NAME='expose3.mod'
x=gf.fFort(SHARED_LIB_NAME,MOD_FILE_NAME)
p = {}
y=x.derived_structure(p)
print('\n','Python: ',y,'\n')

The printed derived-type array p%iq(i) in Python and Fortran are respectively:

Python: {'p': OrderedDict([('iq', array([20, 30, 40, 0]))])}
Fortran: p = 10 20 30 40

As you can see, the Fortran p%iq(2) = 20 is equivalent in Python to y['p']['iq'][0]=20 and the P%iq(1) = 10 element is totally excluded in the Python representation. Is this correct?? Should it not be, in any case, P%iq(2) = 20 equivalent to y['p']['iq'][1]=20? Please clarify. Thanks, Claudio

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.