Currently, if you issue fpm run in a library project without executable program, it does nothing.

It'd be more user friendly if we emitted a message to stderr, something like:

fpm: This package has no executable programs.

or similar.

Likewise for fpm test. If there are no test, we should say something like:

fpm: This package has no tests.

What do you think?

Currently, FPM emits a static library file that has the same name as the project. For example, first line of fpm.toml could be:

name = "datetime"

Then, the static library created is datetime.a. However, the canonical way to name libraries (static and dynamic) is to use a lib prefix: libdatetime.a. Then one can link to this library with -ldatetime.

Current workaround can be to add the lib prefix to the package name in fpm.toml:

name = "libdatetime"

However, this isn't pretty, and would down the road be more awkward once we're installing packages from remote URLs.

Let's add the lib prefix to library files.

Hi there
I started learning OpenMP couple weeks ago and would like to use it to parallelize and speed up fortran standard library codebase.

There is a bug at our CI that I haven't been able to figure out yet. Here is an example of it: https://github.com/fortran-lang/fpm/runs/390475601.

Here is what I know:

1. It only happens on macOS, never on Linux or Windows

2. Restarting the build typically fixes it (sometimes it fails 2x or 3x in a row, but eventually it always passes)

3. The cargo test runs in parallel by default, so I set -j1 to run in serial. That seemed to improve how often it fails (although I could be wrong on that). It still fails sometimes however, so the actual bug is still there.

4. The error is:

thread 'test_2' panicked at 'Unexpected failure.
code=<interrupted>
stderr=``````
command=`"/Users/runner/runners/2.163.1/work/fpm/fpm/target/x86_64-apple-darwin/debug/fpm" "build"`
code=<interrupted>
stdout=``````
stderr=``````
', /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/assert_cmd-0.10.2/src/assert.rs:148:17

Which is caused by the assert_cmd package, which just uses the Rust's std::process::Command, when the output command did not succeed, but it also failed to retrieve the error code, which according to the documentation means the process was interrupted by the system (with some signal like SIGKILL).

cargo build does not build tests. You have to do cargo build --tests to also build tests.

cargo test build tests (if needed) and runs them.

I suggest we follow the same approach.

Except for small/simple projects, I almost always adopt Fortran submodules now to structure new projects. This is primarily for the parallel and incremental builds that they allow.

I realise this may be low down on the list of project priorities currently, but I think it should only require an update to the dependency tracking whereby submodules depend on their parent modules as if they use them.

Much further down the line, I assume fpm will aim for some kind of parallel build support?

My basic outline of how to do this is the following recursive algorithm:

1. for each listed dependency, download/check it out
2. Go into the dependency's fpm.toml file and for each of it's dependencies
   a. if it's already been downloaded, check that it's version is compatible, otherwise recursively repeat this process
3. Collect each of the dependencies dependencies,
4. Return the tree of dependencies
5. Build each of the dependencies, recursively building each of it's dependencies first

I've been trying to get fpm to work with some old-school packages like MINPACK: https://github.com/certik/minpack. These projects are generally just a bunch of fixed format ".f" files. For modern applications that rely on such legacy codes, an interface module is the first step towards safe function calls.

Trying to run fpm build on @certik's MINPACK fails with errors akin to:

fpm build --release
# gfortran (for build/gfortran_release/minpack/rwupdt.o build/gfortran_release/minpack/rwupdt.mod)
# gfortran (for build/gfortran_release/minpack/dmchar.o build/gfortran_release/minpack/dmchar.mod)
# gfortran (for build/gfortran_release/minpack/lmpar.o build/gfortran_release/minpack/lmpar.mod)
# gfortran (for build/gfortran_release/minpack/hybrd.o build/gfortran_release/minpack/hybrd.mod)
fpm: Error when running Shake build system:
  at want, called at src/Build.hs:205:11 in fpm-0.1.0.0-ExRRX9W2aflLvTBBChPpxh:Build
* Depends on: build/gfortran_release/minpack/libminpack.a
  at need, called at src/Build.hs:203:13 in fpm-0.1.0.0-ExRRX9W2aflLvTBBChPpxh:Build
* Depends on: build/gfortran_release/minpack/rwupdt.o
  at &%>, called at src/Build.hs:183:11 in fpm-0.1.0.0-ExRRX9W2aflLvTBBChPpxh:Build
* Depends on: build/gfortran_release/minpack/rwupdt.o build/gfortran_release/minpack/rwupdt.mod
  at error, called at src/Development/Shake/Internal/Rules/Files.hs:245:13 in shake-0.18.5-44KSA7uQF2VObxzEvLYZx:Development.Shake.Internal.Rules.Files
* Raised the exception:
Error, &%> rule failed to produce 1 file (out of 2)
  build/gfortran_release/minpack/rwupdt.o
  build/gfortran_release/minpack/rwupdt.mod - MISSING

I understand that fpm is expecting to find a module file for each single ".f" file. Is it possible to somehow work around this? The farthest I could get it in the fpm source was to remove to the "-Wimplicit-interface" flag which was creating lots of warnings 🙈 .

Am I right to think fpm is currently most suitable for projects composed of multiple module files?

Here are a few good candidate packages to get working with fpm first, that are relatively simple (and so possible to package soon), yet very useful.

Pure Fortran (Simple)

• https://github.com/wavebitscientific/functional-fortran
• https://github.com/jacobwilliams/bspline-fortran
• https://github.com/fortran-lang/stdlib
• https://github.com/astrofrog/fortranlib
• https://github.com/certik/fortran-utils/
• https://github.com/wavebitscientific/datetime-fortran
• https://github.com/jacobwilliams/pyplot-fortran

Pure Fortran (More Complex)

• https://github.com/Reference-LAPACK/lapack
• https://github.com/opencollab/arpack-ng
• https://github.com/jacobwilliams/json-fortran
• https://github.com/jacobwilliams/Fortran-Astrodynamics-Toolkit
• https://github.com/Goddard-Fortran-Ecosystem/pFUnit
• https://github.com/szaghi/StringiFor (Lots of dependencies)

Fortran with non-Fortran dependencies

• https://github.com/scivision/h5fortran
• https://github.com/scivision/nc4fortran
• https://github.com/SHTOOLS/SHTOOLS
• https://github.com/unidata/netcdf-fortran

And obviously any other more complicated package from https://github.com/fortran-lang/stdlib/wiki/List-of-popular-open-source-Fortran-projects.

Any other candidates?

There will be a lot of packages that do not have fpm.toml. Here is my suggested approach how to handle that:

1. Encourage every package to use fpm.toml and to use fpm.

2. Those packages that do not use it yet could be handled as follows: we will maintain a version (fork) at GitHub or GitLab that includes the fpm.toml. It will be this fork that would be used with fpm. From fpm's perspective, each package always contains fpm.toml.

3. The alternative to (or modification of) 2. is to allow fpm.toml to specify where to find sources of the actual package. So our GitLab package can be just one file fpm.toml that would list the metadata and where to download the sources plus any patches to them.

In particular, here is my plan: I will start with forking the packages listed at #17 and implementing fpm.toml together with any modifications that might be needed. I will not submit a PR back initially, but rather simply get my forks working well with fpm, and test it all out and get some usage. Then, as things start to get more serious and the fpm tool matures, we can easily send a PR against the upstream package and start the discussion with upstream authors if they would be willing to use fpm and maintain fpm.toml themselves. And depending on how this conversation goes, we'll either do just 2., or if we need to, we can also implement 3. in fpm. I expect that upstream authors will give us a list of features that they need fpm to have implemented, and once we do, they would be willing to use it.

This issue is similar to #70 but for Spack instead of Conda. Essentially for people that use Spack (myself included sometimes), it would be nice to just be able to spack install any fpm package.

To do that, fpm should be able to take any package and produce a working Spack package out of it, that can then be submitted to Spack.

The generated Spack package would probably call fpm underneath to do the build.

We need to add more tests:

• test that the default directory is target (in some temporary directory)
• test other locations of the build directory (will require to construct relative paths in CMakeLists.txt)

We've been working with @everythingfunctional on the standardization of the layout. First iteration:

$ tree .
.
├── fpm.toml
└── src
    ├── a.f90
    ├── b
    │   └── utils.f90
    └── utils.f90

2 directories, 4 files
$ cat fpm.toml 
Name "a"
$ cat src/a.f90 
module a
use a_utils, only: util1
use a_b_utils, only: util2

implicit none
private
public util1, util2
end module
$ cat src/utils.f90 
module a_utils
implicit none
private
public util1

integer :: util1

end module
$ cat src/b/utils.f90 
module a_b_utils
implicit none
private
public util2

integer :: util2

end module

I would do exactly what Cargo does: https://doc.rust-lang.org/cargo/reference/build-scripts.html

Only I would allow the build script to be any of: any binary, cmake, make, Bash script.

Otherwise the following I would do exactly the same:

• Inputs: environment variables

• Outputs: the build script prints to stdout with lines starting with fpm:, everything else is ignored. At the beginning, I would support the following, and we can add more later:

  • fpm:fc-link-lib=[KIND=]NAME --- Adds a library to link (doc)
  • fpm:fc-flags=FLAGS --- Passes certain flags to the compiler (doc).

  That should be enough to get us started. One of the environment variables passed to the build script is which Fortran compiler is being used, so the build script would know what compiler parameters to pass back (for example ifort in general might require different flags than gfortran). In the same way, the flags might depend on the platform (macOS vs Linux vs Windows), so one of the input variables can be what platform we run on.

Eventually we need to have a central place for packages similar to crates.io.

But for now we will use a git repository (GitHub, GitLab and other places will work) as well as just url for a tarball. That way we don't need to host anything ourselves at first and can get the initial community and ecosystem of packages built up without worrying about security and other issues that will come with maintaining our own repository.

Use case: I'd like to build my program or library that depends on FGSL with fpm. I know how to specify such dependency (in theory, at least) and let's assume that there are no issues in fetching, compiling and linking.

The issue I'm raising here is: what if I want to make sure that the fetched and compiled dependency passes its tests on my machine before proceeding with linking and building my own program or library? How can I do that with fpm?

I believe it's a non-trivial problem considering that each external dependency does not conform to fpm specs. Off the top of my head: maybe an additional key in the TOML file can be specified by the user that specifies how to test the dependency (e.g. make test)? However, I think that it is a sub-optimal solution because requires the user to gain this information for each dependency but it would be hard to have a general reliable method otherwise. Please correct me if I'm off-track.

Now that fpm is in a usable state, I think we should try and create a docker image with the latest version installed, so people can use fpm in their CI.

Currently the new fpm does not seem to have any tests. We have to fix it, and start adding lots of tests.

In the old fpm, I had two initial tests:

https://github.com/fortran-lang/fpm/tree/master/archive/tests

And we need to add a lot more. The idea would be to test any configuration that is supported, and that way we can ensure that it works and things don't break when new functionality is implemented.

Furthermore, if we decide to use a different language for the production implementation, the tests will allow us to ensure that the new implementation works as expected.

They mean the same, but they look and "feel" different. We should have a consistent way to refer to this so it's not confusing to the readers whether they're different things. Especially as we begin to write about these things more publicly?

I've been using both. I don't know which one I prefer. Perhaps 60/40 in favor of fpm.

So, which one should it be?

Following the discussion at the monthly call, I've installed fpm and am trying to convert a few of my Fortran projects to the recommended structure. These projects involve some legacy fixed-style functions and subroutines, an interface module, and several driver programs (tests). The folder structure is something like:

.
├── drivers
│   ├── driver1.f90
|   └── driver2.f90
├── src
│   ├── legacy
|   |   ├── legacy1.f
|   |   └── legacy2.f
|   └── interface.f90
└── fpm.toml

Do I need to add multiple sections such as

[[driver]]
name = "driver1"
source-dir = "drivers"
main = "driver1.f90"

to the TOML file? Right now my build just crashes because of multiple executable files.

I think having multiple executable programs is a quite common type of package, say you have a library for building mesh triangulations, and then with it a small set of command line tools calling routines from this common library.

Another question is can I already specify dependencies on system-wide installed libraries. Say a project (or one of it's dependencies) needs to be linked with -llapack -lblas (with the actual LAPACK and BLAS libraries potentially in some non-default location). Does this fit under issue #99 of custom build scripts?

I had a few other comments about the installation process, but they were already fixed in #101. A remaining point might be that after doing stack install in the fpm folder, the ${HOME}/.local/bin/ folder might not be on the path already, requiring an extra command export PATH="$PATH:$HOME/.local/bin/".

This is related to #69 but it's a separate issue. fpm should be able to create a Conda package for the project, so that it's easy to submit it to conda-forge.

fpm is currently making gfortran fail on warning by hardcoding -Werror. Removing -Werror from fpm.toml doesn't change this behavior (I understand specifying flags via fpm.toml may not be implemented yet.

However, in the meantime, we shouldn't use -Werror because it's making correct code to fail. For example:

[milan@gary test_project_error]$ cat src/mylib.f90 
real :: a, b
a = 2.
b = 2.
print *, a == b
end
[milan@gary test_project_error]$ cat fpm.toml 
name = "package-name"
version = "0.1.0"
license = "BSD3"
author = "Author name here"
maintainer = "[email protected]"
copyright = "2020 Author name here"
dependencies = []
compiler = "gfortran"
devel-options = ["-g", "-Wall", "-Wextra", "-pedantic"]
release-options = ["-O3"]

[library]
source-dirs = "src"
[milan@gary test_project_error]$ fpm build
# gfortran (for build/debug/library/mylib.o build/debug/library/mylib.mod)
src/mylib.f90:4:8:

 print *, a == b
        1
Error: Equality comparison for REAL(4) at (1) [-Werror=compare-reals]
f951: all warnings being treated as errors
fpm-exe: Error when running Shake build system:
  at want, called at src/Build.hs:188:11 in fpm-0.1.0.0-9zFE4ut013U9YSOSmXT3I3:Build
* Depends on: build/debug/library/package-name.a
  at need, called at src/Build.hs:186:13 in fpm-0.1.0.0-9zFE4ut013U9YSOSmXT3I3:Build
* Depends on: build/debug/library/mylib.o
  at &%>, called at src/Build.hs:166:11 in fpm-0.1.0.0-9zFE4ut013U9YSOSmXT3I3:Build
* Depends on: build/debug/library/mylib.o build/debug/library/mylib.mod
  at cmd, called at src/Build.hs:179:19 in fpm-0.1.0.0-9zFE4ut013U9YSOSmXT3I3:Build
* Raised the exception:
Development.Shake.cmd, system command failed
Command line: gfortran -c -Jbuild/debug/library -Wall -Wextra -Wimplicit-interface -Werror -fPIC -fmax-errors=1 -g -fbounds-check -fcheck-array-temporaries -fbacktrace -o build/debug/library/mylib.o src/mylib.f90
Exit code: 1
Stderr:
src/mylib.f90:4:8:

 print *, a == b
        1
Error: Equality comparison for REAL(4) at (1) [-Werror=compare-reals]
f951: all warnings being treated as errors

What do you think?

We need a specification, or at least a start of one, for what should go into the package file and how it should be organized.

I'd suggest we start by answering the question "What does fpm need to know about a package in order to be able to build it?" in as much detail as possible and then trying to organize it logically.

I know it might be a bit early but I'd like to log this issue that is particularly important for me.

To use routinely fpm, I would need a functionality that works a bit like sudo make install to install the executable or library system-wide.

PS: thanks for your great work!

Are we sold on using toml as the file format for a package manifest/configuration file?

I see that Rust uses toml, but many other package mangers use others. Haskell's Stack uses yaml, and I think I've seen others use that. I think I've also seen json, and maybe even xml.

A new project Cargo.toml looks like:

[package]
name = "rusttest"
version = "0.1.0"
authors = ["Brad Richardson <[email protected]>"]
edition = "2018"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]

and a new project package.yaml (for Haskell) looks like:

name:                haskelltest
version:             0.1.0.0
github:              "githubuser/haskelltest"
license:             BSD3
author:              "Author name here"
maintainer:          "[email protected]"
copyright:           "2020 Author name here"

extra-source-files:
- README.md
- ChangeLog.md

# Metadata used when publishing your package
# synopsis:            Short description of your package
# category:            Web

# To avoid duplicated efforts in documentation and dealing with the
# complications of embedding Haddock markup inside cabal files, it is
# common to point users to the README.md file.
description:         Please see the README on GitHub at <https://github.com/githubuser/haskelltest#readme>

dependencies:
- base >= 4.7 && < 5

library:
  source-dirs: src

executables:
  haskelltest-exe:
    main:                Main.hs
    source-dirs:         app
    ghc-options:
    - -threaded
    - -rtsopts
    - -with-rtsopts=-N
    dependencies:
    - haskelltest

tests:
  haskelltest-test:
    main:                Spec.hs
    source-dirs:         test
    ghc-options:
    - -threaded
    - -rtsopts
    - -with-rtsopts=-N
    dependencies:
    - haskelltest

I'm not strongly opposed to toml, but it's the one I'm least familiar with. My gut tells me we should go with yaml. I would not suggest json or xml though, too much syntax/line noise.

After #33 is implemented, the next step is to implement a minimal metadata registry. Here is one way to do that:

1. Have repository https://github.com/fortran-lang/package-registry that would contain a simple JSON file of this form which is the package registry:

[{
    "name": "stdlib",
    "versions": [
        {"version": "0.3.4", "url": "https://github.com/fortran-lang/stdlib/archive/0.3.4.tar.gz"},
        {"version": "0.3.5", "url": "https://github.com/fortran-lang/stdlib/archive/0.3.5.tar.gz"},
    ]
}, {
    "name": "bspline",
    "versions": [
        {"version": "6.0.0", "url": "https://github.com/jacobwilliams/bspline-fortran/archive/6.0.0.tar.gz"},
        {"version": "5.4.2", "url": "https://github.com/jacobwilliams/bspline-fortran/archive/5.4.2.tar.gz"},
    ]
}]

2. We will then have scripts that take this JSON file and download the actual metadata for each package version. So for example, to obtain the metadata for the package bspline version 5.4.2, it would download the tarball https://github.com/jacobwilliams/bspline-fortran/archive/5.4.2.tar.gz, unpack and it would read its fpm.toml, which would contain all the metadata such as short and long description, the list of dependencies, and other things. Then we can automatically create a website which would list all this metadata. This generated website would contain a generated file metadata.json, which the fpm tool can then download to obtain a searchable data base of packages (fpm search).

3. To add a new package to registry, just a new simple entry must be made to the above JSON file by hand, say by issuing a PR against the repository.

4. Later we can automate things more, similarly to how conda-forge works (https://conda-forge.org/docs/maintainer/adding_pkgs.html), where to put a new package in, a PR is sent against https://github.com/conda-forge/staged-recipes/, where the CI checks initial quality and that the package builds, and then if it gets merged, the CI actually creates a new repository for the package etc. In our case, we could have a staging repository, and if a PR is merged, the CI would correctly update the above JSON file.

We can discuss if the JSON file should also contain all the metadata from fpm.toml directly. The advantage of the above approach is that it is not redundant, the JSON only contains the minimal amount of information that can be edited and maintained by hand, and if you want more, you download the tarball and read its fpm.toml, which will be done automatically in the step 2.

Overall, this minimal package registry only contains a minimal JSON file. The actual tarballs and metadata are hosted elsewhere. After this is well implemented and works, we can evolve it into a full package registry (#35).

Just like in Cargo.

We should work on this after #25 is fixed.

Currently the Haskell based fpm is compiling things directly, just like Cargo does it. That's the best default. (The "archived" Rust based fpm was generating CMake.)

However, until fpm is everywhere used by everybody, it would be super helpful to have a CMake and pure Make backends. That way we can move packages like stdlib or any of my own packages into using fpm, and yet not require it from the users --- they would continue using CMake or Make.

Also by generating the CMake / Make automatically, it will be a lot easier to maintain those in stdlib for example.

Since fpm knows (or should know) everything about the Fortran project, it can generate 100% robust CMake and Make files, following the latest practices for CMake / Make, human readable, etc.

I am thinking having these backends would be beneficial for many purposes, such as debugging, anyway. We should also have a Ninja backend.

How should we organize/specify compiler flags?

Should we only have two sets of compiler flags? devel-options and release-options, and they are used to build all dependencies and executables?

Also, should we come up with our own keywords for specifying the compiler options, and have a look up table to determine the correct flag for each compiler?

If I have a source file datetime.f90 that contains a module called datetime_module, then gfortran (and all other compilers I believe) output datetime_module.mod. However, fpm expects datetime.mod and can't find it.

I believe the assumption about .mod filename comes from somewhere around here, however I don't have a solution right now.

I think the safe assumption is that the .mod file will have the module name, for fpm will need to scan the source file to get this info.

Further, a source file can have multiple modules defined, and for each the compiler will emit one .mod file.

Currently, FPM can build only one executable program, namely main.f90.

Here's how Cargo does it:

.
├── Cargo.lock
├── Cargo.toml
├── src/
│   ├── lib.rs
│   ├── main.rs
│   └── bin/
│       ├── named-executable.rs
│       ├── another-executable.rs
│       └── multi-file-executable/
│           ├── main.rs
│           └── some_module.rs
├── benches/
│   ├── large-input.rs
│   └── multi-file-bench/
│       ├── main.rs
│       └── bench_module.rs
├── examples/
│   ├── simple.rs
│   └── multi-file-example/
│       ├── main.rs
│       └── ex_module.rs
└── tests/
    ├── some-integration-tests.rs
    └── multi-file-test/
        ├── main.rs
        └── test_module.rs

So, for Fortran, the main executable could be src/main.f90, and other executables could go in src/bin.

I like this approach, and I am not married to this particular directory structure. I also like the structure that we have now, namely app/, src/, tests/.

Currently compiled binaries are placed in build/debug/ and build/release.
Would it be possible to make this build path dependent on compiler (and maybe architecture)?

e.g. build/gfortran-9.2.0-x86_64_debug/

This reason for this being that I like to build and test my projects on multiple compilers during development; separating the binary output paths allows incremental builds using multiple compilers and allows tests on each compiler to run simultaneously and without clean and rebuild.
This may be considered an 'ugly' directory structure, however since fpm handles all compiler/linker paths and offers the fpm run command, this has no effect on end-user experience.

The workflow for using a non-default compiler could then be along the lines of:

$> export FC=ifort
$> fpm run

or

$> FC=ifort fpm run

or

$> fpm run --compiler=ifort

That is what Cargo does and what I also find intuitive. Very common when I want to just try something out in Fortran.

The CMake syntax used in this Rust script requires at least CMake 3.13.
In general in the Fortran stdlib we currently planned to use CMake 3.14 minimum.

Naming of fpm.toml. Cargo names Cargo.toml with capital C, and as explained in https://doc.rust-lang.org/cargo/faq.html#why-cargotoml, to "ensure that the manifest was grouped with other similar configuration files in directory listings. Sorting files often puts capital letters before lowercase letters, ensuring files like Makefile and Cargo.toml are placed together." If we want to do the same, the candidates are Fpm.toml and FPM.toml. I think fpm.toml looks better. But using a capital letter would make it similar to CMakeLists.txt also. We might want to devise a different name or naming scheme. Any ideas?

This issue is to ask whether fpm will have any built-in support for preprocessing and how this might look.

I bring this up since I noticed that stdlib is listed in #17 under 'Pure Fortran', however it requires the fypp preprocessor to build from repo source.

Put the CMakeLists.txt into target/, and ensure the source directory is not polluted at all.

Rust allows the packages to be decentralized, they do not have to be listed at https://crates.io/.

The Go language does not even have a centralized package registry at all, see this blog post that explains it in the section "Package Management":

https://nullprogram.com/blog/2020/01/21/

Modules are named by a module path that includes its network location. This means there’s no land grab for popular module names.

I don't like the Go's model that the name of the package is the url, I prefer the Rust model where a package has a simple name, but you specify where it can be downloaded. The end result is the same. (Overall, the Rust package management seems much more thought out, and it seems they fixed the issues that Go is tackling long time ago.)

I want to implement the same approach for fpm. That will allow us to build an ecosystem of packages for Fortran, without a "goldrush" to reserve a popular package name in a centralized registry. Then, as we have a healthy ecosystem of packages, we can add a centralized registry later.

How to support packages that do not conform to our "standard layout" (to be specified...). Some examples of such a package would be reference Lapack, or Arpack. The way to do that is that we create a new repository, say certik/lapack.fpm, which will have fpm.toml, in there it would specify the url to the actual sources (https://github.com/Reference-LAPACK/lapack) and a build script, which would build the sources (using CMake in this case) and install them into some $PREFIX provided by fpm and fpm takes it from there. This approach also works for non Fortran packages --- the build script either builds it, or requires it from the system (where it can be provided by, e.g., Spack). Either way this is a clean way to hook this up into the fpm ecosystem.

After #24 and #31 is fixed, set the --target-dir to a temporary directory for tests. That way multiple tests can reuse the same source directory for different tests, in parallel.

After #34 is implemented, we can discuss how to implement a full package registry similar to https://crates.io. It would be an extension of #34, which would also include hosting of the tarballs, and collecting usage statistics and other things. I think the solution in #34 can be evolved into this.

I'm creating this as mostly a reference for how I think we should proceed with having sane defaults to allow minimal fpm.toml files. Right now I'm just reading the toml contents directly into the data structure for the settings for building the project. What I think will make more sense is to have two different data structures. One for reading the settings from the fpm.toml file, and a slightly different one for the settings for the project passed to the rest of the process. This will allow a step for interrogating the file system for some of the defaults based on the presence/absence of information in the fpm.toml file.

First time running stack. On running stack build, it looks like the fpm executable is not installed in any working path. I expected it to be installed either somewhere under stack like pip does, or locally and then added to my path.

For example, I'm working in /home/milan/Work/fortran/fpm, fpm-exe is installed here: /home/milan/Work/fortran/fpm/.stack-work/install/x86_64-linux-tinfo6/68e6a00e61079ec9146947ae047a1ce619d26ace12c1e4cd5fd58ceb496743b0/8.6.5/bin

so I had to add this to my PATH.

Is this expected behavior? If not, is this an issue with stack or with fpm?

fpm-exe should be called just fpm.

As discussed in #33, let's allow installing fpm-enabled packages form their git repositories.

If the following is placed in fpm.toml:

[dependencies]
dependency_name = { git = git_url, tag = git_tag }

Then the package depends on dependency_name, which fpm will fetch from git_url and a specific git tag git_tag.

Here's a concrete example that will work, when this issue is implemented.

[dependencies]
datetime = { git = "https://github.com/wavebitscientific/datetime-fortran", tag = "v1.7.0" }

Question: If tag is omitted, should fpm fetch from the latest commit on master? I think so. Cargo does it like that as well. Then you could also do:

[dependencies]
datetime = { git = "https://github.com/wavebitscientific/datetime-fortran" }

It was just pointed out to me that fpm is used by another project: https://github.com/jordansissel/fpm. Unfortunately it is in a similar field (also a package manager).

Here are some options going forward (I'll update this list if there are more):

1. Rename our fpm
2. Keep the name fpm and ensure that people do not mistake the two projects (what's the best way?)

As to myself, I really like the name fpm to mean a Fortran Package Manager. So I vote for 2., if there is a way to do it.

I am very confident we can make fpm very robust to work for pure Fortran packages. Just like Cargo works well for pure Rust packages or pip works great for pure Python packages.

The problem is with non-Fortran packages. Pip allows to hook in compiling C (and with some work) Fortran code, but it's very fragile in my experience (thus the motivation for Conda that is a binary package manager). Python doesn't have an option to avoid C++/Fortran dependencies because Python itself is slow. Rust allows to (in principle) rewrite everything in Rust. As an example, take png. The system bindings: https://crates.io/crates/libpng-sys they say are unreliable, and you should use a pure Rust implementation: https://lib.rs/crates/lodepng.

I agree it does make things more robust to stick to pure Fortran and for many things we will do that and people will provide pure Fortran implementation of basic tasks. Python cannot do it well due to performance, but Rust and Fortran can.

However, we still need a robust way of handling non-Fortran dependencies, because if there is a robust and well maintained library in another language, we should just use it instead of reimplementing everything. Take HDF5. Here is the Rust package: https://crates.io/crates/hdf5. If you look at the documentation how to build it: https://github.com/aldanor/hdf5-rust they even mention Conda (to install the HDF5 library itself on all platforms --- which in my experience is much more robust than pip). Anyway, the way it works is by this line: https://github.com/aldanor/hdf5-rust/blob/15ec644977e0bee2b77340272730b34209c3765b/Cargo.toml#L11 which causes Cargo to execute this script: https://github.com/aldanor/hdf5-rust/blob/15ec644977e0bee2b77340272730b34209c3765b/build.rs which emits flags how to link against HDF5 correctly. That way the Cargo itself doesn't need to know almost anything, it just parses the output of this file. I think we should follow the same approach in fpm. The flags in this case are emitted by: https://github.com/aldanor/hdf5-rust/blob/d4c3737772bec477739c75566a3d52d0a44f27ba/hdf5-sys/src/lib.rs#L65. I think this is when you link against hdf5 rust package in your own code. How to link against hdf5 library itself is done here I think: https://github.com/aldanor/hdf5-rust/blob/d4c3737772bec477739c75566a3d52d0a44f27ba/hdf5-sys/build.rs#L566, it's quite complicated unfortunately.

But it's clean from the Cargo side, it offloads the responsibility to the package itself. We can provide helpers that fpm packages can use to work with things like pkg-config, cmake packages, etc.

In Rust it looks like each package is on their own, so for example this HDF5 package has messy code for each platform, e.g., here: https://github.com/aldanor/hdf5-rust/blob/d4c3737772bec477739c75566a3d52d0a44f27ba/hdf5-sys/build.rs#L492, you can see they are checking brew, or Windows registry, etc.

The good news is that Fortran codes do not need many non-Fortran dependencies, and so doing what Rust does might work for us. What I've seen is that Fortran codes mostly need some of: Lapack, MPI, FFT, MKL, HDF5, JSON, NETCDF, HYPRE, ...

Of which MPI and Lapack being the most important. I think fpm will have support for all Fortran compilers and I think it can have built-in support for MPI and Lapack also. One reason to special case MPI and Lapack is so that one can switch MPI implementaitons and Lapack implementations easily, and not have the Fortran packages hardcoded with a particular implementation.

With those out of the way, the rest can be done Cargo style, at least for now. Most other packages have just one implementation, so Fortran packages can just depend on a particular package (say Arpack, or Scalapack).

It would work just like cargo init:

$ cargo init myproject1
     Created binary (application) package
$ tree -a myproject1
myproject1
├── Cargo.toml
├── .git
│   ├── config
│   ...
├── .gitignore
└── src
    └── main.rs

10 directories, 18 files

As you can see, it starts a git repository, and so on. It is ready to compile, so:

~$ cd myproject1
myproject1(master)$ cat src/main.rs 
fn main() {
    println!("Hello, world!");
}
myproject1(master)$ cargo run
   Compiling myproject1 v0.1.0 (/tmp/myproject1)
    Finished dev [unoptimized + debuginfo] target(s) in 1.25s
     Running `target/debug/myproject1`
Hello, world!

If you just call cargo init, then it creates a new project in the current directory. Similar to git init.

This is more of a discussion than an issue.

As a developer who routinely develops in FORTRAN and C, I have come across Conan, http://conan.io a C/C++ package manager.

Now in my mind C and FORTRAN are very similar in character ... both have attributes such as:

• compilers
  • compiler vendors
  • compiler versions
  • compiler flags
• bitness (32 vs 64)
• debug/release builds

Etc.

Conan manages these very well via a hash assigned to the package.

Now I tried about two years ago to implement a similar scheme, to what Conan was doing but finally said ... they just do it better.

Today I use VS Code, CMake/Ninja and Conan as my "best practice". ( BTW, the new Ninja version 10. native support for FORTRAN modules ) This set up supports Intel FORTRAN/GFORTRAN and hopefully FLANG soon

So I am curious what do you think you bring new to the table.

Please don't think I am just throwing stones ... I have hundreds of thousands of lines of FORTRAN code, and Conan is bit of a "force fit", but not bad, ... so I truly am looking to improve my best practice.

Following up on the discussion in #60, let's remove the requirement for the compiler setting in the package-specific fpm.toml.

For now, FPM should default to gfortran, and we can put a note in the README or the packaging guide that gfortran is currently the only compiler supported. We can relax this at a later time. The compiler setting can still remain in the toml parser as optional--perhaps we will want to allow package-specific choice of compiler.

It is extremely common to have optional dependencies in Fortran projects (see the examples section at the end of this description).

The way Cargo handles it is described in here: https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html

[dependencies]
foo = { version = "1.0", optional = true }
bar = { version = "1.0", optional = true }

[features]
fancy-feature = ["foo", "bar"]

it seems it is somehow tied to "features" that one can enable somehow, but I don't yet understand the full mechanism. Also there must be some way to propagate this "feature" on/off status inside the code using some macros or something.

In Fortran, I can see at least two ways to implemented it, are there more?

1. We can define some pre-processor definitions and use #ifdef to enable certain code if an optional dependency is used. We can support multiple pre-processors (cpp, fypp, ...).

2. One can do it at the module level: I sometimes have two files, say, openmp.f90 and openmp.dum.f90 both of which implement the openmp module, so the rest of the Fortran code just use openmp no matter what, and only one of the two files is compiled and linked in the buildsystem:

   # OPENMP
   if(WITH_OPENMP)
       set(SRC ${SRC}
           openmp.f90
       )
   else()
       set(SRC ${SRC}
           openmp.dum.f90
       )
   endif()

The advantage of 2. is that you do not have to use any pre-processor, which I try to avoid in my codes. The advantage of 1. is that it's simpler in some ways, you just put a few ifdefs in your code.
I think fpm can support either one, or both.

fpm could for example create some module, say optional_dependencies and export some variable or a function such as openmp_enabled for the "openmp" feature, that you can call in your code and make some decisions.

Either way, we should figure out how to make fpm support optional dependencies and features that the user can configure.

Examples

Example 1

A typical example is a large electronic structure code, that provides its own default exchange correlation functional, but optionally allows to link against the libxc library, in which case one must enable and link against it and some code paths are different (typically some Fortran modules are enabled / disabled) and it allows the code to use functionals from the libxc library.

Example 2

There are many linear and eigensolver libraries, and there is typically some default, but if the user installs a particular 3rd party solver, it can optionally enable it in the Fortran program to use it instead.