fortran-lang / fpm Goto Github PK
View Code? Open in Web Editor NEWFortran Package Manager (fpm)
Home Page: https://fpm.fortran-lang.org
License: MIT License
Fortran Package Manager (fpm)
Home Page: https://fpm.fortran-lang.org
License: MIT License
This is related to #69 but it's a separate issue. fpm
should be able to create a Conda package for the project, so that it's easy to submit it to conda-forge.
Except for small/simple projects, I almost always adopt Fortran submodules now to structure new projects. This is primarily for the parallel and incremental builds that they allow.
I realise this may be low down on the list of project priorities currently, but I think it should only require an update to the dependency tracking whereby submodules depend on their parent modules as if they use
them.
Much further down the line, I assume fpm
will aim for some kind of parallel build support?
After #34 is implemented, we can discuss how to implement a full package registry similar to https://crates.io. It would be an extension of #34, which would also include hosting of the tarballs, and collecting usage statistics and other things. I think the solution in #34 can be evolved into this.
This is more of a discussion than an issue.
As a developer who routinely develops in FORTRAN and C, I have come across Conan, http://conan.io a C/C++ package manager.
Now in my mind C and FORTRAN are very similar in character ... both have attributes such as:
Etc.
Conan manages these very well via a hash assigned to the package.
Now I tried about two years ago to implement a similar scheme, to what Conan was doing but finally said ... they just do it better.
Today I use VS Code, CMake/Ninja and Conan as my "best practice". ( BTW, the new Ninja version 10. native support for FORTRAN modules ) This set up supports Intel FORTRAN/GFORTRAN and hopefully FLANG soon
So I am curious what do you think you bring new to the table.
Please don't think I am just throwing stones ... I have hundreds of thousands of lines of FORTRAN code, and Conan is bit of a "force fit", but not bad, ... so I truly am looking to improve my best practice.
I'm creating this as mostly a reference for how I think we should proceed with having sane defaults to allow minimal fpm.toml
files. Right now I'm just reading the toml contents directly into the data structure for the settings for building the project. What I think will make more sense is to have two different data structures. One for reading the settings from the fpm.toml
file, and a slightly different one for the settings for the project passed to the rest of the process. This will allow a step for interrogating the file system for some of the defaults based on the presence/absence of information in the fpm.toml
file.
I am very confident we can make fpm
very robust to work for pure Fortran packages. Just like Cargo works well for pure Rust packages or pip works great for pure Python packages.
The problem is with non-Fortran packages. Pip allows to hook in compiling C (and with some work) Fortran code, but it's very fragile in my experience (thus the motivation for Conda that is a binary package manager). Python doesn't have an option to avoid C++/Fortran dependencies because Python itself is slow. Rust allows to (in principle) rewrite everything in Rust. As an example, take png
. The system bindings: https://crates.io/crates/libpng-sys they say are unreliable, and you should use a pure Rust implementation: https://lib.rs/crates/lodepng.
I agree it does make things more robust to stick to pure Fortran and for many things we will do that and people will provide pure Fortran implementation of basic tasks. Python cannot do it well due to performance, but Rust and Fortran can.
However, we still need a robust way of handling non-Fortran dependencies, because if there is a robust and well maintained library in another language, we should just use it instead of reimplementing everything. Take HDF5. Here is the Rust package: https://crates.io/crates/hdf5. If you look at the documentation how to build it: https://github.com/aldanor/hdf5-rust they even mention Conda (to install the HDF5 library itself on all platforms --- which in my experience is much more robust than pip). Anyway, the way it works is by this line: https://github.com/aldanor/hdf5-rust/blob/15ec644977e0bee2b77340272730b34209c3765b/Cargo.toml#L11 which causes Cargo to execute this script: https://github.com/aldanor/hdf5-rust/blob/15ec644977e0bee2b77340272730b34209c3765b/build.rs which emits flags how to link against HDF5 correctly. That way the Cargo itself doesn't need to know almost anything, it just parses the output of this file. I think we should follow the same approach in fpm
. The flags in this case are emitted by: https://github.com/aldanor/hdf5-rust/blob/d4c3737772bec477739c75566a3d52d0a44f27ba/hdf5-sys/src/lib.rs#L65. I think this is when you link against hdf5
rust package in your own code. How to link against hdf5 library itself is done here I think: https://github.com/aldanor/hdf5-rust/blob/d4c3737772bec477739c75566a3d52d0a44f27ba/hdf5-sys/build.rs#L566, it's quite complicated unfortunately.
But it's clean from the Cargo side, it offloads the responsibility to the package itself. We can provide helpers that fpm packages can use to work with things like pkg-config
, cmake packages
, etc.
In Rust it looks like each package is on their own, so for example this HDF5 package has messy code for each platform, e.g., here: https://github.com/aldanor/hdf5-rust/blob/d4c3737772bec477739c75566a3d52d0a44f27ba/hdf5-sys/build.rs#L492, you can see they are checking brew, or Windows registry, etc.
The good news is that Fortran codes do not need many non-Fortran dependencies, and so doing what Rust does might work for us. What I've seen is that Fortran codes mostly need some of: Lapack, MPI, FFT, MKL, HDF5, JSON, NETCDF, HYPRE, ...
Of which MPI and Lapack being the most important. I think fpm
will have support for all Fortran compilers and I think it can have built-in support for MPI and Lapack also. One reason to special case MPI and Lapack is so that one can switch MPI implementaitons and Lapack implementations easily, and not have the Fortran packages hardcoded with a particular implementation.
With those out of the way, the rest can be done Cargo style, at least for now. Most other packages have just one implementation, so Fortran packages can just depend on a particular package (say Arpack, or Scalapack).
I've been trying to get fpm to work with some old-school packages like MINPACK: https://github.com/certik/minpack. These projects are generally just a bunch of fixed format ".f" files. For modern applications that rely on such legacy codes, an interface module is the first step towards safe function calls.
Trying to run fpm build
on @certik's MINPACK fails with errors akin to:
fpm build --release
# gfortran (for build/gfortran_release/minpack/rwupdt.o build/gfortran_release/minpack/rwupdt.mod)
# gfortran (for build/gfortran_release/minpack/dmchar.o build/gfortran_release/minpack/dmchar.mod)
# gfortran (for build/gfortran_release/minpack/lmpar.o build/gfortran_release/minpack/lmpar.mod)
# gfortran (for build/gfortran_release/minpack/hybrd.o build/gfortran_release/minpack/hybrd.mod)
fpm: Error when running Shake build system:
at want, called at src/Build.hs:205:11 in fpm-0.1.0.0-ExRRX9W2aflLvTBBChPpxh:Build
* Depends on: build/gfortran_release/minpack/libminpack.a
at need, called at src/Build.hs:203:13 in fpm-0.1.0.0-ExRRX9W2aflLvTBBChPpxh:Build
* Depends on: build/gfortran_release/minpack/rwupdt.o
at &%>, called at src/Build.hs:183:11 in fpm-0.1.0.0-ExRRX9W2aflLvTBBChPpxh:Build
* Depends on: build/gfortran_release/minpack/rwupdt.o build/gfortran_release/minpack/rwupdt.mod
at error, called at src/Development/Shake/Internal/Rules/Files.hs:245:13 in shake-0.18.5-44KSA7uQF2VObxzEvLYZx:Development.Shake.Internal.Rules.Files
* Raised the exception:
Error, &%> rule failed to produce 1 file (out of 2)
build/gfortran_release/minpack/rwupdt.o
build/gfortran_release/minpack/rwupdt.mod - MISSING
I understand that fpm is expecting to find a module file for each single ".f" file. Is it possible to somehow work around this? The farthest I could get it in the fpm source was to remove to the "-Wimplicit-interface" flag which was creating lots of warnings π .
Am I right to think fpm is currently most suitable for projects composed of multiple module files?
Just like in Cargo.
We should work on this after #25 is fixed.
After #33 is implemented, the next step is to implement a minimal metadata registry. Here is one way to do that:
[{
"name": "stdlib",
"versions": [
{"version": "0.3.4", "url": "https://github.com/fortran-lang/stdlib/archive/0.3.4.tar.gz"},
{"version": "0.3.5", "url": "https://github.com/fortran-lang/stdlib/archive/0.3.5.tar.gz"},
]
}, {
"name": "bspline",
"versions": [
{"version": "6.0.0", "url": "https://github.com/jacobwilliams/bspline-fortran/archive/6.0.0.tar.gz"},
{"version": "5.4.2", "url": "https://github.com/jacobwilliams/bspline-fortran/archive/5.4.2.tar.gz"},
]
}]
We will then have scripts that take this JSON file and download the actual metadata for each package version. So for example, to obtain the metadata for the package bspline
version 5.4.2
, it would download the tarball https://github.com/jacobwilliams/bspline-fortran/archive/5.4.2.tar.gz
, unpack and it would read its fpm.toml
, which would contain all the metadata such as short and long description, the list of dependencies, and other things. Then we can automatically create a website which would list all this metadata. This generated website would contain a generated file metadata.json
, which the fpm
tool can then download to obtain a searchable data base of packages (fpm search
).
To add a new package to registry, just a new simple entry must be made to the above JSON file by hand, say by issuing a PR against the repository.
Later we can automate things more, similarly to how conda-forge works (https://conda-forge.org/docs/maintainer/adding_pkgs.html), where to put a new package in, a PR is sent against https://github.com/conda-forge/staged-recipes/, where the CI checks initial quality and that the package builds, and then if it gets merged, the CI actually creates a new repository for the package etc. In our case, we could have a staging repository, and if a PR is merged, the CI would correctly update the above JSON file.
We can discuss if the JSON file should also contain all the metadata from fpm.toml
directly. The advantage of the above approach is that it is not redundant, the JSON only contains the minimal amount of information that can be edited and maintained by hand, and if you want more, you download the tarball and read its fpm.toml
, which will be done automatically in the step 2.
Overall, this minimal package registry only contains a minimal JSON file. The actual tarballs and metadata are hosted elsewhere. After this is well implemented and works, we can evolve it into a full package registry (#35).
fpm is currently making gfortran fail on warning by hardcoding -Werror
. Removing -Werror
from fpm.toml doesn't change this behavior (I understand specifying flags via fpm.toml may not be implemented yet.
However, in the meantime, we shouldn't use -Werror
because it's making correct code to fail. For example:
[milan@gary test_project_error]$ cat src/mylib.f90
real :: a, b
a = 2.
b = 2.
print *, a == b
end
[milan@gary test_project_error]$ cat fpm.toml
name = "package-name"
version = "0.1.0"
license = "BSD3"
author = "Author name here"
maintainer = "[email protected]"
copyright = "2020 Author name here"
dependencies = []
compiler = "gfortran"
devel-options = ["-g", "-Wall", "-Wextra", "-pedantic"]
release-options = ["-O3"]
[library]
source-dirs = "src"
[milan@gary test_project_error]$ fpm build
# gfortran (for build/debug/library/mylib.o build/debug/library/mylib.mod)
src/mylib.f90:4:8:
print *, a == b
1
Error: Equality comparison for REAL(4) at (1) [-Werror=compare-reals]
f951: all warnings being treated as errors
fpm-exe: Error when running Shake build system:
at want, called at src/Build.hs:188:11 in fpm-0.1.0.0-9zFE4ut013U9YSOSmXT3I3:Build
* Depends on: build/debug/library/package-name.a
at need, called at src/Build.hs:186:13 in fpm-0.1.0.0-9zFE4ut013U9YSOSmXT3I3:Build
* Depends on: build/debug/library/mylib.o
at &%>, called at src/Build.hs:166:11 in fpm-0.1.0.0-9zFE4ut013U9YSOSmXT3I3:Build
* Depends on: build/debug/library/mylib.o build/debug/library/mylib.mod
at cmd, called at src/Build.hs:179:19 in fpm-0.1.0.0-9zFE4ut013U9YSOSmXT3I3:Build
* Raised the exception:
Development.Shake.cmd, system command failed
Command line: gfortran -c -Jbuild/debug/library -Wall -Wextra -Wimplicit-interface -Werror -fPIC -fmax-errors=1 -g -fbounds-check -fcheck-array-temporaries -fbacktrace -o build/debug/library/mylib.o src/mylib.f90
Exit code: 1
Stderr:
src/mylib.f90:4:8:
print *, a == b
1
Error: Equality comparison for REAL(4) at (1) [-Werror=compare-reals]
f951: all warnings being treated as errors
What do you think?
We need a specification, or at least a start of one, for what should go into the package file and how it should be organized.
I'd suggest we start by answering the question "What does fpm need to know about a package in order to be able to build it?" in as much detail as possible and then trying to organize it logically.
Hi there
I started learning OpenMP couple weeks ago and would like to use it to parallelize and speed up fortran standard library codebase.
Eventually we need to have a central place for packages similar to crates.io.
But for now we will use a git repository (GitHub, GitLab and other places will work) as well as just url for a tarball. That way we don't need to host anything ourselves at first and can get the initial community and ecosystem of packages built up without worrying about security and other issues that will come with maintaining our own repository.
Currently, if you issue fpm run
in a library project without executable program, it does nothing.
It'd be more user friendly if we emitted a message to stderr, something like:
fpm: This package has no executable programs.
or similar.
Likewise for fpm test
. If there are no test, we should say something like:
fpm: This package has no tests.
What do you think?
How should we organize/specify compiler flags?
Should we only have two sets of compiler flags? devel-options
and release-options
, and they are used to build all dependencies and executables?
Also, should we come up with our own keywords for specifying the compiler options, and have a look up table to determine the correct flag for each compiler?
This issue is to ask whether fpm will have any built-in support for preprocessing and how this might look.
I bring this up since I noticed that stdlib
is listed in #17 under 'Pure Fortran', however it requires the fypp preprocessor to build from repo source.
We need to add more tests:
target
(in some temporary directory)This issue is similar to #70 but for Spack instead of Conda. Essentially for people that use Spack (myself included sometimes), it would be nice to just be able to spack install
any fpm
package.
To do that, fpm
should be able to take any package and produce a working Spack package out of it, that can then be submitted to Spack.
The generated Spack package would probably call fpm
underneath to do the build.
My basic outline of how to do this is the following recursive algorithm:
Line 38 in bf8ee01
The CMake syntax used in this Rust script requires at least CMake 3.13.
In general in the Fortran stdlib we currently planned to use CMake 3.14 minimum.
It would work just like cargo init
:
$ cargo init myproject1
Created binary (application) package
$ tree -a myproject1
myproject1
βββ Cargo.toml
βββ .git
βΒ Β βββ config
β ...
βββ .gitignore
βββ src
βββ main.rs
10 directories, 18 files
As you can see, it starts a git repository, and so on. It is ready to compile, so:
~$ cd myproject1
myproject1(master)$ cat src/main.rs
fn main() {
println!("Hello, world!");
}
myproject1(master)$ cargo run
Compiling myproject1 v0.1.0 (/tmp/myproject1)
Finished dev [unoptimized + debuginfo] target(s) in 1.25s
Running `target/debug/myproject1`
Hello, world!
If you just call cargo init
, then it creates a new project in the current directory. Similar to git init
.
It was just pointed out to me that fpm
is used by another project: https://github.com/jordansissel/fpm. Unfortunately it is in a similar field (also a package manager).
Here are some options going forward (I'll update this list if there are more):
fpm
fpm
and ensure that people do not mistake the two projects (what's the best way?)As to myself, I really like the name fpm
to mean a Fortran Package Manager. So I vote for 2., if there is a way to do it.
Naming of fpm.toml
. Cargo names Cargo.toml
with capital C
, and as explained in https://doc.rust-lang.org/cargo/faq.html#why-cargotoml, to "ensure that the manifest was grouped with other similar configuration files in directory listings. Sorting files often puts capital letters before lowercase letters, ensuring files like Makefile
and Cargo.toml
are placed together." If we want to do the same, the candidates are Fpm.toml
and FPM.toml
. I think fpm.toml
looks better. But using a capital letter would make it similar to CMakeLists.txt
also. We might want to devise a different name or naming scheme. Any ideas?
That is what Cargo does and what I also find intuitive. Very common when I want to just try something out in Fortran.
Currently the Haskell based fpm is compiling things directly, just like Cargo does it. That's the best default. (The "archived" Rust based fpm was generating CMake.)
However, until fpm
is everywhere used by everybody, it would be super helpful to have a CMake and pure Make backends. That way we can move packages like stdlib
or any of my own packages into using fpm
, and yet not require it from the users --- they would continue using CMake or Make.
Also by generating the CMake / Make automatically, it will be a lot easier to maintain those in stdlib
for example.
Since fpm
knows (or should know) everything about the Fortran project, it can generate 100% robust CMake and Make files, following the latest practices for CMake / Make, human readable, etc.
I am thinking having these backends would be beneficial for many purposes, such as debugging, anyway. We should also have a Ninja backend.
Are we sold on using toml as the file format for a package manifest/configuration file?
I see that Rust uses toml, but many other package mangers use others. Haskell's Stack uses yaml, and I think I've seen others use that. I think I've also seen json, and maybe even xml.
A new project Cargo.toml looks like:
[package]
name = "rusttest"
version = "0.1.0"
authors = ["Brad Richardson <[email protected]>"]
edition = "2018"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
and a new project package.yaml (for Haskell) looks like:
name: haskelltest
version: 0.1.0.0
github: "githubuser/haskelltest"
license: BSD3
author: "Author name here"
maintainer: "[email protected]"
copyright: "2020 Author name here"
extra-source-files:
- README.md
- ChangeLog.md
# Metadata used when publishing your package
# synopsis: Short description of your package
# category: Web
# To avoid duplicated efforts in documentation and dealing with the
# complications of embedding Haddock markup inside cabal files, it is
# common to point users to the README.md file.
description: Please see the README on GitHub at <https://github.com/githubuser/haskelltest#readme>
dependencies:
- base >= 4.7 && < 5
library:
source-dirs: src
executables:
haskelltest-exe:
main: Main.hs
source-dirs: app
ghc-options:
- -threaded
- -rtsopts
- -with-rtsopts=-N
dependencies:
- haskelltest
tests:
haskelltest-test:
main: Spec.hs
source-dirs: test
ghc-options:
- -threaded
- -rtsopts
- -with-rtsopts=-N
dependencies:
- haskelltest
I'm not strongly opposed to toml, but it's the one I'm least familiar with. My gut tells me we should go with yaml. I would not suggest json or xml though, too much syntax/line noise.
They mean the same, but they look and "feel" different. We should have a consistent way to refer to this so it's not confusing to the readers whether they're different things. Especially as we begin to write about these things more publicly?
I've been using both. I don't know which one I prefer. Perhaps 60/40 in favor of fpm.
So, which one should it be?
If I have a source file datetime.f90 that contains a module called datetime_module
, then gfortran (and all other compilers I believe) output datetime_module.mod. However, fpm expects datetime.mod and can't find it.
I believe the assumption about .mod filename comes from somewhere around here, however I don't have a solution right now.
I think the safe assumption is that the .mod file will have the module name, for fpm will need to scan the source file to get this info.
Further, a source file can have multiple modules defined, and for each the compiler will emit one .mod file.
I know it might be a bit early but I'd like to log this issue that is particularly important for me.
To use routinely fpm
, I would need a functionality that works a bit like sudo make install
to install the executable or library system-wide.
PS: thanks for your great work!
It is extremely common to have optional dependencies in Fortran projects (see the examples section at the end of this description).
The way Cargo handles it is described in here: https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html
[dependencies]
foo = { version = "1.0", optional = true }
bar = { version = "1.0", optional = true }
[features]
fancy-feature = ["foo", "bar"]
it seems it is somehow tied to "features" that one can enable somehow, but I don't yet understand the full mechanism. Also there must be some way to propagate this "feature" on/off status inside the code using some macros or something.
In Fortran, I can see at least two ways to implemented it, are there more?
We can define some pre-processor definitions and use #ifdef
to enable certain code if an optional dependency is used. We can support multiple pre-processors (cpp, fypp, ...).
One can do it at the module level: I sometimes have two files, say, openmp.f90 and openmp.dum.f90 both of which implement the openmp
module, so the rest of the Fortran code just use openmp
no matter what, and only one of the two files is compiled and linked in the buildsystem:
# OPENMP
if(WITH_OPENMP)
set(SRC ${SRC}
openmp.f90
)
else()
set(SRC ${SRC}
openmp.dum.f90
)
endif()
The advantage of 2. is that you do not have to use any pre-processor, which I try to avoid in my codes. The advantage of 1. is that it's simpler in some ways, you just put a few ifdefs in your code.
I think fpm
can support either one, or both.
fpm
could for example create some module, say optional_dependencies
and export some variable or a function such as openmp_enabled
for the "openmp" feature, that you can call in your code and make some decisions.
Either way, we should figure out how to make fpm
support optional dependencies and features that the user can configure.
A typical example is a large electronic structure code, that provides its own default exchange correlation functional, but optionally allows to link against the libxc library, in which case one must enable and link against it and some code paths are different (typically some Fortran modules are enabled / disabled) and it allows the code to use functionals from the libxc library.
There are many linear and eigensolver libraries, and there is typically some default, but if the user installs a particular 3rd party solver, it can optionally enable it in the Fortran program to use it instead.
Use case: I'd like to build my program or library that depends on FGSL with fpm
. I know how to specify such dependency (in theory, at least) and let's assume that there are no issues in fetching, compiling and linking.
The issue I'm raising here is: what if I want to make sure that the fetched and compiled dependency passes its tests on my machine before proceeding with linking and building my own program or library? How can I do that with fpm
?
I believe it's a non-trivial problem considering that each external dependency does not conform to fpm
specs. Off the top of my head: maybe an additional key in the TOML file can be specified by the user that specifies how to test the dependency (e.g. make test
)? However, I think that it is a sub-optimal solution because requires the user to gain this information for each dependency but it would be hard to have a general reliable method otherwise. Please correct me if I'm off-track.
There will be a lot of packages that do not have fpm.toml
. Here is my suggested approach how to handle that:
Encourage every package to use fpm.toml
and to use fpm
.
Those packages that do not use it yet could be handled as follows: we will maintain a version (fork) at GitHub or GitLab that includes the fpm.toml
. It will be this fork that would be used with fpm
. From fpm
's perspective, each package always contains fpm.toml
.
The alternative to (or modification of) 2. is to allow fpm.toml
to specify where to find sources of the actual package. So our GitLab package can be just one file fpm.toml
that would list the metadata and where to download the sources plus any patches to them.
In particular, here is my plan: I will start with forking the packages listed at #17 and implementing fpm.toml
together with any modifications that might be needed. I will not submit a PR back initially, but rather simply get my forks working well with fpm
, and test it all out and get some usage. Then, as things start to get more serious and the fpm
tool matures, we can easily send a PR against the upstream package and start the discussion with upstream authors if they would be willing to use fpm
and maintain fpm.toml
themselves. And depending on how this conversation goes, we'll either do just 2., or if we need to, we can also implement 3. in fpm
. I expect that upstream authors will give us a list of features that they need fpm
to have implemented, and once we do, they would be willing to use it.
Currently, FPM can build only one executable program, namely main.f90
.
Here's how Cargo does it:
.
βββ Cargo.lock
βββ Cargo.toml
βββ src/
β βββ lib.rs
β βββ main.rs
β βββ bin/
β βββ named-executable.rs
β βββ another-executable.rs
β βββ multi-file-executable/
β βββ main.rs
β βββ some_module.rs
βββ benches/
β βββ large-input.rs
β βββ multi-file-bench/
β βββ main.rs
β βββ bench_module.rs
βββ examples/
β βββ simple.rs
β βββ multi-file-example/
β βββ main.rs
β βββ ex_module.rs
βββ tests/
βββ some-integration-tests.rs
βββ multi-file-test/
βββ main.rs
βββ test_module.rs
So, for Fortran, the main executable could be src/main.f90
, and other executables could go in src/bin
.
I like this approach, and I am not married to this particular directory structure. I also like the structure that we have now, namely app/
, src/
, tests/
.
Here are a few good candidate packages to get working with fpm
first, that are relatively simple (and so possible to package soon), yet very useful.
And obviously any other more complicated package from https://github.com/fortran-lang/stdlib/wiki/List-of-popular-open-source-Fortran-projects.
Any other candidates?
I would do exactly what Cargo does: https://doc.rust-lang.org/cargo/reference/build-scripts.html
Only I would allow the build script to be any of: any binary, cmake, make, Bash script.
Otherwise the following I would do exactly the same:
Inputs: environment variables
Outputs: the build script prints to stdout with lines starting with fpm:
, everything else is ignored. At the beginning, I would support the following, and we can add more later:
fpm:fc-link-lib=[KIND=]NAME
--- Adds a library to link (doc)fpm:fc-flags=FLAGS
--- Passes certain flags to the compiler (doc).That should be enough to get us started. One of the environment variables passed to the build script is which Fortran compiler is being used, so the build script would know what compiler parameters to pass back (for example ifort
in general might require different flags than gfortran
). In the same way, the flags might depend on the platform (macOS vs Linux vs Windows), so one of the input variables can be what platform we run on.
How to support packages that do not conform to our "standard layout" (to be specified...). Some examples of such a package would be reference Lapack, or Arpack. The way to do that is that we create a new repository, say certik/lapack.fpm
, which will have fpm.toml
, in there it would specify the url to the actual sources (https://github.com/Reference-LAPACK/lapack) and a build script, which would build the sources (using CMake in this case) and install them into some $PREFIX provided by fpm
and fpm
takes it from there. This approach also works for non Fortran packages --- the build script either builds it, or requires it from the system (where it can be provided by, e.g., Spack). Either way this is a clean way to hook this up into the fpm
ecosystem.
Rust allows the packages to be decentralized, they do not have to be listed at https://crates.io/.
The Go language does not even have a centralized package registry at all, see this blog post that explains it in the section "Package Management":
https://nullprogram.com/blog/2020/01/21/
Modules are named by a module path that includes its network location. This means thereβs no land grab for popular module names.
I don't like the Go's model that the name of the package is the url, I prefer the Rust model where a package has a simple name, but you specify where it can be downloaded. The end result is the same. (Overall, the Rust package management seems much more thought out, and it seems they fixed the issues that Go is tackling long time ago.)
I want to implement the same approach for fpm. That will allow us to build an ecosystem of packages for Fortran, without a "goldrush" to reserve a popular package name in a centralized registry. Then, as we have a healthy ecosystem of packages, we can add a centralized registry later.
First time running stack. On running stack build
, it looks like the fpm executable is not installed in any working path. I expected it to be installed either somewhere under stack like pip does, or locally and then added to my path.
For example, I'm working in /home/milan/Work/fortran/fpm, fpm-exe is installed here: /home/milan/Work/fortran/fpm/.stack-work/install/x86_64-linux-tinfo6/68e6a00e61079ec9146947ae047a1ce619d26ace12c1e4cd5fd58ceb496743b0/8.6.5/bin
so I had to add this to my PATH
.
Is this expected behavior? If not, is this an issue with stack or with fpm?
fpm-exe should be called just fpm.
Put the CMakeLists.txt into target/, and ensure the source directory is not polluted at all.
As discussed in #33, let's allow installing fpm-enabled packages form their git repositories.
If the following is placed in fpm.toml:
[dependencies]
dependency_name = { git = git_url, tag = git_tag }
Then the package depends on dependency_name
, which fpm will fetch from git_url
and a specific git tag git_tag
.
Here's a concrete example that will work, when this issue is implemented.
[dependencies]
datetime = { git = "https://github.com/wavebitscientific/datetime-fortran", tag = "v1.7.0" }
Question: If tag
is omitted, should fpm fetch from the latest commit on master? I think so. Cargo does it like that as well. Then you could also do:
[dependencies]
datetime = { git = "https://github.com/wavebitscientific/datetime-fortran" }
Currently the new fpm does not seem to have any tests. We have to fix it, and start adding lots of tests.
In the old fpm, I had two initial tests:
https://github.com/fortran-lang/fpm/tree/master/archive/tests
And we need to add a lot more. The idea would be to test any configuration that is supported, and that way we can ensure that it works and things don't break when new functionality is implemented.
Furthermore, if we decide to use a different language for the production implementation, the tests will allow us to ensure that the new implementation works as expected.
cargo build
does not build tests. You have to do cargo build --tests
to also build tests.
cargo test
build tests (if needed) and runs them.
I suggest we follow the same approach.
Following the discussion at the monthly call, I've installed fpm and am trying to convert a few of my Fortran projects to the recommended structure. These projects involve some legacy fixed-style functions and subroutines, an interface module, and several driver programs (tests). The folder structure is something like:
.
βββ drivers
β βββ driver1.f90
| βββ driver2.f90
βββ src
β βββ legacy
| | βββ legacy1.f
| | βββ legacy2.f
| βββ interface.f90
βββ fpm.toml
Do I need to add multiple sections such as
[[driver]]
name = "driver1"
source-dir = "drivers"
main = "driver1.f90"
to the TOML file? Right now my build just crashes because of multiple executable files.
I think having multiple executable programs is a quite common type of package, say you have a library for building mesh triangulations, and then with it a small set of command line tools calling routines from this common library.
Another question is can I already specify dependencies on system-wide installed libraries. Say a project (or one of it's dependencies) needs to be linked with -llapack -lblas
(with the actual LAPACK and BLAS libraries potentially in some non-default location). Does this fit under issue #99 of custom build scripts?
I had a few other comments about the installation process, but they were already fixed in #101. A remaining point might be that after doing stack install
in the fpm folder, the ${HOME}/.local/bin/
folder might not be on the path already, requiring an extra command export PATH="$PATH:$HOME/.local/bin/"
.
We've been working with @everythingfunctional on the standardization of the layout. First iteration:
$ tree .
.
βββ fpm.toml
βββ src
βββ a.f90
βββ b
βΒ Β βββ utils.f90
βββ utils.f90
2 directories, 4 files
$ cat fpm.toml
Name "a"
$ cat src/a.f90
module a
use a_utils, only: util1
use a_b_utils, only: util2
implicit none
private
public util1, util2
end module
$ cat src/utils.f90
module a_utils
implicit none
private
public util1
integer :: util1
end module
$ cat src/b/utils.f90
module a_b_utils
implicit none
private
public util2
integer :: util2
end module
Following up on the discussion in #60, let's remove the requirement for the compiler
setting in the package-specific fpm.toml
.
For now, FPM should default to gfortran, and we can put a note in the README or the packaging guide that gfortran is currently the only compiler supported. We can relax this at a later time. The compiler
setting can still remain in the toml parser as optional--perhaps we will want to allow package-specific choice of compiler.
Currently compiled binaries are placed in build/debug/
and build/release
.
Would it be possible to make this build path dependent on compiler (and maybe architecture)?
e.g. build/gfortran-9.2.0-x86_64_debug/
This reason for this being that I like to build and test my projects on multiple compilers during development; separating the binary output paths allows incremental builds using multiple compilers and allows tests on each compiler to run simultaneously and without clean and rebuild.
This may be considered an 'ugly' directory structure, however since fpm handles all compiler/linker paths and offers the fpm run
command, this has no effect on end-user experience.
The workflow for using a non-default compiler could then be along the lines of:
$> export FC=ifort
$> fpm run
or
$> FC=ifort fpm run
or
$> fpm run --compiler=ifort
Now that fpm is in a usable state, I think we should try and create a docker image with the latest version installed, so people can use fpm in their CI.
There is a bug at our CI that I haven't been able to figure out yet. Here is an example of it: https://github.com/fortran-lang/fpm/runs/390475601.
Here is what I know:
It only happens on macOS, never on Linux or Windows
Restarting the build typically fixes it (sometimes it fails 2x or 3x in a row, but eventually it always passes)
The cargo test
runs in parallel by default, so I set -j1
to run in serial. That seemed to improve how often it fails (although I could be wrong on that). It still fails sometimes however, so the actual bug is still there.
The error is:
thread 'test_2' panicked at 'Unexpected failure.
code=<interrupted>
stderr=``````
command=`"/Users/runner/runners/2.163.1/work/fpm/fpm/target/x86_64-apple-darwin/debug/fpm" "build"`
code=<interrupted>
stdout=``````
stderr=``````
', /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/assert_cmd-0.10.2/src/assert.rs:148:17
Which is caused by the assert_cmd
package, which just uses the Rust's std::process::Command, when the output command did not succeed, but it also failed to retrieve the error code, which according to the documentation means the process was interrupted by the system (with some signal like SIGKILL).
Currently, FPM emits a static library file that has the same name as the project. For example, first line of fpm.toml
could be:
name = "datetime"
Then, the static library created is datetime.a
. However, the canonical way to name libraries (static and dynamic) is to use a lib
prefix: libdatetime.a
. Then one can link to this library with -ldatetime
.
Current workaround can be to add the lib
prefix to the package name in fpm.toml
:
name = "libdatetime"
However, this isn't pretty, and would down the road be more awkward once we're installing packages from remote URLs.
Let's add the lib
prefix to library files.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google β€οΈ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.