FindADIOS.cmake about adios HOT 25 CLOSED

I use autotools and not CMake, but I use adios_config. Instead of guessing dependencies, have you tried using that?

adios_config -c returns compiler flags. adios_config -l returns linker flags.

from adios.

That doesn't help. I'm not guessing dependencies, nor am I asking for that. adios_config is a command-line tool, which unless it integrates with CMake, is essentially the same thing as what I am already doing.

from adios.

The ideal thing would be akin to how VTK detection works with CMake: FindVTK.cmake looks for VTK_DIR, and when it is found, provides a variable USE_VTK_FILE, which can be included, and then provides conventional CMake variables for specifying libraries, include paths, etc.

find_package(ADIOS REQUIRED)
include (${USE_ADIOS_FILE})
...
target_link_libraries(myApplication ${ADIOS_LIBRARIES})

from adios.

That doesn't help. I'm not guessing dependencies, nor am I asking for that. adios_config is a command-line tool, which unless it integrates with CMake, is essentially the same thing as what I am already doing.

You misunderstand, I am not telling you that's the solution, just that it's a more robust method of doing what you're already doing. Your example has mxml, but that's not always the only thing that could be necessary to link with ADIOS.

My CMake is a little rusty, but I'm thinking something like this. Maybe you can fix it up and the ADIOS developers can include it.

find_program(ADIOS_CONFIG_EXECUTABLE NAMES adios_config)
execute_process(COMMAND ${ADIOS_CONFIG_EXECUTABLE} -i RESULT_VARIABLE ADIOS_CFLAGS)
execute_process(COMMAND ${ADIOS_CONFIG_EXECUTABLE} -l RESULT_VARIABLE ADIOS_LIBS)
execute_process(COMMAND ${ADIOS_CONFIG_EXECUTABLE} -i -f RESULT_VARIABLE ADIOS_FCFLAGS)
execute_process(COMMAND ${ADIOS_CONFIG_EXECUTABLE} -l -f RESULT_VARIABLE ADIOS_FLIBS)

from adios.

@andrewcorrigan We run into the same problem that we required a certain component X but adios was compiled with additional support for component Y which pulls in further dependencies.

So I just started writing a module today which is still far from being perfect.

Components support is still missing, but if you are interested in a first draft check out ComputationalRadiationPhysics/picongpu#271

Does basically the same: requires mxml and pulls in the dataspace/staging dependencies if they are compiled in.
Still missing components from the ADIOS Manual:

• lustre api
• serial hdf5 / parallel hdf5
• serial net-cdf / net-cdf4
• fortran90 (nej ;) )
• python (not rly a dependency to compile)
• mpi (used by our project anyway so far but should be pulled to)

from adios.

Does basically the same: requires mxml and pulls in the dataspace/staging dependencies if they are compiled in.

Does adios_config not provide the correct paths for compiling mxml, dataspace, etc.? From my reading of configure.ac, it appears that it is supposed to do so.

from adios.

Hi,

adios_config was created for apps that use automake or manual makefiles. We
are going to generate the same information in a format suitable for apps
that use cmake.

Norbert

On Wed, Mar 12, 2014 at 3:34 PM, Elliott Sales de Andrade <
[email protected]> wrote:

Does basically the same: requires mxml and pulls in the dataspace/staging
dependencies if they are compiled in.

Does adios_config not provide the correct paths for compiling mxml,
dataspace, etc.? From my reading of configure.ac, it appears that it is
supposed to do so.

Reply to this email directly or view it on GitHubhttps://github.com//issues/7#issuecomment-37453675
.

from adios.

@ax3l I will have to take a look at that. I've been distracted with non-ADIOS related endeavors over the past few months, but will give it a shot when I get back into it. Thanks for sharing.

@pnorbert I'm looking forward to official CMake support!

from adios.

As @pnorbert said, adios_config provides all one needs to compile and link.

The only thing is that this is a bit cumbersome to parse in CMake scripts. Imho a CMake module could depend on the information provided by that tool.

from adios.

As @pnorbert said, adios_config provides all one needs to compile and link.

That's why I'm a bit confused why the FindADIOS.cmake in your pull request checks for mxml explicilty?

from adios.

Because I am not using autotools - I have to link to libmxml.a myself since adios pulls in this dependency for the final executables.

But you are right, I am writing right now on a general regex that parses the provided libs from adios_config to absolute paths to avoid exactly that.

from adios.

Because I am not using autotools - I have to link to libmxml.a myself since adios pulls in this dependency for the final executables.

This autotools vs. cmake thing is a red herring. You do the same thing with autotools: call adios_config, and link with the libraries it says to use.

But you are right, I am writing right now on a general regex that parses the provided libs from adios_config to absolute paths to avoid exactly that.

Why do you need to know this? Why can't you just link with everything output from adios_config?

from adios.

Elliott,

Can you do this with cmake? Can you give us an example?

You seem to know something no one else does and therefore everybody tries
to come up with a complicated cmake macro to find the dependencies (not
just Axel).

Thanks
Norbert

On Wed, Mar 12, 2014 at 5:12 PM, Elliott Sales de Andrade <
[email protected]> wrote:

Because I am not using autotools - I have to link to libmxml.a myself
since adios pulls in this dependency for the final executables.

This autotools vs. cmake thing is a red herring. You do the same thing
with autotools: call adios_config, and link with the libraries it says to
use.

But you are right, I am writing right now on a general regex that parses
the provided libs from adios_config to absolute paths to avoid exactly that.

Why do you need to know this? Why can't you just link with everything
output from adios_config?

Reply to this email directly or view it on GitHubhttps://github.com//issues/7#issuecomment-37465950
.

from adios.

@pnorbert: I gave an example in an earlier comment. You then add the particular *CFLAGS and *LIBS to the compiler and link settings as usual. I did not test it out, so there may have been some small syntax errors, but I don't see why it wouldn't work out fine. I was not as confident with CMake then, so I can try to write something more substantial up now.

from adios.

@QuLogic it is not that easy since CMake splits libraries and library names from their pathes. There is simply no such thing as "shoot this long string to your linker".

Your execute_process example is just the beginning, one needs some parsing...

from adios.

@QuLogic it is not that easy since CMake splits libraries and library names from their pathes. There is simply no such thing as "shoot this long string to your linker".

Yes, you are correct, of course. Sometimes I forget cmake's bizarreness. But I'm not sure the parsing needs to be very complicated (i.e., don't think a regex is necessary.) Let me test some ideas out.

from adios.

I rewrote the script to parse the output from adios_config and to create cmake-like absolute links per library.

The absolute path is derived from the -L and then the default paths, prefering static over shared libs right now.

The output, e.g. for enabled DataSpace transports (I will remove the Found something lines later on):

-- ADIOS linker flags (unparsed): -L/sw/xk6/adios/1.6.0/cle4.0_gnu4.7.2/lib -ladios -L/sw/xk6/mxml/2.6/cle4.0_gnu4.5.3/lib -lmxml -L/sw/xk6/dataspaces/1.3.0/cle4.0_gnu4.7.2/lib -L/sw/xk6/dataspaces/1.3.0/cle4.0_gnu4.7.2/lib -L/opt/cray/pmi/default/lib64 -L/opt/cray/ugni/default/lib64 -lm -lmxml -ldspaces -ldscommon -ldart -ldspaces -ldscommon -ldart -lpmi -lugni
-- Found libadios in /sw/xk6/adios/1.6.0/cle4.0_gnu4.7.2/lib/
-- Found libmxml in /sw/xk6/mxml/2.6/cle4.0_gnu4.5.3/lib/
-- Found libm in /usr/lib/
-- Found libmxml in /sw/xk6/mxml/2.6/cle4.0_gnu4.5.3/lib/
-- Found libdspaces in /sw/xk6/dataspaces/1.3.0/cle4.0_gnu4.7.2/lib/
-- Found libdscommon in /sw/xk6/dataspaces/1.3.0/cle4.0_gnu4.7.2/lib/
-- Found libdart in /sw/xk6/dataspaces/1.3.0/cle4.0_gnu4.7.2/lib/
-- Found libdspaces in /sw/xk6/dataspaces/1.3.0/cle4.0_gnu4.7.2/lib/
-- Found libdscommon in /sw/xk6/dataspaces/1.3.0/cle4.0_gnu4.7.2/lib/
-- Found libdart in /sw/xk6/dataspaces/1.3.0/cle4.0_gnu4.7.2/lib/
-- Found libpmi in /opt/cray/pmi/default/lib64/
-- Found libugni in /opt/cray/ugni/default/lib64/
-- Found ADIOS: /sw/xk6/adios/1.6.0/cle4.0_gnu4.7.2/lib/libadios.a;/sw/xk6/mxml/2.6/cle4.0_gnu4.5.3/lib/libmxml.a;/usr/lib/libm.a;/sw/xk6/mxml/2.6/cle4.0_gnu4.5.3/lib/libmxml.a;/sw/xk6/dataspaces/1.3.0/cle4.0_gnu4.7.2/lib/libdspaces.a;/sw/xk6/dataspaces/1.3.0/cle4.0_gnu4.7.2/lib/libdscommon.a;/sw/xk6/dataspaces/1.3.0/cle4.0_gnu4.7.2/lib/libdart.a;/sw/xk6/dataspaces/1.3.0/cle4.0_gnu4.7.2/lib/libdspaces.a;/sw/xk6/dataspaces/1.3.0/cle4.0_gnu4.7.2/lib/libdscommon.a;/sw/xk6/dataspaces/1.3.0/cle4.0_gnu4.7.2/lib/libdart.a;/opt/cray/pmi/default/lib64/libpmi.a;/opt/cray/ugni/default/lib64/libugni.a (found suitable version "1.6.0", minimum required is "1.6.0")

And one can use it like this:

# ...
# you can provide a hint with the environment variable ADIOS_ROOT
# set(ADIOS_USE_STATIC_LIBS ON) # if you want to force static linking
find_package(ADIOS 1.6.0)

if(ADIOS_FOUND)
    include_directories(SYSTEM ${ADIOS_INCLUDE_DIRS})
    set(LIBS ${LIBS} ${ADIOS_LIBRARIES})
endif(ADIOS_FOUND)
# ...

To support cmake components one would have to write a lookup table library name <-> component name.

from adios.

@ax3l: Good job! I barely got home; if only all our problems could be solved while we were in transit... There are a couple of minor things though; I will comment on the PR there.

from adios.

All right, just updated to rely totally on the paths from adios_config (even for "weird" non-default include/ and lib/lib64 install dirs).

The only thing one has to set now is: either provide a hint where to find adios_config with export ADIOS_ROOT=/your/install (the standard cmake style environment variable) or simply set ADIOS' bin/ dir in your PATH.

from adios.

@pnorbert are you interested in a pull request for the final script?

I would change to a version so it fits your license (3-clause BSD) and push it over. We would skip the additional notice, hope you guys are fine with that since we are not UT-BATTELLE employees.

As the second author of the script: @f-schmitt-zih do you agree to fork a BSD version of our script?

from adios.

yes, I agree

from adios.

Hi,

Thanks for this work. We are fine with the BSD version and your names will be on it. If you don't mind, could you give us the cmake script such that we can push it to the repo and add more dependencies to it?

Thanks,
Kimmy

From: Axel Huebl <[email protected]mailto:[email protected]>
Reply-To: ornladios/ADIOS <[email protected]mailto:[email protected]>
Date: Friday, March 14, 2014 8:29 AM
To: ornladios/ADIOS <[email protected]mailto:[email protected]>
Subject: Re: [ADIOS] FindADIOS.cmake (#7)

@pnorberthttps://github.com/pnorbert are you interested in a pull request for the final script?

I would change to a version so it fits your license (3-clause BSD) and push it over. We would skip the additional notice, hope you guys are fine with that since we are not UT-BATTELLE employees.

As the second author of the script: @f-schmitt-zihhttps://github.com/f-schmitt-zih do you agree to fork a BSD version of our script?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/7#issuecomment-37641989.

from adios.

Hi @jyamu,

that sounds great!

I will just use the "usual Git(Hub) workflow" and open a pull request to your master with the modified version.

You will see the public branch that I propose in the pull request, so feel free to branch from that again. That's simple and clean, else we will loose track of different versions.

Edit: I am not sure if you need to extend the script. As long as adios_config gives out the information about additional components, the script will find it.

from adios.

Hi,

We are going to have a ADIOS release in five weeks. Just wondering if everyone is happy with FindADIOS.cmake module now. If there is any problem please let us know.

The module is located https://github.com/ornladios/ADIOS/blob/master/FindADIOS.cmake

Thanks you,
Kimmy

from adios.

@jyamu I just added a slight modification today that handles the tailing white space / new lines without the need of a regex.

Might be useful for you, too - shall I open a pull request for it? Edit: See #21
Example diff in picongpu.

from adios.