MPI tag limit about nek5000 HOT 8 OPEN

yes, however the code checks the maximum tag ID with MPI_TAG_UB.

from nek5000.

Sure, but this check will not make us compliant. Cray typically support up to 2**21.

from nek5000.

You specify lp in SIZE, nid will run from 1 ... #MPI-ranks < lp. You are find if lp<MPI_TAG_UB but what if you want to run on more MPI ranks.

On 16.08.2016, at 19:52, Ron Rahaman [email protected] wrote:

Isn't NID declared in SIZE? If so, isn't it up to the user to make sure that the type of NID can accommodate however many ranks they want?

(Sorry for jumping in; I just saw this as a learning opportunity)

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

from nek5000.

Then isn't that the fault of the MPI implementation? We get MPI_TAG_UB from mpi_get_attr. If MPI_TAG_UB doesn't accommodate the maximum ranks on the machine, then it's not the right implementation for that machine. Then you send a nasty email to your sysadmin.

from nek5000.

No! The MPI standard says MPI_TAG_UB has to be larger than 32768 (see above).

On 16.08.2016, at 19:59, Ron Rahaman [email protected] wrote:

Then isn't that the fault of the MPI implementation? We get MPI_TAG_UB from mpi_get_attr. If MPI_TAG_UB doesn't accommodate the maximum ranks on the machine, then it's not the right implementation for that machine. Then you send a nasty email to your sysadmin.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

from nek5000.

Indeed, as @stgeke says we can only be sure to have 32768 tags, however most MPI implementations had many more in the past.
So I agree this is something one has to change at some point, but so far we could get around it as we still had 2**21=2M tags on the latest Cray. However, chances are that one gets a few bits more in the tag in the future. But sure, it is not standard compliant.

from nek5000.

How about your stuff @jlottes?
It looks like you are using the MPI rank to tag messages.

from nek5000.

Yea, the MPI code in jl/ could probably use a re-thinking in how it does tags. I don't think I gave this too much thought at the time, and probably just used the MPI rank because it seemed as good a thing to choose as anything else. In retrospect, I don't think using the rank buys anything. It shouldn't be too hard to fix things.

from nek5000.