This might also be something for EMC Engineering to work on. I don't have a user from EMC Engineering to which to assign these items at this time, however.

from plfs-core.

I re-ran the experiment with N-N and N-1 POSIX, with and without PLFS. Only N-1 POSIX with PLFS fails.

With 1 pe it succeeds. Here's the command:

Running aprun -n 1 -N 1 -N 1 /users/brettk/code/ionetworking/test_fs/src/fs_test.smog.x -strided 1 -nobj 512 -sync -tmpdirname /users/brettk/tmp -io posix -touch 3 -check 3 -size 4M -target /plfs/scratch1/brettk/n1-posix/out.%s -shift -experiment N1_POSIX_PLFS_Testing.1325794110 -barriers aopen -hints panfs_concurrent_write=1 -type 2

With 2 pes it fails. Here's the command and output:

Running aprun -n 2 -N 2 -N 2 /users/brettk/code/ionetworking/test_fs/src/fs_test.smog.x -strided 1 -nobj 512 -sync -tmpdirname /users/brettk/tmp -io posix -touch 3 -check 3 -size 4M -target /plfs/scratch2/brettk/n1-posix/out.%s -shift -experiment N1_POSIX_PLFS_Testing.1325794807 -barriers aopen -hints panfs_concurrent_write=1 -type 2

pwrite to 4194304 error: Input/output error
Rank 1 Host nid00003 FATAL ERROR 1325794811: read_write_buf:1049 write io posix, ret -1, offset 4194304, obj_size 4194304 (errno=Input/output error) (MPI_Error = -1)
[RANK 1] Waiting 60secs
Rank [1] DEBUG: Query in /users/brettk/smog_db_up needs to be uploaded
FAILED
Rank 1 [Thu Jan 5 13:21:11 2012] [c0-0c0s1n1] application called MPI_Abort(MPI_COMM_WORLD, -1) - process 1
_pmii_daemon(SIGCHLD): [NID 00003] [c0-0c0s1n1] [Thu Jan 5 13:21:12 2012] PE 1 exit signal Aborted
[NID 00003] 2012-01-05 13:21:12 Apid 43537: initiated application termination
Application 43537 exit codes: 134

from plfs-core.

This might be fixed in trunk. Is there a reproducer in our regression testing?

from plfs-core.

In my comment directly above John's is the command that reproduces this error.

Running aprun -n 2 -N 2 -N 2 /users/brettk/code/ionetworking/test_fs/src/fs_test.smog.x -strided 1 -nobj 512 -sync -tmpdirname /users/brettk/tmp -io posix -touch 3 -check 3 -size 4M -target /plfs/scratch2/brettk/n1-posix/out.%s -shift -experiment N1_POSIX_PLFS_Testing.1325794807 -barriers aopen -hints panfs_concurrent_write=1 -type 2

Plug-in your own /plfs... target directory.

from plfs-core.

This is a bug that needs some attention. Users may choose not to use this mode, but while they convert to use MPI/IO they still may use this. It should work for completeness.

from plfs-core.

The issue appears to be fixed. More testing is needed, but all the previous reproducers appear to work correctly now.

from plfs-core.