ck tests fail if default gcc/ck_pr.h is used on MIPS64 about ck HOT 21 CLOSED

Interesting. Any chance we can get objdump -d output of the failed tests?

from ck.

bytelock - validate
https://pastee.org/36m6w

fifo - ck_fifo_spsc
https://pastee.org/sjubc

ring - ck_ring_spsc
https://pastee.org/xq65y

sequence - ck_sequence
https://pastee.org/rbkhj

Regards,
Jurica

from ck.

jurica, any way we could get access to a MIPS64 box? Perhaps we'll also port CK to it.

from ck.

Sorry,

I have asked for MIPS64 box access.
I was told that there is no publicly available MIPS64 box right now.
If it does help, I can test changes (debug) on box that I have access to and send you a report.

There's also QEMU (which will work with 64-bit as well):
http://mipsdebian.imgtec.com/debian/qemu/

If it helps maybe we could get access to one CI20 board (MIPS32 little endian board)?
I will ask if there is CI20 access if you think that this could help package build on MIPS.

Regards,
Jurica

from ck.

I tried to use qemu to reproduce this, but was unsuccessful in getting linux booting on an SMP VM, neither 32 nor 64bits, and the problem won't happen on UP, of course. Can you reproduce the problem on the 32bits board as well ?

Regards,

Olivier

from ck.

There was failiures on MIPS32 as well.
I will check again and post results here.

There is additional problem on MIPS32.
MIPS32 does not support 8B __sync operations.
I have to use libatomic to resolve this.

Regards,
Jurica

from ck.

I don't know MIPS well, but it seems it doesn't support 64BITS ll/sc for 32bits systems, so we're kinda out of luck here. I don't think libatomic is an option for us, as it seems to use pthread for what is not supported natively by gcc.

Regards,

Olivier

from ck.

Yes, you are right.
MIPS32 does not have lld/scd implemented.
I was looking at ck_f_pr.h and ck_pr.h
Could we avoid using 8B operations on MIPS32?

Regards,
Jurica

from ck.

Sure, we just have to provide a separate ck_f_pr.h for it, without the various _64 defines.

Regards,

Olivier

from ck.

Hello,

I did rerun package on MIPS32 (on same box).
More tests fail on MIPS32.

I would like to recheck results on other build machine.
I will be back with results.

Regards,
Jurica

from ck.

The GCC fallback relies on GCC to do the right thing juricast. I think the best route here is to develop a native port to MIPS, which should be fairly straight-forward. As far as MIPS64 is concerned, we still need to do further investigation. However, we still don't have access to a multicore MIPS box.

MIPS32 failures are expected if GCC is not emitting the right sync instruction on there.

I didn't see anything obvious in the disassembly you gave me, for MIPS64. Could you compile the tests with -O0 and try again and give us disassembly of ck_sequence assuming it still fails?

from ck.

I have excluded functionality that require 8B __sync for MIPS32.
Package still fail.
Interesting is that on different MIPS box package fails but less tests fail.
That box has less cores and does not have FPU.
And I was able to once completely build package without failure on loongson box.

I need to look at this more, and I will provide disassembly from tests compiled with -O0 shortly.

I suppose that you are right and it would be nice to have MIPS port for this package.

Regards,
Jurica

from ck.

@juricast - could you also paste us your include/ck_md.h file (post-configure) for both MIPS32 and MIPS64?

from ck.

Hello,

I have tried to build upstream source on different MIPS boards, with -O0 and with -O2.

On Loongson based board with 4 threads package builds successfully on MIPS32 and MIPS64.
For MIPS32 I have removed _64 defines from gcc/ck_pr.h and gcc/ck_f_pr.h

On box with 16 threads (NetLogic-XLP) results are inconsistent.
Make check takes a lot of time to execute.
If there are no other tasks on this box package sometimes execute tests successfully.
When package build fails, if I independently run tests that have failed previously, they execute successfully.

include/ck_md.h for MIPS32:

#ifndef CK_MD_H
#define CK_MD_H
#ifndef CK_MD_CACHELINE
#define CK_MD_CACHELINE (64)
#endif
#ifndef CK_MD_PAGESIZE
#define CK_MD_PAGESIZE (4096)
#endif
#ifndef CK_MD_RTM_DISABLE
#define CK_MD_RTM_DISABLE
#endif /* CK_MD_RTM_DISABLE */
#ifndef CK_MD_POINTER_PACK_DISABLE
#define CK_MD_POINTER_PACK_DISABLE
#endif /* CK_MD_POINTER_PACK_DISABLE */
#ifndef CK_MD_VMA_BITS_UNKNOWN
#define CK_MD_VMA_BITS_UNKNOWN
#endif /* CK_MD_VMA_BITS_UNKNOWN */
#ifndef CK_MD_TSO
#define CK_MD_TSO
#endif /* CK_MD_TSO */
#endif /* CK_MD_H */

include/ck_md.h for MIPS64:

#ifndef CK_MD_H
#define CK_MD_H
#ifndef CK_MD_CACHELINE
#define CK_MD_CACHELINE (64)
#endif
#ifndef CK_MD_PAGESIZE
#define CK_MD_PAGESIZE (4096)
#endif
#ifndef CK_MD_RTM_DISABLE
#define CK_MD_RTM_DISABLE
#endif /* CK_MD_RTM_DISABLE */
#ifndef CK_MD_POINTER_PACK_DISABLE
#define CK_MD_POINTER_PACK_DISABLE
#endif /* CK_MD_POINTER_PACK_DISABLE */
#ifndef CK_MD_VMA_BITS_UNKNOWN 
#define CK_MD_VMA_BITS_UNKNOWN 
#endif /* CK_MD_VMA_BITS_UNKNOWN */
#ifndef CK_MD_TSO
#define CK_MD_TSO
#endif /* CK_MD_TSO */
#endif /* CK_MD_H */

Regards,
Jurica

from ck.

Ok, this is likely your problem. Try changing the CK_MD_TSO to CK_MD_RMO (don't re-run configure) and then try make clean. Does it pass now? Alternatively, pass --memory-model=rmo to configure.

from ck.

@juricast - Have you had a chance to test this yet?

from ck.

Yes I did.
Sorry for taking so long.

I still have problems for MIPS32 on box with 16 threads.
For MIPS64 it seams to be okay (with --memory-model=rmo and -O0).

Thank you!

Regards,
Jurica

from ck.

Terrific. -O1 should work as well. How many cores are on the MIPS32 box? When you say problem, do you mean tests take forever to run / appear to hang or failures? The 8b/4b sync isn't addressed yet.

from ck.

MIPS32 16 threads is failing following tests now (with --memory-model=rmo and -O0):

----[ Testing epoch....
make[2]: Entering directory '/build/git/ck/regressions/ck_epoch/validate'
./ck_stack 16 1
Makefile:9: recipe for target 'check' failed
make[2]: *** [check] Killed
make[2]: Leaving directory '/build/git/ck/regressions/ck_epoch/validate'
----[ Testing hp....
make[2]: Entering directory '/build/git/ck/regressions/ck_hp/validate'
./serial
Free 0x7f0018
Free 0x7f0018
Free 0x7f0018
Allocating entry and freeing in other HP record...
Free 0x7f0018
Free 0x7f0040
./ck_hp_stack 16 100 1
Makefile:8: recipe for target 'check' failed
make[2]: *** [check] Killed
make[2]: Leaving directory '/build/git/ck/regressions/ck_hp/validate'

I will look into it again.
For MIPS32 I have just removed _64 defines from gcc/ck_pr.h and gcc/ck_f_pr.h

from ck.

It seems like OOM killer is kicking in here.

from ck.

CK_MD_RMO should have been used, folks can specify memory model with --memory-model=X.

from ck.