ssrg-vt / popcorn-compiler Goto Github PK
View Code? Open in Web Editor NEWPopcorn Linux compiler toolchain for heterogeneous-ISA execution
popcorn-compiler's Introduction
Popcorn Linux Compiler Toolchain, Copyright Systems Software Research Group at Virginia Tech, 2017. For more information, please visit http://popcornlinux.org or e-mail Rob Lyerly ([email protected]). -------- Overview -------- The goal of the heterogeneous compiler toolchain is to prepare multi-ISA binaries for migration through a series of analyses and transformations. We utilize and extend clang/LLVM in order to prepare heterogeneous binaries. We also use a python-based tool to prepare custom linker scripts to align program objects in the generated binaries. Finally, there are a number of additional libraries needed for migration and runtime state transformation. We need to prepare the binary so that before and after migrating between architectures, the application is able to find the required code and data to seamlessly continue execution. This is done by both using a common layout (where it is possible without significant performance overhead) and state transformation (where state is dictated by the ISA or where a common format would be too costly in performance). The toolchain operates as follows: 1. Parsing/LLVM bitcode generation (clang) - clang frontend, mostly unmodified 2. Middle-end analysis, refactoring & optimization - in addition to standard optimizations, run several passes which adjust the linkage of some variables, insert migration points into the application and notify architecture-specific backends to generate stack frame metadata 3. Backend - modified LLVM backend(s) generate custom data and function location information and compile bitcode to object-code 4. Linker - modified gold linker generates detailed data and function linking information, aligns thread-local storage for all architectures 5. Alignment - python tool uses information provided by gold linker to generate linker scripts that align data & function symbols across binaries 6. Metadata Generation - parse information generated by LLVM backends and add stack transformation metadata to binaries (must be run post-alignment as it requires final symbol layouts) In order to have functionally-identical implementations for all compiled source code, the binary must be compiled into a single IR representation, which is then used by each of the backends to generate architecture-specific code: ---------------- | Orig. Source | ---------------- | | (clang) V ---------------- | LLVM IR | ---------------- | | (opt) V ---------------- | Optimized IR | ---------------- | --------------------------------- (arch-specific backend) | | | V V V --------------- --------------- | aarch64 bin | ... | x86 bin | --------------- --------------- clang is set up to automatically implement this process -- generate a single set of instrumented IR and lower the IR to each target's machine code. The toolchain's installation folder is organized as shown below, along with a brief introduction about important sub-folders: root \ common - source/headers common between components in different folders | lib - libraries needed by the compiler and/or the compiled application \ libelf - library for parsing & reading ELF objects | libopenpop - Popcorn Distributed OpenMP library | migration - functionality for migrating between architectures | musl-1.1.18 - standard C library | stack_transformation - runtime for transforming stacks between ISA-specific layouts | patches - patches for compiler components \ binutils-gold - patch for GNU gold linker | llvm - patches clang/LLVM | tool - post-compilation binary tools for alignment & metadata-generation \ alignment - python tool for generating linker scripts to align binaries | stack_metadata - tools for post-processing binaries in preparation for stack transformation | util - various utilities, including a Makefile template, for heterogeneous applications \ scripts - scripts for patch generation, testing and running binaries See the README file in each subdirectory for more information, and the INSTALL file for installation instructions. ------------- Prerequisites ------------- Hardware requirements: It is highly recommended that the toolchain be built & installed on a machine with at least 4 cores and 8GB of RAM. This is because LLVM is a very large codebase and is built with debugging information by default, making the compile & linking phases taxing in terms of memory consumption. Software requirements: Note: the toolchain has been tested on x86-64 with Ubuntu 16.04 and up. It should work on other architectures and distributions (in particular, Debian 8), but may require installing alternate packages and some extra hacking. It is *highly* recommended that applications be built on x86-64. + gcc and g++ 4.8 or higher for x86-64, gcc 4.8 or higher for aarch64 - On Ubuntu 16.04 & higher, x86-64, install the following packages: build-essential gcc-aarch64-linux-gnu + flex, bison and texinfo (x86-64 only), needed by binutils - On Ubuntu 16.04 & higher, x86-64, install the following packages: flex bison texinfo (for makeinfo) + Python, v2.7 and v3 - v2 is needed for the installation script and the alignment tool - v3 is needed for scripts in util/scripts - Both are pre-installed on Ubuntu 16.04 and up + Cmake - On Ubuntu 16.04 & higher, x86-64, install the following packages: cmake + SVN - On Ubuntu 16.04 & higher, x86-64, install the following packages: subversion ----------- Limitations ----------- The current version of the toolchain only works for aarch64 + x86-64. Applications must be written in C (although anything for which LLVM IR can be generated should be supported), and inline assembly is not supported. All optimizations except auto-vectorization and frame-pointer elimination (required by stackmap intrinsic) are supported.
popcorn-compiler's People
Contributors
Stargazers
Watchers
Forkers
rppt systems-nuts elainejiang abarbala zildj1an yfzm moneytech moharaka xjtuwxg imcg dnigh vsn97 xrdevieee wolegeyun compor gtmshrm vishnukumarkalidasan trellixvulnteam themkotspopcorn-compiler's Issues
External scripts
External scripts that are called:
- mlink_x86Objs.sh and mlink_armObjs.sh: used for the final linking step calling gold to produce the multi-isa binaries
- nmScript.sh -> dump the symbols for vanilla/multi-isa binaries into external text files. They are called from the java code, but I doubt the output (text files) are actually used in the compiling process: when I disable the calls to this script, the compiling process succeeds without issue.
- readMyElfToFile.sh -> call to readelf -SW on vanilla/multi-isa binaries and output to external text files
Integrating the behavior of these scripts natively inside the alignment tool will make the compilation process simpler
GNU C-style destructors not called after migrating
I'm trying to add a couple of destructors using the __attribute__((destructor)) syntax to the migration library for my scheduling. They execute correctly when I compile an application with the Popcorn compiler and run it without migrating. However, if I migrate the application to a remote node (e.g., from x86 to ARM) first and complete execution there, the destructors are never called.
I'm testing on a QEMU setup using the latest versions of the kernel (api branch) and compiler (rack-api branch). I've checked the following situations (--> indicates migration):
SUCCESS (destructors called on program exit)
* x86-only
* x86 --> ARM --> x86
FAILURE (destructors not called on program exit)
* ARM-only (segfaults early due to unrelated error)
* x86 --> ARM
Based on my investigation, the call chain for exit in musl looks something like this:
__libc_start_main env/__libc_start_main.c:76
exit exit/exit.c:20
<destructors> exit/exit.c:27
I modified __libc_start_main to look like this:
int __libc_start_main(int (*main)(int,char **,char **), int argc, char **argv)
{
char **envp = argv+argc+1;
int ret;
__popcorn_stack_base = (void *)(argv - 1);
#ifndef SHARED
__init_libc(envp, argv[0]);
_init();
uintptr_t a = (uintptr_t)&__init_array_start;
for (; a<(uintptr_t)&__init_array_end; a+=sizeof(void(*)()))
(*(void (**)())a)();
#endif
/* Pass control to the application */
ret = main(argc, argv, envp);
// SHOULD SEGFAULT HERE!!
*(int *)ret;
exit(ret);
return 0;
}
That obvious segfault never occurs in the failing scenarios listed above -- so either I'm trying to rely on undefined behavior (entirely a possibility) or control never returns to this function from main (which makes no sense either). I'm not sure of how to continue debugging -- GDB doesn't seem to work on my setup, so I appreciate any advice.
Misaligned variables
Have some misaligned variables. Linker scripts are attached.
- Actually for testing the issue tracker in github
modified__aarch64.x.orig.txt
modified__aarch64.x.txt
modified__elf_x86_64.x.orig.txt
modified__elf_x86_64.x.txt
Segfault when compiling bzlib.c (bzip2 SPEC-2006)
-
Branch: Master
-
Commit: b6b7214
-
Code sample:
- C debug file (add .txt extension to github attach):
bzlib-a1f399.c.txt - sh debug file:
bzlib-a1f399.sh.txt
- C debug file (add .txt extension to github attach):
-
Behavior:
clang-3.7: error: unable to execute command: Segmentation fault
clang-3.7: error: clang frontend command failed due to signal (use -v to see invocation)
clang version 3.7.1 (tags/RELEASE_371/final 326328)
Target: x86_64-unknown-linux-gnu
Thread model: posix
clang-3.7: note: diagnostic msg: PLEASE submit a bug report to http://llvm.org/bugs/ and include the crash backtrace, preprocessed source, and associated run script.
clang-3.7: note: diagnostic msg:
********************
- Compiler output when adding the specified flags: same as without
Wrong linking for __clone
As I discussed this morning, the compiler/alignment tool mishandles __clone() so that it links to the last byte of the previous function in the executable (__unmapself()). Following are the logs
gdb run on aarch64
Program received signal SIGILL, Illegal instruction.
0x0000000000509298 in __unmapself ()
(gdb) bt
#0 0x0000000000509298 in __unmapself ()
#1 0x0000000000507778 in __pthread_create (res=0xa08000, attrp=0x0,
entry=, arg=)
at src/thread/pthread_create.c:272
#2 0x0000000000501310 in main (argc=1, argv=0xfffffffffcf8) at demo.c:238
(gdb)
in musl/src/thread/pthread_create.c
271: a_inc(&libc.threads_minus_1);
272: ret = __clone((c11 ? start_c11 : start), stack, flags, new, &new->tid, TP_ADJ(new), &new->tid);
273:
274: __release_ptc();
objdump aarch64 binary
50776c: aa1a03e5 mov x5, x26
507770: aa0403e6 mov x6, x4
507774: 940006c9 bl 509298 <__unmapself+0x10>
507778: 2a0003f4 mov w20, w0
50777c: 97fffe9e bl 5071f4 <__acquire_ptc>
507780: 34000075 cbz w21, 50778c <__pthread_create+0x3dc>
...
0000000000509288 <__unmapself>:
509288: d2801ae8 mov x8, #0xd7 // #215
50928c: d4000001 svc #0x0
509290: d2800ba8 mov x8, #0x5d // #93
509294: d4000001 svc #0x0
...
0000000000509299 <__clone>:
509299: 927cec21 and x1, x1, #0xfffffffffffffff0
50929d: a9bf0c20 stp x0, x3, [x1,#-16]!
libevent configure failed
Hi, When I try to compile libevent, it got error just at configure:
configure:3510: checking whether the C compiler works
configure:3532: /usr/local/popcorn/bin/clang -target aarch64-linux-gnu -popcorn-alignment conftest.c >&5
/usr/local/popcorn/bin/ld: internal error in global, at ../../gold/aarch64.cc:6228
clang-3.7: error: linker command failed with exit code 1 (use -v to see invocation)
configure:3536: $? = 1
configure:3574: result: no
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME ""
| #define PACKAGE_TARNAME ""
| #define PACKAGE_VERSION ""
| #define PACKAGE_STRING ""
| #define PACKAGE_BUGREPORT ""
| #define PACKAGE_URL ""
| #define PACKAGE "libevent"
| #define VERSION "2.0.21-stable"
| #define NUMERIC_VERSION 0x02001500
| /* end confdefs.h. */
|
| int
| main ()
| {
|
| ;
| return 0;
| }
configure:3579: error: in `/root/dir/libs/libevent-2.0.21-stable':
configure:3581: error: C compiler cannot create executables
See `config.log' for more details
## ---------------- ##How can I fix it?
I used this command:
./configure --prefix=/usr/local/popcorn/aarch64 --host=aarch64-linux-gnu --disable-shared CC=/usr/local/popcorn/bin/clang CFLAGS='-target aarch64-linux-gnu'number of records for stackmap ... doesn't match
-
Description: Compilation works but generated stackmaps don't match across binaries. Probably the reason why the migration fails.
-
Branch: master
-
Commit: b6b7214
-
Code Sample:
bzlib.c.txt -
Compilation Command:
/usr/local/popcorn/bin/clang -nostdinc -mllvm -debug -nostdlib -popcorn-migratable -fno-common -ftls-model=initial-exec -isystem /usr/local/popcorn/aarch64/include -c -o bzlib.o bzlib.c -
Output with "-mllvm -debug-only=regalloc -mllvm -debug-only=stacktransform":
bzlib.debug.txt
struct stat wrong value
Background
I'm trying to port a simple web server to popcorn. I compiled the tiny web server w/o inserting any migration point got wrong result.
Environment configuration
Popcorn-compiler: master (commit: 59ea7bb)
clang, musl-gcc w/ static link works correctly.
Analysis
When compiled w/ popcorn-compiler, the web server always returns "Unknown Error". This is because of the wrong sbuf.st_mode value in line 359.
Debug w/ GDB, popcorn-compiler get the struct stat sbuf with st_mode set 1:
(gdb) p sbuf
$13 = {st_dev = 2049, st_ino = 22551840, st_mode = 1, st_nlink = 0, st_uid = 33188, st_gid = 1000, st_rdev = 1000,
__pad = 0, st_size = 6, st_blksize = 4096, __pad2 = 0, st_blocks = 8, st_atim = {tv_sec = 1533149490,
tv_nsec = 705858674}, st_mtim = {tv_sec = 1532621795, tv_nsec = 412033015}, st_ctim = {tv_sec = 1532621795,
tv_nsec = 412033015}, __unused = {0, 0}}
The musl-gcc version binary show the correct st_mode is 33188 (10064 oct).
(gdb) p sbuf
$6 = {st_dev = 2049, st_ino = 22551840, st_nlink = 1, st_mode = 33188, st_uid = 1000, st_gid = 1000, __pad0 = 0,
st_rdev = 0, st_size = 6, st_blksize = 4096, st_blocks = 8, st_atim = {tv_sec = 1533149490, tv_nsec = 705858674},
st_mtim = {tv_sec = 1532621795, tv_nsec = 412033015}, st_ctim = {tv_sec = 1532621795, tv_nsec = 412033015}, __unused = {0,
0, 0}}
In /usr/include/x86_64-linux-gnu/bits/stat.h defines Regular File: #define __S_IFREG 0100000 /* Regular file. */ So fstat a regular file should return a 1xxxxx in oct, instead of 1 in our case.
Root cause
Rob mentioned for the libraries, we automatically use the aarch64 library. I checked the Linux uapi headers (should be same w/ glibc):
On ARM header (./include/uapi/asm-generic/stat.h):
struct stat {
unsigned long st_dev; /* Device. */
unsigned long st_ino; /* File serial number. */
unsigned int st_mode; /* File mode. */
unsigned int st_nlink; /* Link count. */
On x86 header (./arch/x86/include/uapi/asm/stat.h):
struct stat {
__kernel_ulong_t st_dev;
__kernel_ulong_t st_ino;
__kernel_ulong_t st_nlink;
unsigned int st_mode;
unsigned int st_uid;
unsigned int st_gid;
Consider the GDB info and these headers, this is problem the reason.
SEGFAULT: Cavium-Xeon heterogeneous execution
This is probably a regression fault. It happens when executing a simple example (see below) using the latest master branch commit: f9a26bf
The code is the following:
#define _GNU_SOURCE
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <errno.h>
#include <time.h>
#include <sys/syscall.h>
#include <sys/types.h>
#include <fcntl.h>
#include <string.h>
#include <migrate.h>
void migrated_callback(void *params)
{
char *buffer = params;
int fd;
fd = open("/tmp/bmigrate",
O_CREAT | O_WRONLY | O_APPEND, 0666);
write(fd, buffer, strlen(buffer));
close(fd);
return;
}
void* child(void *_arg)
{
time_t t;
struct tm *now;
char param[80];
char *name = (char *)_arg;
t = time(NULL);
now = localtime(&t);
strftime(param, sizeof(param), "%Y%m%d_%H%M%S\n", now);
printf("Migrate me out %s!\n", name);
migrate(1, migrated_callback, param);
sleep(1);
migrate(0, migrated_callback, param);
printf("I am back! %s\n", name);
return NULL;
}
int main(int argc, char *argv[])
{
fprintf(stderr, "main: start\n");
child("main");
fprintf(stderr, "main: exit\n");
return 0;
}
Stack trace:
[src/init.c:35] --> New execution started <--
[src/init.c:36] PID: 2318
[src/userspace.c:371] procfs stack limits: 0x7ffffffde000 -> 0x7ffffffff000
[src/userspace.c:317] Prepped stack for main thread, addresses 0x7fffff7ff000 -> 0x7ffffffff000
[src/init.c:73] Initializing handle for 'bmigrate_aarch64'
[src/init.c:97] Found 14 per-function unwinding metadata entries
[src/init.c:112] Found 68 callee-saved frame unwinding entries
[src/init.c:127] Found 62 call sites
[src/init.c:142] Found 26 live value location records
[src/init.c:161] Found 4 architecture-specific live value location records
[src/init.c:73] Initializing handle for 'bmigrate_powerpc64'
[src/userspace.c:136] WARNING: could not initialize powerpc64 handle
[src/init.c:73] Initializing handle for 'bmigrate_x86-64'
[src/init.c:97] Found 14 per-function unwinding metadata entries
[src/init.c:112] Found 59 callee-saved frame unwinding entries
[src/init.c:127] Found 62 call sites
[src/init.c:142] Found 26 live value location records
[src/init.c:161] Found 6 architecture-specific live value location records
[src/userspace.c:453] Thread 2318 beginning re-write
[src/userspace.c:462] On stack 0x7fffffffebd0, rewriting to 0x7fffffbff000
[src/rewrite.c:111] --> Initializing rewrite (x86-64 -> aarch64) <--
[src/rewrite.c:124] --> Unwinding source stack to find live activations <--
[src/unwind.c:171] Popping frame (CFA = 0x7fffffffea80)
[src/unwind.c:58] Callee-saved: 3 at FBP + -40 (0x7fffffffea48)
[src/unwind.c:58] Callee-saved: 12 at FBP + -32 (0x7fffffffea50)
[src/unwind.c:58] Callee-saved: 13 at FBP + -24 (0x7fffffffea58)
[src/unwind.c:58] Callee-saved: 14 at FBP + -16 (0x7fffffffea60)
[src/unwind.c:58] Callee-saved: 15 at FBP + -8 (0x7fffffffea68)
[src/unwind.c:58] Callee-saved: 16 at FBP + 8 (0x7fffffffea78)
[src/unwind.c:58] Callee-saved: 6 at FBP + 0 (0x7fffffffea70)
[src/unwind.c:72] Return address: 0x508cf7
[src/unwind.c:171] Popping frame (CFA = 0x7fffffffeac0)
[src/unwind.c:58] Callee-saved: 3 at FBP + -24 (0x7fffffffea98)
[src/unwind.c:58] Callee-saved: 14 at FBP + -16 (0x7fffffffeaa0)
[src/unwind.c:58] Callee-saved: 15 at FBP + -8 (0x7fffffffeaa8)
[src/unwind.c:58] Callee-saved: 16 at FBP + 8 (0x7fffffffeab8)
[src/unwind.c:58] Callee-saved: 6 at FBP + 0 (0x7fffffffeab0)
[src/unwind.c:72] Return address: 0x501130
[src/unwind.c:171] Popping frame (CFA = 0x7fffffffeb50)
[src/unwind.c:58] Callee-saved: 3 at FBP + -8 (0x7fffffffeb38)
[src/unwind.c:58] Callee-saved: 16 at FBP + 8 (0x7fffffffeb48)
[src/unwind.c:58] Callee-saved: 6 at FBP + 0 (0x7fffffffeb40)
[src/unwind.c:72] Return address: 0x5011d7
[src/unwind.c:171] Popping frame (CFA = 0x7fffffffeb80)
[src/unwind.c:58] Callee-saved: 3 at FBP + -16 (0x7fffffffeb60)
[src/unwind.c:58] Callee-saved: 14 at FBP + -8 (0x7fffffffeb68)
[src/unwind.c:58] Callee-saved: 14 at FBP + -8 (0x7fffffffeb68)
[src/unwind.c:58] Callee-saved: 16 at FBP + 8 (0x7fffffffeb78)
[src/unwind.c:58] Callee-saved: 6 at FBP + 0 (0x7fffffffeb70)
[src/unwind.c:72] Return address: 0x5014ad
[src/rewrite.c:379] Number of live activations: 5
[src/rewrite.c:380] Destination stack size: 344
[src/util.c:225] Address of enclosing function: 0x508670
[src/rewrite.c:393] Top of new stack: 0x7fffffbfeea0
[src/rewrite.c:394] Rewriting destination as if entering function @ 0x508670
[src/rewrite.c:133] --> Rewriting from source to destination stack <--
[src/rewrite.c:138] --> Rewriting outermost frame <--
[src/unwind.c:201] Popping frame (CFA = 0x7fffffbfeea0)
[src/unwind.c:215] Return address: 0x508d04
[src/unwind.c:103] New frame bounds: SP=0x7fffffbfeea0, FBP=0x7fffffbfeed0, CFA=0x7fffffbfeee0
[src/rewrite.c:146] --> Rewriting frame 1 <--
[src/rewrite.c:590] Rewriting frame (CFA: 0x7fffffffeac0 -> 0x7fffffbfeee0)
[src/unwind.c:171] Popping frame (CFA = 0x7fffffbfeee0)
[src/unwind.c:58] Callee-saved: 30 at FBP + 8 (0x7fffffbfeed8)
[src/unwind.c:58] Callee-saved: 29 at FBP + 0 (0x7fffffbfeed0)
[src/unwind.c:58] Callee-saved: 19 at FBP + -8 (0x7fffffbfeec8)
[src/unwind.c:58] Callee-saved: 20 at FBP + -16 (0x7fffffbfeec0)
[src/unwind.c:58] Callee-saved: 21 at FBP + -24 (0x7fffffbfeeb8)
[src/unwind.c:58] Callee-saved: 22 at FBP + -32 (0x7fffffbfeeb0)
[src/unwind.c:72] Return address: 0x501144
[src/unwind.c:103] New frame bounds: SP=0x7fffffbfeee0, FBP=0x7fffffbfef70, CFA=0x7fffffbfef80
[src/rewrite.c:154] Old FP saved to 0x7fffffbfeed0
[src/rewrite.c:146] --> Rewriting frame 2 <--
[src/rewrite.c:590] Rewriting frame (CFA: 0x7fffffffeb50 -> 0x7fffffbfef80)
[src/data.c:98] Getting source value: live value at stack address 0x7fffffffeac8
[src/data.c:100] Putting destination value (size=8): live value at stack address 0x7fffffbfeee8
[src/data.c:98] Getting source value: live value at stack address 0x7fffffffead0
[src/data.c:100] Putting destination value (size=80): live value at stack address 0x7fffffbfeef0
[src/data.c:129] Putting arch-specific destination value (size=8): live value in register 19
[src/data.c:398] Saving callee-saved register 19 at 0x7fffffbfeec8 (frame 1)
[src/data.c:136] Arch-specific live value: constant 5247024 / 5247024 / 501030
[src/unwind.c:171] Popping frame (CFA = 0x7fffffbfef80)
[src/unwind.c:58] Callee-saved: 30 at FBP + 8 (0x7fffffbfef78)
[src/unwind.c:58] Callee-saved: 29 at FBP + 0 (0x7fffffbfef70)
[src/unwind.c:58] Callee-saved: 19 at FBP + -8 (0x7fffffbfef68)
[src/unwind.c:58] Callee-saved: 20 at FBP + -16 (0x7fffffbfef60)
[src/unwind.c:72] Return address: 0x5011e8
[src/unwind.c:103] New frame bounds: SP=0x7fffffbfef80, FBP=0x7fffffbfefb0, CFA=0x7fffffbfefc0
[src/rewrite.c:154] Old FP saved to 0x7fffffbfef70
[src/rewrite.c:146] --> Rewriting frame 3 <--
[src/rewrite.c:590] Rewriting frame (CFA: 0x7fffffffeb80 -> 0x7fffffbfefc0)
[src/data.c:129] Putting arch-specific destination value (size=8): live value in register 21
[src/data.c:398] Saving callee-saved register 21 at 0x7fffffbfeeb8 (frame 1)
[src/data.c:136] Arch-specific live value: constant 6295688 / 6295688 / 601088
[src/unwind.c:171] Popping frame (CFA = 0x7fffffbfefc0)
[src/unwind.c:58] Callee-saved: 30 at FBP + 8 (0x7fffffbfefb8)
[src/unwind.c:58] Callee-saved: 29 at FBP + 0 (0x7fffffbfefb0)
[src/unwind.c:58] Callee-saved: 19 at FBP + -8 (0x7fffffbfefa8)
[src/unwind.c:58] Callee-saved: 20 at FBP + -16 (0x7fffffbfefa0)
[src/unwind.c:58] Callee-saved: 21 at FBP + -24 (0x7fffffbfef98)
[src/unwind.c:58] Callee-saved: 22 at FBP + -32 (0x7fffffbfef90)
[src/unwind.c:72] Return address: 0x5014bc
[src/unwind.c:103] New frame bounds: SP=0x7fffffbfefc0, FBP=0x7fffffbfefe0, CFA=0x7fffffbfeff0
[src/rewrite.c:154] Old FP saved to 0x7fffffbfefb0
[src/rewrite.c:159] --> Rewriting frame 4 (starting function) <--
[src/rewrite.c:590] Rewriting frame (CFA: 0x7fffffffebb0 -> 0x7fffffbfeff0)
[src/rewrite.c:169] Finished rewrite!
Can't rewrite stack after proposing migration
I'm encountering a "Could not rewrite stack" error when I try to propose migration of a process (see attached output file). As best as I can tell, the problem originates somewhere in the migration library.
The test file I wrote (see attached migration_test.tar.gz) takes in a path to a popcorn binary, then forks a child process. It runs the popcorn binary in the child process. In the parent process, it waits 1 second, then proposes migration.
The popcorn binary finishes correctly on the origin node (x86_64) if I do not try to migrate.
Steps I took to create error:
- Using the two-VM x86/ARM setup described in the popcorn-rack wiki (including kernel version).
- Built the compiler from the rack branch this morning.
- Compiled NPB CG serial, class A (see attached tar file for the source code, run
make CG CLASS=A). - Copied both CG binaries to x86 and ARM, renamed appropriately (
CG_A_<arch>toCG). - Compiled migration_test (
mkdir migration_test/build ; cd build ; cmake .. ; cmake --build .) - Ran migration_test (
~/migration_test/build/migration_test ~/CG).
migration_test.tar.gz
NPB3.3-SER-C-POPCORN.tar.gz
2017-11-28_migration-test-output.txt
Need to remove old alignment tool
It comprises:
- Removing the tool sources from the tree
- Modifying the install script not to install the tool, and remove dependencies checks (javac)
- Update readme
- Remove old Makefile template
Alignement of TLS symbols
While trying to support the GNU-OMP library (openmp branch of the compiler, under the name libopenpop), I found a problem at the compilation time. This type of warning keep popping out:
<<WARNING: symbol does not have the same name as the containing section>>
Suspecting that the problem comes from the use of the thread local storage section ".tbss" (which are used by the library), I tried to compile this simple program that does not use libopenpop:
__thread volatile int x;
int main()
{
return x+123;
}
Using "readelf -S" we find that the problem comes the .tbss section and the next section (.data.rel.ro) having the same base address:
...
[ 5] .eh_frame PROGBITS 000000000042b388 02b388 000040 00 A 0 0 8
[ 6] .tbss NOBITS 000000000043ffc0 02ffc0 000004 00 WAT 0 0 4
[ 7] .data.rel.ro PROGBITS 000000000043ffc0 02ffc0 000030 00 WA 0 0 8
...
Any idea on how to solve the problem ? How to give each section its own base address?
(I could use an ldscript for the compilation of the vanilla version and try to give both sections different addresses?)
pyalign script bugs
All of these only occur when trying to run pyalign scripts with Python 3, so the easy solution is to add the following line to the top of all alignment-related Python scripts (it's best practice too):
#!/usr/bin/env python2
I don't have the exact errors in front of me (will try to post later), but in __main__.py spaces and tabs are mixed at a few locations in the file. Additionally, there are some Python 2-specific functions that are called (i.e., iteritems on dicts). Again, adding the above shebang will solve some of these errors.
removing gitignore lines
Rob, why did you remove some lines from .gitignore files? Please see commit 21a5988
SIGRTMIN does not induce migration
TL; DR: Deliver SIGRTMIN+1 instead of SIGRTMIN to induce migration when using signal triggering.
When the migration library is compiled with type=signal_trigger, delivering SIGRTMIN to a migrateable process should cause a migration proposal. However, the process instead terminates -- the default action for SIGRTMIN.
On investigation, this stems from a discrepancy between the specified signal -- SIGRTMIN according to lib/migration/include/config.h -- and the actual signal that we set:
[anthony@fox4 11:28 NPB-make-popcorn-quiet-signal]$ gdb ./EP_B
...
(gdb) break sigaction
Breakpoint 1 at 0x51bf28: file src/signal/sigaction.c, line 55.
(gdb) run
Starting program: /home/anthony/NPB-make-popcorn-quiet-signal/EP_B
Breakpoint 1, __sigaction (sig=35, sa=0xfffffffff038, old=0x0) at src/signal/sigaction.c:55
55 src/signal/sigaction.c: No such file or directory.
On the second-to-last-line, we see sig=35 -- this is (on my machines at least) SIGRTMIN+1. The reason likely stems from how glibc implements POSIX threads. From man 7 signal:
The Linux kernel supports a range of 33 different real-time signals, numbered 32 to 64. However, the glibc POSIX threads implementation internally uses two (for NPTL) or three (for LinuxThreads) real-time signals (see pthreads(7)), and adjusts the value of SIGRTMIN suitably (to 34 or 35).
The simple solution then is to deliver SIGRTMIN+1 instead of SIGRTMIN.
Compilation fails with backend ran out of registers
For some benchmarks, inserting stackmaps into the code causes the following error message:
fatal error: error in backend: ran out of registers during register allocation
clang-3.7: error: clang frontend command failed with exit code 70 (use -v to see invocation)
clang version 3.7.1 (tags/RELEASE_371/final 320332)
Target: x86_64-unknown-linux-gnu
Thread model: posix
clang-3.7: note: diagnostic msg: PLEASE submit a bug report to http://llvm.org/bugs/ and include the crash backtrace, preprocessed source, and associated run script.
clang-3.7: note: diagnostic msg:
PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
Preprocessed source(s) and associated run script(s) are located at:
clang-3.7: note: diagnostic msg: /tmp/adapt-89f13a.c
clang-3.7: note: diagnostic msg: /tmp/adapt-89f13a.sh
clang-3.7: note: diagnostic msg:
This is caused by the backend pathologically rematerializing lots of values before a stackmap (in InlineSpiller::reMaterializeAll()). Rematerialized values cannot be spilled (to prevent infinite loops in the register allocator), but these values fill up all available registers, especially on x86. This eventually causes the register allocator to choke in the select or split implementation, e.g., RAGreedy::selectOrSplitImpl().
One solution would be to prevent rematerializing values before stackmaps and instead forcing them to be spilled to the stack. Spilled values will fold into the stackmap, cleanly preventing usage of physical registers (which really aren't using the values but are simply recording their location). However, this causes the creation of a large number of new spill slots which are not currently handled in the StackTransformMetadata pass.
The attached files reproduce this behavior.
Failed assertion in CodeGen/UnwindInfo.cpp
When building libopenpop with -popcorn-migratable, compilation of task.c crashes the compiler due to a failed assertion in /llvm/lib/CodeGen/UnwindInfo.cpp:
depbase=`echo task.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/bash ./libtool --tag=CC --mode=compile /usr/local/popcorn/x86_64/bin/musl-clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -Werror -ftls-model=initial-exec -Wc,-pthread -target x86_64-linux-gnu -nostdinc -isystem /usr/local/popcorn/x86_64/include -popcorn-migratable -Wno-error -pthread -popcorn-alignment -fno-common -MT task.lo -MD -MP -MF $depbase.Tpo -c -o task.lo task.c &&\
mv -f $depbase.Tpo $depbase.Plo
libtool: compile: /usr/local/popcorn/x86_64/bin/musl-clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -Werror -pthread -ftls-model=initial-exec -target x86_64-linux-gnu -nostdinc -isystem/usr/local/popcorn/x86_64/include -popcorn-migratable -Wno-error -pthread -popcorn-alignment -fno-common-MT task.lo -MD -MP -MF .deps/task.Tpo -c task.c -o task.o
clang-3.7: /usr/local/popcorn/src/llvm/lib/CodeGen/UnwindInfo.cpp:49: void llvm::UnwindInfo::recordUnwindInfo(const llvm::MachineFunction&): Assertion `FrameReg == TRI->getFrameRegister(MF) && "Invalid register used as offset for unwinding information"' failed.
0 libLLVMSupport.so.3.7 0x00007fba10ae1950 llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 44
1 libLLVMSupport.so.3.7 0x00007fba10ae1cc7
2 libLLVMSupport.so.3.7 0x00007fba10ae08c2
3 libc.so.6 0x00007fba0daf8030
4 libc.so.6 0x00007fba0daf7fcf gsignal + 207
5 libc.so.6 0x00007fba0daf93fa abort + 362
6 libc.so.6 0x00007fba0daf0e37
7 libc.so.6 0x00007fba0daf0ee2
8 libLLVMCodeGen.so.3.7 0x00007fba13b4d038 llvm::UnwindInfo::recordUnwindInfo(llvm::MachineFunction const&) + 562
9 libLLVMAArch64CodeGen.so.3.7 0x00007fba16a4e486
10 libLLVMCodeGen.so.3.7 0x00007fba1398fb43 llvm::MachineFunctionPass::runOnFunction(llvm::Function&) + 95
11 libLLVMCore.so.3.7 0x00007fba13052f2d llvm::FPPassManager::runOnFunction(llvm::Function&) +315
12 libLLVMCore.so.3.7 0x00007fba1305308c llvm::FPPassManager::runOnModule(llvm::Module&) + 118
13 libLLVMCore.so.3.7 0x00007fba130533bb
14 libLLVMCore.so.3.7 0x00007fba13053a1e llvm::legacy::PassManagerImpl::run(llvm::Module&) + 264
15 libLLVMCore.so.3.7 0x00007fba13053c0f llvm::legacy::PassManager::run(llvm::Module&) + 39
16 libclangCodeGen.so.3.7 0x00007fba0f3a3a0d
17 libclangCodeGen.so.3.7 0x00007fba0f3a3eff clang::CodegenBackendOutput(clang::DiagnosticsEngine&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, llvm::StringRef, llvm::Module*, clang::BackendAction, llvm::raw_pwrite_stream*) + 492
18 libclangCodeGen.so.3.7 0x00007fba0f5c1efe
19 libclangParse.so.3.7 0x00007fba09afacec clang::ParseAST(clang::Sema&, bool, bool) + 789
20 libclangFrontend.so.3.7 0x00007fba0eb643bb clang::ASTFrontendAction::ExecuteAction() + 323
21 libclangCodeGen.so.3.7 0x00007fba0f5bf8ed clang::EmitMultiObjAction::ExecuteAction() + 1065
22 libclangFrontend.so.3.7 0x00007fba0eb63e75 clang::FrontendAction::Execute() + 139
23 libclangFrontend.so.3.7 0x00007fba0eb19609 clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) + 785
24 libclangFrontendTool.so.3.7 0x00007fba0e70a27e clang::ExecuteCompilerInvocation(clang::CompilerInstance*) + 1120
25 clang-3.7 0x000055e32db3c937 cc1_main(llvm::ArrayRef<char const*>, char const*, void*) + 980
26 clang-3.7 0x000055e32db33d6a
27 clang-3.7 0x000055e32db34253 main + 1041
28 libc.so.6 0x00007fba0dae52b1 __libc_start_main + 241
29 clang-3.7 0x000055e32db325ca _start + 42
Stack dump:
0. Program arguments: /usr/local/popcorn/bin/clang-3.7 -cc1 -triple x86_64--linux-gnu -emit-obj -mrelax-all -disable-free -main-file-name task.c -mrelocation-model static -mthread-model posix -mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -target-cpu x86-64 -dwarf-column-info -coverage-file /home/anthony/Research/popcorn-compiler/lib/libopenpop/task.o -nostdsysteminc -nobuiltininc -resource-dir /usr/local/popcorn/bin/../lib/clang/3.7.1 -dependency-file .deps/task.Tpo -sys-header-deps -MP -MT task.lo -isystem /usr/local/popcorn/x86_64/include -isystem /usr/local/popcorn/x86_64/include -D HAVE_CONFIG_H -I . -I ./config/linux -I ./config/posix -I . -isysroot /usr/local/popcorn/x86_64 -Wall -Werror -Wno-error -fdebug-compilation-dir /home/anthony/Research/popcorn-compiler/lib/libopenpop -ferror-limit 19 -fmessage-length 105 -ftls-model=initial-exec -pthread -mstackrealign -fobjc-runtime=gcc -fno-common -fdiagnostics-show-option -fcolor-diagnostics -ffunction-sections -fdata-sections -popcorn-alignment -popcorn-migratable -mllvm -popcorn-instrument=migration -mllvm -optimize-regalloc -mllvm -fast-isel=false -o task.o -x c task.c
1. <eof> parser at end of file
2. Code generation
3. Running pass 'Function Pass Manager' on module 'task.c'.
4. Running pass 'AArch64 Assembly Printer' on function '@GOMP_task'
clang-3.7: error: unable to execute command: Aborted
clang-3.7: error: clang frontend command failed due to signal (use -v to see invocation)
clang version 3.7.1 (tags/RELEASE_371/final 325143) (llvm/tags/RELEASE_371/final 325142)
Target: x86_64--linux-gnu
Thread model: posix
clang-3.7: note: diagnostic msg: PLEASE submit a bug report to http://llvm.org/bugs/ and include the crash backtrace, preprocessed source, and associated run script.
clang-3.7: note: diagnostic msg:
********************
PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
Preprocessed source(s) and associated run script(s) are located at:
clang-3.7: note: diagnostic msg: /tmp/task-da5cca.c
clang-3.7: note: diagnostic msg: /tmp/task-da5cca.sh
clang-3.7: note: diagnostic msg:
********************
Makefile:716: recipe for target 'task.lo' failed
make[1]: *** [task.lo] Error 1
make[1]: Leaving directory '/home/anthony/Research/popcorn-compiler/lib/libopenpop'
Makefile:530: recipe for target 'all' failed
make: *** [all] Error 2
Steps to reproduce:
- Patch
musl-1.1.18with attached patch (note: this just modifies the musl-clang script, so you can manually modify it if you don't want to re-install the library). - Configure
libopenpopwith following command:
CC="/usr/local/popcorn/x86_64/bin/musl-clang" CFLAGS="-target x86_64-linux-gnu -nostdinc -isystem /usr/local/popcorn/x86_64/include -popcorn-migratable -Wno-error" ./configure --prefix=/usr/local/popcorn/x86_64/ --target=x86_64-linux-gnu --host=x86_64-linux-gnu --enable-static --disable-shared
- Build with
make
Attached files include:
-
Output with
-mllvm -debug-only=regalloc -mllvm -debug-only=stacktransformflags
Cannot Build libopenpop
I cannot build libopenpop on the latest update. Below is the output I'm getting. @rlyerly or @beowulf do you have any suggestions how to solve this error?
checking build system type... x86_64-pc-linux-gnu
checking host system type... aarch64-unknown-linux-gnu
checking how to print strings... printf
checking for aarch64-linux-gnu-gcc... /home/test/popcorn/bin/clang
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... yes
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether /home/test/popcorn/bin/clang accepts -g... yes
checking for /home/test/popcorn/bin/clang option to accept ISO C89... none needed
checking whether /home/test/popcorn/bin/clang understands -c and -o together... yes
checking for a sed that does not truncate output... /bin/sed
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for fgrep... /bin/grep -F
checking for ld used by /home/test/popcorn/bin/clang... /home/test/popcorn/bin/ld
checking if the linker (/home/test/popcorn/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/aarch64-linux-gnu-nm -B
checking the name lister (/usr/bin/aarch64-linux-gnu-nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking how to convert x86_64-pc-linux-gnu file names to aarch64-unknown-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /home/test/popcorn/bin/ld option to reload object files... -r
checking for aarch64-linux-gnu-objdump... aarch64-linux-gnu-objdump
checking how to recognize dependent libraries... pass_all
checking for aarch64-linux-gnu-dlltool... no
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for aarch64-linux-gnu-ar... aarch64-linux-gnu-ar
checking for archiver @FILE support... @
checking for aarch64-linux-gnu-strip... aarch64-linux-gnu-strip
checking for aarch64-linux-gnu-ranlib... aarch64-linux-gnu-ranlib
checking for gawk... no
checking for mawk... mawk
checking command to parse /usr/bin/aarch64-linux-gnu-nm -B output from /home/test/popcorn/bin/clang object... ok
checking for sysroot... no
checking for a working dd... /bin/dd
checking how to truncate binary pipes... /bin/dd bs=4096 count=1
checking for aarch64-linux-gnu-mt... no
checking for mt... mt
configure: WARNING: using cross tools not prefixed with host triplet
checking if mt is a manifest tool... no
checking how to run the C preprocessor... /home/test/popcorn/bin/clang -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if /home/test/popcorn/bin/clang supports -fno-rtti -fno-exceptions... yes
checking for /home/test/popcorn/bin/clang option to produce PIC... -fPIC -DPIC
checking if /home/test/popcorn/bin/clang PIC flag -fPIC -DPIC works... yes
checking if /home/test/popcorn/bin/clang static flag -static works... yes
checking if /home/test/popcorn/bin/clang supports -c -o file.o... yes
checking if /home/test/popcorn/bin/clang supports -c -o file.o... (cached) yes
checking whether the /home/test/popcorn/bin/clang linker (/home/test/popcorn/bin/ld) supports shared libraries... yes
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... no
checking whether to build static libraries... yes
checking for --enable-version-specific-runtime-libs... no
checking for --enable-generated-files-in-srcdir... no
checking target system type... aarch64-unknown-linux-gnu
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for aarch64-linux-gnu-strip... (cached) aarch64-linux-gnu-strip
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking whether make sets $(MAKE)... yes
checking for style of include used by make... GNU
checking whether make supports nested variables... yes
checking dependency style of /home/test/popcorn/bin/clang... gcc3
clang-3.7: error: unsupported option '-print-multi-os-directory'
clang-3.7: error: no input files
checking for aarch64-linux-gnu-gcc... (cached) /home/test/popcorn/bin/clang
checking whether we are using the GNU C compiler... (cached) yes
checking whether /home/test/popcorn/bin/clang accepts -g... (cached) yes
checking for /home/test/popcorn/bin/clang option to accept ISO C89... (cached) none needed
checking whether /home/test/popcorn/bin/clang understands -c and -o together... (cached) yes
checking for aarch64-linux-gnu-ar... (cached) aarch64-linux-gnu-ar
checking for aarch64-linux-gnu-ranlib... (cached) aarch64-linux-gnu-ranlib
checking for perl... /usr/bin/perl
checking whether make sets $(MAKE)... (cached) yes
checking for makeinfo... ${SHELL} /home/test/compiler/lib/libopenpop/missing makeinfo
checking for modern makeinfo... yes
checking whether to enable maintainer-specific portions of Makefiles... no
checking for aarch64-linux-gnu-gfortran... no
checking whether we are using the GNU Fortran compiler... no
checking whether no accepts -g... no
checking for ANSI C header files... (cached) yes
checking whether time.h and sys/time.h may both be included... yes
checking whether string.h and strings.h may both be included... yes
checking pthread.h usability... yes
checking pthread.h presence... yes
checking for pthread.h... yes
checking for unistd.h... (cached) yes
checking semaphore.h usability... yes
checking semaphore.h presence... yes
checking for semaphore.h... yes
checking sys/loadavg.h usability... no
checking sys/loadavg.h presence... no
checking for sys/loadavg.h... no
checking sys/sysctl.h usability... no
checking sys/sysctl.h presence... yes
configure: WARNING: sys/sysctl.h: present but cannot be compiled
configure: WARNING: sys/sysctl.h: check for missing prerequisite headers?
configure: WARNING: sys/sysctl.h: see the Autoconf documentation
configure: WARNING: sys/sysctl.h: section "Present But Cannot Be Compiled"
configure: WARNING: sys/sysctl.h: proceeding with the compiler's result
checking for sys/sysctl.h... no
checking sys/time.h usability... yes
checking sys/time.h presence... yes
checking for sys/time.h... yes
looking for a compliant stdint.h in stdint.h, checking for uintmax_t... yes
checking for uintptr_t... yes
checking for int_least32_t... yes
checking for int_fast32_t... yes
checking for uint64_t... yes
checking what to include in gstdint.h... stdint.h (already complete)
checking for dlsym in -ldl... yes
checking for unistd.h... (cached) yes
checking for secure_getenv... no
checking for __secure_getenv... no
checking for getuid... yes
checking for geteuid... yes
checking for getgid... yes
checking for getegid... yes
checking for getloadavg... yes
checking for clock_gettime... yes
checking for strtoull... yes
checking whether the target supports thread-local storage... yes
checking whether the thread-local storage support is from emutls... no
checking whether the target supports hidden visibility... yes
checking whether the target supports dllexport... no
checking whether the target supports symbol aliases... yes
checking for ld used by /home/test/popcorn/bin/clang... (cached) /home/test/popcorn/bin/ld
checking if the linker (/home/test/popcorn/bin/ld) is GNU ld... (cached) yes
checking whether the target supports .symver directive... yes
configure: versioning on shared library symbols is no
checking whether the target supports __sync_*_compare_and_swap... yes
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating omp.h
config.status: creating omp_lib.h
config.status: creating omp_lib.f90
config.status: creating libgomp_f.h
config.status: creating Makefile
config.status: creating libopenpop.spec
config.status: creating config.h
config.status: executing libtool commands
config.status: executing depfiles commands
config.status: executing gstdint.h commands
make all-am
make[1]: Entering directory '/home/test/compiler/lib/libopenpop'
depbase=`echo alloc.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/bash ./libtool --tag=CC --mode=compile /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -ftls-model=initial-exec -Wc,-pthread -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT alloc.lo -MD -MP -MF $depbase.Tpo -c -o alloc.lo alloc.c &&\
mv -f $depbase.Tpo $depbase.Plo
depbase=`echo atomic.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/bash ./libtool --tag=CC --mode=compile /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -ftls-model=initial-exec -Wc,-pthread -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT atomic.lo -MD -MP -MF $depbase.Tpo -c -o atomic.lo atomic.c &&\
mv -f $depbase.Tpo $depbase.Plo
depbase=`echo barrier.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/bash ./libtool --tag=CC --mode=compile /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -ftls-model=initial-exec -Wc,-pthread -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT barrier.lo -MD -MP -MF $depbase.Tpo -c -o barrier.lo barrier.c &&\
mv -f $depbase.Tpo $depbase.Plo
depbase=`echo critical.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/bash ./libtool --tag=CC --mode=compile /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -ftls-model=initial-exec -Wc,-pthread -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT critical.lo -MD -MP -MF $depbase.Tpo -c -o critical.lo critical.c &&\
mv -f $depbase.Tpo $depbase.Plo
depbase=`echo env.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/bash ./libtool --tag=CC --mode=compile /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -ftls-model=initial-exec -Wc,-pthread -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT env.lo -MD -MP -MF $depbase.Tpo -c -o env.lo env.c &&\
mv -f $depbase.Tpo $depbase.Plo
depbase=`echo error.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/bash ./libtool --tag=CC --mode=compile /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -ftls-model=initial-exec -Wc,-pthread -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT error.lo -MD -MP -MF $depbase.Tpo -c -o error.lo error.c &&\
mv -f $depbase.Tpo $depbase.Plo
depbase=`echo icv.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/bash ./libtool --tag=CC --mode=compile /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -ftls-model=initial-exec -Wc,-pthread -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT icv.lo -MD -MP -MF $depbase.Tpo -c -o icv.lo icv.c &&\
mv -f $depbase.Tpo $depbase.Plo
depbase=`echo icv-device.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/bash ./libtool --tag=CC --mode=compile /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -ftls-model=initial-exec -Wc,-pthread -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT icv-device.lo -MD -MP -MF $depbase.Tpo -c -o icv-device.lo icv-device.c &&\
mv -f $depbase.Tpo $depbase.Plo
libtool: compile: /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -pthread -ftls-model=initial-exec -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT alloc.lo -MD -MP -MF .deps/alloc.Tpo -c alloc.c -o alloc.o
libtool: compile: /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -pthread -ftls-model=initial-exec -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT atomic.lo -MD -MP -MF .deps/atomic.Tpo -c atomic.c -o atomic.o
libtool: compile: /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -pthread -ftls-model=initial-exec -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT icv-device.lo -MD -MP -MF .deps/icv-device.Tpo -c icv-device.c -o icv-device.o
libtool: compile: /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -pthread -ftls-model=initial-exec -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT env.lo -MD -MP -MF .deps/env.Tpo -c env.c -o env.o
libtool: compile: /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -pthread -ftls-model=initial-exec -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT critical.lo -MD -MP -MF .deps/critical.Tpo -c critical.c -o critical.o
libtool: compile: /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -pthread -ftls-model=initial-exec -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT icv.lo -MD -MP -MF .deps/icv.Tpo -c icv.c -o icv.o
libtool: compile: /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -pthread -ftls-model=initial-exec -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT barrier.lo -MD -MP -MF .deps/barrier.Tpo -c barrier.c -o barrier.o
libtool: compile: /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -pthread -ftls-model=initial-exec -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT error.lo -MD -MP -MF .deps/error.Tpo -c error.c -o error.o
depbase=`echo iter.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/bash ./libtool --tag=CC --mode=compile /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -ftls-model=initial-exec -Wc,-pthread -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT iter.lo -MD -MP -MF $depbase.Tpo -c -o iter.lo iter.c &&\
mv -f $depbase.Tpo $depbase.Plo
error.c:56:1: warning: Popcorn compatibility in function 'gomp_debug': function takes a variable number of arguments [-Wpass-failed]
}
^
error.c:55:3: warning: Popcorn compatibility in function 'gomp_debug': va_arg not transformable across architectures [-Wpass-failed]
va_end (list);
^
error.c:74:1: warning: Popcorn compatibility in function 'gomp_error': function takes a variable number of arguments [-Wpass-failed]
}
^
error.c:73:3: warning: Popcorn compatibility in function 'gomp_error': va_arg not transformable across architectures [-Wpass-failed]
va_end (list);
^
error.c:91:1: warning: Popcorn compatibility in function 'gomp_fatal': function takes a variable number of arguments [-Wpass-failed]
}
^
depbase=`echo iter_ull.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/bash ./libtool --tag=CC --mode=compile /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -ftls-model=initial-exec -Wc,-pthread -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT iter_ull.lo -MD -MP -MF $depbase.Tpo -c -o iter_ull.lo iter_ull.c &&\
mv -f $depbase.Tpo $depbase.Plo
depbase=`echo kmp.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/bash ./libtool --tag=CC --mode=compile /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -ftls-model=initial-exec -Wc,-pthread -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT kmp.lo -MD -MP -MF $depbase.Tpo -c -o kmp.lo kmp.c &&\
mv -f $depbase.Tpo $depbase.Plo
libtool: compile: /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -pthread -ftls-model=initial-exec -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT iter.lo -MD -MP -MF .deps/iter.Tpo -c iter.c -o iter.o
depbase=`echo loop.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/bash ./libtool --tag=CC --mode=compile /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -ftls-model=initial-exec -Wc,-pthread -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT loop.lo -MD -MP -MF $depbase.Tpo -c -o loop.lo loop.c &&\
mv -f $depbase.Tpo $depbase.Plo
5 warnings generated.
depbase=`echo loop_ull.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/bash ./libtool --tag=CC --mode=compile /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -ftls-model=initial-exec -Wc,-pthread -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT loop_ull.lo -MD -MP -MF $depbase.Tpo -c -o loop_ull.lo loop_ull.c &&\
mv -f $depbase.Tpo $depbase.Plo
libtool: compile: /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -pthread -ftls-model=initial-exec -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT iter_ull.lo -MD -MP -MF .deps/iter_ull.Tpo -c iter_ull.c -o iter_ull.o
depbase=`echo ordered.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/bash ./libtool --tag=CC --mode=compile /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -ftls-model=initial-exec -Wc,-pthread -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT ordered.lo -MD -MP -MF $depbase.Tpo -c -o ordered.lo ordered.c &&\
mv -f $depbase.Tpo $depbase.Plo
libtool: compile: /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -pthread -ftls-model=initial-exec -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT kmp.lo -MD -MP -MF .deps/kmp.Tpo -c kmp.c -o kmp.o
libtool: compile: /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -pthread -ftls-model=initial-exec -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT loop.lo -MD -MP -MF .deps/loop.Tpo -c loop.c -o loop.o
libtool: compile: /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -pthread -ftls-model=initial-exec -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT loop_ull.lo -MD -MP -MF .deps/loop_ull.Tpo -c loop_ull.c -o loop_ull.o
libtool: compile: /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -pthread -ftls-model=initial-exec -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT ordered.lo -MD -MP -MF .deps/ordered.Tpo -c ordered.c -o ordered.o
depbase=`echo parallel.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/bash ./libtool --tag=CC --mode=compile /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -ftls-model=initial-exec -Wc,-pthread -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT parallel.lo -MD -MP -MF $depbase.Tpo -c -o parallel.lo parallel.c &&\
mv -f $depbase.Tpo $depbase.Plo
libtool: compile: /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -pthread -ftls-model=initial-exec -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT parallel.lo -MD -MP -MF .deps/parallel.Tpo -c parallel.c -o parallel.o
kmp.c:110:1: warning: Popcorn compatibility in function '__kmpc_fork_call': function takes a variable number of arguments [-Wpass-failed]
}
^
kmp.c:92:3: warning: Popcorn compatibility in function '__kmpc_fork_call': va_arg not transformable across architectures [-Wpass-failed]
va_end(vl);
^
depbase=`echo sections.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/bash ./libtool --tag=CC --mode=compile /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -ftls-model=initial-exec -Wc,-pthread -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT sections.lo -MD -MP -MF $depbase.Tpo -c -o sections.lo sections.c &&\
mv -f $depbase.Tpo $depbase.Plo
libtool: compile: /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -pthread -ftls-model=initial-exec -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT sections.lo -MD -MP -MF .deps/sections.Tpo -c sections.c -o sections.o
depbase=`echo single.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/bash ./libtool --tag=CC --mode=compile /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -ftls-model=initial-exec -Wc,-pthread -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT single.lo -MD -MP -MF $depbase.Tpo -c -o single.lo single.c &&\
mv -f $depbase.Tpo $depbase.Plo
libtool: compile: /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -pthread -ftls-model=initial-exec -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT single.lo -MD -MP -MF .deps/single.Tpo -c single.c -o single.o
2 warnings generated.
depbase=`echo task.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/bash ./libtool --tag=CC --mode=compile /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -ftls-model=initial-exec -Wc,-pthread -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT task.lo -MD -MP -MF $depbase.Tpo -c -o task.lo task.c &&\
mv -f $depbase.Tpo $depbase.Plo
libtool: compile: /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -pthread -ftls-model=initial-exec -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT task.lo -MD -MP -MF .deps/task.Tpo -c task.c -o task.o
depbase=`echo team.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/bash ./libtool --tag=CC --mode=compile /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -ftls-model=initial-exec -Wc,-pthread -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT team.lo -MD -MP -MF $depbase.Tpo -c -o team.lo team.c &&\
mv -f $depbase.Tpo $depbase.Plo
libtool: compile: /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -pthread -ftls-model=initial-exec -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT team.lo -MD -MP -MF .deps/team.Tpo -c team.c -o team.o
depbase=`echo work.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/bash ./libtool --tag=CC --mode=compile /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -ftls-model=initial-exec -Wc,-pthread -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT work.lo -MD -MP -MF $depbase.Tpo -c -o work.lo work.c &&\
mv -f $depbase.Tpo $depbase.Plo
depbase=`echo lock.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/bash ./libtool --tag=CC --mode=compile /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -ftls-model=initial-exec -Wc,-pthread -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT lock.lo -MD -MP -MF $depbase.Tpo -c -o lock.lo lock.c &&\
mv -f $depbase.Tpo $depbase.Plo
depbase=`echo mutex.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/bash ./libtool --tag=CC --mode=compile /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -ftls-model=initial-exec -Wc,-pthread -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT mutex.lo -MD -MP -MF $depbase.Tpo -c -o mutex.lo ./config/linux/mutex.c &&\
mv -f $depbase.Tpo $depbase.Plo
libtool: compile: /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -pthread -ftls-model=initial-exec -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT work.lo -MD -MP -MF .deps/work.Tpo -c work.c -o work.o
libtool: compile: /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -pthread -ftls-model=initial-exec -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT lock.lo -MD -MP -MF .deps/lock.Tpo -c lock.c -o lock.o
libtool: compile: /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -pthread -ftls-model=initial-exec -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT mutex.lo -MD -MP -MF .deps/mutex.Tpo -c ./config/linux/mutex.c -o mutex.o
team.c:35:10: fatal error: 'migrate.h' file not found
#include "migrate.h"
^
ordered.c:486:1: warning: Popcorn compatibility in function 'GOMP_doacross_wait': function takes a variable number of arguments [-Wpass-failed]
}
^
ordered.c:454:4: warning: Popcorn compatibility in function 'GOMP_doacross_wait': va_arg not transformable across architectures [-Wpass-failed]
va_end (ap);
^
In file included from ordered.c:31:
In file included from ./config/linux/doacross.h:33:
In file included from ./config/linux/wait.h:46:
./config/linux/futex.h:69:3: warning: Popcorn compatibility in function 'GOMP_doacross_wait': inline assembly may have unanalyzable side-effects [-Wpass-failed]
__asm volatile ("" : : : "memory");
^
ordered.c:459:7: warning: Popcorn compatibility in function 'GOMP_doacross_wait': va_arg not transformable across architectures [-Wpass-failed]
va_end (ap);
^
ordered.c:479:7: warning: Popcorn compatibility in function 'GOMP_doacross_wait': va_arg not transformable across architectures [-Wpass-failed]
va_end (ap);
^
In file included from ordered.c:31:
In file included from ./config/linux/doacross.h:33:
In file included from ./config/linux/wait.h:46:
./config/linux/futex.h:69:3: warning: Popcorn compatibility in function 'GOMP_doacross_wait': inline assembly may have unanalyzable side-effects [-Wpass-failed]
__asm volatile ("" : : : "memory");
^
ordered.c:785:1: warning: Popcorn compatibility in function 'GOMP_doacross_ull_wait': function takes a variable number of arguments [-Wpass-failed]
}
^
ordered.c:709:4: warning: Popcorn compatibility in function 'GOMP_doacross_ull_wait': va_arg not transformable across architectures [-Wpass-failed]
va_end (ap);
^
In file included from ordered.c:31:
In file included from ./config/linux/doacross.h:33:
In file included from ./config/linux/wait.h:46:
./config/linux/futex.h:69:3: warning: Popcorn compatibility in function 'GOMP_doacross_ull_wait': inline assembly may have unanalyzable side-effects [-Wpass-failed]
__asm volatile ("" : : : "memory");
^
ordered.c:714:7: warning: Popcorn compatibility in function 'GOMP_doacross_ull_wait': va_arg not transformable across architectures [-Wpass-failed]
va_end (ap);
^
ordered.c:738:4: warning: Popcorn compatibility in function 'GOMP_doacross_ull_wait': va_arg not transformable across architectures [-Wpass-failed]
va_end (ap);
^
In file included from ordered.c:31:
In file included from ./config/linux/doacross.h:33:
In file included from ./config/linux/wait.h:46:
./config/linux/futex.h:69:3: warning: Popcorn compatibility in function 'GOMP_doacross_ull_wait': inline assembly may have unanalyzable side-effects [-Wpass-failed]
__asm volatile ("" : : : "memory");
^
0 libLLVMSupport.so.3.7 0x00007ffff1852275 llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 44
1 libLLVMSupport.so.3.7 0x00007ffff18525f1
2 libLLVMSupport.so.3.7 0x00007ffff18511c9
3 libc.so.6 0x00007fffee7e60e0
4 libclangCodeGen.so.3.7 0x00007ffff0120be0
5 libclangCodeGen.so.3.7 0x00007ffff02c07d4
6 libclangParse.so.3.7 0x00007fffea64e0de clang::ParseAST(clang::Sema&, bool, bool) + 787
7 libclangFrontend.so.3.7 0x00007fffef80f740 clang::ASTFrontendAction::ExecuteAction() + 322
8 libclangCodeGen.so.3.7 0x00007ffff02be55c clang::EmitMultiObjAction::ExecuteAction() + 1068
9 libclangFrontend.so.3.7 0x00007fffef80f1fd clang::FrontendAction::Execute() + 139
10 libclangFrontend.so.3.7 0x00007fffef7c3dad clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) + 775
11 libclangFrontendTool.so.3.7 0x00007fffef388e2c clang::ExecuteCompilerInvocation(clang::CompilerInstance*) + 1120
12 clang-3.7 0x0000000000446a35 cc1_main(llvm::ArrayRef<char const*>, char const*, void*) + 989
13 clang-3.7 0x000000000043de5f
14 clang-3.7 0x000000000043e347 main + 1041
15 libc.so.6 0x00007fffee7d2b45 __libc_start_main + 245
16 clang-3.7 0x000000000043c729
Stack dump:
0. Program arguments: /home/test/popcorn/bin/clang-3.7 -cc1 -triple x86_64--linux-gnu -emit-obj -disable-free -main-file-name team.c -mrelocation-model static -mthread-model posix -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -fuse-init-array -target-cpu x86-64 -momit-leaf-frame-pointer -g -dwarf-column-info -coverage-file /home/test/compiler/lib/libopenpop/team.o -nostdsysteminc -nobuiltininc -resource-dir /home/test/popcorn/bin/../lib/clang/3.7.1 -dependency-file .deps/team.Tpo -sys-header-deps -MP -MT team.lo -isystem /home/test/popcorn/aarch64/include -D HAVE_CONFIG_H -I . -I ./config/linux -I ./config/posix -I . -O2 -Wall -Wall -fdebug-compilation-dir /home/test/compiler/lib/libopenpop -ferror-limit 19 -fmessage-length 0 -ftls-model=initial-exec -pthread -mstackrealign -fobjc-runtime=gcc -fno-common -fdiagnostics-show-option -vectorize-loops -vectorize-slp -ffunction-sections -fdata-sections -popcorn-alignment -popcorn-migratable -mllvm -popcorn-instrument=metadata -mllvm -optimize-regalloc -mllvm -fast-isel=false -popcorn-target=aarch64-linux-gnu -o team.o -x c team.c
1. <eof> parser at end of file
clang-3.7: error: unable to execute command: Segmentation fault
clang-3.7: error: clang frontend command failed due to signal (use -v to see invocation)
clang version 3.7.1 (tags/RELEASE_371/final 332466)
Target: x86_64--linux-gnu
Thread model: posix
clang-3.7: note: diagnostic msg: PLEASE submit a bug report to http://llvm.org/bugs/ and include the crash backtrace, preprocessed source, and associated run script.
depbase=`echo proc.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
/bin/bash ./libtool --tag=CC --mode=compile /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -ftls-model=initial-exec -Wc,-pthread -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT proc.lo -MD -MP -MF $depbase.Tpo -c -o proc.lo ./config/linux/proc.c &&\
mv -f $depbase.Tpo $depbase.Plo
clang-3.7: note: diagnostic msg: Error generating preprocessed source(s).
Makefile:716: recipe for target 'team.lo' failed
make[1]: *** [team.lo] Error 1
make[1]: *** Waiting for unfinished jobs....
libtool: compile: /home/test/popcorn/bin/clang -DHAVE_CONFIG_H -I. -I./config/linux -I./config/posix -I. -Wall -pthread -ftls-model=initial-exec -target x86_64-linux-gnu -O2 -g -Wall -nostdinc -isystem /home/test/popcorn/aarch64/include -popcorn-metadata -popcorn-target=aarch64-linux-gnu -pthread -fno-common -MT proc.lo -MD -MP -MF .deps/proc.Tpo -c ./config/linux/proc.c -o proc.o
12 warnings generated.
In file included from ./config/linux/mutex.c:30:
In file included from ./config/linux/wait.h:46:
./config/linux/futex.h:69:3: warning: Popcorn compatibility in function 'gomp_mutex_lock_slow': inline assembly may have unanalyzable side-effects [-Wpass-failed]
__asm volatile ("" : : : "memory");
^
./config/linux/futex.h:69:3: warning: Popcorn compatibility in function 'gomp_mutex_lock_slow': inline assembly may have unanalyzable side-effects [-Wpass-failed]
work.c:56:3: warning: Popcorn compatibility in function 'gomp_work_share_start': inline assembly may have unanalyzable side-effects [-Wpass-failed]
__asm ("" : "+r" (ws));
^
2 warnings generated.
1 warning generated.
env.c:1212:3: warning: (aarch64--linux-gnu) Stack transformation: unhandled register W22 across call to fprintf [-Wpass-failed]
fprintf (stderr, " OMP_STACKSIZE = '%lu'\n", stacksize);
^
env.c:1215:3: warning: (aarch64--linux-gnu) Stack transformation: unhandled register W22 across call to fprintf [-Wpass-failed]
fprintf (stderr, " OMP_WAIT_POLICY = '%s'\n",
^
env.c:1217:3: warning: (aarch64--linux-gnu) Stack transformation: unhandled register W22 across call to fprintf [-Wpass-failed]
fprintf (stderr, " OMP_THREAD_LIMIT = '%u'\n",
^
env.c:1219:3: warning: (aarch64--linux-gnu) Stack transformation: unhandled register W22 across call to fprintf [-Wpass-failed]
fprintf (stderr, " OMP_MAX_ACTIVE_LEVELS = '%lu'\n",
^
env.c:1222:3: warning: (aarch64--linux-gnu) Stack transformation: unhandled register W22 across call to fprintf [-Wpass-failed]
fprintf (stderr, " OMP_CANCELLATION = '%s'\n",
^
env.c:1224:3: warning: (aarch64--linux-gnu) Stack transformation: unhandled register W22 across call to fprintf [-Wpass-failed]
fprintf (stderr, " OMP_DEFAULT_DEVICE = '%d'\n",
^
env.c:1226:3: warning: (aarch64--linux-gnu) Stack transformation: unhandled register W22 across call to fprintf [-Wpass-failed]
fprintf (stderr, " OMP_MAX_TASK_PRIORITY = '%d'\n",
^
7 warnings generated.
In file included from task.c:471:
./taskloop.c:200:8: warning: (aarch64--linux-gnu) Stack transformation: unhandled stack slot 60 [-Wpass-failed]
fn (arg);
^
In file included from task.c:29:
./libgomp.h:798:5: warning: (aarch64--linux-gnu) Stack transformation: unhandled stack slot 45 across call to free [-Wpass-failed]
free (task->depend_hash);
^
./libgomp.h:686:10: warning: (aarch64--linux-gnu) Stack transformation: unhandled stack slot 56 across call to pthread_getspecific [-Wpass-failed]
return pthread_getspecific (gomp_tls_key);
^
In file included from task.c:471:
./taskloop.c:318:4: warning: (aarch64--linux-gnu) Stack transformation: unhandled stack slot 9 across call to priority_queue_insert [-Wpass-failed]
priority_queue_insert (PQ_TEAM, &team->task_queue, task, priority,
^
./taskloop.c:200:8: warning: (aarch64--linux-gnu) Stack transformation: unhandled register X28 [-Wpass-failed]
fn (arg);
^
In file included from task.c:29:
./libgomp.h:686:10: warning: (aarch64--linux-gnu) Stack transformation: unhandled stack slot 57 across call to pthread_getspecific [-Wpass-failed]
return pthread_getspecific (gomp_tls_key);
^
In file included from task.c:471:
./taskloop.c:318:4: warning: (aarch64--linux-gnu) Stack transformation: unhandled stack slot 9 across call to priority_queue_insert [-Wpass-failed]
priority_queue_insert (PQ_TEAM, &team->task_queue, task, priority,
^
In file included from task.c:29:
In file included from ./libgomp.h:124:
./priority_queue.h:214:5: warning: (aarch64--linux-gnu) Stack transformation: unhandled register X23 across call to prio_splay_tree_lookup [-Wpass-failed]
prio_splay_tree_lookup (&head->t, &k);
^
./priority_queue.h:214:5: warning: (aarch64--linux-gnu) Stack transformation: unhandled register X22 across call to prio_splay_tree_lookup [-Wpass-failed]
9 warnings generated.
make[1]: Leaving directory '/home/test/compiler/lib/libopenpop'
Makefile:530: recipe for target 'all' failed
make: *** [all] Error 2
Checking for prerequisites (see README for more info)...
Configuring libopenpop (aarch64)...
Making libopenpop (aarch64)...
Could not make libopenpop (aarch64) (Command '['make', '-j', '8']' returned non-zero exit status 2)!
./install_compiler.py error on Ubuntu 20.04
Envoronment:
# cat /etc/issue
Ubuntu 20.04 LTS \n \l
# gcc --version
gcc (Ubuntu 9.3.0-10ubuntu2) 9.3.0
# g++ --version
g++ (Ubuntu 9.3.0-10ubuntu2) 9.3.0
Error building llvm:
[ 17%] Building CXX object lib/Transforms/Utils/CMakeFiles/LLVMTransformUtils.dir/CloneFunction.cpp.o
In file included from /usr/local/popcorn/src/llvm/include/llvm/Transforms/Utils/Cloning.h:24,
from /usr/local/popcorn/src/llvm/lib/Transforms/Utils/CloneFunction.cpp:16:
/usr/local/popcorn/src/llvm/include/llvm/IR/ValueMap.h: In member function 'bool llvm::ValueMap<KeyT, ValueT, Config>::hasMD() const':
/usr/local/popcorn/src/llvm/include/llvm/IR/ValueMap.h:102:31: error: cannot convert 'const std::unique_ptr<llvm::DenseMap<const llvm::Metadata*, llvm::TrackingMDRef> >' to 'bool' in return
102 | bool hasMD() const { return MDMap; }
| ^~~~~
| |
| const std::unique_ptr<llvm::DenseMap<const llvm::Metadata*, llvm::TrackingMDRef> >
Alignment tool test cases
It should be pretty easy to create.
Error configuring libelf during install on Debian 9
This is probably due to my distribution being too recent (Debian 9, the problem does not appear on Ubuntu 16.04, I guess it would on recent Ubuntu versions). Still it might be a good idea to fix this if we want to support newer distributions.
The toolchian installation fails at the configure step for libelf:
/* ... */
configuring libelf (x86_64)...
checking whether make sets $(MAKE)... yes
checking for gcc... gcc
checking whether the C compiler works... no
configure: error: in `/home/pierre/Desktop/popcorn-compiler/lib/libelf':
configure: error: C compiler cannot create executables
See `config.log' for more details
Could not configure libelf(Command 'CFLAGS="-O3 -ffunction-sections -fdata-sections -specs /usr/local/popcorn/x86_64/lib/musl-gcc.specs" ./configure --prefix=/usr/local/popcorn/x86_64 --enable-elf64 --disable-shared --enable-extended-format' returned non-zero exit status 77)!
In config.log:
configure:2799: checking whether the C compiler works
configure:2821: gcc -O3 -ffunction-sections -fdata-sections -specs /usr/local/popcorn/x86_64/lib/musl-gcc.specs conftest.c >&5
/usr/bin/ld: /usr/local/popcorn/x86_64/lib/crt1.o: relocation R_X86_64_32S against symbol `_fini' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/6/crtbegin.o: relocation R_X86_64_32 against hidden symbol `__TMC_END__' can not be used when making a shared object
/usr/bin/ld: /usr/local/popcorn/x86_64/lib/libc.a(__libc_start_main.o): relocation R_X86_64_32 against symbol `src_env___libc_start_main__str__dev_null_' can not be used when makin
g a shared object; recompile with -fPIC
/usr/bin/ld: /usr/local/popcorn/x86_64/lib/libc.a(exit.o): relocation R_X86_64_32 against undefined symbol `__fini_array_end' can not be used when making a shared object; recompile
with -fPIC
/usr/bin/ld: /usr/local/popcorn/x86_64/lib/libc.a(__init_tls.o): relocation R_X86_64_32 against `.bss.src_env___init_tls_c_builtin_tls' can not be used when making a shared object;
recompile with -fPIC
/usr/bin/ld: final link failed: Nonrepresentable section on output
collect2: error: ld returned 1 exit status
Unhandled level 2 translation fault
I'm seeing unhandled level 2 translation fault errors when running my experiments. As near as I can tell, they occur on ARM only when trying to load a value from memory, where the address if offset from an address that was previously loaded with an adrp instruction. After migration, however, it appears that the initial address is not reloaded into a register appropriately.
I've only been able to reproduce this bug when I run 50+ applications at once, which makes it difficult for me to provide useful debugging information. I've attached what I can provide:
- The binaries that produce the bug
- The output of
-mllvm -debug-only=stacktransform -mllvm -debug-only=-insert-stackmapsfor each binary - The stacktransform.log output (note, it's highly interspersed due to running so many applications at once)
- The kernel log showing the faulting programs (also highly interspersed).
Here's a concise snippet of the kernel log showing the error:
[331956.720769] CG_A[7194]: unhandled level 2 translation fault (11) at 0x00000028, esr 0x92000006
[331956.720774] pgd = ffff810f19800000
[331956.724325] [00000028] *pgd=0000010f838de003, *pud=0000010f37302003, *pmd=0000000000000000
[331956.732794] CPU: 10 PID: 7194 Comm: CG_A Tainted: G OE 4.4.55+ #18
[331956.732797] Hardware name: Penguin Computing Valkre 2000/MT60-SC4, BIOS T48 10/02/2017
[331956.732801] task: ffff810f836ee800 ti: ffff810f44690000 task.ti: ffff810f44690000
[331956.732805] PC is at 0x502294
[331956.732808] LR is at 0x5022c0
[331956.732811] pc : [<0000000000502294>] lr : [<00000000005022c0>] pstate: 80000000
[331956.732814] sp : 00007fffffbfec70
[331956.732817] x29: 00007fffffbfecf0 x28: 0000000000000000
[331956.732821] x27: 0000000000000000 x26: 0000000000000000
[331956.732826] x25: 0000000000000000 x24: 0000000000000000
[331956.732830] x23: 0000000000000000 x22: 0000000000000000
[331956.732835] x21: 0000000000000000 x20: 0000000000000000
[331956.732839] x19: 0000000000000000 x18: 00007fffffbfc2a0
[331956.732844] x17: 00007fffffbfc30f x16: 0000000000606000
[331956.732848] x15: 0000000000000000 x14: 0000000000000000
[331956.732852] x13: cccccccccccccccd x12: 0000000000000034
[331956.732857] x11: 00007fffffbfe496 x10: 00007fffffffdeb0
[331956.732861] x9 : 000000000000000b x8 : 00000000fffffff7
[331956.732866] x7 : 0000000000000000 x6 : 0000000000000000
[331956.732870] x5 : 0000000000000000 x4 : 0000000000000000
[331956.732874] x3 : 0000000000000000 x2 : 0000000000000000
[331956.732878] x1 : 0000000000000000 x0 : 0000000000000108
[331956.732903] EXITED [7194] remote / 0xb
Everything is in the attached tarball.
Standardize the names of object files created by the popcorn compiler
For the purpose of symbols alignment, I need to be able to map an ARM object file to its x86 counterpart. At the level I'm working I can only compare the file names (present in the MAP file, indicating for each symbol the object file of origin). For now, if my application is composed of a single C file, let's say pierre.c, the compiler will produce:
pierre.o (ARM)
pierre_x86_64.o (x86)
This is not really standardized, it would probably be better to create for ARM pierre_arm.o. And define a similar extension for Power. I guess the Makefile template should be modified.
Soft floating-point emulation routines leaking into aarch64
GCC soft float library routines are leaking into musl, which can cause unhandled level 3 translation faults in programs. E.g., NPB's EP:
Feb 20 09:05:11 fox6 kernel: [66009.923530] EP_A[3919]: unhandled level 3 translation fault (11) at 0x00522330, esr 0x83000007
Feb 20 09:05:11 fox6 kernel: [66009.923538] pgd = ffff810f42255000
Feb 20 09:05:11 fox6 kernel: [66009.927041] [00522330] *pgd=0000010f13e68003, *pud=0000010f1c041003, *pmd=0000010f2017a003, *pte=0000000000000000
Feb 20 09:05:11 fox6 kernel: [66009.937485]
Feb 20 09:05:11 fox6 kernel: [66009.937494] CPU: 7 PID: 3919 Comm: EP_A Tainted: G OE 4.4.55+ #17
Feb 20 09:05:11 fox6 kernel: [66009.937497] Hardware name: Penguin Computing Valkre 2000/MT60-SC4, BIOS T48 10/02/2017
Feb 20 09:05:11 fox6 kernel: [66009.937501] task: ffff810f13e51a00 ti: ffff810f1bdac000 task.ti: ffff810f1bdac000
Feb 20 09:05:11 fox6 kernel: [66009.937506] PC is at 0x522330
Feb 20 09:05:11 fox6 kernel: [66009.937509] LR is at 0x508004
Feb 20 09:05:11 fox6 kernel: [66009.937512] pc : [<0000000000522330>] lr : [<0000000000508004>] pstate: a0000000
Feb 20 09:05:11 fox6 kernel: [66009.937515] sp : 00007fffffbfc8c0
Feb 20 09:05:11 fox6 kernel: [66009.937518] x29: 00007fffffbfc8d0 x28: 0000000000000000
Feb 20 09:05:11 fox6 kernel: [66009.937522] x27: 0000000000601061 x26: 0000000000000000
Feb 20 09:05:11 fox6 kernel: [66009.937527] x25: 00007fffffbfea80 x24: 0000000000601066
Feb 20 09:05:11 fox6 kernel: [66009.937532] x23: 000000007fffffff x22: 0000000000000000
Feb 20 09:05:11 fox6 kernel: [66009.937536] x21: 000000000000000f x20: 0000000000000001
Feb 20 09:05:11 fox6 kernel: [66009.937541] x19: 00007fffffbfcb30 x18: 00007fffffbfcaf0
Feb 20 09:05:11 fox6 kernel: [66009.937545] x17: 00007fffffbfcb5f x16: 0000000000604000
Feb 20 09:05:11 fox6 kernel: [66009.937550] x15: 000000000000000f x14: 0000000000601320
Feb 20 09:05:11 fox6 kernel: [66009.937554] x13: 000000000000003a x12: 000000007fffffff
Feb 20 09:05:11 fox6 kernel: [66009.937559] x11: 000000000000000a x10: 00000000ffffff90
Feb 20 09:05:11 fox6 kernel: [66009.937563] x9 : 00007fffffbfec70 x8 : 00007fffffbfebf0
Feb 20 09:05:11 fox6 kernel: [66009.937568] x7 : 0000000000000001 x6 : 000000000060102f
Feb 20 09:05:11 fox6 kernel: [66009.937572] x5 : 000000000060102f x4 : 0000000000000001
Feb 20 09:05:11 fox6 kernel: [66009.937577] x3 : 0000000000012889 x2 : 00007fffffbfe9e0
Feb 20 09:05:11 fox6 kernel: [66009.937581] x1 : 0000000000000019 x0 : 00007fffffbfcb30
Running objdump shows:
0000000000522330 <__extenddftf2>:
522330: 9e660000 fmov x0, d0
522334: d53b4401 mrs x1, fpcr
522338: d374f801 ubfx
As for where this gets called -- one call-chain (there may be others) is through pop_arg in musl-1.18/src/stdio/vfprintf.c at line 128:
108 static void pop_arg(union arg *arg, int type, va_list *ap)
109 {
110 switch (type) {
111 case PTR: arg->p = va_arg(*ap, void *);
112 break; case INT: arg->i = va_arg(*ap, int);
113 break; case UINT: arg->i = va_arg(*ap, unsigned int);
114 break; case LONG: arg->i = va_arg(*ap, long);
115 break; case ULONG: arg->i = va_arg(*ap, unsigned long);
116 break; case ULLONG: arg->i = va_arg(*ap, unsigned long long);
117 break; case SHORT: arg->i = (short)va_arg(*ap, int);
118 break; case USHORT: arg->i = (unsigned short)va_arg(*ap, int);
119 break; case CHAR: arg->i = (signed char)va_arg(*ap, int);
120 break; case UCHAR: arg->i = (unsigned char)va_arg(*ap, int);
121 break; case LLONG: arg->i = va_arg(*ap, long long);
122 break; case SIZET: arg->i = va_arg(*ap, size_t);
123 break; case IMAX: arg->i = va_arg(*ap, intmax_t);
124 break; case UMAX: arg->i = va_arg(*ap, uintmax_t);
125 break; case PDIFF: arg->i = va_arg(*ap, ptrdiff_t);
126 break; case UIPTR: arg->i = (uintptr_t)va_arg(*ap, void *);
127 break; case DBL: arg->f = va_arg(*ap, double);
128 break; case LDBL: arg->f = va_arg(*ap, long double);
129 }
130 }
This line appears to get executed whenever printf (or related functions) gets called with the %lf conversion specifier (based on a gdb trace I performed). I tried removing the long modifier (l), but that didn't seem to change the behavior.
I've been investigating how to address this -- ideally, we could eliminate quad precision (128-bit floating point) types and force long doubles to be the same size as doubles (which is allowed by the C standard). That said, I have no idea how to achieve that as of right now.
Changing musl Makefile to use -O0 explicitly causes arm64 STLXR -> LDAXR pair to fail on real machines
In case anyone makes this mistake like I did. This will cause atomic arm64 instructions STLXR and LDAXR in a_fetch_and() or a_fetch_or() musl functions to fail and get stuck in an infinite loop. These are called in malloc().
popcorn-compiler/lib/musl-1.1.18/Makefile
Line 121 in b009131
Stack transform timing utility architecture mismatch
At line 293 in lib/stack_transformation/utils/stack_transform_timing.h:
#define TIME_REWRITE( aarch64_bin, x86_64_bin ) \
({ \
int ret; \
struct timespec start = { .tv_sec = 0, .tv_nsec = 0 }; \
struct timespec init = { .tv_sec = 0, .tv_nsec = 0 }; \
struct timespec rewrite = { .tv_sec = 0, .tv_nsec = 0 }; \
struct timespec end = { .tv_sec = 0, .tv_nsec = 0 }; \
struct regset_x86_64 regset; \
struct regset_powerpc64 regset_dest; \
stack_bounds bounds = get_stack_bounds(); \
READ_REGS_X86_64(regset); \
regset.rip = get_call_site(); \
clock_gettime(CLOCK_MONOTONIC, &start); \
st_handle src = st_init(x86_64_bin); \
st_handle dest = st_init(powerpc64_bin); \
...
From the arguments, this macro wants to compare the stack rewriting time for AArch64 and x86-64 binaries; internally, however, the macro uses regset_powerpc64 and powerpc64_bin.
Relocation Overflow Error
Problem
While trying to compile NPB-OMP, I encountered this error. The problem is that some NPB benchmarks uses large tables, which lead to this type of error. This error comes up in at least to benchs: MG, FT with CLASS C.
(EDIT: This problem happens also in the SERIAL version of NPB! Wondering why no one raised the red flag?)
Potential Solution
Rob suggested to use the -mcmodel=large option to overcome this problem. Trying it on the application alone led to a linking error. Which make sense, since we are linking binaries with different models. Trying to recompile all libraries with the large model led to another error signaled by CLang: "ELF TLS only supported in small memory model"! This error happens while compiling libopenpop (OpenMP library).
(EDIT: probably mcmodel=large may work on the serial version of NPB since TLS is not used)
Wondering how things work with GCC for native compilation?
EDIT: they use shared libraries+ "-mcmodel=medium" (an explanation of why it work at: https://software.intel.com/en-us/articles/avoiding-relocation-errors-when-building-applications-with-large-global-or-static-data-on-intel64).
Unable to build bodytrack with clang
The attached zip file has the cpp file where it supposedly failed and the output file.
tmp.tar.gz
Replication:
- in parsec-3.0/pkgs/apps/bodytrack/parsec, create a copy of gcc-pthreads.bldconf and name it popcorn.bldconf
- in popcorn.bldconf, add "-popcorn-migratable" as an argument in build-env
- ./bin/bldconfadd -f -n popcorn -c gcc
- open parsec-3.0/config/popcorn.bldconf and where it says
export CC="${CC_HOME}/bin/gcc"
export CXX="${CC_HOME}/bin/g++"
change it to
export CC="/usr/local/popcorn/bin/clang"
export CXX="/usr/local/popcorn/bin/clang++"
- ./bin/parsecmgt -a fulluninstall -p bodytrack
- ./bin/parsecmgt -a build -p bodytrack -c popcorn
- errors
libelf: different code paths on aarch64 compared to x86-64
There is one macro (HAVE_MMAP) that is not defined when the libelf is configured for aarch64, but defined for x86-64. This leads to some weird behavior on aarch64, observed on at least one application: FT with input B; where the application allocates almost 4 GB of memory.
This simple patch seems to resolve the problem:
diff --git a/lib/libelf/configure b/lib/libelf/configure
index 82b68f5..f2cdea8 100755
--- a/lib/libelf/configure
+++ b/lib/libelf/configure
@@ -4868,11 +4868,11 @@ fi
fi
{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_func_mmap_fixed_mapped" >&5
$as_echo "$ac_cv_func_mmap_fixed_mapped" >&6; }
-if test $ac_cv_func_mmap_fixed_mapped = yes; then
+#if test $ac_cv_func_mmap_fixed_mapped = yes; then
$as_echo "#define HAVE_MMAP 1" >>confdefs.h
-fi
+#fi
rm -f conftest.mmap conftest.txt
for ac_func in ftruncate memcmp memcpy memmove
Argv argument table corruption
It appears that the *argv table get corrupted after a migration. This simple program should demonstrate the problem:
#include <stdio.h>
#include <migrate.h>
int main(int argc, char *argv[])
{
int i;
migrate(1, NULL, NULL);
for(i=0; i<argc; i++)
printf("Arg %d is %s\n", i, argv[i]);
printf("Done! \n");
}
Wrong value when passing "struct stat" as a parameter to a function
Problem description
Hi I got this simple test file when debugging the issue 36:
#include <stdio.h>
#include <unistd.h>
#include <sys/mman.h>
#include <sys/stat.h> // struct stat
#include <fcntl.h> // O_RDONLY
void print_stat(struct stat sbuf)
//void print_stat(struct stat *sbuf)
{
printf(">> stat sbuf.st_mode: 0x%x\n", sbuf.st_mode);
//printf("stat sbuf.st_mode: 0x%x\n", sbuf->st_mode);
}
int main(int argc, char *argv[])
{
struct stat sbuf;
int fd = open("fstat.c", O_RDONLY, 0);
if (fd < 0) {
printf("error");
return 1;
}
fstat(fd, &sbuf);
printf("stat sbuf.st_mode: 0x%x\n", sbuf.st_mode);
printf("sizeof(struct stat): %lu, %lu\n", sizeof(sbuf), sizeof(struct stat));
print_stat(sbuf);
//print_stat(&sbuf);
close(fd);
return 0;
}
For some reason, it has the following results (the value of st_mode is changed):
$ ./2_x86-64
stat sbuf.st_mode: 0x81a4
sizeof(struct stat): 128, 128
>> stat sbuf.st_mode: 0x3a297461
Environment
popcorn-compiler: from commit 4ffd16e, patched with the code from issue 36.
Compilation hangs on NPB/ua/mason.c
Compiling NPB3.3-SER-C-FLAT/ua/mason.c hangs without any progress for minutes.
Versions:
- Compiler: master / ce78b7a
- NPB: make-popcorn-explicit / 9f4662d9107993d2ca2322e55519801aa0f5387c
GCC constructor/destructor attributes do not execute on AArch64
GCC function attributes __attribute__((constructor)) and __attribute__((destructor)) (see here for description) do not execute when starting or ending (respectively) on AArch64 nodes. This is a known issue and documented here for posterity (because I always forget).
Segmentation fault during homogeneous execution on aarch64
Running any of the NPB Serial applications on aarch64 without migrating (i.e., homogeneously) fails before reaching main.
- Application compiles correctly with and without
-popcorn-migratableflag. - Application runs correctly without
-popcorn-migratableflag, but fails with it.
Example:
- Compiler,
masterbranch (commit: 821c6ee) - SNU_NPB-1.0.3,
make-popcornbranch (commit: ca129d29c9d3b01dcfe2c4c2345e0dee4fb7b1fe) - EP, class A
- Default compilation settings (see
EP/Makefile)
dmesg output when running EP_A_aarch64 (gzipped binary attached):
[146742.464428] CPU: 2 PID: 3106 Comm: EP_A_aarch64 Not tainted 4.4.55+ #1
[146742.464429] Hardware name: AppliedMicro (R) AppliedMicro X-C2 (TM) EV-883408-X2-MXB-1 /AppliedMicro X-C2 , BIOS 5.11 08/27/2016
[146742.464431] task: ffffffc7da06a280 ti: ffffffc7d85a4000 task.ti: ffffffc7d85a4000
[146742.464434] PC is at 0x50fd54
[146742.464435] LR is at 0x50fcfc
[146742.464437] pc : [<000000000050fd54>] lr : [<000000000050fcfc>] pstate: 60000000
[146742.464438] sp : 0000007fffffecb0
[146742.464439] x29: 0000007fffffecb0 x28: 0000000000000000
[146742.464441] x27: 0000007fffffecd0 x26: 0000000000000000
[146742.464444] x25: 0000000000000000 x24: 0000000000000000
[146742.464446] x23: 0000000000000001 x22: 0000008000000000
[146742.464448] x21: 0000007fff800000 x20: 0000000000b02040
[146742.464450] x19: 0000000000b02020 x18: 0000007fffffcaf0
[146742.464452] x17: 000000005d6b6361 x16: 0000007fffffe990
[146742.464455] x15: 0000007fffffe9b0 x14: 0000000000000006
[146742.464457] x13: 0000000000000000 x12: 0000000000000000
[146742.464459] x11: 8080808000000000 x10: 000000000000004c
[146742.464461] x9 : 000000000000004f x8 : 000000000000004c
[146742.464463] x7 : 0000007ffffff110 x6 : 0000007ffffff100
[146742.464465] x5 : 0000007ffffff118 x4 : 0000007ffffff108
[146742.464467] x3 : 0000007fffffece0 x2 : 0000000000000014
[146742.464469] x1 : 5f706f7000746e69 x0 : 0000000000000001
GDB output:
[anthony@xgene2 09:42 ~]$ gdb ./EP_A_aarch64
GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.5) 7.11.1
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "aarch64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./EP_A_aarch64...done.
(gdb) run
Starting program: /home/anthony/EP_A_aarch64
Program received signal SIGSEGV, Segmentation fault.
0x000000000050fd54 in elf_version ()
(gdb) bt
#0 0x000000000050fd54 in elf_version ()
#1 0x0000000000509c18 in __st_userspace_ctor () at src/userspace.c:103
#2 0x0000000000502640 in libc_start_init () at src/env/__libc_start_main.c:61
#3 0x0000000000502690 in __libc_start_main (main=0x501210 <main>, argc=1, argv=0x7ffffff1d8)
at src/env/__libc_start_main.c:75
#4 0x0000000000502434 in _start_c (p=<optimized out>) at crt/crt1.c:17
#5 0x0000000000000000 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb)
TLS alignment lead to a not aligned stack on aarch64
When using TLS, the size of the section may lead to having a not aligned stack on aarch64: the TLS is allocated at the same time as the stack within pthread_create().
This problem is confirmed by the kernel dmesg message:
"SP Alignment exception: pc=0000000000508d10 sp=00007ffff6b2a9a8"
A test that show the problem is in the "popcorn-test" repos in the "tls-test" test.
Two possible solutions:
- Align the TLS section in the LDSCRIPT
- Modify the musl library to make sure that the stack is aligned
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.