This link (from README.md) is broken: https://software.intel.com/en-us/articles/runtime-performance-optimization-blueprint-intel-architecture-optimization-with-large-code

The current solution uses a linker script that works with GNU ld but does not with ld.gold and Clang ld. Use a solution that avoids the linker script

Starting point: Running an executable that was linked with -Wl,-zcommon-page-size=2097152 -Wl,-zmax-page-size=2097152 under IODLR_USE_EXPLICIT_HP=1 LD_PRELOAD=/path/to/liblppreload.so.

Expectation: The code mapping is completely mapped to 2 MiB hugepages.

Actual result:

1. The first 2 MiB are still on 4 KiB pages.
2. There is a tail of 1.5 MiB or so that is still on 4 KiB pages.

Discussion:

1. This is due to the liblppreload library parsing the ELF headers and sh_addr begins 20 KiB after the actual 2 MiB aligned r-xp memory mapping start address and thus the library aligns sh_addr up to the next 2 MiB boundary.
2. This is caused by the linker aligning the start addresses to 2 MiB but not the segment size. However, in my tests, the address space after the code segment end and the next segment start wasn't mapped such that the end-address could actually be aligned up instead of down.

An alternative and arguably simpler approach to traversing ELF headers is to parse /proc/$pid/smaps and remap complete r-xp mappings to huge pages. That way one can also detect whether it's is safe to align the end of a mapping up to the next 2 MiB boundary.

The current reference implementation only loads the .text section into 2M pages.. Add a new API to load a DSO into 2M pages.

sudo -E docker-compose build --build-arg http_proxy=$http_proxy --build-arg https_proxy=$https_proxy --build-arg no_proxy=$no_proxy --no-cache

............
Step 67/73 : RUN rm -rf nginx && git clone --depth 1 -b $ASYNCH_NGINX_TAG https://github.com/intel/asynch_mode_nginx.git nginx && cd nginx && ./configure --prefix=/usr/ --with-http_ssl_module --add-dynamic-module=modules/nginx_qat_module --with-cc-opt="-DNGX_SECURE_MEM -I /opt/openssl/1.1.1k/include -Wno-error=deprecated-declarations -Wimplicit-fallthrough=0" --with-ld-opt="-Wl,-rpath=/opt/openssl/1.1.1k/lib -L /opt/openssl/1.1.1k/lib" && make -j && sudo make install && rm -rf /home/${USERNAME}/nginx_build
---> Running in 8d9a9186c3ae
fatal: could not create work tree dir 'nginx': Permission denied
The command '/bin/sh -c rm -rf nginx && git clone --depth 1 -b $ASYNCH_NGINX_TAG https://github.com/intel/asynch_mode_nginx.git nginx && cd nginx && ./configure --prefix=/usr/ --with-http_ssl_module --add-dynamic-module=modules/nginx_qat_module --with-cc-opt="-DNGX_SECURE_MEM -I /opt/openssl/1.1.1k/include -Wno-error=deprecated-declarations -Wimplicit-fallthrough=0" --with-ld-opt="-Wl,-rpath=/opt/openssl/1.1.1k/lib -L /opt/openssl/1.1.1k/lib" && make -j && sudo make install && rm -rf /home/${USERNAME}/nginx_build' returned a non-zero code: 128
ERROR: Service 'wp4.2_php7.4_opt_https' failed to build : Build failed

The metrics for l2_demand_code_MPI and l1_code_read_MPI are inter changed. Fix this.

This document: https://software.intel.com/content/dam/develop/external/us/en/documents/runtimeperformanceoptimizationblueprint-largecodepages-q1update.pdf

Gives the following example:
measure-perf-metric.sh -p 69772 -t 30 -m ITLB_stalls

Which gives an error, because "ITLB" is capitalized and the script requires lower case.

The current reference implementation is in C++. It can only be integrated into C++ code.

Provide a C implementation which can be integrated into both C and C++ code.

What about .data and .bss section? Can I enable hugepages on those?

Below is the bug hint:
$ measure-perf-metric.sh -p 9650 -t 30 -m itlb_mpki
Initializing for metric: itlb_mpki
Checking if process with pid 9650 exists...
Profile application with process id: 9650
Collect perf data for 30 seconds
perf stat -e cycles,instructions,itlb_misses.walk_completed,itlb_misses.walk_completed_4k,itlb_ms.walk_completed_2m_4m,itlb_misses.walk_completed_1g

Calculating metric for: itlb_mpki

Final itlb_mpki metric

FORMULA: metric_ITLB_MPKI(%) = 1000*(a/b)
where, a=itlb_misses.walk_completed
b=instructions

ERROR: metric_ITLB_MPKI can't be derived. Missing pmus

Q ? my perf version is 5.4, I just test on a zsh process (id 9650)

Tools like perf are no longer able to follow the symbols after the .text is remapped and the perf output will not have the symbols like in the following figure.

Figure : perf output will not have the proper symbols after Large Page Mapping

1.35% node perf-12142.map [.] 0x0000562eb19e86aa
1.19% node perf-12142.map [.] 0x0000562eb19e8803
0.71% node perf-12142.map [.] 0x0000562eb19e891f

The solution is to add additional code before the remapping and provide the static symbols to perf in /tmp/perf-PID.map.

Hi,

I try to use liblppreload.so with a rust program and get this error:

symbol lookup error: /usr/lib64/liblppreload.so: undefined symbol: gettid

What did I miss?

It would be nice to have support to use explict hugepages with this library for total control for the end user.

Current version of the IODLR code is causing SEGFAULT due to mismatch in the error string table. This happened after making new update to the IODLR library but after debugging the iodlr-code I was able to deduce it to the missing error string in failure case. The error string is ""map_read_exe_header_failed".
I've a fix ready. I'll soon submit a PR addressing this issue.

I tried to build "wp4.2_php7.4_base_querycacheoff_http". But it fails with the error messages below.

docker-compose build --build-arg http_proxy=$http_proxy --build-arg https_proxy=$https_proxy --build-arg no_proxy=$no_proxy --no-cache wp4.2_php7.4_base_querycacheoff_http

Step 21/31 : RUN echo "soft nofile 1000000\nhard nofile 1000000" | sudo tee -a /etc/security/limits.conf
---> Running in 4564fbdeacf4
soft nofile 1000000
hard nofile 1000000
Removing intermediate container 4564fbdeacf4
---> 38061c225a4e
Step 22/31 : COPY files/1s-bkm.j2 /home/${USERNAME}
---> 0edadcc0a1cd
Step 23/31 : COPY files/2s-bkm.j2 /home/${USERNAME}
---> d7f8830f7bba
Step 24/31 : RUN sudo cp /home/${USERNAME}/2s-bkm.j2 /etc/mysql/my.cnf
---> Running in 959cadd53374
Removing intermediate container 959cadd53374
---> 6f2581fe5fe8
Step 25/31 : RUN sudo service mysql start && sleep 1 && sudo mysqladmin -u root password "" && sudo mysql -u root -e "CREATE USER 'wp_bench'@'localhost' IDENTIFIED BY 'wp_bench'" && sudo mysql -u root -e "GRANT ALL PRIVILEGES on . to 'wp_bench'@'localhost' IDENTIFIED BY 'wp_bench'" && sudo mysql -u root -e "CREATE DATABASE wp_bench" && sudo mysql -u root -e "FLUSH PRIVILEGES" && sudo service mysql stop
---> Running in d6a900e201f9

• Starting MariaDB database server mysqld
  ...done.
  mysqladmin:
  You cannot use 'password' command as mysqld runs
  with grant tables disabled (was started with --skip-grant-tables).
  Use: "mysqladmin flush-privileges password '*'" instead
  ERROR: Service 'wp4.2_php7.4_base_querycacheoff_http' failed to build: The command '/bin/sh -c sudo service mysql start && sleep 1 && sudo mysqladmin -u root password "" && sudo mysql -u root -e "CREATE USER 'wp_bench'@'localhost' IDENTIFIED BY 'wp_bench'" && sudo mysql -u root -e "GRANT ALL PRIVILEGES on . to 'wp_bench'@'localhost' IDENTIFIED BY 'wp_bench'" && sudo mysql -u root -e "CREATE DATABASE wp_bench" && sudo mysql -u root -e "FLUSH PRIVILEGES" && sudo service mysql stop' returned a non-zero code: 255

the metric for the scripts in tools folder does not work anymore for Alder Lake, could you make an update for it?
There are many big differences for the Alder Lake perf events.

It fails with the following message:

Transparent Huge Pages are enabled, mapping ...
Failed to map: map region too small

I followed the readme at: accordingly I compiled the shared library (make -f Makefile.preload), and copied it to /usr/lib64.
Then when I try running applications by loading the library, I'm getting the following error:

$ LD_PRELOAD=/usr/lib64/liblppreload.so <app> 
Mapping to large pages failed: invalid region boundaries

I tested this on Ubuntu 20.04 and 18.04 (x86_64).
I also I have huge-page enabled on the machines and tried with [madvise] and [always] settings, all with the same outcome giving the error above.
This happens with all the applications that I tried.

in https://github.com/intel/iodlr/tree/master/containers/wordpress#docker-proxy, should we require user to become 'root' to use proxy? That is, 'sudo' should not be required.

In https://github.com/intel/iodlr/tree/master/containers/wordpress#siege-lifting-hang, better to add some description or code snippet to show how to change Siege concurrency number in our quickrun.sh or run.sh. Some users, like me, might not be familiar with Siege usage.

There are several places in the code where int64_t or uint64_t are used. Change these to use uintptr_t instead.

static inline uint64_t largepage_align_up(uint64_t addr) {
  return (((addr) + (hps) - 1) & ~((hps) - 1));
}

static inline uint64_t largepage_align_down(uint64_t addr) {
  return ((addr) & ~((hps) - 1));
}

I did a quick check of "wp4.2_php7.4_opt_querycacheon_https_opt" and I'm getting socket errors. I simply ran ./quickrun.sh without any additional parameters.

here are the errors:

**************************
** 2021-07-27 21:46:55 UTC
** Starting Nginx
nginx: [alert] could not open error log file: open() "/usr//logs/error.log" failed (13: Permission denied)
2021/07/27 21:46:55 [emerg] 608#0: dlopen() "/home/base/nginx_install/modules/ngx_ssl_engine_qat_module.so" failed (/home/base/nginx_install/modules/ngx_ssl_engine_qat_module.so: cannot open shared object file: No such file or directory) in /tmp/hhvm-nginxVpa9D1/nginx.conf:3

**************************
** 2021-07-27 21:46:55 UTC
** Starting PHP Engine
Creating PHP FPM config[27-Jul-2021 21:46:55] NOTICE: fpm is running, pid 610
[27-Jul-2021 21:46:55] NOTICE: ready to handle connections

**************************
** 2021-07-27 21:46:56 UTC
** Unfreezing framework
PHP Warning:  file_get_contents(https://e06c9281d9d5:8090/): failed to open stream: Connection refused in /home/base/oss-performance/targets/wordpress/WordpressTarget.php on line 108
Warning: file_get_contents(https://e06c9281d9d5:8090/): failed to open stream: Connection refused in /home/base/oss-performance/targets/wordpress/WordpressTarget.php on line 108
PHP Warning:  file_get_contents(https://e06c9281d9d5:8090/): failed to open stream: Connection refused in /home/base/oss-performance/targets/wordpress/WordpressTarget.php on line 108
Warning: file_get_contents(https://e06c9281d9d5:8090/): failed to open stream: Connection refused in /home/base/oss-performance/targets/wordpress/WordpressTarget.php on line 108
PHP Warning:  file_get_contents(https://e06c9281d9d5:8090/): failed to open stream: Connection refused in /home/base/oss-performance/targets/wordpress/WordpressTarget.php on line 108
Warning: file_get_contents(https://e06c9281d9d5:8090/): failed to open stream: Connection refused in /home/base/oss-performance/targets/wordpress/WordpressTarget.php on line 108

**************************
** 2021-07-27 21:46:56 UTC
** Running sanity check
PHP Warning:  file_get_contents(https://e06c9281d9d5:8090/): failed to open stream: Connection refused in /home/base/oss-performance/base/PerfTarget.php on line 29
Warning: file_get_contents(https://e06c9281d9d5:8090/): failed to open stream: Connection refused in /home/base/oss-performance/base/PerfTarget.php on line 29

*******************************************
** 2021-07-27 21:46:56 UTC
** Starting Siege for single request warmup
/usr/local/bin/siege.config: 32: cannot create /.siege/siege.conf: Directory nonexistent
** SIEGE 4.0.9
** Preparing 1 concurrent users for battle.
The server is now under siege...
[error] socket: unable to connect sock.c:282: Connection refused
[error] socket: unable to connect sock.c:282: Connection refused
[error] socket: unable to connect sock.c:282: Connection refused
[error] socket: unable to connect sock.c:282: Connection refused
..

******************************************
** 2021-07-27 21:46:56 UTC
** Starting Siege for multi request warmup
/usr/local/bin/siege.config: 32: cannot create /.siege/siege.conf: Directory nonexistent
** SIEGE 4.0.9
** Preparing 200 concurrent users for battle.
The server is now under siege...
[error] socket: unable to connect sock.c:282: Connection refused
[error] socket: unable to connect sock.c:282: Connection refused
[error] socket: unable to connect sock.c:282: Connection refused
[error] socket: unable to connect sock.c:282: Connection refused
...

ERROR: Unable to open cookies file: (null)/.siege/cookies.txt

******************************************
** 2021-07-27 21:46:56 UTC
** Starting Siege for multi request warmup
/usr/local/bin/siege.config: 32: cannot create /.siege/siege.conf: Directory nonexistent
** SIEGE 4.0.9
** Preparing 200 concurrent users for battle.
The server is now under siege...
[error] socket: unable to connect sock.c:282: Connection refused
[error] socket: unable to connect sock.c:282: Connection refused
[error] socket: unable to connect sock.c:282: Connection refused
[error] socket: unable to connect sock.c:282: Connection refused
...

*******************************
** 2021-07-27 21:46:57 UTC
** Starting Siege for benchmark
/usr/local/bin/siege.config: 32: cannot create /.siege/siege.conf: Directory nonexistent
** SIEGE 4.0.9
** Preparing 200 concurrent users for battle.
The server is now under siege...
[error] socket: unable to connect sock.c:282: Connection refused
[error] socket: unable to connect sock.c:282: Connection refused
[error] socket: unable to connect sock.c:282: Connection refused
[error] socket: unable to connect sock.c:282: Connection refused
...