Code Monkey home page Code Monkey logo

raplcap's Introduction

Powercap Sysfs C Bindings and Utilities

This project provides the powercap library -- a generic C interface to the Linux power capping framework (sysfs interface). It also provides the following applications:

  • powercap-info - view powercap control type hierarchies or zone/constraint-specific configurations
  • powercap-set - set powercap control type zone/constraint-specific configurations

The aforementioned library and applications should be compatible with all Linux powercap drivers. The library also includes an API, originally created for use with RAPLCap, specifically for managing Intel Running Average Power Limit (RAPL).

If using this project for other scientific works or publications, please reference:

  • Connor Imes, Huazhe Zhang, Kevin Zhao, Henry Hoffmann. "CoPPer: Soft Real-time Application Performance Using Hardware Power Capping". In: IEEE International Conference on Autonomic Computing (ICAC). 2019. DOI: https://doi.org/10.1109/ICAC.2019.00015

    BibTex
    @inproceedings{imes2019copper,
      author={Imes, Connor and Zhang, Huazhe and Zhao, Kevin and Hoffmann, Henry},
      booktitle={2019 IEEE International Conference on Autonomic Computing (ICAC)},
      title={{CoPPer}: Soft Real-Time Application Performance Using Hardware Power Capping},
      year={2019},
      pages={31-41},
      doi={10.1109/ICAC.2019.00015}
    }

Prerequisites

The Linux power capping framework was released with Linux kernel 3.13. You must be running this kernel or newer with the configs CONFIG_POWERCAP and CONFIG_INTEL_RAPL enabled to use the Intel RAPL driver.

To use the intel-rapl control type, ensure that the appropriate kernel module is loaded. Run with proper privileges:

modprobe intel_rapl_msr

Or on kernels older than 5.3:

modprobe intel_rapl

Power Capping

Modern hardware is constrained by power and temperature limitations, often quantified as Thermal Design Power. In processors and other clock-driven hardware components, power consumption P is proportional to capacitance C, the square of the voltage V, and clock frequency f: P ~ C * V^2 * f. A popular mechanism for balancing performance and power consumption is Dynamic Voltage and Frequency Scaling (DVFS). For compute-bound applications, DVFS provides a linear relationship between frequency and performance. However, power is non-linear with frequency since an increase in frequency also requires an increase in voltage.

Although the relationship between performance and power is more difficult to model, hardware can be better at optimizing voltage and frequency than software while still respecting a power cap over a time window. Power capping allows a system administrator to configure an upper limit on the power consumption of various hardware components while letting the hardware more efficiently manage voltage and frequency. Setting a power cap does NOT imply that the component will actually consume that power, only that it will not violate that limit on average over the specified time window.

Usage

Applications

See the man pages or run the applications with the -h or --help option for instructions.

Library

First, there is the powercap-sysfs.h interface for reading/writing to sysfs without the need to maintain state. This is reasonable for simple use cases. See the header files for documentation.

The powercap.h interface provides read/write functions for generic powercap zone and constraint file sets. Users are responsible for managing memory and populating the structs with file descriptors (e.g., code that wrap this interface performs zone/constraint discovery and file descriptor management).

The powercap-rapl.h interface discovers RAPL instances, power zones, and constraints (i.e., long_term and short_term constraints). Users are responsible for managing memory, but the library will manage discovering, opening, and closing files within instances.

Basic lifecycle example:

  // get number of top-level (parent) RAPL instances
  uint32_t count = powercap_rapl_get_num_instances();
  if (count == 0) {
    // none found (maybe the kernel module isn't loaded?)
    perror("powercap_rapl_get_num_instances")
    return -1;
  }
  powercap_rapl_pkg* pkgs = malloc(count * sizeof(powercap_rapl_pkg));
  // initialize
  uint32_t i;
  for (i = 0; i < count; i++) {
    if (powercap_rapl_init(i, &pkgs[i], 0)) {
      // could be that you don't have write privileges
      perror("powercap_rapl_init");
      return -1;
    }
  }
  // do a bunch of stuff with the interface here,
  // e.g., enable desired zones and get/set power caps...
  // now cleanup
  for (i = 0; i < count; i++) {
    if (powercap_rapl_destroy(&pkgs[i])) {
      perror("powercap_rapl_destroy");
    }
  }
  free(pkgs);

Additional Comments

The interfaces do NOT guarantee that values are actually accepted by the kernel, they only notice errors if I/O operations fail. It is recommended that, at least during development/debugging, users read back to see if their write operations were successful.

Additionally, the kernel sysfs bindings (and thus the powercap-rapl interface) do NOT guarantee that RAPL instances are presented in any particular order. For example, the first instance (sysfs directory intel-rapl:0) on a dual-socket system may actually provide access to package-1 instead of package-0, and vice versa. In cases where order matters, e.g., when sockets are managed asymmetrically, the user is responsible for ensuring that the correct powercap instance is being operated on, e.g., by checking its name with powercap_rapl_get_name(...). More concretely, in the example above, powercap_rapl_get_name(&pkgs[0], POWERCAP_RAPL_ZONE_PACKAGE, ...) gives name package-1, and powercap_rapl_get_name(&pkgs[1], POWERCAP_RAPL_ZONE_PACKAGE, ...) is package-0. It might be helpful to sort the pkgs array after initialization (see the powercap implementation of RAPLCap for an example).

Finally, the powercap-rapl interface exposes functions for files that are not (currently) supported by RAPL in order to be compliant with the powercap interface. Use the powercap_rapl_is_zone_file_supported(...) and powercap_rapl_is_constraint_file_supported(...) functions to check in advance if you are unsure if a zone or constraint file is supported. Furthermore, files may exist but always return an error code for some zones or constraints, e.g., the constraint max_power_uw file (powercap_rapl_get_max_power_uw(...)) for zones other than POWERCAP_RAPL_ZONE_PACKAGE.

Building

Compiling

This project uses CMake.

To build, run:

mkdir _build
cd _build
cmake ..
make

To create a shared object library as a release build, specify for cmake:

cmake .. -DBUILD_SHARED_LIBS=On -DCMAKE_BUILD_TYPE=Release

Installing

To install, run with proper privileges:

make install

On Linux, installation typically places libraries in /usr/local/lib and header files in /usr/local/include.

Uninstalling

Install must be run before uninstalling in order to have a manifest. To uninstall, run with proper privileges:

make uninstall

Cross Compiling

To cross-compile for different systems/architectures, use standard CMake toolchain files. See Mastering CMake for reference.

For example, modify the cmake command from the build directory to use your own toolchain.cmake file:

cmake -DCMAKE_TOOLCHAIN_FILE=/path/to/toolchain.cmake ..

Project Source

Find this and related project sources at the powercap organization on GitHub.
This project originates at: https://github.com/powercap/powercap

Bug reports and pull requests for bug fixes and enhancements are welcome.

License

This project is developed by Connor Imes. It is released under the 3-Clause BSD License.

Thanks

Special thanks to Henry Hoffmann (University of Chicago) and Steven Hofmeyr (Lawrence Berkeley National Laboratory) for advising and supporting projects that this code was originally developed for.

raplcap's People

Contributors

cimes-isi avatar connorimes avatar girardm avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

raplcap's Issues

Supporting power limit = 0

This issue is filed mostly to document the idea, although it is unlikely to be implemented.

Currently there is only one way to set a power value to 0, however undesirable that may be. A user must specify a really small watts (still > DBL_EPSILON) value that is ultimately rounded down when converting to microwatts or truncated at the bit-level when writing to MSRs. We might consider allowing watts=0 and only ignoring the watts and seconds fields when they are negative. This would require, in the very least:

  • A silent interface change (resulting in a major version bump and loss of backward compatibility)!
  • An additional burden on the user to set negative values (not as convenient as 0)
  • Changing interface description of and implementations of raplcap_set_limits(...)
  • Updates to rapl-configure (possibly with the addition of a -e/--enabled flag instead of accepting negative values. We note, however, that not all implementations support direct enabling/disabling, e.g., raplcap-libmsr.

Some interfaces might support seconds=0, but in reality this would result in the smallest possible time window due to how the bit values for time windows in MSRs are interpreted.

Intel i9-13900HX is not supported.

Hi there!

It was a long and torturous road that led me here.
My i9-13900HX CPU in a Sager laptop is getting heavily throttled in Linux.

This utility is telling me my CPU is not supported:

[evert@Evert ~]$ sudo rapl-configure-powercap 
[ERROR] [raplcap-powercap] powercap_intel_rapl_get_time_window_us: No data available
Failed to get peak power limit: No data available

And:

[evert@Evert ~]$ sudo rapl-configure-msr 
[ERROR] [raplcap-msr] CPU not supported: Family=6, Model=B7
Failed to initialize: Operation not supported

Rapl-info is providing some useful info:

[evert@Evert ~]$ rapl-info 
Zone 0
  name: package-0
  enabled: 1
  max_energy_range_uj: 262143328850
  Constraint 0
    name: long_term
    power_limit_uw: 135000000
    time_window_us: 95944704
    max_power_uw: 55000000
  Constraint 1
    name: short_term
    power_limit_uw: 162000000
    time_window_us: 2440
    max_power_uw: 0
  Constraint 2
    name: peak_power
    power_limit_uw: 315000000
    max_power_uw: 0
  Subzone 0
    name: core
    enabled: 0
    max_energy_range_uj: 262143328850
    Constraint 0
      name: long_term
      power_limit_uw: 0
      time_window_us: 976
  Subzone 1
    name: uncore
    enabled: 0
    max_energy_range_uj: 262143328850
    Constraint 0
      name: long_term
      power_limit_uw: 0
      time_window_us: 976
Zone 1
  name: psys
  enabled: 1
  max_energy_range_uj: 262143328850
  Constraint 0
    name: long_term
    power_limit_uw: 280000000
    time_window_us: 95944704
  Constraint 1
    name: short_term
    power_limit_uw: 300000000
    time_window_us: 976

msr: ATOM_SILVERMONT_D energy units appear to be incorrect

The energy values on ATOM_SILVERMONT_D (model 0x4D) appear to be too large. When used for measuring power, it produces values > 10k Watts.

The Intel SDM Vol. 4 (as of December 2021) documents in Table 2-10 that the energy unit multiplier is 2^ESU microJoules, like with some other Atom CPUs. RAPLCap currently implements energy measurements as the SDM documents, but we see:

$ rapl-configure-msr        
      enabled: true
   watts_long: 15.000000000000
 seconds_long: 10.000000000000
  watts_short: 18.000000000000
seconds_short: 0.009765625000
       joules: 211875753.099263995886
   joules_max: 281474976.710655987263

However, if we use the standard energy multiplier of 1/2^ESU Joules, we see:

$ rapl-configure-msr      
      enabled: true
   watts_long: 15.000000000000
 seconds_long: 10.000000000000
  watts_short: 18.000000000000
seconds_short: 0.009765625000
       joules: 49373.058593750000
   joules_max: 65536.000000000000

which is much more reasonable. Measuring power then gives values < 10 W.

It appears that the Intel SDM is incorrect.

PKG power limit

Hi,

after i set the PKG power limit to 55 watts and run an application whose peak power is over this value, the measured power consumption is about 50 watts, i.e. 5 watts difference. What is happening?

Hardware: Broadwell E5-2650V4
Release 0.1.1
RAPL setting: two time windows with 1 seconds, PKG to 55 watts
Measured power with: perf stat .... sleep 1

Best,
Bo

[raplcap-libmsr] May show Package as disabled if only one constraint is set/enabled

libmsr enables Package long and short term constraints separately, but doesn't offer a function to check enable statuses. We only say a Package or Platform zone is enabled if both long and short term are enabled.

This is likely only a problem if a constraint was actually disabled, as Package constraints are usually enabled by default.

FYI, it looks like the kernel sysfs interface via raplcap-powercap only checks a single (long term) constraint to determine enabled status. For example, see the 4.13 kernel release: https://github.com/torvalds/linux/blob/v4.13/drivers/powercap/intel_rapl.c#L362

[raplcap-libmsr] Can't set power value separately on power planes that haven’t been enabled before when seconds are not also set

This is a weird issue that we won't likely try to do anything to fix. It's probably a bug in libmsr, and they have removed support for PP0 and PP1 power planes.

Note entirely sure what versions of libmsr this affects. Based on my poorly written notes about this, it seem like the problem is that both the watts and seconds fields have to be set when a zone hasn't been enabled before. libmsr returns seconds=0 when we read before setting a new watts value in isolation, so we’re trying to set seconds=0 if the raplcap user didn't explicitly request a positive seconds value. The problem is exacerbated by not being able to enable the zone prior to setting values. Perhaps we'd have to set values twice?

I noted an error like:

Error: <libmsr> Invalid bit field values: calc_rapl_bits(): Seconds value is too large: (null):/local/third-party-tools/libmsr/src/msr_rapl.c::440

raplcap_set_zone_enabled in README

Hi,
I tried to use the library on my system by using the example provided in the README.
However, to let it work I had to add 'raplcap_set_zone_enabled(..)' before setting the cap. Adding it to the example would probably be helpful for other users.

Best,
Daniele

[powercap] PSYS is a top-level zone, not a subzone of PACKAGE

Currently the raplcap-powercap implementation expects all top-level zones to be PACKAGE type. With the introduction of platforms that actually support PSYS, we see that this is not the case -- PSYS zones are actually top-level zones like PACKAGE.

This also screws up the raplcap API that is centered around sockets. That problem will be addressed later in another issue, which will require API changes.

msr: Ice Lake Xeon CPUs: incorrect DRAM power limit lock bit, no clamping bit

The April 2021 Intel Software Developer's Manual Volume 4 adds Table 2-47 in Section 2.17.6 with new information on the MSR_DRAM_POWER_LIMIT register for Ice Lake Xeon CPUs (0x6A and 0x6C). Per the combined volume Section 14.10.5, bit 31 is the standard lock bit, but this particular Ice Lake documentation says it's at bit 63.

Additionally, there does not appear to be a clamping bit (bit 16 is now Reserved).

It would be really nice if somebody could confirm the correct behavior, since I don't have any Ice Lake Xeon CPUs to test.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.