Code Monkey home page Code Monkey logo

benchmetrics's Introduction

Benchmarks Metrics

This repository contains both dynamic and static data mined from the LLVM Test Suite and SPEC CPU programs. For each benchmark collection, Perf Stat counter metrics and Milepost Features were collected and categorized based on the CLANG optimization flags used to compile the benchmarks. The optimization flags used were: -O0, -O1, -O2, and -O3. The programs of SPEC CPU were run using the 'train' size with one iteration.

Linux Perf

Linux Perf is a suite of performance monitoring tools available on Linux systems, enabling users to collect and analyze detailed performance data. The tool used to extract our data is perf stat, which uses Performance Monitoring Units (PMUs) to record hardware events during code execution.

GCC Milepost Features

Milepost is a set of 56 program features, introduced by Fursin et al. These features include number of basic blocks, number of edges in the control flow graph, number of function calls, etc (see Table 2 in the original paper). To collect the features, we have used the MilePost implementation available in the YaCoS compiler. The data that we report in this repository is the same data that we have used in two other papers to explore the space of compiler optimizations in clang, by Faustino et al and by Damasio et al.

Environment Details

Hardware

  • CPU: Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz
  • CPU Cores Number: 10
  • Threads Number: 20
  • L1 Cache Size: 10 x 32 KB instruction caches and 10 x 32 KB data caches
  • L2 Cache Size: 10 x 256 KB
  • L3 Cache Size: 25 MB
  • RAM: 32GB

Software

  • Operating System: Ubuntu 20.04.6 LTS
  • Python Version: 3.8.107
  • SPEC CPU: 2017
  • LLVM Build: 17.0.6
  • Clang/Clang++: 17.0.6
  • LLVM Test-Suite Build:: Release 17
  • Linux Perf: 5.4.269

External Links

References

  1. Thaís Damásio, Michael Canesche, Vinícius Pacheco, Marcus Botacin, Anderson Faustino da Silva, Fernando Magno Quintão Pereira: A Game-Based Framework to Compare Program Classifiers and Evaders. CGO 2023: 108-121 -- This paper uses MilePost features to find out if two programs solve the same task.

  2. Anderson Faustino da Silva, Edson Borin, Fernando Magno Quintão Pereira, Nilton Luiz Queiroz Junior, Otávio Oliveira Napoli: Program representations for predictive compilation: State of affairs in the early 20's. J. Comput. Lang. 73: 101171 (2022) -- This paper compares the precision of the MilePost features against other program embeddings in various different tasks.

  3. Grigori Fursin, Yuriy Kashnikov, Abdul Wahid Memon, Zbigniew Chamski, Olivier Temam, Mircea Namolaru, Elad Yom-Tov, Bilha Mendelson, Ayal Zaks, Eric Courtois, François Bodin, Phil Barnard, Elton Ashton, Edwin V. Bonilla, John Thomson, Christopher K. I. Williams, Michael F. P. O'Boyle: Milepost GCC: Machine Learning Enabled Self-tuning Compiler. Int. J. Parallel Program. 39(3): 296-327 (2011) -- The original description of the MilePost features (see Table 2).

  4. André Felipe Zanella, Anderson Faustino da Silva, Fernando Magno Quintão Pereira: YACOS: a Complete Infrastructure to the Design and Exploration of Code Optimization Sequences. SBLP 2020: 56-63 -- This paper describes the implementation of the LLVM pass that we have used to collect the MilePost features.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.