dionhaefner / pyhpc-benchmarks Goto Github PK

A suite of benchmarks for CPU and GPU performance of the most popular high-performance libraries for Python :rocket:

License: The Unlicense

Python 96.61% Jupyter Notebook 2.71% Shell 0.68%

benchmarks python tensorflow pytorch jax cupy parallel-computing high-performance-computing gpu

pyhpc-benchmarks's Issues

Compare with DaCe framework?

DaCe (https://github.com/spcl/dace) is a parallel computing framework that also support Numpy frontend, similar to JAX and Numba. It runs on CPU/GPU/FPGA. Would be interesting to add it for comparison!

Compare with TACO Python binding

The Tensor Algebra Compiler (https://github.com/tensor-compiler/taco) seems to be good at sparse/dense linear algebra and has Python frontend: http://tensor-compiler.org/docs/pycomputations/index.html

Add TPU support

Jax now supports using the TPU on Google Colab. It would be nice to try how it performs out of the box.

Compare with an MLIR-based stencil DSL

This project https://github.com/spcl/open-earth-compiler/ provides a DSL frontend for stencil/PDE programs, and rely on MLIR & LLVM to run on NVIDIA and AMD GPUs. It is not a Python frontend, but can be called from Python I think (see https://arxiv.org/abs/2005.13014)

Error while installing GPU environment

Using:
conda env create -f environment-gpu.yml

leads to an error saying:

Exception:

  =========================================================
  The "tensorflow-gpu" package has been removed!

  Please install "tensorflow" instead.

  Other than the name, the two packages have been identical
  since TensorFlow 2.1, or roughly since Sep 2019. For more
  information, see: pypi.org/project/tensorflow-gpu
  =========================================================


  [end of output]

Adding Weld to comparison

https://github.com/weld-project/weld can accelerate Python/numpy code by optimizing data movement.

deprecated jax ops: index_update

the call
for backend in jax; do python run.py benchmarks/isoneutral_mixing/ --device gpu -b $backend -b numpy; done
yields

dTdz = jax.ops.index_update(
AttributeError: module 'jax.ops' has no attribute 'index_update'

and indeed index_update is no longer a thing: https://jax.readthedocs.io/en/latest/jax.ops.html

fastmath

Hi @dionhaefner, great comparisons, thanks for that! Out of interest. Did you ever try to run numba with fastmath=True; does it make any difference, and if, how much?

Adding Transonic to comparison

https://github.com/fluiddyn/transonic can accelerate Python/Numpy code and run on accelerators

turbulent_kinetic_energy returns inconsistent results

I am working on #14.
The command has inconsistent result output:

$ python run.py -r 2 -s 1048576 --device cpu -b pytorch benchmarks/turbulent_kinetic_energy/

Using pytorch version 1.13.0.dev20220617+cu113
Running 3 benchmarks...  [------------------------------------]    0%Error: inconsistent results for size 1048576
Error: inconsistent results for size 1048576
Error: inconsistent results for size 1048576
Running 3 benchmarks...  [####################################]  100%

benchmarks.turbulent_kinetic_energy
===================================
Running on CPU

size          backend     calls     mean      stdev     min       25%       median    75%       max       Δ
------------------------------------------------------------------------------------------------------------------
   1,048,576  pytorch            2     0.573     0.028     0.544     0.559     0.573     0.587     0.601     1.000

(time in wall seconds, less is better)

Looks like two consecutive runs will generate inconsistent results for turbulent_kinetic_energy. I guess the root cause is this line: https://github.com/dionhaefner/pyhpc-benchmarks/blob/master/benchmarks/turbulent_kinetic_energy/tke_pytorch.py#L264

There could be non-deterministic numeric results when running mask = tke[2:-2, 2:-2, -1, taup1] < 0.0

dionhaefner / pyhpc-benchmarks Goto Github PK

pyhpc-benchmarks's Issues

Compare with DaCe framework?

Compare with TACO Python binding

Add TPU support

Compare with an MLIR-based stencil DSL

Error while installing GPU environment

Adding Weld to comparison

deprecated jax ops: index_update

fastmath

Adding Transonic to comparison

turbulent_kinetic_energy returns inconsistent results

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent