Update benchmarks for 8.0 about openmm-org HOT 25 CLOSED

A TONS OF benchmarks

Hardware details:

OS details:
Ubuntu 22.04.3 LTS, Linux 6.2.0-26-generic x86_64, GNU 11.4.0

GPU driver & toolchain:
AMD GPU driver version 6.1.5.50600-1609671, ROCm 5.6.0;
NVIDIA GPU driver 535.86.05, CUDA Toolkit 11.8

OpenMM version:
OpenMM 8.0.0 – OpenMM HIP Plugin 8Mar2023, natively built from source code.

Results:

The performance of AMD GPUs is impressive. However, the performance of NVIDIA Ampere consumer GPUs on larger systems is not satisfactory, 3080Ti/3090 is almost equalized by AMD Radeon VII.

Compared to previous versions, ROCm 5.6.0 has some performance improvements for smaller systems. The performance of the RDNA3 GPU (7900XTX) is not very stable, and occasionally the GPU scheduling is not active.

It is worth mentioning that tests by @muziqaz (openmm/openmm#3338 (comment), his updated tables) over the past few days have shown that for larger systems, the 7900XTX may perform better under Windows (compare to my results under Linux).

BTW, I will be posting a series of blogs of "Switch to AMD" for MD simulation APPs recently, with 4 mainstream APPs: GROMACS, Amber, LAMMPS and OpenMM. The first forum for posting may be the Computational Chemistry Commune, I am also looking for other suitable forums, do anyone have any suggestions?

from openmm-org.

A100:

H100:

4080:

from openmm-org.

It is interesting to see that OCL is stronger on Windows than on Linux.

But it's still much slower than HIP. Many vendors may be abandoning OpenCL, or is SYCL a more modern solution? GROMACS has switched to SYCL.

Now that I ran out of my GPUs on windows I will try few variations of few suggestions by @bdenhollander to maybe improve performance.

Doing so many benchmarks should be a lot of work, waiting for your good news!

@Entropy-Enthalpy do you mind if I add your results to my table?

Sure! Feel free to use them, just mention the source of the data.

Finally, 7900xtx is running with the big dogs, though 4090 performance on small systems is unbelievable

I thought the reason the 4090 works so well on small systems is because the clock frequency of the GPU is so high. All RTX 40 series GPUs can reach ~2.8GHz when running MD simulations at full load. For comparison, Ampere consumer GPUs are ~1.9GHz. In addition, RTX 40xx GPUs even have 100-200MHz overclocking space.

This may also be the reason for the unsatisfactory performance of the H100 PCIe.

from openmm-org.

I posted the aforementioned blog post just now: http://bbs.keinsci.com/thread-39269-1-1.html

from openmm-org.

Here are the numbers for A100.

I also ran the cellulose and STMV benchmarks on multiple devices. Using two A100s:

Using three:

Using four:

from openmm-org.

If anyone else can run benchmarks on other GPUs, that would be great. It would be good present an assortment of Turing and newer GPUs. If anyone has a recent high end AMD GPU, that would also be good to include.

from openmm-org.

This is for RTX 3090:

from openmm-org.

Here are times on H100:

It's significantly faster than A100. Compared to 4080 it's a little faster on the large systems but slower on the small systems. I also tested using multiple GPUs.

Two:

Three:

Four:

The scaling is less good than the A100s, and it gets slower with four GPUs.

I'll aim to update the benchmarks on the website in the next day or two.

from openmm-org.

I missed a few benchmarks that we have on the website. I'll run them for the hardware I have access to. @sef43 could you run a few more benchmarks on the 3090?

python benchmark.py --platform=CUDA --style=table --test=gbsa,rf,apoa1rf,apoa1ljpme

from openmm-org.

Tagging @hmacdope. We are working on benchmarks on consumer grade GPUs via the FAH core version. It looks like our benchmarks project drifted out of sync with the OpenMM benchmarks, so I'll regerate these and we can get extensive consumer grade card statistics as well.

from openmm-org.

@peastman: I'm updating the Folding@home benchmarks project so we can also collect statistics across all consumer-grade cards, and want to confirm that our "standard" benchmarks are intended to be the following:

python benchmark.py --platform=CUDA --style=table --test=gbsa,pme,rf,apoa1rf,apoa1pme,apoa1ljpme,amber20-cellulose,amber20-stmv,amoebapme

Is that correct?

Are we sure we do not want any NPT or mixed-precision benchmarks?

from openmm-org.

It depends what you mean by "standard". The ones you listed are the ones we post on the website. But if you're interested in tracking other things, that's fine too.

from openmm-org.

@sef43 could you run the extra benchmarks on the 3090? That's all I need before I can update the website.

from openmm-org.

Extra benchmarks on 3090:

from openmm-org.

Thanks!

from openmm-org.

Here are times on H100:

Run 1 Run 2 Run 3 Average
pme 1499.74 1502.61 1502.4 1501.5833
apoa1pme 609.277 609.299 608.524 609.0333
amber20-cellulose 162.21 161.718 161.612 161.8467
amber20-stmv 47.8135 47.7666 47.8271 47.8024
amoebapme 32.1292 32.0891 32.1591 32.1258

I guess it's the H100 PCIe version, right...? Because I recently conducted a benchmark test using the H100 PCIe and got similar results.

The H100 comes in several SKUs, with the most common ones being the H100 SXM5 and H100 PCIe. According to the Hopper architecture whitepaper (page 18), there is a significant difference in the chip specifications between these two SKUs, resulting in a notable difference in FP64/FP32 performance (page 20).

As such, it is important to specify the particular SKU on the webpage.

from openmm-org.

You're right. I thought they were the NVLINK version, but it seems they're actually PCIe. That explains why the scaling with multiple GPUs was worse.

from openmm-org.

Here are RTX 4090 benchmarks.

Command:
python benchmark.py --platform=CUDA --style=table --test=gbsa,rf,pme,apoa1rf,apoa1pme,apoa1ljpme,amoebapme,amber20-cellulose,amber20-stmv

Results:

Obviously, RTX 4090 is currently the fastest single GPU in any system, and in the two large systems, amber20-cellulose and amber20-stmv, the RTX 4090 is 20-25% faster than H100 PCIe.

BTW, I'll also benchmark some AMD GPUs using the latest OpenMM-HIP, should I post them here?

from openmm-org.

BTW, I'll also benchmark some AMD GPUs using the latest OpenMM-HIP, should I post them here?

That would be great!

from openmm-org.

The updates are in #98.

from openmm-org.

Great stuff there. Thank you.
Yes, we have been toying with Windows HIP SDK. There are few bugs and issues, but nothing major show stopping. AMD seems to be aware of them.
It is interesting to see that OCL is stronger on Windows than on Linux.
Now that I ran out of my GPUs on windows I will try few variations of few suggestions by @bdenhollander to maybe improve performance.
@Entropy-Enthalpy do you mind if I add your results to my table?
Finally, 7900xtx is running with the big dogs, though 4090 performance on small systems is unbelievable

from openmm-org.

OCL being stronger on Windows rather than Linux is just an internal joke, as nothing ever is faster on Windows compared to Linux 🤣
I believe FAH will still use OCL as a fallback, though with AMD removing more and more older GPUs from drivers, there isn't much of the reason to do that.
7900xtx is running at 2.9-3ghz too while HIPing ;)
I think the reason 4090 is so good, is CUDA's compiler maturity. And arch is more suited probably. On windows it would struggle for sure.
It would be great for Gromacs to look into HIP too, maybe through SYCL, to get AMD GPUs working. Their OCL implementation completely ignore AMD as a hardware device.

from openmm-org.

@muziqaz I think I've figured out why your results of the two larger systems (amber20-cellulose and amber20-stmv) are higher than mine. As you mentioned previously (amd/openmm-hip#5 (comment)), your card is Sapphire Nitro+, which has much higher TGP and clocks (420W, 2680MHz) than my MSI GAMING TRIO CLASSIC (355W, 2500MHz).

If you have time, you could try switching the VBIOS of your Sapphire Nitro+ to the "Silent" mode (using the physical switch on the card) and doing the benchmark again. The "Silent" mode has the same TGP and clocks as MBA model and MSI GAMING TRIO CLASSIC.

from openmm-org.

The difference between your amber results and mine are within normal variation between runs.
The rest of the tests your card runs faster than mine

from openmm-org.

Those are great numbers. I hope that eventually we'll be able to include the HIP platform as part of the main OpenMM package. The main barrier right now is that HIP isn't supported by conda-forge.

from openmm-org.