Code Monkey home page Code Monkey logo

vespa-3rdparty-deps's Introduction

vespa-3rdparty-deps

Patched version of software distributions used in Vespa:

  • boost
  • gtest
  • openssl
  • protobuf

These packages are currently built and distributed on Copr

vespa-3rdparty-deps's People

Contributors

baldersheim avatar arnej27959 avatar toregge avatar vekterli avatar aressem avatar gjoranv avatar geirst avatar bjorncs avatar olaaun avatar mpolden avatar hakonhall avatar lesters avatar hmusum avatar havardpe avatar andreer avatar jonmv avatar

Watchers

Øyvind Grønnesby avatar Eirik Nygaard avatar James Cloos avatar Gleb Sizov avatar Brandon avatar Yahoo CLA Bot avatar  avatar Yngve Aasheim avatar Kristian Aune avatar Grant Irvin avatar Jon Bratseth avatar  avatar Masakazu Nagaya avatar Johan Solbakken avatar  avatar  avatar  avatar Bjørn Meland avatar  avatar  avatar

vespa-3rdparty-deps's Issues

CUDA in Vespa Docker Container

I am trying to use the onnxruntime with CUDA acceleration as described in this article.
However, after starting the docker container with the following Dockerfile:

FROM vespaengine/vespa

USER root

RUN dnf clean all

RUN dnf -y install 'dnf-command(config-manager)'
RUN dnf config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/rhel8/x86_64/cuda-rhel8.repo

RUN dnf -y install $(rpm -q --queryformat '%{NAME}-cuda-%{VERSION}' vespa-onnxruntime)

USER vespa

I get the error message (35) that the CUDA driver version is insufficient. This is the log excerpt from vespa-logfmt:

[2024-07-10 10:15:07.835] WARNING container        Container.ai.djl.util.cuda.CudaUtils Failed to detect GPU count: CUDA driver version is insufficient for CUDA runtime version (35)
[2024-07-10 10:15:07.836] WARNING container        Container.ai.djl.util.cuda.CudaUtils Failed to detect GPU count: CUDA driver version is insufficient for CUDA runtime version (35)
[2024-07-10 10:15:07.836] INFO    container        Container.ai.djl.util.Platform       Found matching platform from: bundle://3c149058-7591-414a-a874-e9111ef9eadd_23.0:3/native/lib/tokenizers.properties
[2024-07-10 10:15:09.064] WARNING container        stderr       terminate called after throwing an instance of 'onnxruntime::OnnxRuntimeException'
[2024-07-10 10:15:09.064] WARNING container        stderr         what():  /builddir/build/BUILD/vespa-onnxruntime-1.18.0/onnxruntime/contrib_ops/cuda/bert/tensorrt_fused_multihead_attention/cudaDriverWrapper.cc:42 onnxruntime::contrib::cuda::CUDADriverWrapper::CUDADriverWrapper() handle != nullptr was false.

The container toolkit on the host is installed and CUDA is functional in other images.

(base) ➜  ~ docker run --rm -it --gpus=all pytorch/pytorch:2.2.1-cuda12.1-cudnn8-runtime nvidia-smi
Wed Jul 10 11:14:00 2024
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.90.07              Driver Version: 550.90.07      CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4090        Off |   00000000:01:00.0 Off |                  Off |
| 45%   49C    P8             22W /  450W |    6872MiB /  24564MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
|   1  NVIDIA GeForce RTX 4090        Off |   00000000:03:00.0 Off |                  Off |
|  0%   46C    P8             22W /  450W |    5164MiB /  24564MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
+-----------------------------------------------------------------------------------------+

I tried manually updating the CUDA drivers inside the vespa container but that prevented the vespa application from starting.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.