Code Monkey home page Code Monkey logo

rust-ptx-builder's Introduction

Rust PTX Builder

Build Status Current Version Docs

New Release: 0.5 ๐ŸŽ‰

Say goodbye to proxy crate approach

This allows us to use single-source CUDA in binary-only crates (ones without lib.rs).

Development breaking changes

The crate does not provide a default panic_handler anymore. From now on, it either up to a user, or other crates (e.g. coming soon ptx-support crate).

Next workaround should work in common cases, although it doesn't provide any panic details in runtime:

#![feature(core_intrinsics)]

#[panic_handler]
unsafe fn breakpoint_panic_handler(_: &::core::panic::PanicInfo) -> ! {
    core::intrinsics::breakpoint();
    core::hint::unreachable_unchecked();
}

API Breaking Changes - less boilerplate code

build.rs script was never so compact and clear before:

use ptx_builder::error::Result;
use ptx_builder::prelude::*;

fn main() -> Result<()> {
    let builder = Builder::new(".")?;
    CargoAdapter::with_env_var("KERNEL_PTX_PATH").build(builder);
}

Documentation improvements

This release comes with a significant documentation improvement! Check it out :)

Purpose

The library should facilitate CUDA development with Rust. It can be used in a cargo build script of a host crate, and take responsibility for building device crates.

Features

  1. Obviously, device crates building.
  2. Announcing device crates sources to cargo, so it can automatically rebuild after changes.
  3. Reporting about missing tools, for example:
[PTX] Unable to get target details
[PTX]
[PTX] caused by:
[PTX]   Command not found in PATH: 'rust-ptx-linker'. You can install it with: 'cargo install ptx-linker'.

Prerequirements

The library depends on a fresh Nightly and ptx-linker. The latter can be installed from crates.io:

cargo install ptx-linker

Usage

First, you need to specify a build script in host crate's Cargo.toml and declare the library as a build-dependency:

[build-dependencies]
ptx-builder = "0.5"

Then, typical build.rs might look like:

use ptx_builder::error::Result;
use ptx_builder::prelude::*;

fn main() -> Result<()> {
    let builder = Builder::new(".")?;
    CargoAdapter::with_env_var("KERNEL_PTX_PATH").build(builder);
}

rust-ptx-builder's People

Contributors

dbeckwith avatar denzp avatar gnzlbg avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

rust-ptx-builder's Issues

error: Linking globals named 'memcpy': symbol multiply defined!

I am getting that linker error when trying to use the coresimd library from a cuda kernel :/

Compiling host v0.1.0 (/stdsimd/crates/nvptx-test)                                                                                                                                          
error: failed to run custom build command for `host v0.1.0 (/Users/gnzlbg/projects/sideprojects/stdsimd/crates/nvptx-test)`                                                                                                       
process didn't exit successfully: `/stdsimd/crates/nvptx-test/target/debug/build/host-5b9ca8737382a47e/build-script-build` (exit code: 1)
--- stderr
[PTX] Unable to build a PTX crate!
[PTX]    Compiling kernel v0.1.0 (/stdsimd/crates/nvptx-test/kernel)
[PTX]    Compiling proxy v0.0.0 (/private/var/folders/mm/y444qqwd0kn52fycyz6nsph40000gn/T/ptx-builder-0.5/kernel/c4e147d83c46a030)
[PTX] error: linking with `ptx-linker` failed: exit code: 1
[PTX]   |
[PTX]   = note: "ptx-linker" "-L" "/Users/gnzlbg/.xargo/lib/rustlib/nvptx64-nvidia-cuda/lib" "/var/folders/mm/y444qqwd0kn52fycyz6nsph40000gn/T/ptx-builder-0.5/kernel/c4e147d83c46a030/target/nvptx64-nvidia-cuda/release/deps/proxy.proxy.43qsgff7-cgu.0.rcgu.o" "-o" "/var/folders/mm/y444qqwd0kn52fycyz6nsph40000gn/T/ptx-builder-0.5/kernel/c4e147d83c46a030/target/nvptx64-nvidia-cuda/release/deps/proxy.ptx" "/var/folders/mm/y444qqwd0kn52fycyz6nsph40000gn/T/ptx-builder-0.5/kernel/c4e147d83c46a030/target/nvptx64-nvidia-cuda/release/deps/proxy.37oceufmlocs4b5j.rcgu.o" "-O1" "-L" "/var/folders/mm/y444qqwd0kn52fycyz6nsph40000gn/T/ptx-builder-0.5/kernel/c4e147d83c46a030/target/nvptx64-nvidia-cuda/release/deps" "-L" "/var/folders/mm/y444qqwd0kn52fycyz6nsph40000gn/T/ptx-builder-0.5/kernel/c4e147d83c46a030/target/release/deps" "-L" "/Users/gnzlbg/.xargo/lib/rustlib/nvptx64-nvidia-cuda/lib" "--start-group" "-Bstatic" "--whole-archive" "/var/folders/mm/y444qqwd0kn52fycyz6nsph40000gn/T/rustcNH6EGz/libkernel-ad470a324c1962cc.rlib" "--no-whole-archive" "--whole-archive" "/var/folders/mm/y444qqwd0kn52fycyz6nsph40000gn/T/rustcNH6EGz/libcoresimd-9761acf5e343c1f3.rlib" "--no-whole-archive" "--whole-archive" "/var/folders/mm/y444qqwd0kn52fycyz6nsph40000gn/T/rustcNH6EGz/libcore-58de11be20fe5a04.rlib" "--no-whole-archive" "--end-group" "/var/folders/mm/y444qqwd0kn52fycyz6nsph40000gn/T/rustcNH6EGz/libcompiler_builtins-20af736248a5f37a.rlib" "-shared" "-Bdynamic"
[PTX]   = note:  [INFO] Going to link 2 bitcode modules and 4 rlibs...
[PTX]                   
[PTX]           error: Linking globals named 'memcpy': symbol multiply defined!
[PTX]           
[PTX] 
[PTX] error: aborting due to previous error
[PTX] 
[PTX] error: Could not compile `proxy`.
[PTX] 
[PTX] To learn more, run the command again with --verbose.

to reproduce the issue:

git clone https://github.com/gnzlbg/stdsimd
cd stdsimd
git checkout nvptx_ci
cd crates/nvptx-test
cargo build

Debugging produced PTX?

I'm trying to figure out how to produce a debuggable binary from the PTX file that I can debug with the NVIDIA tools. It's looking like I'm going to need a non-release-mode version of the PTX file. Could you could expose whether to pass the --release option to xargo or not? Is this even the right thing to do for trying to debug a CUDA kernel produced with this tool?

Files aren't rebuilt if target is removed

Since the produced PTX file are stored in /tmp on Linux it is removed if the computer is rebooted.

However, this won't trigger a rebuild so when trying to build the crate I get an error about the file I try to include not being found.

LLVM ERROR: Module has aliases, which NVPTX does not support.

Hi,

Thanks for creating this great package! I have tried to get rust-ptx-builder to work, but I have not yet succeeded. I get the error 'LLVM ERROR: Module has aliases, which NVPTX does not support.'. To make sure that I was not an error on my part, I have tried to build one of the example projects in rust-ptx-support.

This is what I get:

$ cargo build
   Compiling cuda-println v0.1.0 (/export/scratch1/hendriks/projects/rustacuda_example/rust-ptx-support/examples/cuda-println)
error: failed to run custom build command for `cuda-println v0.1.0 (/export/scratch1/hendriks/projects/rustacuda_example/rust-ptx-support/examples/cuda-println)`
process didn't exit successfully: `/export/scratch1/hendriks/projects/rustacuda_example/rust-ptx-support/target/debug/build/cuda-println-0022aa50cae5c930/build-script-build` (exit code: 1)
--- stderr
[PTX] Unable to build a PTX crate!
[PTX]    Compiling cuda-println v0.1.0 (/export/scratch1/hendriks/projects/rustacuda_example/rust-ptx-support/examples/cuda-println)
[PTX] error: linking with `ptx-linker` failed: exit code: 1
[PTX]   |
[PTX]   = note: "ptx-linker" "-L" "/ufs/hendriks/.xargo/lib/rustlib/nvptx64-nvidia-cuda/lib" "/tmp/ptx-builder-0.5/cuda_println/d623bafb1eb4deca/nvptx64-nvidia-cuda/release/deps/cuda_println-01803fec47121bc1.cuda_println.2jagi3my-cgu.0.rcgu.o" "/tmp/ptx-builder-0.5/cuda_println/d623bafb1eb4deca/nvptx64-nvidia-cuda/release/deps/cuda_println-01803fec47121bc1.cuda_println.2jagi3my-cgu.1.rcgu.o" "-o" "/tmp/ptx-builder-0.5/cuda_println/d623bafb1eb4deca/nvptx64-nvidia-cuda/release/deps/cuda_println-01803fec47121bc1.ptx" "/tmp/ptx-builder-0.5/cuda_println/d623bafb1eb4deca/nvptx64-nvidia-cuda/release/deps/cuda_println-01803fec47121bc1.1w5kxn6gnyc7w18l.rcgu.o" "-O1" "-L" "/tmp/ptx-builder-0.5/cuda_println/d623bafb1eb4deca/nvptx64-nvidia-cuda/release/deps" "-L" "/tmp/ptx-builder-0.5/cuda_println/d623bafb1eb4deca/release/deps" "-L" "/ufs/hendriks/.xargo/lib/rustlib/nvptx64-nvidia-cuda/lib" "--start-group" "-Bstatic" "--whole-archive" "/tmp/rustchm5YG8/libptx_support-bac6370d784080de.rlib" "--no-whole-archive" "--whole-archive" "/tmp/rustchm5YG8/libcore-5040dc495f9f5fc0.rlib" "--no-whole-archive" "--end-group" "/tmp/rustchm5YG8/libcompiler_builtins-44064ef211638321.rlib" "-shared" "-Bdynamic"
[PTX]   = note:  [INFO] Going to link 3 bitcode modules and 3 rlibs...
[PTX]                   
[PTX]            [INFO] Linking with Link Time Optimisation
[PTX]           LLVM ERROR: Module has aliases, which NVPTX does not support.
[PTX]           
[PTX] 
[PTX] error: aborting due to previous error
[PTX] 
[PTX] error: Could not compile `cuda-println`.
[PTX] 

Unfortunately, I cannot seem to get more verbose output than this. I have installed xargo and ptx-linker. I have CUDA_LIBRARY_PATH set correctly. I do not have LLVM installed locally.

I tried with the following versions of rustc:

  • rustc 1.32.0-nightly (14997d56a 2018-12-05) failed: same reason
  • rustc 1.32.0-nightly (b3af09205 2018-12-04) failed: same reason
  • rustc 1.32.0-nightly (5aff30734 2018-11-19) failed: different reason - "note: Unable to find LLVM shared lib in possible locations:"
  • rustc 1.32.0-nightly (653da4fd0 2018-11-08) failed: different reason - "note: Unable to find LLVM shared lib in possible locations:"
  • rustc 1.32.0-nightly (36a50c29f 2018-11-09) failed: different reason - "note: Unable to find LLVM shared lib in possible locations:"
  • rustc 1.31.0-nightly (de9666f12 2018-10-31) failed: different reason - "note: Unable to find LLVM shared lib in possible locations:"

I am not sure which of these errors occurs earlier in the compilation process, the module alias error, or the LLVM shared lib error.

If there is anything I can do to make this issue more specific, please let me know!

npvtx-nvidia-cuda

I only see nvptx64-nvidia-cuda as a supported target, is it also possible to use rust-ptx-builder from a 32-bit host ? (e.g. i686-unknown-linux-gnu or i686-apple-darwin ?)

Error building chapter-1: can't find crate for compiler_builtins

Not sure what to do about this error. I believe I have everything set up correctly, I have xargo and ptx-linker installed. But when I tried to build the example in chapter 1 I get this error:

$ cargo bench
   Compiling backtrace-sys v0.1.16
   Compiling clap v2.31.2
   Compiling criterion-plot v0.1.3
   Compiling criterion-stats v0.1.3
   Compiling synstructure v0.6.1
   Compiling syn v0.12.15
   Compiling simplelog v0.4.4
   Compiling png v0.7.0
   Compiling failure_derive v0.1.1
   Compiling backtrace v0.3.5
   Compiling error-chain v0.11.0
   Compiling failure v0.1.1
   Compiling serde_derive_internals v0.22.2
   Compiling ptx-builder v0.3.1
   Compiling chapter-1 v0.1.0 (file://$HOME/Downloads/rust-inline-cuda-tutorial/chapter-1/host)
   Compiling serde_derive v1.0.36
error: failed to run custom build command for `chapter-1 v0.1.0 (file://$HOME/Downloads/rust-inline-cuda-tutorial/chapter-1/host)`
process didn't exit successfully: `$HOME/Downloads/rust-inline-cuda-tutorial/target/release/build/chapter-1-f9e9b6b73dea461b/build-script-build` (exit code: 1)
--- stderr
[PTX] Unable to build a PTX crate!
[PTX]     Updating git repository `https://github.com/nagisa/math.rs.git`
[PTX]     Updating registry `https://github.com/rust-lang/crates.io-index`
[PTX]    Compiling cty v0.1.5
[PTX]    Compiling nvptx-builtins v0.1.0
[PTX] error[E0463]: can't find crate for `compiler_builtins`
[PTX] 
[PTX] error: aborting due to previous error
[PTX] 
[PTX] For more information about this error, try `rustc --explain E0463`.
[PTX] error: Could not compile `cty`.
[PTX] warning: build failed, waiting for other jobs to finish...
[PTX] error[E0463]: can't find crate for `compiler_builtins`
[PTX] 
[PTX] error: aborting due to previous error
[PTX] 
[PTX] For more information about this error, try `rustc --explain E0463`.
[PTX] error: Could not compile `nvptx-builtins`.
[PTX] 
[PTX] To learn more, run the command again with --verbose.

warning: build failed, waiting for other jobs to finish...
error: build failed

This is with ptx-linker 0.5.1 and rustc 1.27.0-nightly (7360d6dd6 2018-04-15). Any ideas?

I love this project BTW, hope you continue the tutorial!

Perhaps elaborate on necessity of ProxyCrate

It seems confusing that the actual ptx file is generated somewhere in /tmp/ instead of the path given to Builder::new(). Could you explain more in the README? Is this permanent?

:O

??? ???
: O : O
???

Is there a crate somewhere using this ?

It would be nice to have a repository somewhere that shows how to use this.

E.g. a device crate with a kernel, and then a binary crate, that launches the kernel.

Ideally it should at least compile on travis.

Expose build status before building

It would nice if there was a build_needed() method or something on the Builder struct so I could check this value before actually building and do other things if necessary in the build script. Right now at least I want to print something to the console saying that it's building the PTX kernel, but it could be useful for other things as well.

Unable to find extra-filename rustc flag

ptx-builder is failing with "Unable to find extra-filename rustc flag" when I am building my project. When I run cargo rustc --target nvptx64-nvidia-cuda -v .. myself I get:

rustc --crate-name sidequest_core --edition=2018 core/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type cdylib --crate-type rlib --emit=dep-info,link -C opt-level=3 -Cembed-bitcode=no -C debuginfo=2 --crate-type cdylib -C metadata=d9bb1f26303a4731 --out-dir /side/Code/sidequest-meet/target/nvptx64-nvidia-cuda/release/deps --target nvptx64-nvidia-cuda -L dependency=/side/Code/sidequest-meet/target/nvptx64-nvidia-cuda/release/deps -L dependency=/side/Code/sidequest-meet/target/release/deps --extern fnv=/side/Code/sidequest-meet/target/nvptx64-nvidia-cuda/release/deps/libfnv-3eeaa6958d9eb02f.rlib --extern nalgebra=/side/Code/sidequest-meet/target/nvptx64-nvidia-cuda/release/deps/libnalgebra-f28e713a257293dd.rlib --extern num_traits=/side/Code/sidequest-meet/target/nvptx64-nvidia-cuda/release/deps/libnum_traits-72ce644de10ae911.rlib --extern rand=/side/Code/sidequest-meet/target/nvptx64-nvidia-cuda/release/deps/librand-c254e8c85c3e4e8a.rlib --extern rustacuda_core=/side/Code/sidequest-meet/target/nvptx64-nvidia-cuda/release/deps/librustacuda_core-f3f21a55d0150a50.rlib --extern rustacuda_derive=/side/Code/sidequest-meet/target/release/deps/librustacuda_derive-d2b9f2aba2ab89c4.so

The same happens whether or not I include "rlib" as an additional crate type.

Interestingly, all my dependencies' rustc invokations appear to be getting the "extra-filename" param. It is just the final invocation (the one ptx-builder actually cares about) which is missing extra-filename.

If needed, I can take the time to create a minimal repro for you. But I'm guessing it is simply a problem with a newer cargo version which doesn't pass the argument.

I'm running:

  • cargo 1.46.0-nightly (4f74d9b2a 2020-07-08)
  • rustc 1.46.0-nightly (346aec9b0 2020-07-11)
  • Arch BTW

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.