Please create also a fork of cargo. about gccrs HOT 21 CLOSED

Interesting you mention this as we were talking about it on our Zulip channel. I think it would be good for us to try and make a wrapper over gccrs to look as much like rustc as we can and this could possibly be integrated into the official cargo down the line. This is how gccgo approaches this issue with a gccgo integration into the go build cmd. What do you think?

from gccrs.

This severly limits the usuability of the current project, so that having. It s not so much about adding support to cargo. Current cargo .ml files of most rust projects are written to call clang when needed.

Until gccrs becomes mainstream, we should have something replacing that kind of cargo directives.

from gccrs.

but currently cargo will only use rustc for dependencies and clang for C bindings

Cargo doesn't know anything about building C code. If a crate is a wrapper around a C library, it is the responsibility of the build script for that crate to build the C library if necessary. The cc crate is most commonly used for this purpose. It defaults to cc on most unix systems, not clang, but you can override the default using the CC env var.

https://github.com/alexcrichton/cc-rs/blob/f935d6b4d7746a590cb7b0b6032e0f3e5c0eb872/src/lib.rs#L1980-L1994

from gccrs.

Cargo doesn't know anything about building C code. If a crate is a wrapper around a C library, it is the responsibility of the build script for that crate to build the C library if necessary. The cc crate is most commonly used for this purpose. It defaults to cc on most unix systems, not clang, but you can override the default using the CC env var.

That s what I m stating. To have support for automatic rewriting of build scripts in our own version. Otherwise, using gccrs only outside crate wouldn t be that usefull.

from gccrs.

Otherwise, using gccrs only outside crate wouldn t be that usefull.

You said that clang is often used, but this is not the case. The system default is almost always used. The system default is normally gcc.

To have support for automatic rewriting of build scripts in our own version.

Rewriting build scripts isn't going to work. You may have to handle each crate differently.

Otherwise, using gccrs only outside crate wouldn t be that usefull.

Gccrs is a rust compiler, not C compiler. Build scripts don't usually invoke rustc, but if they do, it is advisable to read the RUSTC env var, which cargo reads too.

from gccrs.

Otherwise, using gccrs only outside crate wouldn t be that usefull.

You said that clang is often used, but this is not the case. The system default is almost always used. The system default is normally gcc.

Even the official libc binding crate depends on clang. Zstd support requires clang. Rocksdb binding requires clang.

I couldn t compile any C bindings without having errors like stdarg.h missing even if it works with gcc (something which is fixed by installing clang). rust is simply if for the llvm world. Projects using it simply favor everything coming from llvm.

Rewriting build scripts isn't going to work. You may have to handle each crate differently.

I m thinking about doing this at compile time and on keywords so that when a specific command is called, it gets replaced and it options replaced to gccrs or gcc (if for a C binding).
When you have a dependency hell, rewriting every build script is not the right option.

from gccrs.

Even the official libc binding crate depends on clang. Zstd support requires clang. Rocksdb binding requires clang.

The official libc binding crate doesn't depend on clang in any way. It only asks rustc to pass -lc to the linker. The only external commands it's build script run are the rustc version as found in the RUSTC env var to get the version string. This is used to check if certain things are supported. And the freebsd-version command to check for the freebsd version if it is running on freebsd. The only time a C compiler is used by the libc crate is when running tests to test for compatibility of struct layouts. This uses the default C compiler as it uses the cc crate.

Zstd uses the bindgen crate to create rust bindings using C header files. For this it uses libclang to parse the C headers. There is no way to replace clang with gcc here as gcc doesn't support usage as library as a matter of policy. Even if it did, that would likely mean that bindgen would need to be GPL licensed itself, which they very likely don't want.

The Rocksdb binding also uses bindgen so the same applies.

I m thinking about doing this at compile time and on keywords so that when a specific command is called, it gets replaced and it options replaced to gccrs or gcc (if for a C binding).

There is no keyword to invoke a C compiler. The build script invokes the C compiler like rust code can invoke any other kind of program using std::process::Command or even if it wants directly using fork/exec. Also in the case of bindgen, gcc is simply missing functionality that it needs.

from gccrs.

@bjorn3 more exactly gcc as gcc -E this means replacing a library with several scripts.

from gccrs.

I couldn t compile any C bindings without having errors like stdarg.h missing even if it works with gcc (something which is fixed by installing clang).

stdarg.h is part of the C specification. If GCC doesn't provide it, that is their problem.

rust is simply if for the llvm world. Projects using it simply favor everything coming from llvm.

Projects often favor it for a reason. Some like bindgen use libclang, which gcc doesn't provide an equivalent for. Some favor it because LLVM can cross compile without having to install a separate compiler version for each target. Rustc enables 12 LLVM targets in it's default configuration (https://github.com/rust-lang/rust/blob/d015f0d92144f0e72735a918aee8510b0fe2cff5/config.toml.example#L95) Rustc is planning to switch to lld by default because it is much faster.

This doesn't mean that I think that tying rustc to just LLVM is a good idea. I have been working on an alternative backend for the official rust compiler based on Cranelift for the past two years. Based on this experience I can tell that rust the language is not inherently tied to LLVM. For example LLVM intrinsics have explicitly put behind a feature gate. There are a few things that make alternative backends harder than I wish like the huge amount of SIMD intrinsics, but those shouldn't hinder backends like GCC too much, as those already need to support the exact same SIMD intrinsic set for C compatibility.

from gccrs.

@bjorn3 more exactly gcc as gcc -E this means replacing a library with several scripts.

gcc -E only runs the preprocessor. It doesn't parse the C code.

from gccrs.

I couldn t compile any C bindings without having errors like stdarg.h missing even if it works with gcc (something which is fixed by installing clang).

stdarg.h is part of the C specification. If GCC doesn't provide it, that is their problem.

gcc itself have no problem using it on platform. it s just build scripts trying to compile with llvm/clang.

from gccrs.

Can you give an example of a crate that fails to compile with that error when clang is not installed?

from gccrs.

@bjorn3 not currently. you see one of the problem with llvm is they favor program specification (for debugging) and compilation speed (for faster testings) over optimizations. It doesn t supports automatic parallelization. And profile guided optimisations depends of the front end which in the case of rustc aren t supportted.
Also benchmarks about performance don t push gcc optimizations as far they can go (not tweaking --param= internals).

As a result, I m stuck with single threaded program executing a million instruction virtual machine program at 0.2 instruction per seconds (but leaving no ram free).
And this is the program where I got those compile errors (I had to install clang to solve them).

from gccrs.

not currently. you see one of the problem with llvm is they favor program specification (for debugging) and compilation speed (for faster testings) over optimizations.

llvm-ir is way under-specified to the point that rustc often hits miscompilations or it is unclear if something is UB or not. They favor optimizations over correctness, which is a bad thing. For example strict aliasing for mutable references had to be disabled because of miscompilations.

It doesn t supports automatic parallelization.

Rustc makes it easy to do manual parallelization, which can have much more speedup than automatic parallelization. For example if it is a iterator loop that you want to parallelize, all you have to do is include the rayon crate and replace .iter() with .par_iter() provided that it is actually safe to parallelize.

And profile guided optimisations depends of the front end which in the case of rustc aren t supportted.

It has been supported for a while: -Cprofile-generate/-Cprofile-use.

from gccrs.

lvm-ir is way under-specified to the point that rustc often hits miscompilations or it is unclear if something is UB or not. They favor optimizations over correctness, which is a bad thing. For example strict aliasing for mutable references had to be disabled because of miscompilations.

Still less than gcc.

Rustc makes it easy to do manual parallelization, which can have much more speedup than automatic parallelization. For example if it is a iterator loop that you want to parallelize, all you have to do is include the rayon crate and replace .iter() with .par_iter() provided that it is actually safe to parallelize.

In other languages, I can also do the same thing. But the problem is studying a huge code. Let alone for thread safety. I don t have the ressource for that.

It has been supported for a while: -Cprofile-generate/-Cprofile-use.
It s not in the man page. Are you sure it s supported?

from gccrs.

Still less than gcc.

To the best of my knowledge GCC has a better specified IR than LLVM. I wasn't arguing that LLVM results in faster executables. It may or may not, depending on the program and target architecture. I was arguing that unlike what you say LLVM is more focused on generating faster executables than correctness. As far as I know GCC is more focused on correctness, but just so happens to (sometimes) generate faster executables. Either way, LLVM cares much more about fast executables than you think. It just doesn't always succeed in it as much as GCC. This may simply be because GCC is older, or it may be due to more fundamental reasons.

In other languages, I can also do the same thing. But the problem is studying a huge code. Let alone for thread safety. I don t have the ressource for that.

Unlike other languages rust gives you an error when there is a thread safety problem. It tells you exactly which value couldn't safely be shared between threads. This makes it much easier to parallelize. In simple cases where auto parallelization is possible, your are unlikely to hit any thread safety issues. In the cases where you could hit thread safety issues, auto parallelization wouldn't have worked anyway. As an example rustc could be parallelized despite being a huge code base. (It isn't enabled by default because the last time it was benchmarked, it regressed single thread performance due to extra locks) Auto parallelization would never have been able to find anything worthwhile to parallelize.

from gccrs.

To clarify I am not saying that there shouldn't be a rust frontend for GCC. In fact I really think there should be multiple independent backends or even whole compilers for rust. I am merely saying that rust isn't very dependent on LLVM. Also I personally think a rust frontend for GCC would mostly help with preventing a dependency on LLVM (especially around what unsafe code is and isn't allowed to do) and to enable compilation for more targets unsupported by LLVM.

from gccrs.

In the cases where you could hit thread safety issues, auto parallelization wouldn't have worked anyway.
gcc sometimes can using pgo along insane internal parameter tweaking requiring to real consider the whole program for optimization which sometime involves using hundreds of Gb of ram and day of compilation even for very simple project.

Though it doesn t beat using -par-threasold99 -par-auto with icc (which is quick and use little memory for doing it). That s the time I wish to it should support more things than just fortran and c/c++. But that s completely a differrent unrelated topic.

from gccrs.

Would you like to join our Zulip server? I think this discussion is not relevant to the issue any more https://gcc-rust.zulipchat.com/

I don't wish to close the issue but this is now off topic.

from gccrs.

Instead of forking cargo which will be a very difficult maintenance task, @flip1995 has suggested why not build out a cargo subcommand such as:

1. https://doc.rust-lang.org/cargo/reference/external-tools.html
2. https://crates.io/crates/cargo-edit

This will be much easier for us to maintain and will not distract from rustc development. We can maintain it as we wish, we could also roll our own cargo akin to mrustc but I prefer the idea of a cargo subcommand like cargo gccrs build, cargo gccrs test etc.

Thanks @flip1995 for the idea.

from gccrs.

Lets close this since @CohenArthur has been working on this over on: https://github.com/Rust-GCC/cargo-gccrs

from gccrs.