Comments (7)
From what I can tell, icecc
provides that functionality via chroot
(or some equivalent mechanisms). This is mostly useful for supporting multiple versions of compiler.
If this is the feature what you’re asking for, it’s already partly supported by yadcc
.
Adding paths to alternative compilers to extra_compiler_dirs
when starting daemon on compile server should do the trick, I think. (See the documentation for more details.)
What’s not possible with yadcc
but supported by icecc
is when you want to allow users to use arbitrary compilers that were not known to the compile cluster beforehand. I’d say icecc
is more flexible in this case, but I’m not sure that’s a normal situation in production environment. Besides, that would require root privilege to run the daemon (required by chroot
).
If you’re proposing supporting non-x86 compilers on x86 machine, I’m not sure it’s possible without help from qemu
. IIRC icecc
doesn’t allows this, either.
from yadcc.
chroot
Appreciate your quick response.
Yes, what I am looking for is support arbitrary compilers which can NOT be installed beforehand.
Is there any plan to support it?
from yadcc.
I don’t think it will be implemented any time soon, unfortunately. I failed to imagine a case where it’s desired in production.
Are you using the feature in production environment? If so, do you mind to elaborate your use case?
Although I’m not working on the feature, from a design perspective, I can see some challenges in implementing it (in case someone wants to submit a PR):
- Transferring the compiler to the compile server can be bandwidth-intensive. This is especially true when used with a large compile cluster, where the client is more likely communicating with a “fresh” compile server — In this case even a compile-server-local cache can help little.
- One need to filter syscalls carefully to avoid a “malicious” compiler (e.g. a backdoor or a proxy) being executed. This is even more important if the compile cluster is operated by a different group of people than the users.
- The protocol needs some refactor, both in assigning compile server, and accessing compile cache. This should be relative easy, though.
from yadcc.
FWIW, there is a workaround for this. If you can push your compiler to an NAS that’s accessible to the compile cluster, you might want to check the implementation in branch next
. There the daemon can re-scan periodically to see if new compilers are provided, and use them as appropriate. See https://github.com/Tencent/yadcc/blob/next/yadcc/daemon/cloud/compiler_registry.cc#L44
from yadcc.
Yes, I want to use the feature in production environment. And the use case is compiling Android(AOSP).
For Android build, the toolchain used is from the source tree, that means different source commit may use different toolchain.
As the build time in Android is considerable, so the time used to transferring the compiler is small relatively.
from yadcc.
I agree that’s a totally valid use case. This also reminds me about compiling UE4. Personally I had no experience about compiling either of them but both are so popular among developers that they should be better supported.
Unfortunately for the moment I don’t have much time to implement the feature, I’m open to a PR about this.
TBH, however, since I haven’t tried yadcc
with something like AOSP, I’m expecting several issues (besides this one) to show up in this case. Therefore I suggest you first try adding prebuilt compilers shipped with AOSP to yadcc
manually and see if everything else works as intended.
from yadcc.
ok, thanks for your advise.
from yadcc.
Related Issues (20)
- 文件预处理是由客户端自己完成吗 HOT 9
- 技术原理请教 HOT 2
- Cannot contact delegate daemon. Daemon died? HOT 9
- 能否给daemon加个功能,当scheduler重启后自动重链 HOT 2
- 关于调度机制的问题 HOT 1
- yadcc编译时,提示ld链接无法找到pthread、m、c、dl等动态库 HOT 2
- [2023-01-29 20:02:30.532771] [INFO ] [yadcc/client/cxx/yadcc-cxx.cc:207] Failed on the cloud with [-126]. Failing back to local machine. HOT 2
- 无法缓存C++编译之后的文件 HOT 2
- 安装失败 HOT 2
- can not support CROSS_COMPILE HOT 8
- yadcc v1.x 版本编译报错 HOT 1
- failling back to local machine 126 HOT 3
- yadcc能不能用于安卓系统AOSP构建加速 HOT 3
- 经常编译报错. Child process exited unexpectedly with status HOT 1
- 当我运行./yadcc-daemon时,报Check failed: parts.size() >= 3 (2 vs. 3) Unexpected cgroup setting错误 HOT 2
- build latest llvm failed HOT 1
- 运行daemon错误 HOT 1
- --with-nghttp2 was specified but could not find libnghttp2 pkg-config file.
- 这个是否支持clang? HOT 3
- 如何使用如YADCC_IGNORE_TIMESTAMP_MACROS这样的参数去控制客户端的行为? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from yadcc.