Comments (5)
If my memory serves me well, PCH can be quite large if many headers are included when building it (usually how it’s done). Transferring it can be quite costly even in enterprise-grade network. CMIIW, I haven’t use PCH for a while.
This can be mitigated to a certain degree by caching PCH on compile server. Internally this is done for supporting Java, by uploading dependencies to a central storage and ask compile server to cache them. And it seems to work adequately well.
However, internally our codebase is way too large to use a single PCH to include all headers. Meanwhile I don’t think GCC supports using multiple PCHs when compiling a single TU so splitting PCH is not an option.
That’s why we decided not to use it.
Regarding the second question, I have no experience with MSVC, sorry. But it does look strange to me that MSVC can accept arguments with hyphen, instead of slash.
from yadcc.
Thank you for reply. Just take a example, msvc certainly does not accept -fno-working-directory , it does not have similar function to do same work like the option -fno-working-directory as for gcc. i do this work manually on the preprocess output. The second question is just about whether can i debug on the output while turning the -fworking-directory off, if i use yadcc to compile my source codes distributely.
from yadcc.
If my memory serves me well, PCH can be quite large if many headers are included when building it (usually how it’s done). Transferring it can be quite costly even in enterprise-grade network. CMIIW, I haven’t use PCH for a while.
This can be mitigated to a certain degree by caching PCH on compile server. Internally this is done for supporting Java, by uploading dependencies to a central storage and ask compile server to cache them. And it seems to work adequately well.
However, internally our codebase is way too large to use a single PCH to include all headers. Meanwhile I don’t think GCC supports using multiple PCHs when compiling a single TU so splitting PCH is not an option.
That’s why we decided not to use it.
Regarding the second question, I have no experience with MSVC, sorry. But it does look strange to me that MSVC can accept arguments with hyphen, instead of slash.
As for the first question, may be in most projects, pch is very large. But i just use fastbuild(another distbute complile tool) to compile my ue4 project remotely on serveral cloud mechines. I found if the number of worker machine is not large enough, the time spent on the transferring data and the more time while we split original compilation into preprocess and compilation may be more than that spent while we just compile locally. After i study on this, and found the pch may have a influence upon this. Thus, if the pch is not very large( ue has its own reflection and build system and split its pch well, the size of pch is always just beyond 10MB), Theoretically, i could use pch? wish your reply, thank you very much.
from yadcc.
I can’t recall much issue with debugging using GDB, either because:
- I didn’t need source code during debug much, or
- blade always used a relative path to workspace root and I usually run my binary there, so GDB was able to find the source file correctly.
I don’t have a development environment at hand so I can’t check which was the case, but the latter seems more likely.
Addition of that argument is not strictly necessary, it’s just a hack for lazy people (me) to make caching work. Although not tested, I think caching is also possible without that argument, by ignore that line in preprocessed file when generating cache key, and post-editing the resulting ELF file (dwarf). The relative attribute here should be DW_AT_comp_dir
. (Maybe we can only do post-editing to add that attributes, without removing the argument.)
You can test if removing that argument can help your debugger to find the source file. It probably works.
Regarding PCH, yes I think it should be possible to compile distributedly with it. Though I don’t think 10MB is a acceptable size for relatively large number of translation units*, unless some sort of caching is done. Transferring it once per second can easily reach 100Mbps, while usually in real world scenarios we usually build a dozen of files or more per second.
By the way I’m not sure if caching would make much sense in this case as a change in one of headers would like invalidate a large number of cache entries (if not all).
*: I don’t have a sense about how well the PCH compresses. Preprocessed source compresses well and the result is usually 1/10 or 1/20 (or even less), so I hardly ever worried about it’s size.
from yadcc.
Thank you for your reply, let me test it and discuss with you soon after i have results. Besides, i just do my work based on compilers, including gcc, clang, cl.exe, so i am not able to do work like post-edit...but thanks for your advice, may i can have a test on this to verify my guess.
from yadcc.
Related Issues (20)
- 文件预处理是由客户端自己完成吗 HOT 9
- 技术原理请教 HOT 2
- Cannot contact delegate daemon. Daemon died? HOT 9
- 能否给daemon加个功能,当scheduler重启后自动重链 HOT 2
- 关于调度机制的问题 HOT 1
- yadcc编译时,提示ld链接无法找到pthread、m、c、dl等动态库 HOT 2
- [2023-01-29 20:02:30.532771] [INFO ] [yadcc/client/cxx/yadcc-cxx.cc:207] Failed on the cloud with [-126]. Failing back to local machine. HOT 2
- 无法缓存C++编译之后的文件 HOT 2
- 安装失败 HOT 2
- can not support CROSS_COMPILE HOT 8
- yadcc v1.x 版本编译报错 HOT 1
- failling back to local machine 126 HOT 3
- yadcc能不能用于安卓系统AOSP构建加速 HOT 3
- 经常编译报错. Child process exited unexpectedly with status HOT 1
- 当我运行./yadcc-daemon时,报Check failed: parts.size() >= 3 (2 vs. 3) Unexpected cgroup setting错误 HOT 2
- build latest llvm failed HOT 1
- 运行daemon错误 HOT 1
- --with-nghttp2 was specified but could not find libnghttp2 pkg-config file.
- 这个是否支持clang? HOT 3
- 如何使用如YADCC_IGNORE_TIMESTAMP_MACROS这样的参数去控制客户端的行为? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from yadcc.