Comments (9)
All right. The failures were related to some internal changes I had to make. Fortunately I was able to run with the assistance of a colleague.
Some data:
- The generated
debug.log
was 16GB - The configuration cache itself is 203MB
- With minified output to strip out only tasks, the
speedscope.json
is 1.1MB - I had to anonymize the data before I could upload it to speedscope.app. Therefore, I had to manually map back to our internal module names and tasks
- The number of modules in this particular build is about 1000
With an eyeball look (since we have loads and loads of modules), I was able to see that some tasks stood out:
graphViewMain
which is 115Kb for a relatively small module- Same for
graphViewTest
which is 175KB for that same module - We see the same
graphView
for each of our sourcesets - For some of the largest sourcesets, I see that each
graphViewX
takes 1MB and we have about 5 of these, so 5MB per module
Other tasks are also large, but at least from a first glance graphViewX
appears to be the culprit. When I omit the data for all these, we are still looking at a large number, but maybe the findings from this task can help us with the rest as well.
from dependency-analysis-gradle-plugin.
This is the workaround I used if anybody else runs into this issue:
import com.autonomousapps.tasks.GraphViewTask
project.getPluginManager().withPlugin('com.autonomousapps.dependency-analysis', { plugin ->
project.getTasks().withType(GraphViewTask).configureEach {
it.notCompatibleWithConfigurationCache("https://github.com/autonomousapps/dependency-analysis-gradle-plugin/issues/1098")
}
}
from dependency-analysis-gradle-plugin.
There's a viewer for the configuration cache data: https://github.com/gradle/gcc2speedscope, which may give some insights into what is being stored. @TimvdLippe could you please try that?
from dependency-analysis-gradle-plugin.
I will need security approval to run such tooling, so it will take me a couple of days to do so, but will do 👍
Is there a workaround to disable the configuration cache for the task anyways? I think we can mark it as incompatible ourselves, as we are an outlier in terms of codebase size and then we can wait for a potential fix.
from dependency-analysis-gradle-plugin.
Is there a workaround to disable the configuration cache for the task anyways?
There's Task.notCompatibleWithConfigurationCache, you can look how Gradle itself uses it for a similar purpose. This should be enough to at least prevent loading the cache. I'm not sure if it also disables storing the graph (Gradle still checks for CC errors coming from other tasks), but that's the best option I know of.
from dependency-analysis-gradle-plugin.
I was able to clone the project and build it. Unfortunately I am running into runtime issues with a Suppressed: kotlinx.coroutines.channels.ClosedReceiveChannelException: Channel was closed
and later a Caused by: java.lang.ClassNotFoundException: gcc2speedscope.EventStore$queryProfiles$1
. I lack Kotlin knowledge and am not sure if these are pre-existing issues or related to our internal infrastructure. Will try to debug further with a colleague who has knowledge on Kotlin
from dependency-analysis-gradle-plugin.
Looking at the PR in question, I wonder why we annotated compileFiles
and runtimeFiles
as @InputFiles
? In our internal plugins, we have some tasks that also operate on our classpaths, but they don't have such a large configuration input. Instead, they operate on a @Classpath
and we retrieve the files during task execution. Since we already have the runtimeClassPath
property, shouldn't we annotate that property with @Classpath
and then retrieve the files?
Example from our internal plugin that afaik doesn't have the same configuration cache size hit:
@Classpath
public abstract ConfigurableFileCollection getSourcesClasspath();
from dependency-analysis-gradle-plugin.
@mlopatkin Is that sufficient information for you or do you want me to run anything else?
from dependency-analysis-gradle-plugin.
To unblock our upgrade, we will mark the graphView
tasks as incompatible with the configuration cache. Hopefully in the future the situation improves and we can use the configuration cache again for these tasks.
from dependency-analysis-gradle-plugin.
Related Issues (20)
- ComputeUsagesTask taking 30+ minutes to execute when it used to take 30 seconds for the same project HOT 8
- Count of transitive dependencies that would be required to add when removing an unused dependency
- [TestKit Plugin] Open Rendered elements to be able to create custom ones HOT 2
- thinks I can remove testImplementation after adding compileOnly HOT 2
- Unknown constant pool type HOT 14
- Missing git tag for latest stable release (1.31.0) HOT 1
- Upgrade Moshi dependency to avoid CVE-2022-3635
- Plugin creates variant ambiguity errors with JVM Test Suite Plugin + Jacoco Aggregation Plugin HOT 3
- FR: Suggest converting an android project to a JVM-only project if possible HOT 1
- synthesizeDependenciesMain is failing with exception when having a file collection with an absolute file on windows as dependency
- Make FindDeclaredProcsTask use the JDK of the configured Java toolchain to load classes HOT 1
- ClassNotFoundException when adding plugin to project HOT 1
- File-level private vals of a type from an external dependency result in that dependency requiring to be an api dependency HOT 3
- Could not resolve project -> when a submodule is aar artifact HOT 3
- Unable to find method KotlinModuleMetadata.getKmModule() , explodeJarRelease is failing HOT 1
- False positive HOT 3
- Add wildcard support for exclusion rules HOT 11
- Task ':explodeCodeSource' uses this output of task ':kspKotlin' without declaring an explicit or implicit dependency HOT 3
- GraphVisitor.usesResByRes is extremely slow. (Stuck in computeActualUsageDebug for many minutes) HOT 18
- Run plugin for some specific modules only HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dependency-analysis-gradle-plugin.