Memory Usage explosion in parse command in version 0.2.37 about xclogparser HOT 7 OPEN

+1

from xclogparser.

+1

from xclogparser.

Hello, unfortunately I don't have time to debug exactly what's wrong, but it must be one of the recently added contributions between the last two versions: v0.2.36...v0.2.37

I have a feeling it must be one of these 3 changes:

• #181
• #182
• #189

If you can locally try to revert each of those commits and see which one fixes it, that would be helpful.

from xclogparser.

I can confirm that rolling back to version 0.2.36 worked for me

from xclogparser.

Tested locally. Reverting #182 avoids the massive RAM usage. But the underlying issue seems to be the BuildStep struct that has too many subSteps.

With SwiftCompile as a detail type, it includes each file's compilation, such as:

SwiftCompile normal arm64 /opt/homebrew/var/agent/builds/ip-123-45-67-890-internal/project/pipeline/.../Sources/.../file.swift (in target '...' from project '...') ... etc

I tried it in a project with ~7,000 Swift files, it adds more than 400,000 sub steps into the parsed result, nested in 5 levels down.

In both use cases of --reporter json and --reporter flatJson, the struct becomes too large that JSONEncoder can't handle. The memory usage shoots up on line 42:

An alternative workaround is to use --reporter summaryJson as it drops all the sub steps in the SummaryJsonReporter.

from xclogparser.

Reverting in #199 and then will cut a new release. Thanks for reporting!

from xclogparser.

Same problem with JSON, but not only on last version. Actually on each version I face that issue(

Context:
MacOS: Sonoma 14.2.1
Device: MacbookPro M3 Pro
Xcode: 15.1
XCLogParser installation: rake build v 0.2.36

from xclogparser.