swiftFunctionTimes not always populated in certain cases about xclogparser HOT 9 CLOSED

We haven't tested it with cocoa pods. I will do some tests with it, is there any specific pod that is causing this issue?

I'll take a look at the parsing speeds, I may be able to make it faster.

from xclogparser.

@ecamacho It seems like the issue is building Swift modules with as "Whole Modules" versus "Incremental".

Not sure why, but for some reason the project I'm working on has all their Swift pods doing this. When set back to Incremental, the individual file compilation continues.

Based on what dump contains, you should be able to extract at least the swiftFunctionTimes. Not sure if it's worth trying to list out all the swift files being compiled within the signature in that case (although maybe a qualifier and another array field would be helpful).

Oh, as one very minor thing. I just upgraded to a 2019 MBP which has Catalina. I'm not overly familiar with rake, but Catalina now defaults Terminal to zsh. So as a temp work around till I understand it better I used alias rake="noglob rake" prior to invoking. Adding a note about this for the build instructions would be helpful.

from xclogparser.

@mobileben feel free to test this PR #48 I'm parsing the swift files that were compiled and I also speed up the parsing times.

from xclogparser.

@ecamacho I will test this shortly.

from xclogparser.

@ecamacho quick question here. When should swiftFunctionTimes show up? I have a slightly different build configuration, but am using the correct flags. I can see the generated Swift build times listed in Xcode's Build summary. But when I run xclogparser on it with the parse argument and json reporter, all the swiftFunctionTimes are empty. The dump shows them.

This is using the current master.

I'm going to try and build again, but this time apply the flag across the pods.

I'll try and take a look at the XCLogParser code to see if I can gain any insights. And I'm going to try the branch with the fix since it takes so long to process using master. To give you an idea, the dump JSON is 848MB. So that should give you a relative sense of how large the xcactivitylogs are.

from xclogparser.

Okay update.

Regarding the swift function fix. There is much improvement. The fixed version takes a 1-2 minutes (I did not precisely time). The current master took 1 hour and 15 minutes to process. Nice job on speeding it up!

I do notice if I run a diff, the results between master and swift_func_improv don't match. I haven't had a chance to take a closer look (because the generated files are large) to see if they are just out of order.

Also regarding not finding populated swiftFunctionTimes, I think I either ran into another issue or its working as intended. It appears if you use the the --redacted flag, then the swiftFunctionTimes are ... well redacted :D (ie. they don't show up).

That sort of seems like weird behavior since I would have just expected the filename to be redacted only where needed, like it is done elsewhere.

Oh yeah, this behavior is present in both master and swift_func_improv.

from xclogparser.

Hi, thanks for testing.

Yeah, the order can be different because it's now parsing in parallel the swift function times data.
I've fixed the issue about not having the swift function time when you use the --redacted flag, nice catch. I've pushed the change to the swift_func_improv branch

from xclogparser.

--redacted now has the swiftFunctionTimes.

Just curious, I notice a couple odd things (not related to xclogparser and was curious if you had any thoughts or insights.

• I notice that if I sum the times in MS in the swiftFunctionTimes, they do not sum to the overall duration for the step. Any thoughts as to where that missing time is?
• I've seen some interesting cases in our build summary where compiling an Obj-C file could take 150 seconds+. This shows up in the build summary and in the data from xclogparser. Some of these files themselves are pretty small. Do you happen to know if any blocking occurs during steps? I find it a bit hard to believe some of these file take that long. I do happen to know the argument list for compiling is monstrously large. Other than that, not quite sure why we have some outliers.

Also, regarding that last question. I do notice that the time cited by Build Summary are sometimes smaller than what xclogparser kicks out. So in that example of 150 seconds, I might seen an extra 1-2 seconds on what is shown in the output of xclogparser. Have you noticed any strange behavior like this before?

from xclogparser.

Hi

• Yeah the use of -debug-time-function-bodies is not properly supported by Apple. It was created just as an indicator. You can the response of this question in the Swift forums where they say it's not 100% accurate https://forums.swift.org/t/potential-contributions-to-compilation-time-reporting/7055. So basically, the Swift compilation time reported by the Xcode log is the total sum of several steps involved in compiling a file. The Swift function time it's just an approximation of the compilation of the functions in each file.

• In our case we've observed that issue when dealing with concurrency. So a compilation gets blocked because the cores are busy compiling other files. By default the Swift Compiler uses all the Cores available and that's is not the best strategy, when we artificially limited the number of threads used by swiftc we got better performance. Apple has been working on a fix for that:
  https://bugs.swift.org/browse/SR-11927 It's a good idea to test it with your project.

I'll close this ticket, but feel free to open more issues, your feedback was really helpful.

from xclogparser.