Code Monkey home page Code Monkey logo

xclogparser's People

Contributors

aleksandergrzyb avatar alvarhansen avatar balestrapatrick avatar birwin93 avatar bobbyphtr avatar codedbypm avatar ecamacho avatar f-meloni avatar ignatovv avatar ileitch avatar jinjorge avatar lgilyazov avatar liuxia-bytedance avatar luispadron avatar magauran avatar mariozig avatar mollyiv avatar patrikbillgren avatar polac24 avatar ptrkstr avatar rabc avatar romainboulay avatar ronanrodrigo avatar rudro avatar sunshinejr avatar tinder-maxwellelliott avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

xclogparser's Issues

The file size of parsed json has become huge since Xcode 12.

Hi XCLogParser developers.
Thank you for providing a awesome parser library. 🙇
Let me report a new issue as titled.

I'm currently using xclogparser to parse the xactivitylog file to collect the build time for each module, and I report them to my team members.

For Xcode 11, it worked fine and the file size of the parsed json wasn't so huge.
However, for Xcode 12, the file size of the parsed json becomes so huge (approximately 7.6GB) by executing xclogparser parse --file <file_name> --reporter flatJson command.
I got this json when I parse the xactivitylog whose file size is 3.7MB.
It so huge that I can't parse it on CI (with fastlane's custom lane)

here's the example code of parsing: (Some of filepath are replaced with meaningless paths)

lane :parse_xactivitylog do
  activity_file = "path/to/log.xactivitylog"
  output_path = "path/to/output/activity.json"
  sh("#{xclogparser} parse --file #{activity_file} --reporter flatJson --output #{output_path}")
  json_string = File.read(output_path) // <==== Failed to read json file due to a large file
  activity_json = JSON.parse(json_string)
  ....
end

Since I've been faced with this problem, I tried to parse with other reporter styles then I was able to get some information that led me to identify the problem.

Here's the result that I executed parse command with --reporter summaryJson: (Some of information are masked)

{
  "parentIdentifier" : "",
  "fetchedFromCache" : false,
  "title" : "Building workspace XXXX with scheme YYYY and configuration Debug",
  "warningCount" : 2526,
  "duration" : xxxxxxx.xxxxx,
  "startTimestamp" : xxxxxxx.xxxxx,
  "signature" : "Building workspace XXXX with scheme YYYY and configuration Debug",
  "errors" : [
    ...
  ],
  "compilationEndTimestamp" : xxxxxxx.xxxxx,
  "compilationDuration" : xxxxxxx.xxxxx,
  "endDate" : "xxxxxxxxxxxxxxxxxx",
  "errorCount" : 81,
  "domain" : "Xcode.IDEActivityLogDomainType.BuildLog",
  "type" : "main",
  "identifier" : "xxxxxxxxxxxxxxxxxx",
  "buildStatus" : "succeeded",
  "schema" : "YYYY and configuration Debug",
  "subSteps" : [

  ],
  "endTimestamp" : xxxxxxx.xxxxx,
  "architecture" : "",
  "machineName" : "ABC",
  "buildIdentifier" : "xxxx-xxxx-xxxx-xxxx",
  "startDate" : "xxxxxxxxxxxxxxxxxx",
  "warnings" : [
    {
      ... there are so many warning details
    }
  ],
  "documentURL" : "",
  "detailStepType" : "none"
}

According to this result, I seem to have over 2500 warnings... 😨.
Also, I think that since Xcode12, the number of warnings has increased and the parsed json now contains a large number of detail characters for those warnings, so the file size has become bloated.
However, I don't know why I get so many warnings and bloated file sizes in Xcode 12 and later.

What I actually want to do

Actually, I just want to get the build time for each module and each module's name by referring to the steps that are "type": "target" only.
So in my case, I don't need any error details.
BTW, with --reporter summaryJson, I can get the total build time, but not for each module.(I mean, --reporter summaryJson can't do what I want to do.)

Based on the above, I have some questions for you.

  • Are there any good ideas(or options) I can only get the steps of "type" : "target" so that I can analyze the build time of each module without creating a huge json file?
  • Or, is it possible to create some options like being able to reduce(or redact/ignore) the output of the error details?
  • Or, could you tell me how Spotify is able to parse and use the xactivitylog generated from a huge project like your service with a reasonable file size?

I hope you can see this issue. Thanks 🙇

Can't install it

I'm getting this error whent trying to install the tool with Swift Package Manager

image

HTML parser is not working anymore

Hey There, I don't know what happened but out of nowhere after everything was working and the html page generated was showing all the errors and logs, now whenever I try to generate with html parser the html page is generated but displays empty data then the html page just breaks and does not want to load anymore.

I tried the following approaches

xclogparser parse --file pathtologfile.xcactivitylog --reporter html
xclogparser parse --workspace pathtoworkspacefile.xcworkspace --reporter html
xclogparser parse --project mytaxi --reporter html

any clue why this might be happening?
my Xcode version is 11.5
xclogparser version: XCLogParser 0.2.18
and I am sure we are using Swift5

Thank you

Docker Image

Hi there 👋 :

I'm curious what the intention is behind the Docker image. Is this so it can be built with Swift Linux, or is this so the tool can actually be ran from a Docker container?

I'm wondering how you use this tool "at Scale" at Spotify? Do your developers install the tool and run on demand, or are you monitoring for XCActivityLog's and uploading them to AWS S3 (or something similar) for processing and archiving in the cloud.

I work on a developer productivity team @square and have some interest in using this tool to provide insights into local developer experience.

xcactivitylog is not generated

This is not exactly an issue with the xclogparser, but with xcodebuild. Whenever I build within Xcode I get the xcactivitylog file, however if I delete derived data folder, then build with xcodebuild the log is just not there. (We use a fresh VM for every new CI build) If I rerun xcodebuild (so I rebuild) the xcactivitylog is generated.

Did you encounter this issue before?

Custom DerivedData path in Xcode 12.5

Hey guys,

whenever we run clean xcodebuild with custom -derivedDataPath logs look different compare to Xcode output. The main diff is that logs are missing Build target TargetName. It's just a list of compiled files.

Prior iOS 12.5 it worked OK 🤷‍♂️ Do you know smth about it? Maybe there is a some workaround

I've attached the output
json+log+xcresult.zip

When trying to set `--derived_data`, if there is an error, the error indicates the flag is `--derived-data`

The error is in LogFinder.swift. I can create a PR in a little bit to fix this.

Also as an aside, do you happen to know if, when you use --derived_data if you can make it not look for project directories? At least when I use -derivedDataPath, it writes the contents of the project (eg. Build, Logs, etc) directly into that directory.

So for example -derivedDataPath foo would yield something like:

foo/Build
foo/Logs
etc

I'm having to add an extra directory to -derivedDataPath to get it to work with xclogparser. I'm working on some else's build system, so I can't arbitrarily change path like that (or would at least need to have discussions about doing it).

dump versus parse time durations

I was analyzing my build using xclogparser invoked with parse --reporter json. As a test, I focused on one type == target. This had 22 defined subSteps. My thought was that the duration of the target should equal the sum of the durations of the subSteps. Each of these sub steps did not have any other sub steps.

Summing the sub steps results in a value of 1.0292 whereas the duration value for the target was defined as 5.7726 (I rounded these down)

I took at a look of the Xcode build output for the file I parsed. I did happen to notice that the steps that Xcode listed as 0.1 seconds were actually must smaller values. My guess is they just round up to 0.1 seconds. For example 0.007 sec appears to be rounded up to 0.1 sec. Unfortunately the total time for this target was not displayed in Xcode.

The time disparity is quite large. So I ran a dump to see if any clues could be found there. I found the same matching signature in the dump. I also verified the individual subSections from the dump matched the subSteps defined in the parse export to help verify I was looking at the same data.

I also wrote another script to do the math from the dump and got the same values.

Any thoughts on where that extra time came from? Should I expect summing the sub steps/ sub sections to match the target's duration time? Is the xcactivitylog missing information or is something being dropped on the floor?

Or am I looking at the data the wrong way where really the total time should be the target's duration plus the sum of the subStep durations? And if this is the case, how does one classify what "work" is being done by the target?

We're trying to optimize our build and find the places where we are slow, but it is a little unsettling to see a target's duration be so far off from it's subSteps

Thanks!

HTML Report Fails to parse build errors

Swift/Xcode Version

swift 5, Xcode 12.0

Problem

Hi,
I discovered a possible issue when there are no SubSteps as part of the failure.

The scenario:
attempting to archive a build where the provisioning profile is incorrect or not found. This generates a failure before build compilation of any build steps have been performed. The result is a failure in the root log activities. After doing some local testing I found that the activity logs parse the errors and are stored in the root/main document model. The issue presents its self when attempting to writeTargetFiles without any subSteps being performed yet.

an empty array of errors has been written to file but the error count has been updated in the UI so that's great but you cannot see any of the errors generated.

I added these lines here to correct it but feels off to me.

        if let buildErrors = build.errors, buildErrors.count > 0 {
           stepsWithErrors.append(build)
        }
        if let buildWarnings = build.warnings, buildWarnings.count > 0 {
            stepsWithWarnings.append(build)
        }

Report before:
Screen Shot 2020-12-03 at 2 06 30 PM

After:
Screen Shot 2020-12-03 at 2 09 39 PM

I have not done any extensive testing and would really like to get some input on this.
Thanks!

Invalid or no xcactivitylog file

Looks like there were some previous posts but none of the existing suggestions seem to work for.

Versions

Xcode 11.5

Problem

Running the following command leads to no xcactivitlylog being generated

set -o pipefail && env NSUnbufferedIO=YES xcodebuild clean build \
    -project Project.xcodeproj \
    -scheme 'Development' \
    -configuration Debug  \
    -derivedDataPath /tmp/test_build

Running the following command generates a xcactivitlylog but running xclogparser parse throws an error saying is not a valid xcactivitylog file. This just add -resultBundlePath

set -o pipefail && env NSUnbufferedIO=YES xcodebuild clean build \
    -project Project.xcodeproj \
    -scheme 'Development' \
    -configuration Debug  \
    -derivedDataPath /tmp/test_build
     -resultBundlePath /tmp/test_build/result_bundle

What's even weirder, is if I rerun the second command, it then produces a valid xcactivitylog. I've been able to repro that behavior twice now.

Is there something I should be doing here? I've also confirmed that waiting a few minutes to run xclogparser doesn't fix the problem

Expose Reporter.makeLogReporter()

I'm using XCLogParser in a command-line tool that filters out some build steps.
I would still like to be able to use the HTML reporter for the edited builds steps but HtmlReporter does not have a public initializer.
A quick way to achieve what I want would be to make Reporter.makeLogReporter() public. This way all reporters could be initialized if needed

online analysis tool which work with XCLogParser?

We use CXLogParser. we want to upload daily reports to an online analysis tool to have historical trend analysis for individual file's build or unit test of each test case? Do you know to use which online analysis tool?

Displaying the nearest matching/all available project names when no --project is found

Hi!

Right now, if you pass xclogparser the --project option and give it an incorrect project name, LogFinder throws an error

    Error. There is no directory for the project \(name) in the DerivedData
    folder in \(derivedDataDir.path). Please specify the path to the xcactivitylog
    with --file or the right DerivedData folder with --derived_data

Would it be possible to include in the message the nearest matching/all projects available in the DerivedData directory? I would find this helpful if, for example, I incorrectly enter a --project name and don't easily have the ability to inspect the DerivedData directory.

Happy to submit a PR if so, just curious if that's intentionally been omitted and if this would be welcomed. Also - love the project - and thanks for publishing it!

xcodebuild does not generate a valid xcactivitylog

I use xcodebuild command to compile my project,generate new xcactivitylog, but there is no data in it. Does anyone know how to fix it?

xcodebuild build -project xxx.xcodeproj -scheme xxx -configuration Release -sdk iphonesimulator

[Question] Is there any documentation about xcactivitylog?

Thanks for great OSS. I'll try to use this.
But if I understand correctly, xcactivitylog just has values without key name.
I'm curious about how did you name them? Is there any documentation? or this can be broken after updating Xcode?

I get some files with xcodebuild -resultBundlePath but not .xcactivitylog

Version

Version 12.0 (12A7209)

Problem

Run the following command

xcodebuild build -workspace TestDemoPod.xcworkspace -scheme TestDemoPod -configuration Debug -sdk iphonesimulator14.0 -arch x86_64 -resultBundlePath TeseDemoPod/Result

I get these files
企业微信20201210-152435@2x

I can't get .xcactivitylog file with xcodebuild -resultBundlePath command.
What should I do?
THX

10.14.4+ Support

It seems like XCLOgParser doesnt work on any MacOS after 10.14.4. I am getting this error when running the parser

"This copy of libswiftCore.dylib requires an OS version prior to 10.14.4."

Question: Can the event "Create directory..." be removed from the start of targets?

I tried this out a couple of months ago and it was correctly showing the start and stop times of each target.

Screen Shot 2021-10-19 at 2 01 51 PM

I just tried again today and for some reason it's reporting all targets starting at the same time.

Screen Shot 2021-10-19 at 2 02 15 PM

Each target is now starting with a Create directory Target.framework and Create directory Headers step that happens when the build starts. Then there's a long period of inactivity before the target actually starts building.

Screen Shot 2021-10-19 at 2 21 50 PM

As far was I can tell, both builds were done with Xcode 12.5.1 and no build settings have changed that should affect that. But XCLogParser hasn't had an update so it's unlikely to be a cause of the change in behaviour.

Any recommendations for build settings I can look at that could be causing this?

`outputPath` and `machineName` are inaccessible due to 'internal' protection level

I may be possibly doing things that are were not intended.

I had a tool I wrote in python to take the parse JSON from xclogparser which converted into data we could use with some of our other tools. I decided to expand on this, but this time to create a Swift command line tool to do this, leveraging the XCLogParser library.

I really don't care about the reporter, rather I want to use the IDEActivityLog directly.

In experimenting/getting used to the library code, I bypassed using CommandHandler and Command and created my own code based on XCLogParser source. Currently it looks something like:

let logOptions = config.logOptions()
let actionOptions = config.actionOption()

do {
    let logFinder = LogFinder()
    let logURL = try logFinder.findLatestLogWithLogOptions(logOptions)
    let activityLogParser = ActivityParser()
    let activityLog = try activityLogParser.parseActivityLogInURL(logURL, redacted: config.redact)
    // TODO: Revist
    let buildParser = ParserBuildSteps(machineName: nil)
    let buildSteps = try buildParser.parse(activityLog: activityLog)
    // TODO: Revist
    let reporterOutput = ReporterOutputFactory.makeReporterOutput(path: "")
} catch {   
}

The line following you see // TODO: Revisit is where I did things like actionOptions.machineName and actionOptions.outputPath. The real values I am using are nil and the empty string, but for code clarity, it is nice to use it as what is probably intended.

The problem here is that machineName and outputPath are not public, and hence inaccessible. LogOptions more or less gets cleanly consumed by findLatestLogWithLogOptions whereas ActionOptions tends to need it's data pulled out.

Obviously going through Command/CommandHandler works because it's internal to the library. However, to get the raw data, one doesn't have this nicety that I know of.

It would be helpful to either have getters or to mark them as public.

Add GitHub CODEOWNERS

In the MNF TSC meeting today we agreed it would be a good idea to add a GitHub CODEOWNERS file to the root of the repo to document who the maintainers of this repository are. These don't have to be blocking but will still help us keep the most up to date maintainers list possible.

Please reach out to me if you have any issues, or need anyone to be added to the org / repo.

Question around placing the script in a scheme's post-action

Hey all,

Firstly thank you for this tool, it's absolutely fantastic and is a massive help to my use case: tracking build times on DataDog.

This statement exists in the readme around placing the script within a scheme's post-action that I think might be slightly ambiguous:

In this way, the last build log can be parsed as soon as a build finishes...

I interpreted this to mean that when you build the scheme, the xcactivitylog for that build is parsed at the end of the build. However upon further investigation it looks as though the xcactivitylog is only created after the post-build script is ran, which means that it's always reporting on the current_build - 1.

This is a problem in my case as the last build could have been yesterday, and DataDog requires timestamps on any metrics to be "defined as not more than 10 minutes in the future or more than 1 hour in the past".

I managed to get around this by offloading the running of the script to the background and waiting for a second or so. This allows Xcode to generate the xcactivitylog for the current build, before it is parsed (and then sent to DataDog in my case).

If this is not a misunderstanding by me, or problem on how I've set this all up, it might be worth highlighting this in the readme?

xcactivitylog is not a valid xcactivitylog

I've been testing some builds generated with xcodebuild. The command generally looks like:

xcodebuild -workspace Project.xcworkspace -scheme Project  -derivedDataPath 'd_d/Project' clean build -resultBundlePath testResult/Project.test_result

I've noticed that when I try and run xclogparser using the --derived_data argument, I get the following error:

Project/Logs/Build/UUID.xcactivitylog is not a valid xcactivitylog

If I run file on the xcactivitylog, it returns

gzip compressed data, original size modulo 2^32 1025420490 gzip compressed data, reserved method, ASCII, has CRC, encrypted, from FAT filesystem (MS-DOS, OS/2, NT), original size modulo 2^32 1025420490

I'm not quite too sure where the reserved method, ASCII, has CRC, encrypted, from FAT filesystem (MS-DOS, OS/2, NT) came from. It consistently builds this way.

Invalid xcactivitylog file

Hi, When I use Xcode to build, It will generate xcactivitylog.
But When I use xcodebuild cli, For example

xcodebuild -workspace xx.xcworkspace -scheme xx -sdk iphoneos -showBuildTimingSummary -resultBundlePath ./a/a -derivedDataPath ./a clean build

it will generate two xcactivitylog, One is a clean job log, and one is a build job log. But the build job log is not a valid gzip.
I have to wait one day, but nothing changes.

So, Is it possible to crash in generate xcactivitylog?

Swift Type check times stopped reporting

I was using 0.2.16 and just saw that I was out of date.
Using 0.2.16 I was seeing Swift Type check times reported in the HTML report.
I updated to 0.2.26 and re-ran the report and see
Did you compile your project with the flags -Xfrontend -debug-time-expression-type-checking

Yes, I did compile with those flags. I ran a report with 0.2.16, it looked fine.
Updated to 0.2.26 and ran the report on the exact same project and nothing.
I double-checked, just in case something strange happened, and the flag is absolutely present.

The debug-time-function-bodies report is displaying properly.

Could not find an SDK that supports macOS 11.4

I try use homebrew to install, but failed.

"Could not find an SDK that supports macOS 11.4.
You may have have an outdated or incompatible Xcode."

my system version is macOS 11.4,Xcode version is 12.5

截屏2021-07-16 下午3 11 01

Read me update for feature for rootoutput to save HTML.

I see the feature for rootoutput to save HTML reports in specific folder is merged #105 but I don't see the feature in Read me with an example which I can refer

On trying below command, getting error on using rootOutput.. please suggest

 xclogparser parse --file 02E5868D-9B8A-4D9A-BB7D-846287FD9380.xcactivitylog --rootOutput path --reporter html
Unrecognized arguments: --rootOutput, "path"

Xclogparser version used : 0.2.19

Build status sanitizing

We saw some Clean succeeded at our logs as build status. And checking on ParserBuildSteps it just extract extra terms when it contains "Build ".

public func parse(activityLog: IDEActivityLog) throws -> BuildStep {
    // ...
    buildStatus = activityLog.mainSection.localizedResultString.replacingOccurrences(of: "Build ", with: "")
    // ...
    return mainBuildStep
}

Difference in how targets are handled when -resultBundlePath is used

This is very likely an Xcode thing.

I've noticed that when you build with Xcode or with xcodebuild using the -resultBundlePath, you get different output regarding targets.

More specifically, when building with Xcode, target names with the hash show up in the signature. However when xcodebuild with -resultBundlePath, the signature is empty. The target name is in the title showing up as "Target XYZ" whereas when running from Xcode it would show up as "Build target XYZ".

I wrote a tool which works with this one to better analyze the numbers, which is why I ran across this ... my code broke. I worked around it, but I wonder if you noticed this before.

Note that I could not check running xcodebuild without the -resultBundlePath to verify if the issue is actually running xcodebuild or -resultBundlePath due to #16.

xclogparser parses the second to the latest .xcactivitylog

As noted in the README for XCLogParser, .xcactivitylog file is generated a few seconds after the build action is completed. I added the xclogparser parse command into the build's post action and what I see is that the xclogparser parses the xcactivitylog for the second to the latest build log.

I am using Xcode 12.5.1 and I am not sure if this is by design or a bug. Can someone also confirm that adding xclogparser as the post build action is guaranteed to wait for the latest .xcactivitylog file to be generated and then parse it?

Swift Type Checks times table fails to load properly in HTML report if there are too many rows

With large build logs, that table throws an error in the Browser's console and as a result, the table doesn't show the search field nor is paginated. Originally reported by @BalestraPatrick

jquery-3.3.1.slim.min.js:2 Uncaught RangeError: Maximum call stack size exceeded
    at Function.oe [as find] (jquery-3.3.1.slim.min.js:2)
    at w.fn.init.find (jquery-3.3.1.slim.min.js:2)
    at HTMLTableElement.<anonymous> (jquery.dataTables.min.js:102)
    at Function.each (jquery-3.3.1.slim.min.js:2)
    at w.fn.init.each (jquery-3.3.1.slim.min.js:2)
    at w.fn.init.q [as dataTable] (jquery.dataTables.min.js:96)
    at w.fn.init.f.fn.DataTable (jquery.dataTables.min.js:179)
    at drawSwiftTypeChecks (app.js:599)
    at drawCharts (app.js:138)
    at app.js:115

Mismatch between text and image when describing Xcode Integration

On the Xcode Integration part of the documentation the text reads "To do that, open the scheme editor in a project and expand the "Build" panel on the left side." while the image displays the script being added to the "Run" phase:

post-action-run-script

I can send a PR but wanted to double check which phase it should be (or perhaps it works in both?).

Thanks for the awesome tool 🙌

Populating `documentURL` for other `BuildStepType`s

I realize this is something that Apple is doing with their xcactivitylog. It appears with CompileC and CompileSwift that the documentURL will contain the file name of the file that is being compiled. This is populated by the DVTDocumentLocation.

It seems it would also be useful to have this information available to other compile types such as CompileStoryboard and CompileXIB. It does look like we should be able to derive this from the title.

`--project` value finds incorrect project

We have a few projects that have the same prefix. For example our main project is Foo. But we also have some other smaller modules that could be something like foo-bar or FooUtils.

When calling

xclogparser parse --project Foo --reporter json

Or when using as a library, setting logOption.projectName = "Foo", we find that we get the results of foo-bar in the event foo-bar was built after Foo.

This is caused by a couple of things. First is that in findDerivedDataForProject the directory name generation is based on whether or not the directories in derived data start with projectName. The second is that this testing is done with lowercase versions of both the projectName and the directory name in derived data.

I would expect an exact name match.

Optionally so as to not break behavior, perhaps a --strictNames flag or something similar could be used.

Is XCLogParser available as a library?

I'm making a log system which stores xcode build logs.
I'd like to integrate XCLogParser into the source code with Swift Package Manager.
Does XCLogParser have an Interface to integrate other source code?

swiftTypeCheckTimes and swiftFunctionTimes only appearing for main application target

Using xclogparser v0.2.18

I've been playing around with using -Xfrontend -debug-time-expression-type-checking -Xfrontend -debug-time-function-bodies to analyze what functions and type checkers are slowing down our build times.

I've noticed that only the source files in the main application target seem to have swiftTypeCheckTimes and swiftFunctionTimes populated. All the source files in our dynamic frameworks and static libraries don't though. I've confirmed those targets are also built with the correct compiler flags, and that the method/typechecks + durations are in the build logs in xcode.

Any ideas? Thanks!

swiftFunctionTimes not always populated in certain cases

I recently turned on swift timings (-driver-time-compilation -Xfrontend -debug-time-compilation -Xfrontend -debug-time-function-bodies -Xfrontend -debug-time-expression-type-checking) and noticed a couple of things.

  • The parsing speed is pretty slow.
  • There are cases where the swiftFunctionTimes is an empty array even though it appears the data exists in the dump.

This happens when using Cocoapods. Some of the pods build by using a single CompileSwift step with all swift files as arguments (versus individual CompileSwift steps per file). This results in a Build Summary that looks like this:

Screen Shot 2019-12-12 at 10 52 34 PM

as opposed to something like this:

Screen Shot 2019-12-13 at 2 59 20 PM

This results in the signature for the step being "CompileSwift normal x86_64" versus something like "CompileSwift normal x86_64 source.swift". In other words, it is lacking the source file name.

The commandDetailDesc from the dump does contain the actual swift files (along with the other flags). I also can see that the text field contains what looks like the swift timing information. It appears xclogparse probably uses the signature and the existence of a source in the signature as the cue when to generate swiftFunctionTimes.

Crash when parsing log file

I have a log that contains the next line

0.08ms	<redacted-folder>/src/pages/headers-list/cells-content/mail-bill/analytic/PaymentAnalyticHandlerInReadView.swift:45:-0#0#0#-7@118"file://<redacted-folder>/src/pages/headers-list/cells-content/mail-bill/analytic/PaymentAnalyticHandler.swift0000000000000000^8797"CompileSwift normal x86_64 <redacted-folder>/src/pages/headers-list/cells-content/mail-bill/analytic/PaymentAnalyticHandler.swift (in target '<redacted-appname>' from project '<redacted-app>')

This line causes the crash Thread 1: Fatal error: Unexpectedly found nil while unwrapping an Optional value at SwiftCompilerTimeOptionParser.swift:59. (PaymentAnalyticHandlerInReadView.swift:45 contains simple method with a single function call.)

Seems like we should remove force optionals unwrapping

Cannot get unit tests stats due to missing test logs either in derived data or result bundle path

I'd like to gather the test coverage statistics for a given target in a project. The Xcode version that I use is 12.4. I did run tests for a project via xcodebuild command:

$ xcodebuild test -scheme Demo -sdk iphonesimulator -destination 'platform=iOS Simulator,name=iPhone 11 Pro,OS=14.4' -resultBundlePath ~/Demo/result

According to the blog post mentioned in the README file, there should be .xcactivitylog log generated at the result bundle path, however, with Xcode 12.4, it is not. There's a .xcresult file instead.

I also checked the derived data and same results.

I tried to simply parse the build activity log, but no data about unit tests has been found:

$ xclogparser parse --project Demo --reporter html

Could you please help me to figure out the way to get the unit tests coverage stats? 🙏

I will gladly update the documentation if needed 😊

Process gets killed after a while when parsing a large xcactivitylog

Our project contains a ton of targets and dependencies. The generated xcactivitylog file is around 4.2MB in size. For comparison xcactivitylog for a newly created project is around 7KB in size.

Whenever I try parsing my project's xcactivitylog using HTML reporter xclogparser's process gets killed by the system after about a minute of waiting. It works just fine on smaller projects/targets though. Also it succeeds in dumping the log into a JSON file.

Seems to me XCLogParser is just not able to process a file this big in a reasonable amount of time.

All builds were made in Xcode 12.5.1 (12E507).

Unfortunately, I cannot provide the xcactivitylog file, as it contains sensitive info. Is there a way to enable verbose output for the parse command, so that I could provide you with some more information on what's going on?

Add the Option to Generate HTML Report Directly Under --output Folder

We want to publish the HTML report for each build in Jenkins. For this purpose, we need to know the name of the root folder that includes the index.html file.
Right now, if you specify an output path using the --output flag, the report is being generated not under that folder, but another folder is being created (with the build timestamp as its name), and the report is being generated to it.
This makes the integration with the Jenkins plugin a bit tricky, since I need to either parse the output path from the shell output (the full path of index.html is being printed to the terminal), or ls to the folder I configured using the --output flag.

If I could assign the exact folder path while running XCLogParser, it would ease up the integration.
I assume that generating the timestamp folder is more relevant while running on dev local machines, to avoid duplicates, but it's less relevant in CI environments, since the build folder (including derived_data) is being cleaned after each build. So maybe an additional CI=true flag could be useful here?

WDYT?

[Question] What steps should I do after adding SPM dependency?

Hey
In the documentation when installation you say: simply use the Swift Package Manager commands directly. I already did that but my problem was trying to run xclogparse on CI (Bamboo). How can I make that the executable is there on agent without adding it manually but with Swift Package, is it doable?
Thanks!
P.S: I don't want to use Homebrew :)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.