ec-digit-csirc / sysdiagnose Goto Github PK

recent iOS has network activity reports ("App Privacy report" in the Settings). Are those also in sysdiagnose?

If you run the tools from another folder, it wont' work :

$ python ~/tools/forensic/sysdiagnose/initialyze.py file FILE      
Could not read version info, bailing out. Something is wrong: [Errno 2] No such file or directory: 'VERSION.txt'

This is because the get_version function checks for VERSION.txt in the local folder instead of the script folder. It would be nice to allow to run it from another folder

Like how to do a venv / conda env, installation of dependencies, etc. etc.

Be consistent, f-strings are the way to go.
Document a minimum python version of Python >= 3.6

• discuss with @kwouffe on which analysers are still needed.

Its buggy like the appinstallation used to

create analzyer for that.

Maybe a GPS file output + timestamp. Load it in google earth.
Use: https://github.com/EC-DIGIT-CSIRC/sysdiagnose/blob/main/analyzers/sysdiagnose-wifi-gelocation.py

In sysdiagnose-uuid2path.py +62 there is a variable which seems that it should be a different one...

def export_to_json(json, filename="./uuid2path.json"):
    json_u2p = json.dumps(json, indent=4)
    try:
        fd = open(filename, "w")
        fd.write(json_ps)       # XXX FIXME! What is json_ps?? XXX FIXME
        fd.close()
    except Exception as e:
        print(f"Impossible to dump the UUID to Path to {filename}. Reason: {str(e)}\n")
    return

@ddurvaux could you take a look pls?

• which version of Google authenticator did I have installed?

#10 only fixed that print()s have f-strings (where applicable).
However, there are more pure strings which should be converted to f-strings.

Example:

data/4/sysdiagnose_2023.05.19_09-39-55+0200_iPhone-OS_iPhone_20E252/WiFi/com.apple.wifi.known-networks.plist

• add more log files
  line 24: add more logs:
  ``timestamps_files = { ...```
• powerlog (line 70)

Go over the file and look at all TODOs

https://github.com/EC-DIGIT-CSIRC/sysdiagnose/blob/main/analyzers/sysdiagnose-timeliner.py#L60

--> Implement it and fix the parser as well (might be wrong) . Take a look at sysdiagnose-powerlogs.py (probably sqlite2json)

settle on british or US english, not both.

• see all Wifi (B)SSIDs that the iPhone saw
• see which Wifis the iPhone connected to
• check if we can also see the mac addresses and map to device types (?)
• update the timeliner for it
• be able to detect where the WiFi entry came from (iCloud sync?)

Today its handled with

d[k]=binascii.hexlify(v).decode('utf-8')

But it should handle differently outputs from the plist module

• Data
• UiD

depending on the data it output with b' or b"

Maybe reading the code of the plist module would figure out how they handle different classes.

• decide on which way: developer certificate or via a jailbroken iOS
• see in log files what the installation says
• write analyzer
• app installation should be visible in the timeliner (--> @aaronkaplan )

• agree on the scenarios
• rehears them internally
• screen record them

Retrieve proces and their PID from taskinfo.txt.

• AWS instance with enough RAM (128GB) + NVMEs
• ssh tunnel to VM
• load data into it

Apply this to all files where the version check is present. It's really a useless if statement.
Reasoning: Since all files, use python3 syntax only (f-strings, print(), etc.), the file won't run anyway in python2.x . So get rid of the version check all together! If you try to run it in python2, the interpreter will spit out the file anyway.

"z" -> "s"
analyZer or analyser

I identified 2 types of IPS files located in ./crashes_and_spins/ :

• panic-full-XXX,ips
• stacks-*.ips

Should be easy as the IPS format is... a json.
So maybe a parser to put together all the IPS files so it is easier to analyse/ingest in a consistent way

Possible scenarios:

• list of all wifi networks seen (+ timestamp) the device has connected to (+ BSSIDs)
• app installation: when was which app installed? app install history
• ... TBD

Example of a broken timestamp format:

+052643-10-21T13:46:40.000Z

    parser.add_option("-i", dest="inputfile", action="store", type="string",            # XXX FIXME! Here we should pass on a glob **.plist like in mobileactivation.py's main()

undefined

The option to generate a graphviz graph seems to have dissapear.
Fix the option.

Parser is currently outpouting to stdout

docstrings docstrings docstrings

• create a hello world / calc.exe app which does nothing
• It is just there to demo that we can detect rogue apps (-> #27 )

Timeliner complains about JSON format of powerlog export

ERROR while extracting timestamp from ./parsed_data//4//sysdiagnose-powerlogs.json. Reason: not all arguments converted during string formatting

produces:

"{\n    \"application\": [],\n    \"asset\": [],\n    \"client\": [],\n    \"download_policy\": [],\n    \"download_state\": [],\n    \"download\": [],\n    \"finished_download\": [],\n    \"job_restore\": [],\n    \"job_software\": [],\n    \"job\": [],\n    \"persistent_job\": [],\n    \"persistent_manager\": [],\n    \"purchase\": []\n}

aaron@host:~/Desktop/git/work/ec/sysdiagnose$ python initialyze.py file data/sysdiagnose_2023.02.24_15-37-15+0100_iPhone-OS_iPhone_20D67.tar.gz
fff3dcf24bed8acfc7734a22b0b29565e6890a425f90d91ae4789d5b874e9de7
Traceback (most recent call last):
  File "/Users/aaron/Desktop/git/work/ec/sysdiagnose/initialyze.py", line 290, in <module>
    main()
  File "/Users/aaron/Desktop/git/work/ec/sysdiagnose/initialyze.py", line 278, in main
    init(arguments['<sysdiagnose_file>'])
  File "/Users/aaron/Desktop/git/work/ec/sysdiagnose/initialyze.py", line 225, in init
    with open(new_case_json['sysdiagnose.log'], 'r') as f:
FileNotFoundError: [Errno 2] No such file or directory: './data/1/sysdiagnose_2023.02.24_15-37-15+0100_iPhone-OS_iPhone_20D67/sysdiagnose.log'

--> identify why and fix

TBD if we want to use python's logger. That's a clearer way to separate log messages from print messages.. Related to #10 however, since the logger prefers late bindings and not f-strings.

Need to implement the export and parsing via httsp://github.com/ydkhatri/UnifiedLogReader. The UnifiedLogReader has a different format than the OS X /usr/bin/log show command.

• remove the "sysdiagnose-" prefix from the parser and analyser names
• s/-/_/g in the parser/analyser names
• test if everything still works
• do it in a separate branch

This is a precondition to #40

While the timeline is compatible with TimeSketch, no events are shown there :(.

type hints type hints type hints

Useful for debugging

Building parsers for:

• com.apple.wifi.plist
• com.apple.wifi-private-mac-networks.plist
• com.apple.wifi-known-networks.plist
• com.apple.wifi.recent-networks.json
• wifi_scan.txt and wifi_scan_cache.txt
• security.txt

subject says it all

• unit tests for all parsers
• for all analysers
• create a github workflow for every commit
• use codecov or similar to measure