fox-it / dissect Goto Github PK

When I run the 'services()' function on an opened Target image, I get thrown out of the generator eventually after running into the 'Unit_X-Snappy' service. The error itself is as follows: "Field 'Unit_X-Snappy' is an invalid or reserved field name".

When looking at the Dissect code I see the error is thrown at the creation of a 'TargetRecordDescriptor' of the 'ServicesPlugin' function, which is located in the following file: dissect\target\plugins\os\unix\linux\services.py.

In the mean time I can kind of fix it by putting a try-except block around de record creation, but I still miss any services after 'Unit_X-Snappy'. Anybody that can help me with this issue?

When loading the demo in chrome, the following error occurs within the browser. It appears there is a permission issue for https://try.dissect.tools/pyodide/pyodide.js.map as it is returning a 403. Reproducible on windows 10 and 11 running chrome on different ISPs.

Loading distutils
pyodide.asm.js:10 Loaded distutils
worker.ts:228 Python initialization complete
pyodide.asm.js:10 distutils already loaded from default channel
pyodide.asm.js:10 Loading micropip, pyparsing, packaging
pyodide.asm.js:10 Loaded micropip, packaging, pyparsing
pyodide.asm.js:10 Loading pyyaml
pyodide.asm.js:10 Loaded pyyaml
pyodide.asm.js:10 Loading msgpack
pyodide.asm.js:10 Loaded msgpack
pyodide.asm.js:10 Uncaught (in promise) PythonError: Traceback (most recent call last):
File "/lib/python3.10/site-packages/_pyodide/_base.py", line 435, in eval_code
.run(globals, locals)
File "/lib/python3.10/site-packages/_pyodide/_base.py", line 304, in run
coroutine = eval(self.code, globals, locals)
File "", line 11, in
File "/lib/python3.10/site-packages/dissect/target/tools/utils.py", line 15, in
from dissect.target.loaders.targetd import TargetdLoader
File "/lib/python3.10/site-packages/dissect/target/loaders/targetd.py", line 4, in
import ssl
File "/lib/python3.10/site-packages/_pyodide/_importhook.py", line 174, in find_spec
raise ModuleNotFoundError(
ModuleNotFoundError: The module 'ssl' is unvendored from the Python standard library in the Pyodide distribution, you can install it by calling: await micropip.install("ssl"). See https://pyodide.org/en/stable/usage/wasm-constraints.html for more details.

at new_error (pyodide.asm.js:10:218127)
at pyodide.asm.wasm:0xdef7c
at pyodide.asm.wasm:0xdf077
at Module._pythonexc2js (pyodide.asm.js:10:900305)
at Module.callPyObjectKwargs (pyodide.asm.js:10:119175)
at Module.callPyObject (pyodide.asm.js:10:119387)
at PyProxyClass.apply (pyodide.asm.js:10:126775)
at Object.apply (pyodide.asm.js:10:125862)
at Object.runPython (pyodide.asm.js:10:149972)
at worker.ts:245:1

DevTools failed to load source map: Could not load content for https://try.dissect.tools/pyodide/pyodide.js.map: Fetch through target failed: Target not supported; Fallback: HTTP error: status code 403, net::ERR_HTTP_RESPONSE_CODE_FAILURE

ReFS is currently not supported and therefor gives us a hard time investigating systems we encounter with it.

More information can be found on https://docs.microsoft.com/en-us/windows-server/storage/refs/refs-overview.

Add support for TOML-files inside the configuration parser

Hi,
i recently started trying dissect but i get error messages all the time. My command was:

target-query /mnt/SCHARDT.001 -f hostname,domain,os,version,ips -d ';'
2023-12-12T13:19:53.452782Z [error ] Unable to import dissect.target.plugins.filesystem.yara [dissect.target.plugin]
2023-12-12T13:19:53.605162Z [error ] /mnt/SCHARDT.001: Failed to load target with loader RawLoader('/mnt/SCHARDT.001')

TargetError: Failed to load target: /mnt/SCHARDT.001

Traceback (most recent call last):
File "/usr/local/bin/target-query", line 8, in
sys.exit(main())
^^^^^^
File "/usr/local/lib/python3.11/dist-packages/dissect/target/tools/utils.py", line 250, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/dissect/target/tools/query.py", line 243, in main
for target in Target.open_all(args.targets, args.children):
File "/usr/local/lib/python3.11/dist-packages/dissect/target/target.py", line 307, in open_all
raise TargetError(f"Failed to find any loader for targets: {paths}")
dissect.target.exceptions.TargetError: Failed to find any loader for targets: ['/mnt/SCHARDT.001']

Do you have any ideas, what I'm doing wrong?
Thanks for your help.

p.s.: There is a longer and more detailed error output if you need it.

Currently dissect requires dissect.ntfs==3.3 but dissect.target requires dissect.ntfs>=3.4.dev,<4.0.dev

This causes a ResolutionImpossible dependency conflict when installing with pip.

https://github.com/fox-it/dissect.target/blob/main/setup.py#L61
https://github.com/fox-it/dissect/blob/main/setup.py#L18

To use developement versions of all the dissect.* packages in the test cases, we are going to define a dev extra.

This can be done in tox.ini like:

[testenv]
+ extras = dev
  deps =

And in pyproject.toml like:

[project.optional-dependencies]
dev = [
  "dissect.cstruct>=4.0.dev,<5.0.dev",
  "dissect.util>=3.0.dev,<4.0.dev",
]

The versions in the default dependencies in pyproject.toml should be specified like:

dissect.cstruct>=4.dev,<5 # Before cstruct 4 is released dissect.cstruct>=4,<5 # After cstruct 4 is released 
dissect.extfs>=3,<4 # For all 3.x version, basically all older, non cstruct dissect pkgs
dissect.btrfs>=1,<2 # For all 1.x versions, all newer dissect pkgs
dissect.xyz>=3.5,<4 # For some packages that need to depend on a specific newer minimum subversion, see dissect.target & dissct.esedb pyproject.toml

The versions in the dev extra dependency in pyproject.toml should be specified as:

dissect.cstruct>=4.0.dev,<5.0.dev
dissect.extfs[dev]>=3.0.dev,<4.0.dev # For all 3.x version, basically all older, non cstruct dissect pkgs
dissect.btrfs[dev]>=1.0.dev,<2.0.dev # For all 1.x versions, all newer dissect pkgs
dissect.xyz[dev]>=3.6.dev,<4.0.dev # For some packages that need to depend on a specific newer minimum subversion, see dissect.target & dissct.esedb pyproject.toml

The dev extra should also specify any full or similar extra that is used in tox.ini's testenv of the same package, for example acquire's definition of the dev extra:

[project.optional-dependencies]
 ...
dev = [
  "acquire[full]", "dissect.cstruct>=4.0.dev,<5.0.dev",
  "dissect.target[dev]>=3.7.dev,<4.0.dev",
]

Hello,

I have attempted to forward output via RDump to Splunk using the following command:

$ target-query [PATH_OF_IMAGE] -f users | rdump -w splunk://172.20.32.126:3000?rdtag=test123
[reading from stdin]
$

I do not observe the RDTag in the output being sent to Splunk in transit (Following screenshot is from a PCAP of data sent in transit).

I also do not observe the RDTag in the data that is received by Splunk.

I only came across the ability to add the RDTag in the source code (https://github.com/fox-it/flow.record/blob/e234f617a4aab625808598385f4f5a4a4f68eea6/flow/record/adapter/splunk.py#L12).
Should the RDTag I specified be taking effect or have I done something wrong when executing the command?

PS: The version of the software I am using is as follows:

$ rdump --version
flow.record version 3.5

dear team ,
im facing an issue while execute the following command
target-query test.tar -f tasks after extract some event i get the following
File "/usr/local/bin/target-query", line 8, in
sys.exit(main())
File "/usr/local/lib/python3.10/dist-packages/dissect/target/tools/query.py", line 234, in main
for record_entries in entry:
File "/usr/local/lib/python3.10/dist-packages/dissect/target/plugins/os/windows/tasks.py", line 390, in tasks
for entry in self.parse_task(f):
File "/usr/local/lib/python3.10/dist-packages/dissect/target/plugins/os/windows/tasks.py", line 397, in parse_task
raise InvalidTaskError()
dissect.target.exceptions.InvalidTaskError: None

note that the .tar was part of acquire output

Branch the below projects with cstruct patches and dependency on >=4.0.dev <=5.0.dev

No.    | Project              | Size                 | Files to check       | Est. type fixes      | Est. failing structs    
-------+----------------------+----------------------+----------------------+----------------------+-------------------------
#1     | acquire              | 1                    | 1                    | 0                    | 0                   
#2     | foxhound             | 1                    | 1                    | 0                    | 0                   
#3     | dissect.ole          | 5                    | 1                    | 0                    | 0                   
#4     | dissect.ffs          | 6                    | 1                    | 0                    | 0                   
#5     | dissect.regf         | 6                    | 1                    | 0                    | 0                   
#6     | dissect.btrfs        | 7                    | 1                    | 0                    | 0                   
#7     | dissect.shellitem    | 8                    | 1                    | 0                    | 0                   
#8     | dissect.sql          | 8                    | 1                    | 0                    | 0                   
#9     | dissect.esedb        | 9                    | 1                    | 0                    | 0                   
#10    | dissect.vmfs         | 10                   | 1                    | 0                    | 0                   
#11    | dissect.cim          | 12                   | 1                    | 0                    | 0                   
#12    | dissect.fat          | 7                    | 2                    | 0                    | 0                   
#13    | dissect.executable   | 10                   | 2                    | 0                    | 0                   
#14    | dissect.fve          | 17                   | 2                    | 0                    | 0                   
#15    | dissect.thumbcache   | 7                    | 3                    | 0                    | 0                   
#16    | dissect.evidence     | 8                    | 3                    | 0                    | 0                   
#17    | dissect.eventlog     | 4                    | 6                    | 0                    | 0                   
#18    | dissect.squashfs     | 6                    | 2                    | 1                    | 0                   
#19    | dissect.ntfs         | 11                   | 4                    | 2                    | 0                   
#20    | dissect.xfs          | 6                    | 2                    | 4                    | 0                   
#21    | dissect.extfs        | 6                    | 4                    | 7                    | 0                   

Hi!
I really like acquire bc of the ease of dealing with images / vmdks.
Can you state what is the go-to way if I want to acquire my own artifacts? As far as I have seen, there is no option to provide a list of paths or similar. In this case, I will have to implement my own module, isn't it?
Thanks in advance for answering!

It seems that the sam plugin provides the wrong user creation timestamps, or it's intended by design, and instead provides the last modification timestamp, e.g. a users last password change. The expected creation time of the user should be "2021-12-28 06:57:23 UTC"

See below comparison with x-ways forensics and dissect, all timestamps in UTC.

x-ways

Registry report:

User ID of  | 0x000003EA (1002) |   | 2021-12-28 06:57:23

Registry viewer:
Screenshot shows last password change, which is the "creation" timestamp of dissect output below

dissect
target-query -q TestImage.E01 -f sam -j | jq . -C | less -r

{
  "hostname": "<redacted>",
  "domain": "<redacted>",
  "rid": 1002,
  "fullname": "<redacted>",
  "username": "<redacted>",
  "comment": "",
  "lockout": "1601-01-01T00:00:00.000000",
  "creation": "2021-12-28T07:41:13.536482",
  "lastlogin": "1601-01-01T00:00:00.000000",
  "flags": 528,
  "failedlogins": 0,
  "logins": 0,
  "lm": "<redacted>",
  "ntlm": "<redacted>",
  "_source": "TestImage.E01",
  "_classification": null,
  "_generated": "2022-12-28T03:27:52.017157",
  "_version": 1,
  "_type": "record",
  "_recorddescriptor": [
    "windows/registry/sam",
    1346789399
  ]
}

If this is by design, I would suggest to change "creation" timestamp to "last modified" or similar wording to avoid any confusions.

I'm getting with every command the error: TtypeError 'type' object is not subscriptable.
In installed dissect with pip install dissect on a fully patch Ubuntu 20.04 LTS system with python 3.8.10

Any idea what can be wrong?