Code Monkey home page Code Monkey logo

ddbnl / office365-audit-log-collector Goto Github PK

View Code? Open in Web Editor NEW
96.0 96.0 38.0 255.07 MB

Collect / retrieve Office365, AzureAD and DLP audit logs and output to PRTG, Azure Log Analytics Workspace, SQL, Graylog, Fluentd, and/or file output.

Home Page: https://ddbnl.github.io/office365-audit-log-collector/

License: MIT License

Rust 99.74% Dockerfile 0.26%
api audit audit-log azure azuread collect extract extractor fluentd graylog log log-analytics-workspace log-collector office-365 office365 powerbi prtg retrieve sql workspace

office365-audit-log-collector's People

Contributors

ddbnl avatar www avatar yasuhiroharada avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

office365-audit-log-collector's Issues

How to run LINUX-OfficeAuditLogCollector-V2.0

Hi,
I would like to thank you for the awesome work this project.
I got the Windows version working without any issues and get the messages in Graylog.
I would like to run the Linux version, but usage is not documented.
Any chance you could help me out.
TY in advanced for your feedback.
BR
GG

Memory Leak?

Good morning and thank you so much for putting this program together!

I run office365-audit-log-collector in a LXC. Generally, I give it about 1GB RAM. However, it quickly hangs and locks up the container pegging at 1GB. So I increase it and it pegs the RAM, again. Raise it again and, same result. So it doesn't seem resources are the issue as the log-collector will take whatever you give it and lock up the container.

Is there anything I can do to prevent this activity?

Here is my config, in relevant part, thank you!:

log:  # Log settings. Debug will severely decrease performance                                                                                               
  path: '/var/log/officecollector/collector.log'                                                                                                                              
  debug: False                                                                                                                                                                               
collect:  # Settings determining which audit logs to collect and how to do it                                                               
  contentTypes:                                                                                                                                                                  
    Audit.General: True                                                                                                                                     
    Audit.AzureActiveDirectory: True                                                                                                                                 
    Audit.Exchange: True                                                                                                                                               
    Audit.SharePoint: True                                                                                                                                                              
    DLP.All: True                                                                                                                                                                       
  rustEngine: True  # Use False to revert to the old Python engine. If running from python instead of executable, make sure to install the python wheel in the RustEngineWheels folder                                                          
#  schedule: 0 0 10  # How often to run in days/hours/minutes. Delete this line to just run once and exit.                  
  maxThreads: 50  # Maximum number of simultaneous threads retrieving logs                                                                      
  retries: 3  # Times to retry retrieving a content blob if it fails                                                                                   
  retryCooldown: 30  # Seconds to wait before retrying retrieving a content blob                                                        
  autoSubscribe: True  # Automatically subscribe to collected content types. Never unsubscribes from anything.   
  skipKnownLogs: True  # Remember retrieved log ID's, don't collect them twice                                                                     
  resume: False  # Remember last run time, resume collecting from there next run                                                     
  hoursToCollect: 3  #Look back this many hours for audit logs (can be overwritten by resume)                                                  
filter:  # Only logs that match ALL filters for a content type are collected. Leave empty to collect all                          
  Audit.General:                                                                                                                                          
  Audit.AzureActiveDirectory:                                                                                                                                         
  Audit.Exchange:                                                                                                                                                        
  Audit.SharePoint:                                                                                                                                                                        
  DLP.All:

I run via crontab:
*/10 * * * * /root/officeauditlogcollector/officecollector.sh

Improvement: Run Collector as SingleInstance (Linux)

I run the collector via cron. I noticed that the collector does not check if an instance is already running before starting. This can cause multiple instances to run in an infinite loop and load the CPU to 100%. I would need to write myself a bash-script for this requirement.

I would find an additional - native - configuration option helpful, like:

collect:
  runAsSingleInstance: True

Error access_token missing field

Hi guys !

I've encountered an issue when trying to run LINUX-OfficeAuditLogCollector-V2.3 using this command : ./LINUX-OfficeAuditLogCollector-V2.3 client_id="xxxxx" tenant_id="xxxx" secret_key="_xxxx" --config graylog.yaml

Here is what is returned : Starting run @ 2023-11-21 11:20:11.651446. Content: deque(['Audit.General', 'Audit.AzureActiveDirectory', 'Audit.Exchange', 'Audit.SharePoint', 'DLP.All']).
thread '' panicked at 'called Result::unwrap() on an Err value: reqwest::Error { kind: Decode, source: Error("missing field access_token", line: 1, column: 560) }', src/api_connection.rs:57:14
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace
Traceback (most recent call last):
File "AuditLogCollector.py", line 712, in
File "AuditLogCollector.py", line 71, in run
File "AuditLogCollector.py", line 84, in run_once
File "AuditLogCollector.py", line 105, in receive_results_from_rust_engine
pyo3_runtime.PanicException: called Result::unwrap() on an Err value: reqwest::Error { kind: Decode, source: Error("missing field access_token", line: 1, column: 560) }
[109274] Failed to execute script 'AuditLogCollector' due to unhandled exception!

Do you guys maybe have an idea of what's wrong ?
Thanks in advance.

AuditLogCollector.py - AttributeError: 'collections.deque' object has no attribute 'insert'

So close to getting this running but am hitting the following issue when starting the AuditLogCollector.py script.

Exception in thread Thread-2:
Traceback (most recent call last):
File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner
self.run()
File "/usr/lib/python3.4/threading.py", line 868, in run
self._target(*self._args, **self._kwargs)
File "AuditLogCollector.py", line 119, in monitor_blobs_to_collect
self._graylog_interface.stop()
File "/var/opt/office365-audit-log-collector/GraylogInterface.py", line 24, in stop
self.queue.insert(0 if not gracefully else -1, 'stop monitor thread')
AttributeError: 'collections.deque' object has no attribute 'insert'

Lacking certificate validation

The HTTP(S) requests performed by the scripts have been configured to not validate the identity (certificate) of the Office365 services:

[...]
r = requests.post(auth_url, headers=headers, data=data, verify=False)
[...]

This makes the connections vulnerable to Man-In-The-Middle attacks, which could result in the exposure of access credentials and other sensitive information.

Since the scripts use the "requests" module (which supports/uses "certifi") and Office365 provides verifiable certificates, it should not be an issue to enable validation.

Issue running LINUX-OfficeAuditLogCollector-V2.3

Hi guys !

I'm new using Graylog and wanted to use this log collector to get 365 defender logs on Graylog.
I've tried running LINUX-OfficeAuditLogCollector-V2.3 but can't seem to succeed do i have to use a CRON to run it automatically ?

Sorry if i'm mistaken i'm a beginner.
Best regards.

OfficeCollector Hangs and does not pull new logs

Good afternoon. First and foremost, thank you for your hard work and for putting this together. It is truly amazing.

Unfortunately, after a month of working, I have encountered a problem. I use your OfficeAuditLogCollector to pull data/logs from Office365/Azure for use in Graylog. I followed the how-to you made for the Graylog community and it worked perfectly.

However, as of today, it no longer pulls data. If I run the application it will give me:

.Making API request using URL: "https://manage.office.com/api/v1.0/<redacted>/activity/feed/subscriptions/list"
Starting new HTTPS connection (1): login.microsoftonline.com:443
https://login.microsoftonline.com:443 "POST /<redacted>/oauth2/token HTTP/1.1" 200 1482
Logged in
Starting new HTTPS connection (1): manage.office.com:443
https://manage.office.com:443 "GET /api/v1.0/<redacted>/activity/feed/subscriptions/list HTTP/1.1" 200 342
Starting run @ 2022-07-05 17:43:06.895347. Content: deque(['Audit.General', 'Audit.AzureActiveDirectory', 'Audit.Exchange', 'Audit.SharePoint', 'DLP.All']).
Traceback (most recent call last):
  File "AuditLogCollector.py", line 712, in <module>
  File "AuditLogCollector.py", line 71, in run
  File "AuditLogCollector.py", line 84, in run_once
  File "AuditLogCollector.py", line 125, in receive_results_from_rust_engine
  File "AuditLogCollector.py", line 448, in _handle_retrieved_content
TypeError: string indices must be integers
[5436] Failed to execute script 'AuditLogCollector' due to unhandled exception!
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: SendError { .. }', src/api_connection.rs:254:57
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Any suggestions?

Thank you!

Trouble getting it started

Hi There,
ist there any working config for Linux?

I have problems to install this in Linux (Ubuntu 20.04).
I have registered an app, but fail to run the script.

Issues when connecting to GCC Tenant

I've been having issues connecting using a few different systems and I'm thinking it's related to being a GCC tenant and using a different management API URL. I will paste the two different error snippets I've received.

Is it possible to set the URL in the config.yaml file?

('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))
[5724] Failed to execute script 'AuditLogCollector' due to unhandled exception!
HTTPSConnectionPool(host='manage.office.com', port=443): Max retries exceeded with url: /api/v1.0/[TENANT_ID]/activity/feed/subscriptions/list (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1129)')))
[36496] Failed to execute script 'AuditLogCollector' due to unhandled exception!

Error: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

When running the Collector (Version 2.1) I receive following Error:

Starting run @ 2024-01-16 12:53:51.777429. Content: deque(['Audit.General', 'Audit.Exchange', 'Audit.SharePoint', 'DLP.All']).
Traceback (most recent call last):
  File "AuditLogCollector.py", line 699, in <module>
  File "AuditLogCollector.py", line 67, in run
  File "AuditLogCollector.py", line 80, in run_once
  File "AuditLogCollector.py", line 113, in receive_results_from_rust_engine
  File "json/__init__.py", line 357, in loads
  File "json/decoder.py", line 337, in decode
  File "json/decoder.py", line 355, in raw_decode
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
[449236] Failed to execute script 'AuditLogCollector' due to unhandled exception!
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: SendError { .. }', src/api_connection.rs:254:57
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Any Idea whats going on here?

uname -a
Linux hostname 6.1.0-17-cloud-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.69-1 (2023-12-30) x86_64 GNU/Linux

Data stored in SQL is getting deleted

Hello,

I have saved the output to SQL. But i see that data is getting deletd from the tables. Also no last_run file is getting created under windows. I have schedueld task to run the script every hour.

Thanks

No new logs since first collection / initial run

Dear @ddbnl

I noticed that since the initial collection run, I don't get any new logs.

Making API request using URL: "https://manage.office.com/api/v1.0/TENANT-ID/activity/feed/subscriptions/list"
Starting new HTTPS connection (1): login.microsoftonline.com:443
https://login.microsoftonline.com:443 "POST /TENANT-ID/oauth2/token HTTP/1.1" 200 1510
Logged in
Starting new HTTPS connection (1): manage.office.com:443
https://manage.office.com:443 "GET /api/v1.0/TENANT-ID/activity/feed/subscriptions/list HTTP/1.1" 200 342
Starting run @ 2022-05-03 09:39:11.194626. Content: deque(['Audit.General', 'Audit.AzureActiveDirectory', 'Audit.Exchange', 'Audit.SharePoint', 'DLP.All']).
Audit.General - resuming from: 2022-05-03 07:30:03+00:00
Getting available content for type: "Audit.General"
Audit.AzureActiveDirectory - resuming from: 2022-05-03 07:30:03+00:00
Retrieving Audit.General. Start time: 2022-05-03T07:30:03+00:00. End time: 2022-05-03T07:39:11.
Getting available content for type: "Audit.AzureActiveDirectory"
Making API request using URL: "https://manage.office.com/api/v1.0/TENANT-ID/activity/feed/subscriptions/content?contentType=Audit.General&startTime=2022-05-03T07:30:03+00:00&endTime=2022-05-03T07:39:11"
Audit.Exchange - resuming from: 2022-05-03 07:30:03+00:00
Retrieving Audit.AzureActiveDirectory. Start time: 2022-05-03T07:30:03+00:00. End time: 2022-05-03T07:39:11.
Starting new HTTPS connection (1): login.microsoftonline.com:443
Getting available content for type: "Audit.Exchange"
Audit.SharePoint - resuming from: 2022-05-03 07:30:03+00:00
Making API request using URL: "https://manage.office.com/api/v1.0/TENANT-ID/activity/feed/subscriptions/content?contentType=Audit.AzureActiveDirectory&startTime=2022-05-03T07:30:03+00:00&endTime=2022-05-03T07:39:11"
Retrieving Audit.Exchange. Start time: 2022-05-03T07:30:03+00:00. End time: 2022-05-03T07:39:11.
Starting new HTTPS connection (1): login.microsoftonline.com:443
Getting available content for type: "Audit.SharePoint"
Making API request using URL: "https://manage.office.com/api/v1.0/TENANT-ID/activity/feed/subscriptions/content?contentType=Audit.Exchange&startTime=2022-05-03T07:30:03+00:00&endTime=2022-05-03T07:39:11"
DLP.All - resuming from: 2022-05-03 07:30:03+00:00
Retrieving Audit.SharePoint. Start time: 2022-05-03T07:30:03+00:00. End time: 2022-05-03T07:39:11.
Starting new HTTPS connection (1): login.microsoftonline.com:443
https://login.microsoftonline.com:443 "POST /TENANT-ID/oauth2/token HTTP/1.1" 200 1510
Getting available content for type: "DLP.All"
Making API request using URL: "https://manage.office.com/api/v1.0/TENANT-ID/activity/feed/subscriptions/content?contentType=Audit.SharePoint&startTime=2022-05-03T07:30:03+00:00&endTime=2022-05-03T07:39:11"
https://login.microsoftonline.com:443 "POST /TENANT-ID/oauth2/token HTTP/1.1" 200 1510
Logged in
Retrieving DLP.All. Start time: 2022-05-03T07:30:03+00:00. End time: 2022-05-03T07:39:11.
Starting new HTTPS connection (1): login.microsoftonline.com:443
Logged in
Starting new HTTPS connection (1): manage.office.com:443
Making API request using URL: "https://manage.office.com/api/v1.0/TENANT-ID/activity/feed/subscriptions/content?contentType=DLP.All&startTime=2022-05-03T07:30:03+00:00&endTime=2022-05-03T07:39:11&PublisherIdentifier=TENANT-ID"
https://login.microsoftonline.com:443 "POST /TENANT-ID/oauth2/token HTTP/1.1" 200 1510
Starting new HTTPS connection (1): manage.office.com:443
Starting new HTTPS connection (1): manage.office.com:443
Logged in
https://login.microsoftonline.com:443 "POST /TENANT-ID/oauth2/token HTTP/1.1" 200 1510
Starting new HTTPS connection (1): manage.office.com:443
Logged in
https://manage.office.com:443 "GET /api/v1.0/TENANT-ID/activity/feed/subscriptions/content?contentType=Audit.General&startTime=2022-05-03T07:30:03+00:00&endTime=2022-05-03T07:39:11 HTTP/1.1" 400 82
https://manage.office.com:443 "GET /api/v1.0/TENANT-ID/activity/feed/subscriptions/content?contentType=DLP.All&startTime=2022-05-03T07:30:03+00:00&endTime=2022-05-03T07:39:11&PublisherIdentifier=TENANT-ID HTTP/1.1" 400 82
Got 1 content blobs of type: "Audit.General"
Starting new HTTPS connection (1): manage.office.com:443
Got 1 content blobs of type: "DLP.All"
https://manage.office.com:443 "GET /api/v1.0/TENANT-ID/activity/feed/subscriptions/content?contentType=Audit.AzureActiveDirectory&startTime=2022-05-03T07:30:03+00:00&endTime=2022-05-03T07:39:11 HTTP/1.1" 400 82
Got 1 content blobs of type: "Audit.AzureActiveDirectory"
https://manage.office.com:443 "GET /api/v1.0/TENANT-ID/activity/feed/subscriptions/content?contentType=Audit.Exchange&startTime=2022-05-03T07:30:03+00:00&endTime=2022-05-03T07:39:11 HTTP/1.1" 400 82
Got 1 content blobs of type: "Audit.Exchange"
https://manage.office.com:443 "GET /api/v1.0/TENANT-ID/activity/feed/subscriptions/content?contentType=Audit.SharePoint&startTime=2022-05-03T07:30:03+00:00&endTime=2022-05-03T07:39:11 HTTP/1.1" 400 82
Got 1 content blobs of type: "Audit.SharePoint"
Finished. Total logs retrieved: 0. Total logs with errors: 0. Run time: 0:00:01.596500.
GraylogInterface reports: 0 successfully sent, 0 errors

My settings:

log:  # Log settings. Debug will severely decrease performance
  path: '/var/log/office365-audit-log-collector.log'
  debug: True
collect:  # Settings determining which audit logs to collect and how to do it
  contentTypes:
    Audit.General: True
    Audit.AzureActiveDirectory: True
    Audit.Exchange: True
    Audit.SharePoint: True
    DLP.All: True
  maxThreads: 50
  retries: 3  # Times to retry retrieving a content blob if it fails
  retryCooldown: 3  # Seconds to wait before retrying retrieving a content blob
  autoSubscribe: True  # Automatically subscribe to collected content types. Never unsubscribes from anything.
  skipKnownLogs: True  # Remember retrieved log ID's, don't collect them twice
  resume: True  # Remember last run time, resume collecting from there next run
  hoursToCollect: 24 # Look back this many hours for audit logs (can be overwritten by resume)
filter:  # Only logs that match ALL filters for a content type are collected. Leave empty to collect all
  Audit.General:
  Audit.AzureActiveDirectory:
  Audit.Exchange:
  Audit.SharePoint:
  DLP.All:
output:
  graylog:
    enabled: true
    address: 127.0.0.1
    port: 5555

last_run_times
{"Audit.General": "2022-05-03T07:39:11Z", "Audit.AzureActiveDirectory": "2022-05-03T07:39:11Z", "Audit.Exchange": "2022-05-03T07:39:11Z", "Audit.SharePoint": "2022-05-03T07:39:11Z", "DLP.All": "2022-05-03T07:39:11Z"}%

known_content is empty... normal?

known_logs has content but not up2date (nothing from today or yesterday for example)

Any hints how to debug this?

First thing I will try is backup known_* & last_run_times files, remove them and run again.

Finished. Total logs retrieved: 15074. Total logs with errors: 0. Run time: 0:00:14.717247. GraylogInterface reports: 15074 successfully sent, 0 errors

This works and I get new logs.

Panic: `OS can't spawn a new worker thread`

I'm getting this error sometimes:

thread '<unnamed>' panicked at 'OS can't spawn a new worker thread: Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }', /home/azureuser/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.17.0/src/runtime/blocking/pool.rs:291:14

I'll run with RUST_BACKTRACE=1 to try and get more info, I have no other signals of what could be happening.

Feature Request - Collect since previous one

Collect is based on 1 hour time delta. Sometimes, collect may not have start for some times, and so data will be missed.

Would be better to keep a timestamp, like in known_content file, to calculate correct start date to use.

Filter not working

Hi All

i am trying to collect only logs from a specific Workload but the filter option is not working for me. To perform some tests i have merged the full config with the filter one without success.

Following my config:

log:  # Log settings. Debug will severely decrease performance
  path: 'collector.log'
  debug: True
collect:  # Settings determining which audit logs to collect and how to do it
  contentTypes:
    Audit.General: True
    Audit.AzureActiveDirectory: True
    Audit.SharePoint: True
    Audit.Exchange: False
    DLP.All: False
  rustEngine: True  # Use False to revert to the old Python engine. If running from python instead of executable, make sure to install the python wheel in the RustEngineWheels folder
#  schedule: 0 1 0  # How often to run in days/hours/minutes. Delete this line to just run once and exit.
  maxThreads: 50  # Maximum number of simultaneous threads retrieving logs
  retries: 3  # Times to retry retrieving a content blob if it fails
  retryCooldown: 3  # Seconds to wait before retrying retrieving a content blob
  autoSubscribe: True  # Automatically subscribe to collected content types. Never unsubscribes from anything.
  skipKnownLogs: True  # Remember retrieved log ID's, don't collect them twice
  resume: False  # Remember last run time, resume collecting from there next run
  hoursToCollect: 24  # Look back this many hours for audit logs (can be overwritten by resume)
filter:  # Only logs that match ALL filters for a content type are collected. Leave empty to collect all
  Audit.General:
    Policy: Spoof
  Audit.AzureActiveDirectory:
    Operation: UserLoginFailed
  Audit.SharePoint:
    Operation: FileDeleted
#  Audit.AzureActiveDirectory:
#  Audit.Exchange:
#  Audit.SharePoint:
#  DLP.All:
output:
  file:  # CSV output
    enabled: True
    separateByContentType: True  # Creates a separate CSV file for each content type, using file name from 'path' as a prefix
    path: 'output.csv'
    separator: ';'
    cacheSize: 500000  # Amount of logs to cache until each CSV commit, larger=faster but eats more memory
  azureLogAnalytics:
    enabled: False
    workspaceId:
    sharedKey:
    maxThreads: 50  # Maximum simultaneous threads sending logs to workspace
  azureTable:  # Provide connection string to executable at runtime with --table-string
    enabled: False
    tableName: AuditLogs  # Name of the table inside the storage account
    maxThreads: 10  # Maximum simultaneous threads sending logs to Table
  azureBlob:  # Write CSV to a blob container. Provide connection string to executable at runtime with --blob-string
    enabled: False
    containerName: AuditLogs  # Name of the container inside storage account
    blobName: AuditLog  # When separatedByContentType is true, this is used as file prefix and becomes e.g. AuditLog_AuditExchange.csv
    tempPath: './output'
    separateByContentType: True
    separator: ';'
    cacheSize: 500000  # Amount of logs to cache until each CSV commit, larger=faster but eats more memory
  sql:  # Provide connection string to executable at runtime with --sql-string
    enabled: False
    cacheSize: 500000  # Amount of logs to cache until each SQL commit, larger=faster but eats more memory
    chunkSize: 2000  # Amount of rows to write simultaneously to SQL, in most cases just set it as high as your DB allows. COUNT errors = too high
  graylog:
    enabled: False
    address:
    port:
  prtg:
    enabled: False
    channels:
  fluentd:
    enabled: False
    tenantName:
    address:
    port:

In the csv that I get i see all the entries of the Audit logs not only the filtered ones.

What am I missing?

Thx
Mattia

Getting occasional failures, via Linux cron job script [latest binary]

Typically shows the same output every time it errs:

Starting run @ 2022-10-05 06:50:04.387915. Content: deque(['Audit.General', 'Audit.AzureActiveDirectory', 'Audit.Exchange', 'Audit.SharePoint', 'DLP.All']).
Traceback (most recent call last):
File "AuditLogCollector.py", line 699, in
File "AuditLogCollector.py", line 67, in run
File "AuditLogCollector.py", line 80, in run_once
File "AuditLogCollector.py", line 113, in receive_results_from_rust_engine
File "json/init.py", line 357, in loads
File "json/decoder.py", line 337, in decode
File "json/decoder.py", line 355, in raw_decode
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
[1061404] Failed to execute script 'AuditLogCollector' due to unhandled exception!
thread '' panicked at 'called Result::unwrap() on an Err value: SendError { .. }', src/api_connection.rs:254:57
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

TypeError: string indices must be integers

Hi,

I'm trying to get your nice solution running but it fails on start.

ENV: Ubuntu 20.04

./Linux/LINUX-OfficeAuditLogCollector-V1.3 AAD-Tenant-ID AAD-App-ID AAD-App-SecretKey --config ./fullConfig.yaml

Starting run @ 2022-04-27 16:03:17.386048. Content: ['Audit.General', 'Audit.AzureActiveDirectory', 'Audit.Exchange', 'Audit.SharePoint', 'DLP.All'].
Traceback (most recent call last):
  File "AuditLogCollector.py", line 740, in <module>
  File "AuditLogCollector.py", line 299, in run_once
  File "AuditLogCollector.py", line 279, in _prepare_to_run
  File "AuditLogCollector.py", line 383, in _auto_subscribe
TypeError: string indices must be integers
[118569] Failed to execute script 'AuditLogCollector' due to unhandled exception!

fullConfig.yaml:

log:  # Log settings. Debug will severely decrease performance
  path: 'collector.log'
  debug: False
collect:  # Settings determining which audit logs to collect and how to do it
  contentTypes:
    Audit.General: True
    Audit.AzureActiveDirectory: True
    Audit.Exchange: True
    Audit.SharePoint: True
    DLP.All: True
  maxThreads: 50
  retries: 3  # Times to retry retrieving a content blob if it fails
  retryCooldown: 3  # Seconds to wait before retrying retrieving a content blob
  autoSubscribe: True  # Automatically subscribe to collected content types. Never unsubscribes from anything.
  skipKnownLogs: True  # Remember retrieved log ID's, don't collect them twice
  resume: True  # Remember last run time, resume collecting from there next run
  hoursToCollect: 24  # Look back this many hours for audit logs (can be overwritten by resume)
filter:  # Only logs that match ALL filters for a content type are collected. Leave empty to collect all
  Audit.General:
  Audit.AzureActiveDirectory:
  Audit.Exchange:
  Audit.SharePoint:
  DLP.All:
output:
  file:  # CSV output
    enabled: False
    separateByContentType: True  # Creates a separate CSV file for each content type, appends content name to path
    path: 'output'
    separator: ';'
    cacheSize: 500000  # Amount of logs to cache until each CSV commit, larger=faster but eats more memory
  azureLogAnalytics:
    enabled: False
    workspaceId:
    sharedKey:
  azureTable:  # Provide connection string to executable at runtime with --table-string
    enabled: False
    tableName: AuditLogs  # Name of the table inside the storage account
  azureBlob:  # Write CSV to a blob container. Provide connection string to executable at runtime with --blob-string
    enabled: False
    containerName: AuditLogs  # Name of the container inside storage account
    blobName: AuditLog  # When separatedByContentType is true, this is used as file prefix and becomes e.g. AuditLog_AuditExchange.csv
    tempPath: './output'
    separateByContentType: True
    separator: ';'
    cacheSize: 500000  # Amount of logs to cache until each CSV commit, larger=faster but eats more memory
  sql:  # Provide connection string to executable at runtime with --sql-string
    enabled: False
    cacheSize: 500000  # Amount of logs to cache until each SQL commit, larger=faster but eats more memory
    chunkSize: 2000  # Amount of rows to write simultaneously to SQL, in most cases just set it as high as your DB allows. COUNT errors = too high
  graylog:
    enabled: true
    address: 127.0.0.1
    port: 5555
  prtg:
    enabled: False
    channels:

Docker image?

Great tool, thanks!
Are you planning on publishing a Docker image to Docker Hub? We could send a PR.

TypeError: string indices must be integers

I receive the following error when running the collector. I read code fairly well but I cannot seem to locate the error causing the issue.

AuditLogCollector.py --general --exchange --azure_ad --sharepoint --dlp -g -gA -gP

Exception in thread Thread-2:
Traceback (most recent call last):
File line 916, in _bootstrap_inner
self.run()
File line 864, in run
self._target(*self._args, **self._kwargs)
File "AuditLogCollector.py", line 111, in monitor_blobs_to_collect
logging.log(level=logging.DEBUG, msg='Retrieving content blob: "{0}"'.format(blob_json['contentUri']))

TypeErrror:string indices must be integers.

I have removed all paths and IDs for security.

Thanks

Daily run after first day of CSV collection prepends extraneous semi-colons on new entries.

I've been running this for a week in a scheduled task to collect logs to CSV. It has run successfully over this time, but now that I examine the output, I see that each run after the first run adds extraneous semi-colons for each new entry.

EDIT3:- I note that the Row of Header Names is removed from the CSV after the first run too.

The amount of added semi colons doubles on each daily run. This makes importing to excel impossible as the data importer freaks out.

I've included the YAML I'm using - is there a mistake in there somewhere maybe?

image
CSVConfig.zip

Edit - Manually re-ran the BAT I use and caught this output.....

Starting run @ 2023-06-20 12:12:22.184290. Content: deque(['Audit.General', 'Audit.AzureActiveDirectory', 'Audit.Exchange', 'Audit.SharePoint', 'DLP.All']).
Rust engine finished receiving all content
Interfaces\FileInterface.py:96: DtypeWarning: Columns (0,1,2,3,5,6,9,10,11,12,14,15,16,17,18,19,20,22,23,24,25,27,28,29,30,31,33,34,37,38,39,40,42,43,44,45,46,47,48,50,51,52,53,55,56,57,58,59,61,62,65,66,67,68,70,71,72,73,74,75,76,78,79,80,81,83,84,85,86,87,89,90,93,94,95,96,98,99,100,101,102,103,104,106,107,108,109,111,112,113,114,115,117,118,121,122,123,124,126,127,128,129,130,131,132,134,135,136,137,139,140,141,142,143,145,146,149,150,151,152,154,155,156,157,158,159,160,162,163,164,165,167,168,169,170,171,173,174,177,178,179,180,182,183,184,185,186,187,188,190,191,192,193,195,196,197,198,199,201,202,205,206,207,208,210,211,212,213,214,215,216,218,219,220,221,223,224,225,226,227,229,230,233,234,235,236,238,239,240,241,242,243,244,246,247,248,249,251,252,253,254,255,257,258,261,262,263,265,266,267,268,269,270,272,273,274,275,276,277,279,280,281,282,283,285,286,289,290,291,292,294,295,296,297,298,299,300,302,303,304,305,307,308,309,310,311,313,314,317,318,319,320,322,323,324,325,326,327,328,330,331,332,333,335,336,337,338,339,341,342,345,346,347,348,350,351,352,353,354,355,356,358,359,360,361,363,364,365,366,367,369,370,373,374,375,376,378,379,380,381,382,383,384,386,387,388,389,391) have mixed types. Specify dtype option on import or set low_memory=False.
Writing 2 logs of type Audit.AzureActiveDirectory to F:\Office365 audit log collector\Logs\365Log_AuditAzureActiveDirectory.csv
Interfaces\FileInterface.py:96: DtypeWarning: Columns (0,1,2,3,4,6,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,28,30,31,32,33,34,35,37,38,39,40,41,42,43,48,50,51,52,53,54,55,56,57,58,59,60,61,62,64,65,66,67,68,69,70,72,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,98,99,100,101,102,103,105,106,111,113,114,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,139,142,143,144,145,146,147,148,149,150,151,152,153,154,155,156,157,158,159,160,161,162,163,165,166,167,168,169,171,173,174,175,180,182,183,184,185,186,187,188,189,190,191,192,193,194,195,196,197,198,199,200,201,202,203,205,208,209,210,211,212,213,214,215,216,217,218,219,220,221,222,223,224,225,226,227,228,230,231,232,233,234,235,236,237,239,241,242,243,244,245,246,251,253,254,255,256,257,258,259,260,261,262,263,264,265,266,267,269,272,273,274,275,276,277,278,279,281,282,283,284,285,286,287,288,289,290,291,292,293,296,297,298,299,301,302,303,304,305,306,307,308,310,313,314,315,316,317,318,319,320,321,322,323,324,325,326,327,328,329,330,332,334,335,336,337,338,339,340,341,342,343,344,345,346,348,349,350,351,352,357,359,360,361,362,363,364,365,366,367,368,370,371,372,373,374,376,379,380,381,382,383,384,385,386,387,388,389,390,391,392,393,394,395,396,397,398,399,400,402,403,404,405,406,408,409,411,412,413,414,419,421,422,423,424,425,426,427,428,429,430,431,432,433,434,435,436,437,438,439,440,442,445,446,447,448,449,450,451,452,453,454,455,456,457,458,459,460,461,462,463,464,465,466,467,469,470,471,472,473,475,476,477,478,479,480,482,485,487,488,490,491,492,493,494,495,496,497,498,499,500,501,502,503,504,505,506,508,511,512,513,514,515,516,517,518,519,520,521,522,523,524,525,526,527,528,529,530,532,533,534,535,536,538,539,541,542,543,544,545,546,547,548,549,550,555,556,557,558,559,560,561,562,563,564,565,566,567,569,570,571,572,573,575,578,579,580,581,582,583,584,585,586,587,588,589,590,591,592,593,594,595,596,597,598,599,601,602,603,604,605,606,607,609,610,612,615,617,618,619,620,621,622,623,625,626,627,628,629,630,631,632,633,634,635,636,637,638,639,640,642,645,646,647,648,649,650,651,652,653,654,655,656,657,658,659,660,661,662,663,665,666,668,669,670,671,672,673,674,675,676,677,678,680,681,686,688,689,690,691,692,693,694,695,696,697,698,700,703,704,705,706,707,708,709,710,711,712,713,714,715,716,717,718,719,720,721,722,723,725,726,727,728,729,731,733,734,735,736,737,738,743,745,746,747,748,749,750,752,753,754,755,756,757,758,759,760,762,765,766,767,768,769,770,771,772,773,774,775,776,777,778,779,780,781,782,783,785,786,788,789,790,791,792,793,794,795,796,798,801,803,804,805,806,807,809,810,811,812,813,814,815,816,817,818,819,820,821,822,823,824,826,829,830,831,832,833,834,835,836,837,838,839,840,841,842,843,844,845,846,847,848,849,850,852,853,854,855,856,858,859,860,861,866,868,869,870,871,873,874,875,876,877,878,879,880,881,882) have mixed types. Specify dtype option on import or set low_memory=False.
Writing 17 logs of type Audit.SharePoint to F:\Office365 audit log collector\Logs\365Log_AuditSharePoint.csv
Finished. Total logs retrieved: 338. Total retries: 0. Total logs with errors: 0. Run time: 0:02:50.215925.
FileInterface reports: 19 successfully sent, 0 errors

F:\Office365 audit log collector>pause
Press any key to continue . . .

EDIT2:- Workaround for removing leading Semi-colons on Notepad++
Remove Leading Semi-colons

PermissionError: [Errno 13] Permission denied: 'known_logs'

Hi,

since 2023-07-13 I don't get any logs...
I was running LINUX-OfficeAuditLogCollector-V1.4.2 and updated to V2.1 but the issue persists.
The user has write access to the log file by the way.
This is my config:

log:  # Log settings. Debug will severely decrease performance
  path: '/var/log/office365-audit-log-collector.log'
  debug: True
collect:  # Settings determining which audit logs to collect and how to do it
  contentTypes:
    Audit.General: True
    Audit.AzureActiveDirectory: True
    Audit.Exchange: True
    Audit.SharePoint: True
    DLP.All: True
  maxThreads: 50
  retries: 3  # Times to retry retrieving a content blob if it fails
  retryCooldown: 3  # Seconds to wait before retrying retrieving a content blob
  autoSubscribe: True  # Automatically subscribe to collected content types. Never unsubscribes from anything.
  skipKnownLogs: True  # Remember retrieved log ID's, don't collect them twice
  resume: False  # Remember last run time, resume collecting from there next run
  hoursToCollect: 24 # Look back this many hours for audit logs (can be overwritten by resume)
filter:  # Only logs that match ALL filters for a content type are collected. Leave empty to collect all
  Audit.General:
  Audit.AzureActiveDirectory:
  Audit.Exchange:
  Audit.SharePoint:
  DLP.All:
output:
  graylog:
    enabled: true
    address: 127.0.0.1
    port: 5555

Compared to 1.4.2 the "debug: True" isn't not as chatty anymore.

Any hints appreciated.
Many thanks and best regards, Flo.

Error logging in: "'access_token'"

Hello

When I try to run the collector with the tenant ID, client key, secret key and the fullConfig.yaml I get the error message "Error logging in: "'access_token'". I don't know what could be the problem behind this.

Thanks in advance.

No logs received

Hi,

I'm just starting with this nice solution but I'm stuck and I can't get any logs.

Auditing is enabled in tenant

Get-AdminAuditLogConfig | FL UnifiedAuditLogIngestionEnabled
UnifiedAuditLogIngestionEnabled : True
Starting run @ 2022-04-27 16:26:48.850581. Content: ['Audit.General', 'Audit.AzureActiveDirectory', 'Audit.Exchange', 'Audit.SharePoint', 'DLP.All'].
Retrieving Audit.General. Start time: 2022-04-26T14:22:09. End time: 2022-04-27T14:26:49.
Retrieving Audit.AzureActiveDirectory. Start time: 2022-04-26T14:22:09. End time: 2022-04-27T14:26:49.
Retrieving Audit.Exchange. Start time: 2022-04-26T14:22:09. End time: 2022-04-27T14:26:49.
Retrieving Audit.SharePoint. Start time: 2022-04-26T14:22:09. End time: 2022-04-27T14:26:49.
Retrieving DLP.All. Start time: 2022-04-26T14:22:09. End time: 2022-04-27T14:26:49.
Finished. Total logs retrieved: 0. Total logs with errors: 0. Run time: 0:00:02.546457.
GraylogInterface reports: 0 successfully sent, 0 errors

Strangely the log file "collector.log" is not created when I run the collector:
./LINUX-OfficeAuditLogCollector-V1.3 AAD-Tenant-ID AAD-App-ID AAD-App-SecretKey --config ./fullConfig.yaml

fullConfig.yaml

log:  # Log settings. Debug will severely decrease performance
  path: 'collector.log'
  debug: True
collect:  # Settings determining which audit logs to collect and how to do it
  contentTypes:
    Audit.General: True
    Audit.AzureActiveDirectory: True
    Audit.Exchange: True
    Audit.SharePoint: True
    DLP.All: True
  maxThreads: 50
  retries: 3  # Times to retry retrieving a content blob if it fails
  retryCooldown: 3  # Seconds to wait before retrying retrieving a content blob
  autoSubscribe: True  # Automatically subscribe to collected content types. Never unsubscribes from anything.
  skipKnownLogs: True  # Remember retrieved log ID's, don't collect them twice
  resume: True  # Remember last run time, resume collecting from there next run
  hoursToCollect: 72  # Look back this many hours for audit logs (can be overwritten by resume)
filter:  # Only logs that match ALL filters for a content type are collected. Leave empty to collect all
  Audit.General:
  Audit.AzureActiveDirectory:
  Audit.Exchange:
  Audit.SharePoint:
  DLP.All:
output:
  graylog:
    enabled: true
    address: 127.0.0.1
    port: 5555

OSError: [Errno 99] Cannot assign requested address

Hello,

During initial collection, I had this crash.
Tenant is big, like 11K users.

I guess I can start it again, but I don't know if it means a gap in log collection, related to known_content file ?

Exception in thread Thread-3:
Traceback (most recent call last):
  File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "/home/ubuntu/office365-audit-log-collector-master/GraylogInterface.py", line 36, in monitor_queue
    self._send_message_to_graylog(msg=msg)
  File "/home/ubuntu/office365-audit-log-collector-master/GraylogInterface.py", line 60, in _send_message_to_graylog
    sock = self._connect_to_graylog_input()
  File "/home/ubuntu/office365-audit-log-collector-master/GraylogInterface.py", line 48, in _connect_to_graylog_input
    s.connect((self.gl_address, int(self.gl_port)))
OSError: [Errno 99] Cannot assign requested address

Under debug mode '-d' file AuditLogCollector.log shows 0 content blob

Hello,
First I like to thank you guys for making such useful script/tool.

I have most things setup, with small change that instead of tcp stream to be gray log its logstash which is configured to received json data on given port defined under

./AuditLogCollector.py

The output of AuditLogCollector.log

DEBUG:root:Getting available content for type: "Audit.Exchange"
DEBUG:root:Getting available content for type: "Audit.AzureActiveDirectory"
DEBUG:root:Making API request using URL: "https://manage.office.com/api/v1.0/71a9fb03-b345-4df8-a29a-461c5b94797e/activity/feed/subscriptions/content?contentType=Audit.Exchange&startTime=2019-08-12T20:58:54&endTime=2019-08-12T21:58:54&PublisherIdentifier=0110bb34-3fc5-4d21-b8fa -827437ab7597"
DEBUG:root:Making API request using URL: "https://manage.office.com/api/v1.0/71a9fb03-b345-4df8-a29a-461c5b94797e/activity/feed/subscriptions/content?contentType=Audit.AzureActiveDirectory&startTime=2019-08-12T20:58:54&endTime=2019-08-12T21:58:54&PublisherIdentifier=0110bb34-3fc5-4d21-b8fa -827437ab7597"
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): login.microsoftonline.com:443
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): login.microsoftonline.com:443
DEBUG:urllib3.connectionpool:https://login.microsoftonline.com:443 "POST /71a9fb03-b345-4df8-a29a-461c5b94797e/oauth2/token HTTP/1.1" 200 1427
DEBUG:root:Logged in
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): manage.office.com:443
DEBUG:urllib3.connectionpool:https://login.microsoftonline.com:443 "POST /71a9fb03-b345-4df8-a29a-461c5b94797e/oauth2/token HTTP/1.1" 200 1427
DEBUG:root:Logged in
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): manage.office.com:443
DEBUG:urllib3.connectionpool:https://manage.office.com:443 "GET /api/v1.0/71a9fb03-b345-4df8-a29a-461c5b94797e/activity/feed/subscriptions/content?contentType=Audit.AzureActiveDirectory&startTime=2019-08-12T20:58:54&endTime=2019-08-12T21:58:54&PublisherIdentifier=0110bb04-3fc5-4d21-b8fa%20-827437ab7597 HTTP/1.1" 200 2
DEBUG:root:Got 0 content blobs of type: "Audit.AzureActiveDirectory"
DEBUG:urllib3.connectionpool:https://manage.office.com:443 "GET /api/v1.0/71a9fb03-b345-4df8-a29a-461c5b94797e/activity/feed/subscriptions/content?contentType=Audit.Exchange&startTime=2019-08-12T20:58:54&endTime=2019-08-12T21:58:54&PublisherIdentifier=0110bb34-3fc5-4d21-b8fa%20-827437ab7597 HTTP/1.1" 200 2
DEBUG:root:Got 0 content blobs of type: "Audit.Exchange"

On 0365 i made considerable sensitive changes in order to generate audit logs. See attachments
comp audit
log audit

Also, the time is in UTC, but when retrieved also seen in screenshot its in local-time, should API request be made against local time ?

No data in Graylog

Hi,

It runs as it should, but I can't find any messages under Graylog :

.\WIN-OfficeAuditLogCollector-V2.1.exe $env:tenant_id $env:client_id $env:secret_key --config .\fullConfig.yaml
Starting run @ 2024-01-30 16:51:01.129884. Content: deque(['Audit.General', 'Audit.AzureActiveDirectory', 'Audit.Exchange', 'Audit.SharePoint', 'DLP.All']).
Rust engine finished receiving all content
Finished. Total logs retrieved: 6817. Total retries: 0. Total logs with errors: 0. Run time: 0:00:21.965862.
GraylogInterface reports: 6673 successfully sent, 0 errors
```

![image](https://github.com/ddbnl/office365-audit-log-collector/assets/80531900/2f3f198a-69d6-43f7-b5bb-01f31b2d6c3e)

last_run_times causes process to hang

Running on debain

Whenever last_run_times exists in the directory the command runs from, the program will hang at reading the logs.

Ctrl-C gives the following:

Traceback (most recent call last):
File "AuditLogCollector.py", line 93, in receive_results_from_rust_engine
ValueError: No logs ready

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "AuditLogCollector.py", line 692, in
File "AuditLogCollector.py", line 60, in run
File "AuditLogCollector.py", line 73, in run_once
File "AuditLogCollector.py", line 95, in receive_results_from_rust_engine
File "AuditLogCollector.py", line 54, in force_stop
File "logging/init.py", line 2082, in info
File "logging/init.py", line 1446, in info
File "logging/init.py", line 1565, in _log
File "AuditLogCollector.py", line 55, in force_stop
File "threading.py", line 1006, in join
RuntimeError: cannot join thread before it is started
[26644] Failed to execute script 'AuditLogCollector' due to unhandled exception!

error in latest version

Hello,

I'm trying the latest version but I got this error

thread 'main' panicked at src\api_connection.rs:59:33:
Could not parse API login reply: error decoding response body: missing field `access_token` at line 1 column 623
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

the request is:

./OfficeAuditLogCollector.exe --tenant-id "xxxxxxx" --client-id "xxxxx" --secret-key "xxxxx" --config config.yaml

config file:

collect:
  skipKnownLogs: True
  workingDir: ./
  maxThreads: 50
  globalTimeout: 5
  retries: 3
  hoursToCollect: 168
  contentTypes:
    Audit.General: True
    Audit.AzureActiveDirectory: True
    Audit.Exchange: True
    Audit.SharePoint: True
    DLP.All: True 
output:
  file:
    path: 'output.csv'
    separateByContentType: True
    separator: ';'

I'm using the client on window system

Improvement: Additional parameter for the work files

The collector needs four files in my configuration:

collector.log
config.yaml
known_content
known_logs

With the parameter path I can determine where the logs are stored, but not where the working files are placed.

log:
  path: "collector.log"

However, this path only applies to the log files. I would find it helpful if there was also a parameter to set the location of the working files. But that would mean a small redsign of the YAML structure. For instance:

path:
  log: "collector.log"
  workingdir: "/usr/local/o365collector"

Error: Error logging in: "'access_token'"

Hi. Im getting this error when lauching .exe : Error logging in: "'access_token'"

I check so many times tenant id, client id an key, and all its ok. I dont know what I should check.

Thanks

Errors on Debian 10

Using the Linux executable I get:
Error loading Python lib '/tmp/_MEIfpmUZU/libpython3.8.so.1.0': dlopen: /lib/x86_64-linux-gnu/libm.so.6: version GLIBC_2.29 not found (required by /tmp/_MEIfpmUZU/libpython3.8.so.1.0)

Updated Debian and ldd --version = ldd (Debian GLIBC 2.28-10) 2.28

Tried running from the source python files but I get hung up on pip installing the requirements.txt

ERROR: Could not find a version that satisfies the requirement kivy-deps.angle==0.3.2 (from versions: none)
ERROR: No matching distribution found for kivy-deps.angle==0.3.2

http 400 error

Hi, i was wondering if you may be able to point me in the right direction, we have set up our Audit logs, and we have set up the app integration and granted it permission to the Office365 Management API, created a secret etc, and it seems to authenticate OK to login.microsoftonline.com:443, but then the next step seems to throw a http 400, from what i can seee we have set this up correctly. can anyone advise why we might be getting a 400? doesnt seem to matter which log type we pull:

(i've remved the actual values for our tenant on the command bellow:

python AuditLogCollector.py '7xxx' 'xxx 'xxx' --azure_ad -p 'TEST365LOGS' -g -gA 10.50.2.128 -gP 5566 -d -l /tmp/logs/debug.log

results from debug.log:

cat /tmp/logs/debug.log

INFO:root:Starting run @ 2019-08-21 07:15:20.942284
DEBUG:root:Getting available content for type: "Audit.AzureActiveDirectory"
DEBUG:root:Making API request using URL: "https://manage.office.com/api/v1.0/7xxx/activity/feed/subscriptions/content?contentType=Audit.AzureActiveDirectory&startTime=2019-08-20T20:15:20&endTime=2019-08-20T21:15:20&PublisherIdentifier=TEST365LOGS"
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): login.microsoftonline.com:443
DEBUG:urllib3.connectionpool:https://login.microsoftonline.com:443 "POST /7xxx/oauth2/token HTTP/1.1" 200 1492
DEBUG:root:Logged in
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): manage.office.com:443
DEBUG:urllib3.connectionpool:https://manage.office.com:443 "GET /api/v1.0/xxx/activity/feed/subscriptions/content?contentType=Audit.AzureActiveDirectory&startTime=2019-08-20T20:15:20&endTime=2019-08-20T21:15:20&PublisherIdentifier=TEST365LOGS HTTP/1.1" 400 71
DEBUG:root:Got 1 content blobs of type: "Audit.AzureActiveDirectory"

any assistance would be great, thanks for your help

ERROR:root:Error logging in: "'access_token'"

Hello,
I ran without issue AuditLogSubscriber.py.

Now when starting AuditLogCollector.py, it keeps running, taking 100% of 1 core, but no entries appears in Graylog.
Going to debug mode, I have this log with this error:
ERROR:root:Error logging in: "'access_token'"

INFO:root:Starting run @ 2019-03-22 01:41:24.883658
DEBUG:root:Getting available content for type: "Audit.General"
DEBUG:root:Making API request using URL: "https://manage.office.com/api/v1.0/XXXXXXXX/activity/feed/subscriptions/content?contentType=Audit.General&startTime=2019-03-22T00:41:24&endTime=2019-03-22T01:41:24&PublisherIdentifier=XXXXXXXXXXXX"
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): login.microsoftonline.com
DEBUG:root:Getting available content for type: "Audit.Exchange"
DEBUG:root:Making API request using URL: "https://manage.office.com/api/v1.0/XXXXXXXXXX/activity/feed/subscriptions/content?contentTyp
e=Audit.Exchange&startTime=2019-03-22T00:41:24&endTime=2019-03-22T01:41:24&PublisherIdentifier=XXXXXXXXX"
DEBUG:root:Getting available content for type: "Audit.Sharepoint"
DEBUG:root:Making API request using URL: "https://manage.office.com/api/v1.0/XXXXXXXXXXX/activity/feed/subscriptions/content?contentTyp
e=Audit.Sharepoint&startTime=2019-03-22T00:41:25&endTime=2019-03-22T01:41:25&PublisherIdentifier=XXXXXXXXX"
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): login.microsoftonline.com
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): login.microsoftonline.com
DEBUG:root:Getting available content for type: "Audit.AzureActiveDirectory"
DEBUG:root:Making API request using URL: "https://manage.office.com/api/v1.0/cxxxxxxxxxxxxxx/activity/feed/subscriptions/content?contentType=Audit.AzureActiveDirectory&startTime=2019-03-22T00:41:25&endTime=2019-03-22T01:41:25&PublisherIdentifier=XXXXXXXXX"
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): login.microsoftonline.com
DEBUG:root:Getting available content for type: "DLP.All"
DEBUG:root:Making API request using URL: "https://manage.office.com/api/v1.0/XXXXXXXX/activity/feed/subscriptions/content?contentType=DLP.All&startTime=2019-03-22T00:41:25&endTime=2019-03-22T01:41:25&PublisherIdentifier=XXXXXXXXX"
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): login.microsoftonline.com
DEBUG:urllib3.connectionpool:https://login.microsoftonline.com:443 "POST /XXXXXXXXX/oauth2/token HTTP/1.1" 400 779
ERROR:root:Error logging in: "'access_token'"
DEBUG:urllib3.connectionpool:https://login.microsoftonline.com:443 "POST /XXXXXXXXXXXXXX/oauth2/token HTTP/1.1" 400 779
ERROR:root:Error logging in: "'access_token'"
DEBUG:urllib3.connectionpool:https://login.microsoftonline.com:443 "POST /XXXXXXXXXX/oauth2/token HTTP/1.1" 400 779
ERROR:root:Error logging in: "'access_token'"
DEBUG:urllib3.connectionpool:https://login.microsoftonline.com:443 "POST /XXXXXXXXXXXXX/oauth2/token HTTP/1.1" 400 779
DEBUG:urllib3.connectionpool:https://login.microsoftonline.com:443 "POST /XXXXXXXXXXXXXXXXX/oauth2/token HTTP/1.1" 400 779
ERROR:root:Error logging in: "'access_token'"
ERROR:root:Error logging in: "'access_token'"

Can you help me on this ?

Thanks in advance and thanks for what looks like a great plugin :)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.