View the GitHub project here or download the latest release here.

• Overview
• Getting Started
  • Setup
  • Configuration
    • AddressBookSqlConnectionSettings.json
    • EnterpriseVaultConnectionSettings.json
    • DefaultProcessingSettings.json
    • DefaultWorkerSettings.json
    • DefaultMimeTypeSettings.csv
• Address Book Database
  • Tables
    • UserRecord
    • UserAddress
    • UserPhoneNumber
    • UserSID
    • IngestionHistory
• Address Book Active Directory Sync Console
  • Sync Logic
  • Settings File
    • AddressBookDatabaseConnectionSettings
    • SyncSettings
    • ActiveDirectoryServers
  • Arguments
    • Examples
  • Return Codes
• License

Written By: Jason Wells

This script provides an interface to assist with locating and processing EV stores and archives in Nuix. The script supports filtering by a custodian which is selected from a user record database (the address book) which is stored in SQL.

Additionally this repository contains:

• A SQL script to build the requisite tables in a database
• A tool to sync data from Active Directory or a CSV into the address book

Begin by downloading the latest release of this code. Extract the contents of the archive into your Nuix scripts directory. In Windows the script directory is likely going to be either of the following:

• %appdata%\Nuix\Scripts - User level script directory
• %programdata%\Nuix\Scripts - System level script directory

Before you can make use of this script you will need to perform some configuration. Within the settings sub directory, there are several JSON files which provide configuration settings to the script.

This file allows you to provide database connection settings which will allow the script to interact with the address book database.

Example

{
	"server": "localhost",
	"port": null,
	"database": "UserData",
	"instance": null,
	"domain": null,
	"user": null,
	"password": null
}

Note: Providing a value of null for a setting instructs the script to ignore that setting.

Note: If this file cannot be found or the database cannot be reached using the settings provided, the script will still run with limited functionality. The script will start with a warning and the address book functionality will be unavailable, but the script should otherwise be functional.

The combination of settings needed depends on your MSSQL environment and how it is setup.

This file allows you to provide SQL connection settings to one or more Enterprise Vault SQL backend servers. The script connects to these servers to query information about available vault stores and associated vault archives.

Example - Single Server

[
	{
		"name": "Server 1",
		"sql_connection_settings": {
			"server": "192.168.1.10",
			"port": null,
			"database": "EnterpriseVaultDirectory",
			"instance": "",
			"domain": null,
			"user": null,
			"password": null
		}
	}
]

Example - Multiple Servers

[
	{
		"name": "Server 1",
		"sql_connection_settings": {
			"server": "192.168.1.10",
			"port": null,
			"database": "EnterpriseVaultDirectory",
			"instance": "",
			"domain": null,
			"user": null,
			"password": null
		}
	},
	{
		"name": "Server 2",
		"sql_connection_settings": {
			"server": "192.168.1.20",
			"port": null,
			"database": "EnterpriseVaultDirectory",
			"instance": "SQLEXPRESS",
			"domain": null,
			"user": null,
			"password": null
		}
	},
	{
		"name": "Server 3",
		"sql_connection_settings": {
			"server": "192.168.1.30",
			"port": null,
			"database": "EnterpriseVaultDirectory",
			"instance": "",
			"domain": null,
			"user": "sa",
			"password": "sapassword"
		}
	}
]

Note: Providing a value of null for a setting instructs the script to ignore that setting.

The combination of settings needed depends on your MSSQL environment and how it is setup.

This file defines the default processing settings used by the "Data Processing Settings" tab. If this file does not exist the script will use its own internal default settings (seen below). See Processor.setProcessingSettings for details on what settings are available and what they do.

Example

{
  "addBccToEmailDigests": false,
  "addCommunicationDateToEmailDigests": false,
  "analysisLanguage": "en",
  "calculateAuditedSize": true,
  "calculateSSDeepFuzzyHash": false,
  "carveFileSystemUnallocatedSpace": false,
  "createThumbnails": false,
  "detectFaces": false,
  "digests": [
    "MD5"
  ],
  "enableExactQueries": false,
  "extractEndOfFileSlackSpace": false,
  "extractFromSlackSpace": false,
  "extractNamedEntitiesFromProperties": false,
  "extractNamedEntitiesFromText": false,
  "extractNamedEntitiesFromTextStripped": false,
  "extractShingles": false,
  "hideEmbeddedImmaterialData": false,
  "identifyPhysicalFiles": true,
  "maxDigestSize": 250000000,
  "maxStoredBinarySize": 1000000000,
  "processFamilyFields": false,
  "processText": true,
  "processTextSummaries": false,
  "recoverDeletedFiles": false,
  "reportProcessingStatus": "none",
  "reuseEvidenceStores": true,
  "skinToneAnalysis": false,
  "smartProcessRegistry": false,
  "stemming": false,
  "stopWords": false,
  "storeBinary": false,
  "traversalScope": "full_traversal"
}

This file defines the default settings used by the "Worker Settings" tab. If this file does not exist the script will use its own internal default settings (seen below). See Processor.setParallelProcessingSettings for details on what these settings do. Note that currently only workerCount, workerMemory and workerTemp are supported by the script.

Example

{
	"workerCount": 4,
	"workerMemory": 2048,
	"workerTemp": "C:\\WorkerTemp",
	"workerTimeout": 3600
}

This CSV file configures how mime type settings will be configured through the API before processing begins.

Note: If a mime type value is encountered that is not a known type in the version of Nuix you are using to run the script, the settings will be skipped for that particular type to prevent errors. A message will also be logged noting this.

The address book information is stored in a series of tables which hold information about users. See the included file CreateDatabaseTables.sql for details.

Used to store the main details of a user.

Used to store the email addresses associate to a user.

Used to store the phone numbers associate to a user.

Used to store the SIDs associate to a user.

When ingestion is performed with a custodian selected, this table records to which case and when that ingestion was performed.

This is a headless .Net Framework 4.5 executable (no GUI) which performs the sync operation between a data source (usually Active Directory) and the address book database from which the script pulls the custodian information.

For each record in input source:

1. If input record has an empty or only whitespace character value for EmployeeID the record is skipped. Note that alternate source attributes can be used for this value by configuring the setting UniqueIdentifierAttribute in the SyncSettings section of the file Settings.json. Since this is the only way to uniquely identify each entry this is a required field that must have a uniquely identifying value.
2. If the database does not already contain a record with this EmployeeID, a new record is added. If the database does already contain a record with this EmployeeID:
   1. If settings specify to update user base data:
      1. If Name is different from value in database, update it
      2. If Title is different from value in database, update it
      3. If Department is different from value in database, update it
      4. If Location is different from value in database, update it
   2. Update database with any email addresses not already present for user (values are added, but never modified or removed)
   3. Update database with any phone numbers not already present for user (values are added, but never modified or removed)
   4. Update database with any SID values which are not already present for user record (values are added, but never modified or removed)

By default the tool will look for the file Settings.json located in the same directory as the executable. This is a JSON file containing information regarding:

• SQL Server connection settings for the database server hosting the address book database.
• Sync settings controlling some aspects regarding how data is collected from Active Directory.
• The directory log files are written to.
• Settings for each Active Directory server to collect user information from.

Note: See the arguments section below for details on specifying where the settings file is located.

{
	"AddressBookDatabaseConnectionSettings":{
		"Server": "localhost",
		"Port": null,
		"Instance": "SQLEXPRESS",
		"Database": "UserData",
		"User": null,
		"Password": null,
		"Domain": null,
	},
	"SyncSettings":{
		"EmailAddressesIncludesMailNicknameProperty": true,
		"EmailAddressesIncludesMailProperty": true,
		"EmailAddressesIncludesProxyAddressesProperty": true,
		"CollectX400ProxyAddresses": true,
		"CollectX500ProxyAddresses": true,
		"CollectSmtpProxyAddresses": true,
		"CollectSipProxyAddresses": true,
		"UpdateUserInformation": true,
		"UniqueIdentifierAttribute":  "employeeId" // Valid values: employeeId, employeeNumber
	},
	"LogDirectory": "C:\\Temp\\ADSyncLogs\\",
	"ActiveDirectoryServers":[
		{
			"Name": "Server 1",
			"Domain": null,
			"ContextContainer": null
		},
		{
			"Name": "Server 2",
			"Domain": null,
			"ContextContainer": null
		}
	]
}

This file is comprised of several sections:

• AddressBookDatabaseConnectionSettings
• SyncSettings
• ActiveDirectoryServers

Contains information about how to connect to the SQL server instance hosting the address book database.

SQL Server Authentication

Authentication to the SQL server hosting the address book database will be performed in different ways, depending on the settings provided.

1. Trusted connection - If no values are provided for: User, Password or Domain then a trusted connection will be used to connect to the SQL server. The credentials of user running the tool will be used to authenticate against the SQL server.
2. SQL Server Authentication - If value are provided for User and Password, then those credentials will be used to authenticate against a login setup within the SQL server instance.
3. Domain Authentication - If values are provided for User, Password and Domain then those credentials will be used to authenticate with the specified Domain.

This section allows you to specify 1 or more Active Directory servers to collect sync data from. Each server entry takes the following form:

{
	"Name": "Server 1",
	"Domain": null,
	"ContextContainer": null,
	"UserName": null,
	"Password": null
}

The tool also supports several option arguments to modify how it behaves.

  -t, --templatefile    (Default: None) When specified the tool will save a
                        blank template CSV (suitable for import) to the
                        location specified.

  -i, --inputfile       (Default: None) Used to specify a CSV file to import.
                        When this option is provided, import will not occur
                        from active directory.

  -e, --exportfile      (Default: None) Used to specify an file to export to.
                        When this option is provided, entries will be read from
                        Active Directory but written to the output file rather
                        than the address book database.  Supports extensions:
                        CSV, TXT

  -l, --limit           (Default: 0) Used to specify a maximum number of
                        records to process.  A value of 0 means no limit.

  -s, --settingsfile    (Default: None) Used to specify an alternate settings
                        JSON file.  If not provided default is to look for
                        Settings.json in same directory as executable.

  -n, --servername      (Default: None) When provided, only the Active
                        Directory server in the settings file with the provided
                        name will be processed.

  --help                Display this help screen.

Specify the settings file

ActiveDirectorySyncConsole.exe --settings "C:\SomeDirectory\AlternativeSettings.json"

Export to a CSV

ActiveDirectorySyncConsole.exe --exportfile "C:\SomeDirectory\ADUserSample.csv"

Export 100 records to a CSV

ActiveDirectorySyncConsole.exe --exportfile "C:\SomeDirectory\ADUserSample.csv" --limit 100

Import from a CSV

ActiveDirectorySyncConsole.exe --importfile "C:\SomeDirectory\ADUserSample.csv"

Generate Template CSV

ActiveDirectorySyncConsole.exe --templatefile "C:\SomeDirectory\WillImportLater.csv"

Process only specific AD server

ActiveDirectorySyncConsole.exe --servername "Server 1"

Process only 100 records from specific AD server

ActiveDirectorySyncConsole.exe --servername "Server 1" --limit 100

The tool returns a few different codes depending on errors which may have occurred.

Copyright 2018 Nuix

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.