ericzimmerman / sqlecmd Goto Github PK

View Code? Open in Web Editor NEW

41.0 19.0 19.0 2.84 MB

License: MIT License

C# 100.00%

sqlecmd's Introduction

SQLECmd

This repo that contains all the Maps used by Eric Zimmerman's SQLECmd.

Ongoing Projects

SQLECmd Map Ideas - Development roadmap for SQLECmd Maps. Please feel free to contribute by adding ideas or by finishing tasks in the To Do column. Any help is appreciated!

Command Line Interface

SQLECmd version 0.5.0.0

Author: Eric Zimmerman ([email protected])
https://github.com/EricZimmerman/SQLECmd

        d               Directory to process that contains SQLite files. This or -f is required
        f               File to process. This or -d is required

        csv             Directory to save CSV formatted results to.
        json            Directory to save JSON formatted results to.

        dedupe          Deduplicate -f or -d files based on SHA-1. First file found wins. Default is TRUE
        hunt            When true, all files are looked at regardless of name and file header is used to identify SQLite files, else filename in map is used to find databases. Default is FALSE

        maps            The path where event maps are located. Defaults to 'Maps' folder where program was executed

        sync            If true, the latest maps from https://github.com/EricZimmerman/SQLECmd/tree/master/SQLMap/Maps are downloaded and local maps updated. Default is FALSE

        debug           Show debug information during processing
        trace           Show trace information during processing


Examples: SQLECmd.exe -f "C:\Temp\someFile.db" --csv "c:\temp\out"
          SQLECmd.exe -d "C:\Temp\" --csv "c:\temp\out"
          SQLECmd.exe -d "C:\Temp\" --hunt --csv "c:\temp\out"

          Short options (single letter) are prefixed with a single dash. Long commands are prefixed with two dashes

Documentation

SQLECmd parses any SQLite database from any OS. As long as a Map exists for the database, SQLECmd will parse it! If there's a Map that's missing, please create an issue or submit your own via a Pull Request.

Download Eric Zimmerman's Tools

All of Eric Zimmerman's tools can be downloaded here. Use the Get-ZimmermanTools PowerShell script to automate the download and updating of the EZ Tools suite. Additionally, you can automate each of these tools using KAPE!

Special Thanks

Open Source Development funding and support provided by the following contributors:

SANS Institute and SANS DFIR.
Tines

sqlecmd's People

Contributors

Stargazers

Watchers

Forkers

cpapaioannou hmahalik f0r3ns1cat0r andrewrathbun hyuunnn stark4n6 anelshaer chadtilbury hartl3y94 cluelessatcoding dfirdetective wwwgeneral ronrader randomaccess3 hamiltonrg chris-p-bakin rohan836 matthieugras jfdubya

sqlecmd's Issues

`--json` switch doesn't appear to work

** SQLECmd version # **

most current

Describe the bug

When running .\SQLECmd.exe -d "D:\DFIRArtifactMuseum\Windows\WindowsTimeline" --json "D:\DFIRArtifactMuseum\Windows\WindowsTimeline"

the following message appears:

--csv is required. Exiting

For context:

SQLECmd/SQLECmd/Program.cs

Lines 79 to 81 in 518d038

    
           new Option<string>( 
        
               "--json", 
        
               "Directory to save JSON formatted results to"),

A brief review of Program.cs shows there's no similar logic that we see here for --csv:

SQLECmd/SQLECmd/Program.cs

Lines 182 to 192 in 518d038

    
           if (string.IsNullOrEmpty(csv)) 
        
           { 
        
               var helpBld = new HelpBuilder(LocalizationResources.Instance, Console.WindowWidth); 
        
               var hc = new HelpContext(helpBld, _rootCommand, Console.Out); 
        
               helpBld.Write(hc); 
        
               Log.Warning("--csv is required. Exiting"); 
        
               Console.WriteLine(); 
        
               return; 
        
           }

Expected behavior

JSON output, although CSV is perfectly fine. If there's no plans to fix this, I can just PR to remove all references to JSON.

Extract Microsoft Edge Screenshots from History file

SQLECmd version
1.0.0.0

Describe the solution you'd like
I already implemented smap file (provided below), but I would like to see if its possible to add image extraction of the data column. The column is stored as BLOB in the table edge_visits. For now I added a simple hint to the output so users can find these images themselves.

Description: Chromium Browser History Screenshots
Author: Michal Minar
Email: [email protected]
Id: 2ff9fe76-b191-422b-abf7-976e8e540326
Version: 1.0
CSVPrefix: ChromiumBrowser
FileName: History
IdentifyQuery: SELECT count(*) FROM sqlite_master WHERE type='table' AND (name='edge_visits');
IdentifyValue: 1
Queries:
    -
        Name: Chromium Browser History Screenshots
        Query: |
                SELECT 
                datetime( visit_time / 1000000 + ( strftime( '%s', '1601-01-01' ) ), 'unixepoch', 'localtime' ) as VisitTime, 
                u.url as URL, 
                u.title as Title,
                'See the image yourself under History (file)/edge_visits (table)/data (column)' as Hint
                FROM edge_visits ev 
                JOIN visits v 
                on v.id = ev.visit_id 
                JOIN urls u 
                on u.id=v.url 
                WHERE ev.data NOT NULL
                ORDER BY visit_time ASC;
        BaseFileName: ScreenshotsList

# Documentation
# https://medium.com/@DCSO_CyTec/microsoft-edge-forensics-screenshot-history-703b9b8392f8
# So far I found only a few (one to be precise) articles regarding screenshots saved into History file when browsing the Web using Microsoft Edge.

Additional context
MS Edge added a new settings option to their browser, which saves screenshots when browsing the web. So far I found only one article and I was able to extract screenshots on my local machine using the steps provided that article.

ID to Link SQLECmd Map and KAPE Target File

Is your feature request related to a problem? Please describe.
Currently, we have to rely on comments to link a SQLECmd map file to a KAPE target file. We do this because we dynamically generate our own language-specific targets from the target files and want to associate the query defined within the SQL map file to the target.

Describe the solution you'd like
It would be great to either share the map/target file's ID in each's definition, or generate a common ID to use for linking the two files.

Describe alternatives you've considered
Keying off of comments. However, this is a bit of a hack, and does not always work as expected.

No Map for NIST SQLite DB

** SQLECmd version # **
The version of SQLECmd you are running
SQLECmd version 1.0.0.0

Describe the bug
A clear and concise description of what the bug is, including the full command line you are using.
Missing map when processing the NIST NSRLRDS_2024.03.1_modern_minimal.db file downloaded from: https://s3.amazonaws.com/rds.nsrl.nist.gov/RDS/rds_3.00_demo/RDSv3_Modern_Minimal_Demo.zip.

Thank you for your continued work with the programs you maintain!

To Reproduce
Steps to reproduce the behavior:

Form Admin terminal, run .\SQLECmd.exe -f "E:\RDS_2024.03.1_modern_minimal\RDS_2024.03.1_modern_minimal.db" --csv "E:"
Results yielded no csv output, but did create a missing map message.
See error

Expected behavior
Process the db and export a csv.

Screenshots

Additional context
When processing the NIST NSRLRDS_2024.03.1_modern_minimal.db file, I received a message that there was a missing map. At least one database was found with no corresponding map (Use --debug for more details about discovery process) File name: E:\RDS_2024.03.1_modern_minimal\RDS_2024.03.1_modern_minimal.db, Tables: FILE,MFG,OS,PKG,VERSION

SQLECmd.dll Unable to Load SQLite.interop.dll on Linux Ubuntu

** SQLECmd version # **
SQLECmd.dll 1.0.0.0

Describe the bug
Basically the DLL not runs correctly on Ubuntu or Debian Linux System Debian 4.9.303-1 (2022-03-07) x86_64 GNU/Linux

Unhandled exception: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> System.DllNotFoundException: Unable to load shared library 'SQLite.Interop.dll' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libSQLite.Interop.dll: cannot open shared object file: No such file or directory
   at System.Data.SQLite.UnsafeNativeMethods.sqlite3_config_none(SQLiteConfigOpsEnum op)
   at System.Data.SQLite.SQLite3.StaticIsInitialized()
   at System.Data.SQLite.SQLiteLog.PrivateInitialize(String className)
   at System.Data.SQLite.SQLiteLog.Initialize(String className)
   at System.Data.SQLite.SQLiteConnection..ctor(String connectionString, Boolean parseViaFramework)
   at System.Data.SQLite.SQLiteConnection..ctor(String connectionString)
   at ServiceStack.OrmLite.Sqlite.SqliteOrmLiteDialectProvider.CreateConnection(String connectionString)
   at ServiceStack.OrmLite.Sqlite.SqliteOrmLiteDialectProviderBase.CreateConnection(String connectionString, Dictionary`2 options)
   at ServiceStack.OrmLite.OrmLiteConfig.ToDbConnection(String dbConnectionStringOrFilePath, IOrmLiteDialectProvider dialectProvider)
   at ServiceStack.OrmLite.OrmLiteConnection.get_DbConnection()
   at ServiceStack.OrmLite.OrmLiteConnection.Open()
   at SQLECmd.Program.ProcessFile(String fileName, Boolean hunt, Boolean dedupe, String csv)
   at SQLECmd.Program.DoWork(String f, String d, String csv, String json, Boolean dedupe, Boolean hunt, String maps, Boolean sync, Boolean debug, Boolean trace)
   --- End of inner exception stack trace ---
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Span`1& arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at System.Delegate.DynamicInvokeImpl(Object[] args)
   at System.Delegate.DynamicInvoke(Object[] args)
   at System.CommandLine.NamingConventionBinder.ModelBindingCommandHandler.InvokeAsync(InvocationContext context) in /_/src/System.CommandLine.NamingConventionBinder/ModelBindingCommandHandler.cs:line 87
   at System.CommandLine.Invocation.InvocationPipeline.<>c__DisplayClass4_0.<<BuildInvocationChain>b__0>d.MoveNext() in /_/src/System.CommandLine/Invocation/InvocationPipeline.cs:line 58
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass22_0.<<UseParseErrorReporting>b__0>d.MoveNext() in /_/src/System.CommandLine/Builder/CommandLineBuilderExtensions.cs:line 611
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass15_0.<<UseHelp>b__0>d.MoveNext() in /_/src/System.CommandLine/Builder/CommandLineBuilderExtensions.cs:line 473
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass26_0.<<UseVersionOption>b__0>d.MoveNext() in /_/src/System.CommandLine/Builder/CommandLineBuilderExtensions.cs:line 725
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass24_0.<<UseTypoCorrections>b__0>d.MoveNext() in /_/src/System.CommandLine/Builder/CommandLineBuilderExtensions.cs:line 672
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<<UseSuggestDirective>b__23_0>d.MoveNext() in /_/src/System.CommandLine/Builder/CommandLineBuilderExtensions.cs:line 646
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass21_0.<<UseParseDirective>b__0>d.MoveNext() in /_/src/System.CommandLine/Builder/CommandLineBuilderExtensions.cs:line 586
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<<UseDebugDirective>b__8_0>d.MoveNext() in /_/src/System.CommandLine/Builder/CommandLineBuilderExtensions.cs:line 285
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<<RegisterWithDotnetSuggest>b__7_0>d.MoveNext() in /_/src/System.CommandLine/Builder/CommandLineBuilderExtensions.cs:line 236
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass11_0.<<UseExceptionHandler>b__0>d.MoveNext() in /_/src/System.CommandLine/Builder/CommandLineBuilderExtensions.cs:line 377

Processing module option reset

In KAPE is there a way to “reset” module processing options without having to close and reopen the app? For example if you were setting a date range or computer name can that info be edited or removed from the options box without restarting the app?

Adding OG File Name Variable for Output CSV

** SQLECmd version **
1.0.0.0

Is your feature request related to a problem? Please describe.
Processing multiple similar formatted files that use the same Map cause indistinguishable resulting CSVs.

Describe the solution you'd like
When running with seek across Maps, would be great to have a potential variable to the "BaseFileName" field that would add the original file name that is processed to the final CSV filename.

Describe alternatives you've considered
N/A

Additional context
See screenshot below, the original DB files on the right and the resulting files on the left. I have to manually open each CSV to figure out which correlates to which original DB.

	new Option<string>(
	"--json",
	"Directory to save JSON formatted results to"),

	if (string.IsNullOrEmpty(csv))
	{
	var helpBld = new HelpBuilder(LocalizationResources.Instance, Console.WindowWidth);
	var hc = new HelpContext(helpBld, _rootCommand, Console.Out);

	helpBld.Write(hc);

	Log.Warning("--csv is required. Exiting");
	Console.WriteLine();
	return;
	}