Code Monkey home page Code Monkey logo

data-import's Introduction

This repository contains samples of ways to get data into QRadar, mostly utilizing public REST APIs.

These samples are provided for reference purposes on an "as is" basis, and are without warranties of any kind.

Any issues discovered using the samples should not be directed to QRadar support, but be reported on the Github issues tracker.

aql-to-reference-data

An example python script that allows you to execute an AQL query and use the result to populate a reference set or reference table

stix-import

An example python script that allows you to import STIX indicators either from an XML file or from a TAXII server

assets

An example python script that allows you to import data from a CSV file into the asset model

excel

Excel spreadsheet to retrieve and update reference data (tables and maps of sets) directly from an excel spreadsheet

rfisi-threat-import

Sample code demonstrating the creation and feeding of reference collections defined by the Ready For IBM Security Intelligence partner program

tor-nodes-import

A sample of extracting TOR relay node IPs from the TOR Bundle and importing those as reference data for QRadar

data-import's People

Contributors

chrismeenan avatar filipschepers avatar jasonkeirstead avatar manahan avatar rory-bray avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

data-import's Issues

Asset import crashing qradar

Using update_assets.py, I crashed qradar's tomcat server after updating 2000 assets. It seems that with each POST request, a new thread is created and never released. After ~ 2000 assets, tomcat runs out of memory and the web ui crashes.

assets: sample for creating new assets

Hi!

i am currently looking for a way to create new assets via the Qradar API. Could you provide a sample of how this is done with the latest version of qradar (7.2.5) ?
The samples around the internet only show how you could upload a single huge csv file in the web interface but this is hardly a state-of-the-art way to work with big data :)

Regards

Error executing stix_import.py

After downloading and install required dependencies received the following error:

Traceback (most recent call last):
File "stix_import.py", line 293, in
main()
File "stix_import.py", line 185, in main
vocabs._VOCAB_MAP["stixVocabs:IndicatorTypeVocab-1.0"] = IndicatorType
AttributeError: 'module' object has no attribute '_VOCAB_MAP'

Currently running python v2.7.10_2 on OSX v10.10.5 with the following dependencies:
stix (1.2.0.1.dev2)
pytz (2015.4)
libtaxii (1.1.107)

urllib2 error

I was receiving the following error:

{"message":"Version (0.1) from header parameter (Version) has been removed and is no longer valid. Please refer to documentation for list of valid versions.","details":{},"description":"","code":41,"http_response":{"message":"The request was well-formed but was unable to be followed due to semantic errors","code":422}}
Traceback (most recent call last):
File "import.py", line 296, in
main()
File "import.py", line 269, in main
indicators += process_package_dict( args, stix_package.to_dict() )
File "import.py", line 166, in process_package_dict
response = urllib2.urlopen(req)
File "/usr/lib/python2.7/urllib2.py", line 127, in urlopen
return _opener.open(url, data, timeout)
File "/usr/lib/python2.7/urllib2.py", line 407, in open
response = meth(req, response)
File "/usr/lib/python2.7/urllib2.py", line 520, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python2.7/urllib2.py", line 445, in error
return self._call_chain(_args)
File "/usr/lib/python2.7/urllib2.py", line 379, in _call_chain
result = func(_args)
File "/usr/lib/python2.7/urllib2.py", line 528, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 422: Unprocessable Entity

Removing 'version':'0.1', from line 153 in the headers variable fixed it for me.

Run Time Error 94

The reference sets which are empty is causing the run time error '94' , Invalid use of Null.
The issue is happening for both update and retrieve options.

Tried to debug but unable to find the cause and the debug options hits the below block of code.

If Response.StatusCode <> Ok Then
    If IsEmpty(Response) Then
          MsgBox Response.StatusDescription
    Else
          MsgBox Response.Data("message")
    End If
          Exit Sub
End If 

Mac address update

I have list of ip address and their mac addresses looks like there is no properties for mac address. Is there work around for updating mac address on qradar

The use of this field has been deprecated. Received 'datetime' object

Getting the error while trying to retrieve data from hailataxii.com with the following stix_import command.

Command:
./stix_import.py -i x.x.x.x -t 11111111-22222-33333-444444-555555555 -x 'hailataxii.com' --taxii_endpoint '/taxii-discovery-service' -c guest.Abuse_ch -r Taxii_IP -y AddressObjectType --taxii_start_time '2016-07-01 12:00:00' --taxii_end_time '2016-08-18 12:00:00'

Error:
/usr/lib/python2.7/site-packages/stix/utils/deprecated.py:48: UserWarning: The use of this field has been deprecated. Received 'datetime' object.
warnings.warn(msg)
Error posting data ["202.9.36.111"]

{"message":"Version (0.1) from header parameter (Version) has been removed and is no longer valid. Please refer to documentation for list of valid versions.","details":{},"description":"","code":41,"http_response":{"message":"The request was well-formed but was unable to be followed due to semantic errors","code":422}}
Traceback (most recent call last):
File "./stix_import.py", line 293, in
main()
File "./stix_import.py", line 266, in main
indicators += process_package_dict( args, stix_package.to_dict() )
File "./stix_import.py", line 163, in process_package_dict
response = urllib2.urlopen(req)
File "/usr/lib64/python2.7/urllib2.py", line 126, in urlopen
return _opener.open(url, data, timeout)
File "/usr/lib64/python2.7/urllib2.py", line 406, in open
response = meth(req, response)
File "/usr/lib64/python2.7/urllib2.py", line 519, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib64/python2.7/urllib2.py", line 444, in error
return self._call_chain(_args)
File "/usr/lib64/python2.7/urllib2.py", line 378, in _call_chain
result = func(_args)
File "/usr/lib64/python2.7/urllib2.py", line 527, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 422: Unprocessable Entity

What is taxii_endpoint for stix-import?

I have a taxii server which have services discovery, collection-management, poll and inbox.
Now what is endpoint service is it discovery or collection or poll ?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.