anamico / node-red-contrib-proofpoint Goto Github PK

Should look to see if we can spit out reputations per flagged url

Could have a bank of checkboxes to allow the user to "turn on" or "turn off" any of the 4 Proofpoint SIEM response sections.

Should also use that to adjust the query to reduce data where possible.

Currently, the request to proofpoint is returned as a blob and then the next stage parses over the entire parsed json response structure streaming out reputations as they are detected.

This has a level (albeit small) of latency (while the response is streamed to the response buffer - plus, depending on how request works, may also then parse the response buffer) and a memory overhead as the entire parsed json structure resides in memory to be passed from the request response handler to the reputation searching component.

An alternate approach is to stream the response from the request directly to a json lexical streaming parser, and based on the known structure of the proofpoint response, start processing (and discarding) each file reputation as it is hydrated from the stream.

This removes both the memory and latency overheads at the client end (though doesn't overcome the server side latency, it's the best we can do).

Provide a reputation score mapping feature to override based on drop-down selectors (or custom string?) to make the pollproofpoint node more flexible

persist? 2021-04-15T06:10:45Z
persist var? proofpointPersistence-eca69f22_a9908 2021-04-15T06:10:45Z
decoded persistent file content { lastTimestamp: '2021-04-15T06:10:45Z' }
22 Apr 15:13:29 - [info] [poll proofpoint siem:eca69f22.a9908] lastTimestamp
22 Apr 15:13:29 - [red] Uncaught Exception:
22 Apr 15:13:29 - Error: Callback was already called.
at C:\Users\xxxx.node-red\node_modules\async\dist\async.js:318:36
at C:\Users\xxxx.node-red\node_modules\node-red-contrib-proofpoint\proofpoint\poll.js:34:29
at read (C:\Users\xxxx.node-red\node_modules\node-red-contrib-proofpoint\proofpoint\util.js:100:13)
at FSReqCallback.readFileAfterClose [as oncomplete] (internal/fs/read_file_context.js:63:3)