Comments (8)
Files are closed when the file handles are needed elsewhere and not until then, but they are also closed when the file is finished. There are better ways to know when the file is done. For example, you could monitor the XML file, or you could have a pipe for the XML output. Or some IPC mechanism. What's your preference? Having the XML go to a named pipe or a numbered pipe would be easiest.
from tcpflow.
Thanks for the quick reply!
Interesting - all my files seem to be getting closed almost immediately. I wonder if inotify is interfering with it somehow.
I can monitor the XML file for changes, but I was concerned that it would just keep growing and become massive if I don't restart tcpflow occasionally. Is a file guaranteed to be finished once it's been entered into the XML file?
Alternatively, do you think it makes sense to write the files in /tmp until they're complete, at which point they get moved to the output directory (and the XML gets written to)?
from tcpflow.
Nothing is guaranteed in this life. If out-of-order packets are delivered, the file will be re-opened so that they can be written. Many people who use tcpflow think that tcp is much cleaner than it actually is.
from tcpflow.
So if I want to process then delete the output files, is there any way I can be certain that the file won't be reopened, without completely killing tcpflow?
Am I better off doing something like waiting for the entry in the XML file, then waiting another 5 minutes before processing the file?
from tcpflow.
What problem are you really trying to solve? We are building an API for tcpflow that will allow you to link in a shared library.
from tcpflow.
I'm using tcpflow as one component for a network-based IDS. I feed the output files from tcpflow to foremost (to carve actual PDF/JPG/etc files from things like HTTP responses), then do analysis on the output files from foremost to look for things like shellcode.
So, tcpflow will be essentially running 24/7, constantly outputting more data.
I did a quick test where I did what I described in my previous comment (wait for the XML entry, then sleep for a little while before touching the file) and it appeared to work so far - but I haven't extensively tested it yet.
from tcpflow.
If you have the XML file go to a pipe and read the pipe, you should be fine. I do not think that the XML file flushes after each connection, but you could easily add that.
from tcpflow.
The XML file now flushes after each connection. I want to modify the system so that the connections only terminate when both the FIN is received and all of the data is complete.
from tcpflow.
Related Issues (20)
- Python 3 HOT 4
- configure script falls in infinite loop of "=yes" output on Centos8 and Centos7 HOT 3
- configure error: tcpflow requires a version of Boost that has Boost interval_map and interval_set HOT 6
- Release tag HOT 6
- Generate an error if -R option is provided without the -r option. HOT 4
- Source violates -Werror=format-security HOT 2
- tcpflow: can't parse filter expression: syntax error HOT 2
- Cannot use -S enable_report=NO in write protected directory HOT 6
- .findx file not written in real-time HOT 4
- Provide option to perform reverse DNS lookup of IP addresses HOT 7
- Feature Request: need to Add Stream number HOT 6
- Lack update of read_end_offset in shift_file (tcpip.cpp)
- Is there a way to find media file's HTTP link of non web browser app? HOT 3
- Introduced change creates problem with spurious data HOT 3
- Length in IP header ignored by the code HOT 1
- upgrade to use be20_api
- Some TCP keepalives corrupt the extracted data streams HOT 4
- Remove misslead requirement for python-2.7 HOT 1
- Performance description HOT 1
- Discrepancies between TCPFlow v1.5.1 and v1.6.1 number of flows. Violations occurring with 1.6.1 but not with 1.5.1 as well. HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tcpflow.