dgets / recscan Goto Github PK

Try to keep the try/catch blocks a little slimmer in order to more concisely pinpoint what particular line of code is really effin' it all up, without having to resort to using the (slow) debugger.

RecScan is claiming that /home/ouah/Downloads/ethminer-0.11.0-Linux.tar.gz, being used for a test archive, is not named properly (ie doesn't have one of the proper suffixes).

Need to watch out for other flags, attributes, or whatever else may be tucked away as another tar entry, when we are reading from the stream.

I was just thinking, I suppose the 'PK' prefix to -Zip up there probably shows my age a bit. It's just WinZip now, isn't it? A truly appropriate thought for today, being my 40th birthday.

Now that I'm doing development in a native Windows partition for awhile, I guess it would be a good time to start writing the Zip support into the MultiZip class. It's at least a good opportunity to make sure that I'm keeping the different levels of functionality somewhat similar for devel & debugging.

The static fields describing the different archival and compression options are really a cumbersome and obtuse way to be dealing with all of that information. Instead of a 'flat' listing of HashMap values, I think that perhaps some nested HashMap, and/or Array values, would probably be better for accessing these options and flags.

Once we hit post-alpha work, there will have to be checking for whether or not the specified archive is in the current directory. If not, this path will obviously have to be stored, as well, and taken into account when moving to the temp dir, etc. At present, in alpha work, there is only going to be support for specifying an archive name within the current directory.

Not totally sure that I've been learning testing the right way yet, but I'm working on it. If anybody wants to assist with critiquing what I've done here, that'd be great & much appreciated.

Something in Arc.init() is throwing the following tantrum:

Verbose:
Initializing tgtInvalid/unhandled archive
Fucked1: Unhandled or invalid archive
Fubar in getRawlist(): Problem in Arc.init()
Borkage: getRawlist() borked

Came across this when trying to see if maybe the problem in dealing with a tar.gz file was related to the decompression as opposed to the actual untarring. I read about multiple input streams being utilized for some _tar.gz_s here, so it seemed like a decent new route to take debugging. Going to be working on this right away, en route to #15.

No idea where this exception with a null message is coming from right now, but it is tracked to this method, at least. I've got a feeling that the StackExchange post on reading a tar directory here may well come in useful. There's probably way too much superfluous code in my method that seemed necessary, as I was stumbling through the apache libraries at first.

Self-explanatory. Will help out for the most simple debugging, also.

It has become apparent in commit 8a248be that the Archive class needs to also contain flags for listing files, as well as decompressing (derp).

Not sure how to read the TarInputStream to EOF just yet. Seems like it should be a ridiculously simple problem to fix, but I'm having trouble locating anything about it in the api. :| I'll have to look into whether or not it has something to do with watching the byte count manually... Seems like it should be handled a lot more professionally than that, though.

Then again, it looks like there may be other issues that need correcting along the way. I just attempted testing with an archive other than my normal test archive. The different runs crashed in different places, with different exceptions. :(

Here is the output from the new archive:

Verbose:
Initializing tgt
Executing tgt.getArcType()
Archive is tar.gz
Opening fis . . .
Opening zis . . .
Opening tarInput . . .
Fucked: tarListDir() java.io.IOException: Stream Closed
Borkage: java.io.IOException: Stream Closed

Here's the output from the original test archive:

Verbose:
Initializing tgt
Executing tgt.getArcType()
Archive is tar.gz
Opening fis . . .
Opening zis . . .
Opening tarInput . . .
Read entry #1
Pulling entry: pax_global_header
-Found Global Pax Header-
Read entry #2
Pulling entry: ossec-hids-2.9.1/
-Valid Directory Found-
Read entry #3
Pulling entry: ossec-hids-2.9.1/.gitignore
-Valid File Found-
NP Fucked: tarListDir() java.lang.NullPointerException

ProcessBuilder is almost certainly not going to be the type of construct that we're going to have to use for any archive that is compressed and tarballed (or otherwise archived) separately. Not sure if this will have to be taken care of with some other method that will allow 'piping', or if things will have to be decompressed, and then unarchived (or contents listed) separately. It seems a little bit overkill to have to do things like that, but then again I guess that's what a shell pipe is doing, anyway.

It may be useful to limit the levels of recursion that RecScan will descend and process; this could be very handy in cases where the archive is large, or drive space is too limited.

First time working with Input-/OutputStreams & Tar library variants thereof, and, of course, they're giving me some trouble. Need to figure out why this is botched and probably completely rewrite the code handing off the different Streams, as it was all cut 'n pasted and modified for 'compatibility' afterwards (when I was sleep deprived, at that). Obviously, I don't understand the code well enough yet, and it's probably a good idea, just to wrap my head around all of that better.

More data to follow once I've had a chance to collect it.

Clean out /tmp directory after usage, on case of fault related program exit.