oleg-cherednik / zip4jvm Goto Github PK

When adding a file to compression, a file with the same name exists in the archive. an override option should be provided.

Toy example:

I created one directory with a symlink to it and a regular file with a symlink to it like so:

$ mkdir tmp
$ cd tmp
$ mkdir a; ln -s a b; touch c; ln -s c d
$ cd ..
$ tree tmp
tmp
├── a
├── b -> a
├── c
└── d -> c

Ideally what would happen is the following:

$ zip -yr tmp.zip tmp
$ unzip tmp.zip -d newtmp
Archive:  tmp.zip
   creating: newtmp/tmp/
   creating: newtmp/tmp/a/
 extracting: newtmp/tmp/c            
    linking: newtmp/tmp/d            -> c 
    linking: newtmp/tmp/b            -> a 
finishing deferred symbolic links:
  newtmp/tmp/d           -> c
  newtmp/tmp/b           -> a
$ tree newtmp 
newtmp
└── tmp
    ├── a
    ├── b -> a
    ├── c
    └── d -> c

I then attempted to use Zip4j to zip these files.

First attempt is a naive approach. Just setSymbolicLinkAction to INCLUDE_LINK_ONLY:

import net.lingala.zip4j.ZipFile;
import net.lingala.zip4j.exception.ZipException;
import net.lingala.zip4j.model.ZipParameters;

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.stream.Stream;

public class ZipTest {
    public static void main(String[] args) throws IOException {
        try (ZipFile zipFile = new ZipFile("test.zip");
             Stream<Path> paths = Files.walk(Paths.get("tmp"))) {
            paths.forEach(path -> {
                ZipParameters zp = new ZipParameters();
                if ((!Files.isSymbolicLink(path)) && Files.isDirectory(path)) {
                    zp.setFileNameInZip(path + "/");
                } else {
                    zp.setFileNameInZip(path.toString());
                }
                zp.setSymbolicLinkAction(ZipParameters.SymbolicLinkAction.INCLUDE_LINK_ONLY);
                try {
                    zipFile.addFile(path.toFile(), zp);
                } catch (ZipException e) {
                    throw new RuntimeException(e);
                }
            });
        }
    }
}

Let's see what it contains:

$ unzip test.zip -d test && tree test
Archive:  test.zip
   creating: test/tmp/
   creating: test/tmp/a/
 extracting: test/tmp/c              
    linking: test/tmp/d              -> c 
 extracting: test/tmp/b              
finishing deferred symbolic links:
  test/tmp/d             -> c
test
└── tmp
    ├── a
    ├── b
    ├── c
    └── d -> c

2 directories, 3 files

That didn't work. The symlink b to directory a, did not stay a symlink.

Then I tried only setting INCLUDE_LINK_ONLY for paths tested to be symlinks, and manually set CompressionLevel.NO_COMPRESSION and CompressionMethod.STORE.

import net.lingala.zip4j.ZipFile;
import net.lingala.zip4j.exception.ZipException;
import net.lingala.zip4j.model.ZipParameters;
import net.lingala.zip4j.model.enums.CompressionLevel;
import net.lingala.zip4j.model.enums.CompressionMethod;

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.stream.Stream;

public class ZipTest {
    public static void main(String[] args) throws IOException {
        try (ZipFile zipFile = new ZipFile("test.zip");
             Stream<Path> paths = Files.walk(Paths.get("tmp"))) {
            paths.forEach(path -> {
                ZipParameters zp = new ZipParameters();
                if ((!Files.isSymbolicLink(path)) && Files.isDirectory(path)) {
                    zp.setFileNameInZip(path + "/");
                } else {
                    zp.setFileNameInZip(path.toString());
                }
                if (Files.isSymbolicLink(path)) {
                    zp.setSymbolicLinkAction(ZipParameters.SymbolicLinkAction.INCLUDE_LINK_ONLY);
                    zp.setCompressionLevel(CompressionLevel.NO_COMPRESSION);
                    zp.setCompressionMethod(CompressionMethod.STORE);
                }
                try {
                    zipFile.addFile(path.toFile(), zp);
                } catch (ZipException e) {
                    throw new RuntimeException(e);
                }
            });
        }
    }
}

Unzipping output:

$ unzip test.zip -d test   
Archive:  test.zip
   creating: test/tmp/
   creating: test/tmp/a/
 extracting: test/tmp/c              
    linking: test/tmp/d              -> c 
 extracting: test/tmp/b              
finishing deferred symbolic links:
  test/tmp/d             -> c

It still won't link b to a!

Originally posted by @ajpfahnl in srikanth-lingala/zip4j#486

Add an icon for the project as specified here https://stackoverflow.com/questions/40197177/how-to-create-a-logo-for-mvnrepository-artifact

Update Readme during mvn release:perform automatically. There is hint https://stackoverflow.com/questions/34629326/update-versions-contained-in-readme-on-maven-release how to to this.

Add check that all parts of split zip are available.

Usually we set one encryption and compression settings for all entires. So we should provide this settings directly in ZipSettings:

ZipSettings settings = ZipSettings.builder()
                                  .encryption(Encryption.AES_256, password)
                                  .compression(Compression.DEFLATE, CompressionLevel.NORMAL)
                                  .build();

Additionally, we should not allow set a password to ZipEntrySettings without providing encryption algorithm.

In case zip file contains another zip files and so on, we should add a boolean option recursive to extract all included zip files.

When we decompress the zip file, we based on the CentralDirecrtory records. But between LocalEntries it could be additional data. Yes, this is not up to standard, but still. We should extract these data as well.

thanks for maintaining such a top notch library. I wanted to upgrade from 1.3.2 to 2.1.2, but have a question concerning the API change.

Use case:
I have to process incoming zip files. They can be encrypyed or not, I do not know upfront. If they are encrypted i know how to generate the password from the file name.

With the old API, I could create a ZipFile and check its encryption status, and if true I could set the password:

Zipfile zip = new ZipFile(filename);
if( zip.isEncrypted() ){
String password = generatePassword(filename);
zip.setPassword(password);
}
// ... process zip
AFAIK, with the new API, I can only set the password when creating the ZipFile. I cannot set it afterwards. Also not via the ZipParameters, which was also possible with the old API.

Question:
How can i handle this use case with the new API?
Or in other term, what happens when I proactively/preventively supply a password, but the zip is not actually encrypted. Does extraction still work correctly?

String meaninglessPassword = generatePassword(nonEncryptedFilename);
Zipfile zip = new ZipFile(nonEncryptedFilename, meaninglessPassword);
// ... process zip correctly?
Thanks for looking into this.

Zip file encrypted with String Encryption (bit 6 is set) should correctly work for ZipInf.

is it possible to add (or change existing) validity check that will verify if all split files exists?

it happens that sometimes one or few zip split files (z01, z02 ...etc.) are missing on the disk (due to various reasons, not related to zip4j). The extractAll method will start the extraction and will fail only when it encounter the missing file with
Caused by: java.io.FileNotFoundException: zip split file does not exist: /home/ec2-user/bigZip.z11

it would be nice if i could test for missing files in advance. (like zipFile.isValidZipFile() add zipFile.isValidSplitZipFile() )

There's a problem with saving zip in Store compression. WinRar could not open it. The problem is that GeneralPurposeFlag bit3 and correct values are not consistent.

In tests, another tool says, that only Deflated archive could have EXT part.

Rename StandardSplitSrcZip to PkwareSplitSrcZip

When try to unzip zstd archive with AES encryption, it does not work for image. It work for simple text files, but not for image. MAC not match.

Add 96 - JPEG variant compression support.

For STORE compression we know the compression size (it's equal to uncompressed size), therefore no need to add some blocks like DataDescriptor.

zstd compression ratio is very high, decompression is very fast, directory archive uses zstd compression files

https://github.com/mcmilk/7-Zip-zstd

Right now we support streams but generated from files only.
We need to add support of real streams. In this case it's not possible to read CentralDirectory, so all metadata should be read from LocalFileHeader.

When add new entry to zip, not we ignore not found entry. It seems that it should throw exception.

SplitZipOutputStream.openNextDisk()

Here we validate that new disk file name does not aready exists. This should be done before all writes.

Looks like total disk is not correct when ready Zip64. After investigation, I have found that totalDisk is differ in EndCentralDirectory and Zip64.EndCetralDirectoryLocator.

I have found out that many programs like WinRar or WinZip use default system charset when zipping and not set utf8 flag.

It's impossible to solve this issue in general. zip4jvm by default uses utf8 encoding, if not cleared flag by the user (in this case it uses ibm437).

• I this it could be useful to add the ability to set charset externally for correctly extract zip archive (but not for creating a zip archive).
• Additionally, add convert API to convert such archives to correct one using utf8.

ZipInputStream stop finding next entry if entry is a folder with Data Descriptor. This zip could be extracted by unzip in Linux.

Seems this is caused by ZipInputStream.readUntilEndOfEntry

    if (localFileHeader.isDirectory()
        || (localFileHeader.getCompressedSize() == 0 && !localFileHeader.isDataDescriptorExists())) {
      return;
    }

apk file keeps file content in the extra field section. But this section has not correct format for extra field. According this, decompose and view has a problem.

ZipInfoTest should be split into two tests ZipInfoPrintShortInfoTest and ZipInfoDecomposeTest

CentralDirectory.FileHeader.versionMadeBy should be depending on current OS