oleg-cherednik / zip4jvm Goto Github PK
View Code? Open in Web Editor NEWzip files support for JDK application: split, zip64, encryption, streaming
Home Page: https://github.com/oleg-cherednik/zip4jvm
License: Apache License 2.0
zip files support for JDK application: split, zip64, encryption, streaming
Home Page: https://github.com/oleg-cherednik/zip4jvm
License: Apache License 2.0
When adding a file to compression, a file with the same name exists in the archive. an override option should be provided.
Toy example:
I created one directory with a symlink to it and a regular file with a symlink to it like so:
$ mkdir tmp
$ cd tmp
$ mkdir a; ln -s a b; touch c; ln -s c d
$ cd ..
$ tree tmp
tmp
├── a
├── b -> a
├── c
└── d -> c
Ideally what would happen is the following:
$ zip -yr tmp.zip tmp
$ unzip tmp.zip -d newtmp
Archive: tmp.zip
creating: newtmp/tmp/
creating: newtmp/tmp/a/
extracting: newtmp/tmp/c
linking: newtmp/tmp/d -> c
linking: newtmp/tmp/b -> a
finishing deferred symbolic links:
newtmp/tmp/d -> c
newtmp/tmp/b -> a
$ tree newtmp
newtmp
└── tmp
├── a
├── b -> a
├── c
└── d -> c
I then attempted to use Zip4j to zip these files.
First attempt is a naive approach. Just setSymbolicLinkAction
to INCLUDE_LINK_ONLY
:
import net.lingala.zip4j.ZipFile;
import net.lingala.zip4j.exception.ZipException;
import net.lingala.zip4j.model.ZipParameters;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.stream.Stream;
public class ZipTest {
public static void main(String[] args) throws IOException {
try (ZipFile zipFile = new ZipFile("test.zip");
Stream<Path> paths = Files.walk(Paths.get("tmp"))) {
paths.forEach(path -> {
ZipParameters zp = new ZipParameters();
if ((!Files.isSymbolicLink(path)) && Files.isDirectory(path)) {
zp.setFileNameInZip(path + "/");
} else {
zp.setFileNameInZip(path.toString());
}
zp.setSymbolicLinkAction(ZipParameters.SymbolicLinkAction.INCLUDE_LINK_ONLY);
try {
zipFile.addFile(path.toFile(), zp);
} catch (ZipException e) {
throw new RuntimeException(e);
}
});
}
}
}
Let's see what it contains:
$ unzip test.zip -d test && tree test
Archive: test.zip
creating: test/tmp/
creating: test/tmp/a/
extracting: test/tmp/c
linking: test/tmp/d -> c
extracting: test/tmp/b
finishing deferred symbolic links:
test/tmp/d -> c
test
└── tmp
├── a
├── b
├── c
└── d -> c
2 directories, 3 files
That didn't work. The symlink b
to directory a
, did not stay a symlink.
Then I tried only setting INCLUDE_LINK_ONLY
for paths tested to be symlinks, and manually set CompressionLevel.NO_COMPRESSION
and CompressionMethod.STORE
.
import net.lingala.zip4j.ZipFile;
import net.lingala.zip4j.exception.ZipException;
import net.lingala.zip4j.model.ZipParameters;
import net.lingala.zip4j.model.enums.CompressionLevel;
import net.lingala.zip4j.model.enums.CompressionMethod;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.stream.Stream;
public class ZipTest {
public static void main(String[] args) throws IOException {
try (ZipFile zipFile = new ZipFile("test.zip");
Stream<Path> paths = Files.walk(Paths.get("tmp"))) {
paths.forEach(path -> {
ZipParameters zp = new ZipParameters();
if ((!Files.isSymbolicLink(path)) && Files.isDirectory(path)) {
zp.setFileNameInZip(path + "/");
} else {
zp.setFileNameInZip(path.toString());
}
if (Files.isSymbolicLink(path)) {
zp.setSymbolicLinkAction(ZipParameters.SymbolicLinkAction.INCLUDE_LINK_ONLY);
zp.setCompressionLevel(CompressionLevel.NO_COMPRESSION);
zp.setCompressionMethod(CompressionMethod.STORE);
}
try {
zipFile.addFile(path.toFile(), zp);
} catch (ZipException e) {
throw new RuntimeException(e);
}
});
}
}
}
Unzipping output:
$ unzip test.zip -d test
Archive: test.zip
creating: test/tmp/
creating: test/tmp/a/
extracting: test/tmp/c
linking: test/tmp/d -> c
extracting: test/tmp/b
finishing deferred symbolic links:
test/tmp/d -> c
It still won't link b
to a
!
Originally posted by @ajpfahnl in srikanth-lingala/zip4j#486
Add an icon for the project as specified here https://stackoverflow.com/questions/40197177/how-to-create-a-logo-for-mvnrepository-artifact
Update Readme during mvn release:perform automatically. There is hint https://stackoverflow.com/questions/34629326/update-versions-contained-in-readme-on-maven-release how to to this.
Add check that all parts of split zip are available.
Usually we set one encryption and compression settings for all entires. So we should provide this settings directly in ZipSettings
:
ZipSettings settings = ZipSettings.builder()
.encryption(Encryption.AES_256, password)
.compression(Compression.DEFLATE, CompressionLevel.NORMAL)
.build();
Additionally, we should not allow set a password to ZipEntrySettings
without providing encryption algorithm.
In case zip file contains another zip files and so on, we should add a boolean option recursive to extract all included zip files.
When we decompress the zip file, we based on the CentralDirecrtory records. But between LocalEntries it could be additional data. Yes, this is not up to standard, but still. We should extract these data as well.
thanks for maintaining such a top notch library. I wanted to upgrade from 1.3.2 to 2.1.2, but have a question concerning the API change.
Use case:
I have to process incoming zip files. They can be encrypyed or not, I do not know upfront. If they are encrypted i know how to generate the password from the file name.
With the old API, I could create a ZipFile and check its encryption status, and if true I could set the password:
Zipfile zip = new ZipFile(filename);
if( zip.isEncrypted() ){
String password = generatePassword(filename);
zip.setPassword(password);
}
// ... process zip
AFAIK, with the new API, I can only set the password when creating the ZipFile. I cannot set it afterwards. Also not via the ZipParameters, which was also possible with the old API.
Question:
How can i handle this use case with the new API?
Or in other term, what happens when I proactively/preventively supply a password, but the zip is not actually encrypted. Does extraction still work correctly?
String meaninglessPassword = generatePassword(nonEncryptedFilename);
Zipfile zip = new ZipFile(nonEncryptedFilename, meaninglessPassword);
// ... process zip correctly?
Thanks for looking into this.
Zip file encrypted with String Encryption (bit 6 is set) should correctly work for ZipInf.
is it possible to add (or change existing) validity check that will verify if all split files exists?
it happens that sometimes one or few zip split files (z01, z02 ...etc.) are missing on the disk (due to various reasons, not related to zip4j). The extractAll method will start the extraction and will fail only when it encounter the missing file with
Caused by: java.io.FileNotFoundException: zip split file does not exist: /home/ec2-user/bigZip.z11
it would be nice if i could test for missing files in advance. (like zipFile.isValidZipFile() add zipFile.isValidSplitZipFile()
)
There's a problem with saving zip in Store compression. WinRar could not open it. The problem is that GeneralPurposeFlag bit3 and correct values are not consistent.
In tests, another tool says, that only Deflated archive could have EXT part.
Rename StandardSplitSrcZip
to PkwareSplitSrcZip
When try to unzip zstd archive with AES encryption, it does not work for image. It work for simple text files, but not for image. MAC not match.
Add 96 - JPEG variant
compression support.
For STORE compression we know the compression size (it's equal to uncompressed size), therefore no need to add some blocks like DataDescriptor.
zstd compression ratio is very high, decompression is very fast, directory archive uses zstd compression files
Right now we support streams but generated from files only.
We need to add support of real streams. In this case it's not possible to read CentralDirectory, so all metadata should be read from LocalFileHeader.
When add new entry to zip, not we ignore not found entry. It seems that it should throw exception.
SplitZipOutputStream.openNextDisk()
Here we validate that new disk file name does not aready exists. This should be done before all writes.
Looks like total disk is not correct when ready Zip64. After investigation, I have found that totalDisk is differ in EndCentralDirectory
and Zip64.EndCetralDirectoryLocator
.
I have found out that many programs like WinRar or WinZip use default system charset when zipping and not set utf8 flag.
It's impossible to solve this issue in general. zip4jvm by default uses utf8 encoding, if not cleared flag by the user (in this case it uses ibm437).
ZipInputStream stop finding next entry if entry is a folder with Data Descriptor. This zip could be extracted by unzip in Linux.
Seems this is caused by ZipInputStream.readUntilEndOfEntry
if (localFileHeader.isDirectory()
|| (localFileHeader.getCompressedSize() == 0 && !localFileHeader.isDataDescriptorExists())) {
return;
}
apk
file keeps file content in the extra field section. But this section has not correct format for extra field. According this, decompose and view has a problem.
ZipInfoTest
should be split into two tests ZipInfoPrintShortInfoTest
and ZipInfoDecomposeTest
CentralDirectory.FileHeader.versionMadeBy
should be depending on current OS
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.