brendan-duncan / archive Goto Github PK
View Code? Open in Web Editor NEWDart library to encode and decode various archive and compression formats, such as Zip, Tar, GZip, ZLib, and BZip2.
License: MIT License
Dart library to encode and decode various archive and compression formats, such as Zip, Tar, GZip, ZLib, and BZip2.
License: MIT License
Scenario
dart archive.dart default.ods
(see code below)Get 'default.ods' file from following link:
https://github.com/sestegra/spreadsheet_decoder/raw/master/test/files/default.ods
archive.dart
import 'dart:io';
import 'package:archive/archive.dart';
void main(List<String> args) {
var input = args[0];
var archive = new ZipDecoder().decodeBytes(new File(input).readAsBytesSync(), verify: true);
var zip = new ZipEncoder().encode(archive);
archive = new ZipDecoder().decodeBytes(zip, verify: true);
}
$ dart archive.dart default.ods
Unhandled exception:
ArchiveException: Invalid CRC for file in archive.
#0 ZipDecoder.decodeBuffer (package:archive/src/zip_decoder.dart:29:11)
#1 ZipDecoder.decodeBytes (package:archive/src/zip_decoder.dart:10:12)
#2 main (file:///Users/sestegra/reyer/repos/spreadsheet_decoder/example/archive.dart:64:30)
#3 _startIsolate.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:263)
#4 _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:151)
I'm trying to automatically download and extract ImageMagick, I'm downloading it as a zip from the following url: http://www.imagemagick.org/download/binaries/ImageMagick-6.8.8-4-Q16-x86-windows.zip
But it is failing. The file seems to be correct as I can extract it with different tools (e.g. the Windows build in one and IZArc). The following exception is thrown:
Uncaught Error: RangeError: 229376 must be in the range [0..229376)
Stack Trace:
#0 _throwRangeError (dart:typed_data-patch/typed_data.dart:3372)
#1 Uint8List.[]= (dart:typed_data-patch/typed_data.dart:716)
#2 OutputStream.writeByte (package:archive/src/util/output_stream.dart:33:12)
#3 Inflate._decodeHuffman (package:archive/src/zlib/inflate.dart:213:25)
#4 Inflate._parseDynamicHuffmanBlock (package:archive/src/zlib/inflate.dart:195:19)
#5 Inflate._parseBlock (package:archive/src/zlib/inflate.dart:60:34)
#6 Inflate._inflate (package:archive/src/zlib/inflate.dart:31:23)
#7 Inflate.Inflate (package:archive/src/zlib/inflate.dart:10:13)
#8 ZipFile.content (package:archive/src/zip/zip_file.dart:82:22)
#9 ZipDecoder.decodeBuffer (package:archive/src/zip_decoder.dart:21:39)
#10 ZipDecoder.decodeBytes (package:archive/src/zip_decoder.dart:10:24)
#11 installPrerequisites.<anonymous closure>
#12 _Future._propagateToListeners.<anonymous closure> (dart:async/future_impl.dart:453)
#13 _rootRun (dart:async/zone.dart:683)
#14 _RootZone.run (dart:async/zone.dart:832)
#15 _Future._propagateToListeners (dart:async/future_impl.dart:445)
#16 _Future._complete (dart:async/future_impl.dart:303)
#17 _Future._asyncComplete.<anonymous closure> (dart:async/future_impl.dart:354)
#18 _asyncRunCallback (dart:async/schedule_microtask.dart:18)
#19 _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:119)
Unhandled exception:
RangeError: 229376 must be in the range [0..229376)
#0 _rootHandleUncaughtError.<anonymous closure>.<anonymous closure> (dart:async/zone.dart:677)
#1 _asyncRunCallback (dart:async/schedule_microtask.dart:18)
#2 _asyncRunCallback (dart:async/schedule_microtask.dart:21)
#3 _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:119)
I'm calling it via:
final archiveFile = new File(filename);
final archiveBytes = archiveFile.readAsBytesSync();
final archive = new pak.ZipDecoder().decodeBytes(archiveBytes); // Crash
Thanks for your support!
The latest version seems to be generating archives that can't be decompressed with standard tar
implementations. To reproduce:
import 'dart:io';
import 'package:archive/archive.dart';
void main() {
new File("out.tar.gz").writeAsBytesSync(
new GZipEncoder().encode(new TarEncoder().encode(new Archive())));
}
Running tar xzf out.tar.gz
with GNU tar 1.27.1 prints:
gzip: stdin: invalid compressed data--crc error
gzip: stdin: invalid compressed data--length error
tar: Child returned status 1
tar: Error is not recoverable: exiting now
The same script produces a valid tar file with archive
1.0.29.
Seems like the Archive
class cannot properly handle encoded characters in file headers.
(please note that this has been tried with archive~1.0.33
, please close issue when already fixed)
Hi brendan,
it seems there is an error in zip_encoder.dart
int d1 = ((date.month + 1 & 0x7) << 5) | (date.day);
int d2 = ((date.year - 1980 & 0x7f) << 1) | (date.month + 1 >> 3);
I think you should replace date.month + 1 with date.month
I added "archive: any" (which now resolves to 1.0.27) and after "pub get" I received:
A transformer imported unknown package "args" (in "package:args/args.dart").
Adding "args: any" didn't solve the problem.
I also created an issue for pub, BTW:
dart-lang/pub#1473
My pubspec.yaml:
name: something
description: A Dart app that uses Angular 2
version: 0.0.1
environment:
sdk: '>=1.19.0 <2.0.0'
dependencies:
angular2: ^2.0.0
googleapis: any
googleapis_auth: any
archive: any
dev_dependencies:
browser: ^0.10.0
dart_to_js_script_rewriter: ^1.0.1
transformers:
Perhaps it's not possible with this package, but I can't figure out how to create a new Archive from files within my app. For example I want to zip up some files within a sub-directory of my app. All the examples only call out manipulating an already existing archive file.
In your example it looks like you recursively write the files found in the archive.
In the root of my archive is a directory and I get the error :
E/flutter ( 3054): [ERROR:topaz/lib/tonic/logging/dart_error.cc(16)] Unhandled exception:
E/flutter ( 3054): FileSystemException: Cannot create file, path = '/data/data/com.rockvole.kbmlviewer/app_flutter/out/how2instructions-master/' (OS Error: Is a directory, errno = 21)
E/flutter ( 3054): #0 _File.throwIfError (dart:io/file_impl.dart:628)
E/flutter ( 3054): #1 _File.createSync (dart:io/file_impl.dart:271)
E/flutter ( 3054): #2 _PreferencesPageState._decompressFile (/data/data/com.rockvole.kbmlviewer/cache/kbml_viewerQGEUIX/kbml_viewer/lib/preferences.dart:56:11)
E/flutter ( 3054): #3 _PreferencesPageState.getPreferenceText.<anonymous closure>.<anonymous closure> (/data/data/com.rockvole.kbmlviewer/cache/kbml_viewerQGEUIX/kbml_viewer/lib/preferences.dart:124:19)
E/flutter ( 3054): #4 _RootZone.runUnary (dart:async/zone.dart:1381)
E/flutter ( 3054): #5 _FutureListener.handleValue (dart:async/future_impl.dart:129)
E/flutter ( 3054): #6 _Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:633)
E/flutter ( 3054): #7 _Future._propagateToListeners (dart:async/future_impl.dart:662)
E/flutter ( 3054): #8 _Future._complete (dart:async/future_impl.dart:467)
E/flutter ( 3054): #9 _SyncCompleter.complete (dart:async/future_impl.dart:51)
E/flutter ( 3054): #10 _completeOnAsyncReturn (dart:async-patch/dart:async/async_patch.dart:255)
E/flutter ( 3054): #11 _PreferencesPageState._downloadFile (/data/data/com.rockvole.kbmlviewer/cache/kbml_viewerQGEUIX/kbml_viewer/lib/preferences.dart:45:5)
E/flutter ( 3054): #12 _RootZone.runUnary (dart:async/zone.dart:1381)
E/flutter ( 3054): #13 _FutureListener.handleValue (dart:async/future_impl.dart:129)
E/flutter ( 3054): #14 _Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:633)
E/flutter ( 3054): #15 _Future._propagateToListeners (dart:async/future_impl.dart:662)
E/flutter ( 3054): #16 _Future._complete (dart:async/future_impl.dart:467)
E/flutter ( 3054): #17 _SyncCompleter.complete (dart:async/future_impl.dart:51)
E/flutter ( 3054): #18 _completeOnAsyncReturn (dart:async-patch/dart:async/async_patch.dart:255)
E/flutter ( 3054): #19 getApplicationDocumentsDirectory (package:path_provider/path_provider.dart:44:3)
E/flutter ( 3054): #20 _RootZone.runUnary (dart:async/zone.dart:1381)
E/flutter ( 3054): #21 _FutureListener.handleValue (dart:async/future_impl.dart:129)
E/flutter ( 3054): #22 _Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:633)
E/flutter ( 3054): #23 _Future._propagateToListeners (dart:async/future_impl.dart:662)
E/flutter ( 3054): #24 _Future._complete (dart:async/future_impl.dart:467)
E/flutter ( 3054): #25 _SyncCompleter.complete (dart:async/future_impl.dart:51)
E/flutter ( 3054): #26 _completeOnAsyncReturn (dart:async-patch/dart:async/async_patch.dart:255)
E/flutter ( 3054): #27 MethodChannel.invokeMethod (package:flutter/src/services/platform_channel.dart:154:5)
E/flutter ( 3054): #28 _RootZone.runUnary (dart:async/zone.dart:1381)
E/flutter ( 3054): #29 _FutureListener.handleValue (dart:async/future_impl.dart:129)
E/flutter ( 3054): #30 _Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:633)
E/flutter ( 3054): #31 _Future._propagateToListeners (dart:async/future_impl.dart:662)
E/flutter ( 3054): #32 _Future._completeWithValue (dart:async/future_impl.dart:477)
E/flutter ( 3054): #33 _Future._asyncComplete.<anonymous closure> (dart:async/future_impl.dart:507)
E/flutter ( 3054): #34 _microtaskLoop (dart:async/schedule_microtask.dart:41)
E/flutter ( 3054): #35 _startMicrotaskLoop (dart:async/schedule_microtask.dart:50)
I converted a complex Java program to Dart. Part of the program is to go throug various zip archives and read one entry of multiple existing ones in each zip file. Replacing the Java ZipFile/ZipEntry with your Archive/ArchiveFile was easy. The problem arrived when running the program. Unit tests went through but the Dart program took about 30 seconds compared to 2 seconds for the Java version.
I know from other measures I have done that Dart is on average 20-40% slower than Java for equivalent algorithms, for some library functions I found Dart equivalents up to 100% slower than their Java counterparts.
The performance of your archive packages points to something more serious. Probably you use very inefficient algorithms? Or could it be that you decode all zip entries of an archive file no matter whether they get used? In my example I always use only one entry and I would expect that the content gets decompressed only when referenced and not before to save time and memory.
Could you have a look at this issue?
This would be to capture some strong mode fixes in #23.
Right now it's in the dev_dependencies
section but it really needs to be in dependencies
since the public tool bin/tar.dart
relies on it.
See dart-lang/pub#1414.
hi,
the library works fine in Dartium, but it doesn't work when compiled using dart2js unfortunately.
the error occurs when calling decodeBytes();
you can find my code here: https://github.com/TomTasche/OpenDocument.dart/blob/76bfe183247ac76091b4cd77485caf986e1e9d1f/web/opendocument.dart#L24
is there anything we can do to make it work in JS too?
Hi and thanks for the package. In my usage, I was zipping a folder with an executable and I noticed the unixPermissions was not properly set (always null) when using ZipFileEncoder and the executable permission was lost.
The issue can be seen in the following unit tests (maybe something simpler could be found but this is a real usage scenario I'm using - tested on Linux but I guess it is wrong also on MacOS):
import 'dart:io';
import 'package:archive/archive_io.dart';
import 'package:test/test.dart';
import 'package:path/path.dart';
void main() {
test('zip_executable', () async {
// Only tested on linux so far
if (Platform.isLinux) {
var path = join('.dart_tool', 'archive', 'test', 'zip_executable');
var srcPath = join(path, 'src');
try {
await Directory(path).deleteSync(recursive: true);
} catch (_) {}
await Directory(srcPath).create(recursive: true);
// Create an executable file and zip it
var file = File(join(srcPath, 'test.bin'));
await file.writeAsString('bin', flush: true);
await Process.run('chmod', ['+x', file.path]);
// Check permission
// permission executable b001001001 = 0x49
expect((await file.stat()).mode & 0x49, 0x49);
var dstFilePath = join(path, 'test.zip');
ZipFileEncoder().zipDirectory(Directory(srcPath), filename: dstFilePath);
// Read
List<int> bytes = await File(dstFilePath).readAsBytes();
// Decode the Zip file
Archive archive = ZipDecoder().decodeBytes(bytes);
var archiveFile = archive.first;
// Fails! unixPermissions is null
expect(archiveFile.unixPermissions & 0x49, 0x49);
}
});
}
When I try to encode a zip file with filenames that contain german umlauts (ÜäßĂĂĂĂ) the encoder does not complain and creates a zip file. However the ZipDecoder cannot decode the file and fails with the following exception. Also many other unzipping programs fail to decode such an archive.
FormatException: Bad UTF-8 encoding 0xf6 (at offset 4)
dart:convert Utf8Decoder.convert
package:archive/src/util/input_stream.dart 183:25 InputStream.readString
package:archive/src/zip/zip_file_header.dart 44:26 new ZipFileHeader
package:archive/src/zip/zip_directory.dart 53:23 new ZipDirectory.read
package:archive/src/zip_decoder.dart 21:30 ZipDecoder.decodeBuffer
package:archive/src/zip_decoder.dart 15:12 ZipDecoder.decodeBytes
test/zip_file_names_test.dart 22:38 main.<fn>
Here is a test code to reproduce this:
test('file encoding zip file', () {
String origialFileName = "fileÜäßĂĂĂĂ.txt";
List<int> bytes = Utf8Codec().encode("test");
ArchiveFile archiveFile = ArchiveFile(origialFileName, bytes.length, bytes);
Archive archive = Archive();
archive.addFile(archiveFile);
ZipEncoder encoder = ZipEncoder();
ZipDecoder decoder = ZipDecoder();
var encodedBytes = encoder.encode(archive);
Archive archiveDecoded = decoder.decodeBytes(encodedBytes);
ArchiveFile decodedFile = archiveDecoded.files.first;
expect(decodedFile.name, origialFileName);
});
My workaround right now is to encode the filename to utf8 before it is passed to the ArchiveFile like this:
String utf8EncodedFileName =
String.fromCharCodes(Utf8Encoder().convert(origialFileName));
I think this should happen in the encoder.
The pub site can show example code for a package.
https://www.dartlang.org/tools/pub/package-layout#examples
Let me know if you want me to try working on this.
related issue : dart-lang/site-www#413
According to the issue dart-lang/sdk#35352 users might fall into performance trap when using this package instead of builtin ZLibCodec on Dart VM.
Can we change this package in a way that it falls back to faster builtin implementation when dart:io library is available?
this is the best library to decompress files but the lack of rar suport make it limp a little, it will be amazing to get unrar functions
I'm trying to write a client app that downloads and processes .zip or .bz2 files (created by 7-zip).
How should the downloaded String be converted to the List int that BZip2Decoder or ZipDecoder expect?
Thanks
Running tests in dart 2 mode
dart --preview-dart-2 test/input_stream_test.dart
type 'List' is not a subtype of type 'List' in type cast where
List is from dart:core
List is from dart:core
int is from dart:core
dart:core Object._as
../../archive/lib/src/util/input_stream.dart 18:14 new InputStream
../..//archive/test/input_stream_test.dart 6:29 main.
Because of the bad behaviour I reported here the code
archive/lib/src/bzip2_encoder.dart
Line 1312 in b6ddae3
causes problems if bbSize = 0! (What will happen in every call)
Always make sure not to shift in negativ numbers. Here is my temp fix I put in. But of course this will only be in there until the next update.
if (i < 255) {
int bbStart = _ftab[ss << 8] & CLEARMASK;
int bbSize = (_ftab[(ss + 1) << 8] & CLEARMASK) - bbStart;
int shifts = 0;
if (bbSize > 0) {
while ((bbSize >> shifts) > 65534) {
shifts++;
}
for (j = bbSize - 1; j >= 0; j--) {
int a2update = ptr[bbStart + j];
int qVal = (j >> shifts) & 0xffff;
quadrant[a2update] = qVal;
if (a2update < BZ_N_OVERSHOOT) {
quadrant[a2update + nblock] = qVal;
}
}
_assert(((bbSize - 1) >> shifts) <= 65535);
}
As @rakudrama pointed out you can also force the sign back:
_assert(((bbSize - 1) >> shifts).toSigned(32) <= 65535);
ZipDecoder.decodeBytes(...) expects List<int>
. Is there a reason why you don't use Uint8List? It should consume less memory.
I'm validating your library to check whether we could using it in pub, instead of invoking the external tar
command every time. I've found the FormatException: Invalid radix-8 number
exception with the following stacktrace:
dart:core int.parse
package:archive/src/tar/tar_file.dart 177:17 TarFile._parseInt
package:archive/src/tar/tar_file.dart 65:15 TarFile.TarFile.read
package:archive/src/tar_decoder.dart 23:24 TarDecoder.decodeBuffer
package:archive/src/tar_decoder.dart 10:12 TarDecoder.decodeBytes
I've also created a simple test for it:
import 'dart:io';
import 'package:archive/archive.dart';
import 'package:test/test.dart';
void main() {
group('pub archives', () {
HttpClient client;
setUpAll(() {
client = new HttpClient();
});
tearDownAll(() {
client.close(force: true);
});
test('logfmt 0.4.0', () async {
final HttpClientRequest rq = await client.getUrl(Uri.parse(
'https://storage.googleapis.com/pub-packages/packages/logfmt-0.4.0.tar.gz'));
final HttpClientResponse rs = await rq.close();
final List<int> data = (await rs.toList())
.fold([], (List<int> a, List<int> b) => a..addAll(b));
expect(data.length, 10240);
final Archive archive =
new TarDecoder().decodeBytes(new GZipDecoder().decodeBytes(data));
expect(archive.toList(), isNotEmpty);
});
});
}
Please let me know what you think of this issue. I can provide further .tar.gz examples, which have the same parsing error message.
I have downloaded a zip from a link and extracted the files to the same folder in a sub directory,
I can clearly see these files from local file explorer but when i connect it to my pc, the folders are not listed.
The code I used
extractFiles() async{
final dir = await Storage().localPath;
final _directory = Directory(dir+"/bizzapp/zip");
List<FileSystemEntity> _files;
_files = _directory.listSync(recursive: true, followLinks: false);
// Read the Zip file from disk.
List<int> bytes = new File(_files[0].path).readAsBytesSync();
// Decode the Zip file
Archive archive = new ZipDecoder().decodeBytes(bytes);
// Extract the contents of the Zip archive to disk.
for (ArchiveFile file in archive) {
String filename = file.name;
if (file.isFile) {
List<int> data = file.content;
new File(dir+'/bizzapp/' + filename)
..createSync(recursive: true)
..writeAsBytesSync(data);
} else {
new Directory(dir+'/bizzapp/' + filename)
..create(recursive: true);
}
}
}
With the upgrade of Dart SDK to 1.24 I get an error:
Failed to precompile archive:tar: 'package:convert/src/hex/decoder.dart': malformed type: line 21 pos 13: cannot resolve class 'ChunkedConverter' from 'HexDecoder' extends ChunkedConverter<String, List<int>, String, List<int>> { ^
This causes me to keep 1.23 for now.
GNU tar puts filenames in files when they exceed tar's native length. The fix is pretty straightforward
String nextName = null;
for (var archiveFile in archive) {
if (archiveFile.name == '././@LongLink') {
nextName = archiveFile.rawContent.readString();
continue;
}
var name = nextName ?? archiveFile.name;
nextName = null;
...process the file here...
}
Encoder and Decoder classes would be derived from Converter, and a Codec class should be provided to combine the two, following the pattern used by the json, utf8, etc converters.
https://api.dartlang.org/apidocs/channels/stable/#dart-convert.Codec@id_encode
Try decompressing all of https://storage.googleapis.com/dart-archive/channels/dev/release/latest/dartium/dartium-linux-x64-release.zip
For me this gets caught in a [seemingly infinite] loop at the chrome binary, with 1g (99%) of old gen full of allocated lists.
Hello,
I am getting this crash message when I try to extract a zip file for the second time:
Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x6b in tid 7836 (1.ui), pid 7818
The error occurs when file.content is called.
final archive = new ZipDecoder().decodeBytes(request.bodyBytes);
for (ArchiveFile file in archive) {
String filename = file.name;
if (file.isFile) {
List<int> data = file.content;
new File("$databasesPath/" + filename)
..createSync(recursive: true)
..writeAsBytesSync(data);
} else {
new Directory('$databasesPath/' + filename)
..create(recursive: true);
}
}
If you decode a tar file like this:
import 'dart:io';
import 'package:archive/archive.dart';
import 'package:archive/archive_io.dart';
void main(args) {
if (args.length != 2) {
print('usage: example.dart a.tar folder');
return;
}
String tarPath = args[0];
Uri dir = Uri.base.resolve("${args[1]}/");
// Switching to reading using `decodeBuffer` fixes it.
var archive = TarDecoder().decodeBytes(File(tarPath).readAsBytesSync());
for (var file in archive) {
if (file.name.endsWith('/')) continue;
var outputUri = dir.resolve(file.name);
File.fromUri(outputUri).parent.createSync(recursive: true);
// Switching to writing using `toUnit8List` fixes it.
// new File.fromUri(outputUri).writeAsBytesSync(file.rawContent.toUint8List());
new OutputFileStream(outputUri.toFilePath()).writeInputStream(file.rawContent);
}
}
then you get exceptions and/or truncated data. Either using decodeBuffer
to read the data, or toUint8List
to write the data, makes it work again.
I tried to create zip file with following code:
ZipFileEncoder encoder = ZipFileEncoder();
encoder.create(join(documentsDirectory.path, 'product.zip'));
encoder.addFile(File('${documentsDirectory.path}/TestDB.db'));
List _supplierList = await DBProvider.db.getAllSupplier();
_supplierList.forEach((Supplier _supplier) {
encoder.addFile(File(_supplier.photo));
_generateItemArchive(encoder, _supplier.id);
});
encoder.close();//here exception raised.
void _generateItemArchive(ZipFileEncoder encoder, int id) async {
List _itemList = await DBProvider.db.getAllItem(id);
_itemList.forEach((Item _item) {
encoder.addFile(File(_item.photo));
});
What I am missing? The zip file created with above code is invalid. Can you please guid?
Brendan,
Thanks for the archive library for Dart!
Quick question...I have a dart web app where I read text files, parse them and export the parsed text files using your archive library to make a downloadable zip file.
The library keeps crashing when I try to encode an Archive that has one ArchiveFile in it:
Archive archive = new Archive();
// String content = "This is test content.";
List content = [1,2,3,4,5,6,7,8,9,10];
ArchiveFile arcfile = new ArchiveFile('test.txt', content.length*8, content);
archive.addFile(arcfile);
//CRASHES AT THIS POINT
List tar_data = new TarEncoder().encode(archive);
List tar_bz2 = new BZip2Encoder().encode(tar_data);
String base64text = Convert.BASE64.encode(tar_bz2);
One question I have is for the ArchiveFile constructor, is the "size" parameter the size of the content in bytes? or is it the size in ints?
Either way, the code crashes with this stack trace:
Uncaught Unhandled exception:
NoSuchMethodError: The method 'toRadixString' was called on null.
Receiver: null
Tried calling: toRadixString(8)
#0 Object._noSuchMethod (dart:core-patch/object_patch.dart:44)
#1 Object.noSuchMethod (dart:core-patch/object_patch.dart:47)
#2 TarFile._writeInt (package:archive/src/tar/tar_file.dart:198:22)
#3 TarFile.write (package:archive/src/tar/tar_file.dart:117:5)
#4 TarEncoder.encode (package:archive/src/tar_encoder.dart:19:10)
#5 onSaveButton (http://localhost:63342/CDL2MatchboxWebVer/web/main.dart:40:41)
#6 _EventStreamSubscription._EventStreamSubscription. (dart:html:44616)
Thanks in advance for any help!
It will make folks understand better what release versions do :)
We're seeing some issues when inflating deflated content. We're trying to switch over from using a JS inflater to the one from this package. Identical code, with just the inflater switched out, is throwing this exception:
#0 Inflate._parseUncompressedBlock (package:archive/src/zlib/inflate.dart:151:7)
#1 Inflate._parseBlock (package:archive/src/zlib/inflate.dart:54:32)
#2 Inflate._inflate (package:archive/src/zlib/inflate.dart:31:23)
#3 Inflate.Inflate.buffer (package:archive/src/zlib/inflate.dart:16:13)
I'm not sure if it's an issue w/ how we're driving the Inflate
class or an actual issue in the inflate code. I'll try and work up a reproducible test case.
On Windows systems, it is not possible to determine if an archived entity is in fact a directory or file.
Currently this is the code snipped that would do it:
final unixAttributes = zfh.externalFileAttributes >> 16;
final isDirectory = unixAttributes & 0x7000 == 0x4000;
final isFile = unixAttributes & 0x3F000 == 0x8000;
There are cases when the above will say isDirectory = false
and isFile = false
. For many Zip archives, the external attributes bytes are set to 0
and thus no meaningful determination can be made base on that.
This is what OpenJDK 8 does for isDirectory
:
Hi, thanks for the package. Archiving works well except one device - I pasted error below.
Package version: 2.0.8
Device: Huawei P20 EML-L29
Android version: 8.1
NoSuchMethodError: NoSuchMethodError: The method 'toUint8List' was called on null.
Receiver: null
Tried calling: toUint8List()
File "input_file_stream.dart", line 288, in InputFileStream.toUint8List
File "zip_encoder.dart", line 81, in ZipEncoder.getFileCrc32
File "zip_encoder.dart", line 131, in ZipEncoder.addFile
File "zip_file_encoder.dart", line 56, in ZipFileEncoder.addFile
File "zip_file_encoder.dart", line 44, in ZipFileEncoder.addDirectory
File "zip_file_encoder.dart", line 20, in ZipFileEncoder.zipDirectory
I'm trying to use Decode by adding bytes (Deflate.addBytes) as they are generated with FLUSH_PARTIAL and pull bytes off with Deflate.getBytes. When I call Deflate.addBytes then Deflate.getBytes the second time I get the bytes from the first call PLUS the new bytes added to the output buffer. I think, you should either have:
_output.clear()
in Deflate.getBytes (after grabbing the bytes to return), or add some new function e.g.
List<int> takeBytes() {
_flushPending();
List<int> bytes = _output.getBytes();
_output.clear();
return bytes;
}
I used this function and my code works.
FYI: I am not doing the whole file in memory because the data are too large. As I take the output buffer, I write into a temporary File in the browser and create a url for the user to download.
on Mac OS X file.isFile
always false
i used code as example:
import 'dart:io';
import 'package:archive/archive.dart';
import 'package:archive/archive_io.dart';
void main() {
// Read the Zip file from disk.
List<int> bytes = new File('/Users/axzq/Desktop/ĺ˝ćĄŁ.zip').readAsBytesSync();
// Decode the Zip file
Archive archive = new ZipDecoder().decodeBytes(bytes);
// Extract the contents of the Zip archive to disk.
for (ArchiveFile file in archive) {
String filename = file.name;
if(filename.contains('__MACOSX')) {
continue;
}
// file.isFile always false
if (file.isFile) {
List<int> data = file.content;
new File('out/' + filename)
..createSync(recursive: true)
..writeAsBytesSync(data);
} else {
new Directory('out/' + filename)
..create(recursive: true);
}
}
}
I would think that this line:
archive/lib/src/zlib/inflate.dart
Line 202 in 89a489c
if (len != ~nlen)
. The error should be thrown if the second two bytes complement is NOT equal to the first two right? (https://www.w3.org/Graphics/PNG/RFC-1951#noncompressed). Or am I missing something?How can I properly decompress symbolic links contained in a TAR.GZ archive? The FileArchive
API let us check if an entity is a file, but not if it is a symbolic link and its target path.
Is it possible? Did I miss an API?
How to Unzip Password Files
and
How to Compress a Password File
E/flutter (18353): [ERROR:flutter/shell/common/shell.cc(181)] Dart Error: Unhandled exception:
E/flutter (18353): ArchiveException: Could not find End of Central Directory Record
E/flutter (18353): #0 ZipDirectory._findSignature (package:archive/src/zip/zip_directory.dart:145:5)
E/flutter (18353): #1 new ZipDirectory.read (package:archive/src/zip/zip_directory.dart:27:20)
E/flutter (18353): #2 ZipDecoder.decodeBuffer (package:archive/src/zip_decoder.dart:21:21)
E/flutter (18353): #3 ZipDecoder.decodeBytes (package:archive/src/zip_decoder.dart:17:12)
E/flutter (18353): #4 writeCounter (file:///C:/src/projetos/widget_dinamico/lib/main.dart:83:38)
E/flutter (18353):
E/flutter (18353): #5 main (file:///C:/src/projetos/widget_dinamico/lib/main.dart:133:1)
E/flutter (18353):
E/flutter (18353): #6 _startIsolate. (dart:isolate/runtime/libisolate_patch.dart:289:19)
E/flutter (18353): #7 _RawReceivePortImpl._handleMessage (dart:isolate/runtime/libisolate_patch.dart:171:12)
I already do many checks of file type and/or content. I already criate a new ZIP file in computer, transfer to phone and test, and got same error.
My code is:
import 'dart:convert';
import 'dart:async';
import 'package:flutter/material.dart';
import 'package:flutter_webview_plugin/flutter_webview_plugin.dart';
import 'dart:async' show Future;
import 'package:flutter/services.dart' show rootBundle;
import 'package:path_provider/path_provider.dart';
import 'dart:io';
import 'package:flutter_document_picker/flutter_document_picker.dart';
import 'package:archive/archive.dart';
import 'package:archive/archive_io.dart';
const kAndroidUserAgent =
'Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Mobile Safari/537.36';
Future<String> loadAsset() async {
return await rootBundle.loadString('assets/teste3.xhtml');
}
final data = loadAsset();
String selectedUrl;
String extPath;
//recuperando localpath
Future<String> get _localPath async {
final directory = await getApplicationDocumentsDirectory();
print(directory.path);
extPath = directory.path;
return directory.path;
}
Future<File> get _localFile async {
final path = await _localPath;
return File('$path/teste.zip');
}
Future<File> writeCounter(String counter) async {
final path = await FlutterDocumentPicker.openDocument();
final ffile = await _localFile;
File(path).copy(ffile.path);
// Read the Zip file from disk.
List<int> bytes = new File(ffile.path).readAsBytesSync();
// Decode the Zip file
Archive archive = new ZipDecoder().decodeBytes(bytes);
// Extract the contents of the Zip archive to disk.
for (ArchiveFile cfile in archive) {
String filename = cfile.name;
if (cfile.isFile) {
List<int> cdata = cfile.content;
new File('out/' + filename)
..createSync(recursive: true)
..writeAsBytesSync(cdata);
} else {
new Directory('out/' + filename)
..create(recursive: true);
}
}
selectedUrl = extPath + '/out/OEBPS/1102014248.xhtml';
// Write the file
return File(extPath + '/out/OEBPS/1102014248.xhtml'); // file.writeAsString('$counter');
}
void main() async {
// escrevendo no arquivo
writeCounter(await data);
runApp(new MyApp());
}
class MyApp extends StatelessWidget {
final flutterWebviewPlugin = new FlutterWebviewPlugin();
final _codeCtrl =
new TextEditingController(text: '' );
@override
Widget build(BuildContext context) {
return new MaterialApp(
title: 'Flutter WebView Demo',
theme: new ThemeData(
primarySwatch: Colors.blue,
),
routes: {
'/': (_) => const MyHomePage(title: 'Flutter WebView Demo'),
'/widget': (_) => new
WebviewScaffold(
url: selectedUrl,
appBar: new AppBar(
title: const Text('Widget webview'),
actions: <Widget>[
new RaisedButton(
onPressed: () {
// escrevendo no arquivo
//data.then((String rres){writeCounter(rres);});
/* abrindo o epub */
//selectedUrl = "file:///data/user/0/com.example.widgetdinamico/app_flutter/teste.xhtml";
final future =
flutterWebviewPlugin.evalJavascript(_codeCtrl.text);
future.then( (String result) {
//_history.add('eval: $result');
print('eval: $result');
});
},
child: const Text('Color')
)
],
),
clearCache: true,
clearCookies: true,
withZoom: false,
withLocalStorage: true,
)
},
);
}
}
class MyHomePage extends StatefulWidget {
const MyHomePage({Key key, this.title}) : super(key: key);
final String title;
@override
_MyHomePageState createState() => new _MyHomePageState();
}
class _MyHomePageState extends State<MyHomePage> {
// Instance of WebView plugin
final flutterWebviewPlugin = new FlutterWebviewPlugin();
// On destroy stream
StreamSubscription _onDestroy;
// On urlChanged stream
StreamSubscription<String> _onUrlChanged;
// On urlChanged stream
StreamSubscription<WebViewStateChanged> _onStateChanged;
StreamSubscription<WebViewHttpError> _onHttpError;
StreamSubscription<double> _onScrollYChanged;
StreamSubscription<double> _onScrollXChanged;
final _urlCtrl = new TextEditingController(text: selectedUrl);
final _codeCtrl =
new TextEditingController(text: "confirm('teste');" /*'window.navigator.userAgent'*/); /* \$('rb-f').css({"color": "red", "border": "2px solid red"}); */
final _scaffoldKey = new GlobalKey<ScaffoldState>();
final _history = [];
@override
void initState() {
super.initState();
flutterWebviewPlugin.close();
_urlCtrl.addListener(() {
selectedUrl = _urlCtrl.text;
});
// Add a listener to on destroy WebView, so you can make came actions.
_onDestroy = flutterWebviewPlugin.onDestroy.listen((_) {
if (mounted) {
// Actions like show a info toast.
_scaffoldKey.currentState.showSnackBar(
const SnackBar(content: const Text('Webview Destroyed')));
}
});
// Add a listener to on url changed
_onUrlChanged = flutterWebviewPlugin.onUrlChanged.listen((String url) {
print('_onUrlChanged');
if (mounted) {
setState(() {
_history.add('onUrlChanged: $url');
});
}
});
_onScrollYChanged =
flutterWebviewPlugin.onScrollYChanged.listen((double y) {
if (mounted) {
setState(() {
_history.add("Scroll in Y Direction: $y");
});
}
});
_onScrollXChanged =
flutterWebviewPlugin.onScrollXChanged.listen((double x) {
if (mounted) {
setState(() {
_history.add("Scroll in X Direction: $x");
});
}
});
_onStateChanged =
flutterWebviewPlugin.onStateChanged.listen((WebViewStateChanged state) {
print('_onStateChanged');
if (mounted) {
setState(() {
_history.add('onStateChanged: ${state.type} ${state.url}');
});
}
});
_onHttpError =
flutterWebviewPlugin.onHttpError.listen((WebViewHttpError error) {
if (mounted) {
setState(() {
_history.add('onHttpError: ${error.code} ${error.url}');
});
}
});
}
@override
void dispose() {
// Every listener should be canceled, the same should be done with this stream.
_onDestroy.cancel();
_onUrlChanged.cancel();
_onStateChanged.cancel();
_onHttpError.cancel();
_onScrollXChanged.cancel();
_onScrollYChanged.cancel();
flutterWebviewPlugin.dispose();
super.dispose();
}
@override
Widget build(BuildContext context) {
return new Scaffold(
key: _scaffoldKey,
appBar: new AppBar(
title: const Text('Plugin example app'),
),
body: new Column(
mainAxisAlignment: MainAxisAlignment.center,
children: [
new Container(
padding: const EdgeInsets.all(24.0),
child: new TextField(controller: _urlCtrl),
),
new RaisedButton(
onPressed: () {
flutterWebviewPlugin.launch(selectedUrl,
rect: new Rect.fromLTWH(
0.0, 0.0, MediaQuery.of(context).size.width, 300.0),
userAgent: kAndroidUserAgent);
},
child: const Text('Open Webview (rect)'),
),
new RaisedButton(
onPressed: () {
flutterWebviewPlugin.launch(selectedUrl, hidden: true);
},
child: const Text('Open "hidden" Webview'),
),
new RaisedButton(
onPressed: () {
flutterWebviewPlugin.launch(selectedUrl);
},
child: const Text('Open Fullscreen Webview'),
),
new RaisedButton(
onPressed: () {
flutterWebviewPlugin.evalJavascript(_codeCtrl.text);
Navigator.of(context).pushNamed('/widget');
},
child: const Text('Open widget webview'),
),
new Container(
padding: const EdgeInsets.all(24.0),
child: new TextField(controller: _codeCtrl),
),
new RaisedButton(
onPressed: () {
final future =
flutterWebviewPlugin.evalJavascript(_codeCtrl.text);
future.then((String result) {
setState(() {
_history.add('eval: $result');
print('eval: $result');
});
});
},
child: const Text('Eval some javascript'),
),
new RaisedButton(
onPressed: () {
setState(() {
_history.clear();
});
flutterWebviewPlugin.close();
},
child: const Text('Close'),
),
new RaisedButton(
onPressed: () {
flutterWebviewPlugin.getCookies().then((m) {
setState(() {
_history.add('cookies: $m');
});
});
},
child: const Text('Cookies'),
),
new Text(_history.join('\n'))
],
),
);
}
}
Large archive files take a lot of memory when the buffer needs to be built in memory. Please add support to read and write using files, without the need to build up the buffer in memory.
I tried to find a way to decompress a zip file and get a stream of decompressed content for a file which I can then pipe to an IoSink.
My expectation is that this way not the entire file needs to be decompressed entirely in RAM before it can be written to disk.
Is this supported?
One option could be:
....
file.crc32 = zf.crc32;
file.compress = compress;
file.lastModTime = zf.lastModFileDate << 16 | zf.lastModFileTime;
then decode this way to have a proper Dart DateTime:
DateTime getZipDate( int dosTime ) {
int year = ((dosTime >> 25) & 0x7f) + 1980;
int month = ((dosTime >> 21) & 0x0f); // - 1;
int day = (dosTime >> 16) & 0x1f;
int hours = (dosTime >> 11) & 0x1f;
int minutes= (dosTime >> 5) & 0x3f;
int seconds = (dosTime << 1) & 0x3e;
var date = DateTime( year, month, day, hours, minutes, seconds).toUtc();
return date;
}
This way works for me...
Hello,
Thank you for all the work here. I have been using archive library in my dart application for a while now. However there are some cases where I am required to use 7z encoder and decoder. Since there seems to be no support at this time, I am relying on the local system's 7z.exe to do the job. It would be really great to have a support for 7z encoder and decoder within archive library and I believe it will definitely add to the archive library's strength.
Any suggestions/workaround for this support are highly appreciated. Thanks in advance.
Can you publish one more version, to capture the unused symbols PR? Much appreciated!
ZipFileEncoder.addDirectory
only adds files within the directory and ignores the sub-directories. Need support to be able to archive full directory structure. please provide an example if this is possible.
Thanks in advance đ
Archive archive = new Archive();
List bdata = [1,2,3,4,5,6,7,8,9];
ByteData bytes = new ByteData.view(new Uint8List.fromList(bdata).buffer);
String name = 'abc.txt';
ArchiveFile afile = new ArchiveFile(name, bytes.lengthInBytes, new InputStream(bytes), 8);
archive.addFile(afile);
List zip_data = new ZipEncoder().encode(archive);
I got an error in the last line. If I change the compression method from 8 to 0 in the line
ArchiveFile afile = new ArchiveFile(name, bytes.lengthInBytes, new InputStream(bytes), 8);
everything is OK.
We should analyze and test this package on a CI system; people often use travis for Dart packages. I'm happy to help set this up if you'd like.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
đ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. đđđ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google â¤ď¸ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.