codecavepro / openmcdf Goto Github PK
View Code? Open in Web Editor NEWStructured Storage .net component - pure C#
License: Mozilla Public License 2.0
Structured Storage .net component - pure C#
License: Mozilla Public License 2.0
Hi
I tried to unpack a large msdf file (about 500 Mb) and got large memory consumption (about 6-8 Gb).
I think what this because a new byte[]
operation take place many times.
Can you made some changes like this:
Sector.cs - add function
public void GetData(byte[] buffer, int offset, int length)
{
if (IsStreamed)
{
stream.Seek((long)size + (long)this.id * (long)size, SeekOrigin.Begin);
stream.Read(buffer, offset, Math.Min(size, length));
}
}
And use it at the StreamView.cs (function int Read(byte[] buffer, int offset, int count)
) instead of sectorChain[secIndex].GetData()
function
Thanks
I got an ArgumentOutOfRangeException when run this code
static void Main(string[] args)
{
var file = new CompoundFile(@"cf_test6.bin");
var cfStream = file.RootStorage.TryGetStorage("Layer2").TryGetStream("ids");
// ArgumentOutOfRangeException
var r = new StreamReader(cfStream.AsIOStream());
// If replace by this code line, no exception
//var r = new StreamReader(new MemoryStream(cfStream.GetData()));
int count = 0;
while (r.ReadLine() != null)
{
count++;
}
Console.WriteLine(count);
Console.ReadLine();
}
I found that the method CompoundFile.LoadDirectories
fail to handle the case where the starting sector of a directory entry is invalid.
With the sample file FTC07.zip, the starting sector of the directory entry #42 is equal to 6553868 which is invalid. Adding the below line to Line 685, we can check directoryEntries[42].StartSetc
Console.WriteLine(directoryEntries[42].StartSetc); // 6553868
We should check de.StartSetc
in the method CompoundFile.LoadDirectories
. If de.StartSetc
is greater than this.sectors.Count
, we should raise an exception that the file is invalid.
private void LoadDirectories()
{
List<Sector> directoryChain
= GetSectorChain(header.FirstDirectorySectorID, SectorType.Normal);
if (header.FirstDirectorySectorID == Sector.ENDOFCHAIN)
header.FirstDirectorySectorID = directoryChain[0].Id;
StreamView dirReader
= new StreamView(directoryChain, GetSectorSize(), directoryChain.Count * GetSectorSize(), sourceStream);
while (dirReader.Position < directoryChain.Count * GetSectorSize())
{
IDirectoryEntry de
= DirectoryEntry.New(String.Empty, StgType.StgInvalid, directoryEntries);
//We are not inserting dirs. Do not use 'InsertNewDirectoryEntry'
de.Read(dirReader);
// We should check de.StartSetc here
if(de.StartSetc > sectors.Count)
{
throw new CFException("Compound File is invalid");
}
}
}
Sorry for my bad description about the issue.
Best regards,
Nhut M. Ngo
I can't create a compound file more than 268434944 bytes. I got StackOverflow exception when try to do this.
SectorCollection.cs has a line
private const int MAX_SECTOR_V4_COUNT_LOCK_RANGE = 524287; //0x7FFFFF00 for Version 4
This is limited max file size to 524287*512 = 268434944 bytes.
What is wrong?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.