mirage / ocaml-vhd Goto Github PK

View Code? Open in Web Editor NEW

9.0 7.0 20.0 912 KB

Read and write .vhd format data

License: Other

Makefile 0.16% C 3.33% OCaml 96.51%

ocaml-vhd's People

Contributors

Stargazers

Watchers

Forkers

xapi-project simonjbeaumont euanh bobball talex5 lesserevil avsm yomimono robhoes minli1 edwintorok craigfe msgpo kit-ty-kate alessandro-barbieri dra27

ocaml-vhd's Issues

replace O_DIRECT by fadvise()

Hi all,
I am working on ZFS SR support, and I am constantly butting on O_DIRECT which is not supported by ZFS (the symptom is EINVAL on open()).

This time, it's XAPI calling vhd-tool with --direct hard-coded

And through vhd-tool it comes here. I suspect the intent of --direct is to avoid filling the disk cache in this instance. Both operations are simple in-order complete read and in-order complete write.

I think those cases could be covered by posix_fadvise() and fsync().

I am ready to create a patch, but I would like to know where you think I should make the change before I get to work.

Thanks,
Nicolas.

Cannot build on Fedora 21

The spec file in buildroot needs to be updated to include "blktap" as a dependency, since that provides vhd/libvhd.h, but even with that change ocaml-vhd cannot build as the search path does not include %{libdir}/blktap/ which is where buildroot deposits the code. This 'works' on other distributions as xen-devel includes a version of vhd/libvhd.h that seems to be 'good enough'.

It seems that buildroot depends on xen-c6 not just for an updated Xen, but for the headers being re-added to the Xen RPM:

# yum provides */libvhd.h
xen-devel-4.2.2-23.el6.centos.alt.x86_64 : Development libraries for Xen tools
Repo        : xen-c6
Matched from:
Filename    : /usr/include/vhd/libvhd.h

+ ocaml setup.ml -configure
Configuration: 
ocamlfind: ........................................... /usr/bin/ocamlfind
ocamlc: .............................................. /usr/bin/ocamlc.opt
ocamlopt: ............................................ /usr/bin/ocamlopt.opt
ocamlbuild: .......................................... /usr/bin/ocamlbuild
Package name: ........................................ vhdlib
Package version: ..................................... 0.9.1
os_type: ............................................. Unix
system: .............................................. linux
architecture: ........................................ amd64
ccomp_type: .......................................... cc
ocaml_version: ....................................... 4.01.1
standard_library_default: ............................ /usr/lib64/ocaml
standard_library: .................................... /usr/lib64/ocaml
standard_runtime: .................................... /usr/bin/ocamlrun
bytecomp_c_compiler: ................................. gcc -fno-defer-pop -Wall -D_FILE_OFFSET_BITS=64 -D_REENTRANT -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protec
tor-strong --param=ssp-buffer-size=4 -grecord-gcc-switches  -m64 -mtune=generic -fPIC
native_c_compiler: ................................... gcc -Wall -D_FILE_OFFSET_BITS=64 -D_REENTRANT -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --pa
ram=ssp-buffer-size=4 -grecord-gcc-switches  -m64 -mtune=generic
model: ............................................... default
ext_obj: ............................................. .o
ext_asm: ............................................. .s
ext_lib: ............................................. .a
ext_dll: ............................................. .so
default_executable_name: ............................. a.out
systhread_supported: ................................. true
Install architecture-independent files dir: .......... /usr/local
Install architecture-dependent files in dir: ......... $prefix
User executables: .................................... $exec_prefix/bin
System admin executables: ............................ $exec_prefix/sbin
Program executables: ................................. $exec_prefix/libexec
Read-only single-machine data: ....................... $prefix/etc
Modifiable architecture-independent data: ............ $prefix/com
Modifiable single-machine data: ...................... $prefix/var
Object code libraries: ............................... $exec_prefix/lib
Read-only arch-independent data root: ................ $prefix/share
Read-only architecture-independent data: ............. $datarootdir
Info documentation: .................................. $datarootdir/info
Locale-dependent data: ............................... $datarootdir/locale
Man documentation: ................................... $datarootdir/man
Documentation root: .................................. $datarootdir/doc/$pkg_name
HTML documentation: .................................. $docdir
DVI documentation: ................................... $docdir
PDF documentation: ................................... $docdir
PS documentation: .................................... $docdir
findlib_version: ..................................... 1.4
is_native: ........................................... true
suffix_program: ...................................... 
Remove a file.: ...................................... rm -f
Remove a directory.: ................................. rm -rf
Turn ocaml debug flag on: ............................ true
Turn ocaml profile flag on: .......................... false
Compiler support generation of .cmxs.: ............... true
OCamlbuild additional flags: ......................... 
Create documentations: ............................... true
Compile tests executable and library and run them: ... false
ocaml setup.ml -build 
+ make
/usr/bin/ocamlopt.opt unix.cmxa -I /usr/lib64/ocaml/ocamlbuild /usr/lib64/ocaml/ocamlbuild/ocamlbuildlib.cmxa myocamlbuild.ml /usr/lib64/ocaml/ocamlbuild/ocamlbuild.cmx -o myocamlbuild
ocamlfind ocamlc -c lib/vhd_stubs.c
+ ocamlfind ocamlc -c lib/vhd_stubs.c
lib/vhd_stubs.c:11:24: fatal error: vhd/libvhd.h: No such file or directory
 #include <vhd/libvhd.h>
                        ^
compilation terminated.
Command exited with code 2.
E: Failure("Command ''/usr/bin/ocamlbuild' lib/libvhdlib_stubs.a lib/dllvhdlib_stubs.so lib/vhdlib.cma lib/vhdlib.cmxa lib/vhdlib.a lib/vhdlib.cmxs lib_test/open.byte -tag debug' terminated with error code 10")
Makefile:5: recipe for target 'build' failed

cli: add a "--skip" argument

In cases where we have partially transferred a file, we could avoid re-uploading the same data twice by supplying "--skip ".

This would also be useful when performing a download using a byte-range request?

"Unsupported footer cookie" error for Hyper-V VHDs

I'm importing a bunch of VHDs from Hyper-V on Windows 2012 to XenServer 6.5 SP1, which uses this library to stream the data over http. After reaching 100%, the import operation fails with the following error:

(/opt/xensource/libexec/vhd-tool serve --source-format vhd --source-protocol n...) exited with code 1
vhd-tool failed, returning VDI_IO_ERROR
vhd-tool output: vhd-tool: Unsupported footer cookie: expected conectix, got

This seems to be due to some slight difference in the VHD file formats. For example, I can import a VHD file that was exported from VirtualBox without any errors. If I then run Convert-VHD src.vhd dst.vhd in PowerShell on Windows 2012, which doesn't really do anything since the source and destination formats are the same, import of dst.vhd fails with the cookie error. The same error happens for any VHD file that was created by Hyper-V.

I looked at the files in a hex editor and there is absolutely nothing wrong with the header or the footer. The conectix cookie is where it should be and only the UUID and checksum fields differ. Could it be a problem with the way this library parses VHD files? Since it's extracting an empty string for the cookie, my only guess is that it's looking at the wrong offset.

Don't use string comparison over filenames

When computing the blocks I should copy across in an incremental copy, I see this:

Including blocks in /var/run/sr-mount/3dc60352-df3e-1b5b-ef2f-4d4b43980f33/././11269300-4efd-4229-8c2f-63b44bd20480.vhd
Including blocks in /var/run/sr-mount/3dc60352-df3e-1b5b-ef2f-4d4b43980f33/././6e5f5c37-3103-4713-9693-9880c38e7afa.vhd
Including blocks in /var/run/sr-mount/3dc60352-df3e-1b5b-ef2f-4d4b43980f33/./11269300-4efd-4229-8c2f-63b44bd20480.vhd
Including blocks in /var/run/sr-mount/3dc60352-df3e-1b5b-ef2f-4d4b43980f33/./985000a9-47e1-4f27-bd8d-82b2ec30c6d3.vhd
Including blocks in /var/run/sr-mount/3dc60352-df3e-1b5b-ef2f-4d4b43980f33/466fc1cb-89a7-4344-9d6e-4ac3dae71e97.vhd
Including blocks in /var/run/sr-mount/3dc60352-df3e-1b5b-ef2f-4d4b43980f33/91afebd0-1d90-47b3-9d27-6dc54bfa8c01.vhd

Clearly several of these strings refer to the same file.

String escape lost in recent commit 7d96f3d

The recent commit 7d96f3d makes the code safe-string compliant but most likely introduced a bug:

@@ -57,9 +57,9 @@ let print_ocaml out t =
   Printf.fprintf out "  let t = ref Disk.empty in\n";
   Int64Map.iter
     (fun ofs cstr ->
-      let buf = String.make (Cstruct.len cstr) '\000' in
-      Cstruct.blit_to_string cstr 0 buf 0 (Cstruct.len cstr);
-      Printf.fprintf out "  t := Disk.write_string !t %LdL \"%s\";\n" ofs (String.escaped buf)
+      let buf = Bytes.make (Cstruct.len cstr) '\000' in
+      Cstruct.blit_to_bytes cstr 0 buf 0 (Cstruct.len cstr);
+      Printf.fprintf out "  t := Disk.write_string !t %LdL \"%s\";\n" ofs (buf |> Bytes.to_string)
     ) t;
   Printf.fprintf out "  !t\n"

In the new code String.escaped is missing.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.