Whiteouts in layers about moby HOT 5 CLOSED

There might be (at least) one scenario where you want to have whiteouts: if you ship a layer including some sample code, you might want to remove it (or overwrite it with something else) when you derive the layer.

On the current platform, we don't use whiteouts in that case, because this data is on a separate volume (therefore not requiring a whiteout).

There are also special cases of whiteouts: permission changes. Those can be useful when you want to give access to some files, or remove access. This was a frequent problem on the current platform (e.g. readability of logs when daemons are installed using system packages instead of the default user).

There is no convention in AUFS—just an opaque format. I'd personally advocate against making that part of the API, since it effectively ties the implementation with AUFS, which is probably a Very Bad Thing, but YMMV.

Last but not least, in some cases (mail servers), you might also want to preserve inode numbers—and in that case, you would have to also preserve the xino translation file of AUFS.

... Unless we want to support bind-mounted volumes; then those concerns are waived.

from moby.

On Sun, Jan 20, 2013 at 10:57 PM, jpetazzo [email protected] wrote:

There might be (at least) one scenario where you want to have whiteouts:
if you ship a layer including some sample code, you might want to remove it
(or overwrite it with something else) when you derive the layer.

Could you walk me through an example use case in more detail?

There are also special cases of whiteouts: permission changes. Those can
be useful when you want to give access to some files, or remove access.
This was a frequent problem on the current platform (e.g. readability of
logs when daemons are installed using system packages instead of the
default user).

Same question as above.

Last but not least, in some cases (mail servers), you might also want to
preserve inode numbers—and in that case, you would have to also preserve
the xino translation file of AUFS.

Same question as above.

Thanks!

from moby.

On Sun, Jan 20, 2013 at 10:57 PM, jpetazzo [email protected] wrote:

There is no convention in AUFS—just an opaque format. I'd personally
advocate against making that part of the API, since it effectively ties the
implementation with AUFS, which is probably a Very Bad Thing, but YMMV.

We are in agreement here. Frankly, sticking to vanilla tarballs as the
image format is something I would like to preserve if at all possible. I know
Andrea feels the same way.

from moby.

1. Whiteouts Needed To Remove Sample Code.
   The current implementation of dotCloud service ships with some sample code. It is never seen by dotCloud users, since their code replaces it immediately after container deployment (before the container gets a chance to see traffic) — unless code deployment fails; in which case the container is rolled back to its previous version, which happens to be the pristine version of the cloudlet. Of course, we could argue that this sample code could be removed in the first place from the base layer; but it's here for a good reason: to help the service developer to ensure that his service actually works.
2. Whiteouts Needed To Preserve Permission Changes.
   For a very long time, dotCloud Java service has been plagued by the fact that its logs were readable only by the jetty user, not by the dotCloud user. The files in base layer didn't have to be modified, but their permissions did.
3. Preserving Inode Numbers.
   http://grox.net/doc/postfix/html/faq.html#copying
   "Postfix names a queue file after its inode number and after the microsecond part of the time of day. Thus, if a queue file has a name based on someone elses inode number there is a small chance that the file name will collide with another queue file."

from moby.

Ok, after a few off-band discussions: the consensus seems to be that whiteouts and layers are a (very) useful tool, but should not be exposed as a top-level object to the end-user. The main reason being that it would make things unnecessarily complicated (every user needs to acquire the off-band knowledge of which layer works with which, and it is very easy to shoot yourself in the foot by combining layers that were not meant to be combined). At the same time, there is not much benefit to allowing arbitrary combination of layers: even if you are 99% sure your layer can be safely "rebased" to a new base layer, why not run that build script again just to make sure you're right?

So, instead, let's use layers (+ their associated whiteouts) as a optimization for storing and transferring images. In other words, the payload for docker is always a tarball. But it may optionally be a sparse tarball, which will reference the bottom layers by ID, and only store changes. Of course these sparse tarballs will only be usable to a receiver capable of (a) decoding the sparse format and (b) retrieving the bottom layers from their ID.

from moby.