Add support for external blobs #22
Conversation
src/lib.rs
Outdated
| /// This is useful when `blobs/sha256` might contain symlinks pointing outside the oci | ||
| /// directory, e.g. when sharing blobs across OCI repositories. The LXC OCI template uses this | ||
| /// feature. | ||
| pub fn open_with_external_blobs(dir: &Dir, blobs_dir: Dir) -> Result<Self> { |
There was a problem hiding this comment.
I'm ok with this as is to be clear but it definitely feels inconsistent to have one take a ref and the other an owned value.
With just a small refactoring we could have both take owned values instead right?
There was a problem hiding this comment.
Or we take refs and do the try_clone() for both, if I had to make a call I'd probably say let's do that, but not strongly held.
There was a problem hiding this comment.
I actually prefer to take owned values for both. It would remove the try_clone, but what I really would like to avoid is sharing the underlying file handle, where you could seek through a reference, read through the other and observe the side effect of the seek. I know I've linked the example for files, but I assume there would be similar semantics for directories. I would like to entirely avoid having to deal with such behavior.
There was a problem hiding this comment.
I guess one concern with this approach is that it's a breaking change.
There was a problem hiding this comment.
I guess one concern with this approach is that it's a breaking change.
Yes. Now let's try to roll up a few other cleanups at least to make the semver bump worth it.
Oh, I see, it's because you can pass whatever blobs directory you want, it doesn't have to go through a symlink. Took me a while to understand. |
This feature is especially useful when we want to share blobs across OCI repositories. See the `--dest-shared-blob-dir` option in skopeo [1] as well as the LXC OCI template [2] (specifically OCI_USE_CACHE). Usually this is done via symlinking oci/blobs or oci/blobs/sha256, but ocidir-rs doesn't follow symlinks which lead outside the oci directory (this is by design). This commit doesn't change the capability model, it only addresses the blob sharing use case by adding a new open_with_external_blobs function which takes the blobs directory as a separate cap_std::Dir. The library won't allow any accesses outside the blobs directory, but instead of reading the blobs directory from the OCI directory, it will directly read the user supplied blobs directory. closes containers#21 [1] https://github.com/containers/skopeo/blob/main/docs/skopeo-copy.1.md [2] https://github.com/lxc/lxc/blob/main/templates/lxc-oci.in Signed-off-by: Ariel Miculas-Trif <amiculas@cisco.com>
They now both take owned values of type cap_std::Dir for both parameters instead of references to cap_std::Dir. The reason for owned values instead of references is avoiding the undesired effects of sharing the same underlying file handle [1]. [1] https://doc.rust-lang.org/nightly/std/fs/struct.File.html#method.try_clone Signed-off-by: Ariel Miculas-Trif <amiculas@cisco.com>
ariel-miculas
force-pushed
the
support-symlinks-for-blob
branch
from
44088b6 to
0a88546
Compare
|
@cgwalters any objections to the latest changes? If not, I'll go ahead and merge this. |
ariel-miculas
changed the title
…I repos Use open_with_external_blobs which adds support for external blobs, added in ocidir-rs [1]. We'll use the latest version of ocidir-rs from github and we'll switch to version 0.4.0 when it gets released. Closes project-machine#131 [1] containers/ocidir-rs#22 Signed-off-by: Ariel Miculas-Trif <amiculas@cisco.com>
This feature is especially useful when we want to share blobs across OCI repositories. See the
--dest-shared-blob-diroption in skopeo [1] as well as the LXC OCI template [2] (specifically OCI_USE_CACHE).closes #21
[1] https://github.com/containers/skopeo/blob/main/docs/skopeo-copy.1.md
[2] https://github.com/lxc/lxc/blob/main/templates/lxc-oci.in