Skip to content

Commit be28041

Browse files
authored
Use pzstd to compress the Docker in SWI slim images (#22005)
<!-- Please make sure you've read and understood our contributing guidelines: https://github.com/Azure/SONiC/blob/gh-pages/CONTRIBUTING.md failure_prs.log skip_prs.log Make sure all your commits include a signature generated with `git commit -s` ** If this is a bug fix, make sure your description includes "fixes #xxxx", or "closes #xxxx" or "resolves #xxxx" Please provide the following information: --> #### Why I did it Same with #21824 Enabled the docker inram feature for slim image. It would extract the docker image to ram during the boot, so it would take extra times during boot. ##### Work item tracking - Microsoft ADO **(number only)**: 31323281 #### How I did it Use pzstd which is more efficient tool to compress and decompress the docker file to reduce the boot time. Currently, we do not modify the "FILESYSTEM_DOCKERFS=dockerfs.tar.gz" in onie-image.conf, so for slim image, we still use dockerfs.tar.gz as file name but actually with zstd compressed. We tried to find out a way to adjust the file name in onie-image.conf, but it seems not easy to do that. So we use the file cmd to determine the compressing type in union-mount.j2, then use the related cmd to extract the docker file. Plan to support zstd for all types of images in the future to unify the docker file image . #### How to verify it Boot swi slim image and normal image, boot mellanox bin image, all boot successfully. <!-- If PR needs to be backported, then the PR must be tested against the base branch and the earliest backport release branch and provide tested image version on these two branches. For example, if the PR is requested for master, 202211 and 202012, then the requester needs to provide test results on master and 202012. --> #### Which release branch to backport (provide reason below if selected) <!-- - Note we only backport fixes to a release branch, *not* features! - Please also provide a reason for the backporting below. - e.g. - [x] 202006 --> - [ ] 201811 - [ ] 201911 - [ ] 202006 - [ ] 202012 - [ ] 202106 - [ ] 202111 - [ ] 202205 - [ ] 202211 - [ ] 202305 #### Tested branch (Please provide the tested image version) <!-- - Please provide tested image version - e.g. - [x] 20201231.100 --> - [ ] <!-- image version 1 --> - [ ] <!-- image version 2 --> #### Description for the changelog <!-- Write a short (one line) summary that describes the changes in this pull request for inclusion in the changelog: --> <!-- Ensure to add label/tag for the feature raised. example - PR#2174 under sonic-utilities repo. where, Generic Config and Update feature has been labelled as GCU. --> #### Link to config_db schema for YANG module changes <!-- Provide a link to config_db schema for the table for which YANG model is defined Link should point to correct section on https://github.com/Azure/sonic-buildimage/blob/master/src/sonic-yang-models/doc/Configuration.md --> #### A picture of a cute animal (not mandatory but encouraged)
1 parent c371cd3 commit be28041

File tree

5 files changed

+58
-4
lines changed

5 files changed

+58
-4
lines changed

build_debian.sh

+11-1
Original file line numberDiff line numberDiff line change
@@ -391,6 +391,12 @@ sudo LANG=C DEBIAN_FRONTEND=noninteractive chroot $FILESYSTEM_ROOT apt-get -y in
391391
zstd \
392392
nvme-cli
393393

394+
sudo cp files/initramfs-tools/pzstd $FILESYSTEM_ROOT/etc/initramfs-tools/hooks/pzstd
395+
sudo chmod +x $FILESYSTEM_ROOT/etc/initramfs-tools/hooks/pzstd
396+
397+
sudo cp files/initramfs-tools/file $FILESYSTEM_ROOT/etc/initramfs-tools/hooks/file
398+
sudo chmod +x $FILESYSTEM_ROOT/etc/initramfs-tools/hooks/file
399+
394400
# Have systemd create the auditd log directory
395401
sudo mkdir -p ${FILESYSTEM_ROOT}/etc/systemd/system/auditd.service.d
396402
sudo tee ${FILESYSTEM_ROOT}/etc/systemd/system/auditd.service.d/log-directory.conf >/dev/null <<EOF
@@ -851,7 +857,11 @@ if [[ $MULTIARCH_QEMU_ENVIRON == y || $CROSS_BUILD_ENVIRON == y ]]; then
851857
fi
852858

853859
## Compress docker files
854-
pushd $FILESYSTEM_ROOT && sudo tar -I pigz -cf $OLDPWD/$FILESYSTEM_DOCKERFS -C ${DOCKERFS_PATH}var/lib/docker .; popd
860+
if [ "$BUILD_REDUCE_IMAGE_SIZE" = "y" ]; then
861+
pushd $FILESYSTEM_ROOT && sudo tar -I pzstd -cf $OLDPWD/$FILESYSTEM_DOCKERFS -C ${DOCKERFS_PATH}var/lib/docker .; popd
862+
else
863+
pushd $FILESYSTEM_ROOT && sudo tar -I pigz -cf $OLDPWD/$FILESYSTEM_DOCKERFS -C ${DOCKERFS_PATH}var/lib/docker .; popd
864+
fi
855865

856866
## Compress together with /boot, /var/lib/docker and $PLATFORM_DIR as an installer payload zip file
857867
pushd $FILESYSTEM_ROOT && sudo tar -I pigz -cf platform.tar.gz -C $PLATFORM_DIR . && sudo zip -n .gz $OLDPWD/$INSTALLER_PAYLOAD -r boot/ platform.tar.gz; popd

files/dsc/install_debian.j2

+4
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,11 @@ image_dir=image-$image_version
1515

1616
INSTALLER_PAYLOAD=fs.zip
1717
DOCKERFS_DIR=docker
18+
{% if BUILD_REDUCE_IMAGE_SIZE == "y" -%}
19+
FILESYSTEM_DOCKERFS=dockerfs.tar.zstd
20+
{%- else -%}
1821
FILESYSTEM_DOCKERFS=dockerfs.tar.gz
22+
{%- endif %}
1923
BL_CONF=boot.conf
2024

2125
DATA_PARTUUID=6ED62003-DD8D-44B8-9538-0A2B7C7E628F

files/initramfs-tools/file

+18
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
#!/bin/sh
2+
set -e
3+
PREREQ=""
4+
prereqs() {
5+
echo "$PREREQ"
6+
}
7+
case "$1" in
8+
prereqs)
9+
prereqs
10+
exit 0
11+
;;
12+
esac
13+
. /usr/share/initramfs-tools/hook-functions
14+
# Include file binary
15+
copy_exec /usr/bin/file /usr/bin
16+
# Include magic database
17+
copy_exec /usr/lib/file/magic.mgc /etc
18+
exit 0

files/initramfs-tools/pzstd

+16
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
#!/bin/sh
2+
set -e
3+
PREREQ=""
4+
prereqs() {
5+
echo "$PREREQ"
6+
}
7+
case "$1" in
8+
prereqs)
9+
prereqs
10+
exit 0
11+
;;
12+
esac
13+
. /usr/share/initramfs-tools/hook-functions
14+
# Include pzstd binary
15+
copy_exec /usr/bin/pzstd /usr/bin
16+
exit 0

files/initramfs-tools/union-mount.j2

+9-3
Original file line numberDiff line numberDiff line change
@@ -130,9 +130,15 @@ extract_dockerfs()
130130
{
131131
echo "Extracting {{ FILESYSTEM_DOCKERFS }}"
132132
if [ -f "${rootmnt}/host/$image_dir/{{ FILESYSTEM_DOCKERFS }}" ] && [ "$secureboot" = false ]; then
133-
# Extract dockerfs.tar.gz into /var/lib/docker unless the system booted with secureboot
134-
# In secureboot dockerfs.tar.gz cannot be trusted as it does not have a signature
135-
tar xz --numeric-owner -f ${rootmnt}/host/$image_dir/{{ FILESYSTEM_DOCKERFS }} -C ${rootmnt}/var/lib/docker
133+
# Check if the file is zstd compressed
134+
file_type=$(file -b --mime-type "${rootmnt}/host/$image_dir/{{ FILESYSTEM_DOCKERFS }}")
135+
if [ "$file_type" = "application/zstd" ]; then
136+
echo "Detected zstd compression, extracting with pzstd..."
137+
pzstd -d -q ${rootmnt}/host/$image_dir/{{ FILESYSTEM_DOCKERFS }} -c | tar x --numeric-owner -C ${rootmnt}/var/lib/docker
138+
else
139+
echo "Using default extraction method (gzip assumed)..."
140+
tar xz --numeric-owner -f "${rootmnt}/host/$image_dir/{{ FILESYSTEM_DOCKERFS }}" -C "${rootmnt}/var/lib/docker"
141+
fi
136142
elif [ "$bootloader" = "aboot" ] && unzip -l "$swi_path" | grep -q {{ FILESYSTEM_DOCKERFS }}; then
137143
# Aboot swi images also support extracting dockerfs.tar.gz directly from them
138144
unzip -qp "$swi_path" {{ FILESYSTEM_DOCKERFS }} | tar xz --numeric-owner -C ${rootmnt}/var/lib/docker

0 commit comments

Comments
 (0)