Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improving build time and size of cilium-agent rock #7

Merged
merged 2 commits into from
Apr 3, 2024
Merged

Improving build time and size of cilium-agent rock #7

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
5c05749
Changed bpftool mirror, pull lvm-10 from upstream instead of building…
berkayoz Mar 28, 2024
00b2cf4
Move bazel cache clean to after build
berkayoz Mar 28, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 49 additions & 39 deletions cilium/rockcraft.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,17 @@ package-repositories:
suites: ["llvm-toolchain-jammy-15"]
components: ["main"]
key-id: 6084F3CF814B57C1CF12EFD515CF4D18AF4F7421
- type: apt
url: http://apt.llvm.org/focal/
suites: ["llvm-toolchain-focal-10"]
components: ["main"]
key-id: 6084F3CF814B57C1CF12EFD515CF4D18AF4F7421

parts:
bazelisk:
plugin: nil
build-packages:
- wget
overlay-script: |
wget https://github.com/bazelbuild/bazelisk/releases/download/v1.19.0/bazelisk-linux-$CRAFT_ARCH_BUILD_FOR
mv bazelisk-linux-$CRAFT_ARCH_BUILD_FOR /usr/bin/bazelisk
Expand All @@ -41,6 +48,7 @@ parts:
source-type: git
source: https://github.com/cilium/proxy.git
source-tag: v1.27
source-depth: 1
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are all around the PR so we don't fetch the entire history of a git repo, saves us some disk space and clone time.

build-packages:
- autoconf
- automake
Expand Down Expand Up @@ -81,6 +89,7 @@ parts:
git rev-parse HEAD >SOURCE_VERSION
make bazel-bin/cilium-envoy
make install
rm -rf /root/.cache/bazel

builder-img-deps:
plugin: nil
Expand Down Expand Up @@ -112,6 +121,7 @@ parts:
source-type: git
source: https://github.com/cilium/cilium.git
source-tag: v1.15.2
source-depth: 1
source-subdir: images/builder
build-environment:
- CGO_ENABLED: 0
Expand All @@ -125,6 +135,7 @@ parts:
source-type: git
source: https://github.com/protocolbuffers/protobuf.git
source-tag: v24.0
source-depth: 1
source-submodules:
- third_party/googletest
- third_party/abseil-cpp
Expand Down Expand Up @@ -215,6 +226,7 @@ parts:
source-type: git
source: https://github.com/kubernetes-sigs/iptables-wrappers.git
source-commit: "e139a115350974aac8a82ec4b815d2845f86997e"
source-depth: 1
override-build: |
mv /usr/sbin /usr/sbin-tmp
ln -sf $CRAFT_STAGE/usr/sbin /usr/sbin
Expand All @@ -227,54 +239,36 @@ parts:
bpftool:
plugin: make
source-type: git
source: https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git
source-commit: "93270357daa949e4bed375b40d0a100ce04f3399"
source: https://github.com/libbpf/bpftool.git
source-tag: v7.0.0
Comment on lines +242 to +243
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We used to pull bpf-next with a specific source commit that pointed to v7.0.0 version of bpftool. The issue is bpf-next is huge and pulls a lot of kernel code just to build simple bpftool, the git server being slow there also does not help.

So instead we are switching to use the official bpftool mirror that only syncs bpftool related header files and code, which is way smaller and builds way faster.

This saves ~8GiB of disk space.

source-depth: 1
source-subdir: "tools/bpf/bpftool"
source-subdir: src
source-submodules:
- "libbpf"
build-packages:
- zlib1g-dev
- libelf-dev
stage-packages:
- libelf1
organize:
usr/local/sbin/bpftool: usr/sbin/bpftool

llvm-10-bpf:
plugin: cmake
source-type: git
source: https://github.com/llvm/llvm-project.git
source-branch: "llvmorg-10.0.0"
source-subdir: "llvm"
cmake-generator: Ninja
override-pull: |
craftctl default
git cherry-pick 29bc5dd19407c4d7cad1c059dea26ee216ddc7ca
git cherry-pick 13f6c81c5d9a7a34a684363bcaad8eb7c65356fd
git cherry-pick ea72b0319d7b0f0c2fcf41d121afa5d031b319d5
override-build: |
cmake $CRAFT_PART_SRC_WORK -G "Ninja" \
-DLLVM_TARGETS_TO_BUILD="BPF" \
-DLLVM_ENABLE_PROJECTS="clang" \
-DBUILD_SHARED_LIBS="OFF" \
-DCMAKE_BUILD_TYPE="Release" \
-DLLVM_BUILD_RUNTIME="OFF" \
-DCMAKE_INSTALL_PREFIX="/usr/local"

ninja clang llc llvm-objcopy

cp -r $CRAFT_PART_BUILD/bin $CRAFT_PART_INSTALL
stage:
- -bin
organize:
bin/clang-10: usr/bin/clang
bin/llc: usr/bin/llc
bin/llvm-objcopy: usr/bin/llvm-objcopy
Comment on lines -241 to -270
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are dropping llvm-10-bpf part entirely. This was just statically built llvm tools with only the bpf target turned on. The cherry picked commits are there to enable the bpf targets / specific to that. Cilium also states that technically any llvm+clang >= 10 with bpf target turned on is good, check here for reference. So instead of pulling the whole llvm-10 source and building it, we pull it and stage it from the apt repository.

- libiberty-dev
- llvm-15
- clang-15
- clang-tools-15
- lldb-15
- lld-15
- clang-format-15
- libc++-15-dev
- libc++abi-15-dev
build-environment:
- EXTRA_CFLAGS: --static
- LLVM_CONFIG: "/usr/bin/llvm-config-15"
- LLVM_STRIP: "/usr/bin/llvm-strip-15"
Comment on lines +251 to +263
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are related to bpftool mirror's build requirements. We build it statically with LLVM+clang 15.


gops:
after: [build-deps]
plugin: go
source-type: git
source: https://github.com/google/gops.git
source-tag: v0.3.27
source-depth: 1
build-environment:
- CGO_ENABLED: 0
override-build: |
Expand All @@ -286,6 +280,7 @@ parts:
source-type: git
source: https://github.com/containernetworking/plugins.git
source-tag: v1.4.0
source-depth: 1
override-build: |
./build_linux.sh
cp -r $CRAFT_PART_BUILD/bin $CRAFT_PART_INSTALL
Expand All @@ -300,17 +295,25 @@ parts:
source-type: git
source: "https://github.com/cilium/hubble.git"
source-tag: v0.13.2
source-depth: 1
override-build: |
craftctl default
mkdir -p $CRAFT_PART_INSTALL/etc/bash_completion.d
$CRAFT_PART_INSTALL/usr/local/bin/hubble completion bash > $CRAFT_PART_INSTALL/etc/bash_completion.d/hubble

cilium:
after: [build-deps, builder-img-deps, llvm-10-bpf]
after: [build-deps, builder-img-deps]
plugin: make
source-type: git
source: https://github.com/cilium/cilium.git
source-tag: v1.15.2
source-depth: 1
build-packages:
- clang-10
- llvm-10
stage-packages:
- clang-10
- llvm-10
build-environment:
- DISABLE_ENVOY_INSTALLATION: 1
override-build: |
Expand All @@ -324,3 +327,10 @@ parts:
cp $CRAFT_PART_SRC/images/cilium/init-container.sh $CRAFT_PART_INSTALL/
cp $CRAFT_PART_SRC/plugins/cilium-cni/install-plugin.sh $CRAFT_PART_INSTALL/
cp $CRAFT_PART_SRC/plugins/cilium-cni/cni-uninstall.sh $CRAFT_PART_INSTALL/

cp -a $CRAFT_PART_INSTALL/usr/bin/clang-10 $CRAFT_PART_INSTALL/usr/bin/clang
cp -a $CRAFT_PART_INSTALL/usr/bin/llc-10 $CRAFT_PART_INSTALL/usr/bin/llc
cp -a $CRAFT_PART_INSTALL/usr/bin/llvm-objcopy-10 $CRAFT_PART_INSTALL/usr/bin/llvm-objcopy
Comment on lines +331 to +333
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just duplicating symlinks for clang, llc and llvm-objcopy which are the only binaries needed by cilium afaik. Sadly it only looks for these direct names and doesn't scan for versioned tools.

override-prime: |
craftctl default
rm -rf /root/.cache/go-build
Comment on lines +334 to +336
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again some compile caches for go which doesn't help us very much, so removing this saves ~2G disk space.

Loading