bootc enablement for ppc64le #578

sacsant · 2024-06-01T07:48:11Z

I am trying to figure out how bootc (bootable container) works and started experimenting with it on Power(ppc64le arch).

I came across https://developers.redhat.com/articles/2024/05/07/image-mode-rhel-bootable-containers# and attempted to create a bootable container image on Power using the recipe mentioned in bootc image builder project .

I used following command (Fedora40 running on PowerVM Logical Partition [LPAR])

podman run --pull=newer --rm --privileged --pid=host quay.io/sacsant/fedora-bootc:40 bootc install to-disk --target-no-signature-verification --wipe --block-setup direct --filesystem xfs /dev/sdc
WARN[0000] Failed to decode the keys ["storage.options.thinpool"] from "/usr/share/containers/storage.conf"
time="2024-05-25T06:03:33-04:00" level=warning msg="Failed to decode the keys ["storage.options.thinpool"] from "/usr/share/containers/storage.conf""
time="2024-05-25T06:03:34-04:00" level=warning msg="Failed to decode the keys ["storage.options.thinpool"] from "/usr/share/containers/storage.conf""
Installing image: docker://quay.io/sacsant/fedora-bootc:40
Digest: sha256:6c88359c24a8c351bf2f742c0f5cfd9a5b32b1e6e475499fc9612c017e2cf860
Wiping /dev/sdc1
Wiping device /dev/sdc1
Wiping /dev/sdc2
Wiping device /dev/sdc2
Wiping /dev/sdc3
Wiping device /dev/sdc3
/dev/sdc3: 4 bytes were erased at offset 0x00000000 (xfs): 58 46 53 42
Wiping /dev/sdc
Wiping device /dev/sdc
/dev/sdc: 2 bytes were erased at offset 0x000001fe (dos): 55 aa
/dev/sdc: calling ioctl to re-read partition table: Success
ERROR Installing to disk: Creating rootfs: Unsupported architecture: powerpc64

Can someone help me debug this problem? Is code missing for ppc64le arch?

prb112 · 2024-06-01T16:41:55Z

It's x86_64 and aarch64 only L251
anyhow::bail!("Unsupported architecture: {}", std::env::consts::ARCH);

It'd be great to get ppc64le support.

sacsant · 2024-06-10T07:33:30Z

Following the s390x enablement tracker, I was able to make some progress on this.

Updated the image.go to include ppc64le arch
Updated partition_tables.go with ppc64le specific data
To get around the (panic: required mounts for bootupd stage [/boot/efi] missing) issue for now commented the /boot/efi line from images repo
AnacondaContainerInstaller specific customisation code in image.go also possibly needs some update to avoid following errors. For now I commented these statements to move forward

cmd/bootc-image-builder/image.go:237:7: img.Users undefined (type *"github.com/osbuild/images/pkg/image".AnacondaContainerInstaller has no field or method Users)
cmd/bootc-image-builder/image.go:238:7: img.Groups undefined (type *"github.com/osbuild/images/pkg/image".AnacondaContainerInstaller has no field or method Groups)
cmd/bootc-image-builder/image.go:240:50: img.KickstartKernelOptionsAppend undefined (type *"github.com/osbuild/images/pkg/image".AnacondaContainerInstaller has no field or method KickstartKernelOptionsAppend)
cmd/bootc-image-builder/image.go:242:7: img.KickstartNetworkOnBoot undefined (type *"github.com/osbuild/images/pkg/image".AnacondaContainerInstaller has no field or method KickstartNetworkOnBoot)
cmd/bootc-image-builder/image.go:246:14: img.Users undefined (type *"github.com/osbuild/images/pkg/image".AnacondaContainerInstaller has no field or method Users)
cmd/bootc-image-builder/image.go:246:36: img.Groups undefined (type *"github.com/osbuild/images/pkg/image".AnacondaContainerInstaller has no field or method Groups)

I have run into following boot loader related error:

Installing image: docker://quay.io/sacsant/fedora-bootc:latest
Initializing ostree layout
Initializing sysroot
ostree/deploy/default initialized as OSTree stateroot
Deploying container image
Deployment complete
Running bootupctl to install bootloader
Installing for powerpc-ieee1275 platform.
/usr/sbin/grub2-install: error: the chosen partition is not a PReP partition.
error: boot data installation failed: installing component BIOS: Failed to run "/usr/sbin/grub2-install" "--target" "powerpc-ieee1275" "--boot-directory" "/run/osbuild/mounts/boot" "--no-nvram" "/dev/loop0"
ERROR Installing to filesystem: Installing bootloader: Task Running bootupctl to install bootloader failed: ExitStatus(unix_wait_status(256))
Traceback (most recent call last):
File "/run/osbuild/bin/org.osbuild.bootc.install-to-filesystem", line 53, in
r = main(args["options"], args["inputs"], args["paths"])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/run/osbuild/bin/org.osbuild.bootc.install-to-filesystem", line 48, in main
subprocess.run(pargs, env=env, check=True)
File "/usr/lib64/python3.12/subprocess.py", line 571, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['bootc', 'install', 'to-filesystem', '--source-imgref', 'containers-storage:[overlay@/run/osbuild/containers/storage+/run/containers/storage]b3a026083aa7f61628b5fdc3ab90b5b692139668ec583485d0f2e48fe34e46d9', '--skip-fetch-check', '--generic-image', '--karg', 'rw', '--karg', 'console=tty0', '--karg', 'console=ttyS0', '--target-imgref', 'quay.io/sacsant/fedora-bootc:latest', '/run/osbuild/mounts']' returned non-zero exit status 1.

Any suggestions on how to solve this error? I am unable to connect the dots and having trouble identifying code that needs a change to fix this.

sacsant · 2024-06-10T07:35:25Z

To be specific I am currently focussed on creating a qcow2 based bootable image.

podman run --rm -it --privileged --pull=newer --security-opt label=type:unconfined_t -v $(pwd)/config.toml:/config.toml -v $(pwd)/output:/output -v /var/lib/containers/storage:/var/lib/containers/storage quay.io/sacsant/bootc-image-builder:latest --type qcow2 --rootfs xfs --local quay.io/sacsant/fedora-bootc:latest

cgwalters · 2024-06-10T13:30:10Z

/usr/sbin/grub2-install: error: the chosen partition is not a PReP partition.
error: boot data installation failed: installing component BIOS: Failed to run "/usr/sbin/grub2-install" "--target" "powerpc-ieee1275" "--boot-directory" "/run/osbuild/mounts/boot" "--no-nvram" "/dev/loop0"
ERROR Installing to filesystem: Installing bootloader: Task Running bootupctl to install bootloader failed: ExitStatus(unix_wait_status(256))

That's in https://github.com/coreos/bootupd/

sacsant · 2024-06-11T13:05:18Z

Thanks Colin for the pointer. Since I am completely new to this project/code, I am still unable to connect the dots.

Should I use the coreos/bootupd code as reference and make similar changes to other repositories (bootc-image-builder or images or perhaps bootc??) or should I change coreos/bootupd? Can you guide me on this?

I noticed that coreos/bootupd uses powerpc64 as a value to identify IBM Power architecture while other projects like bootc-image-builder and likes use ppc64le as a value. Is there a theory behind this difference ?

mkumatag · 2024-06-11T14:02:28Z

I noticed that coreos/bootupd uses powerpc64 as a value to identify IBM Power architecture while other projects like bootc-image-builder and likes use ppc64le as a value. Is there a theory behind this difference ?

This is based on how your programming language detects the platform

e.g:
golang

# go env GOARCH
ppc64le      <=============================

rust:

# rustc --target=powerpc64le-unknown-linux-gnu --print=cfg
debug_assertions
panic="unwind"
target_abi=""
target_arch="powerpc64"             <=============================
target_endian="little"
target_env="gnu"
target_family="unix"
target_has_atomic="16"
target_has_atomic="32"
target_has_atomic="64"
target_has_atomic="8"
target_has_atomic="ptr"
target_os="linux"
target_pointer_width="64"
target_vendor="unknown"
unix

cgwalters · 2024-06-11T14:39:39Z

What's extra fun here is that it's actually only for ppc64le on Fedora derivatives that the "Rust vs RPM" architecture naming is different. Rust=powerpc64 RPM=ppc64le whereas they match for s390x, x86_64 and aarch64. Note that the Go and RPM names are otherwise different for amd64 vs x86_64 etc.

sacsant · 2024-06-20T16:10:29Z

Going back to the original error of "Unsupported architecture: powerpc64" , I added following code in bootc
} else if cfg!(target_arch = "powerpc64") { // BIOS-BOOT sgdisk_partition( &mut sgdisk.cmd, 1, "0:+4M", "BIOS-BOOT", Some("9E1A2D38-C612-4316-AA26-8B49521E5A8B"), );

Is the above code correct?

With this code using install to-disk command the operation proceeds further but fails eventually with

DEBUG exec: "sgdisk" "-Z" "/dev/sdb" "-U" "R" "-n" "1:0:+4M" "-c" "1:BIOS-BOOT" "-t" "1:9E1A2D38-C612-4316-AA26-8B49521E5A8B" "-n" "3:0:0" "-c" "3:root" "-t" "3:0FC63DAF-8483-4772-8E79-3D69D8477DE4"
[14381.355893] sdb: sdb1 sdb3
DEBUG Created partition table
[14381.356751] sdb: sdb1 sdb3
DEBUG argv0=Some("exe")
ERROR Installing to disk: Creating rootfs: Missing partition for index 3

Following partition layout was written to the disk
Device Start End Sectors Size Type
/dev/sdb1 256 1279 1024 4M PowerPC PReP boot
/dev/sdb3 1280 26214394 26213115 100G Linux filesystem

What partition is missing and is referenced by index 3? I guess there should be /dev/sdb2 pointing to /boot . Can someone help decode this error?

sacsant · 2024-06-20T19:26:34Z

What partition is missing and is referenced by index 3? I guess there should be /dev/sdb2 pointing to /boot . Can someone help decode this error?

I was able to modify the code to move past this issue.

Using block setup: direct
Initializing partitions
Creating boot filesystem (xfs) on device /dev/sdb2 (size=510M)

mkfs.xfs -m uuid=e14c7dd8-4297-4cb4-b41b-eb027167d941 -L boot /dev/sdb2
Creating root filesystem (xfs) on device /dev/sdb3 (size=99.5G)
mkfs.xfs -m uuid=ad5cdc91-0ca9-4cf5-afda-5aeaf53b1370 -L root /dev/sdb3
Mounting /run/bootc/mounts/rootfs
Mounting /run/bootc/mounts/rootfs/boot
Initializing ostree layout
Initializing sysroot
ostree/deploy/default initialized as OSTree stateroot
Deploying container image...Freed objects: 118 bytes
done
Running bootupctl to install bootloader

Now both the methods qcow2 and install to-disk fail during boot loader install

Running bootupctl to install bootloader

bootupctl backend install --write-uuid --update-firmware --auto --device /dev/sdb /run/bootc/mounts/rootfs
Installing for powerpc-ieee1275 platform.
/usr/sbin/grub2-install: error: the chosen partition is not a PReP partition.
error: boot data installation failed: installing component BIOS: Failed to run "/usr/sbin/grub2-install" "--target" "powerpc-ieee1275" "--boot-directory" "/run/bootc/mounts/rootfs/boot" "--no-nvram" "/dev/sdb"
ERROR Installing to disk: Installing bootloader: Task Running bootupctl to install bootloader failed: ExitStatus(unix_wait_status(256))

sacsant · 2024-06-21T14:08:49Z

With some additional changes (read hard coded values) I was able to successfully use bootc install to-disk and to-filesystem commands on ppc64le architecture. I was also able to successfully verify boot of these images (disk device as well as qcow2) on IBM Power.

cgwalters · 2024-06-21T14:33:48Z

Cool! Feel free to send a PR, it's OK if it's not polished, we can iterate on it!

sacsant · 2024-06-25T04:47:45Z

I have submitted following pull requests for initial set of changes
PR 497 against bootc-image-builder
PR 636 against bootc

Working on following changes:

images : Remove dependency on /boot/efi for ppc64le arch
bootc-image-builder compile time failure

cmd/bootc-image-builder/image.go:237:7: img.Users undefined (type *"github.com/osbuild/images/pkg/image".AnacondaContainerInstaller has no field or method Users)
cmd/bootc-image-builder/image.go:238:7: img.Groups undefined (type *"github.com/osbuild/images/pkg/image".AnacondaContainerInstaller has no field or method Groups)
cmd/bootc-image-builder/image.go:240:50: img.KickstartKernelOptionsAppend undefined (type *"github.com/osbuild/images/pkg/image".AnacondaContainerInstaller has no field or method KickstartKernelOptionsAppend)
cmd/bootc-image-builder/image.go:242:7: img.KickstartNetworkOnBoot undefined (type *"github.com/osbuild/images/pkg/image".AnacondaContainerInstaller has no field or method KickstartNetworkOnBoot)
cmd/bootc-image-builder/image.go:246:14: img.Users undefined (type *"github.com/osbuild/images/pkg/image".AnacondaContainerInstaller has no field or method Users)
cmd/bootc-image-builder/image.go:246:36: img.Groups undefined (type *"github.com/osbuild/images/pkg/image".AnacondaContainerInstaller has no field or method Groups)

bootupd changes

yoheiueda · 2024-06-25T05:57:15Z

Hi @sacsant

I am working on s390x support, and I just raised this PR osbuild/images#758

images : Remove dependency on /boot/efi for ppc64le arch

I think my patch may be relevant to this issue.

sacsant · 2024-06-25T06:05:30Z

Hi @sacsant

I am working on s390x support, and I just raised this PR osbuild/images#758

images : Remove dependency on /boot/efi for ppc64le arch

I think my patch may be relevant to this issue.

Indeed. Thanks @yoheiueda

sacsant · 2024-07-02T12:30:52Z

@cgwalters to identify correct partition for grub2-install, I have tried using following code in lib/src/bootloader.rs

// get PowerPC-PReP-boot device information
let result = Command::new("realpath")
.arg("/dev/disk/by-partlabel/PowerPC-PReP-boot")
.output()?;
if !result.status.success() {
anyhow::bail!("realpath failed with {}", result.status);
}

Unfortunately this code fails with return code 1 which if I read correctly corresponding to errno EPERM (Operation not permitted).

Creating root filesystem (xfs) on device /dev/sdb3 (size=99.5G)

mkfs.xfs -m uuid=2e332d8b-8cca-46df-8d63-9d874333cada -L root /dev/sdb3
Mounting /run/bootc/mounts/rootfs
Mounting /run/bootc/mounts/rootfs/boot
Initializing ostree layout
Initializing sysroot
ostree/deploy/default initialized as OSTree stateroot
Deploying container image...Freed objects: 118 bytes
done
ERROR Installing to disk: Installing bootloader: realpath failed with exit status: 1

Is there a restriction on running commands like realpath in bootc environment?
If I manually run the command it does return correct output
$ realpath /dev/disk/by-partlabel/PowerPC-PReP-boot
/dev/sdb1

The disk is partitioned correctly as follows.

fdisk -l /dev/sdb
Disk /dev/sdb: 100 GiB, 107374182400 bytes, 26214400 sectors
Disk model: VDASD
Units: sectors of 1 * 4096 = 4096 bytes
Sector size (logical/physical): 4096 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: DE468C00-7382-4C25-A58A-1802A3AC9666

Device Start End Sectors Size Type
/dev/sdb1 256 1279 1024 4M PowerPC PReP boot
/dev/sdb2 1280 131839 130560 510M Linux filesystem
/dev/sdb3 131840 26214394 26082555 99.5G Linux filesystem

sacsant · 2024-07-03T13:59:51Z

mkfs.xfs -m uuid=2e332d8b-8cca-46df-8d63-9d874333cada -L root /dev/sdb3
Mounting /run/bootc/mounts/rootfs
Mounting /run/bootc/mounts/rootfs/boot
Initializing ostree layout
Initializing sysroot
ostree/deploy/default initialized as OSTree stateroot
Deploying container image...Freed objects: 118 bytes
done
ERROR Installing to disk: Installing bootloader: realpath failed with exit status: 1

Is there a restriction on running commands like realpath in bootc environment? If I manually run the command it does return correct output $ realpath /dev/disk/by-partlabel/PowerPC-PReP-boot /dev/sdb1

The failure is due to non-existent directory

Initializing ostree layout
Initializing sysroot
ostree/deploy/default initialized as OSTree stateroot
Deploying container image...Freed objects: 118 bytes
done
realpath: /dev/disk/by-partlabel/PowerPC-PReP-boot: No such file or directory

ERROR Installing to disk: Installing bootloader: realpath failed with exit status: 1

sacsant · 2024-07-04T07:05:39Z

I have submitted #667 to include ppc64le specific boot loader install logic.

cgwalters · 2024-08-01T20:52:22Z

At this point, I think we've fixed everything that's needed in this specific project! There's clearly more work - such as enabling c9s bootc that are probably best tracked in e.g. https://gitlab.com/fedora/bootc/tracker/-/issues/15

cgwalters added area/install Issues related to `bootc install` area/osintegration Relates to an external OS/distro base image labels Jun 1, 2024

sacsant mentioned this issue Jul 4, 2024

lib/bootloader: Write to PReP partition on ppc64le #667

Merged

cgwalters closed this as completed Aug 1, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

bootc enablement for ppc64le #578

bootc enablement for ppc64le #578

sacsant commented Jun 1, 2024 •

edited

Loading

prb112 commented Jun 1, 2024

sacsant commented Jun 10, 2024

sacsant commented Jun 10, 2024

cgwalters commented Jun 10, 2024

sacsant commented Jun 11, 2024

mkumatag commented Jun 11, 2024

cgwalters commented Jun 11, 2024

sacsant commented Jun 20, 2024

sacsant commented Jun 20, 2024

sacsant commented Jun 21, 2024

cgwalters commented Jun 21, 2024

sacsant commented Jun 25, 2024

yoheiueda commented Jun 25, 2024

sacsant commented Jun 25, 2024

sacsant commented Jul 2, 2024

sacsant commented Jul 3, 2024

sacsant commented Jul 4, 2024

cgwalters commented Aug 1, 2024

bootc enablement for ppc64le #578

bootc enablement for ppc64le #578

Comments

sacsant commented Jun 1, 2024 • edited Loading

prb112 commented Jun 1, 2024

sacsant commented Jun 10, 2024

sacsant commented Jun 10, 2024

cgwalters commented Jun 10, 2024

sacsant commented Jun 11, 2024

mkumatag commented Jun 11, 2024

cgwalters commented Jun 11, 2024

sacsant commented Jun 20, 2024

sacsant commented Jun 20, 2024

sacsant commented Jun 21, 2024

cgwalters commented Jun 21, 2024

sacsant commented Jun 25, 2024

yoheiueda commented Jun 25, 2024

sacsant commented Jun 25, 2024

sacsant commented Jul 2, 2024

sacsant commented Jul 3, 2024

sacsant commented Jul 4, 2024

cgwalters commented Aug 1, 2024

sacsant commented Jun 1, 2024 •

edited

Loading