Petalinux tools provide a reference Linux distribution tailored for Xilinx embedded processors, such as Microblaze™ and Zynq™ devices. It’s a layer on top of Yocto, OpenEmbedded and BitBake tools, that makes working with these devices more straightforward. The following notes are the result of working with Petalinux on a custom product development. As always, the Xilinx documentation is indispensable to navigate the complex web of options, although it’s not sufficient to understand the whole flow.

Creating a Petalinux project

In general, most people will start working on a Petalinux project on a development platform and not completely from scratch. The platform vendor will supply a BSP file, which is a format with everything Petalinux needs to reproduce the project from scratch, as packaged by the vendor.

A simple Petalinux command will create the project:

$ petalinux-create --type project --source /path/to/file.bsp

This project will have its components set up and configured, such u-boot, rootfs, kernel, device partitions, etc. to match the underlying hardware platform.

Creating a project from scratch requires setting these parameters by the user. The first requirement is to import the Vivado-exported hardware description file, with .hdf extension for Vivado <= 2019.1 or .xsa extension for 2019.2 and up.

$ petalinux-create --type project --name MYPROJECT --template zynqMP
$ cd MYPROJECT
$ petalinux-config --get-hw-description /path/to/Vivado-exported.hdf (or .xsa)

Customizing the project to match the hardware platform would require setting the base project at a minimum. A text menu (aka menuconfig) will show up after the petalinux-config command and allows this configuration to be done. The rootfs is the Linux partition that will hold the kernel and all the software packages in the filesystem. Typically, it would have to be configured as well to add necessary packages to the application running on the device. To configure it:

$ petalinux-config -c rootfs

After this, the kernel may need configuration as well, though the default settings are commonly sufficient.

$ petalinux-config -c kernel

Building the project

Once the project configuration is done, the building would follow. This step retrieves packages from the internet, patching as necessary, and full source compilation takes place. This whole process can take about 20 to 40 minutes, depending on host and network speed. Petalinux makes heavy use of caching, so once packages are compiled, they would rarely need to be re-compiled, saving valuable time on subsequent builds.

$ petalinux-build

Testing on hardware

Okay, the project is built!. Now let’s get it running on the target hardware. The fastest way to achieve that, is to boot directly from the host, loading all binaries over JTAG to the target. The target boot selection should be set to JTAG and restarted for this to take effect.

$ petalinux-boot --jtag --u-boot --fpga --kernel [--hw_server-url TCP:$HW_SRV_IP:3121]

In my experience, u-boot and the FPGA fabric would load fine, but the kernel would have trouble recognizing the Ethernet PHY using this method. I had to resort to booting without the –kernel switch and stop at u-boot, and then load the kernel over TFTP.

This is an efficient way to upload build images to the target, but eventually loading the images at boot time is needed. To package the binaries into a file suitable for loading into SD card or QSPI memory, the package command is used:

$ petalinux-package --boot --u-boot --fsbl --fpga --pmufw

This command will create a BOOT.BIN in images/linux directory, containing the u-boot, first stage bootloader, PMU and the FPGA bitstream. The kernel should already be in the same directory and named image.ub.

The images can be burned to QSPI memory using Vivado Hardware Manager or transfered to the target over SSH, and then using the following commands. The /dev/mtd partitions are configured with petalinux-config.

$ flashcp -v BOOT.BIN /dev/mtd0
$ flashcp -v image.ub /dev/mtd2

Cross platform libraries

Petalinux can also export its library components to Xilinx SDK or Vitis, in order to cross compile applications on the host. The following command takes a destination directory where to export these libraries, which will be pointed at from SDK or Vitis.

$ petalinux-package --sysroot --dir /path/to/sdk_or_vitis_library_dir

This process is time consuming, so a coffee break is granted. Fortunately, this packaging is not required to be repeated often.

Locking kernel changes

The modifications done by means of petalinux-config -c kernel are not automatically recorded in the meta-user directory structure, and thus are volatile if the project is (eg.) moved to a different location. In order to move those changes from the kernel config file to the meta-user structure, I use this command:

$ petalinux-build -c kernel -x finish [-f]

It will create a meta-user/recipes-kernel/linux directory, with a linux-xlnx_%.bbappend file and under linux-xlnx directory, the devtool-fragment.cfg file with the custom kernel config settings. Care should be taken with this command as it will overwrite any changes such as patch files already in meta-user/recipe-kernel.

# CONFIG_NF_TABLES_ARP is not set
CONFIG_NF_NAT_IPV4=y
CONFIG_NF_NAT_MASQUERADE_IPV4=y
CONFIG_NFT_CHAIN_NAT_IPV4=y
CONFIG_IP_NF_NAT=y
CONFIG_IP_NF_TARGET_MASQUERADE=y
# CONFIG_IP_NF_TARGET_NETMAP is not set
CONFIG_IP_NF_TARGET_REDIRECT=y

A similar procedure will be needed when modifying u-boot configuration settings, i.e. petalinux-config -c u-boot

Patching a system file

While doing this development, a need came across to modify the /etc/fw_env.config file, provided by the u-boot-fw-utils package, added to rootfs. This package allows Linux to read and modify u-boot environment variables in its partition, /dev/mtd1 in this platform. The /etc/fw_env.config file should match the partition size among other parameters. The default file came as follows:

/dev/mtd1       0x4000      0x4000
/dev/mtd2       0x4000      0x4000

And these lines had to be changed to match the partition size, and to comment out the second device:

/dev/mtd1       0x80000     0x40000
#/dev/mtd2       0x80000     0x40000

The solution was to add a bitbake append recipe that had a sed command to patch the existing file in place. The file petalinux-user-image.bbappend was added to project-spec/meta-user/recipes-core/images/ with the following content:

# petalinux-user-image.bbappend content

rootfs_postprocess_function() {

    # Resize /dev/mtd1 in config file for fw_setenv/fw_printenv to calculate CRC properly
    if [ -e ${IMAGE_ROOTFS}/etc/fw_env.config ]; then
        sed -i 's/0x4000/0x80000/' ${IMAGE_ROOTFS}/etc/fw_env.config
        sed -i 's/0x4000/0x40000/' ${IMAGE_ROOTFS}/etc/fw_env.config
        sed -i 's/\/dev\/mtd2/#\/dev\/mtd2/' ${IMAGE_ROOTFS}/etc/fw_env.config
    fi

}

ROOTFS_POSTPROCESS_COMMAND_append = " \
  rootfs_postprocess_function; \
  "

This recipe was taken from the Xilinx Wiki, where lots of good and pretty up to date content is to be found: RootFS post process command

Flash lock protect

The fw_setenv command from u-boot-fw-utils package has a bug that after writing to the configuration partition, it lock-protects -not only this partition- but the entire QSPI device from writes. The lock is so effective, that Linux, u-boot, not even Vivado are able to override it. U-boot should have been able to unlock it with the following command, though it would stall and would report a DMA error when canceled.

ZynqMP> sf probe
ZynqMP> sf protect unlock 0 0x2600000
        <Ctrl-C>
ZynqMP> DMA Timeout: 0x0

The solution was to boot the whole system from Petalinux and once in Linux, issue the flash_unlock command, unlocking all QSPI partitions.

$ petalinux-boot --jtag --u-boot --fpga --kernel [--hw_server-url TCP:$HW_SRV_IP:3121]

Then from Linux:

# flash_unlock -i /dev/mtd1     # to check lock status - it was indeed locked
# flash_unlock -u /dev/mtd1     # to unlock partition

TFTP boot

The Trivial File Transport Protocol (TFTP) is a very basic non-authenticated service that serves files over the network. In this case it is used to load binaries from the target at boot time, saving valuable time. Petalinux can be configured to copy the binaries to the TFTP directory automatically after each build. This directory is typically /tftpboot on the host, though this can be configured in Petalinux and also in the TFTP package as well. In Fedora and Ubuntu, this package is called tftp-server. To install on Fedora:

$ sudo dnf install tftp-server

TFTP files are looked in the /var/lib/tftpboot directory, as configured in /usr/lib/systemd/system/tftp.service file.

At boot on the target, u-boot is stopped from loading the kernel, so we have a chance to load a kernel from the TFTP server. The serverip can be set in u-boot and can also be configured in Petalinux to have a fixed IP.

ZynqMP> setenv serverip 192.168.1.147       # host IP holding binaries
ZynqMP> tftpboot ${netstart} image.ub
############################################################
############################################################
<...>
############################################################
############################################################
ZynqMP> bootm ${netstart}                   # boot kernel

Also, to load the default address and image, and immediately boot the kernel, a shortcut script can be used:

ZynqMP> run netboot