This document is a WORK IN PROGRESS.
This is just a quick personal cheat sheet: treat its contents with caution!

Yocto¶

The Yocto Project is a Linux Foundation work group whose goal is to produce tools and processes that will enable the creation of Linux distributions for embedded software that are independent of the underlying architecture of the embedded software itself.

Some distros (like Gentoo or Arch) are not officially supported by the Yocto/OpenEmbedded project, however, they generally works fine as Yocto/OE build host (but sometimes with one or two caveats about building on a hardened profile).

Prerequisite(s)

QEMU KVM

Reference(s)

https://wiki.gentoo.org/wiki/Yocto
https://wiki.archlinux.org/index.php/Yocto
https://www.yoctoproject.org
https://www.yoctoproject.org/docs/current/mega-manual/mega-manual.html (reference manual)
https://www.yoctoproject.org/docs/current/brief-yoctoprojectqs/brief-yoctoprojectqs.html (quick start)
http://www.informit.com/articles/article.aspx?p=2514911 (core image differences explanation)
https://community.nxp.com/docs/DOC-94953 (useful bitbake commands)
https://www.yoctoproject.org/docs/what-i-wish-id-known/
https://www.yoctoproject.org/docs/transitioning-to-a-custom-environment/

Install¶

Look at the following link to get a detailed packages dependency list depending on the Yocto version you want:

https://www.yoctoproject.org/docs/latest/ref-manual/ref-manual.html#ref-manual-system-requirements (for latest Yocto)
https://www.yoctoproject.org/docs/3.0/ref-manual/ref-manual.html#ref-manual-system-requirements (e.g. for Yocto v3.0)
https://www.yoctoproject.org/docs/2.0/ref-manual/ref-manual.html#intro-requirements (e.g. for Yocto v2.0)

Install the needed Yocto dependencies to perform builds:

emergepacmanaptyumdnf

Double check here: https://wiki.gentoo.org/wiki/Yocto

# emerge -n patch make sed dev-lang/python:2.7 m4 bison cvs openjade quilt sgmltools-lite docbook-xml-dtd docbook-dsssl-stylesheets xmlto docbook-sgml-utils libpcre boost subversion texi2html chrpath fakeroot lzop bc

Double check here: https://wiki.archlinux.org/index.php/Yocto

# vi /etc/pacman.conf # Enable the multilib repository (for the multilib-devel group)
    > ...
    > [multilib]
    > Include = /etc/pacman.d/mirrorlist
    > ...
# pacman -S git diffstat unzip texinfo python chrpath wget xterm sdl rpcsvc-proto socat cpio inetutils multilib-devel

Double check here: https://www.yoctoproject.org/docs/latest/ref-manual/ref-manual.html#ubuntu-packages

# apt install gawk wget git-core diffstat unzip texinfo gcc-multilib build-essential chrpath socat cpio python python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm

Double check here: https://www.yoctoproject.org/docs/latest/ref-manual/ref-manual.html#centos-7-packages

# yum install -y epel-release
# yum makecache
# yum install gawk make wget tar bzip2 gzip python3 unzip perl patch diffutils diffstat git cpp gcc gcc-c++ glibc-devel texinfo chrpath socat perl-Data-Dumper perl-Text-ParseWords perl-Thread-Queue python36-pip xz which SDL-devel xterm
# pip3 install GitPython jinja2

Double check here: https://www.yoctoproject.org/docs/latest/ref-manual/ref-manual.html#fedora-packages

# dnf install gawk make wget tar bzip2 gzip python3 unzip perl patch diffutils diffstat git cpp gcc gcc-c++ glibc-devel texinfo chrpath ccache perl-Data-Dumper perl-Text-ParseWords perl-Thread-Queue perl-bignum socat python3-pexpect findutils which file cpio python python3-pip xz python3-GitPython python3-jinja2 SDL-devel xterm rpcgen

Then check your git, tar, python and GCC versions: https://www.yoctoproject.org/docs/latest/ref-manual/ref-manual.html#required-git-tar-python-and-gcc-versions !

Install Yocto:

$ mkdir -p ~/projects/yocto
$ cd ~/projects/yocto
$ git clone -b dora git://git.yoctoproject.org/poky.git
$ git checkout -b zeus origin/zeus # checkout to the wanted version (e.g. zeus)

Config¶

Run the setup script to define Yocto Project's build environment on your build host:

$ cd ~/projects/yocto/poky
$ BUILD_NAME=$(git log --pretty='format:%aI--%h' | head -1)
$ echo $BUILD_NAME

$ source oe-init-build-env build--$BUILD_NAME--qemux86-64 # e.g. for a default qemux86-64 target
$ pwd # you should know be located in the "build" directory
    > ~/projects/yocto/poky/build--$BUILD_NAME--qemux86-64

You might want to point to another downloads directory before building, it might serve as common downloads folder for future builds (also with other images and/or target, as said here https://v-connect.io/yocto/hands-on-yocto-configuration.html):

$ mkdir ~/projects/yocto/downloads
$ vi conf/local.conf
    > ...
    > DL_DIR ?= "/home/user/projects/yocto/downloads"
    >...

It might also be possible to share the SSTATE_DIR between builds (as said in the manual "You can reuse the directory SSTATE_DIR for multiple builds"):

$ vi conf/local.conf
    > ...
    > SSTATE_DIR = "/home/user/projects/yocto/sstate-cache
    > ...

Use¶

If the build folder has already been created, and you want to resume (or redo...) builds, then you shall source oe-init-build-env first:

$ cd ~/projects/yocto/poky
$ source oe-init-build-env build--$BUILD_NAME--qemux86-64
$ pwd
    > ~/projects/yocto/poky/build--$BUILD_NAME--qemux86-64

Preserve disk space during builds:

$ vi ~/projects/yocto/poky/build/conf/local.conf
    > ...
    > # To help conserve disk space during builds, you can add the following statement.
    > # Adding this statement deletes the work directory used for building a recipe once the
    > # recipe is built.
    > INHERIT += "rm_work"
    > ...

Select a target machine (e.g. for a default qemux86-64):

$ vi ~/projects/yocto/poky/build--$BUILD_NAME--qemux86-64/conf/local.conf
    > ...
    > MACHINE = "qemux86-64"
    > ...

Run a build for a minimal image:

$ cd ~/projects/yocto/poky/build--$BUILD_NAME--qemux86-64
$ bitbake core-image-minimal

Run qemux86-64 (following the previous use case example):

$ cd ~/projects/yocto/poky/build
$ runqemu qemux86-64

Modify number of parallel running tasks when building:

In general, the default settings for all the following variables result in the most efficient build times when dealing with single socket systems (i.e. a single CPU): if you have multiple CPUs, you might try increasing the default values to gain more speed.

$ vi ~/projects/yocto/poky/build/conf/local.conf
    > ...
    > #
    > # Parallelism Options
    > #
    > # The maximum number of threads BitBake simultaneously executes:
    > #BB_NUMBER_THREADS ?= "4"
    > # Default to setting automatically based on cpu count
    > #BB_NUMBER_THREADS ?= "${@oe.utils.cpu_count()}"
    > #
    > # The number of threads BitBake uses during parsing:
    > #BB_NUMBER_PARSE_THREADS ?= "4"
    > #Default to setting automatically based on cpu count
    > # BB_NUMBER_PARSE_THREADS ?= "${@oe.utils.cpu_count()}"
    > #
    > # Extra options passed to the make command during the do_compile task in order to specify
    > # how many parallel compilation should happen on the local build host:
    > #PARALLEL_MAKE ?= "-j 4"
    > # Default to setting automatically based on cpu count
    > #PARALLEL_MAKE ?= "-j ${@oe.utils.cpu_count()}"
    > #
    > # PARALLEL_MAKEINST: Extra options passed to the make command during the do_install task in
    > # order to specify parallel installation on the local build host. Default value is the same
    > # that PARALLEL_MAKE wich is optimal in most cases.
    > #
    > #
    > # Example: For a quad-core machine, BB_NUMBER_THREADS = "4", PARALLEL_MAKE = "-j 4" would
    > # be appropriate for example.
    > ...

Print layers (also called metadata layers):
```
$ bitbake-layers show-layers
```
Find a layer:
- http://git.yoctoproject.org/
- https://caffelli-staging.yoctoproject.org/software-overview/layers/
- http://layers.openembedded.org/layerindex/branch/master/layers/

Add an existing layer (e.g. Raspberrypi3), for a new target and build it:

$ cd ~/projects/yocto

$ git clone git://git.yoctoproject.org/meta-raspberrypi
$ cd meta-raspberrypi
$ git checkout -b zeus origin/zeus # checkout to same version than poky

$ cd ~/projects/yocto/poky/build

$ vi conf/bblayers.conf
        > ...
        > BBLAYERS ?= " \
        >   ...
        >   /home/user/projects/yocto/meta-raspberrypi \
        >   "
        > ...
$ vi conf/local.conf
        > ...
        > MACHINE = "raspberrypi3"
        > ...

$ bitbake core-image-minimal # this might take a while (especially the first time...)

Create and add his own layer:

$ cd ~/projects/yocto
$ bitbake-layers create-layer meta-layername

$ cd ~/projects/yocto/poky/build
$ bitbake-layers add-layer ~/projects/yocto/meta-layername

Add a recipe (e.g. the htop 2.2.0 recipe compatible with yocto zeus https://layers.openembedded.org/layerindex/recipe/995/) to a layer (e.g. to the meta-layername layer):

$ cd ~/projects/yocto/meta-layername

$ mkdir -p recipes-test
$ cd recipes-test
$ wget http://cgit.openembedded.org/meta-openembedded/plain/meta-oe/recipes-support/htop/htop_2.2.0.bb # ⚠️  make sure this htop version is compatible with the used yocto verions ⚠️

$ mkdir -p files
$ cd files
$ wget http://cgit.openembedded.org/meta-openembedded/plain/meta-oe/recipes-support/htop/files/0001-Use-pkg-config.patch
$ wget http://cgit.openembedded.org/meta-openembedded/plain/meta-oe/recipes-support/htop/files/0001-Ask-for-python3-specifically.patch

$ cd ~/projects/yocto/poky/build
$ bitbake htop
$ vi ~/projects/yocto/poky/meta/recipes-core/images/core-minimal-image.bb
    > ...
    > IMAGE_INSTALL += "htop"

$ bitbake core-image-minimal -C rootfs # add the newly comiled package to the rootfs (rebuilding the rootfs)

You can use the -e BitBake option to display the parsing environment for a configuration:
```
$ bitbake -e <recipename>
```
E.g. to look at PACKAGECONFIG:
```
$ bitbake -e <recipename> | grep ^PACKAGECONFIG=
```

Create his own image in a layer (e.g. meta-test layer) (see https://hub.mender.io/t/how-to-create-custom-images-using-yocto-project/902):

$ mkdir -p ~/projects/yocto/meta-test/recipes-core/images/

$ vi ~/projects/yocto/meta-test/recipes-core/images/test-image.bb
    > SUMMARY = "Simple test image."
    >
    > #CORE_IMAGE_EXTRA_INSTALL += "htop"
    >
    > IMAGE_INSTALL = "packagegroup-core-boot ${CORE_IMAGE_EXTRA_INSTALL}"
    >
    > LICENSE = "MIT"
    >
    > inherit core-image

$ cd ~/projects/yocto/poky/build
$ bitbake test-image

If, with the above commands ends up failing, one might try this way:

$ vi ~/projects/yocto/meta-test/recipes-core/images/test-image.bb
    > SUMMARY = "Simple test image."
    >
    > IMAGE_INSTALL = "packagegroup-core-boot ${CORE_IMAGE_EXTRA_INSTALL}"
    >
    > LICENSE = "MIT"
    >
    > inherit core-image
    >
    > #IMAGE_INSTALL += "htop"

$ bitbake test-image -C rootfs

Create his own package group (e.g. just containing htop) in a layer (e.g. meta-test layer) for an image (e.g. test-image) (see https://hub.mender.io/t/how-to-create-custom-images-using-yocto-project/902):

$ mkdir -p ~/projects/yocto/meta-test/recipes-core/packagegroups/
$ vi ~/projects/yocto/meta-test/recipes-core/packagegroups/test-packagegroup-plop.bb
    > DESCRIPTION = "Test packagegroup"
    > SUMMARY = "Simple test packagegroup just containing htop"
    >
    > PACKAGE_ARCH = "${MACHINE_ARCH}"
    >
    > inherit packagegroup
    >
    > RDEPENDS_${PN} = " \
    >     htop \
    > "

$ vi ~/projects/yocto/meta-test/recipes-core/images/test-image.bb
    > SUMMARY = "Simple test image."
    >
    > CORE_IMAGE_EXTRA_INSTALL += "test-packagegroup-plop"
    >
    > IMAGE_INSTALL = "packagegroup-core-boot ${CORE_IMAGE_EXTRA_INSTALL}"
    >
    > LICENSE = "MIT"
    >
    > inherit core-image

$ cd ~/projects/yocto/poky/build
$ bitbake test-image

Bring up a dependency explorer for an image (e.g. test-image) to check file dependencies:
```
$ bitbake -u taskexp -g test-image
```
Bring up a dependency explorer for a recipe (e.g. htop):
```
$ bitbake -g -u taskexp htop
```

Reports component sizes (in Bytes) for the root file system (e.g. only for components heavier than 100 kB):

$ cd ~/projects/yocto/poky/build
$ ../scripts/tiny/dirsize.py 100000 > dirsize-100k.log
$ cat dirsize-100k.log

Handle passwords and users (see https://www.yoctoproject.org/docs/current/mega-manual/mega-manual.html#ref-classes-extrausers)

e.g. in the test-image, add the "username" user with the password userpwd and modify the root password to toor:

$ vi ~/projects/yocto/poky/meta-test/recipes-core/images/test-image.bb
    > ...
    > inherit extrausers
    > EXTRA_USERS_PARAMS = "\
    >   useradd -P userpwd username; \
    >   usermod -P toor root; \
    > "
    > ...
$ bitbake test-image

Fetch all the necessary sources without starting the build (all sources files are downloaded into the DL_DIR):
```
$ bitbake -c target runall="fetch"
```

Print debug output from bitbake (e.g. when building htop):

$ bitbake -D htop # base logging level
$ bitbake -DD htop # intermediate loggin level
$ bitbake -DDD htop # high loggin level

For useful bitbake commands: see https://community.nxp.com/docs/DOC-94953

SDK¶

See https://www.yoctoproject.org/docs/current/mega-manual/mega-manual.html#sdk-dev-environment
See https://www.yoctoproject.org/docs/2.1/sdk-manual/sdk-manual.html

Generate the SDK (e.g. for the test-image):

$ vi ~/projects/poky/build/conf/local.conf # select a machine for which the SDK will be built (e.g. x86_64)
    > ...
    > SDKMACHINE ?= "x86_64"
    > ...

$ bitbake -c populate_sdk test-image # or "bitbake -c populate_sdk_ext test-image" for extensive sdk

Find the generated files under ~/projects/poky/build/tmp/deploy/sdk with the .sh SDK installer for cross development environment.

Install the SDK:

$ ./~/projects/poky/build/tmp/deploy/sdk/poky-glibc-x86_64-test-image-core2-64-qemux86-64-toolchain-3.0.1.sh

The default SDK installation location will be /opt/poky/3.0.1

Develop a simple hello word with the SDK for GNU auto tools based projects:

$ mkdir ~/projects/helloword
$ cd ~/projects/helloword
$ touch readme
$ vi hello.c
    > #include <stdio.h>
    >
    > main()
    >   {
    >       printf("Hello World!\n");
    >   }

$ vi configure.ac
    > AC_INIT(hello,0.1)
    > AM_INIT_AUTOMAKE([foreign])
    > AC_PROG_CC
    > AC_CONFIG_FILES(Makefile)
    > AC_OUTPUT

$ vi Makefile.am
    > bin_PROGRAMS = hello
    > hello_SOURCES = hello.c

$ source /opt/poky/3.0.1/environment-setup-core2-64-poky-linux

$ autoreconf # if you get errors indicating missing files: run "$ autoreconf -i" to install those

$ echo ${CONFIGURE_FLAGS} # print your configuration flags: look at --host and --with-libtool-sysroot for the next cmd

$ ./configure --host=x86_64-poky-linux --with-libtool-sysroot=/opt/poky/3.0.1/sysroots/core2-64-poky-linux # for example

$ make
$ make install DESTDIR=./tmp
$ file ./tmp/usr/local/bin/hello

Develop a simple hello word with the SDK for makefile based projects:

$ mkdir ~/projects/helloword
$ cd ~/projects/helloword

$ vi main.c
    > #include "module.h"
    > void sample_func();
    > int main()
    > {
    >   sample_func();
    >   return 0;
    > }

$ vi module.h
    > #include <stdio.h>
    > void sample_func();

$ vi module.c
    > #include "module.h"
    > void sample_func()
    > {
    >   printf("Hello World!");
    >   printf("\n");
    > }

$ source /opt/poky/3.0.1/environment-setup-core2-64-poky-linux

$ echo ${CC}

$ vi makefile # make will not cross compile (default compiler is gcc)
    > # CC="gcc"
    > all: main.o module.o
    >   ${CC} main.o module.o -o target_bin
    > main.o: main.c module.h
    >   ${CC} -I . -c main.c
    > module.o: module.c module.h
    >   ${CC} -I . -c module.c
    > clean:
    >   rm -rf *.o
    >   rm target_bin

$ vi makefile # make will cross compile
    > CC=x86_64-poky-linux-gcc  -m64 -march=core2 -mtune=core2 -msse3 -mfpmath=sse --sysroot=/opt/poky/3.0.1/sysroots/core2-64-poky-linux
    > all: main.o module.o
    >   ${CC} main.o module.o -o target_bin
    > main.o: main.c module.h
    >   ${CC} -I . -c main.c
    > module.o: module.c module.h
    >   ${CC} -I . -c module.c
    > clean:
    >   rm -rf *.o
    >   rm target_bin

$ make

Extract the root file system, e.g. extract the rootfs of test-image (the SDK need to be build and installed before hand):

$ mkdir /tmp/test-rootfs
$ source /opt/poky/3.0.1/environment-setup-core2-64-poky-linux
$ runqemu-extract-sdk ~/Projects/poky/build/tmp/deploy/images/test-image-qemux86-64-20200113150625.rootfs.tar.bz2 /tmp/test-rootfs

Booting the images from a USB flash drive¶

After building, you will find bootable images (e.g. in ~/projects/yocto/project-name/build_name/tmp/deploy/images/target-name), which can be used to directly boot Yocto from a USB flash drive.

First, insert a USB flash drive. Assuming the USB flash drive takes device /dev/sdf, use dd to copy the live image to it. For example:
```
# dd if=core-image-target-20200107053738.hddimg of=/dev/sdf
# sync
# eject /dev/sdf
```
This should give you a bootable USB flash device.
Insert the device into a bootable USB socket on the target, and power on. This should result in a system booted to the Sato graphical desktop (depending on your build config).
If you want a terminal, use the arrows at the top of the UI to move to different pages of available applications, one of which is named 'Terminal'. Clicking that should give you a root terminal.
If you want to ssh into the system, you can use the root terminal to ifconfig the IP address and use that to ssh in. The root password is empty (depending on your build config), so to log in type 'root' for the user name and hit 'Enter' at the Password prompt: and you should be in.

Troubleshooting¶

If you find you're getting corrupt images on the USB (it doesn't show the syslinux boot prompt, or the boot prompt contains strange characters), try doing this first:
```
# dd if=/dev/zero of=/dev/sdf bs=1M count=512
```

If this cheat sheet has been useful to you, then please consider leaving a star here.

Yocto¶

Table of contents¶

Install¶

Config¶

Use¶

SDK¶

Booting the images from a USB flash drive¶

Troubleshooting¶

Harden/More secure images¶

Test images¶

Troubleshooting¶