Understanding Yocto Project, Yocto Starter Pack
Let us start with what is Yocto actually?
Most of the time people get confused with Yocto and its usage. Here is the most suitable information from Yocto website home page
“Yocto is NOT a linux distribution but it creates a custom one for you”
Using Yocto one can create its own linux distribution suitable to his/her needs. There are different ways to create our own custom embedded linux based build system. Here are they
- Build everything manually (Creating own Makefiles and scripts for packaging in to build)
- It has more flexibility
- Time consuming, as to create manual build system will take time
- Binary distributions (Debian, Fedora / .rpm , .deb etc.)
- Not much flexible and hard to customize
- Not possible to build from source
- May not available for all machines and architectures
- Custom Build system creator projects (Yocto, OpenWRT, BuildRoot, etc.)
- Mostly flexible
- Build from the source
- Leverage available tools and software for creating build system
Who can use Yocto and what it is use for?
If you want to build your own linux distribution for your embedded product then Yocto is the best choice. One can leverage the work already done by OE and Yocto community and different SOC vendor companies. Yocto provides full flexibility for creating distribution for the embedded product.
It is widely adopted in the industry and one can get support for most of the embedded platforms. In addition to semiconductor vendors, there are a number of vendors providing commercially supported Yocto software and services.
Here are the benefits of using Yocto
1.Develop using common linux OS for all major architecture
2.Flexible framework allowing reuse of software
3.Changing HW platforms easy
4.Access to collections of layers and recipes
Yocto maintains OpenEmbedded core and poky is reference distribution of the Yocto project. Poky provides
So let's start
Poky = OpenEmbedded Build system + Set of meta data from Yocto / Poky to enable building own distribution
Open Embedded Build system = BitBake + OpenEmbedded Core (meta/ directory)
Bitbake: A task executor and scheduler. It is a build engine that works through recipes written in a specific format in order to perform sets of tasks
OpenEmbedded-Core: OpenEmbedded-Core (OE-Core) is a common layer of metadata (i.e. recipes, classes, and associated files) used by OpenEmbedded-derived systems, which includes the Yocto Project. It is comprised of foundation recipes, classes and associated files that are meant to be common among many different OpenEmbedded system.
Metadata: Task definitions / Set of instructions. This contains the files that OpenEmbedded build system parses when building an image. It includes recipes, configuration files and instruction on how to build an image
Yocto project uses different terminologies as mentioned below. Its packages are grouped in to layers and recipes are part of layers which is nothing but individual piece of software.
For example one can create different layer based on its functionality and category. Like layer for connectivity which includes all software related to connectivity in the embedded product, layer for BSP which includes all software components related to board board support package, layer for user application which may include all software components related to software applications used by end users.
Remember layer are kind of repositories or folder we can say. We can have multiple recipes within each layer
Here are the different components of the Yocto project
Configuration files (.conf) - Global definition variables [types of machine architecture, global variables, build path, compiler flags]
Classes (.bbclass) - encapsulation and inheritance of build logic [defines how we build linux kernel, how to generate RPM package, how to create root file system image]
Recipes (.bb) - logical units of SW / Images to build [individual piece of SW to be built, what packages get included in final file system image. Recipes includes meta data for the SW such as where we download upstream sources from, build or runtime dependencies, what feature we enable in our application, configuration, compilation options, define what files goes in to what output package]
Layers (bblayers): Are repositories that contain related metadata, set of recipes
Now let us start with its concept & workflow of yocto projects. Here are the different stages of Yocto build compilation / Build
Work-flow
- Source fetching
- This stage fetches source for the compilation either from local copy or upstream sources.
- Configuration / Metadata
- This stage configures policy and set respective flags for the compilation
- Build / Compile / packaging
- This compiles the source and generates packages as .rpm, .deb or .ipk formats
- QA Checks
- This performs basic checks on packages
- Image / SW generation
- This generates final image which includes rootfs, kernel and all application software. This image can be flashed to embedded product directly
Now let us start with using Yocto.
First download Latest Yocto Source from the following link:
http://downloads.yoctoproject.org/releases/yocto/yocto-3.1/poky-dunfell-23.0.0.tar.bz2
Extract the source to work directory, after extracting we can see the following tree structure.
├── bitbake ------------------ This is main build engine for the build system
├── contrib
├── documentation ----------- Contains documentation for the Yocto project
├── LICENSE
├── LICENSE.GPL-2.0-only
├── LICENSE.MIT
├── MEMORIAM
├── meta -------------------- This is OpenEmbedded Core which incudes base recipes
├── meta-poky --------------- This is meta data for poky reference distro
├── meta-selftest
├── meta-skeleton
├── meta-yocto-bsp ---------- This is BSP layer which supports following boards beaglebone-yocto, edgerouter, genericx86 & genericx86-64
├── oe-init-build-env ------- Initial script which sets all environment variable and initialize build environment , This is first script which has to be executed before any other command in the yocto
├── README.hardware -> meta-yocto-bsp/README.hardware
├── README.OE-Core
├── README.poky -> meta-poky/README.poky
├── README.qemu
└── scripts
Please remember this Yocto download is just base version of Yocto project which enables developer to create Linux distribution for any of the following four boards types
* beaglebone
* edgerouter
* Genericx86
* genericX86-64
As we can see here in meta-yocto-bsp layer it supports the above mentioned embedded platforms. Refer meta-yocto-bsp/README.hardware file for more detail
Now, let say for example we want to build for Raspberry Pi embedded platform which is very famous hence all necessary BSP and application software are available in Yocto project.
Consider we want to build for Raspberry Pi 3 Model B board and here is the board configurations
Version : Raspberry pi 3 Model B
Quad-core 64-bit ARM cortex A53 CPU
Clocked at 1.2GHz
New BCM2837 chip
400MHz VideoCore IV GPU
1GB LPDDR2-900 SDRAM
New BCM43438 chip for WiFi/BT
As we can see in downloaded Yocto source we do not see any BSP related to raspberry PI by default , so we need to download from its repository. Yocto provide wide range of layers and recipes for different embedded platform. Here is the link:
http://layers.openembedded.org/layerindex/branch/dunfell/layers/
Please note that selected Branch is dunfell branch , but not any other branch
Now on search bar search for “raspberry”
Here is the link for raspberry pi layers
git://git.yoctoproject.org/meta-raspberrypi
Either we can use git clone to clone raspberry layer repository or just use the following link for downloading source in .zip
http://git.yoctoproject.org/clean/cgit.cgi/meta-raspberrypi/snapshot/meta-raspberrypi-dunfell.tar.gz
Download and extract to poky-dunfell-23.0.0 directory,Now we are ready for the compilation. First step is to run the following command
“source oe-init-build-env”
Which will create build directory and it will change its directory to build directory.If you want different directory name than build then execute the following command
“source oe-init-build-env buildRaspberryPi”
Which will create buildRaspberryPi directory and change directory to that one.
Now we need to configure the board type. Which we can configure using MACHINE variable in conf/local.conf file
Set this variable in local.conf file
MACHINE ??= "raspberrypi3-64"
Now we can see in the same file there is one variable called PACKAGE_CLASSES using which we can configure which type of packages we want to use, user can configure its package as .rpm, .deb or .ipk format.
PACKAGE_CLASSES ?= "package_rpm"
So , by default yocto will only look in to the following bblayers when searching for recipes, check conf/bblayers.conf file
poky-dunfell-23.0.0/meta
poky-dunfell-23.0.0/meta-poky
poky-dunfell-23.0.0/meta-yocto-bsp
Now we also need to include downloaded raspberry pi layer so add that layer to conf/bblayers.conf as the following
poky-dunfell-23.0.0/meta
poky-dunfell-23.0.0/meta-poky
poky-dunfell-23.0.0/meta-yocto-bsp
poky-dunfell-23.0.0/meta-raspberrypi-dunfell
Now we are ready for image compilation,There are multiple targets to build, so when we executed the command “source oe-init-build-env” it had printed the following available targets
Common targets are:
core-image-minimal
core-image-sato
meta-toolchain
meta-ide-support
core-image-minimal - Using this target we can create minimal image which doesn’t include any GUI components
core-image-sato - Using this target we can create full image which also includes GUI components
Now you will have a question , how are these targets configured? Actually building target is also using recipes (.bb files).We can find image recipes here
Target -> Recipe file
core-image-minimal -> meta/recipes-core/images/core-image-minimal.bb
core-image-sato -> meta/recipes-sato/images/core-image-sato.bb
Now one may wonder what features or packages are includes in each target image. We can find that information in image recipe file. Check IMAGE_FEATURE variable which is nothing but describes which features are available in respective target image. So we can see here GUI components like x11 are available in sato image but not in minimal image.
If you want more information about each variable, one can look in to meta/conf/documentation.conf and search for the variable you are looking for
Now one can ask question which are different features are already available which I can configure as a part of my image, here is the answer, open the file meta/classes/core-image.bbclass and search for FEATURE_PACKAGES_<feature name>. This describes mapping between available features and its recipe.
For example IMAGE_FEATURE = x11 corrosponds to FEATURE_PACKAGES_x11 which is nothing but packagegroup-core-x11 recipe
FEATURE_PACKAGES_x11 = "packagegroup-core-x11"
We can find packagegroup-core-x11.bb at the following location meta/recipes-graphics/packagegroups/packagegroup-core-x11.bb
So now we are clear which new feature or package we can add as a part of target image. Now let see how to build / compile image.
Execute the following command from the build directory.
bitbake core-image-minimal
If you see error of bitbake not found then go to main Yocto work directory and execute “source oe-init-build-env” then execute above command
For the first time compilation it will take long time (few hours). After compilation is done it will store final SD card image in the following directory tmp/deploy/images/raspberrypi3-64/<image name>.rpi-sdimg
Now we can use dd command to flash the image on to SD card
For example if SD card is at /dev/sdb
Then execute the following command
dd if=<image name>.rpi-sdimg of=/dev/sdb
Once it is flashed just insert SD card to raspberry Pi board and you are done. We can see raspberry Pi sunning your own Linux distribution.
