Skip to content

Latest commit

 

History

History

Lesson_Build_tools

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
UefiLessonsPkg/BuildOptionsApp
UefiLessonsPkg/BuildOptionsApp
 
 
README.md
README.md
 
 

README.md

In this lesson we are going to look how EDKII configures its build tools.

As you might remember when you source edksetup.sh script for the first time, this script creates Conf folder from the contents inside the https://github.com/tianocore/edk2/tree/master/BaseTools/Conf directory.

In this lesson we are going to look at the content of the Conf/tools_def.txt file which is created from the https://github.com/tianocore/edk2/blob/master/BaseTools/Conf/tools_def.template.

This file defines configurations for the edk2 build tools.

Every configuration is representent in a format TARGET_TOOLCHAIN_ARCH_COMMANDTYPE_ATTRIBUTE, for example:

RELEASE_VS2013x86_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Gw

In this example:

TARGET = RELEASE
TOOLCHAIN = VS2013x86
ARCH = X64
COMMANDTYPE = CC
ATTRIBUTE = FLAGS

This means, when the project is build in RELEASE mode with a VS2013x86 toolchain for X64 architecture use the FLAGS flags for the compiler CC.

If we use * in place of some part of the format string, it means for every. For example:

*_GCC5_X64_NASM_FLAGS            = -f elf64

This means, when the project is build with a GCC5 toolchain for the X64 architecture use the FLAGS flags for the nasm assembler NASM. And this configuration would be active for every build mode (RELEASE/DEBUG/NOOPT).

There can be many * in the configuration string. For example:

*_*_*_BROTLI_PATH        = BrotliCompress

This means that the PATH attribute for the BROTLI command is BrotliCompress. And this is true for builds for any arch, toolchain or build mode.

Another example:

*_CLANG38_*_MAKE_PATH               = make

I guess no explanation is needed at this point. I just wanted to point out, that the * could be in any part of the format string.

DEF and ENV

The tools_def.txt file has several syntactic constructions to help format strings definition.

DEFINE statement is used to create a named data that can be later expanded via DEF statement. This is similar to the C language define statemnt. For example:

DEFINE IASL_OUTFLAGS           = -p
...
*_GCC49_*_ASL_OUTFLAGS         = DEF(IASL_OUTFLAGS)

There can be chains of defines:

DEFINE GCC_ASM_FLAGS             = -c -x assembler -imacros AutoGen.h
DEFINE GCC48_ASM_FLAGS           = DEF(GCC_ASM_FLAGS)
DEFINE GCC49_ASM_FLAGS           = DEF(GCC48_ASM_FLAGS)
DEFINE GCC5_ASM_FLAGS            = DEF(GCC49_ASM_FLAGS)
*_GCC5_X64_ASM_FLAGS             = DEF(GCC5_ASM_FLAGS) -m64

Another syntactic construction is ENV. It is used when the value should be obtained from the environment:

*_GCC5_ARM_CC_PATH               = ENV(GCC5_ARM_PREFIX)gcc

This way if we cross compile EDKII for the ARM architecture we can set gcc prefix:

export GCC5_ARM_PREFIX=<...>

Precedence rules

The more defined statements override the more common ones. The tools_def.txt has a comment about the override rules:

TARGET_TOOLCHAIN_ARCH_COMMANDTYPE_ATTRIBUTE (Highest)
******_TOOLCHAIN_ARCH_COMMANDTYPE_ATTRIBUTE
TARGET_*********_ARCH_COMMANDTYPE_ATTRIBUTE
******_*********_ARCH_COMMANDTYPE_ATTRIBUTE
TARGET_TOOLCHAIN_****_COMMANDTYPE_ATTRIBUTE
******_TOOLCHAIN_****_COMMANDTYPE_ATTRIBUTE
TARGET_*********_****_COMMANDTYPE_ATTRIBUTE
******_*********_****_COMMANDTYPE_ATTRIBUTE
TARGET_TOOLCHAIN_ARCH_***********_ATTRIBUTE
******_TOOLCHAIN_ARCH_***********_ATTRIBUTE
TARGET_*********_ARCH_***********_ATTRIBUTE
******_*********_ARCH_***********_ATTRIBUTE
TARGET_TOOLCHAIN_****_***********_ATTRIBUTE
******_TOOLCHAIN_****_***********_ATTRIBUTE
TARGET_*********_****_***********_ATTRIBUTE
******_*********_****_***********_ATTRIBUTE (Lowest)

Tools and flags review

Here is some peak on the parameters that currently can be defined for the commands:

TARGET = 
  RELEASE | DEBUG | NOOPT

TOOLCHAIN = 
  VS2008 | VS2008x86 | VS2010 | VS2010x86 | VS2012 | VS2012x86 | VS2013 | VS2013x86 | VS2015 | VS2015x86 | VS2017 | VS2019 
  GCC48 | GCC49 | GCC5
  CLANG35 | CLANG38 | CLANGPDB | CLANGDWARF
  XCODE5
  RVCT | RVCTLINUX | RVCTCYGWIN	                       # ARM RealView Tools Windows | ARM RealView Tools Linux | ARM RealView Tools - Cygwin

ARCH = 
  IA32 | X64 | EBC | AARCH64 | ARM | RISCV64

ATTRIBUTE = 
  PATH | FLAGS | OUTFLAGS | XIPFLAGS | GUID | DLL | FAMILY | BUILDRULEFAMILY

And here are descriptions for some of the common commands:

COMMANDTYPE =
  APP			# C compiler for applications
  ASL			# ACPI Compiler for generating ACPI tables
  ASLCC			# ACPI Table C compiler
  ASLDLINK		# ACPI Table C Dynamic linker
  ASLPP			# ASL C pre-processor
  ASM			# A Macro Assembler for assembly code in some libraries
  ASMLINK		# The Linker to use for assembly code generated by the ASM tool
  CC			# C compiler for PE32/PE32+/Coff images
  DLINK			# The C dynamic linker
  MAKE			# Required for tool chains. This identifies the utility used to process the Makefiles generated by the first phase of the build
  PCH			# The compiler for generating pre-compiled headers
  PP			# The C pre-processor command
  SLINK			# The C static linker
  TIANO			# This special keyword identifies a compression tool used to generate compression sections as well as the library needed to uncompress an image in the firmware volume
  VFR			# The VFR file compiler which creates IFR code
  VFRPP			# The C pre-processor used to process VFR files

Setting TARGET/TOOLCHAIN/ARCH for the project

Configuration strings are represented in the form TARGET_TOOLCHAIN_ARCH_COMMANDTYPE_ATTRIBUTE.

The first three parameters TARGET/TOOLCHAIN/ARCH are usually constant for the build environment, so we've even fixed them in our Conf/target.txt file:

...
TARGET                = RELEASE
TARGET_ARCH           = X64
TOOL_CHAIN_TAG        = GCC5
...

But as you remember we can always override them from command prompt to build different configurations:

build <...> --arch=X64 --buildtarget=RELEASE --tagname=GCC5

Modify build tools attributes via INF files

It is possible to modify build tools options in the INF files.

Let's create new app:

./createNewApp.sh BuildOptionsApp

And add it to our package DSC file UefiLessonsPkg/UefiLessonsPkg.dsc:

[Components]
  ...
  UefiLessonsPkg/BuildOptionsApp/BuildOptionsApp.inf

Add this code to the UefiLessonsPkg/BuildOptionsApp/BuildOptionsApp.c:

#include <Library/UefiBootServicesTableLib.h>
#include <Library/UefiLib.h>

EFI_STATUS
EFIAPI
UefiMain (
  IN EFI_HANDLE        ImageHandle,
  IN EFI_SYSTEM_TABLE  *SystemTable
  )
{
  #ifdef MY_DEBUG
    Print(L"MY_DEBUG is definedi\n");
  #endif
  #ifdef MY_RELEASE
    Print(L"MY_RELEASE is defined\n");
  #endif
  #ifdef MY_ALL_TARGETS
    Print(L"MY_ALL_TARGETS is defined\n");
  #endif

  return EFI_SUCCESS;
}

Build our application:

$ build --buildtarget=RELEASE

And copy result to the QEMU shared folder

$ cp Build/UefiLessonsPkg/RELEASE_GCC5/X64/BuildOptionsApp.efi ~/UEFI_disk/

If you execute our app under UEFI shell now, it wouldn't produce any output as we didn't define any of the defines.

Now let's add this [BuildOptions] section to the application INF file:

[BuildOptions]
  RELEASE_GCC5_X64_CC_FLAGS = "-DMY_RELEASE"
  DEBUG_GCC5_X64_CC_FLAGS = "-DMY_DEBUG"
  *_GCC5_CC_X64_FLAGS = "-DMY_ALL_TARGETS"

Now build and test our application in the RELEASE mode:

$ build --buildtarget=RELEASE
$ cp Build/UefiLessonsPkg/RELEASE_GCC5/X64/BuildOptionsApp.efi ~/UEFI_disk/

This version would give you this output:

FS0:\> BuildOptionsApp.efi
MY_RELEASE is defined
MY_ALL_TARGETS is defined

And if you build with the DEBUG mode:

$ build --buildtarget=DEBUG
$ cp Build/UefiLessonsPkg/DEBUG_GCC5/X64/BuildOptionsApp.efi ~/UEFI_disk/

You would get:

FS0:\> BuildOptionsApp.efi
MY_DEBUG is defined
MY_ALL_TARGETS is defined

If you look closely to the build log you could even see our defines in the EDK2 build output. For example this is a sample from the RELEASE build:

...
"gcc" -MMD -MF /<...>/Build/UefiLessonsPkg/RELEASE_GCC5/X64/UefiLessonsPkg/BuildOptionsApp/BuildOptionsApp/OUTPUT/AutoGen.obj.deps -g -Os -fshort-wchar -fno-builtin -fno-strict-aliasing -Wall -Werror -Wno-array-bounds -include AutoGen.h -fno-common -ffunction-sections -fdata-sections -DSTRING_ARRAY_NAME=BuildOptionsAppStrings -m64 -fno-stack-protector "-DEFIAPI=__attribute__((ms_abi))" -maccumulate-outgoing-args -mno-red-zone -Wno-address -mcmodel=small -fpie -fno-asynchronous-unwind-tables -Wno-address -flto -DUSING_LTO -Os -Wno-unused-but-set-variable -Wno-unused-const-variable "-DMY_RELEASE" "-DMY_ALL_TARGETS" -c -o /<...>/Build/UefiLessonsPkg/RELEASE_GCC5/X64/UefiLessonsPkg/BuildOptionsApp/BuildOptionsApp/OUTPUT/./AutoGen.obj -I/<...>/UefiLessonsPkg/BuildOptionsApp -I/<...>/Build/UefiLessonsPkg/RELEASE_GCC5/X64/UefiLessonsPkg/BuildOptionsApp/BuildOptionsApp/DEBUG -I/<...>/MdePkg -I/<...>/MdePkg/Include -I/<...>/MdePkg/Test/UnitTest/Include -I/<...>/MdePkg/Include/X64 /<...>/Build/UefiLessonsPkg/RELEASE_GCC5/X64/UefiLessonsPkg/BuildOptionsApp/BuildOptionsApp/DEBUG/AutoGen.c
...

The = sign in the our statements actually means "append". This is why we have both defines in the end. If you want to completely override some attribute, you should use "==" syntax. EDK2 has many necessary options for the CC_FLAGS, so you probably shouldn't override this variable, but you might need this functionality for some other attributes.

FAMILY attribute

With the help of a FAMILY attribute, it is possible to define rules for several toolchains. For example if we have these strings in the tools_def.txt file:

*_GCC48_*_*_FAMILY               = GCC
*_GCC49_*_*_FAMILY               = GCC
*_GCC5_*_*_FAMILY                = GCC

Then we can add append like this:

[BuildOptions]
  GCC:*_*_*_CC_FLAGS = "-DMY_FAMILY"

And it would be active if our toolchain one of the GCC48/GCC49/GCC5

Command substitution in the tools options

As we were adding defines to the gcc in our example, I want to point out one interesting usage of this feature.

Look at the https://github.com/tianocore/edk2/blob/master/EmbeddedPkg/Library/VirtualRealTimeClockLib/VirtualRealTimeClockLib.inf BuildOptions section:

[BuildOptions]
  GCC:*_*_*_CC_FLAGS = -DBUILD_EPOCH=`date +%s`

Here you can see that it is even possible to use command substitution in the BuildOptions. Fun, right?

In case you don't know this command gives current date as Unix time (which is the number of (non-leap) seconds since 1970-01-01). Try to execute it in you Linux shell:

$ date +%s
1657034765

In the code [https://github.com/tianocore/edk2/blob/master/EmbeddedPkg/Library/VirtualRealTimeClockLib/VirtualRealTimeClockLib.c] this define is used like this:

EFI_STATUS
EFIAPI
LibGetTime (
  OUT EFI_TIME               *Time,
  OUT EFI_TIME_CAPABILITIES  *Capabilities
  )
{
  UINTN       EpochSeconds;
  ...
  EpochSeconds = BUILD_EPOCH;
  ...
}

Modify build tools attributes via DSC files

Besides the INF files it is also possible to modify build tool attributes via the DSC file.

You can do it globally for all the package modules via the [BuildOptions] section like we did in the INF file:

[BuildOptions]
  RELEASE_GCC5_X64_CC_FLAGS = "-DMY_RELEASE_DSC"

Or only for individual modules:

[Components]
  ...
  
  UefiLessonsPkg/BuildOptionsApp/BuildOptionsApp.inf {
    <BuildOptions>
      RELEASE_GCC5_X64_CC_FLAGS = "-DMY_RELEASE_DSC"
  }

In the first case this would lead to recompilation of all the modules listed in the DSC. And in the second case build system will recompile only one module.

Section tag

In the examples above we've always used [BuildOptions] name for our section. But this is just a short form. The most full form for the section is [BuildOptions.$(arch).CodeBase.ModuleType]. Here are:

  • $(arch) - target architecture. You can use common for any architecture or define particular rules just for particalar one like X64, AARCH64, ...
  • CodeBase - this is equal to the EDKII
  • ModuleType - with this option we can define options for different classes of modules, e.g. DXE_RUNTIME_DRIVER, SMM_CORE. This value corresponds to the MODULE_TYPE value in the INF files

So the section [BuildOptions.X64.EDKII.DXE_RUNTIME_DRIVER] would change build options for the modules of type DXE_RUNTIME_DRIVER if they are compiled for the X64 arch.

The [BuildOptions] is the shortest form, but it is possible to have all intermidiate forms. For example [BuildOptions.X64] section name is allowed.

Also it is possible to list several names as a section name. Look at this as an example:

[BuildOptions.common.EDKII.DXE_RUNTIME_DRIVER, BuildOptions.common.EDKII.COMBINED_SMM_DXE, BuildOptions.common.EDKII.DXE_SMM_DRIVER, BuildOptions.common.EDKII.SMM_CORE, BuildOptions.Common.EDK.DXE_RUNTIME_DRIVER]

With this you could define the same modifiers for different situations.

Override priority

Generally if we use override == syntax, the DSC override would take the precedence over the INF override. But if we use [BuildOptions.$(arch).CodeBase.ModuleType] section names, things get a little bit more complicated.

Here is complete override sequence from the EDKII docs:

HIGHEST PRIORITY
- DSC file's component scoped <BuildOptions> for individual INF files
- [BuildOptions.$(arch).CodeBase.ModuleType]
- [BuildOptions.$(arch).CodeBase]
- [BuildOptions.common.CodeBase]
- [BuildOptions.$(arch)]
- [BuildOptions.common]
- [BuildOptions]
- INF file's [BuildOptions] section
- tools_def.txt
LOWEST PRIORITY

Links

Here are links to the EDKII docs about BuildOptions section: