uuu deploy issue

[agentdenton]

Branch: master

Hello, I have a question about the uuu_git recipe. I'm trying to deploy the uuu binary into the DEPLOY_IMAGE_DIR using the following bbappend file:

FILESEXTRAPATHS:prepend := "${THISDIR}/files:"

inherit deploy

SRC_URI += "file://bootloader.uuu"
SRC_URI += "file://wic.uuu"

do_deploy:class-native() {
    install -d ${DEPLOYDIR}

    # Install flash scripts
    cp ${WORKDIR}/bootloader.uuu ${DEPLOYDIR}
    cp ${WORKDIR}/wic.uuu ${DEPLOYDIR}

    # Install the uuu binary
    install -m 0755 uuu/uuu ${DEPLOYDIR}
}
addtask do_deploy after do_compile

This doesn't work because bitbake fails to generate the 'do_deploy' function, resulting in the error message below.

NOTE: Executing Tasks
ERROR: uuu-native-1.4.165-r0 do_deploy: ExecutionError('/home/orion/orion-bsp/build/tmp/work/x86_64-linux/uuu-native/1.4.165-r0/temp/run.do_deploy.59194', 127, None, None)
ERROR: Logfile of failure stored in: /home/orion/orion-bsp/build/tmp/work/x86_64-linux/uuu-native/1.4.165-r0/temp/log.do_deploy.59194
Log data follows:
| DEBUG: Executing python function sstate_task_prefunc
| DEBUG: Python function sstate_task_prefunc finished
| DEBUG: Executing shell function do_deploy
| WARNING: exit code 127 from a shell command.
| /home/orion/orion-bsp/build/tmp/work/x86_64-linux/uuu-native/1.4.165-r0/temp/run.do_deploy.59194: 140: do_deploy: not found
ERROR: Task (virtual:native:/home/orion/orion-bsp/layers/meta-freescale/recipes-devtools/uuu/uuu_git.bb:do_deploy) failed with exit code '1'
NOTE: Tasks Summary: Attempted 237 tasks of which 229 didn't need to be rerun and 1 failed.

This can be verified by looking at the environment:

bitbake -e uuu-native

The environment:

do_deploy="    install -d /home/orion/orion-bsp/build/tmp/work/x86_64-linux/uuu-native/1.4.165-r0/deploy-uuu-native \
 \
    # Install flash scripts \
    cp /home/orion/orion-bsp/build/tmp/work/x86_64-linux/uuu-native/1.4.165-r0/bootloader.uuu /home/orion/orion-bsp/build/tmp/work/x86_64-linux/uuu-native/1.4.165-r0/deploy-uuu-native \
    cp /home/orion/orion-bsp/build/tmp/work/x86_64-linux/uuu-native/1.4.165-r0/wic.uuu /home/orion/orion-bsp/build/tmp/work/x86_64-linux/uuu-native/1.4.165-r0/deploy-uuu-native \
 \
    # Install the uuu binary for Linux \
    install -m 0755 uuu/uuu /home/orion/orion-bsp/build/tmp/work/x86_64-linux/uuu-native/1.4.165-r0/deploy-uuu-native \

If I remove the :class-native modifier, then everything works the way it's intended. However, that means I can't build uuu for the target anymore because uuu-native has already populated the DEPLOY_IMAGE_DIR:

bitbake uuu

Error message:

NOTE: Executing Tasks
ERROR: uuu-1.4.165-r0 do_deploy_setscene: The recipe uuu is trying to install files into a shared area when those files already exist. Those files and their manifest location are:
  /home/orion/orion-bsp/build/tmp/deploy/images/imx6sxea-com/wic.uuu
    (matched in manifest-imx6sxea_com-uuu-native.deploy)
  /home/orion/orion-bsp/build/tmp/deploy/images/imx6sxea-com/bootloader.uuu
    (matched in manifest-imx6sxea_com-uuu-native.deploy)
  /home/orion/orion-bsp/build/tmp/deploy/images/imx6sxea-com/uuu
    (matched in manifest-imx6sxea_com-uuu-native.deploy)
Please verify which recipe should provide the above files.

The build has stopped, as continuing in this scenario WILL break things - if not now, possibly in the future (we've seen builds fail several months later). If the system knew how to recover from this automatically it would, however there are several different scenarios which can
 result in this and we don't know which one this is. It may be you have switched providers of something like virtual/kernel (e.g. from linux-yocto to linux-yocto-dev), in that case you need to execute the clean task for both recipes and it will resolve this error. It may be you
changed DISTRO_FEATURES from systemd to udev or vice versa. Cleaning those recipes should again resolve this error, however switching DISTRO_FEATURES on an existing build directory is not supported - you should really clean out tmp and rebuild (reusing sstate should be safe). It
 could be the overlapping files detected are harmless in which case adding them to SSTATE_ALLOW_OVERLAP_FILES may be the correct solution. It could also be your build is including two different conflicting versions of things (e.g. bluez 4 and bluez 5 and the correct solution for
 that would be to resolve the conflict. If in doubt, please ask on the mailing list, sharing the error and filelist above.
ERROR: uuu-1.4.165-r0 do_deploy_setscene: If the above message is too much, the simpler version is you're advised to wipe out tmp and rebuild (reusing sstate is fine). That will likely fix things in most (but not all) cases.
WARNING: Logfile for failed setscene task is /home/orion/orion-bsp/build/tmp/work/cortexa9t2hf-neon-poky-linux-gnueabi/uuu/1.4.165-r0/temp/log.do_deploy_setscene.59383
WARNING: Setscene task (/home/orion/orion-bsp/layers/meta-freescale/recipes-devtools/uuu/uuu_git.bb:do_deploy_setscene) failed with exit code '1' - real task will be run instead
ERROR: uuu-1.4.165-r0 do_deploy: The recipe uuu is trying to install files into a shared area when those files already exist. Those files and their manifest location are:
  /home/orion/orion-bsp/build/tmp/deploy/images/imx6sxea-com/wic.uuu
    (matched in manifest-imx6sxea_com-uuu-native.deploy)
  /home/orion/orion-bsp/build/tmp/deploy/images/imx6sxea-com/bootloader.uuu
    (matched in manifest-imx6sxea_com-uuu-native.deploy)
  /home/orion/orion-bsp/build/tmp/deploy/images/imx6sxea-com/uuu
    (matched in manifest-imx6sxea_com-uuu-native.deploy)
Please verify which recipe should provide the above files.

The build has stopped, as continuing in this scenario WILL break things - if not now, possibly in the future (we've seen builds fail several months later). If the system knew how to recover from this automatically it would, however there are several different scenarios which can
 result in this and we don't know which one this is. It may be you have switched providers of something like virtual/kernel (e.g. from linux-yocto to linux-yocto-dev), in that case you need to execute the clean task for both recipes and it will resolve this error. It may be you
changed DISTRO_FEATURES from systemd to udev or vice versa. Cleaning those recipes should again resolve this error, however switching DISTRO_FEATURES on an existing build directory is not supported - you should really clean out tmp and rebuild (reusing sstate should be safe). It
 could be the overlapping files detected are harmless in which case adding them to SSTATE_ALLOW_OVERLAP_FILES may be the correct solution. It could also be your build is including two different conflicting versions of things (e.g. bluez 4 and bluez 5 and the correct solution for
 that would be to resolve the conflict. If in doubt, please ask on the mailing list, sharing the error and filelist above.
ERROR: uuu-1.4.165-r0 do_deploy: If the above message is too much, the simpler version is you're advised to wipe out tmp and rebuild (reusing sstate is fine). That will likely fix things in most (but not all) cases.
ERROR: Logfile of failure stored in: /home/orion/orion-bsp/build/tmp/work/cortexa9t2hf-neon-poky-linux-gnueabi/uuu/1.4.165-r0/temp/log.do_deploy.60115
ERROR: Task (/home/orion/orion-bsp/layers/meta-freescale/recipes-devtools/uuu/uuu_git.bb:do_deploy) failed with exit code '1'
NOTE: Tasks Summary: Attempted 1157 tasks of which 1149 didn't need to be rerun and 1 failed.

So, the :class-native modifier does make sense here but doesn't work. Obviously, it's quite rare to install a flash tool onto the target as well, but nevertheless, it should work.

I'm not sure if I'm doing something wrong or it is a bug. If possible can someone clarify this for me?

[angolini]

Could you please share what are the bitbake commands that you've been using?

When I use bitbake core-image-base -c populate_mfgtool I can get the final image and the uuu binaries installed in the bundle.

[agentdenton]

I don't have a task like that available:

NOTE: Running task 1 of 1 (/home/imx/imx-dev/layers/poky/meta/recipes-core/images/core-image-base.bb:do_listtasks)
NOTE: recipe core-image-base-1.0-r0: task do_listtasks: Started
do_build                              Default task for a recipe - depends on all other normal tasks required to 'build' a recipe
do_checkuri                           Validates the SRC_URI value
do_clean                              Removes all output files for a target
do_cleanall                           Removes all output files, shared state cache, and downloaded source files for a target
do_cleansstate                        Removes all output files and shared state cache for a target
do_compile                            Compiles the source in the compilation directory
do_configure                          Configures the source by enabling and disabling any build-time and configuration options for the software being built
do_deploy_source_date_epoch
do_deploy_source_date_epoch_setscene   (setscene version)
do_devshell                           Starts a shell with the environment set up for development/debugging
do_fetch                              Fetches the source code
do_flush_pseudodb
do_image
do_image_complete
do_image_complete_setscene             (setscene version)
do_image_qa
do_image_qa_setscene                   (setscene version)
do_image_wic
do_install                            Copies files from the compilation directory to a holding area
do_listtasks                          Lists all defined tasks for a target
do_package                            Analyzes the content of the holding area and splits it into subsets based on available packages and files
do_package_qa_setscene                Runs QA checks on packaged files (setscene version)
do_package_setscene                   Analyzes the content of the holding area and splits it into subsets based on available packages and files (setscene version)
do_package_write_ipk_setscene         Creates the actual IPK packages and places them in the Package Feed area (setscene version)
do_packagedata_setscene               Creates package metadata used by the build system to generate the final packages (setscene version)
do_patch                              Locates patch files and applies them to the source code
do_populate_lic_deploy
do_populate_lic_setscene              Writes license information for the recipe that is collected later when the image is constructed (setscene version)
do_populate_sdk                       Creates the file and directory structure for an installable SDK
do_populate_sdk_ext
do_populate_sdk_setscene              Creates the file and directory structure for an installable SDK (setscene version)
do_populate_sysroot_setscene          Copies a subset of files installed by do_install into the sysroot in order to make them available to other recipes (setscene version)
do_prepare_recipe_sysroot
do_pydevshell                         Starts an interactive Python shell for development/debugging
do_rootfs                             Creates the root filesystem (file and directory structure) for an image
do_rootfs_wicenv
do_sdk_depends
do_unpack                             Unpacks the source code into a working directory
do_write_wks_template
NOTE: recipe core-image-base-1.0-r0: task do_listtasks: Succeeded
NOTE: Tasks Summary: Attempted 1 tasks of which 0 didn't need to be rerun and all succeeded.
NOTE: Writing buildhistory
NOTE: Writing buildhistory took: 2 seconds

I have a very minimal project with a custom image and a custom layer because I'm trying to understand the overall NXP BSP infrastructure

Here is my custom image:

# imx-image-dev.bb

require recipes-fsl/images/fsl-image-multimedia.bb

SUMMARY = "A console-only development image for I.MX SoC"

LICENSE = "MIT"

IMAGE_FEATURES += "ssh-server-openssh"
IMAGE_FEATURES += "dev-pkgs"

DEPENDS += "uuu-native"

I'm using kas for layer management; here is the generated bblayers.conf:

BBLAYERS ?= " \
    /home/imx/imx-dev/layers/meta-freescale \
    /home/imx/imx-dev/layers/meta-freescale-distro \
    /home/imx/imx-dev/layers/meta-imx-extra \
    /home/imx/imx-dev/layers/meta-openembedded/meta-multimedia \
    /home/imx/imx-dev/layers/meta-openembedded/meta-oe \
    /home/imx/imx-dev/layers/meta-openembedded/meta-python \
    /home/imx/imx-dev/layers/poky/meta \
    /home/imx/imx-dev/layers/poky/meta-poky \
    /home/imx/imx-dev/layers/poky/meta-selftest \
    /home/imx/imx-dev/layers/poky/meta-skeleton \
    /home/imx/imx-dev/layers/poky/meta-yocto-bsp"
BBPATH ?= "${TOPDIR}"
BBFILES ??= ""

My custom layer's name is meta-imx-extra. To build the project, I use the commands below:

source layers/poky/oe-init-build-env
bitbake -k imx-image-dev

Bitbake builds the uuu-native package and puts the output inside the x86_64-linux directory. However, I wanted to place the uuu binary into the DEPLOY_DIR_IMAGE automatically. So, I created the uuu_git.bbappend file:

inherit deploy

do_deploy() {
    install -d ${DEPLOYDIR}
    install -m 0755 uuu/uuu ${DEPLOYDIR}
}
addtask do_deploy after do_compile

Because of this 'deploy task', I can't build the uuu_git recipe for the target anymore. If I try, Bitbake will attempt to place the 32-bit uuu binary inside DEPLOY_DIR_IMAGE, which already contains the native uuu binary. So, I tried adding :class-native to do_deploy to separate deploy tasks for native and target versions.

I'm fully aware that I'm doing everything in an unconventional manner regarding NXP BSP development. I encountered this issue while experimenting with the meta-freescale layer. Also, English isn't my native language, so I may have phrased my questions in a confusing manner. Sorry for that.

I tried using 'grep' to find 'populate_mfgtool', but came up empty. Could you tell me where it's defined so I can look at the implementation?

[angolini]

Take a look here https://github.com/Freescale/meta-freescale/blob/master/classes/image_populate_mfgtool.bbclass

I think you can try to inherit the class, so you can generate the mfgtool bundle with the uuu binary

[agentdenton]

Thanks, I'll check it out.

[agentdenton]

Sorry, I left the discussion unanswered. The project I'm working on requires an older version of Yocto. I cannot use the image_populate_mfgtool class, so I ended up writing a custom recipe that deploys the uuu binaries to the DEPLOY_DIR_IMAGE only for the host.