Plugins of Shopware (6.4) are not updated in the "correct" order and therefore possible trigger errors

[func0der]

Deployer Version

v7.5.5

Target OS

22.04.5

Which PHP version are you using?

PHP 8.1

Content of deploy.php or deploy.yaml

There is basically nothing special in here. I just replaced the "no database" part on the build agent with one on the server, since Shopware 6.4 does not really support that setup properly. It always tries to connect to the database to find "all" plugins and gives out "connection issues" on the build agent.

Anyhow, this makes the deployment and the commands more reliable in this case, since everything it needs is available on the server for sure.

<?php
namespace Deployer;

require 'recipe/shopware.php';


// Probably unrelated to the issue - START
// Not compatible with shopware 6.4
task('sw-build-without-db:get-remote-config')->disable();
task('sw-build-without-db:build')->disable();

task('deploy:clear_paths:find_paths', static function() {
    $output = run("cd {{release_path}} && find . -type d -name node_modules -not -wholename '*/node_modules/*/node_modules'");

    set('clear_paths', explode("\n", $output));
});

before('deploy:clear_paths', 'deploy:clear_paths:find_paths');

task('sw-build-with-db:build', static function () {
    run('cd {{release_path}} && PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=1 CI=1 SHOPWARE_SKIP_BUNDLE_DUMP=1 ./bin/build-js.sh');
});
task('sw-build-with-db', [
    'sw:cache:clear',
    'sw-build-with-db:build',
]);
// Probably unrelated to the issue - END

// Hooks

after('sw:plugin:update:all', 'sw-build-with-db');
after('deploy:failed', 'deploy:unlock');

Steps to reproduce

1. Setup Shopware 6.4
2. Install two plugins: dne/custom-css-js in version 2.0.3 and acris/tiny-mce in version 2.1.5′ ¹
3. Update dne/custom-css-js to version 3.0.1 and acris/tiny-mce to 2.1.9
4. Deploy it somewhere so that the sw:plugin:update:all task (https://github.com/deployphp/deployer/blob/master/recipe/shopware.php#L117) runs. ²

You should get an error messages along the lines of:

cd /app/releases/21 && /usr/local/bin/php /app/releases/21/bin/console plugin:update AcrisTinyMceCS
 Shopware Plugin Lifecycle Service
 =================================
  Update 1 plugin(s):
  * Tiny Mce (v2.1.5)
 20:04:54 CRITICAL  [console] Error thrown while running command "plugin:update AcrisTinyMceCS". Message: "Unable to compile the theme "Theme STRATUS | Pro - Set1". Error while trying to concatenate Styles: An exception occurred while executing a query: SQLSTATE[42S22]: Column not found: 1054 Unknown column 'dne_custom_js_css.description' in 'field list'" ["exception" => Shopware\Storefront\Theme\Exception\ThemeCompileException { …},"command" => "plugin:update AcrisTinyMceCS","message" => "Unable to compile the theme "Theme STRATUS | Pro - Set1". Error while trying to concatenate Styles: An exception occurred while executing a query: SQLSTATE[42S22]: Column not found: 1054 Unknown column 'dne_custom_js_css.description' in 'field list'"]

---

¹ Both of the extensions are payed ones, but those are the only ones I got right now. dne_custom_js_css has been open source before and should be available. I think this is easily reproducable with any plugin that comes alphabetically before the DNE one in the plugin:list

² I don't know if that is important, but i could be, because of caches and stuff. An update of acris/checkout-voucher before acris/tiny-mce is actually successful. But that could also be related to it now having any frontend assets (css/js) in the version I am using.

[func0der]

Reason being, that dne/custom-css-js adds a new column description in its migration in between the version 2.0.3 and 3.0.1 (not sure exactly where, but it does not really matter).

Since plugin:list command is alphabetically ordered we are getting the tinymce plugin update first. In that update the theme compiling or probably the asset build is triggered. This leads to theme compilation, which in terms tries to access the table of dne/custom-css-js, which had not run its migration and therefore fails.

It is probably a shopware problem that we are missing the proper tools to migrate plugins in a specific order, if that is at all possible without having dependency hell.
But since this task is thankfully part of the deployer recipe, we could have some kind of failsafe mechanism to avoid getting stuck with "broken" update chains.

Since this is not only a problem with deployments, but also with local mass updates of plugins or the update to a newer shopware versions, if you are using * for the plugin's version constraints, I adapted the task content into a console command.

Everything inside is basically the same, but I added a wrapper for retries:

        $updatedPluginCount = 0;
        $skippedPlugins = new EntityCollection();
        $failSafe = $plugins->count() * 3; // Three times be a little excessive, but locally I have the "time"
        do {
            foreach ($plugins as $key => $plugin) {
                try {
                    $this->pluginLifecycleService->updatePlugin($plugin, $context);
                } catch (Exception $exception) {
                    $io->note(sprintf('Experienced error while updating plugin "%s" update. Defering. Error: %s', $plugin->getName(), $exception->getMessage()));

                    $skippedPlugins->set($key, $plugin);
                    continue;
                }
                ++$updatedPluginCount;

               $io->text(sprintf('Plugin "%s" has been updated successfully.', $plugin->getName()));
            }

            $plugins = $plugins->filter(fn(Entity $entity) => in_array($entity, $skippedPlugins->getElements()));
            $skippedPlugins->clear();
            $failSafe--;
        } while (count($plugins) > 0 && $failSafe > 0);

        if ($failSafe === 0) { // in this case we need to abort deployment with an error
            $io->error('Something went seriously wrong, we hit the fail safe.');
        }

        if ($updatedPluginCount !== 0) {
            $io->success(sprintf('Updated %d plugin(s).', $updatedPluginCount));
        }

(there is more plugin checks in there, but it is not important for the logic I am trying to convey).

The output of the plugin update commands should be kept, so that the deployment log is a testament of the way the update process went. But the != 0 exit codes should not lead to an aborted deployment.

I can not see any other viable option for this scenario, if you do not want to deploy EACH plugin update separately or define a update order (each time) you are deploying plugin updates.

What do you guys think?