[BUG] fatal error with WPLoader on 4.3.0, LoadSandbox tries to use DB from wp-config, not test config

[andronocean]

Version 3.5

No, new bug in 4.3.0

Environment

OS: macOS 14.6.1
PHP version: 8.2.22
Installed Codeception version: 5.1.2
Installed wp-browser version: 4.3.0
WordPress version: 6.6.1
Local development environment: Roots' Trellis VM environment using Lima as VM manager
WordPress structure and management: Bedrock

Can you perform the test manually?

Yes.

Codeception configuration file

Paste, in a fenced YAML block, the content of your Codeception configuration file; remove any sensitive data!

namespace: Example\Tests
support_namespace: Support
paths:
    tests: tests
    output: tests/_output
    data: tests/Support/Data
    support: tests/Support
    envs: tests/_envs
actor_suffix: Tester
params:
    - tests/.env
extensions:
    enabled:
        - Codeception\Extension\RunFailed
        - lucatume\WPBrowser\Extension\ChromeDriverController
        - lucatume\WPBrowser\Extension\BuiltInServerController
        - lucatume\WPBrowser\Extension\IsolationSupport
    config:
        lucatume\WPBrowser\Extension\ChromeDriverController:
            port: '%CHROMEDRIVER_PORT%'
            suites:
                - EndToEnd
        lucatume\WPBrowser\Extension\BuiltInServerController:
            suites:
                - Integration
            workers: 5
            port: '%BUILTIN_SERVER_PORT%'
            docroot: '%WORDPRESS_DOCROOT%'
            env:
                DATABASE_TYPE: sqlite
                DB_ENGINE: sqlite
                DB_DIR: '%codecept_root_dir%/tests/Support/Data'
                DB_FILE: db.sqlite
                WPBROWSER_SITEURL: '%WORDPRESS_URL%/wp'
                WPBROWSER_HOMEURL: '%WORDPRESS_URL%'
    commands:
        - lucatume\WPBrowser\Command\RunOriginal
        - lucatume\WPBrowser\Command\RunAll
        - lucatume\WPBrowser\Command\GenerateWPUnit
        - lucatume\WPBrowser\Command\DbExport
        - lucatume\WPBrowser\Command\DbImport
        - lucatume\WPBrowser\Command\MonkeyCachePath
        - lucatume\WPBrowser\Command\MonkeyCacheClear
        - lucatume\WPBrowser\Command\DevStart
        - lucatume\WPBrowser\Command\DevStop
        - lucatume\WPBrowser\Command\DevInfo
        - lucatume\WPBrowser\Command\DevRestart
        - lucatume\WPBrowser\Command\ChromedriverUpdate

Suite configuration file

Paste, in a fenced YAML block, the content of the suite configuration file; remove any sensitive data!

# End-to-End suite configuration
#
# Run full-system tests for user flows in a real browser.
# These tests send requests to a production-like WordPress test installation and manipulate a fully interactive page.

actor: EndToEndTester
suite_namespace: Example\Tests\EndToEnd
bootstrap: _bootstrap.php
modules:
    enabled:
        - lucatume\WPBrowser\Module\WPWebDriver
        - lucatume\WPBrowser\Module\WPDb
        - lucatume\WPBrowser\Module\WPFilesystem
        - lucatume\WPBrowser\Module\WPLoader
    config:
        lucatume\WPBrowser\Module\WPWebDriver:
            url: '%WORDPRESS_E2E_URL%'
            adminUsername: '%WORDPRESS_E2E_ADMIN_USER%'
            adminPassword: '%WORDPRESS_E2E_ADMIN_PASSWORD%'
            adminPath: '%WORDPRESS_E2E_ADMIN_PATH%'
            browser: chrome
            host: '%CHROMEDRIVER_HOST%'
            port: '%CHROMEDRIVER_PORT%'
            path: '/'
            window_size: 1200x1000
            capabilities:
              acceptInsecureCerts: true
              "goog:chromeOptions":
                args:
                  - "--headless=new"
                  - "--disable-gpu"
                  - "--disable-dev-shm-usage"
                  - "--proxy-server='direct://'"
                  - "--proxy-bypass-list=*"
                  - "--no-sandbox"
                  - "user-agent=HeadlessChromeDriver"
        lucatume\WPBrowser\Module\WPDb:
            dbUrl: 'mysql://%WORDPRESS_E2E_DB_USER%:%WORDPRESS_E2E_DB_PASSWORD%@%WORDPRESS_E2E_DB_HOST%:3306/%WORDPRESS_E2E_DB_NAME%'
            dump: 'tests/Support/Data/dump.sql'
            populate: true
            cleanup: true
            reconnect: false
            url: '%WORDPRESS_E2E_URL%'
            urlReplacement: false
            tablePrefix: '%WORDPRESS_TABLE_PREFIX%'
        lucatume\WPBrowser\Module\WPFilesystem:
            wpRootFolder: '%WORDPRESS_ROOT_DIR%'
        lucatume\WPBrowser\Module\WPLoader:
            loadOnly: true
            wpRootFolder: '%WORDPRESS_ROOT_DIR%'
            dbUrl: 'mysql://%WORDPRESS_E2E_DB_USER%:%WORDPRESS_E2E_DB_PASSWORD%@%WORDPRESS_E2E_DB_HOST%:3306/%WORDPRESS_E2E_DB_NAME%'
            domain: '%WORDPRESS_E2E_DOMAIN%'

For completeness, here’s my tests/.env file too:

# TESTING ENVIRONMENT CONFIGURATION FOR WP-BROWSER AND CODECEPTION

# The path to the WordPress root directory, the one containing the wp-load.php file.
# This can be a relative path from the directory that contains the codeception.yml file,
# or an absolute path.
WORDPRESS_ROOT_DIR=web/wp

# Tests will require a MySQL database to run.
# Integration tests use a local SQLite database for speed
# The database will be created if it does not exist.
# Do not use a database that contains important data!
WORDPRESS_DB_URL=sqlite://%codecept_root_dir%/tests/Support/Data/db.sqlite

# The Integration suite will use this table prefix for the WordPress tables.
TEST_TABLE_PREFIX=test_

# This table prefix used by the WordPress site in end-to-end tests.
WORDPRESS_TABLE_PREFIX=wp_

# The URL and domain of the WordPress site used in integration tests (uses the builtin PHP server)
WORDPRESS_URL=http://localhost:33893
WORDPRESS_DOMAIN=localhost:33893
WORDPRESS_ADMIN_PATH=/wp/wp-admin

# The username and password of the administrator user of the WordPress site used in integration tests.
WORDPRESS_ADMIN_USER=admin
WORDPRESS_ADMIN_PASSWORD=password

# The port on which the PHP built-in server will serve the WordPress installation.
BUILTIN_SERVER_PORT=33893

# The path to the directory that should be served on localhost, the one containing the wp-config.php file.
WORDPRESS_DOCROOT=web

# The host and port of the ChromeDriver server that will be used in end-to-end tests.
CHROMEDRIVER_HOST=localhost
CHROMEDRIVER_PORT=9515

# The URL and domain of the WordPress site used in end-to-end tests, running on our development VM
WORDPRESS_E2E_URL=https://example.test
WORDPRESS_E2E_DOMAIN=example.test
WORDPRESS_E2E_ADMIN_PATH=/wp/wp-admin

# End-to-end tests use a separate database on the development VM:
WORDPRESS_E2E_DB_HOST=192.168.64.10
WORDPRESS_E2E_DB_NAME=example_com_test_suite
WORDPRESS_E2E_DB_USER=wpbrowser
WORDPRESS_E2E_DB_PASSWORD=##########

# The username and password of the administrator user of the WordPress site used in end-to-end tests.
WORDPRESS_E2E_ADMIN_USER=tester
WORDPRESS_E2E_ADMIN_PASSWORD=##########

Describe the bug

I updated to 4.3.0 and my End-to-End and Functional test suites started crashing immediately with a fatal error (see output below). Both of these suites use WPLoader in loadOnly: true mode (difference between them is one uses WPWebDriver and other WPBrowser module)

It appears that the changes to the LoadSandbox class are causing it to load my Bedrock configuration. The sandboxed WP then tries to create a database connection using the credentials in the Bedrock config, instead of connecting to the database I’ve specified in suite config and my tests/.env file. This fails because my Bedrock config is meant to run inside of my local development VM, and therefore it’s set for localhost access to my development database.

As shown in the tests/.env file above, I’ve given WPLoader credentials to access a database running in the VM remotely (WORDPRESS_E2E_DB_HOST=192.168.64.10). (I have it using its own database and a different MySQL user.) The output below shows that instead the sandboxed WordPress is trying to use the 'example_com'@'localhost' user while running on my host Mac.

Output

Here’s the output of vendor/bin/codecept run EndToEnd --debug

Codeception PHP Testing Framework v5.1.2 https://stand-with-ukraine.pp.ua

  [Connecting To Db] {"config":{"tablePrefix":"wp_","populate":true,"cleanup":true,"reconnect":false,"dump":["tests/Support/Data/dump.sql"],"populator":"","urlReplacement":false,"originalUrl":"","waitlock":10,"createIfNotExists":false,"dbUrl":"mysql://wpbrowser:##PASSWORD##@192.168.64.10:3306/example_com_test_suite","url":"https://example.test","dsn":"mysql:host=192.168.64.10;port=3306;dbname=example_com_test_suite","user":"wpbrowser","password":"##PASSWORD##"},"options":[]}
  [Db] Connected to default example_com_test_suite
  The WordPress installation will be loaded after all other modules have been initialized.
  [Query] INSERT INTO `wp_options` (`option_name`, `option_value`, `autoload`) VALUES (?, ?, ?)
  [Parameters] ["admin_email_lifespan",2533080438,"yes"]
  [Query] INSERT INTO `wp_options` (`option_name`, `option_value`, `autoload`) VALUES (?, ?, ?)
  [Parameters] ["_transient_doing_cron",1725462669,"yes"]

In class-wpdb.php line 1982:

  [PHPUnit\Framework\Error\Warning (2)]
  mysqli_real_connect(): (HY000/1045): Access denied for user 'example_com'@'localhost' (using password: YES)


Exception trace:
  at /path/on/host/computer/example/site/web/wp/wp-includes/class-wpdb.php:1982
 Codeception\Subscriber\ErrorHandler->errorHandler() at n/a:n/a
 mysqli_real_connect() at /path/on/host/computer/example/site/web/wp/wp-includes/class-wpdb.php:1982
 wpdb->db_connect() at /path/on/host/computer/example/site/web/wp/wp-includes/class-wpdb.php:767
 wpdb->__construct() at /path/on/host/computer/example/site/web/wp/wp-includes/load.php:697
 require_wp_db() at /path/on/host/computer/example/site/web/wp/wp-settings.php:132
 require_once() at /path/on/host/computer/example/site/web/wp-config.php:9
 require_once() at /path/on/host/computer/example/site/web/wp/wp-load.php:55
 include_once() at /path/on/host/computer/example/site/vendor/lucatume/wp-browser/src/WordPress/LoadSandbox.php:32
 lucatume\WPBrowser\WordPress\LoadSandbox->load() at /path/on/host/computer/example/site/vendor/lucatume/wp-browser/src/Module/WPLoader.php:623
 lucatume\WPBrowser\Module\WPLoader->_loadWordPress() at /path/on/host/computer/example/site/vendor/lucatume/wp-browser/src/Module/WPLoader.php:554
 lucatume\WPBrowser\Module\WPLoader->_beforeSuite() at /path/on/host/computer/example/site/vendor/codeception/codeception/src/Codeception/Subscriber/Module.php:52
 Codeception\Subscriber\Module->beforeSuite() at /path/on/host/computer/example/site/vendor/symfony/event-dispatcher/EventDispatcher.php:206
 Symfony\Component\EventDispatcher\EventDispatcher->callListeners() at /path/on/host/computer/example/site/vendor/symfony/event-dispatcher/EventDispatcher.php:56
 Symfony\Component\EventDispatcher\EventDispatcher->dispatch() at /path/on/host/computer/example/site/vendor/codeception/codeception/src/Codeception/SuiteManager.php:148
 Codeception\SuiteManager->run() at /path/on/host/computer/example/site/vendor/codeception/codeception/src/Codeception/Codecept.php:260
 Codeception\Codecept->runSuite() at /path/on/host/computer/example/site/vendor/codeception/codeception/src/Codeception/Codecept.php:216
 Codeception\Codecept->run() at /path/on/host/computer/example/site/vendor/codeception/codeception/src/Codeception/Command/Run.php:646
 Codeception\Command\Run->runSuites() at /path/on/host/computer/example/site/vendor/codeception/codeception/src/Codeception/Command/Run.php:467
 Codeception\Command\Run->execute() at /path/on/host/computer/example/site/vendor/lucatume/wp-browser/src/Command/RunAll.php:28
 lucatume\WPBrowser\Command\RunAll->execute() at /path/on/host/computer/example/site/vendor/symfony/console/Command/Command.php:326
 Symfony\Component\Console\Command\Command->run() at /path/on/host/computer/example/site/vendor/symfony/console/Application.php:1078
 Symfony\Component\Console\Application->doRunCommand() at /path/on/host/computer/example/site/vendor/symfony/console/Application.php:324
 Symfony\Component\Console\Application->doRun() at /path/on/host/computer/example/site/vendor/symfony/console/Application.php:175
 Symfony\Component\Console\Application->run() at /path/on/host/computer/example/site/vendor/codeception/codeception/src/Codeception/Application.php:112
 Codeception\Application->run() at /path/on/host/computer/example/site/vendor/codeception/codeception/app.php:45
 {closure}() at n/a:n/a
 call_user_func() at /path/on/host/computer/example/site/vendor/codeception/codeception/app.php:7
 require() at /path/on/host/computer/example/site/vendor/codeception/codeception/codecept:7
 include() at /path/on/host/computer/example/site/vendor/bin/codecept:119

run [-o|--override OVERRIDE] [-e|--ext EXT] [--report] [--html [HTML]] [--xml [XML]] [--phpunit-xml [PHPUNIT-XML]] [--colors] [--no-colors] [--silent] [--steps] [-d|--debug] [--shard SHARD] [--filter FILTER] [--grep GREP] [--bootstrap [BOOTSTRAP]] [--no-redirect] [--coverage [COVERAGE]] [--coverage-html [COVERAGE-HTML]] [--coverage-xml [COVERAGE-XML]] [--coverage-text [COVERAGE-TEXT]] [--coverage-crap4j [COVERAGE-CRAP4J]] [--coverage-cobertura [COVERAGE-COBERTURA]] [--coverage-phpunit [COVERAGE-PHPUNIT]] [--no-exit] [-g|--group GROUP] [-s|--skip SKIP] [-x|--skip-group SKIP-GROUP] [--env ENV] [-f|--fail-fast [FAIL-FAST]] [--no-rebuild] [--seed SEED] [--no-artifacts] [--] [<suite> [<test>]]

  [Db] Disconnected from default
PHP Fatal error:  Uncaught lucatume\WPBrowser\WordPress\InstallationException: WordPress failed to load for the following reason: cOMMAND DID NOT FINISH PROPERLY. in /path/on/host/computer/example/site/vendor/lucatume/wp-browser/src/WordPress/InstallationException.php:49
Stack trace:
#0 /path/on/host/computer/example/site/vendor/lucatume/wp-browser/src/WordPress/LoadSandbox.php(105): lucatume\WPBrowser\WordPress\InstallationException::becauseWordPressFailedToLoad('COMMAND DID NOT...')
#1 [internal function]: lucatume\WPBrowser\WordPress\LoadSandbox->obCallback('\n\n\nCOMMAND DID ...', 9)
#2 {main}
  thrown in /path/on/host/computer/example/site/vendor/lucatume/wp-browser/src/WordPress/InstallationException.php on line 49

The smoking gun is in the exception trace, where it says require_once() at /path/to/example/site/web/wp-config.php:9 . In Bedrock, that wp-config file loads application config prior to requiring wp-settings.php:

require_once dirname(__DIR__) . '/vendor/autoload.php';
require_once dirname(__DIR__) . '/config/application.php';
require_once ABSPATH . 'wp-settings.php';

To Reproduce

1. Have a local WordPress installation running in a VM (I assume containers would be similarly affected). (For normal, working operation, the SQL database in the VM must be set to allow remote access by the user that WPLoader should use.)
2. Configure test suites as above so tests run on the host, accessing the site running in the VM, and populating the database remotely.
3. The local project folders are mounted inside the VM filesystem, so files are the same in both places.
4. Try to run the test suite on the host machine

(This might be onerous to set up; if necessary I could give you a bare-bones repo with a configured Trellis & Bedrock project)

Expected behavior

What has worked perfectly until 4.3.0 is to have the WebDriver/WPBrowser tests access the same VM environment that runs my development instance of a site. This helps me maintain parity. I run the test suites directly on my host for convenience (wonderful for IDE integration).

Setting DB credentials for WPLoader should ensure that those are always used by the tests.

Additional context (and thoughts!)

I’m using the local SQLite db option for Integration tests, so I haven’t experienced any issues there with a loadOnly: false setup. Everything there is confined to the host machine, no VM involvement.

I’ve tested this with both PHPUnit 9.6 and 10.5, so that doesn’t appear to be a factor. I don’t think my setup is too bizarre...

Thoughts: I’m not sure how WPLoader can avoid applying the Bedrock config if it has to include wp-load.php and all that that reaches out to. Whatever it did before worked, however.

I could modify the Bedrock bootstrap config to, for example, check an environment variable to determine which .env file(s) to load (I’m already doing something similar inside the VM to check request headers for Chromedriver requests and load production config instead of development.) But I also don’t like making my code too test-aware.

(Or maybe I should just create a docker container for my tests already 😄)

[lucatume]

Thank you @andronocean for the detailed report and research: this seriously helps me.

This looks like a regression, since it was working before.

I will look into this in the next days and post updates in this issue.

[lucatume]

While I have not found a fix yet, I've found the origin of the issue: in version 4.3.0 the WPLoader::_beforeSuite method was added to fix #744.
The issue was the module would not load WordPress when the EventDispatcherBridge was not part of the configuration, in the main codeception.yml file, usually.

Your configuration does not use the lucatume\WPBrowser\Extension\EventDispatcherBridge extension: this means that, in versions before 4.3.0, WordPress would not be loaded.

Are you using WordPress functions in your EndToEnd tests? If you do, they should crash.

If you downgrade wp-browser to version 4.2.5 and add the lucatume\WPBrowser\Extension\EventDispatcherBridge extension to your configuration, then you should have the same issue.

The issue appeared, for your configuration, in version 4.3.0 because WPLoader is actually loading WordPress without relying on the EventDispatcherBridge extension.

The issue does not come up when using a SQLite database in the WPLoader module configuration since the DB_HOST, DB_USER, DB_PASSWORD and DB_NAME constants will not be relevant to the loading of the database file.

The fix for #744 is correct: WPLoader should load WordPress whether the EventDispatcherBridge extension is active or not.
The fix for your issue is making sure the loaded WordPress installation will use the database credentials specified in the WPLoader module configuration, and not the ones provided by the wp-config.php (or relative files in the case of Bedrock) file, and this is what I'm looking into.

[andronocean]

Aha! That makes perfect sense. My EndToEnd tests on this project treat the site like a black box and do everything via the browser/HTTP interface, and database setup is either in the dump or done via the WPDb module, so I never ran into missing WP. I'd actually been wondering about what WP functions would do. And sure enough, they fail on 4.2.5.

I've never clarified a use-case for the EventDispatcherBridge, so yes, I left it out. Seems like I've just been lucky.

As for a fix: from my side, the simplest and best option would be to dynamically set which .env file Bedrock is loading. If WPLoader could define a constant (or set an environment variable) that's visible to wp-config (and descendants) during the sandbox load, I can use that to make sure the right environment data gets loaded. As a rough example:

// in some sort of WPLoader bootstrap file
define('WP_ENV_FILE', dirname(__FILE__) . '/tests/.env');

// in wp-config.php or Bedrock application.php
if ( ! defined( 'WP_ENV_FILE' ) ) {
	define( 'WP_ENV_FILE', realpath( dirname( __FILE__ ) . '/../.env' ) );
}

Then Bedrock could load that file with Dotenv as usual, without ever needing to detect if tests are running.

(That obviously wouldn't work for vanilla WordPress sites using hardcoded wp-config constants.)

[andronocean]

To add to that thought... in loadOnly: false mode, WPLoader supports a configFile parameter. Maybe that could be used here, too?

[lucatume]

That obviously wouldn't work for vanilla WordPress sites using hardcoded wp-config constants.

This I've already dealt with: redefinition of constants will trigger a warning that can be handled and suppressed.

Your idea about the definition of an env var or constant to indicate WordPress is being loaded by WPLoader is a good one. I've been going back and forth about other possible solutions, but they all turn out pretty ugly or complicated; your approach is simple and robust.

I will update this issue with further changes.

[lucatume]

To add to that thought... in loadOnly: false mode, WPLoader supports a configFile parameter. Maybe that could be used here, too?

Another possible solution to keep in mind.

[lucatume]

@andronocean I've pushed a fix in #755; I'm defining an env var, WPBROWSER_LOAD_ONLY=1 when loadOnly: true that should provide a good hook for configurations to change depending on the env.

If you could test it out and let me know, that would be great.

composer require --dev lucatume/wp-browser:dev-v4-issue-753

[lucatume]

To add to that thought... in loadOnly: false mode, WPLoader supports a configFile parameter. Maybe that could be used here, too?

Another possible solution to keep in mind.

I will reply to this myself with one admission and one information:

1. I've working on this project so long I forget what's in there.
2. the configFile configuration parameter already works and has for some time now

#755 adds both options: an env var to detect and support for configFile in loadOnly: true mode. I've updated the documentation and am merging that PR.

[andronocean]

@lucatume Thank you for this — I updated to 4.3.3 and modified my configuration accordingly, and it's all working perfectly. The new env var is extremely useful!

Another error appeared on the loadOnly:true tests, and thanks to the env var I could resolve it immediately. I'd seen it once before a while ago, but had no way to handle it then. Again, it was related to integration with the Roots stack (Acorn this time), trying to take over Codeception's run command with its own console handler and leading to an InstallationException in wp-browser.

Happily, WPBROWSER_LOAD_ONLY let me detect WPLoader and do putenv( 'APP_RUNNING_IN_CONSOLE=false' );, which makes Acorn behave itself.

When I have time I can write up a proper separate issue or make a pull request. I think the exception message thrown could be clarified as it is a very specific failure mode. Would you prefer an issue first or just a PR?

[lucatume]

Would you prefer an issue first or just a PR?

If you can provide a PR, that would be much appreciated. You explore issues carefully, and a PR from someone that is taking the time to think it out is an excellent contribution.

Edit: will I ever learn to use quote?