Skip to main content

Behat

Vortex uses Behat for Behavior-Driven Development (BDD) testing. Behat lets you write human-readable stories that describe the behavior of the application. Behat tests primarily focus on critical user journeys, serving as end-to-end validations.

Vortex provides full Behat support, including configuration in behat.yml and a browser container to run tests interactively in a real browser with a VNC viewer.

Additional features include:

  1. Behat Drupal Extension - an extension to work with Drupal.
  2. Behat Steps - a library of re-usable Behat steps.
  3. Behat Screenshot - extension to capture screenshots on-demand and on failure.
  4. Behat Progress formatter - extension to show progress as TAP and failures inline.
  5. Parallel profiles - configuration to allow running tests in parallel.

Running tests​

# Run all Behat tests
ahoy test-bdd
# Run specific feature file
ahoy test-bdd tests/behat/features/homepage.feature
# Run scenarios with specific tag
ahoy test-bdd -- --tags=@smoke

Discovering available step definitions​

# Generate step definitions reference
ahoy test-bdd -- --definitions=l

FeatureContext​

The FeatureContext.php file comes with included steps from Behat steps package.

You can add your custom steps into this file.

Profiles​

Behat's default profile runs every scenario except those tagged @skipped.

In the continuous integration environment, the profile can be overridden using the $VORTEX_CI_BEHAT_PROFILE environment variable.

Parallel runs​

In the continuous integration pipeline, Behat tests can run within multiple runners to increase the speed of the test suite. To achieve this, Behat tags are used to mark features and scenarios.

Out of the box, Vortex provides support for unlimited parallel runners, but only 2 parallel profiles, p0 and p1:

  • p0 (the first runner) is the catch-all: it runs every scenario not tagged @p1, plus all @smoke scenarios.
  • p1 (the second runner) runs scenarios tagged @p1, plus all @smoke scenarios.

In practice: leave a feature untagged to run it on the first runner, tag it @p1 to move it to the second runner, or tag it @smoke to run it on every runner. An untagged feature always lands on the first runner, so forgetting to tag never orphans a test.

You can add more p* profiles in your behat.yml by copying the existing p1 profile and changing several lines of configuration.

If the pipeline has only one runner and VORTEX_CI_BEHAT_PROFILE is unset, the default profile is used and all tests run there except those tagged @skipped. An explicitly set VORTEX_CI_BEHAT_PROFILE stays active even on a single runner.

Skipping tests​

Add @skipped tag to a feature or scenario to exclude it from the test run.

Screenshots​

Test screenshots are stored into .logs/screenshots location by default, which can be overwritten using $BEHAT_SCREENSHOT_DIR variable (courtesy of Behat Screenshot package).

In continuous integration pipeline, screenshots are stored as build artifacts. In GitHub Actions, they can be downloaded from the Summary tab. In CircleCI they are accessible in the Artifacts tab.

Animated screenshots​

Per-step screenshots are combined into an animated GIF for each scenario, making a failing test easier to follow. This is enabled in behat.yml under the animation key of the DrevOps\BehatScreenshotExtension block.

Animation forces a full-page capture after every passed step, so its cost grows with the number of steps and roughly doubles the wall time of a run. Vortex passes BEHAT_SCREENSHOT_ANIMATION_SKIP=1 into the Behat step in continuous integration to skip animation there, while keeping it available for local runs where a recording is worth the wait.

Set animation.enabled to false in behat.yml to disable animation everywhere, or tag an individual scenario or feature with @screenshots:animated:skip to exempt it.

Format​

Out of the box, Vortex comes with Behat Progress formatter output formatter to show progress as TAP and failures inline. This lets a test run continue after a failure while maintaining a minimal output.

Reporting​

Behat writes test reports in JUnit format to the .logs/test_results/behat directory. The continuous integration pipeline stores them as artifacts and uses them to track test performance and stability.

Boilerplate test features​

Vortex provides BDD tests boilerplate covering core user journeys (homepage, login, search) and the shipped module integrations (Redis, ClamAV, redirects, robots.txt, XML sitemap, accessibility).

These boilerplate tests run in continuous integration pipeline when you install Vortex and can be used as a starting point for writing your own.

Writing tests​

For project-specific test writing conventions (user story format, standard user types, test data conventions), see your project's docs/testing.md file.

The docs/testing.md file is scaffolded when you install Vortex and should be maintained by your project team to document agreed-upon testing practices.