A JUnit5 TestEngine to integrate the FHIR Validator into the JUnit5 ecosystem.
Supports writing expected validation outcome as tests in json or yaml files.
The engine can be used with any JUnit5 test-runner that supports file based DiscoverySelectors, e.g. the JUnit Console Launcher
- The FHIR Validator must be in the Classpath. Download the latest release of validator_cli.jar or publisher.jar.
- Download latest release of the junit-platform-console-standalone.
- Download latest release of fhir-validator-junit-engine.jar.
Execute the following in a terminal:
java -jar junit-platform-console-standalone-1.8.1.jar -cp fhir-validator-junit-engine.jar:validator_cli.jar -f test.yaml
This process can be automated, e.g. using npm
, see example-project for a working example.
Tests are written using specification files in either YAML or JSON. There is a json-schema that can be used to get intellisense and documentation when writing tests in a supported text-editor. Visual Studio Code supports this for json files, but requires an extension for yaml support.
Following is an example in YAML, most fields are optional but included here for completeness.
title: Human readable name of the Test Suite.
validator:
version: "4.0"
ig:
- ../fsh-generated/resources
- hl7.fhir.us.core#1.0.1
tx: http://tx.fhir.org
txLog: logs/tx.txt
sct: us
tests:
# 1. Test will validate 1 resource against a profile and expects 1 error and 1 warning.
- title: Messages with missing destination.endpoint shall give ERROR.
profile: http://example.com/fhir/StructureDefinition/MyMessage
fileMatch: test-resources/message-with-missing-destination.json
tags:
- fast
- message
expectedIssues:
- severity: WARNING
- severity: ERROR
type: INVARIANT
expression: Bundle.entry[0].resource.ofType(MessageHeader).destination[0].endpoint
message: minimum required = 1, but only found 0
# 2. Test has multiple file-matches and will generate multiple tests, non of which expects any errors.
- fileMatch:
- ../fsh-generated/resources/**
- "!../fsh-generated/resources/ImplementationGuide-*"
tags: ig
Currently the best way to run custom JUnit engines is to use the official console launcher.
- Test files are selected using one or multiple
-f, --select-file
arguments, the values adhere to the .gitignore variant of the glob pattern. - Tests can be filtered by tags, e.g.
--include-tag fast
or--exclude-tag ig
. - Tests can be executed in parallel by adding the argument
--config no.nav.execution.parallel.enabled=true
. - Text coloring in logs can be disabled with
--config no.nav.disable-ansi-colors=true
.
The junit-runner can be configured to output test-reports in JUnit XML format, this can be used in existing tools that already support this format.
The following is an example on how to integrate tests in a GitHub-action workflow. FHIR resources are built using SUSHI and the generated IG is tested using the fhir-validator-junit engine. Because the engine internally uses the official FHIR validator it uses the same package-cache as other FHIR tools (validator, publisher, sushi).
on:
pull_request:
branches:
- main
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: 🗂️ Cache FHIR Packages
uses: actions/cache@v2
with:
path: |
~/.fhir/packages
key: fhir-packages
- name: 🍣 Run SUSHI
run: |
npm install -g fsh-sushi
sushi /ig
- name: 📥 Download test tools
run: |
wget -q https://github.com/navikt/fhir-validator-junit-engine/releases/latest/download/fhir-validator-junit-engine.jar
wget -q https://github.com/hapifhir/org.hl7.fhir.core/releases/latest/download/validator_cli.jar
wget -q https://repo1.maven.org/maven2/org/junit/platform/junit-platform-console-standalone/1.8.1/junit-platform-console-standalone-1.8.1.jar
- name: ✔️ Run tests
run: java -jar junit-platform-console-standalone-1.8.1.jar -cp fhir-validator-junit-engine.jar:validator_cli.jar -f /tests/*.yaml --reports-dir /test-results
continue-on-error: true
- name: 📤 Publish test results
uses: EnricoMi/publish-unit-test-result-action@v1
with:
commit: ${{ github.event.workflow_run.head_sha }}
files: "/test-results/TEST-*.xml"