Add support for adding file attachments in junit report format

[trakos]

Soon, gitlab will add ability to show images attached in junit report file on merge request/pipeline view:

• https://docs.gitlab.com/ee/ci/unit_test_reports.html#viewing-junit-screenshots-on-gitlab
• https://gitlab.com/gitlab-org/gitlab/-/merge_requests/26922

This was also possible in jenkins, with junit-attachments plugins and supported by some libraries like mocha:

• https://plugins.jenkins.io/junit-attachments/
• https://www.npmjs.com/package/mocha-junit-reporter#attachments
• https://kohsuke.org/2012/03/13/attaching-files-to-junit-tests/

The format is a bit weird, because it's simply supposed to be added to system-out element, example from gitlab:

<?xml version="1.0" encoding="UTF-8"?>
<testsuite name="rspec" tests="1" skipped="0" failures="1" errors="0" time="0.017557" timestamp="2020-03-09T15:16:55-04:00" hostname="Maximes-MacBook-Pro.local">
  <properties>
    <property name="seed" value="9386"/>
  </properties>
  <testcase classname="spec.hello_world_spec" name="the home page return hello world" file="./spec/hello_world_spec.rb" time="0.016340">
    <failure message="" type="RuntimeError">Failure/Error: raise
RuntimeError:
./spec/hello_world_spec.rb:8:in `block (2 levels) in &lt;top (required)&gt;'</failure>
    <system-out>[[ATTACHMENT|tmp/capybara/screenshot_return-hello-world.html]]</system-out>
  </testcase>
</testsuite>

Example use-case for phpunit would be with library like symfony/panther: it already has option to make screenshots on failed assertions, but for now they are not linked in any way in the report: symfony/panther#392

It is possible to achieve that simply by echoing the attachment during the phpunit test, but I find this way flawed: it interferes with tests that have to check stdout and with --disallow-test-output option, it also pollutes job output in CI, and has to be done in the test itself or in tearDown.

I submit a proposition for basic interface that could be implemented by TestCases to attach files to junit report. For now it would only be used by junit format, since I'm not aware of other formats requiring such feature.

I think it's simple enough, and doesn't introduce any BC, WDYT?

[sebastianbergmann]

I am reluctant to touch the JUnit logger.

[shaedrich]

@trakos Could you make it a package or something so it can still be of use?

[trakos]

Hey @shaedrich, since then, I've implemented it directly in to symfony/panther in PR symfony/panther#474. It's possible to simply echo the screenshot file path in the right format to get it to the junit output. You can check that symfony/panther PR to see how to implement it.

It's not as nice, but gets the job done.

[shaedrich]

Okay, thanks, @trakos 👍🏻