Renovate doesn't update docker-compose with yaml anchors anymore

[mueller-ma]

How are you running Renovate?

A Mend.io-hosted app

If you're self-hosting Renovate, tell us which platform (GitHub, GitLab, etc) and which version of Renovate.

No response

Please tell us more about your question or problem

In the past Renovate had no problems when a Docker compose file had yaml anchors. Now it doesn't extract the image anymore:

• https://github.com/mueller-ma/renovate-reproduce/blob/main/docker-compose.yml
• Dependency Dashboard mueller-ma/renovate-reproduce#2

docker compose can use and parse this docker-compose file. You can see that docker includes the image in both services:

$ docker compose -f docker-compose.yml config
name: tmp
services:
  service-a:
    environment:
      SERVICE: a
    image: debian:11
    networks:
      default: null
  service-b:
    environment:
      SERVICE: b
    image: debian:11
    networks:
      default: null
networks:
  default:
    name: tmp_default
x-shared_setting:
  image: debian:11

Logs (if relevant)

Logs

DEBUG: packageFiles with updates
{
  "baseBranch": "main"
  "config": {
    "docker-compose": [
      {
        "deps": [],
        "packageFile": "docker-compose.yml"
      }
    ]
  }
}

[Badbond]

We experienced a similar issue with Helmfiles when upgrading Renovate from 37.440.7 to 38.142.1. The new YAML parsing library seems to by default use YAML 1.2 specification which dropped merge keys << (was introduced as deprecated in the first place). We found that specifying a YAML directive on the file caused Renovate to parse the file with the YAML 1.1 spec and it opened an upgrade PR again for a dependency that was defined in an anchor and merged in the proper place.

Not sure if this would work for Docker compose files too, but adding this to the top of the file fixed it for us in helmfile.yaml files:

%YAML 1.1
---

See also https://ktomk.github.io/writing/yaml-anchor-alias-and-merge-key.html for some more context on YAML 1.1 vs 1.2.

[rarkins]

Great info!

Are either of these possible?

1. Is there a string/pattern we could match against prior to parsing, which we could reliably use to know if it needs YAML 1.1 parsing?
2. If it parses successfully with YAML 1.2 defaults, can we use the parsed result to someone determine that we should parse it again but with YAML 1.1?

[mueller-ma]

1. Maybe use the string <<: * for that, but I'm not sure if that handles all cases.

[Badbond]

but adding this to the top of the file fixed it for us in helmfile.yaml files

I unfortunately have to come back on this. It seems Helmfile doesn't support the YAML directive. But perhaps it is still useful for Docker or other managers working on YAML files.

Is there a string/pattern we could match against prior to parsing, which we could reliably use to know if it needs YAML 1.1 parsing?

I think the YAML directive in my example above is part of the spec, but as mentioned above it seems that not every tool is compatible with it. As a handgesture to those tools, perhaps next to the YAML directive we can support a similar string as a comment on the top of the file, much like a shebang, to tell Renovate it is YAML 1.1 instead, either:

# YAML 1.1
# %YAML 1.1

If it parses successfully with YAML 1.2 defaults, can we use the parsed result to someone determine that we should parse it again but with YAML 1.1?

Not sure if I completely follow, but if I recall correctly from the Renovate debugging I've done, it seems that during parsing the invalid 1.2 input is omitted/sanitized, but the result is still successful. I don't think we can identify from this it should reparse.

[rarkins]

Plan A:

• Our yaml parse utility can usually determine when to use YAML 1.1

Plan B:

• Support one or more magic strings like suggested above (could be needed in addition to the first plan too)