Breaking changes policy

[rickeylev]

As part of establishing a 1.0 release (#1361), I want to write down our policy and process for breaking changes.

The basic policy and procedure I'm leaning towards is something like:

In order to introduce a breaking change, the breaking behavior must (when reasonably possible)
be opt-in in the first release (N) it's available for.

The basic idea is: it's easier to handle breaking changes when you can incrementally upgrade a code base. e.g., upgrade the dependency version N, update apis to the new version's requirements, then upgrade the dependency to N+1.

I'm not keen on a date-based scheme (e.g. an incompatible change must retain a compatible option for 6 months); I'd rather just use version releases.

I'm torn about what happens after that, though. I see two approaches:

• The next release forces the new behavior on; the toggle is a no-op or removed. This has the advantage of being easier on us maintainers; defunct code sticks around for less longer.
• The next release enables the new behavior by default, but users are able to opt-out. The release after that one forces the new behavior on. This has the advantage of being easier for users (you can opt-out problem cases reactively to more clearly identify what needs to be fixed). But, I suspect most people won't notice an incompatible change until it's the default behavior, so I'm not sure how much value the extra step really has. This is a bit more of a burden on us maintainers, though.

[aignas]

Overall, this LGTM. I would love to have this text in developing and an added note in the pull request template, which points to it.

What do we do with bzlmod, which is still beta and may see more breaking? Do we keep the same behaviour?

[rickeylev]

Good question. For things considered beta, I think we should record that an incompatible change has occurred (in commit description and changelog), but not require an opt-in toggle. I expect we'll have a similar situation with Phillip's work redoing how the pip integration works (that it'll span multiple releases to uncover bugs and reach feature parity).

And yes, this policy will go in contributing.md, alongside the commit description guidelines. (I just wanted to start with an issue for discussion purposes.)

[rickeylev]

Update from our maintainer's meeting: we'll go with a 3 step process.

1. A breaking change must be disabled by default when released in version N. Users must opt into the behavior.
2. In version N+1, the breaking change can be enabled by default. Users must opt out of the behavior.
3. In version N+2, the breakage change is always enabled and the ability to control it is also removed. Users cannot opt out of the behavior.

Note the +1 and +2 are just examples. The release in step 2 just has to be after the release in step 1; the release in step 3 has to be after the release in step 2. I'll write up a more formal thing and send PR to modify the md files