Release Engineering Process and Merge Queue Management

• PRs ready to be merged are listed on https://merge-list.zephyrproject.io/, if the PR is not listed and is not greyed out and is in bold, that means it is ready to be merged. There are always exceptions of course, those are very minimal however, if in doubt, please ask on #release.
• For a while now we have been merging PRs in batches. Usually, a merge batch is initiated by a single person and the expectation is to go through all ready to merge PRs, do a final quick review and merge if no issues were identified, examples of when a merge would be delayed or needs further actions:
  • Identified wrong assignee. ACTION: The PR should be re-assigned.
  • Large change with minimal reviews, i.e. too many areas being changed. ACTION: add more people as reviewers, ping additional reviewers.
  • CI was last run long time ago, ACTION: re-run CI
  • This is usually the case for PRs that fall under the 4 eyes rule.
  • In general, PRs should not be cherry-picked, the queue needs to be processed in full.
• A new batch is only started when CI on main is complete. We should not interrupt running CI actions in the middle. The time it takes from a merge until the CI job has completed can range between 2 hours and 5-6 hours depending on the time of the day. If the action is interrupted in the middle or just before it was started after a long wait, this can be disruptive, and we will not get a clean status of the main branch.
• Once a batch is started by someone, they are responsible to followup on issues that might arise, for example, if a merged PR is causing CI failures, authors of the PR need to be notified and an issue should be created where it makes sense. If the problem can be solved with a hotfix, a PR should be submitted to fix the problem. When CI is broken with new issues that are still being investigated, no further PRs shall be merged. In cases where the issue is causing failures across the tree with no immediate fix in sight, a revert of the offending PR should be considered.