[JENKINS-55927] Hook event should not trigger Branch Indexing

[Dohbedoh]

JENKINS-55927

Noticed this issue while troubleshooting an environment where Bitbucket event were triggering branch indexing. Haven't yet identified exactly what kind of push events has empty changes. Though per my understanding of the design of the Branch API / SCM API and Pipeline Multibranch, SCM events should not trigger branch indexing ? And this is rather unexpected. Unless I am missing something ?

cc @jenkinsci/bitbucket-branch-source-plugin-developers

Your checklist for this pull request

• Make sure you are requesting to pull a topic/feature/bugfix branch (right side) and not your master branch!
• Ensure that the pull request title represents the desired changelog entry
• Please describe what you did
• Link to relevant issues in GitHub or in Jenkins JIRA
• Link to relevant pull requests, esp. upstream and downstream changes
• Did you provide a test-case? That demonstrates feature works or fixes the issue.

[nfalco79]

I think, but I need to check, Approved, Approved Removed, Declined event could have not changes. For example declined will trigger the job deletion in multibranch pipeline

[Dohbedoh]

@nfalco79 Actually according to the code and the HookProcessor that do this, it should really be a push event (could also be a mirror synchronized) but not pull requests... Unless declining a pull requests somehow generate a push event on the base branch..

For example declined will trigger the job deletion in multibranch pipeline

Right, so per my understanding of the SCM API, a decline should just be a REMOVED event and mark the branch job as dead. Branch Indexing will be responsible later on for the cleanup of orphaned items.

• https://github.com/jenkinsci/scm-api-plugin/blob/698.v8e3b_c788f0a_6/docs/implementation.adoc?plain=1#L153-L157
• https://github.com/jenkinsci/scm-api-plugin/blob/698.v8e3b_c788f0a_6/docs/implementation.adoc?plain=1#L1945-L1950

Here the behavior of triggering a full scan on a SCMHead event (single branch) can be detrimental. A full scan can be long and expensive (resources and BranchIndexing thread pool on the server but also API requests against Bitbucket).

That is why I think this might be a mistake.

[nfalco79]

As described in JIRA issue I think these changes will cause to the user that have setup "Discover pull requests" strategies different than "The current pull request revision" they never get a build for PRs which target branch is the same branch subject of the merged.
If I understand correct the configuration in the JIRA issue the reported would trigger manual reindex to update all PRs. Than this should be archived using a build strategies to ignore some specific index events (maybe already implemented in other plugins)

[Dohbedoh]

The JIRA mentions the Received hook from Bitbucket. Processing push event on log line. per my reading, this is really coming from the event subscriber / hook processors (entry point is BitbucketSCMSourcePushHookReceiver). It would happen regardless of the multibranch configuration as far as I can tell.

[nfalco79]

Here the behavior of triggering a full scan on a SCMHead event (single branch) can be detrimental. A full scan can be long and expensive (resources and BranchIndexing thread pool on the server but also API requests against Bitbucket).

I agree but if you do not perform a BranchIndexing you do not know what are branches involved in changes.

[Dohbedoh]

Looking at the history of that change, it was here from the start, before we implemented the SCM API Event mechanism and preserved when we added it https://github.com/jenkinsci/bitbucket-branch-source-plugin/pull/37/files#diff-a5efe0e74b0f75735db75b177d0b8da3426c13be4a855727e8eeb8193f74ff33. Though we were relying on a custom made plugin at that time that does not exist anymore: https://github.com/topicusfinan/bitbucket-webhooks-plugin/

I think we really need to figure out what scenario causes an empty changes. I am quite convinced triggering indexing instead of using a particular Event is wrong.. If we wanted to have indexing triggered, I would think we'd create a SCMSourceEvent instead. Maybe someone closer to the design of SCM API / Branch API could confirm cc @rsandell @amuniz

Per the JIRA issue, this could be a remote merge, that is we have an Open PR, but instead of merging it from Bitbucket, we merge it by hand. Hence we end up with a PR without changes. If that is the case, this really shouldn't trigger an indexing. I would be surprised if someone was relying on such a behavior.

[nfalco79]

I would take time to test some supported webhook event (I'm collecting them) like:
new tag, that must automatically trigger the tag discover branch
PR merged trigger removal
New PR
Approved
Disapproved

The behaviour to trigger a new build for all the PRs with Merge strategy (build PR commit with merge into master) is by what who setup this expect to ensure the PR is good for merge and avoid trigger manul build. Who do not want -> build strategy to inhibit builds (but not indexing)

[amuniz]

It's a really long time I worked on this plugin, so I don't really know. But looking at the change introducing the empty check I think they were interpreted as an empty changeset, so it corresponds to a PR creation/deletion which actually requires a re-index to create/remove the job in Jenkins. Whether that's still needed or not, I don't know, but I think if all the known cases are manually tested and they work, then we are ok.

[Dohbedoh]

But per the design of the API (and like all implementation do) those events do not run branch indexing... A CREATE SCMEvent would create the PR job. A DELETED SCMEvent would mark it as stale...
And those event with empty changes are actually push events, they originate from activity on branches.

Just need to figure out what a push event with empty changes originate from. I think I have a reproducer with Bitbucket Server with Native webhook and also with the add on. Will update this in the JIRA shortly.

[Dohbedoh]

Found 2 scenario where you get a push event with empty changes, in Bitbucket Server with the Native webhook as well as with the Bitbucket add-on. I found 2 scenarios where the empty changes may happen:

Rebase a target branch to a source branch while a PR is open

git checkout main
git checkout -B source-1
echo "test" >> source-1.md
git add . && git commit -m "source-1: $(date)" && git push --set-upstream origin source-1
git push --set-upstream origin source-1
# Open a PR for from the source branch `source-1` to the target branch `main`
git checkout main
git rebase -i source-1
git push -f

This will generate 2 events. The first one with empty changes and the next one with the correct changes made to the target branch. (The PR is automatically merged because there is no diff and that's a default behavior of Bitbucket server).. So the first event is useless. It does not give any clue about what just happened.

Auto-Sync of Fork when a branch cannot be synchronized

When the Multibranch Pipeline is a fork and Automatic Fork Syncing is enabled. If a branch on a fork cannot be synchronized, the attempt from Bitbucket to synchronize it generates a push event with empty changes. This seems also useless. The event does not contain information that allow us to know what just happened, and the refs is actually unchanged...

---

Both the add-on and the native webhook results behave the same. Per my understanding the add-on relies on the native mechanism but add some extra configuration..

So far, I don't see a valid scenario where a push event with empty changes should be considered. Or why it would trigger branch indexing. If we are skeptical about whether anybody expect this behavior, I propose to add a system property to allow use to re-enable this.

[nfalco79]

The behaviour to trigger a new build for all the PRs with Merge strategy (build PR commit with merge into master) is by what who setup this expect to ensure the PR is good for merge and avoid trigger manual build.

Issue #316

[Dohbedoh]

Actually this behavior is not what happens with Bitbucket Branch Source at the moment. Those events that trigger branch indexing are happening on very specific scenario where you would not expect it (see my previous comment).
What people are asking in #316 are IMO a feature for SCM API / Branch API. By default the API does not go and observe source branches when a target one is updated.