Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add FLAKY_TEST_SECTION_RESET and use it to reset FCM in between flakes. #1201

Open
wants to merge 18 commits into
base: main
Choose a base branch
from

Conversation

jonsimantov
Copy link
Contributor

@jonsimantov jonsimantov commented Feb 2, 2023

Description

Provide details of the change, and generalize the change in the PR title above.

When GetToken fails, it seems like it will usually keep failing unless FCM is reinitialized. This change allows us to reinitialize FCM between attempts, to try to get GetToken to get a token.

This change adds an extra flaky test macro, FLAKY_TEST_SECTION_RESET, that can be put in the middle of a FLAKY_TEST_SECTION_BEGIN and ..._END pair. If there is a FLAKY_TEST_SECTION_RESET, this code will be run after each flaky failure, before the retry, and can be used to reset or reinitialize things before the test runs again.

(Unfortunately, this doesn't seem to 100% fix the problem in TestReceiveToken, which may be caused by an underlying issue with the SDK. Here is an example of it still being broken. 🙁)

It does seem to fix the issues with SendMessageToToken and SendMessageToTopic. However, when retrying SendMessageToToken, we can get into a situation where it receives the message from the previous attempt; add some logic to recognize this situation and consume the extra message. (Here is a test run where this occurred before this fix.)


Testing

Describe how you've tested these changes. Link any manually triggered Integration tests or CPP binary SDK Packaging Github Action workflows, if applicable.

Will run the Android integration tests a few times and analyze the logs to see if the flaky test triggers and if the reinitialize helps.


Type of Change

Place an x the applicable box:

  • Bug fix. Add the issue # below if applicable.
  • New feature. A non-breaking change which adds functionality.
  • Other, such as a build process or documentation change.

Notes

  • Bug fixes and feature changes require an update to the Release Notes section of release_build_files/readme.md.
  • Read the contribution guidelines CONTRIBUTING.md.
  • Changes to the public API require an internal API review. If you'd like to help us make Firebase APIs better, please propose your change in a feature request so that we can discuss it together.

@github-actions github-actions bot added the tests: in-progress This PR's integration tests are in progress. label Feb 2, 2023
@github-actions
Copy link

github-actions bot commented Feb 2, 2023

Integration test with FLAKINESS (succeeded after retry)

Requested by @jonsimantov on commit refs/pull/1201/merge
Last updated: Thu Feb 2 15:34 PST 2023
View integration test log & download artifacts

Failures Configs
messaging [TEST] [FLAKINESS] [Android] [1/3 os: windows] [1/4 android_device: android_target]
(1 failed tests)  FirebaseMessagingTest.TestSendMessageToToken

Add flaky tests to go/fpl-cpp-flake-tracker

@github-actions github-actions bot added the tests: failed This PR's integration tests failed. label Feb 2, 2023
@github-actions github-actions bot added tests: failed This PR's integration tests failed. and removed tests: failed This PR's integration tests failed. labels Feb 2, 2023
@jonsimantov jonsimantov added the no-lint Skip lint checks label Feb 2, 2023
@jonsimantov jonsimantov removed the tests: failed This PR's integration tests failed. label Feb 2, 2023
@github-actions github-actions bot added the tests: failed This PR's integration tests failed. label Feb 2, 2023
@firebase-workflow-trigger firebase-workflow-trigger bot removed the tests: in-progress This PR's integration tests are in progress. label Feb 2, 2023
@github-actions github-actions bot added tests: in-progress This PR's integration tests are in progress. tests: failed This PR's integration tests failed. and removed tests: failed This PR's integration tests failed. labels Feb 2, 2023
@github-actions github-actions bot added the tests: succeeded This PR's integration tests succeeded. label Feb 2, 2023
@firebase-workflow-trigger firebase-workflow-trigger bot removed the tests: in-progress This PR's integration tests are in progress. label Feb 2, 2023
@github-actions github-actions bot added tests: in-progress This PR's integration tests are in progress. and removed tests: succeeded This PR's integration tests succeeded. labels Feb 2, 2023
@github-actions github-actions bot added the tests: succeeded This PR's integration tests succeeded. label Feb 2, 2023
@firebase-workflow-trigger firebase-workflow-trigger bot removed the tests: in-progress This PR's integration tests are in progress. label Feb 2, 2023
@github-actions github-actions bot added tests: in-progress This PR's integration tests are in progress. tests: succeeded This PR's integration tests succeeded. and removed tests: succeeded This PR's integration tests succeeded. labels Feb 2, 2023
@firebase-workflow-trigger firebase-workflow-trigger bot removed the tests: in-progress This PR's integration tests are in progress. label Feb 2, 2023
@github-actions github-actions bot added tests: in-progress This PR's integration tests are in progress. and removed tests: succeeded This PR's integration tests succeeded. labels Feb 2, 2023
@jonsimantov jonsimantov added skip-release-notes Skip release notes check and removed no-lint Skip lint checks labels Feb 2, 2023
@github-actions github-actions bot added the tests: succeeded This PR's integration tests succeeded. label Feb 2, 2023
@firebase-workflow-trigger firebase-workflow-trigger bot removed the tests: in-progress This PR's integration tests are in progress. label Feb 2, 2023
@github-actions github-actions bot added tests: in-progress This PR's integration tests are in progress. tests: succeeded This PR's integration tests succeeded. and removed tests: succeeded This PR's integration tests succeeded. labels Feb 2, 2023
@firebase-workflow-trigger firebase-workflow-trigger bot removed the tests: in-progress This PR's integration tests are in progress. label Feb 2, 2023
@github-actions github-actions bot added the tests: failed This PR's integration tests failed. label Feb 2, 2023
@jonsimantov jonsimantov removed tests: succeeded This PR's integration tests succeeded. tests: failed This PR's integration tests failed. labels Feb 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
skip-release-notes Skip release notes check
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant