Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: overhaul of server streaming retries #1653

Merged
merged 106 commits into from
Sep 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
106 commits
Select commit Hold shift + click to select a range
0f9988a
fix: reduce duplicate code in streaming retries and add a test
leahecole Aug 1, 2024
6d1841f
add comments
leahecole Aug 2, 2024
23ef391
WIP experiment
leahecole Jul 9, 2024
6a3552e
experiments
leahecole Jul 9, 2024
ff26514
WIP - loss example
leahecole Jul 10, 2024
ce6ef1c
fix typo, use pipe not pumpify
leahecole Jul 10, 2024
2b500f8
example of piping errors together
leahecole Jul 10, 2024
1a587c8
pipeline example
leahecole Jul 10, 2024
b4066e5
experiment continued
leahecole Jul 10, 2024
6fd50ad
no buffer, data loss
leahecole Jul 10, 2024
8c5cb44
stream pipeline example
leahecole Jul 10, 2024
8462d22
example that shows before and after
leahecole Jul 10, 2024
fd1a305
set immediate exampel
leahecole Jul 10, 2024
abae9eb
call both tests
leahecole Jul 10, 2024
416c167
another scenario
leahecole Jul 10, 2024
0678f91
more experiment
leahecole Jul 11, 2024
4533036
more experimentation
leahecole Jul 12, 2024
cbd38bc
more experimenting
leahecole Jul 12, 2024
90961b7
some reorganization
leahecole Jul 12, 2024
1644761
more experiments
leahecole Jul 12, 2024
e163c4e
WIP: more exampels
leahecole Jul 15, 2024
47fd5a8
WIP
leahecole Jul 16, 2024
3c09df7
WIP: fixing tests
leahecole Jul 16, 2024
659a0f5
move streamingRetryRequest to streaming.ts, remove delayStream
leahecole Jul 29, 2024
f9ac498
WIP - move SRR into streamProxy class
leahecole Jul 30, 2024
6cbe715
WIP - reduce duplicate code between forwardEventsREtries and streamin…
leahecole Jul 31, 2024
783f2f1
WIP - exploration of SRR removal
leahecole Aug 1, 2024
035025c
merge with cbt2
leahecole Aug 2, 2024
e31781a
fix mistake from rebase
leahecole Aug 2, 2024
3421457
Merge remote-tracking branch 'origin' into cbt2
leahecole Aug 2, 2024
bb92ac5
wip - remove bind from non error emit
leahecole Aug 4, 2024
61c3e27
WIP - stream retries pipeline - success case
leahecole Aug 4, 2024
68b4814
WIP - redoing SRR, successful calls
leahecole Aug 5, 2024
1f4d845
wip - redoing retries, fails immediately
leahecole Aug 5, 2024
cf61b49
WIP - some retrying
leahecole Aug 5, 2024
5df8279
WIP - theoretically working
leahecole Aug 5, 2024
57fe9da
some test cleanup
leahecole Aug 5, 2024
c221dc7
WIP
leahecole Aug 6, 2024
bfb7f97
WIP
leahecole Aug 6, 2024
e522f74
Merge branch 'main' of github.com:googleapis/gax-nodejs into cbt2
leahecole Aug 6, 2024
bd6ae25
WIP - resetting retries to zero
leahecole Aug 6, 2024
a9ac063
change .end to .destroy
leahecole Aug 6, 2024
b7cdeca
Working version of new retries
leahecole Aug 6, 2024
021cfd3
linted test app
leahecole Aug 6, 2024
7a466ef
WIP - no lint errors, but lots commented out
leahecole Aug 6, 2024
eda25aa
cleanup
leahecole Aug 6, 2024
036b4a6
remove files that shouldn't be here
leahecole Aug 7, 2024
4cd45ea
remove files
leahecole Aug 7, 2024
67836de
remove protos that don't need to be there
leahecole Aug 7, 2024
0829c67
remove more files
leahecole Aug 7, 2024
f993c06
remove text files
leahecole Aug 7, 2024
c710a15
more cleanup
leahecole Aug 7, 2024
04f4ff7
remove more files
leahecole Aug 7, 2024
1800589
remove old directories
leahecole Aug 7, 2024
5eb5a3c
finish file cleanup
leahecole Aug 7, 2024
04338d4
more cleanup
leahecole Aug 7, 2024
b3e9e54
fix one unit test
leahecole Aug 7, 2024
1a6237b
fix some unit tests
leahecole Aug 7, 2024
de35694
revert previous change
leahecole Aug 7, 2024
cf189cb
fix some unit tests
leahecole Aug 8, 2024
b03fbae
fix another unit test
leahecole Aug 9, 2024
9e69b2c
fix one more unit test
leahecole Aug 9, 2024
4f56355
WIP - event forwarding is working!
leahecole Aug 14, 2024
3658772
fix test behavior - move data push inside of case
leahecole Aug 14, 2024
97fa51a
WIP - fixing shouldRetry test
leahecole Aug 14, 2024
fe09b9f
fix another unit test
leahecole Aug 22, 2024
9c7f1c5
fix another unit test
leahecole Aug 22, 2024
3131f92
fix two more tests
leahecole Aug 22, 2024
90b8cc5
fix more unit tests, remove console logs
leahecole Aug 22, 2024
a52f87d
WIP - trying final test again
leahecole Aug 23, 2024
b3dc0c8
add a lil stream destruction
leahecole Aug 28, 2024
40c486d
WIP - update returns in inner function
leahecole Aug 28, 2024
524ec6c
remove return
leahecole Aug 28, 2024
550c391
add status/end boolean, remove retrying boolean
leahecole Aug 28, 2024
dfe6653
some testing fixes
leahecole Aug 29, 2024
44c59d4
more fixes
leahecole Aug 29, 2024
d0b979e
lots of lint fixes
leahecole Aug 29, 2024
2c8460c
fix todos, fix test behavior
leahecole Aug 29, 2024
5785b0e
some lint fixes, some todo cleanups
leahecole Aug 29, 2024
0d4c2da
resolve some todos
leahecole Aug 29, 2024
40730d7
add a test, todo cleanup
leahecole Sep 3, 2024
9726e19
WIP - timeout stuff
leahecole Sep 5, 2024
f6b47fa
Timeout test
leahecole Sep 5, 2024
500f9d9
lint fixes
leahecole Sep 5, 2024
6c9f92d
remove retries param
leahecole Sep 5, 2024
8fcb6be
helper function
leahecole Sep 5, 2024
20529ea
resolving todos
leahecole Sep 5, 2024
7a8af41
resolve lint
leahecole Sep 5, 2024
e7ffe0d
Merge branch 'main' of github.com:googleapis/gax-nodejs into cbt2
leahecole Sep 6, 2024
0121d71
clean up tests
leahecole Sep 9, 2024
2a4e6bc
final fix of tests
leahecole Sep 9, 2024
70ea6aa
re-add accidentally deleted file
leahecole Sep 9, 2024
789a1b4
remove erroneous print statement
leahecole Sep 9, 2024
b186f69
private function
leahecole Sep 17, 2024
fae0c03
address review comment
leahecole Sep 18, 2024
ae95d4c
fixing docstring
leahecole Sep 18, 2024
4370e53
fix lint
leahecole Sep 18, 2024
d8abbc0
add comments in test
leahecole Sep 18, 2024
9b3ac82
type coercion change
leahecole Sep 23, 2024
1095232
resolve review comment
leahecole Sep 23, 2024
5f70f80
better typing in unit tests
leahecole Sep 23, 2024
ff7f29f
fix lint
leahecole Sep 23, 2024
ff0b5a1
Merge branch 'main' into cbt2
leahecole Sep 24, 2024
face12f
fix - make sure package.json matches main
leahecole Sep 24, 2024
8dc2109
try/catch for timeout test
leahecole Sep 24, 2024
7b577f0
fix lint
leahecole Sep 24, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions gax/src/gax.ts
Original file line number Diff line number Diff line change
Expand Up @@ -386,6 +386,15 @@ export function convertRetryOptions(
// this is in seconds and needs to be converted to milliseconds and the totalTimeoutMillis parameter
if (options.retryRequestOptions.totalTimeout !== undefined) {
totalTimeoutMillis = options.retryRequestOptions.totalTimeout * 1000;
} else {
if (options.maxRetries === undefined) {
totalTimeoutMillis = 30000;
warn(
'retry_request_options_no_max_retries_timeout',
'Neither maxRetries nor totalTimeout were passed. Defaulting to totalTimeout of 30000ms.',
'MissingParameterWarning'
);
}
}

// for the variables the user wants to override, override in the backoff settings object we made
Expand Down
Loading
Loading