Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

remove js payload limit #1823

Merged
merged 50 commits into from
Jan 9, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
10f0707
remove js payload limit
djskinner Sep 29, 2022
e57249c
remove js payload limit
djskinner Sep 29, 2022
fa453ab
remove js payload limit
djskinner Sep 30, 2022
df3f6b5
add json-payload unit tests
djskinner Sep 30, 2022
aa5aa7e
dry up test suite
djskinner Oct 3, 2022
805cd29
remove js payload limit
djskinner Oct 3, 2022
051d36b
remove js payload limit
djskinner Oct 3, 2022
2e4daae
remove js payload limit
djskinner Oct 3, 2022
32481fa
remove js payload limit
djskinner Oct 3, 2022
6d6b459
Update CHANGELOG.md
djskinner Oct 6, 2022
1c8d135
add e2e coverage
djskinner Oct 13, 2022
7310991
Merge branch 'next' into plat-8731
djskinner Oct 13, 2022
3238dae
Merge branch 'next' into plat-8731
djskinner Nov 1, 2022
0240d64
add e2e coverage
djskinner Nov 1, 2022
700053d
Merge branch 'plat-8731' of github.com:bugsnag/bugsnag-js into plat-8731
djskinner Nov 1, 2022
6812fb7
add e2e coverage
djskinner Nov 1, 2022
0defa0b
Merge branch 'next' into plat-8731
djskinner Nov 17, 2022
8e6b15c
Test if 400 status has any influence
Cawllec Nov 3, 2022
98e3eb1
Use options only response code for failing scenario
Cawllec Nov 3, 2022
08ea261
Use the post response instead of options
Cawllec Nov 3, 2022
01479ea
Log feedback inplementation
Cawllec Nov 4, 2022
e1767a7
Use actual steps
Cawllec Nov 4, 2022
e1797bb
Return the event instead of err
Cawllec Nov 4, 2022
945a5a6
Return the full err
Cawllec Nov 4, 2022
c3203fc
Remove feedback from log and rely on signle response only
Cawllec Nov 4, 2022
fae0eda
Attempt cross-platform xhr response
Cawllec Nov 8, 2022
a694dc6
Add basic node log implementation and extra logs to ie10
Cawllec Nov 8, 2022
e8d94f3
Update node to maze-runner v7
Cawllec Nov 8, 2022
cd74970
Update to ensure errors are logged correctly
Cawllec Nov 8, 2022
7e88ff0
Temporarily disable delivery test on node and on ie10
Cawllec Nov 9, 2022
37d53ef
Update to Node V7 and add express oversized scenario
Cawllec Nov 17, 2022
e0a77dd
Ensure logs uri is passed through to containers and use http lib in o…
Cawllec Nov 18, 2022
e7c5056
Remove problematic HTTP code
Cawllec Nov 21, 2022
f0556b8
Remove potentially problematic big-data generation
Cawllec Nov 21, 2022
168bd62
Crop even more out
Cawllec Nov 21, 2022
24a0e26
Fix incorrect URL
Cawllec Nov 21, 2022
a5aeabd
Ensure the correct version of url is present
Cawllec Nov 21, 2022
50caa80
Ensure the correct version of url is present
Cawllec Nov 21, 2022
140aef4
Use correct parse method
Cawllec Nov 23, 2022
f445c2f
Merge pull request #1866 from bugsnag/plat-8731-testing
djskinner Nov 28, 2022
9bfbdbe
Merge branch 'next' into plat-8731
djskinner Nov 29, 2022
82f6719
Merge branch 'plat-8731' of github.com:bugsnag/bugsnag-js into plat-8731
djskinner Nov 29, 2022
879587a
use remote logging to assert event oversized log message
djskinner Nov 29, 2022
b1a23c8
use remote logging to assert event oversized log message
djskinner Nov 29, 2022
af22e4c
enhance node delivery test for future persistence functionality
djskinner Dec 1, 2022
b9ed23a
revert e2e tests for oversized delivery on electron
djskinner Dec 1, 2022
22557f5
Merge branch 'next' into plat-8731
Dec 15, 2022
094c21a
update CHANGELOG.md
Dec 15, 2022
aef0a36
Merge branch 'next' into plat-8731
Jan 9, 2023
c455ae4
revert to running Safari 16 tests on browser stack
Jan 9, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# Changelog

## TBD

### Changed

- Attempt to send all event and session payloads, regardless of size
djskinner marked this conversation as resolved.
Show resolved Hide resolved

## v7.18.0 (2022-09-22)

### Changed
Expand Down
6 changes: 2 additions & 4 deletions packages/core/lib/json-payload.js
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,11 @@ Serialized payload was ${payload.length / 10e5}MB (limit = 1MB)
metadata was removed`
}
payload = jsonStringify(event, null, null, { redactedPaths: EVENT_REDACTION_PATHS, redactedKeys })
if (payload.length > 10e5) throw new Error('payload exceeded 1MB limit')
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the change can be summarised as: don't throw here so each delivery mechanism attempts the send. For delivery mechanisms that support retries we explicitly make it not retryable if the payload was above the threshold (regardless of 400 status code or not)

}
return payload
}

module.exports.session = (event, redactedKeys) => {
const payload = jsonStringify(event, null, null)
if (payload.length > 10e5) throw new Error('payload exceeded 1MB limit')
module.exports.session = (session, redactedKeys) => {
const payload = jsonStringify(session, null, null)
return payload
}
5 changes: 5 additions & 0 deletions packages/delivery-electron/delivery.js
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,11 @@ const delivery = (client, filestore, net, app) => {
} else {
const err = new Error(`Bad status code from API: ${response.statusCode}`)
err.isRetryable = isRetryable(response.statusCode)
// do not retry oversized payloads regardless of status code
if (body.length > 10e5) {
client._logger.warn(`Discarding over-sized event (${body.length / 10e5}) after failed delivery`)
err.isRetryable = false
}
cb(err)
}
})
Expand Down
7 changes: 6 additions & 1 deletion packages/delivery-node/delivery.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,13 @@ const request = require('./request')

module.exports = (client) => ({
sendEvent: (event, cb = () => {}) => {
const body = payload.event(event, client._config.redactedKeys)

const _cb = err => {
if (err) client._logger.error(`Event failed to send…\n${(err && err.stack) ? err.stack : err}`, err)
if (body.length > 10e5) {
client._logger.warn(`Discarding over-sized event (${body.length / 10e5}) after failed delivery`)
}
djskinner marked this conversation as resolved.
Show resolved Hide resolved
cb(err)
}

Expand All @@ -17,7 +22,7 @@ module.exports = (client) => ({
'Bugsnag-Payload-Version': '4',
'Bugsnag-Sent-At': (new Date()).toISOString()
},
body: payload.event(event, client._config.redactedKeys),
body,
agent: client._config.agent
}, (err, body) => _cb(err))
} catch (e) {
Expand Down
10 changes: 9 additions & 1 deletion packages/delivery-x-domain-request/delivery.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,22 @@ const payload = require('@bugsnag/core/lib/json-payload')
module.exports = (client, win = window) => ({
sendEvent: (event, cb = () => {}) => {
const url = getApiUrl(client._config, 'notify', '4', win)
const body = payload.event(event, client._config.redactedKeys)

const req = new win.XDomainRequest()
req.onload = function () {
cb(null)
}
req.onerror = function () {
client._logger.error('Event failed to send…\n')
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

node does logger.error(Event failed to send…` so I thought maybe the browser delivery methods should too

if (body.length > 10e5) {
client._logger.warning(`Discarding over-sized event (${body.length / 10e5}MB) after failed delivery`)
}
}
req.open('POST', url)
setTimeout(() => {
try {
req.send(payload.event(event, client._config.redactedKeys))
req.send(body)
} catch (e) {
client._logger.error(e)
cb(e)
Expand Down
15 changes: 13 additions & 2 deletions packages/delivery-xml-http-request/delivery.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,26 @@ module.exports = (client, win = window) => ({
try {
const url = client._config.endpoints.notify
const req = new win.XMLHttpRequest()
const body = payload.event(event, client._config.redactedKeys)

req.onreadystatechange = function () {
if (req.readyState === win.XMLHttpRequest.DONE) cb(null)
if (req.readyState === win.XMLHttpRequest.DONE) {
const status = req.status
if (status === 0 || status >= 400) {
client._logger.error('Event failed to send…\n')
if (body.length > 10e5) {
client._logger.warning(`Discarding over-sized event (${body.length / 10e5}MB) after failed delivery`)
}
}
cb(null)
}
}
req.open('POST', url)
req.setRequestHeader('Content-Type', 'application/json')
req.setRequestHeader('Bugsnag-Api-Key', event.apiKey || client._config.apiKey)
req.setRequestHeader('Bugsnag-Payload-Version', '4')
req.setRequestHeader('Bugsnag-Sent-At', (new Date()).toISOString())
req.send(payload.event(event, client._config.redactedKeys))
req.send(body)
} catch (e) {
client._logger.error(e)
}
Expand Down