Changed the ShortenerTransform to use breadth first traversal

[danielmorell]

Description of the change

This is an additional performance improvement for our data processing and cleaning. We don't want to traverse and shorten deep in a large data structure if we are going to later drop that structure.

By changing to a breadth first traversal for the ShortenerTransform we ensure we only traverse data structures we will pass to further transforms.

One change that comes out of this impacts how we pass shortened objects between transforms. The ShortenerTransform no longer turns list and other built-in types into strings e.g. '[1, 2, 3, ...]'. It now returns a list e.g. [1, 2, 3, '...'].

Type of change

• Bug fix (non-breaking change that fixes an issue)
• New feature (non-breaking change that adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Maintenance
• New release

Related issues

• Fix Limits on locals() logging do not prevent runaway traversal #329
  Note: We could further optimize this by limiting our collection of locals prior to traversing the payload body.

Checklists

Development

• Lint rules pass locally
• The code changed/added as part of this pull request has been covered with tests
• All tests related to the changed code pass in development

Code review

• This pull request has a descriptive title and information useful to a reviewer. There may be a screenshot or screencast attached
• "Ready for review" label attached to the PR and reviewers assigned
• Issue from task tracker has a link to this pull request
• Changes have been reviewed by at least one other engineer

[pawelsz-rb]

@danielmorell , I have reviewed the code and performed my old tests, it mostly looks good to me, with the exception of dictionary handling
"6": "{...}" was changed to just "6": {}. To me as a user, this would not indicate that something got truncated. For example, with this change, my output var looks like this:
{ "1": 1, "2": { "3": { "4": { "level4": "good", "level5": { "6": {}, "toplevel": "ok" } } } } } where before I got
{ "1": 1, "2": { "3": { "4": { "level4": "good", "level5": { "6": "{...}", "toplevel": "ok" } } } } } . I think I liked {...} better . Maybe we can use {"..." : "..."} as the default to indicate a truncated dict - something we use for the list, which looks good now:

before:

[ "good_5", "[...]" ]
after:
[ "good_5", [ "..." ] ]

[pawelsz-rb]

lgtm