Fix test to handle nested observation dicts

[dm-ackerman]

Description

The previous version of the test worked for a dict observation, but not a dict of a dict. This fix will allow observations with unlimited nesting to be correctly tested (at least for the specific test that was failing).

Fixes #1167, Depends on #1170

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)

Checklist:

• I have run the pre-commit checks with pre-commit run --all-files (see CONTRIBUTING.md instructions to set it up)
• I have run pytest -v and no errors are present.
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I solved any possible warnings that pytest -v has generated that are related to my code to the best of my knowledge.
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[elliottower]

This would be good to get merged but it looks like it failed because int has no dtype? Not sure what that is about

[dm-ackerman]

Tests are failing due to AgileRL version change. #1193 should resolve that.

[elliottower]

Looks good to me seems to be passing now too