Large amounts of NRE's after upgrading to Release 22

[victornor]

Describe the bug
After upgrading from Release 21 to Release 22 and Unity 2021 to Unity 6, a bunch of exceptions and 0 observation errors started occurring during inference.
Nothing about my agent setup changed and I'm still making calls to sensor.AddObservation(x); in CollectObservations.

The console is filled with "Fewer observations (0) made than vector observation size (6). The observations will be padded." and eventually Dispose is called on TensorProxy, which throws Null Reference Exceptions because data.dataOnBackend is null.

Console logs / stack traces

2024/11/06 04:35:08.534 20970 21086 Error Unity NullReferenceException: Object reference not set to an instance of an object.
2024/11/06 04:35:08.534 20970 21086 Error Unity   at Unity.MLAgents.Inference.TensorProxy.Dispose () [0x0000c] in /home/v/git/XXX/Assets/Plugins/com.unity.ml-agents/Runtime/Inference/TensorProxy.cs:77 
2024/11/06 04:35:08.534 20970 21086 Error Unity   at Unity.MLAgents.Inference.TensorProxy.Finalize () [0x00003] in /home/v/git/XXX/Assets/Plugins/com.unity.ml-agents/Runtime/Inference/TensorProxy.cs:72 
2024/11/06 04:35:08.534 20970 21086 Error Unity UnityEngine.DebugLogHandler:Internal_LogException(Exception, Object)
2024/11/06 04:35:08.534 20970 21086 Error Unity UnityEngine.DebugLogHandler:LogException(Exception, Object)
2024/11/06 04:35:08.534 20970 21086 Error Unity UnityEngine.Logger:LogException(Exception, Object)
2024/11/06 04:35:08.534 20970 21086 Error Unity UnityEngine.Debug:LogException(Exception)
2024/11/06 04:35:08.534 20970 21086 Error Unity UnityEngine.<>c:<RegisterUECatcher>b__0_0(Object, UnhandledExceptionEventArgs) (at /home/bokken/build/output/unity/unity/Runtime/Export/Scripting/UnhandledExceptionHandler.bindings.cs:46)

2024/11/06 04:34:25.236 20970 21024 Warn Unity Fewer observations (0) made than vector observation size (6). The observations will be padded.
2024/11/06 04:34:25.236 20970 21024 Warn Unity UnityEngine.DebugLogHandler:Internal_Log(LogType, LogOption, String, Object)
2024/11/06 04:34:25.236 20970 21024 Warn Unity UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])
2024/11/06 04:34:25.236 20970 21024 Warn Unity UnityEngine.Logger:LogFormat(LogType, String, Object[])
2024/11/06 04:34:25.236 20970 21024 Warn Unity UnityEngine.Debug:LogWarningFormat(String, Object[])
2024/11/06 04:34:25.236 20970 21024 Warn Unity Unity.MLAgents.Sensors.VectorSensor:Write(ObservationWriter) (at /home/v/git/XXX/Assets/Plugins/com.unity.ml-agents/Runtime/Sensors/VectorSensor.cs:56)
2024/11/06 04:34:25.236 20970 21024 Warn Unity Unity.MLAgents.Policies.HeuristicPolicy:StepSensors(List`1) (at /home/v/git/XXX/Assets/Plugins/com.unity.ml-agents/Runtime/Policies/HeuristicPolicy.cs:133)
2024/11/06 04:34:25.236 20970 21024 Warn Unity Unity.MLAgents.Policies.HeuristicPolicy:RequestDecision(AgentInfo, List`1) (at /home/v/git/XXX/Assets/Plugins/com.unity.ml-agents/Runtime/Policies/HeuristicPolicy.cs:38)
2024/11/06 04:34:25.236 20970 21024 Warn Unity Unity.MLAgents.Agent:SendInfoToBrain() (at /home/v/git/XXX/Assets/Plugins/com.unity.ml-agents/Runtime/Agent.cs:1137)
2024/11/06 04:34:25.236 20970 21024 Warn Unity Unity.MLAgents.Agent:SendI

Environment (please complete the following information):

• Unity Version: Unity 6
• OS + version: Arch Linux and Android 14 (dev build)
• ML-Agents version: Release 22
• Torch version: Not Applicable (exceptions during inference)
• Environment: Not Applicable (exceptions during inference)

[miguelalonsojr]

Please try to reproduce this with the examples environments. Note: we have not yet upgraded to Unity. The current supported version is 2023.2. Please downgrade to that version and see if the issue persists.