"Received a packet with an invalid Magic Value" due to huge message

[scs-b]

Description

This bug has only happened a few times in the past 6 months, and we're not sure what's causing it.

We are letting the game run on repeat for hours or days, and this happened after successfully running the game for over 24 hours.

The log says that the client received a seemingly huge message, but the server is not complaining about trying to write too much to the FastBufferWriter. This is how we send the message;

public void SendInitState(InitState message) {
    using var writer = new FastBufferWriter(4096, Allocator.Temp);
    writer.WriteValueSafe(message);
    _sharedData.NetworkManager.CustomMessagingManager.SendNamedMessageToAll(Constants.InitStateMessage, writer, NetworkDelivery.ReliableFragmentedSequenced);
}

Error message

[Netcode] Received a packet with an invalid Magic Value. Please report this to the Netcode for GameObjects team at https://github.com/Unity-Technologies/com.unity.netcode.gameobjects/issues and include the following data: Offset: 4, Size: 6357069, Full receive array: (25mb of data here)
[Netcode] Received a packet too small to contain a BatchHeader. Ignoring it.
[Netcode] Received a packet too small to contain a BatchHeader. Ignoring it.
ArgumentOutOfRangeException: ArgumentOutOfRange_NeedNonNegNum
Parameter name: count
  at System.ArraySegment`1[T]..ctor (System.Byte[] array, System.Int32 offset, System.Int32 count) [0x00011] in <467a840a914a47078e4ae9b0b1e8779e>:0 
  at Unity.Netcode.Transports.UTP.BatchedReceiveQueue.PopMessage () [0x0003a] in <cb73a45f761942c68e8e5de2a8859e77>:0 
  at Unity.Netcode.Transports.UTP.UnityTransport.ReceiveMessages (System.UInt64 clientId, Unity.Networking.Transport.NetworkPipeline pipeline, Unity.Networking.Transport.DataStreamReader dataReader) [0x00046] in <cb73a45f761942c68e8e5de2a8859e77>:0 
  at Unity.Netcode.Transports.UTP.UnityTransport.ProcessEvent () [0x000c4] in <cb73a45f761942c68e8e5de2a8859e77>:0 
  at Unity.Netcode.Transports.UTP.UnityTransport.Update () [0x000c3] in <cb73a45f761942c68e8e5de2a8859e77>:0 

Environment

• OS: Windows 11
• Unity Version: 2022.3.30f1
• Netcode Version: 1.11.0
• Unity Transport Version: 1.4.1

[NoelStephensUnity]

Are you using the default UnityTransport or a 3rd party transport?

[scs-b]

We're using the default UnityTransport, installed automatically as a NGO 1.11.0 dependency.

[NoelStephensUnity]

@scs-b
I have the same question as #3153 about using relay and whether it is a pure server or host you are using. This will help me narrow down where the issue is coming from.

For both this issue and #3153, if you are using Unity relay then my next question is whether you have you run the same kind of burn in tests locally without the use of relay or any services and if so do you get the same issue ?