You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When large scale of pods ( >150) will be created in parallel Azure CNI will fail with this error:
Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox "c99e647e8519b46b4c492d3076ea2da13e861a39ccb982aca89eb5ec5cafae55": plugin type="azure-vnet" failed (add): Failed to initialize key-value store of network plugin: error Acquiring store lock: timed out locking store" pod="default/k8-parallel-100-joblh8pf-gvn6s"
The text was updated successfully, but these errors were encountered:
The issue is due to the serialized approach of Azure CNI during the pod creation. each CNI process will acquire lock at the beginning of the process and release it at the end and when CNI add calls take place in parallel in large numbers some of them will fail waiting behind the lock. The issue is seen more in Windows.
To address the issue a new CNI version called Statless CNI has been designed and implemented that enable paralle pod creation and removes the process locks. #2276
The first target is for Windows AKS Swift Scenario and rollout has started for K8s 1.30
When large scale of pods ( >150) will be created in parallel Azure CNI will fail with this error:
Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox "c99e647e8519b46b4c492d3076ea2da13e861a39ccb982aca89eb5ec5cafae55": plugin type="azure-vnet" failed (add): Failed to initialize key-value store of network plugin: error Acquiring store lock: timed out locking store" pod="default/k8-parallel-100-joblh8pf-gvn6s"
The text was updated successfully, but these errors were encountered: