Between 11:43-11:47 we found one of our switches was having a mac-address flapping event on multiple ports. This caused SAN volumes to be intermittently lost and then renegotiated multiple times, the flow on effect being some VMs experienced a loss of IO.

We have assigned some static mac address rules to ensure this no longer occurs while we investigate the underlying cause.