This is part three of a three-part series about FabricPath. The slides are from a presentation I presented to my colleagues. The source of the presented information comes from a lot of Googling, day-to-day practice and this very good piece of documentation: “Nexus 7000 FabricPath White Paper Version 2.0″.
Within the FabricPath domain FabricPath assures a loop free layer 2 topology. Even if a loop would form the TTL field in the FabricPath header would prevent the FabricPath frames to loop forever. CE edge ports still need to be protected, in FP, STP is used to protect the CE edge against forming Ethernet loops over the FabricPath domain.
Imagine connecting two CE edge ports of two edge switches together with a cross cable. Without protection the following will happen: A CE broadcast frame is encapsulated in a FabricPath frame at edge switch 2001 (with default TTL of 32) it will be forwarded to every edge switch in the FabricPath domain including switch 2002. Switch 2002 will decapsulate it and send it out of all its CE edge ports. It will be received by switch 2001, the frame will again be encapsulated in FabricPath (with a TTL of 32) and send into the FabricPath domain again. This process would continue forever. This shows that even if it is not possible to create loops within the FabricPath domain that loops created at the CE edge are still forwarded over the FP domain. Luckily this will not happen because one of the CE ports will be blocked by STP. It is extremely important to protect the CE edge, if not done properly Ethernet loops can still form and the FP domain will forward looped frames forever!
STP at the CE edge also allows interaction with classical (non FabricPath) switches.
From CE perspective the FabricPath domain acts as a single bridge. This means that all switches send the same bridge ID within their BPDUs send out of the CE interfaces. This bridge ID is a reserved MAC address: C84C.75FA.6000. BPDUs are not forwarded between FabricPath switches, the result of this is that every edge switch forms its own spanning-tree domain. In a topology as shown on a the sheet this is no problem but in more complex topologies as shown on the next sheet this can lead to convergence problems.
NOTE: the FabricPath bridge must be the root bridge! If a superior BPDU is received on one of the CE interfaces then this port is suspended. This is not configurable, this behavior is part of the FP STP design!
BPDUs are not forwarded between FabricPath interfaces, thus also TCN BPDUs are not forwarded. This can lead to convergence problems.
The left topology shows CE switches S1 and S2 connected to edge switches 2001 and 2002. CE switch S3 is connected to S1 and S2. Edge switches 2001 and 2002 behave like a single STP root bridge, the result of this is that STP at S3 blocks its port towards S2 to break the loop in this network. The MAC table of edge switch 2001 learned the MAC of host A via its interface connecting towards S1. this topology works perfectly until the link between S1 and S3 fails, this is shown in the right hand figure. After failure of the link, switch S3 will transition the interface connected towards S2 from blocking into forwarding state. During this transition a TCN BPDU will be send. S2 and edge switch 2002 will flush their MAC tables after receiving the TCN BPDU. The TCN BPDU will however never reach switch 2001, switch 2001 still points to S1 for MAC address A. Convergence is not optimal in this situation. How this problem can be solved is shown in the next slide.
Convergence can be optimized by allowing TCN BPDUs to travel over the FabricPath domain. This is possible by configuring spanning-tree domains at the edge switches. Every spanning tree domain is assigned a unique ID and TCN BPDUs are forwarded between edge member switches of this STP domain. The edge switches that receive the TCN will also propagate them back into the CE network. The flushing problem as explained in the previous sheet is resolved.
VPC can be used with FabricPath but only a modified version of VPC called VPC+ can work together with FabricPath. The differences between VPC and VPC+ are:
- The peer link interfaces are configured as FabricPath core interfaces.
- A FabricPath virtual switch ID is configured for VPC+
The FabricPath virtual witch ID that is used under the VPC+ configuration is used in the FabricPath network to address the VPC+ peers as a single FabricPath switch. Just like any other SID this virtual SID must also be unique. All frames originated form a VPC+ LAG will use this virtual SID in the OSA.
It is possible to define multiple topologies in FabricPath. VLANS are assigned to topologies and these topologies are assigned to interfaces. Every topology has its own MDT trees and roots. Also IS-IS parameters can be configured per topology. This example shows a shared twin datacenter network, customers residing in topology 0 can host servers in both data center locations. Customers in topology 1 or 2 can only host servers within one of the data center locations.
The opinions expressed in this blog are my own views and not those of Cisco.