Ali Abbas

MPLS – Multiprotocol Label Switching

The only purpose of this post is to demystify MPLS and hopefully give you a solid ground on which to build more knowledge on MPLS.

So what is MPLS?

Simply said, MPLS is just a forwarding/routing mechanism as opposed to traditional IP routing (static route/BGP/OSPF/RIP etc…) which enables faster IP processing/forwarding by integrating layer 2 information with layer 3 routing…

Having said that, we ought to remember than in traditional IP routing, whenever a router receives a packet, the packet is analyzed (DST field) against its forwarding table to determine the next hop. With MPLS, the packets no longer needs to be examined by the router as they are simply routed by looking at the packet tag and forwarded on a pre-configured Label Switch Path (LPS)

Before we go on, let’s define some terminology Read more…

I am not going to go over in details on the hoods of PPP (another post will be written for this purpose).

A deep understanding of Point-to-Point WAN networks isn’t required as we will barely scratch that surface when dealing with the “unnumbered IP command”.

So what is the unnumbered IP command?

The unnumbered IP command enables you to enable IP routing/processing on an interface by assigning it an IP already in used of another interface. That is to say, the unnumbered IP command enables you to have 2 interfaces, a multi-access interface and a point to point interface (often a serial port) to share the same IP address.

–> Now why would you do that? Simple, to save IPs and Read more…

This is probably one of the most ignored and forgotten feature of IOS since 12.2(25)S and 12.2(27)SBC.

I am positing it  here as I never stopped coming across routers and switches with this feature not active. Please note you need to enough memory,to use this feature; that is to say, the available space in memory to hold a copy of the interfaces configuration.

As you may guess, a router or switch with a monstrous configuration, can take a while to display the running configuration when issuing

edge1#sh run

as it needs to fetch all the configuration from various places in memory.

Quoting Cisco

When invoked, NVGEN queries each system component and each instance of interface or other configuration objects. A running configuration file is constructed as NVGEN traverses the system performing these queries.

To speed things up, IOS ships in with a feature called Configuration Generation Performance Enhancement , which caches the interfaces configurations, which in return speed up NVGEN.

Activate caching with

edge1(config)#parser config cache interface

and voilà.

This post assumes that you have a basic understanding of OSPF… if not, I suggest jumping over http://en.wikipedia.org/wiki/OSPF for a first quick read. However for the sake of this post, I will go over some basic reminders.

The “hello” packet

The OSPF routers sends a periodic packet referred to as the hello packet ‘multicast 224.0.0.5′ which is composed of the OSPF header + different fields ID necessary for routers to neighbor and become adjacent. The hello packet is by default sent at a 10 seconds interval on a multi-access network and each 30 second on a point to point network.

The HELLO PACKET (roughly 50 bytes) looks as following

[ OSPF HEADER ] | Network Mask | Hello Interval | Options | Router Priority | Router Dead Interval | DR | BDR | Neighbor

The OSPF HEADER (20 bytes) looks as following

Version number | Type | Packet Length | Router ID | Area ID | Checksum

The neighboring and adjacent process

Like I explained earlier, OSPF uses the hello packet not only to discover another peer router, but also to neighbor with this router. For 2 OSPF routers to neighbor, they must belong to the same AREA (Area ID), use the same Authentication schema, have the same hello and dead intervals. Past the agreement phase, the routers becomes “neighbors”.

Only when they are neighbors, OSPF routers will start exchanging their database… this process is referred as Adjacency.

Now let’s imagine a singular segment on which we have 10 OSPF routers… in theory, each router would peer with each other and start exchanging their database with each others. The number of adjacency is then calculated as followed

(n (n – 1) ) / 2

So 10 routers, will give us 45 Adjacency

To minimize the amount of information shared, OSPF will elect a Designated Router (DR) and a Backup Designated Router (BDR). Once the DR and BDR are elected, every other OSPF router will start exchanging database only with the DR and BDR and no longer with each other.

Now keep in mind, as we said earlier OSPF routers use multicast IP 224.0.0.5 to send their hello packets but also exchange their databases… in presence of a DR/BDR, the other routers will send their updates on multicast 224.0.0.6, which in return the DR/BDR will resend on multicast 224.0.0.5

So how does the DR and BDR election takes place?

It is quite simple, if you are used to the switch root bridge election, this will not look much different. The BDR and DR takes place through the HELLO PACKET by comparing the Priority ID (which if you recall is located in the hello packet as shown earlier).

The router with the highest Priority ID is elected the Designated Router (DR), the next router with second highest Priority ID will become the BDR. Now keep in mind, by default all router interfaces have a priority ID of 1… if on a particular segment, all the Priority ID of all routers match, the Router ID (OSPF header) will then be the next ID to compare in order to elect the DR/BDR. Again in the same mind set, the OSPF router with the highest Router ID will be elected the DR or BDR.

Keep in mind that once the DR/BDR are elected, if a new OSPF router is added with the highest priority of all, the DR/BDR will not change… to start the election process, you will have to clear up the OSPF process

Once the DR and BDR are elected, the BDR will only listen to the exchange between the peers and the DR and elects itself as the DR if the current DR was to fail.

As a last thing to remember, without DR/BDR, we calculated 45 Adjacency for 10 routers on a multi-access segment. Now how many adjacency do we have with a DR and BDR? Simple!

2*n – 1 –> 2×10 – 1 = 19 Adjacency … so from 45 Adjacency, we dropped down to 19 Adjacency with a DR and a BDR.

If you were to only elect a DR without BDR, then you would naturally obtain 9 Adjacency.

To keep in mind

• If you do not want a router to participate in the DR/BDR election, sets its Priority ID to 0, it will then be shown as DROTHER.
• You can override the RID of the OSPF router by creating a loopback interface with a different IP than the one used on the router’s interface
• The BDR and DR election only take place on broadcast and non-broadcast multi-access… That is to say routers on serial WAN would not have a BDR/DR election

It is important to note that TCP flow control mechanism as well as Ethernet flow control mechanism are completely 2 different mechanism, which strive to achieve the same unique goal but when in used, are completely unaware of each other.

As a matter of fact, Ethernet flow control can fully alienate your network if not planned and used carefully …

So What is TCP flow control?

Flow control is a mechanism implemented in the TCP stack which enables a receiver endpoint to notify a sender that it can no longer receive data in its buffer. The buffer size is what is simply referred as the TCP Window Size, and is transmitted in each ACK. The receiver can therefore let the sender know, how much bytes it is able to process at once.

[ let's assume, the receiver machine can only process 8K in its buffer]

(sender) <——– ACK 1022 WIN 4096 <——– (receiver)
(sender) ———> 4K | SEQ 1022 ————–> (receiver)

[ assuming that the buffer of the receiver is now full with the first 4K ]

(sender) <——– ACK 2024 WIN 0 <——– (receiver)

[ the sender is now "blocked" from sending more data till the receiver sends a second acknowledgment]

(sender) <——– ACK 2024 WIN 4096 <——– (receiver)

Ok so now, what is Ethernet flow control?

From layer 4 (TCP flow control), we jump now to layer 2 (Ethernet flow control).

Ethernet flow control is different from TCP flow control as it makes usage of the MAC control frame “pause frame” to notify the end device to stop sending frames. It is important to keep in mind that, the sender of the pause frame sets the 2bit quanta time which defines how long the endpoint must wait to start retransmitting frames and finally to keep in mind that pause frames are not forwarded. That is to say, a MAC control frame will not be forwarded through a trunk port, nor to the adjacent device.

What is the problem when using Ethernet flow control?

If you have read so far, you can start guessing what may occur, if you have “ethernet flow control” enabled on your switch. Instead of dropping the packets when the tcp window size is exhausted, the switch will not drop the packet but generate its own pause frame and send it to the sender host. Now keep in mind that pause frames completely cease all transmission on the data link layer… that is to say if meanwhile PCX was getting a file of PCB, it would as well be “paused”. Because pause frame only work on layer 2 “data-link”, all communications associated to the targeted switch port, will completely cease for the pause period of time.

But what happens meanwhile with the TCP flow control?

Like said earlier, the TCP flow control isn’t aware of the data flow control… the TCP flow control allows TCP to throttle the amount of data it is sending, because the switch no longer drops packets due to “ethernet flow control”, TCP becomes unaware that it is sending more data than what the endpoint window size can receive and thus keeps increasing the amount of data it is sending… the result is an overloaded receiver and a switch which keeps generating pause frames, till the TCP flow control detects congestion and readjusts the sending window.

And what happens when you have IGMP snooping off?

Imagine a multicast scenario, where you have a server and a workstation on 2x 1Gb port and another workstation on a 100Mb. If the server starts sending multicast packets at 1Gpbs (in the absolute ), Ethernet flow control will directly start to throttle down the speed at which the server sends the packet to the lowest port speed of the switch. Remember we are talking multicast here and because packets would be delivered to the 100Mb port… Ethernet flow control on the switch would force the server to only send at 100Mbps. While this is good in practice, remember without IGMP snooping,the switch would be sending all the multicast packets to all the switch ports, thus to endpoints which are unsolicited in the mutlicast group, will cause Ethernet flow control to trigger bad and slow performance.

Conclusion

IGMP snooping has always been a problem in VRRP setup (aka. Checkpoint HA), causing fluctuation on the interface state (referred as flapping interfaces).

While it is possible to disable IGMP per VLAN, I would recommend disabling IGMP snooping per MAC Multicast Address (i.e 01:50:5e:xx:yy:zz)