Sunday, December 15, 2013

MPLS Tools and Tricks - Part 1, show mpls forwarding-table (aka your best friend)

First off, I know it's been a while since I've posted anything. Between failing the CCIE (again), moving and changing jobs... I've been busy. Back off. That being said, I'm getting the itch again, and it's time to start studying, I don't care if it takes 20 tries over 3 different blueprints I'm going to get my number in R&S. So to get things rolling, I thought I'd start with my favorite topic - MPLS.

Let's get some lingo straight first, that way you impress all your friends with new acronyms.

LSR - Label Switch Router. This guy's life is label switching, LSRs make up your MPLS backbone.
LER - Label Edge Router. Your provider edge (PE), and as the name implies the edge of your MPLS network.
LSP - Label Switch Path. The MPLS path for a particular source and destination.

Now that we know who all the players are, lets take a look at our topology

You'll notice we're dealing with more "P" routers than usual, but having a few extra LSRs should help visual the MPLS process. First off, before setting up any VRFs, or doing any l2 vpns let's take a look at probably the single most important command for troubleshooting MPLS.  That's right kids, I'm talking about show mpls forwarding-table. As a sample we'll look at the LFIB (Label Forwarding Information Base) on P1.

P1#show mpls forwarding-table
Local  Outgoing      Prefix            Bytes Label   Outgoing   Next Hop
Label  Label or VC   or Tunnel Id      Switched      interface
16     Pop Label     0             Et1/0
17     Pop Label   0             Et1/1
         Pop Label   0             Et1/0
18     Pop Label             0             Et1/2
19     Pop Label             0             Et1/1
20     Pop Label   0             Et1/2
21     Pop Label     0             Et1/2
         Pop Label     0             Et1/3
22     Pop Label     0             Et1/1
         Pop Label     0             Et1/3
23     Pop Label     0             Et1/1
         Pop Label     0             Et1/2
24     26            0             Et1/2
         26            0             Et1/3
25     Pop Label              0             Et1/3
26     Pop Label    0             Et1/3

Alright... what are looking at right?? I'll get to that, first let's examine why this command is so darn important, simple. By issuing it on our MPLS enabled routers, we can establish our LSP between two given endpoints. After all, MPLS is multiprotocol label switching, so when we're troubleshooting it we should always give the labels the majority of our attention. Now, what are we looking at? The first column is our local labels, generated by the router we're on. The second column, when all is working well, will have either an out going label received (in this case) via LDP or it will show "Pop Label". Without digging too deep, Pop Label comes from penultimate hop popping, really fun to say by the way. This is the process whereby the second to the LSR in an LSP will pop the label off an mpls packet, this is an efficiency mechanism. 

Now back to show mpls forwarding-table. Let's say there is a VPN built between PE1 and PE2, routing tables on the CEs look correct, import export statements look good while you're troubleshooting. However, CE1 and not ping CE2. So spoiler alert, I played proctor and jacked up P3 with some control plane policing, more on that later. Further more, since the LFIB is built off the FIB I adjusted the ospf cost on P4's link to PE2 so that P3 was the preferred path. VPNs are built between the PEs, and for a VPN to work there has to be a fully functional LSP between their peering address (loopbacks in our example). So starting on PE1, we'll verify we have labels to get to (PE2's loopback).

PE1#show mpls forwarding-table
Local  Outgoing      Prefix            Bytes Label   Outgoing   Next Hop
Label  Label or VC   or Tunnel Id      Switched      interface
26     24      0             Et1/0
         26      0             Et1/1

So far so good. We have egress labels of 24 and 26 going to P1 and P2 respectfully. Next hop, I'm going to look at P1, but you could go to either.

P1#show mpls forwarding-table
Local  Outgoing      Prefix            Bytes Label   Outgoing   Next Hop
Label  Label or VC   or Tunnel Id      Switched      interface
24     No Label    1523          Et1/2

Trouble in paradise, P1 shows that it's forwarding the packet out with no label towards P3, and has no other path. Let's see what P2 shows.

P2#show mpls forwarding-table
Local  Outgoing      Prefix            Bytes Label   Outgoing   Next Hop
Label  Label or VC   or Tunnel Id      Switched      interface
26     No Label    126           Et1/3

Same thing, at this point we can jump onto P3 and start diagnosing. First up, look at the labels.

P3#show mpls forwarding-table
Local  Outgoing      Prefix            Bytes Label   Outgoing   Next Hop
Label  Label or VC   or Tunnel Id      Switched      interface
26     No Label    1218          Et1/0

Verify it has any labels

P3#show mpls forwarding-table
Local  Outgoing      Prefix            Bytes Label   Outgoing   Next Hop
Label  Label or VC   or Tunnel Id      Switched      interface
16     No Label    0             Et1/2
         No Label    0             Et1/3
17     No Label            0             Et1/3
18     No Label            0             Et1/2
19     No Label    0             Et1/3
         No Label    0             Et1/1
20     No Label    0             Et1/2
         No Label    0             Et1/1
21     No Label  0             Et1/3
22     No Label  0             Et1/2
23     No Label    0             Et1/2
         No Label    0             Et1/3
24     No Label            0             Et1/1
25     No Label  0             Et1/0
26     No Label    0             Et1/0

Well that's not good! Why doesn't P3 have any labels?? I know I've told you the problem already, but humor me. So what next? We verify the router has any mpls enabled interfaces, then if we have any neighbors.

P3#show mpls interfaces
Interface              IP            Tunnel   BGP Static Operational
Ethernet1/0          Yes (ldp)     No       No  No     Yes
Ethernet1/1          Yes (ldp)     No       No  No     Yes
Ethernet1/2          Yes (ldp)     No       No  No     Yes
Ethernet1/3          Yes (ldp)     No       No  No     Yes
P3#show mpls ldp neighbor


So from this we see that all the expected interfaces are in fact MPLS enabled and using LDP. However we have no LDP. Since LDP is our current environment's mechanism for exchanging labels, this would explain why we're having issues. This point requires you to know a little more about LDP, LDP sends hello messages out on udp port 646. However, the LDP session is actually built between the LDP router-ids (loopback interfaces for us, this is also the LDP/TDP default behavior) on TCP port 646. That being said, you need to check a couple things on R3

Can we ping our neighbors? Can we ping their LDP RIDs (loopbacks here) from our RID? We'll pick on R1 for these tests


Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to, timeout is 2 seconds:
Success rate is 100 percent (5/5), round-trip min/avg/max = 16/20/28 ms
P3#ping source lo0

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to, timeout is 2 seconds:
Packet sent with a source address of
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/21/60 ms

Alright that looks good, so now back our TCP and UDP ports... do we have any access-lists configured?

P3#sh ip access
Extended IP access list 100
    10 permit ospf any any (949 matches)
    20 permit tcp any any eq bgp
    30 permit tcp any eq bgp any
    40 permit udp any any eq 646 (314 matches)
Extended IP access list 101
    10 permit icmp any any (10 matches)
Extended IP access list 103
    10 permit ip any any (2229 matches)

We do! Great... now where are they applied??
P3#show run | inc 101
 match access-group 101
access-list 101 permit icmp any any

"match access-group" sounds like a class-map, let's take a look.

class-map match-all ICMP
 match access-group 101
class-map match-all ALL
 match access-group 103
class-map match-any ROUTING
 match access-group 100

P3#show run policy-map
Building configuration...

Current configuration : 141 bytes
policy-map CoPP
 class ROUTING
 class ICMP
   police rate 100 pps
     exceed-action drop
 class ALL
 class class-default

And finally, where is it applied?

P3#show policy-map interface  <-- Nothing here

P3#show policy-map control-plane <-- Nailed it!
 Control Plane

  Service-policy input: CoPP

    Class-map: ROUTING (match-any)
     ... out ommitted

So now we can modify CoPP to allow for tcp communication on 646. After doing that LDP will come back up, and CE1 will be able to ping CE2. Mystery solved.

1 comment:

  1. Thanks for sharing this nice content blog.Its about the MPLS tools and tricks.Its a good article and happy to hear from you.I need essay writing services details for my essay works.I have to write an effective essay for my school competition on last week.For that I need your helps and need to know the essay writing services details to make my essay perfect.Can you please share the best essay writing services details.Is there any good services and have you experience with one of them?If yes please share it with me.