How to Troubleshoot Nexus 5500s and 1 GE SFPs?

May 23 2013 , Written by Cisco & Cisco Router, Network Switch Published on #Cisco Switches - Cisco Firewall

The newer versions of NX-OS support 1G or 10G on the N5K, but the Nexus5000 is not auto-speed sensing. A mini case study follows.

One of my customers just put some GLC-SX-MM optics and a fiber patch between a Catalyst 3750 and a Nexus5000. He happened to look at status on Network Assistant on the 3750, and this error appeared:

Description: Gi1/1/1: This port has been disabled because Non Compliant Gigabit Interface Converter (GBIC) connector detected.
Recommendation: Replace connector with cisco compliant Gigabit Interface Converter (GBIC) connector. Refer switch technical documentation to determine cisco compliant connector. Enable the port again.

This seemed odd, since the GLC-SX-MM is supported on both devices. I tried shut/no shut on both ends of the port, but that did not help. I then wondered - does he have a bad SFP? I poked around a bit.

The CLI show log info on the 3750 switch was also unhappy:

*Apr 16 01:48:41.190: %PHY-4-SFP_NOT_SUPPORTED: The SFP in Gi1/1/1 is not supported
*Apr 16 01:48:41.190: %PM-4-ERR_DISABLE: gbic-invalid error detected on Gi1/1/1, putting Gi1/1/1 in err-disable state

Strange, the GLC-SX-MM is supported on the Catalyst 3750. The interface capabilities looked fine:

3750_sw1#sh int gig 1/1/1 capa
GigabitEthernet1/1/1
Model:                 WS-C3750X-48P
Type:                  1000BaseSX SFP
Speed:                 1000
Duplex:                full
Trunk encap. type:     802.1Q,ISL
Trunk mode:            on,off,desirable,nonegotiate
Channel:               yes
Broadcast suppression: percentage(0-100)
Flowcontrol:           rx-(off,on,desired),tx-(none)
Fast Start:            yes
QoS scheduling:        rx-(not configurable on per port basis),
                         tx-(4q3t) (3t: Two configurable values and one fixed.)
CoS rewrite:           yes
ToS rewrite:           yes
  UDLD:                  yes
Inline power:          no
SPAN:                  source/destination
PortSecure:            yes

Dot1x:                 yes
3750_sw1#

The status on the 3750 was not connected:

3750_sw1#sh int gig 1/1/1 status
Port Name Status Vlan Duplex Speed Type
Gi1/1/1 notconnect 1 auto auto 1000BaseSX SFP
3750_sw1#

I then went to look at the N5K. It claimed it has an invalid SFP of an unkown type:

N5K1# sh int e1/5 status
--------------------------------------------------------------------------------
Port          Name               Status    Vlan      Duplex Speed   Type
--------------------------------------------------------------------------------
Eth1/5       --                 sfpInvali 1         full    10G     SFP-1000BAS
N5K1#
N5K1# sh int e1/5 capa
Ethernet1/5
Model:                 N5K-C5548UP-SUP
Type (SFP capable):    10Gbase-(unknown)
Speed:                 1000,10000
Duplex:                full
Trunk encap. type:     802.1Q
Channel:               yes
Broadcast suppression: no
Flowcontrol:           rx-(off/on),tx-(off/on)
Rate mode:             none
QOS scheduling:        rx-(6q1t),tx-(1p6q0t)
CoS rewrite:           no
ToS rewrite:           no
SPAN:                  yes
UDLD:                  yes
Link Debounce:         yes
Link Debounce Time:    yes
MDIX:                  no
Pvlan Trunk capable:   yes
TDR capable:           no
FabricPath capable:    yes
Port mode:             Switched
FEX Fabric:            yes
N5K1#

The Nexus5000 also claimed there was no transceiver:

N5K1#sh log | inc 1/5

...
2012 Apr 6 18:10:26 N5K1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet1/5 is down (None)
...
2012 Apr 6 19:06:10 N5K1 %ETHPORT-5-IF_HARDWARE: Interface Ethernet1/5, hardware type changed to No-Transceiver

I scratched my head a bit, and then came up with the underlying issue - when installing 1GE optics in the N5K, you need to manually set the speed. If you look back, you will see the previous 'sh int e1/5 status' on the N5K shows a speed of 10G. So after the appropriate "speed 1000" command to the e1/5 interface on the N5K, the link came up on both ends.

N5K1# sh int e1/5 stat
--------------------------------------------------------------------------------
Port Name Status Vlan Duplex Speed Type
--------------------------------------------------------------------------------
Eth1/5 -- connected 1 full 1000 SFP-1000BAS
N5K1#

Summary: The newer versions of NX-OS support 1G or 10G on the N5K, but the N5K is not auto-speed sensing. If you run into something similar, even error messages on the remote device, check the port speed on the N5K.

---Resource from http://www.netcraftsmen.net

More…

Cisco Nexus Switches: Layer 2 Configuration Strategies

Cisco Nexus 6000 Switches: High-Density, Compact Form Factor

Cisco Catalyst 6500 vs. Cisco Nexus 7000 Switch

To Answer the Questions from Cisco Nexus 1000V InterCloud Users and Analysts

Cisco New ASR 5500 is for Next-Generation Mobile Internet

Share this post

Repost0