Nexus 5K FC and FCoE basic guide
Nexus 5K FC and FCoE basic guide
Introduction: for this installation guide we will use single Nexus with one server connected via FCoE and Storage connected via FC.
1. The FC license should be installed on the Nexus:
Nexus# show license usage
Feature Ins Lic Status Expiry Date Comments
Count
--------------------------------------------------------------------------------
FCOE_NPV_PKG No - Unused -
FM_SERVER_PKG No - Unused -
ENTERPRISE_PKG Yes - Unused Never -
FC_FEATURES_PKG Yes - In use Never -
VMFEX_FEATURE_PKG No - Unused -
ENHANCED_LAYER2_PKG No - Unused -
LAN_BASE_SERVICES_PKG No - Unused -
LAN_ENTERPRISE_SERVICES_PKG No - Unused -
--------------------------------------------------------------------------------
2. Enable features fcoe, npiv, fport-channel-trunk, lldp
Nexus(conf)# feature fcoe Nexus(conf)# feature npiv Nexus(conf)# feature fport-channel-trunk Nexus(conf)# lldp Nexus# show run | i feature feature fcoe feature vmfex logging level feature-mgr 0 feature npiv feature fport-channel-trunk feature vrrp feature tacacs+ feature interface-vlan feature lacp feature dhcp feature vpc feature lldp feature fex
3. Create VLAN for FCoE, it will be bound to VSAN. The recommendation is to use same VLAN and VSAN number:
Nexus(conf)# vlan 200 Nexus(conf)# name vsan_vlan Nexus(conf)# fcoe vsan 200 (binding vlan to vsan)
4. Create virtual Fibre Channel (vfc) interfaces. Each physical Nexus port that will carry FCoE traffic must have a corresponding vfc interface. Generally, you will want to create the vfc interface with the same number as the physical interface
Nexus(conf)# interface vfc 25 Nexus(conf-if)# bind interface ethernet 1/25 Nexus(conf-if)# switchport description Server-A Nexus(conf-if)# no shutdown
5. At this point the vfc interface is created, but it won’t work yet; you’ll need to place it into a VSAN that is mapped to an FCoE enabled VLAN.
Nexus(conf)# vsan database Nexus(conf-vsan-db)# vsan 200 interface vfc 25 Nexus(conf-vsan-db)# exit
6. By default VFC interface participating as trunk, so the physical interface should be configured as trunk also
Nexus(conf)# interface Ethernet 1/25 Nexus(conf-if)# switchport mode trunk
7. Adding specified VSAN to the VFC interface
Nexus(conf)# interface vfc 25 Nexus(conf-if)# switchport trunk allowed vsan 200
8. Configuring FC interfaces. Regardless the FCoE virtual interfaces, for the FC we should use physical interfaces converted from Ethernet to FC. For this procedure we need to choose one or more interfaces and convert them.
Be aware, converting Eth to FC requesting Nexus reboot.
Nexus(conf)# slot #(slot number)
Nexus(conf-slot)# port (port number) type {ethernet | fc}
Nexus(conf)# slot 1
Nexus(conf-slot)# port 32 type fc
Nexus(conf-slot)# copy running-config startup-config
Nexus(conf-slot)# reload
9. Now we should to prepare zones, zoneset and device aliases for our SAN network. First of all you should get WWNs from server and storage.
Let’s say, we have storage WWN: 10:00:00:00:c9:d0:c0:d7
And server WWN: 22:b0:00:05:9b:77:d4:7f
10. Configure device alias database
Nexus(config)# device-alias database Nexus(config-device-alias-db)# device-alias name Server-A pwwn 22:b0:00:05:9b:77:d4:7f Nexus(config-device-alias-db)# device-alias name Storage-A pwwn 10:00:00:00:c9:d0:c0:d7 Nexus(config-device-alias-db)# device-alias commit (you should use device-alias commit each time when changing database)
11. Zone configuration. Each zone should include at least one server and one storage. Always use only one server at each zone, but you still can use some storage WWNs within same zone.
Nexus(config)# zone name Trololo_zone vsan 200 Nexus(config-zone)# member pwwn 10:00:00:00:c9:d0:c0:d7 Nexus(config-zone)# member pwwn 22:b0:00:05:9b:77:d4:7f
12. Configure zoneset. Each zoneset can include different amount of zones.
Nexus(config)# zoneset name Trololo vsan 200 Nexus(config-zoneset)# member Trololo_zone Nexus(config-zoneset)# member Another_zone Nexus(config-zoneset)# exit Nexus(config)# zoneset activate name Trololo vsan 200
Troubleshooting steps
- To check which WWNs you see and from which interfaces use command show flogi database
flogi database
Nexus# show flogi database
--------------------------------------------------------------------------------
INTERFACE VSAN FCID PORT NAME NODE NAME
--------------------------------------------------------------------------------
fc2/1 2002 0x8d00a1 50:0a:09:82:98:63:e4:47 50:0a:09:80:88:63:e4:47
[na_tkny_vi01_0d]
fc2/2 2002 0x8d00a0 50:0a:09:82:88:63:e4:47 50:0a:09:80:88:63:e4:47
[na_tkny_vi02_0d]
You can see WWN, interface and name from aliases database. At the flogi database you will see WWNs from FC and FCoE
- To check WWNs from FCoE use command show fcoe database
------------------------------------------------------------------------------- INTERFACE FCID PORT NAME MAC ADDRESS ------------------------------------------------------------------------------- vfc3 0x490100 21:00:00:1b:32:0a:e7:b8 00:c0:dd:0e:5f:76
- Use show vsan command to check configured VSANs
Nexus# show vsan
vsan 1 information
name:VSAN0001 state:active
interoperability mode:default
loadbalancing:src-id/dst-id/oxid
operational state:down
vsan 2002 information
name:VSAN2002 state:active
interoperability mode:default
loadbalancing:src-id/dst-id/oxid
operational state:up
vsan 4079:evfp_isolated_vsan
vsan 4094:isolated_vsan
- Use show zone command to check already configured zones
Nexus# show zone zone name Trololo1 vsan 2002 pwwn 21:00:00:24:ff:41:f1:47 [Server-A] pwwn 50:0a:09:82:98:63:e4:47 [Storage-A] zone name Trololo2 vsan 2002 pwwn 21:00:00:24:ff:41:f1:47 [Server-B] pwwn 50:0a:09:82:88:63:e4:47 [Storage-B]
- Use show zoneset to check configured zonesets and zones attached to zonesets
Nexus# show zoneset
zoneset name SAN_B vsan 2002
zone name Trololo1 vsan 2002
pwwn 21:00:00:24:ff:41:f1:47 [Server-A]
pwwn 50:0a:09:82:98:63:e4:47 [Storage-A]
zone name Trololo2 vsan 2002
pwwn 21:00:00:24:ff:41:f1:47 [Server-B]
pwwn 50:0a:09:82:88:63:e4:47 [Storage-B]
- Physical and logical interfaces check:
Nexus# show interface fc 1/32 Nexus# show interface eth 1/25 Nexus# show interface vfc 25
- SAN hierarchy:
-VSAN A
|- zoneset 1
|- zone A
|- zone B
-VSAN B
|- zoneset 2
|- zone C
|- zone D
|- zoneset 3
|- zone E
|- zone F