Nexus 5K basic instalation guide VPC
Basic Cisco Nexus 5K installation guide (VPC)
1. Basic topology two Nexuses with VPC link between them and MGMT interfaces connected to OOB switch:
2. MGMT interface configuration:
N5K-A(config)# int mgmt 0 N5K-A(config-if)# ip address 192.168.3.100/24 N5K-A(config-if)# vrf member management (add interface to preconfigured VRF management)
3. Default gateway configuration for VRF management
N5K-A(config)# vrf context management N5K-A(config-vrf)# ip route 0.0.0.0/0 192.168.3.254
TIP: Each procedure like ping, trace route, copy via addresses on the interface management should be done via VRF management:
N5K-A# ping 192.168.3.254 PING 192.168.3.254 (192.168.3.254): 56 data bytes ping: sendto 192.168.3.254 64 chars, No route to host Request 0 timed out ping: sendto 192.168.3.254 64 chars, No route to host N5K-A# ping 192.168.3.254 vrf management PING 192.168.3.254 (192.168.3.254): 56 data bytes 64 bytes from 192.168.3.254: icmp_seq=0 ttl=63 time=2.183 ms 64 bytes from 192.168.3.254: icmp_seq=1 ttl=63 time=2.043 ms
4. Upgrade Procedure:
Step1: Select and download the kickstart and system software files to a local server.
Step2: Verify that the required space is available in the bootflash: directory for the image file(s) to be copied.
N5K-A# dir bootflash: 0 Jan 01 07:54:28 2009 20090101_075428_poap_4561_init.log 0 Jan 01 08:14:55 2009 20090101_081455_poap_4472_init.log 4165 Apr 02 14:27:59 2009 20090402_141959_poap_4444_init.log 54390 Apr 03 08:14:45 2009 20090403_073813_poap_4472_init.log 324 Apr 03 12:32:20 2009 MDS201308200524288410.lic 592 Apr 03 13:00:52 2009 MDS201308200601208920.lic 319 Apr 04 14:31:08 2009 MDS201308210728363480.lic 248 Apr 03 11:53:17 2009 convert_pfm1.log 342 Apr 03 11:53:17 2009 fcoe_mgr_cnv.log 1123 Apr 03 11:52:16 2009 fwm_pre_issu_dump.txt 268 Jan 01 07:57:22 2009 license_SSI1705099E_3_1.lic 34672128 May 20 07:48:12 2013 n5000-uk9-kickstart.6.0.2.N1.2.bin 35589120 Apr 03 09:34:56 2009 n5000-uk9-kickstart.6.0.2.N2.1.bin 238082390 May 20 07:49:48 2013 n5000-uk9.6.0.2.N1.2.bin 244168123 Apr 03 09:34:10 2009 n5000-uk9.6.0.2.N2.1.bin 4542 Apr 03 11:52:16 2009 stp.log.1 4096 Jan 01 07:53:18 2009 vdc_2/ 4096 Jan 01 07:53:18 2009 vdc_3/ 4096 Jan 01 07:53:18 2009 vdc_4/ 641 Apr 03 11:53:17 2009 vfc_cnv.log 4096 Jan 01 07:53:19 2009 virt_strg_pool_bf/ Usage for bootflash://sup-local 670674944 bytes used 980230144 bytes free 1650905088 bytes total
TIP: We recommend that you keep the kickstart and system image files for at least one previous software release to use if the new image files do not load successfully.
Step3: (Optional) If you need more space on the bootflash, delete unnecessary files to make space available.
N5K-A# delete bootflash:n5000-uk9-kickstart.4.0.1a.N1.0.62.bin N5K-A# delete bootflash:n5000-uk9.4.0.1a.N1.0.62.bin
Step4: Copy the new kickstart and system images to the switch bootflash by using a transfer protocol such as ftp, tftp, scp, or sftp. The examples in this procedure use scp.
copy scp://user@192.168.3.1/n5000-uk9.4.2.1.N1.1.bin bootflash:n5000-uk9.4.2.1.N1.1.bin copy scp://user@192.168.3.1/n5000-uk9-kickstart.4.2.1.N1.1.bin bootflash:n5000-uk9-kickstart.4.2.1.N1.1.bin
Step5: Enter the show install all impact command to display the impact of the upgrade.
N5K-A# show install all impact kickstart bootflash:n5000-uk9-kickstart.4.2.1.N1.1.bin system bootflash:n5000-uk9.4.2.1.N1.1.bin Verifying image bootflash:/n5000-uk9-kickstart.4.2.1.N1.1.bin for boot variable "kickstart". [####################] 100% -- SUCCESS Verifying image bootflash:/n5000-uk9.4.2.1.N1.1.bin for boot variable "system". [####################] 100% -- SUCCESS Verifying image type. [########### ] 50% [####################] 100% -- SUCCESS Extracting "system" version from image bootflash:/n5000-uk9.4.2.1.N1.1.bin. [####################] 100% -- SUCCESS Extracting "kickstart" version from image bootflash:/n5000-uk9-kickstart.4.2.1.N 1.1.bin. [####################] 100% -- SUCCESS Extracting "bios" version from image bootflash:/n5000-uk9.4.2.1.N1.1.bin. [####################] 100% -- SUCCESS Extracting "fex" version from image bootflash:/n5000-uk9.4.2.1.N1.1.bin. [####################] 100% -- SUCCESS Notifying services about system upgrade. [####################] 100% -- SUCCESS Compatibility check is done: Module bootable Impact Install-type Reason ------ -------- -------------- ------------ ------ 1 yes disruptive reset Reset due to single supervisor 100 yes disruptive reset Reset due to single supervisor Images will be upgraded according to following table: Module Image Running-Version New-Version Upg-Required ------ ---------- ---------------------- ---------------------- ------------ 1 system 4.1(3)N1(1) 4.2(1)N1(1) yes 1 kickstart 4.1(3)N1(1) 4.2(1)N1(1) yes 1 bios v1.3.0(09/08/09) no 100 fex 4.1(3)N1(1) 4.2(1)N1(1) yes
Step6: Enter the install all command to install the new images, specifying the new image names that you downloaded in the previous step.
N5K-A# install all kickstart bootflash:n5000-uk9-kickstart.4.2.1.N1.1.bin system bootflash:n5000-uk9.4.2.1.N1.1.bin Verifying image bootflash:/n5000-uk9-kickstart.4.2.1.N1.1.bin for boot variable "kickstart". [####################] 100% -- SUCCESS Verifying image bootflash:/n5000-uk9.4.2.1.N1.1.bin for boot variable "system". [####################] 100% -- SUCCESS Verifying image type. [####################] 100% -- SUCCESS Extracting "system" version from image bootflash:/n5000-uk9.4.2.1.N1.1.bin. [####################] 100% -- SUCCESS Extracting "kickstart" version from image bootflash:/n5000-uk9-kickstart.4.2.1.N1.1.bin. [####################] 100% -- SUCCESS Extracting "bios" version from image bootflash:/n5000-uk9.4.2.1.N1.1.bin. [####################] 100% -- SUCCESS Extracting "fex" version from image bootflash:/n5000-uk9.4.2.1.N1.1.bin. [####################] 100% -- SUCCESS Notifying services about system upgrade. [####################] 100% -- SUCCESS Compatibility check is done: Module bootable Impact Install-type Reason ------ -------- -------------- ------------ ------ 1 yes disruptive reset Reset due to single supervisor 100 yes disruptive reset Reset due to single supervisor Images will be upgraded according to following table: Module Image Running-Version New-Version Upg-Required ------ ---------- ---------------------- ---------------------- ------------ 1 system 4.1(3)N1(1) 4.2(1)N1(1) yes 1 kickstart 4.1(3)N1(1) 4.2(1)N1(1) yes 1 bios v1.3.0(09/08/09) no 100 fex 4.1(3)N1(1) 4.2(1)N1(1) yes Switch will be reloaded for disruptive upgrade. Do you want to continue with the installation (y/n)? [n] y Install is in progress, please wait. Setting boot variables. [####################] 100% -- SUCCESS Performing configuration copy. [####################] 100% -- SUCCESS Module 1: Refreshing compact flash and upgrading bios/loader/bootrom. Warning: please do not remove or power off the module at this time. [####################] 100% -- SUCCESS Pre-loading modules. [This step might take up to 20 minutes to complete - please wait.] [# ] 0%2010 Jun 10 18:27:25 N5K1 %$ VDC-1 %$ %SATCTRL-2-SATCTRL_IMAGE: FEX100 Image update in progress. [##### ] 20% [###### ] 25%2010 Jun 10 18:32:54 N5K1 %$ VDC-1 %$ %SATCTRL-2-SATCTRL_IMAGE: FEX100 Image update complete. Install pending [####################] 100% -- SUCCESS Finishing the upgrade, switch will reboot in 10 seconds. switch# switch# switch# writing reset reason 31, Broadcast message from root (Thu Jun 10 18:33:16 2010): INIT: Sending processes the TERM signal Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "stp" (PID 2843) is forced exit. Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "netstack" (PID 2782) is forced exit. Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "Security Daemon" (PID 2706) is forced exit. Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "Cert_enroll Daemon" (PID 2707) is forced exit. Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "igmp" (PID 2808) is forced exit. Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "Radius Daemon" (PID 2806) is forced exit. Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "AAA Daemon" (PID 2708) is forced exit. Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "vshd" (PID 2636) is forced exit. Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "vlan_mgr" (PID 2737) is forced exit. Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "vdc_mgr" (PID 2681) is forced exit. Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "urib" (PID 2718) is forced exit. Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "u6rib" (PID 2717) is forced exit. Jun 10 18:33:17 %TTYD-2-TTYD_ERROR TTYD Error ttyd bad select Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "statsclient" (PID 2684) is forced exit. Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "smm" (PID 2637) is forced exit. Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "res_mgr" (PID 2688) is forced exit. Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "licmgr" (PID 2641) is forced exit. Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "l3vm" (PID 2715) is forced exit. Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "icmpv6" (PID 2781) is forced exit. Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "eth_dstats" (PID 2700) is forced exit. Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "arp" (PID 2780) is forced exit. Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "fs-daemon" (PID 2642) is forced exit. Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "ascii-cfg" (PID 2704) is forced exit. Jun 10 18:33:17 %LIBSYSMGR-3-SIGTERM_FORCE_EXIT Service "adjmgr" (PID 2771) is forced exit. Jun 10 18:33:19 Unexporting directories for NFS kernel daemon...done. Stopping NFS kernel daemon: rpc.mountd rpc.nfsddone. Unexporting directories for NFS kernel daemon... done. Stopping portmap daemon: portmap. Stopping kernel log daemon: klogd. Sending all processes the TERM signal... done. Sending all processes the KILL signal... done. Unmounting remote filesystems... done. Deactivating swap...umount: none busy - remounted read-only done. Unmounting local filesystems...umount: none busy - remounted read-only done. mount: you must specify the filesystem type Starting reboot command: reboot Rebooting... Restarting system.
Step6: Verify that the switch is running the required software release.
N5K-A# show version Cisco Nexus Operating System (NX-OS) Software TAC support: http://www.cisco.com/tac Documents: http://www.cisco.com/en/US/products/ps9372/tsd_products_support_series_home.html Copyright (c) 2002-2013, Cisco Systems, Inc. All rights reserved. The copyrights to certain works contained herein are owned by other third parties and are used and distributed under license. Some parts of this software are covered under the GNU Public License. A copy of the license is available at http://www.gnu.org/licenses/gpl.html. Software BIOS: version 3.6.0 loader: version N/A kickstart: version 6.0(2)N2(1) system: version 6.0(2)N2(1) Power Sequencer Firmware: Module 1: version v1.0 Module 3: version v5.0 Microcontroller Firmware: version v1.2.0.1 SFP uC: Module 1: v1.0.0.0 QSFP uC: Module not detected BIOS compile time: 05/09/2012 kickstart image file is: bootflash:///n5000-uk9-kickstart.6.0.2.N2.1.bin kickstart compile time: 7/24/2013 3:00:00 [07/24/2013 10:49:21] system image file is: bootflash:///n5000-uk9.6.0.2.N2.1.bin system compile time: 7/24/2013 3:00:00 [07/24/2013 12:42:28] Hardware cisco Nexus5548 Chassis ("O2 32X10GE/Modular Universal Platform Supervisor") Intel(R) Xeon(R) CPU with 8253856 kB of memory. Processor Board ID FOC171520SH Device name: N5K-A bootflash: 2007040 kB Kernel uptime is 0 day(s), 5 hour(s), 42 minute(s), 52 second(s) Last reset Reason: Unknown System version: 6.0(2)N2(1) Service: plugin Core Plugin, Ethernet Plugin, Fc Plugin
5. License installation:
Step1: Copy the new/exist license to the switch bootflash by using a transfer protocol such as ftp, tftp, scp, or sftp. The examples in this procedure use scp.
N5K-A# copy scp://root@192.168.3.3/tftp/MDS201308210736044450.lic bootflash:MDS201308210736044450.lic
Step2: Perform the installation by entering the install license command on the active supervisor module from the switch console.
N5K-A# install license bootflash:license_file.lic Installing license ..done
Step3: View all license files installed on the switch using the show license command.
N5K-A# show license MDS201308200524288410.lic: SERVER this_host ANY VENDOR cisco INCREMENT LAN_ENTERPRISE_SERVICES_PKG cisco 1.0 permanent uncounted \ VENDOR_STRING=<LIC_SOURCE>MDS_SWIFT</LIC_SOURCE><SKU>L-N55-LAN1K9=</SKU> \ HOSTID=VDH=SSI1705099E \ NOTICE="<LicFileID>20130820052428841</LicFileID><LicLineID>1</LicLineID> \ <PAK>3231J7961B3</PAK>" SIGN=DDA6E6404312 MDS201308200601208920.lic: SERVER this_host ANY VENDOR cisco N5K-A# show license usage Feature Ins Lic Status Expiry Date Comments Count -------------------------------------------------------------------------------- FCOE_NPV_PKG No - Unused - FM_SERVER_PKG No - Unused - ENTERPRISE_PKG Yes - Unused Never - FC_FEATURES_PKG Yes - In use Never - VMFEX_FEATURE_PKG Yes - Unused Never - ENHANCED_LAYER2_PKG No - Unused - LAN_BASE_SERVICES_PKG Yes - In use Never - LAN_ENTERPRISE_SERVICES_PKG Yes - Unused Never - --------------------------------------------------------------------------------
6. VPC creation:
Step1: Create VPC domain
N5K-A(config)# vpc domain 1
Step2: VPC configuration
N5K-A(config-vpc-domain)# role priority 2000 (less is better) N5K-A(config-vpc-domain)# peer-keepalive destination 192.168.3.101 source 192.168.3.100 (use MGMT interfaces on both devices) N5K-A(config-vpc-domain)# delay restore 120 N5K-A(config-vpc-domain)# auto-recovery
Step3: VPC on the peer device:
N5K-B(config)# vpc domain 1 N5K-B(config-vpc-domain)# role priority 4000 N5K-B(config-vpc-domain)# peer-keepalive destination 192.168.3.100 source 192.168.3.101 N5K-B(config-vpc-domain)# delay restore 120 N5K-B(config-vpc-domain)# auto-recovery
Step4: Create interface port-channel for VPC: (for both devices)
N5K-A(config)# interface port-channel 1 N5K-A(config-if)# switchport mode trunk N5K-A(config-if)# vpc peer-link
Step5: Configure port-channel interfaces: (for both devices)
N5K-A(config)# int eth1/1 N5K-A(config-if)# switchport mode trunk N5K-A(config-if)# channel-group 1 mode active N5K-A(config)# int eth1/2 N5K-A(config-if)# switchport mode trunk N5K-A(config-if)# channel-group 1 mode active
Step6: Connect interfaces between the nexuses:
Step7: Check the VPC status:
N5K-A# show vpc Legend: (*) - local vPC is down, forwarding via vPC peer-link vPC domain id : 1 Peer status : peer adjacency formed ok vPC keep-alive status : peer is alive Configuration consistency status : success Per-vlan consistency status : success Type-2 consistency status : success vPC role : primary Number of vPCs configured : 0 Peer Gateway : Disabled Dual-active excluded VLANs : - Graceful Consistency Check : Enabled Auto-recovery status : Enabled (timeout = 240 seconds) vPC Peer-link status --------------------------------------------------------------------- id Port Status Active vlans -- ---- ------ -------------------------------------------------- 1 Po1 up 1,3-4,10-17,101,110,112,166,168-171,180,412
7. TIP! The configuration on the both devices should be same.