Cisco Nexus 6000 and 5600 with
Fabric Extender 2000 Switch
Architecture
BRKARC-3452
Nicolas Delecroix
Technical Marketing Engineer
Data Center Group – Nexus Business Unit
[email protected]
Nexus 5000 Evolution
Nexus 5020
56-Port, 2RU
Customers
Nexus 5010
28-Port, 1RU
BRKARC-3452
Nexus 5548
48-Port, 1RU
Nexus 5596T
10Gbase-T, 2RU
Nexus 5596
96-Port, 2RU
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
3
Introducing Cisco Nexus 5600/6000
Nexus 6004
Nexus 5020
56-Port, 2RU
Customers
Nexus 5010
28-Port, 1RU
BRKARC-3452
Nexus 5548
48-Port, 1RU
Nexus 5596T
10Gbase-T, 2RU
Nexus 5596
96-Port, 2RU
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
4
96 port 40G, 4RU
Introducing Cisco Nexus 5600/6000
Nexus 5672
48 port 10G, 6 port 40G, 1RU
Nexus 56128
Nexus 6004
96 port 10G, 8 port 40G, 2RU
Nexus 5020
56-Port, 2RU
Customers
Nexus 5010
28-Port, 1RU
BRKARC-3452
Nexus 5548
48-Port, 1RU
Nexus 5596T
10Gbase-T, 2RU
Nexus 5596
96-Port, 2RU
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
5
96 port 40G, 4RU
Cisco Nexus 6000 and 5600 with
Fabric Extender 2000 Switch
Architecture
BRKARC-3452
Nicolas Delecroix
Technical Marketing Engineer
Data Center Group – Nexus Business Unit
[email protected]
Nexus 5600/6000 Architecture - Agenda
Overview
Fabric Extender
Performance
Internal Architecture
Transceivers/Cables
Access Control List
Analytics
Multicast
Quality of Service
Applications
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
7
Nexus 5672 Chassis
Port-Side View
• All ports provide Ethernet and FCoE
• 16 Unified Ports (orange) also provide 2/4/8G FC
• L2 and L3 VXLAN
48 fixed 1/10G SFP+ interfaces
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
8
6 fixed 40G QSFP interfaces
Cisco Nexus 5672 Chassis
Power Supply and Fans
Power Supply 1+1
Redundancy
Fan Module 2+1
Redundancy
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
Console Mgmt0 USB
9
Nexus 56128 Chassis
Port-Side View
• 48 Fixed 10G SFP+ Ports, 4 fixed 40G QSFP+ ports
• 2 Expansion Slots
• Expansion module provides 24 Unified ports 10G Ethernet/FCoE or 2/4/8G FC,
and 2 ports 40G Ethernet/FCoE
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
10
Cisco Nexus 56128 Chassis
Power Supply and Fans
Power Supply 2+2
Redundancy
Fan Module 3+1
Redundancy
Console Mgmt0 USB
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
11
Cisco Nexus 6004 Chassis
12 QSFP+ ports
Expansion Module
N6K-6004-M12Q
Port-Side View
4RU
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
12
Cisco Nexus 6004 Chassis
Power Supply and Fans
• Six power supply slots; a minimum of three is required. Supports both 3 + 1 and 3 + 3 redundancy.
• Each power supply is rated 1100W, 90—240 VAC.
• The chassis has four fan trays. A minimum of three is required.
Power Supply 3 + 3
Grid Redundancy
or 3 + 1 Redundancy
Console Mgmt0 USB
Fan Module 3+1
Redundancy
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
13
Unified Port Line-Card Expansion Module - N6004X-M20UP
For Nexus 6004
• 20 * 10G SFP+ unified ports expansion module
• 2/4/8G FC port
• 1G/10G and FCoE
• Max number of native FC ports per chassis is 48
(as of NXOS 7.0(2)N1(1) release)
• SFP+ port allows support for a larger variety of
optical transceivers
• Same transceivers supported as on 5600 10G
ports – including 10G long distance, 1G copper
and optical
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
14
Cisco Nexus 5600/6000 Airflow
• The units have a “port side” and a “fan side”. No front or back
• The units support both port side exhaust and port side intake
– Port side exhaust (used for in server rack airflow alignment)
– Port side intake (used for network rack airflow alignment)
• Different power supply and fan modules are required for different airflow
directions
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
15
N5600/6000 and N5500 Comparison
Feature
Nexus 5600/6000
Nexus 5500
L2 Throughput
Line rate
Line rate
L3 Throughput
Line rate
160Gbps
Latency
~1us
1.8us
Native 40GE Support
Yes
4xQSFP module – 10G flows
1GE Support
Yes
Yes
Unified Ports
FEX Scaling
Yes
48 (6004 L2), 24 (6004 L3, 5600)
Yes
24 with L2, 16 with L3
MAC Table
IPv4 Unicast Routes
256K (shared with ARP/ND)
32K
32K
16K
IPv4 Multicast Routes
32K
8K (shared with ARP)
IPv4 ARP/Host Route Entries
128K
16K (shared with IPv6 ND and
mroute)
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
16
N5600/6000 and N5500 Comparison (cont.)
Feature
Nexus 5600/6000
Nexus 5500
IPv6 Unicast Routes
8K (shared with IPv4)
8K (shared with IPv4)
IPv6 ND Entry/Host Route Entries
85K (shared with MAC and ARP)
8K (shared with mroute and ARP)
IPv6 Multicast Routes
32K
8K (shared with ARP, ND. Limit is
with /64 routes)
VLAN
4K
4K
VRF
4K
1K
Segment ID
Yes
No
ACL
4K
4K
L4ops for ACL
24 total (16 for UDP/TCP, 8 for TCP flag)
8 for UDP and 8 for TCP
Packet Buffer
640KB per 10G port
vPC, FabricPath
25MB per 3xQSFP (or 12xSFP) ports
2.1MB per 10G port
Yes
Dynamic Fabric Automation
Yes
No
VXLAN
Yes*
No
*Software roadmap feature
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
17
Yes
Nexus 5600/6000 Architecture - Agenda
Overview
Fabric Extender
Performance
Internal Architecture
Transceivers/Cables
Access Control List
Analytics
Multicast
Quality of Service
Applications
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
18
Nexus 2000 100M/1G Fabric Extender Portfolio
Nexus 2224TP
Nexus 2248TP
Nexus 2248TP-E
2 x 10G SFP+
4 x 10G SFP+
4 x 10G SFP+
Host Facing Ports
24 x 100/1000BASE-T
RJ45
48 x 100/1000BASE-T
RJ45
48 x 100/1000BASE-T
RJ45
Oversubscription
1 : 1.2
1 : 1.2
1 : 1.2
FCoE
N/A
N/A
N/A
Buffer
3.8MB
7.6MB
32MB
No
No
Yes
Uplink Ports
Shared Buffer
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
19
Nexus 2000 1G/10G Fabric Extender Portfolio
Nexus 2232PP
Nexus 2232TM
Nexus 2232TM-E
Nexus 2248PQ
8 x 10G SFP+
8 x 10G SFP+
8 x 10G SFP+
4 x 40G QSFP+
Host Facing Ports
32 x SFP/SFP+
(1/10G)
32 x 1/10GBASE-T
RJ45
32 x 1/10GBASE-T
RJ45
48 x SFP/SFP+
(1/10G)
Oversubscription
1:4
1:4
1:4
1:3
FCoE
Yes
No
Yes (30m / Cat6a/7)
Yes
Buffer
10.2MB
10.2MB
10.2MB
32MB
No
No
No
Yes
Uplink Ports
Shared Buffer
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
20
Nexus 2000 Fabric Extender Scalability
• Nexus 5600 and 6000 support all types of FEX, including HP, Fujitsu, Dell, IBM FEX for
blade servers. Adapter-FEX and VM-FEX are also supported.
Cisco Nexus 2000 FEX
48xFEX
Cisco Nexus 5600/6000
+
Nexus
5000
BRKARC-3452
Nexus 5500
Nexus 5600
Nexus
6004
Nexus
7000
Nexus 7700
L2
12
24
24
48
48
64
L3
x
16
24
24
48
64
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
21
Nexus 5600/6000 Architecture - Agenda
Overview
Fabric Extender
Performance
Internal Architecture
Transceivers/Cables
Access Control List
Analytics
Multicast
Quality of Service
Applications
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
22
Nexus 5672 - 72 x 10G Unicast Throughput
100
90
80
70
60
Throughput
(% of line rate)
50
40
30
20
10
0
RFC 2544 Unicast L2 Port Pair
RFC 2544 Unicast L3 Port Pair
64
100
100
80
100
100
96
100
100
128
100
100
256
100
100
512
100
100
1000
100
100
1024
100
100
Packet Size (Bytes)
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
23
1280
100
100
1518
100
100
2240
100
100
4096
100
100
9100
100
100
Nexus 5672 - 72 x 10G Multicast Throughput
100
90
80
70
Throughput
(% of line rate)
60
50
40
30
20
10
0
RFC 3918 Mcast L2 One to Many Accumulated
RFC 3918 Mcast L3 One to Many Accumulated
RFC 3918 Mcast L2 One to Many Distributed
RFC 3918 Mcast L3 One to Many Distributed
64
100
100
100
100
80
100
100
100
100
96
100
100
100
100
128
100
100
100
100
256
100
100
100
100
512
100
100
100
100
1000
100
100
100
100
1024
100
100
100
100
1280
100
100
100
100
Packet Size (Bytes)
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
24
1518
100
100
100
100
2240
100
100
100
100
4096
100
100
100
100
9100
100
100
100
100
Nexus 5672 – 72 x 10G Unicast 100% Load Latency
1600
1400
1200
1000
Average Latency
(Nano Seconds)
800
600
400
200
0
RFC 2544 unicast L2 port pair
RFC 2544 unicast L3 port pair
64
1001
937
128
1021
963
256
1091
1029
512
1111
1048
1024
1105
1047
1280
1104
1046
Packet Size (Bytes)
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
25
1518
1104
1045
2240
1103
1044
4096
1101
1043
9216
1098
1040
Nexus 6004 Performance – Fully Loaded Switch
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
26
Nexus 6004 Performance – Spirent Traffic Generator
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
27
Nexus 6004 - 96 x 40G Unicast Throughput
100
90
80
70
60
Throughput
(% of line rate)
50
40
30
20
10
0
RFC 2544 Unicast L2 Port Pair
RFC 2544 Unicast L3 Port Pair
64
100
100
80
100
100
96
100
100
128
100
100
256
100
100
512
100
100
1000
100
100
1024
100
100
1280
100
100
1518
100
100
Packet Size (Bytes)
Spirent third party performance report for Nexus 6004
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
28
2240
100
100
4096
100
100
9100
100
100
Nexus 6004 - 96 x 40G Multicast Throughput
Throughput
(% of line rate)
100
90
80
70
60
50
40
30
20
10
0
RFC 3918 Mcast L2 One to Many Accumulated
RFC 3918 Mcast L3 One to Many Accumulated
RFC 3918 Mcast L2 One to Many Distributed
RFC 3918 Mcast L3 One to Many Distributed
64
100
100
100
100
80
100
100
100
100
96
100
100
100
100
128
100
100
100
100
256
100
100
100
100
512
100
100
100
100
1000
100
100
100
100
1024
100
100
100
100
1280
100
100
100
100
1518
100
100
100
100
Packet Size (Bytes)
Spirent third party performance report for Nexus 6004
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
29
2240
100
100
100
100
4096
100
100
100
100
9100
100
100
100
100
Nexus 6004 - 384 x 10G Unicast 100% Load
Latency
1.4
1.2
Average Latency1
(microseconds)
0.8
0.6
0.4
0.2
0
RFC 2544 Unicast L2
Port Pair
RFC 2544 Unicast L3
Port Pair
64
80
96
128
256
512
1000
1024
1280
1518
2240
4096
9100
1.028
0.99
1.005
1.023
1.121
1.118
1.111
1.116
1.11
1.107
1.11
1.108
1.101
1.032
1.001
1.011
1.03
1.122
1.121
1.116
1.118
1.113
1.111
1.112
1.11
1.106
Packet Size (Bytes)
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
30
Nexus 6004 - 384 x 10G Multicast 100% Load
Latency
1.4
1.2
1
Average Latency
(microseconds)
0.8
0.6
0.4
0.2
0
RFC 3918 Mcast L2 One-to-Many
Accumulated
RFC 3918 Mcast L3 One-to-Many
Accumulated
RFC 3918 Mcast L2 One-to-Many
Distributed
RFC 3918 Mcast L2 One-to-Many
Distributed
BRKARC-3452
64
80
96
128
256
512
1000 1024 1280 1518 2240 4096 9100
1.004 0.989 1.029 1.017 1.095 1.086 1.079 1.084 1.083 1.082 1.082 1.08 1.077
1.067 1.051 1.061 1.064 1.087 1.083 1.079 1.081 1.08 1.079 1.078 1.076 1.073
0.951 0.949 0.954 0.962 1.019 1.042 1.043 1.042 1.042 1.042 1.043 1.043 1.043
0.96 0.958 0.963 0.974 1.019 1.044 1.044 1.044 1.044 1.044 1.044 1.044 1.044
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
31
Nexus 6004 - 384 x 10G Unicast and Multicast
100% Load Jitter
12
10
8
Average Jitter
(nanoseconds)
6
4
2
0
Unicast L2 Jitter (ns)
Unicast L3 Jitter (ns)
Multicast L2 Jitter (ns)
Multicast L3 Jitter (ns)
64
8
8
9
9
80
5
5
6
5
96
8
9
9
9
128
5
5
5
5
256
6
6
5
5
512
8
8
8
8
1000
6
6
6
6
1024
7
7
7
7
Packet Size (Bytes)
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
32
1280
6
6
6
6
1518
10
10
10
9
2240
10
10
9
9
4096
9
9
9
9
9100
9
10
9
9
Nexus 5600/6000 Architecture - Agenda
Overview
Fabric Extender
Performance
Internal Architecture
Transceivers/Cables
Access Control List
Analytics
Multicast
Quality of Service
Applications
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
33
Cisco Nexus 5672 Internal Architecture
Switch Fabric
Unified Fabric Controller (UFC)
Supervisor
UPC 1
UPC 2
UPC 3
UPC 4
UPC 5
UPC 6
UPC 0
FC PHY
48xSFP+
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
6xQSFP+
Cisco Public
34
CPU
Cisco Nexus 56128 Internal Architecture
Slot 2
Slot 1
2x
QSFP+
24xSFP+
FC PHY
FC PHY
UPC 7
UPC 8
UPC 9
2x
QSFP+
24xSFP+
FC PHY
FC PHY
UPC 10
UPC 11
UPC 12
Supervisor
Unified Fabric Controller (UFC)
UPC 1
UPC 2
UPC 3
UPC 4
48xSFP+
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
35
UPC 5
UPC 6
2x
QSFP+
2x
QSFP+
CPU
Cisco Nexus 6004 Internal Architecture
Switch Fabric
UFC 1
UPC 1
UFC 2
UPC 2
UPC 3
UFC 3
UFC 4
UPC 4
UPC 1
UPC 2
UPC 0
FC PHY
20xSFP+
12xQSFP+
CPU
Supervisor
12x40G LEM (M12Q)
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
20x10G LEM (M20UP)
Cisco Public
36
Unified Port Controller - UPC ASIC
• Multimode MAC; built-in PHY for 1, 10, and 40GE
• Packet parsing and rewriting
• Lookup engine and access control: L2, L3, FabricPath, ACL, FCoE, and policing
• Buffering and queuing: Buffer management, PFC for lossless traffic, queuing (Strict Priority
Queuing and DWRR), and packet replication (SPAN and multicast)
• Extra fabric bandwidth for SPAN and multi-destination traffic
Front Panel Ports
Twelve 10 GE
Fabric Connections
224Gbps
UPC
448Gbps
224Gbps
Three 40 GE
BRKARC-3452
UPC
© 2014 Cisco and/or its affiliates. All rights reserved.
448Gbps
Cisco Public
37
Cisco Nexus 5600/6000 Packet Processing Flow
Ingress Pipeline
VLAN Table
MAC Table
ARP/ND Table
IPv4/IPv6 Routing Table
FabricPath
FC routing Table
Ingress UPC
 Buffer Allocation
 Buffer Accounting
 PAUSE Signal
1, 10, and 40 GE
Unicast VOQ
(8 per egress port)
Forwarding
Lookup
MAC
Packet
Rewrite
Parser
Ingress ACL
Ingress Policing
Buffer
Management
Buffer





BRKARC-3452
Security ACL
QoS ACL
SPAN Filter
Policing/Stats
Storm Control
Ingress
SPAN
Replication
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
38
8000 Multicast VOQs
Unified Crossbar
Fabric






Nexus 5600/6000 Packet Processing Flow
Egress Pipeline
Unified Crossbar
Fabric
Egress UPC
 Unicast buffer
 Multicast
congestion
management
Unicast Queue
Buffer
Management
Scheduling
Multicast
Replication
Egress ACL
Egress Policing*
Editing
TX SPAN
Replication
Buffer
Multicast Queue
To Ingress
Buffer Management
*Egress policing is not supported with current software
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
39
MAC
Cisco Nexus 5600/6000 Key Forwarding Tables
 Host table: 256,000 -entry hashing table;
actual capacity is slightly less than 256,000

Host table default carving: 128,000 MAC,
128,000 IP hosts
 Host table: Shared between MAC, ARP, and
ND and /32 host route

LPM table: 32,000 entries. Also known as
summary routes
256,000
Entries
Host Table*
MAC Region
LPM Table(32,000)
Summary Routes
IP Host Region
(ARP/ND/Host
route/(*,G))
* Hardware table size. Please check configuration limit for software scaling
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
40
Mroute Table(64,000)*
(S,G)
Cisco Nexus 5600/6000 Host Table Carving
• In Layer 2, virtualized environments, it can be interesting to have more than
128k MAC addresses on the switch
• MAC/ARP Hardware Resource Carving allows to change the size of MAC and
IP regions to allocate more space for the MAC region.
256,000 Entries
Host Table
MAC Region
IP Host Region
(ARP & ND & /32 Host route)
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
41
Cisco Nexus 5600/6000 Carving Profiles
• Use of pre-defined carving profiles.
• HRT = Host Route Table = IP host region.
• STM = Station Table Management = MAC region.
Template Profile
Description
hrt-128-stm-128
HRT size: 128k, STM size: 128k (default profile)
hrt-96-stm-160
HRT size: 96k, STM size: 160k
hrt-64-stm-192
HRT size: 64k, STM size: 192k
hrt-32-stm-224
HRT size: 32k, STM size: 224k
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
42
Cisco Nexus 5600/6000 MAC/ARP Resource
Carving CLI
• Specify the resource template to use:
•
•
switch(config)# hardware profile route resource servicetemplate template-name
Save the config and reload the switch
• Show commands:
•
•
•
•
BRKARC-3452
show
show
show
show
hardware profile route resource
hardware profile route resource
running-config hardware profile
startup-config hardware profile
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
43
template
template default
route resource template
route resource template
Nexus 5600/6000 Architecture - Agenda
Overview
Fabric Extender
Performance
Internal Architecture
Transceivers/Cables
Access Control List
Analytics
Multicast
Quality of Service
Applications
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
44
QSFP 40G Bidirectional Transceiver
40G deployment with 10G cabling
• QSFP pluggable 40Gbit/s Bidirectional (BiDi) Transceiver that works with
existing Multi-mode fiber infrastructure
– Using the existing 10Gbit/s Multi-mode infrastructure
– QSFP-40G-SR-BD has dual LC Connector
– 40Gbit/s BiDi transceiver has two 20Gbit/s channels each transmitted and received
simultaneously on two wavelengths
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
45
QSFP-40G-SR-BD
 Supports 100m with OM3 with
MMF and 150m with OM4
(OM4+) MMF
40G-BD
40G-BD
Multimode
LC Duplex
 LC duplex connectors
 Wavelength: ~850nm and
~900nm
 Pull-Tab Color: grey
Optical
DeMux
40G-BD
10G-SR
10G-SR
10G-SR
10G-SR
Multimode
LC Duplex
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
46
QSFP-40G-SR4
40G-SR4
40G-SR4
MPO-12 MPO-12
• Support 100M with OM3 MMF
and 150M with OM4 MMF
10G-SR
10G-SR
10G-SR
10G-SR
40G-SR4
MPO-12  LC Breakout
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
47
QSFP-40G-CSR4
• Supports 300M with OM3 MMF
and 400M with OM4 MMF
40G-CSR4
40G-SR4/CSR4
• Compatible with 10G SFP-SR
MPO-12 MPO-12
10G-SR
10G-SR
10G-SR
10G-SR
40G-CSR4
MPO-12  LC Breakout
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
• Compatible with Cisco QSFP40G-SR4
48
Multimode Ribbon Fiber
40GE
For QSFP-40G-SR4 and
QSFP-40G-CSR4
12-Fiber MPO Connector
• MPO/MTP connector with 12
pins
• Use 4 fiber pairs
− 4 TX and 4 RX allows for 40GE
− 4 unused fibers in the center
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
49
QSFP-40GE-LR4
• QSFP-40GE-LR4 can only be connected to each other.
10KM with Single mode fiber.
4x ROSA
Multiplexer
LC
TIA
Host
Card
IEEE Standard
40GBASE-LR4
4x TOSA
LC
Can’t connect to 10Gbase-LR
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
50
FET-40G
•
•
•
•
Low-cost QSFP optical transceiver connecting FEX to Cisco Nexus 6004
Supported on Cisco Nexus 5600/6004 and Nexus 2248PQ-10G
Interoperable with FET-10G
Support for 100m distance with OM3
Cisco Nexus 5600/6000
Cisco Nexus 5600/6000
FET-40G
FET-40G
FET-10G
FET-40G
Cisco Nexus 2248PQ-10G
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
51
Cisco Nexus 2232PP and 2232TM-E 2232TM
Cisco Nexus 2248TP-E 2248TP
Nexus 5600/6000 Architecture - Agenda
Overview
Fabric Extender
Performance
Internal Architecture
Transceivers/Cables
Access Control List
Analytics
Multicast
Quality of Service
Applications
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
52
ACL Types and Features
• Security ACL
− MAC, IPv4, and IPv6 ACLs
− PACL: ACL enabled under L2 interface
− VACL: ACL enabled for L2 VLAN traffic
− RACL: ACL enabled for routed traffic
− RBACL: Role-based ACL for CTS (roadmap feature)
• ACL for QoS classification and Policing
• Policy Based Routing (PBR)
– User-configured ACL to redirect traffic
• ACL for SPAN/ERSPAN
• ACL for control traffic
– To redirect control traffic to CPU, not user-configurable
– CoPP
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
53
ACL Scaling and TCAM Partition
• 4096 Access Control Entries per UPC are organized in blocks of 64 ACE
entries.
• Each IPv6 ACL (without port range) requires two TCAM entries.
Default TCAM Partition
VACL(1024)
4096
Entries
IFACL(1152)
QoS(448)
RBACL(1024)
SPAN(64)
Control Traffic(256)
BRKARC-3452
switch# sh platform afm info tcam 0 region ifacl
ifacl tcam TCAM configuration for asic id 0:
[ vacl tcam]: range
0 - 1023
[ifacl tcam]: range 1024 - 2175 *
[ qos tcam]: range 2176 - 2623
[rbacl tcam]: range 2624 - 3775
[ span tcam]: range 3776 - 3839
[ sup tcam]: range 3840 - 4095
TCAM [ifacl tcam]: [v:1, size:1152, start:1024
end:2175]
TCAM Region
In use tcam entries: 24
Usage
1024-1039,2168-2175
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
54
TCAM Carving
• TCAM Carving allows to change the size of each region
• Create a template:
switch(config)# hardware profile tcam resource template t1
switch(config-tcam-templ)#
• Set region sizes:
switch(config-tcam-templ)# ifacl ?
<320-3584>
Enter size of ifacl
increments)
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
55
region
(in
64
entry
TCAM Carving
• Similarly, other region sizes can be changed:
switch(config-tcam-templ)# ?
ifacl Configure size of ifacl region
no
Change size to default value
qos
Configure size of qos region
rbacl Configure size of rbacl region
span
Configure size of span region
vacl
Configure size of vacl region
• Display the configuration for a template along with the current usage:
switch(config)# show hardware profile tcam resource template name
<my_template>
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
56
TCAM Carving
switch(config-tcam-templ)# show hardware profile tcam resource template name t1
------------------------------------------------------------------Template name: t1
Current state: Created
Region Size-allocated Current-size Current-usage
Available/free
------------------------------------------------------------------Vacl
1152
1024
28
996
Ifacl
1024
1152
266
971
Rbacl
1152
1152
3
1149
Qos
448
448
30
418
Span
64
64
6
58
Sup
256
256
74
182
-------------------------------------------------------------------
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
57
TCAM Carving
• Commit a template:
switch(config)# hardware profile tcam resource service-template t1
Details of the t1 template you are trying to commit are as follows:
------------------------------------------------------------------------------Template name: t1
Current state: Committed (Startup/Running)
Region
Features
Size-allocated
Current-size
Current-usage
Available/free
------------------------------------------------------------------------------Vacl
Vacl
1152
2048
11
2037
[…]
------------------------------------------------------------------------------
Template committed.
Please do the following for the template to be applied:
1> Save running config :
"copy running-config startup-config"
2> Reboot the switch
"reload”
BRKARC-3452
:
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
58
Nexus 5600/6000 Architecture - Agenda
Overview
Fabric Extender
Performance
Internal Architecture
Transceivers/Cables
Access Control List
Analytics
Multicast
Quality of Service
Applications
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
59
Why Analytics? – A Data Center Today:
• 1G -> 10G -> 40G -> 100G
• Virtualized
• More and more applications
• Those applications are more and more complex (Hadoop…)
• More and more flows
• Latency is very important
• Ports speed mismatch
• Buffer discrepancy between the devices
• With spine/leaf, design is less complex, but there’s more speed and more
applications
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
60
Switched Port Analyzer (SPAN)
Switch
Host-B
Host-A
Monitor Destination
Monitor Source
Sniffer Device
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
61
Encapsulated Remote SPAN (ERSPAN)
Host-B
GRE Encapsulated
monitored traffic
IP Cloud
Host-A
Monitor Destination
Monitor Source
Sniffer Device
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
62
Cisco Nexus 5600/6000 SPAN
SPAN Features
Nexus 5600/6000
Nexus 5500
Total SPAN sessions
31 bi-directional (16 with current NXOS)
4 bi-directional
Local SPAN sessions
31 bi-directional (16 with current NXOS)
4 bi-directional
ERSPAN sessions
Prioritize data over SPAN
16
Yes (through scheduling)
4
Yes (SPAN policing)
Line-rate SPAN throughput
Yes
No
ERSPAN destination session
ERSPAN with 1588 PTP timestamp
Yes
Yes*
No
No
MTU Truncated SPAN and ERSPAN
ACL filter for SPAN and ERSPAN
Yes
Yes
Yes
Yes
SPAN-on-Drop
SPAN-on-Latency
Yes
Yes
No
No
SPAN with multiple destination ports
Yes (each destination port uses one
SPAN session)
No
*Software roadmap feature
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
63
High-Performance SPAN
Extra Fabric
Bandwidth
for SPAN
Scheduler Prioritizes
Production Data
Data
Unified
Crossbar
Fabric
SPAN
Separate Buffer
Pool for SPAN
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
64
SPAN on Drop - NX-OS 7.0(1)N1(1), Q1/2014
SPAN-On-Drop
Tail-Drop
SPAN-on-Drop
BRKARC-3452
Ingress
Data Buffer
Monitoring
Station
SPAN Buffer
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
65
SPAN
on Drop - NX-OS 7.0(1)N1(1), Q1/2014
SPAN-on-Drop
• Copies the dropped packet to capture device
• Correlate the packet drop with application
• Works for unicast packets only
• Packet drops can be monitored on ingress only
• There is a dedicated buffer space for SPAN, so SPAN-on-Drop does not affect production
traffic
• Supports both local SPAN and ERSPAN
• Configuration uses existing SPAN commands
• One SPAN-on-Drop session is supported
• Can have multiple source ports, and multiple destination ports
• Source cannot be FEX HIF port. But FEX fabric port is supported
• Source port(s) can be a part of a SPAN-on-Drop session, and a local SPAN session
simultaneously
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
66
SPAN on Drop - NX-OS 7.0(1)N1(1), Q1/2014
SPAN-on-Drop
The source interface is the ingress port for which we want to monitor drops.
switch(config)# monitor session 1 type span-on-drop
switch(config-span-on-drop)# source interface e3/1
switch(config-span-on-drop)# destination interface e3/2
switch(config)# monitor session 2 type span-on-drop-erspan
switch(config-span-on-drop-erspan)# source interface e3/1
switch(config-span-on-drop-erspan)# destination ip 100.1.1.2
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
67
Latency Monitoring
Latency Monitoring:
Min: 856ns
Max: 1208ns
Avg: 901ns
T0
T1 – T0
T0
T1 – T0
•
•
TimeStamp
Packet
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
Latency Histogram:
Latency Range: 800ns - 10usec
Stats:
Within the Range: x packets
Out of the Range: y packets
68
FIFO Latency
Per Port-Pair Data
Latency
LatencyMonitoring
Monitoring - NX-OS 7.0(1)N1(1), Q1/2014
• Latency Monitoring provides {min, average, max} latency between a
specified port pair and also maintains latency histogram (accuracy in
few nanosecs)
• By default instantaneous Latency Monitoring is enabled between pair
of ports
• Latency Histogram can be enabled for specific port-pair to provide
histogram instead of instantaneous mode
• Measures switch latency for each packet, no sampling required
• Fully implemented in HW, no CPU impact, no traffic impact
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
69
Instantaneous Mode
• Enabled by default on all pairs of ports. No configuration is required
switch# show hardware profile latency monitor interface e1/23 interface e1/22
-------------------------------------------------------------------------------Latency Statistics in nano seconds
-------------------------------------------------------------------------------| Ingress Port | Egress Port
|
Minimum
|
Maximum
|
Average
|
-------------------------------------------------------------------------------|
Ethernet1/22|
Ethernet1/23|
856|
1208|
901|
--------------------------------------------------------------------------------
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
70
Custom Histogram
• Count the number of packets falling in a specific range of latency
• Example for port-pair e1/1 and e1/3:
switch(config)# interface e1/3
switch(config-if)# packet latency low-latency 800 high-latency 10000 mode custom
interface e1/1
switch# show hardware profile latency monitor histogram interface e1/3 interface e1/1
-------------------------------------------------------------------------------|
Egress Interface : Ethernet1/3 Ingress Interface : Ethernet1/1
|
-------------------------------------------------------------------------------|
All Latency Values are in nano seconds
|
-------------------------------------------------------------------------------|
Range |
800 <= Latency < 10000 |
Outside the first range
|
------------------------------------------------------------------------------| counter |
3542818572
|
16290988
|
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
71
SPAN-on-Latency
Latency Monitoring:
Min: 856ns
Max: 1208ns
Avg: 901ns
T0
T1 – T0
T2
T3 – T2
TimeStamp
Packet
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
If Latency Threshold > 10 usec:
SPAN to 1/64
Monitoring
Station
72
SPAN-on-Latency
• This feature replicates packets from the interface when latency exceeds
a configured threshold
• Latency threshold is per-port
• This again helps to identify the victim of congestion
• Replicated traffic uses the SPAN buffer so it doesn't impact the
production traffic
• One SPAN-on-Latency session is supported in hardware
• Uses similar SPAN CLI, with new session type “span-on-latency”
• ERSPAN-on-latency is also supported
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
73
SPAN-on-Latency Configuration
• The SPAN-on-Latency source port is the egress port on which we monitor the latency
• The SPAN-on-Latency session makes a copy of all high-latency packets egressing on
this port, coming from any ingress port
• Sample configuration to SPAN packets if latency goes beyond 10us when egressing on
e1/3:
Always Tx: packets egressing on 1/3
(any source) with latency >10us will
be replicated to the SPAN dest 1/4
monitor session 1 type span-on-latency
source interface Ethernet1/3 tx
destination interface Ethernet1/4
interface Ethernet1/3
hardware profile latency monitor threshold 10001
interface Ethernet1/4
switchport mode monitor
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
74
SPAN-on-Latency
• Source port can be an regular ethernet port, not a port-channel. Can be
a port-channel member
• Source port cannot be FEX HIF port. But FEX fabric port is supported
• Multiple sources can be configured – latency threshold is per SPAN-ondrop TX source port
• A SPAN-on-Latency source port cannot be in another SPAN session
• Destination is only a single ethernet port, not port-channel
• Only one destination port can be configured
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
75
Nexus 5600/6000 Architecture - Agenda
Overview
Fabric Extender
Performance
Internal Architecture
Transceivers/Cables
Access Control List
Analytics
Multicast
Quality of Service
Applications
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
76
Cisco Nexus 5600/6000 Multicast Features
Summary
• IGMP snooping for IGMPv1, v2, and v3
• PIM-SM
• PIM-SSM without vPC and PIM-SSM with vPC+
• PIM-BiDir without vPC and PIM-BiDir with vPC+
• PIM Policy
• MSDP
• Anycast RP with PIM (RFC 4610)
• Anycast RP with MSDP
• 8000 IGMP snooping entries
• 8000 mroutes with vPC, 16000 mroutes without vPC supported by software
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
77
Efficient Multicast Replication
• Optimized multicast replication
• Fabric replication and egress replication; one copy is replicated to
egress UPC, where there is a receiver—minimizing the traffic load on
the switch fabric and eliminating the switch fabric congestion
• Line-rate multicast replication
Egress
UPC
Ingress UPC
BRKARC-3452
Unified
Crossbar
Fabric
© 2014 Cisco and/or its affiliates. All rights reserved.
Egress
UPC
Cisco Public
78
Multicast VOQ
• 8000 multicast VOQs to eliminate HOLB and help ensure high throughput
• Tracks the fan-out of the egress UPC; packets with different egress UPC fanouts are
assigned to different VOQs so that it can be scheduled and served independently
Ingress UPC
Multicast VOQ Unicast VOQ
Egress UPC 1
224.1.1.1
Unified
Crossbar
Fabric
Egress UPC 2
224.1.1.1
224.1.1.2
Egress UPC 3
224.1.1.2
224.1.1.2
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
79
Cisco Nexus 5600/6000 Key Forwarding Tables
 Host table: 256,000 -entry hashing table;
actual capacity is slightly less than 256,000

Host table default carving: 128,000 MAC,
128,000 IP hosts
 Host table: Shared between MAC, ARP, and
ND and /32 host route

LPM table: 32,000 entries. Also known as
summary routes
256,000
Entries
Host Table*
MAC Region
• Mroute table: 64,000 entries
LPM Table(32,000)
Summary Routes
IP Host Region
(ARP/ND/Host
route/(*,G))
* Hardware table size. Please check configuration limit for software scaling
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
80
Mroute Table(64,000)*
(S,G)
MET Table
• 64,000 entries in MET table
• Currently supported: 8,000 vPC, 16,000 non-vPC
Ingress UPC
Multicast VOQ Unicast VOQ
• Today the MET table is the limiting factor for multicast
scale
MET: 123
4
16384
16385
122
123
MET: 123
1
1, 4, 5
5
65536
Egress UPC 1
2, 10
2
MET: 123
16384
16385
10
Switch Fabric
BRKARC-3452
122
123
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
65536
81
Egress UPC 2
Multicast Hashing over Port Channel
• N5600/6000 implements flow based hashing for multi-destination traffic, and it supports multicast load
sharing over Port Channel with 5-tuple packet header.
• Traffic is replicated to all egress UPCs where Port Channel member resides
• Egress UPC runs hash calculation and one egress port is chosen to send out multicast packets. The
UPC ASIC that is not supposed to send out packet will drop packet (Egress UPC 1 in the example)
Multicast VOQ
Ingress UPC
Unicast VOQ
Egress UPC 1
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Port 3
Selected
Unified
Crossbar
Fabric
Port 1
Port 2
Egress UPC 2
Port 3
Port 3
Selected
Port 4
Cisco Public
82
Nexus 5600/6000 Architecture - Agenda
Overview
Fabric Extender
Performance
Internal Architecture
Transceivers/Cables
Access Control List
Analytics
Multicast
Quality of Service
Applications
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
83
Cisco Nexus 5600/6000 QoS Features
• Eight classes of service;
– 2 reserved for control traffic, 6 for data traffic
• Traffic classification
– DSCP, CoS, and ACL
• Strict Priority Queuing and DWRR
– DCBX 802.1Qaz
• Packet marking
– DSCP, CoS, and ECN
• Ingress and egress policing
– 4096 policers per ASIC
• No drop system class
• Flexible buffer management
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
84
Cisco Nexus 5600/6000 QoS Processing Flow
Ingress UPC
Trust CoS/DSCP
L2/L3/L4 info with ACL
MAC
Traffic
Classification
Ingress
Policing
MTU
Checking
Per-class
Buffer Usage
Monitoring
VOQs for Unicast
(8 per egress port)
Multicast Queues
Truncate or Drop
Packets if MTU is Violated
Egress Queues
PAUSE ON/OFF
Signal
MAC
Unicast
ECN
Marking
Egress
Policing
Egress
Scheduling
Multicast
Egress UPC
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
Strict Priority +
DWRR Scheduling
85
Crossbar
Fabric
Ingress
Cos/DSCP
Marking
If Buffer Usage Crosses Threshold:
• Tail drop for drop class
• Assert pause signal to MAC
for no-drop system class
Increased Packet Buffer
• 25MB packet buffer is shared by every three 40GE ports or twelve 10GE ports.
150MB of total buffer
• Nexus 56128:
300MB of total buffer
• Nexus 6004:
800MB of total buffer
Ingress UPC
Multicast VOQ
16MB
Unicast VOQ
• Nexus 5672:
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Egress UPC
9MB
Unified
Crossbar
Fabric
Cisco Public
86
Ingress Buffer - Flexible Buffer Management
• Shared buffer is good for burst
absorption.
UPC Ingress Buffer (16MB)
• Dedicated buffer is good for
predictable performance for each
port.
SPAN
• N5600/6000 buffer management is
flexible: dedicated plus shared.
Shared Packet Buffer
~14MB
• Long-distance FCoE, video editing
(i.e., AVID), Big Data, and
distributed storage
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
Control
Port 1
Dedicated
87
Port 2
Dedicated
Port 3
Dedicated
Tune Shared vs Dedicated Buffer Allocation at
Ingress
• “queue-limit” under “network-qos” policy specifies the dedicated buffer for each port and each class.
• Without “queue-limit” each class of service will get 100 KB of dedicated buffer.
• The size of dedicated buffer can be different for different classes of service. The policy applies to all ports
in the chassis.
• Total ingress buffer minus the dedicated buffer and buffer for control and SPAN will be in the shared
buffer pool.
• The following example sets the dedicated buffer for “class-default” to be 400 KB for all ports.
switch(config)# policy-map type network-qos Policy-buffer
switch(config-pmap-nq)# class type network-qos class-default
switch(config-pmap-nq-c)# queue-limit 400000 bytes
switch(config-pmap-nq-c)# system qos
switch(config-sys-qos)# service-policy type network-qos Policy-buffer
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
88
Egress Buffer Design
• 9-MB buffer is reserved for
egress traffic
UPC Egress Buffer (9MB)
• Unicast traffic can be buffered at
egress and ingress.
Unicast Buffer
• Multicast is buffered at egress in
case of interface
oversubscription.
Dedicated per port
Multicast Buffer
Shared ~6MB
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
89
Unicast Flow Control and Buffering
• Unicast is queued at egress buffer pool first and then ingress buffer after egress queue
is full.
• Take advantage of ingress buffer from multiple port or ASIC for unicast burst absorption.
• Ensure fairness among multiple ingress ports
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
90
Identify Packet Drops Due to Congestion
switch# show interface ethernet 1/7
switch# show queuing interface ethernet 1/7
Ethernet1/1 queuing information:
TX Queuing
qos-group sched-type oper-bandwidth
0
WRR
100
Ethernet1/1 is down (SFP not inserted)
Dedicated Interface
<snip>
RX
<snip>
0 input error 0 short frame 0 overrun 0 underrun 0
ignored
0 watchdog 0 bad etype drop 0 bad proto drop 0 if down
drop
0 input with dribble
0 Rx pause
TX
RX Queuing
qos-group 0
q-size: 100160, HW MTU: 1500 (1500 configured)
drop-type: drop, xon: 0, xoff: 0
Statistics:
Pkts received over the port
: 710025080
Ucast pkts sent to the cross-bar
: 474177592
Mcast pkts sent to the cross-bar
:0
Ucast pkts received from the cross-bar : 0
Pkts sent to the port
:0
Pkts discarded on ingress
:
235847488 input discard
Unicast drop at ingress
0 unicast packets 0 multicast packets 0 broadcast
packets
0 output packets 0 bytes
0 jumbo packets
0 output error 0 collision 0 deferred 0 late collision
235847488
Per-priority-pause status
(Inactive), Tx (Inactive)
0 lost carrier 0 no carrier 0 babble 0 output discard
0 Tx pause Broadcast/Multicast/Unknown
Ingress drop for each class
unicast flooding drop at egress
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
Dedicated buffer for this class
91
: Rx
Identify Congested Egress Port with VOQ Counters
I
X
I
A
I
X
I
A
1/7
Ingress UPC 3
Unified Crossbar
Fabric
Egress UPC 3
1/8
Ingress UPC 4
1/10
 Using “show interface” CLI, we noticed drops on ingress ports
 How do we know which egress port is affected?
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
92
I
X
I
A
Step 1: Map Interface to UPC Number
N6004# show hardware internal bigsur all-ports
Bigsur Port Info:
Port
|asic|inst|inst|
name
|idx |slot|asic|eport|logi|flag|adm|opr|if_index|diag|ucVer
---------+----+----+----+-----+----+----+---+---+--------+----+----sup1
|0
|17 |0
| 0 - |0
|b3 |en |dn |15010000|pass| 0.00
sup0
|0
|17 |0
| 1 - |1
|b3 |en |dn |15020000|pass| 0.00
40gb1/1 |1
|0
|0
| 0 - |0
|b3 |en |dn |1a070000|pass| 0.00
40gb1/2 |1
|0
|0
| 1 - |1
|b3 |en |dn |1a071000|pass| 0.00
40gb1/3 |1
|0
|0
| 2 - |2
|b3 |en |dn |1a072000|pass| 0.00
40gb1/4 |2
|0
|1
| 0 - |3
|b3 |en |dn |1a073000|pass| 0.00
40gb1/5 |2
|0
|1
| 1 - |4
|b3 |dis|dn |1a074000|pass| 0.00
40gb1/6 |2
|0
|1
| 2 - |5
|b3 |dis|dn |1a075000|pass| 0.00
40gb1/7 |3
|0
|2
| 0 p |6
|b3 |en |up |1a076000|pass| 0.00
40gb1/8 |3
|0
|2
| 1 p |7
|b3 |en |up |1a077000|pass| 0.00
40gb1/9 |3
|0
|2
| 2 p |8
|b3 |en |up |1a078000|pass| 0.00
40gb1/10 |4
|0
|3
| 0 p |9
|b3 |en |up |1a079000|pass| 0.00
40gb1/11 |4
|0
|3
| 1 - |10 |b3 |dis|dn |1a07a000|pass| 0.00
40gb1/12 |4
|0
|3
| 2 p |11 |b3 |en |up |1a07b000|pass| 0.00
Done.
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
93
Step 2: Identify Congested Egress Port with VOQ Counters
N6004# show platform software qd info counters voq asic-num 3
+----------+------------------------------+------------------------+-----------+
|
port|
TRANSMIT|
TAIL DROP| HEAD DROP|
+----------+------------------------------+------------------------+-----------+
Eth1/8
QUEUE-3
3888954496
2248133231
0
+----------+------------------------------+------------------------+-----------+
Congested egress port and class of service
N6004# show platform software qd info counters voq asic-num 4
+----------+------------------------------+------------------------+-----------+
|
port|
TRANSMIT|
TAIL DROP| HEAD DROP|
+----------+------------------------------+------------------------+-----------+
Eth1/8
QUEUE-3
2574819434
2248180087
0
+----------+------------------------------+------------------------+-----------+
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
94
Ingress
UPC number
Nexus 5600/6000 Architecture - Agenda
Overview
Fabric Extender
Performance
Internal Architecture
Transceivers/Cables
Access Control List
Analytics
Multicast
Quality of Service
Applications
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
95
Nexus 5600/6000 Applications
Compact Aggregation
Nexus 7000
CORE
L3
Nexus 7000
Nexus 6004
AGG.
vPC to Nexus 7000 or 6004
Nexus 6004/
Nexus 5600
ACCESS
Nexus 2000
FEX
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
96
Nexus 5600/6000 Applications
Large Scale Fabric (Layer 2 or Layer 3)
CORE
Nexus 7000
L3
SPINE
Nexus 6004
FabricPath
Nexus 6004/
Nexus 5600+2000
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
LEAF
Cisco Public
97
Nexus 5600/6000 Applications
High Performance Computing (HPC)
CORE/AGG.
Nexus 6004
L3/L2
Fabric
Nexus 3548
Nexus 5600/6004
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
ACCESS
Cisco Public
98
Target Q3 CY14
Nexus 6004X Chassis
• Nexus 6004X has 8 slots (similar to the 6004EF chassis)
• Nexus 6004X chassis supports VXLAN functionality
• Provides Flexible Ordering Options – mix/match 10/40G LEMs
• Supports new 12 x 40G LEM VXLAN capable (backward compatible with
Nexus 6004EF chassis)
• VXLAN capable LEM will work in non-VXLAN mode in 6004EF chassis
• Same software as on Nexus 6004
• Supports all the optics, FEX models as on Nexus 6004
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
99
Nexus 5600/6000 Key Takeaways
Rich Feature set
STP/vPC/FabricPath/DFA, FEX, L3, VXLAN,
Adapter-FEX/VM-FEX
Performance
Line rate L2 and L3 with low latency
Scalability
256K MAC / 128K ARP / 32K LPM routes,
16K bridge domains with segment ID
Innovations
Intelligent SPAN, Buffer monitoring,
Latency monitoring
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
100
Nexus 2000/5600/6000 More Information
• http://cisco.com/go/nexus2000
• http://cisco.com/go/nexus5000
• http://cisco.com/go/nexus6000
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
101
Complete Your Online Session Evaluation
• Give us your feedback and you
could win fabulous prizes. Winners
announced daily.
• Complete your session evaluation
through the Cisco Live mobile app
or visit one of the interactive kiosks
located throughout the convention
center.
Don’t forget: Cisco Live sessions will be available
for viewing on-demand after the event at
CiscoLive.com/Online
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
102
Continue Your Education
• Demos in the Cisco Campus
• Walk-in Self-Paced Labs
• Table Topics
• Meet the Engineer 1:1 meetings
BRKARC-3452
© 2014 Cisco and/or its affiliates. All rights reserved.
Cisco Public
103

fstack - OpenStack Benelux Conference 2014

Detailed Syllabus