Enterprise design & high availability

What you will learn in this module. By the end of these chapters you will be able to look at any enterprise network drawing and explain why it is built the way it is: why there are two of every box, where Layer 2 ends and Layer 3 begins, why the spanning-tree root and the default gateway must live on the same switch, how a supervisor can fail without users noticing, which wireless deployment model fits a campus or a branch, when a workload belongs in the cloud, and how a switch actually forwards a frame in hardware. You will also build a redundant collapsed-core block in the lab and review a broken design until traffic stops taking the long way.

Prerequisites. You should be comfortable with CCNA-level VLANs and trunks, spanning tree (root bridge, root port, blocking), EtherChannel with LACP, HSRP basics and single-area OSPF. If any of these feel shaky, revisit the CCNA modules on STP, EtherChannel and FHRP first. This module does not re-teach them; it uses them as building blocks for design.

An analogy: a network is a city's road system

Picture a well-planned city. Every house sits on a quiet local lane. Lanes feed into arterial roads where the traffic lights, speed checks and turn restrictions live. Arterial roads join a fast highway with no traffic lights at all, because its only job is to move a lot of cars quickly between districts. Every district is planned as a copy of the same template, so a new district can be added without redesigning the city. If a lane is dug up for repairs, only that street is affected, and there is always a second road into every district.

An enterprise network works the same way. Local lanes are the access layer where users plug in. Arterial roads are the distribution layer where policy and routing decisions happen. The highway is the core. Districts are modules (a building, a data centre, a WAN edge). The second road is redundancy. The rule that a dug-up lane only affects one street is the idea of a failure domain.

Where this module sits in the exam

The ENCOR 350-401 Architecture domain opens with design. This module covers the design-level items; later modules go deep on the technologies they mention.

Blueprint item	What you must be able to do	Chapters
1.1 Design principles	Explain 2-tier, 3-tier, fabric and cloud designs; high availability with redundancy, FHRP and SSO	2 to 7
1.2 WLAN deployment models	Compare centralised, distributed, controller-less, controller-based, cloud and remote-branch designs	9
1.3 On-premises vs cloud	Choose where infrastructure should live and how it connects	8
1.7 Hardware vs software switching	Explain process switching, CEF, RIB, FIB, adjacency table, CAM and TCAM	10

SD-WAN and SD-Access, LISP and VXLAN, HSRP object tracking and deep wireless configuration each get their own modules later. Here you learn why they exist and where they fit.

The lab network you will use

Both labs use the same small collapsed-core block. Two distribution/core switches, DS1 and DS2, are joined by two links (Gi0/6 and Gi0/7). One access switch, AS1, has an uplink to each: Gi0/0 to DS1 and Gi0/1 to DS2. PC1 sits in VLAN 10 (10.1.10.0/24) and PC2 in VLAN 20 (10.1.20.0/24). DS1 owns .2 and DS2 owns .3 in each VLAN, and HSRP provides the virtual gateway .1. A WAN router R1 connects to DS1 over 10.0.1.0/30 and to DS2 over 10.0.2.0/30, and reaches the data-centre server SRV (172.16.0.10). OSPF area 0 runs between R1 and the distribution switches.

Before you pick a single switch model, you need a way to judge whether a design is good. Architects of buildings use principles such as "load-bearing walls line up floor to floor" and "every floor has two exits". Network designers use four: hierarchy, modularity, resiliency and flexibility. Two measurements keep those principles honest: the size of each failure domain and the oversubscription ratio of each layer.

Hierarchy: every layer has one job

Hierarchy means splitting the network into layers with clearly separated roles: access, distribution and core (the next chapter looks at each). Because every device in a layer does the same job, the network behaves predictably. You know where to look for a policy (distribution), where users attach (access) and which devices must never be slowed down (core). Hierarchy also makes route summarisation possible: a distribution pair can advertise one summary such as 10.1.0.0/16 for a whole building, so a flapping user subnet does not ripple through the core.

Modularity: build with repeatable blocks

Modularity means building the enterprise from self-contained blocks: an access-distribution block per building, a data-centre block, a WAN edge block, an internet edge block. Each block has the same internal design and connects to the core in the same way. The benefits are practical. A new building is a copy of a tested template. A change in one block (say a new VLAN in building B) does not touch the others. Troubleshooting starts by finding the block, then working inside it. Modularity also lets different teams own different blocks.

Resiliency: survive failures, normal and abnormal

Resiliency is the ability to keep working when something goes wrong. That includes the obvious failures (a link, a line card, a whole switch, a power feed) and the abnormal ones: a broadcast storm, a loop created by a user's home switch, a traffic flood, a bad software upgrade. Redundancy is the raw material: two uplinks, two distribution switches, dual supervisors and power supplies. But redundancy only becomes resiliency when failover is fast and automatic, which is the job of EtherChannel, FHRP, routing protocols, SSO and NSF, all covered in later chapters.

Availability is measured in "nines". 99.9 % allows about 8.8 hours of downtime a year, 99.99 % about 53 minutes, 99.999 % about 5 minutes. Each extra nine means removing another single point of failure and shortening another convergence time.

Flexibility: ready for what comes next

Flexibility is the ability to adapt without a redesign: adding wireless, IoT, a new cloud region, a merger, a new security policy. Hierarchical and modular designs are naturally flexible because you change one block or add one block. Designs with fixed, hand-crafted exceptions are not. Scalability is closely related: can the design grow from 5 to 50 access switches without changing its shape?

Failure domains: how far does one fault spread?

A failure domain is the part of the network that is affected when one component fails or misbehaves. A Layer 2 VLAN is a failure domain for broadcasts and loops: a storm reaches every port in that VLAN. A spanning-tree instance is a failure domain for topology changes. The good design habit is to keep Layer 2 domains small, ideally inside one access-distribution block, and to use routed links between blocks, because a routing protocol does not forward broadcasts and contains a failure to one subnet.

Oversubscription: honest about bandwidth

Oversubscription is the ratio between the bandwidth that could arrive from below and the bandwidth available upward. Not every user transmits at line rate at the same time, so some oversubscription is normal and saves money. Common starting guidance for a campus is up to about 20:1 from access to distribution and about 4:1 from distribution to core. Data-centre fabrics aim much lower, often 3:1 or better, because servers really do send at high rates.

Think of a big railway network. Local trains stop at every small station (access). Several local lines meet at a junction where passengers change trains and tickets are checked (distribution). Junctions are linked by express trains that never stop at small stations (core). A small town, though, does not need an express line: one junction that also acts as the main station is enough. That is the difference between a three-tier campus and a two-tier (collapsed-core) campus.

The three layers and their jobs

Layer	Main job	Typical features
Access	Connect end devices: PCs, phones, APs, printers, cameras	VLANs, PoE, 802.1X, port security, DHCP snooping, dynamic ARP inspection, QoS trust boundary, PortFast and BPDU guard
Distribution	Aggregate access switches; the Layer 2/Layer 3 boundary	SVIs and FHRP gateways, STP root, route summarisation, ACLs and policy, redundant uplinks to core
Core	Fast, reliable transport between distribution blocks, data centre, WAN and internet edge	Layer 3 only, high-speed links, fast convergence, minimal policy, no users attached

The core is deliberately simple. Anything that costs CPU or adds risk (complex ACLs, NAT, packet inspection) belongs at the distribution layer or in dedicated blocks such as the internet edge. A core switch should do one thing: move packets between blocks as fast and as reliably as possible.

Why a separate core? The full-mesh problem

Without a core, every distribution block must connect directly to every other block and to the WAN, data centre and internet edge. The number of block-to-block connections grows as n(n-1)/2.

A common rule of thumb is to introduce a dedicated core once you have more than two or three distribution blocks, or when the campus spans several buildings, or when you need one clean meeting point for the data centre, WAN and internet edge.

Two-tier: the collapsed core

In a collapsed-core design the core and distribution functions run on the same pair of switches. Access switches uplink to the pair; the pair also connects to the WAN routers, firewalls or a small server room. This is the right design for a single building or a medium site: fewer boxes, less cabling, less cost, and still fully redundant if built in pairs. In the lab, DS1 and DS2 are a collapsed core: they are the default gateways for VLANs 10 and 20 (distribution role) and route towards R1 and the data centre (core role).

The trade-offs: the collapsed pair carries both user-facing policy and core transport, so a mistake there affects everything; scaling to many buildings eventually brings back the full-mesh problem; and upgrades must be planned more carefully because there is no separate core to carry traffic.

Summarise at the distribution layer

Hierarchy pays off in routing. If each building uses one block of addresses, its distribution pair can advertise a single summary to the core. In OSPF, with the building in its own area and the distribution switches as ABRs:

! On each distribution switch acting as ABR for area 1
router ospf 1
 area 1 range 10.1.0.0 255.255.0.0

CORE1# show ip route ospf
O IA     10.1.0.0/16 [110/3] via 10.0.11.2, 00:05:12, TenGigabitEthernet1/0/1
                     [110/3] via 10.0.12.2, 00:05:12, TenGigabitEthernet1/0/2

A user subnet flapping inside the building no longer triggers SPF recalculation across the whole campus. The multi-area OSPF module covers the details.

Every campus design must answer one question early: where does Layer 2 stop and Layer 3 begin? Think of a housing society. Inside the gate, neighbours walk freely between houses (Layer 2, same VLAN). At the gate, a guard checks where you are going and sends you on the right road (Layer 3, routing). You can put the gate at the society entrance, or at each building's door. Both work, but they change how far a problem spreads and how far a resident can wander. This chapter compares the three common answers.

Option 1: Layer 2 access (the looped triangle)

Access switches are pure Layer 2. Every access switch has an uplink trunk to each distribution switch, and the two distribution switches are linked by a Layer 2 trunk (usually an EtherChannel). The VLAN's gateway lives on distribution SVIs protected by an FHRP. Each access switch plus the distribution pair forms a triangle, which is a physical loop, so spanning tree blocks one uplink per VLAN.

• Pros: a VLAN can span several access switches; simple access configuration; works with any access switch.
• Cons: STP blocks half the uplinks (unless you load-share per VLAN); failover depends on STP and FHRP timers; the VLAN is a larger failure domain; the STP root and FHRP active must be aligned (next chapter).

This is the design used in the lab: AS1 has Gi0/0 to DS1 and Gi0/1 to DS2, and DS1 and DS2 are linked by Gi0/6 and Gi0/7.

Two variants avoid the loop. In a loop-free U, the two access switches of a pair are joined by a Layer 2 link and the distribution link is routed, so VLANs span only that access pair and nothing blocks. In a loop-free inverted U, the distribution link is Layer 2 and each access switch has only one uplink, so nothing blocks but a single uplink failure isolates that switch. Both are niche; the looped triangle and routed access are what you will meet most.

Option 2: Routed access

The Layer 3 boundary moves down to the access switch. Each uplink is a routed point-to-point link, the access switch is the default gateway for its own VLANs, and it runs OSPF or EIGRP towards the distribution pair.

• Pros: no STP blocking on uplinks, both uplinks forward with equal-cost multipath (ECMP); no FHRP needed because the access switch is the gateway; routing convergence is fast and deterministic; broadcasts and loops stay on one switch, so failure domains are tiny.
• Cons: a VLAN cannot span access switches; more IP planning (a /30 or /31 per uplink, subnets per access switch); the access switches need Layer 3 features and licensing.

! Routed access switch ACC-R1 (example)
ip routing
interface GigabitEthernet1/0/49
 description UPLINK-DIST1
 no switchport
 ip address 10.0.101.1 255.255.255.252
 ip ospf network point-to-point
interface GigabitEthernet1/0/50
 description UPLINK-DIST2
 no switchport
 ip address 10.0.102.1 255.255.255.252
 ip ospf network point-to-point
interface Vlan110
 ip address 10.1.110.1 255.255.255.0
router ospf 1
 router-id 10.255.0.21
 passive-interface default
 no passive-interface GigabitEthernet1/0/49
 no passive-interface GigabitEthernet1/0/50
 network 10.0.101.0 0.0.0.3 area 1
 network 10.0.102.0 0.0.0.3 area 1
 network 10.1.110.0 0.0.0.255 area 1

ACC-R1# show ip route ospf
O*IA  0.0.0.0/0 [110/2] via 10.0.101.2, 00:02:10, GigabitEthernet1/0/49
                [110/2] via 10.0.102.2, 00:02:10, GigabitEthernet1/0/50

Two equal-cost default routes: both uplinks carry traffic. Making the access area totally stubby (or the access switch an EIGRP stub) keeps its routing table tiny and stops it from ever being used as a transit path.

Option 3: Virtual switching (StackWise Virtual)

The two distribution switches are combined into one logical switch with one control plane (StackWise Virtual on Catalyst 9000, covered in the high-availability chapter). The access switch bundles its two uplinks into a single multichassis EtherChannel (MEC) with LACP. To spanning tree that is one port, so nothing blocks. The gateway is a single SVI on the logical switch, so no FHRP is needed, and VLANs can still span access switches.

Question	L2 access	Routed access	Virtual switching
Uplinks forwarding	One per VLAN	Both (ECMP)	Both (MEC)
STP role	Active, blocking	Only on edge ports	Loop protection only
FHRP needed	Yes	No	No
VLAN spans access switches	Yes	No	Yes
Failure domain	Whole VLAN	One access switch	Whole VLAN

Imagine an office with two reception desks, one at each end of a long corridor. Visitors always enter through the east door, but the only receptionist on duty sits at the west desk. Every visitor walks the whole corridor, then walks back. Nothing is broken, yet everything is slower and the corridor is crowded. That is what happens in an L2 access design when spanning tree sends frames up one uplink while the HSRP active gateway sits on the other distribution switch. This chapter shows how to align them and then walks through the first lab, Build a redundant collapsed-core block.

Why alignment matters

In the looped triangle, spanning tree picks one root bridge per VLAN and blocks the access uplink that leads away from it. HSRP independently picks one active router per VLAN. If DS2 is the STP root for VLAN 10 but DS1 is HSRP active, PC1's frames go AS1 → DS2 (the forwarding uplink), cross the DS1–DS2 link and only then reach the gateway on DS1. The inter-switch link carries all of VLAN 10's routed traffic and every packet takes an extra hop.

The design rules

1. Per VLAN, one owner. The same distribution switch is STP root and HSRP active; the other is secondary root and HSRP standby.
2. Load-share by VLAN. DS1 owns VLAN 10, DS2 owns VLAN 20, so both uplinks carry traffic.
3. Preempt on both switches. Without standby preempt, a recovered owner stays standby and alignment is lost after the first failure. In production add standby 10 preempt delay minimum 60 so the switch waits for routing to converge before taking over.
4. Bundle the inter-switch link with LACP. Two links in one Port-channel look like one link to STP, so a member failure causes no topology change, and LACP detects mis-cabling that mode on would hide.
5. Both distribution switches in the routing protocol, so the WAN router has two equal-cost paths back to every user subnet.

Lab walkthrough: build a redundant collapsed-core block

Start state: addressing, SVIs and HSRP virtual IPs (10.1.10.1 and 10.1.20.1) exist; nothing is aligned, the DS1–DS2 links are separate, and DS2 is missing from OSPF.

Task 1: LACP bundle between DS1 and DS2

! On DS1 and on DS2
interface range g0/6 - 7
 channel-group 1 mode active
interface port-channel 1
 switchport mode trunk
 switchport trunk allowed vlan 10,20

DS1# show etherchannel summary
Group  Port-channel  Protocol    Ports
------+-------------+-----------+-----------------------------------------------
1      Po1(SU)         LACP      Gi0/6(P)    Gi0/7(P)

SU means Layer 2 and in use; P means bundled. Anything else (I stand-alone, s suspended, D down) means the bundle is not healthy.

Task 2: STP roots per VLAN

DS1(config)# spanning-tree vlan 10 root primary
DS1(config)# spanning-tree vlan 20 root secondary
DS2(config)# spanning-tree vlan 20 root primary
DS2(config)# spanning-tree vlan 10 root secondary

The root primary macro writes priority 24576, or 4096 less than the current root if that is already lower; root secondary writes 28672. It is a one-time calculation saved as a number in the configuration.

AS1# show spanning-tree vlan 10
  Root ID    Priority    24586
             Cost        4
             Port        1 (GigabitEthernet0/0)
Interface           Role Sts Cost      Prio.Nbr Type
Gi0/0               Root FWD 4         128.1    P2p
Gi0/1               Altn BLK 4         128.2    P2p

24586 is 24576 plus the VLAN number (extended system ID). Gi0/0 towards DS1 is the root port; for VLAN 20 the roles reverse.

Task 3: HSRP owners with preempt

DS1(config)# interface vlan 10
DS1(config-if)# standby 10 priority 110
DS1(config-if)# standby 10 preempt
DS1(config)# interface vlan 20
DS1(config-if)# standby 20 preempt
! DS2 mirrors it: priority 110 + preempt for group 20, preempt for group 10

DS1# show standby brief
Interface   Grp  Pri P State   Active          Standby         Virtual IP
Vl10        10   110 P Active  local           10.1.10.3       10.1.10.1
Vl20        20   100 P Standby 10.1.20.3       local           10.1.20.1

Task 4: DS2 into OSPF

DS2(config)# router ospf 1
DS2(config-router)# router-id 1.1.1.3
DS2(config-router)# network 10.0.2.0 0.0.0.3 area 0
DS2(config-router)# network 10.1.0.0 0.0.255.255 area 0

R1# show ip route ospf
O        10.1.20.0/24 [110/2] via 10.0.2.2, 00:00:21, GigabitEthernet0/1
                      [110/2] via 10.0.1.2, 00:00:21, GigabitEthernet0/0

Two equal-cost paths: return traffic can reach the users through either distribution switch.

Tasks 5 and 6: failover test and the virtual MAC

Shut interface vlan 10 on DS1. DS2 logs %HSRP-5-STATECHANGE: Vlan10 Grp 10 state Standby -> Active and PC1 still pings 172.16.0.10. Bring the SVI back with no shutdown; after the hold and preempt timers DS1 logs Speak -> Active or Standby -> Active and owns VLAN 10 again. Finally, show standby vlan 10 shows the virtual MAC.

DS1# show standby vlan 10
Vlan10 - Group 10
  State is Active
  Virtual IP address is 10.1.10.1
  Active virtual MAC address is 0000.0c07.ac0a (MAC In Use)
  Preemption enabled
  Priority 110 (configured 110)

A plane has two pilots. If the captain falls ill, the first officer is already in the cockpit, already knows the altitude, speed and route, and simply takes the controls. The engines never stop. Air-traffic control keeps talking to the same flight number. Network devices have exactly this idea inside one chassis: a second supervisor (the pilot) that knows the full state, hardware that keeps forwarding (the engines), and neighbours that are told not to panic (air-traffic control). This chapter covers the tools that make a single box, or a pair acting as one box, survive its own failures.

Layers of redundancy

So far you have protected against link failures (EtherChannel, dual uplinks), gateway failures (HSRP, VRRP, GLBP) and path failures (OSPF ECMP). Device-level HA protects against failures inside a device: a supervisor engine crash, a software fault, a stack member dying. It matters most where a single box is a single point of failure, such as a modular core chassis or an access switch with users attached.

Supervisor redundancy modes: RPR, RPR+ and SSO

A modular switch (for example a Catalyst 9400 or 9600) can hold two supervisors: one active, one standby. How ready the standby is depends on the redundancy mode:

Mode	Standby state	Switchover impact
RPR	Partially booted, not synchronised	Line cards reset; minutes of outage
RPR+	Fully booted, configuration synchronised	Links stay up but state is rebuilt; tens of seconds
SSO	Fully booted, configuration and state synchronised (STANDBY HOT)	Layer 2 state and links kept; typically around a second or less

SSO (stateful switchover) continuously copies the running configuration and protocol state (interface state, MAC tables, STP, LACP, and more) to the standby. On failure the standby takes over with no link flaps and no spanning-tree reconvergence.

redundancy
 mode sso

CORE1# show redundancy states
       my state = 13 -ACTIVE
     peer state = 8  -STANDBY HOT
           Mode = Duplex
Redundancy Mode (Operational) = sso
Redundancy Mode (Configured)  = sso

NSF: keep forwarding while routing restarts

SSO synchronises Layer 2 state, but routing protocol adjacencies traditionally restart on the new supervisor. NSF (non-stop forwarding) lets the hardware keep forwarding packets using the last known FIB while the new active supervisor rebuilds its routing table. To stop neighbours from tearing down adjacencies and rerouting around the switch, the routing protocol uses graceful restart (GR): the restarting router signals that it is restarting, and NSF-aware (helper) neighbours keep its routes and adjacency for a grace period while databases resynchronise.

• NSF-capable: the device that can restart gracefully (dual supervisors with SSO).
• NSF-aware / helper: a neighbour that understands graceful restart and keeps forwarding towards the restarting device. Most modern IOS XE devices are NSF-aware by default.

router ospf 1
 nsf                        ! Cisco NSF; "nsf ietf" for the RFC version
router bgp 65001
 bgp graceful-restart

NSR: neighbours never notice

NSR (non-stop routing) goes one step further. The standby supervisor keeps a live copy of the routing protocol state itself: adjacencies, databases, BGP sessions and TCP state. After a switchover the new active continues the conversation as if nothing happened, so no helper is needed and neighbours never see a restart. NSR is useful when neighbours are not NSF-aware or belong to another organisation, such as a service provider.

router ospf 1
 nsr

StackWise: many switches, one brain

StackWise (for example StackWise-480 on Catalyst 9300) joins up to eight access switches with stack cables into one logical switch with one management IP and one configuration. One member is active, one is standby (SSO between them), the rest are members. You can build a cross-stack EtherChannel with uplinks on different members, so losing a member does not isolate the stack. The member with the highest stack priority (1 to 15) becomes active.

ACC-STACK# show switch
Switch#   Role    Mac Address     Priority Version  State
-------------------------------------------------------------
*1       Active   00aa.bb00.1100     15     V01     Ready
 2       Standby  00aa.bb00.2200     14     V01     Ready
 3       Member   00aa.bb00.3300     1      V01     Ready

StackWise Virtual: two chassis, one logical switch

StackWise Virtual (SVL) joins two distribution or core switches (Catalyst 9400, 9500 or 9600) over a StackWise Virtual link of one or more Ethernet ports. The pair has one control plane (active plus hot standby with SSO) and two data planes, both forwarding. Access switches connect with a multichassis EtherChannel, so no STP blocking and no FHRP are needed. A separate dual-active detection (DAD) link prevents both switches from becoming active if the SVL fails.

stackwise-virtual
 domain 10
interface range TenGigabitEthernet1/0/47 - 48
 stackwise-virtual link 1
interface TenGigabitEthernet1/0/46
 stackwise-virtual dual-active-detection
! save and reload both switches to form the pair; verify with show stackwise-virtual

A metro rail network with a ring line and radial lines has a nice property: from any station you can reach any other station with at most one change, and if one ring segment closes, trains simply use the other direction. Compare that with an old tree-shaped bus network where every trip goes to the central depot first. Data-centre and modern campus fabrics are the metro: every edge switch is exactly one hop from every other edge switch through a set of equal paths. This chapter covers fabric designs (ENCOR 1.1 lists "fabric" next to 2-tier and 3-tier) and the WAN and branch designs that connect an enterprise's sites.

Why the data centre moved to spine-leaf

Traditional three-tier data centres were built for north-south traffic: clients outside talking to servers inside. Modern applications are split into many services that talk to each other, so most traffic is now east-west, server to server. In a three-tier design with spanning tree, east-west traffic often climbs to the aggregation or core layer and back down, half the links are blocked, and latency differs depending on where two servers sit.

A spine-leaf (Clos) fabric fixes this with two strict rules:

• Every leaf (top-of-rack switch where servers, firewalls and routers attach) connects to every spine.
• Spines never connect to each other, and leaves never connect to each other (apart from special pairs such as vPC or MLAG peers).

So any server reaches any other server on another leaf in exactly leaf → spine → leaf: the same number of hops and predictable latency. The links are routed and every spine offers an equal-cost path, so traffic is spread with ECMP and nothing is blocked.

Scaling and oversubscription in a fabric

You scale a fabric out, not up. Need more server ports? Add a leaf. Need more bandwidth between leaves? Add a spine and one more uplink on every leaf. The limits are simple: the number of spines is capped by the uplink ports on each leaf, and the number of leaves is capped by the ports on each spine.

LEAF1# show ip route 10.20.3.0
Routing entry for 10.20.3.0/24
  Known via "ospf 1", distance 110, metric 3, type intra area
  Routing Descriptor Blocks:
  * 10.255.1.1, from 10.255.0.3, via HundredGigE1/0/49
    10.255.2.1, from 10.255.0.3, via HundredGigE1/0/50

Two spines, two equal-cost next hops towards LEAF3's subnet.

Underlay and overlay

A fabric has two layers. The underlay is the routed physical network (OSPF, IS-IS or eBGP between leaves and spines) that simply makes every switch's loopback reachable. The overlay builds tenant networks on top, typically VXLAN tunnels between leaves, so a VLAN or subnet can appear on any leaf without stretching spanning tree. In data centres the overlay control plane is usually BGP EVPN.

The same idea came to the campus as SD-Access: a routed underlay (often IS-IS built by LAN automation), LISP as the control plane that tracks where each endpoint is, VXLAN as the data plane, and Scalable Group Tags as the policy plane, managed by Catalyst Center. Its roles are edge nodes (where users attach), border nodes (exits to other networks) and control-plane nodes (the LISP map server). You get routed access for resiliency, yet a subnet can still exist on every edge node. The SD-WAN/SD-Access and LISP/VXLAN modules go deep; here you only need to recognise fabric as a design option.

WAN and branch designs

Branches range from a two-person sales office to a thousand-person regional hub, so there is no single branch design. Choose by size and by how much downtime the branch can tolerate:

Branch size	Typical design	Weak point
Small	One router or SD-WAN edge, one or two WAN links (for example broadband plus 4G/5G), access switch with integrated services	Single device
Medium	Two routers or edges, two transports (MPLS plus internet), FHRP or routing towards a switch stack	Single switch stack
Large	Dual edges, dual transports, a small collapsed core or full access-distribution block	Cost and complexity

Transports include MPLS L3VPN (provider-managed, SLAs), dedicated or broadband internet with IPsec or DMVPN, and cellular backup. SD-WAN overlays all of them, measures loss, latency and jitter on each, and steers each application to the best path. The internet edge module usually has redundant firewalls, two ISPs and BGP for multihoming.

Owning a car and using a ride-hailing app both get you to work. Owning costs a lot up front, but the car is always in your driveway, you can modify it, and daily trips are cheap once it is paid for. Ride-hailing costs nothing up front, scales instantly when your whole family needs to travel, and someone else handles servicing, but you pay every trip, depend on the app and the roads, and cannot choose the engine. On-premises infrastructure is the owned car; public cloud is ride-hailing. ENCOR expects you to explain the difference and know how a network engineer connects the two.

Deployment models

On-premises

The organisation owns and runs the hardware in its own data centre or a rented colocation space. Full control, full responsibility.

Private cloud

On-premises (or dedicated hosted) infrastructure run with cloud-style self-service, automation and virtualisation for one organisation.

Public cloud

A provider's shared infrastructure, rented on demand and billed by use, reached over the internet or private interconnects.

Hybrid cloud

On-premises or private cloud connected to public cloud, with workloads placed where they fit best. This is what most enterprises actually run.

Multicloud

Using two or more public cloud providers, for resilience, features or commercial reasons.

Service models describe how much the provider manages: IaaS (you get virtual machines, storage and virtual networks and manage the OS and everything above), PaaS (you deploy code or containers onto a managed platform) and SaaS (you just use the application, for example hosted email or CRM). Networking itself can also be consumed as a service, such as cloud-managed switches and access points whose management plane lives in the provider's cloud.

Comparing on-premises and cloud

Factor	On-premises	Public cloud
Cost model	CapEx: buy hardware up front, depreciate over years	OpEx: pay per hour, per GB and per request
Scaling	Size for peak; adding capacity takes weeks	Elastic; scale in minutes, scale down to save money
Control	Full control of hardware, software versions, topology	Limited to what the provider exposes
Latency and locality	Close to campus users and factory systems	Depends on WAN or internet path to the region
Compliance	Data stays where you put it	Choose regions carefully; shared-responsibility model
Operations	Your team patches, replaces and cools everything	Provider runs facilities, hardware and hypervisor

The shared-responsibility model is key. With IaaS, the provider secures the buildings, hardware and hypervisor; you still own the guest operating systems, applications, identities, data and your virtual network rules such as security groups and route tables. Moving to the cloud does not move security responsibility away from you.

How the network connects to the cloud

• Site-to-site IPsec VPN over the internet: quick and cheap; use two tunnels to two provider gateways and run BGP over them.
• Private or dedicated interconnect: a physical circuit from your router, usually in a colocation facility, into the provider's network. Predictable latency and bandwidth; build two in different locations for resilience.
• SD-WAN cloud on-ramp: virtual SD-WAN routers (for example Catalyst 8000V) inside the cloud region become part of the SD-WAN fabric, and branches can reach SaaS directly instead of hairpinning through the head office.
• Virtual network functions in the cloud: virtual routers, firewalls and even wireless controllers (Catalyst 9800-CL) run as cloud instances.

EDGE1# show ip bgp summary
Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
169.254.10.1    4        64512     220     218       14    0    0 03:12:41        2
169.254.11.1    4        64512     219     218       14    0    0 03:12:39        2

Two VPN tunnels to the cloud gateway, each with an eBGP session learning the two cloud prefixes. If one tunnel fails, BGP converges to the other. Link-local 169.254.x.x addresses inside the tunnels are common with cloud VPN gateways.

Think of a restaurant chain. A tiny café can have its own cook who decides everything alone. A big mall food court sends every order to one central kitchen. A chain of small outlets might have a head chef who writes the menu centrally, while each outlet cooks locally so it keeps serving even when the phone line to head office is down. And some chains let a remote company run their ordering system from an app. Wireless networks are deployed in exactly these ways. ENCOR 1.2 asks you to compare them at a design level; the enterprise wireless module later covers RF, roaming and controller configuration.

Controller-less versus controller-based

The first split is whether there is a controller at all.

• Controller-less (autonomous): each access point is a complete, independent device with its own configuration, SSIDs, security and RF settings. Fine for one or two APs; painful beyond that, because every change is repeated on every AP and features such as coordinated radio resource management and fast roaming are limited. Some material also calls an AP running an embedded wireless controller (one AP acting as controller for its neighbours) controller-less, because there is no separate controller box.
• Controller-based (lightweight APs): APs are managed by a wireless LAN controller (WLC) using CAPWAP. This is the split-MAC model: real-time functions (beacons, acknowledgements, encryption) stay on the AP, while management, authentication, RF management and roaming decisions move to the controller. CAPWAP uses UDP 5246 for control and UDP 5247 for data.

Where the controller and the data path live

Model	Controller	Client data path	Best fit
Centralised	WLC (appliance or pair) in the data centre or services block	Tunnelled in CAPWAP from AP (local mode) to the WLC, then switched there	Campus with good LAN bandwidth; central policy and easy roaming
Distributed	Controller function close to the edge: embedded on Catalyst 9000 switches, or central WLC with locally switched APs	Switched at the access layer (in SD-Access, VXLAN from AP to fabric edge)	Large campuses and fabric designs; avoids hairpinning all traffic to one place
Cloud	Management in the cloud (cloud-managed APs) or a virtual controller such as Catalyst 9800-CL in a public or private cloud	Local at the site; only management goes to the cloud	Many small sites, lean IT teams, fast rollout
Remote branch	Central WLC over the WAN with FlexConnect APs, or an embedded controller on a branch AP	Locally switched onto the branch VLAN; can keep working if the WAN fails	Branches and retail stores

FlexConnect for branches

A FlexConnect AP keeps its CAPWAP control connection to a central WLC but switches client traffic onto a local VLAN. If the WAN fails, it enters standalone mode: already-connected clients stay connected, and with local authentication configured new clients can still join. On a Catalyst 9800 controller, APs become FlexConnect through a site tag that is not a local site:

wireless profile flex BR1-FLEX
 native-vlan-id 30
wireless tag site BR1-SITE
 flex-profile BR1-FLEX
 no local-site

WLC1# show ap summary
Number of APs: 2
AP Name     Slots  AP Model     Ethernet MAC    Radio MAC       Location  Country  IP Address   State
AP-HQ-01    2      C9120AXI-D   00aa.bb11.0001  00aa.bb22.0001  HQ-FL1    IN       10.1.30.21   Registered
AP-BR1-01   2      C9120AXI-D   00aa.bb11.0002  00aa.bb22.0002  Branch1   IN       10.50.30.21  Registered

Design factors beyond the model

• Client density: auditoriums, classrooms and stadiums are designed for capacity, not coverage. Use more APs at lower power, prefer 5 GHz and 6 GHz, use narrower channels to get more of them, and plan the number of clients per radio.
• Location services: accurate location needs every point heard by at least three APs at about -75 dBm or better, with APs placed along the perimeter and staggered, not just down the corridor.
• Controller redundancy: N+1 (one spare WLC for many), N+N (two sets sharing load), N+N+1, or an HA SSO pair in which a standby controller holds AP and client state, so a failure does not force APs to rejoin.
• Latency and bandwidth between AP and controller: local mode depends on the LAN or WAN path for every packet, which is why remote sites use FlexConnect.

A busy post office can sort letters in three ways. A clerk can read every envelope and look the address up in a big book (slow, but always correct). A clerk can keep sticky notes for addresses seen recently, so only the first letter to a new address is slow. Or the office can print a complete sorting table for every possible destination before the morning mail arrives, with a pre-printed label for each outgoing van, and feed it to an automatic sorting machine. Routers and switches went through exactly these three stages: process switching, fast switching and Cisco Express Forwarding (CEF) in hardware. ENCOR 1.7 asks you to explain them and the tables behind them.

Control plane and data plane

The control plane is the device's brain: routing protocols, spanning tree, ARP, building tables. The data plane is its muscle: moving each packet from an input port to an output port as fast as possible. The management plane is how you reach the device (SSH, SNMP, NETCONF). Good switching design keeps the data plane in hardware and the control plane on the CPU.

From process switching to CEF

Method	How it works	Weakness
Process switching	The CPU handles every packet: routing table lookup, ARP lookup, rewrite	Slow, CPU bound
Fast switching	First packet to a destination is process-switched; the result is cached and later packets use the cache ("route once, switch many")	Demand-driven: first packet always slow; cache churn with many flows
CEF	Tables are built in advance from the routing table and ARP, before any packet arrives	Tables use memory; hardware tables have finite size

CEF is topology-driven: whenever the routing table or ARP changes, CEF updates its tables immediately, so there is no first-packet penalty. It is the default on all modern Cisco platforms.

The four tables

RIB (routing information base)

The routing table built by the control plane from connected, static and dynamic routes. show ip route.

FIB (forwarding information base)

CEF's copy of the RIB, optimised for lookup, with recursive next hops already resolved to a directly connected next hop. show ip cef.

Adjacency table

For every directly connected next hop, the pre-built Layer 2 header (destination MAC, source MAC, EtherType) learned from ARP. show adjacency detail.

CAM / MAC address table

Layer 2 switching: VLAN plus MAC to port, an exact-match lookup. show mac address-table.

Special adjacencies handle exceptions: glean (the destination is on a connected subnet but its ARP entry is missing, so the packet is punted to trigger ARP), punt (send to the CPU, for example traffic addressed to the device), drop, discard and null (routes to Null0).

CAM and TCAM: the hardware memories

CAM (content-addressable memory) is searched by content rather than by address, and answers in one lookup. It is binary: every bit must match exactly (0 or 1), which is perfect for the MAC address table, where a VLAN plus MAC either matches or does not.

TCAM (ternary CAM) adds a third state, X, "don't care". Each entry is a value, mask and result (VMR). That makes it ideal for lookups that are not exact: longest-prefix match for the FIB, ACLs, QoS classification and policy-based routing. All entries are compared in parallel, so a 1,000-line ACL is checked as fast as a 10-line one. TCAM is expensive and limited; on Catalyst switches SDM templates decide how it is shared between routes, MAC addresses, ACLs and QoS.

Verification on the lab network

R1# show ip cef 10.1.20.0/24
10.1.20.0/24
  nexthop 10.0.1.2 GigabitEthernet0/0
  nexthop 10.0.2.2 GigabitEthernet0/1

R1# show ip cef exact-route 172.16.0.10 10.1.20.12
172.16.0.10 -> 10.1.20.12 =>IP adj out of GigabitEthernet0/1, addr 10.0.2.2

AS1# show mac address-table vlan 10
Vlan    Mac Address       Type        Ports
----    -----------       --------    -----
  10    0000.0c07.ac0a    DYNAMIC     Gi0/0
  10    0050.7966.6801    DYNAMIC     Gi0/2

CEF shares load per destination by default: a hash of source and destination address picks one of the equal-cost next hops, so one flow stays on one path and packets are not reordered. On AS1 the HSRP virtual MAC is learned on Gi0/0, the uplink towards DS1, which is exactly what an aligned design should show.

When hardware falls back to software

Some packets are always punted to the CPU: packets addressed to the device, TTL expiry, IP options, glean adjacencies, and traffic that needs features the ASIC cannot do. If TCAM runs out, new routes or ACL entries cannot be programmed in hardware and matching traffic may be software-switched or dropped, which shows up as high CPU. On Catalyst 9000 check resources with show platform hardware fed switch active fwd-asic resource tcam utilization and the template with show sdm prefer.

A building inspector does not start by knocking down walls. She takes the architect's drawing, walks the building floor by floor, and marks every place where reality differs from the plan. A design review of a network works the same way: you start from the design intent, check each layer in a fixed order, and list every difference. Most "the network is slow" tickets in redundant campuses are not broken links; they are redundancy that no longer matches the design. This chapter gives you the workflow and walks through the second lab, Design review: traffic takes the long way.

The design-review workflow

1. Design intent. Write down what should be true: which switch owns which VLAN (STP root and HSRP active), the inter-switch bundle (members and protocol), and which switches are in OSPF.
2. Bundles. show etherchannel summary and show lacp neighbor on both ends. Healthy is Po(SU) with every member (P).
3. Spanning tree. On the access switch, show spanning-tree vlan N shows the root port; show cdp neighbors maps that port to a switch. On distribution, show spanning-tree root.
4. FHRP. show standby brief on both switches: active, standby, priority, preempt flag.
5. Routing. show ip ospf neighbor on distribution, show ip route ospf on the WAN router for equal-cost paths.
6. Verify forwarding. Ping and traceroute from hosts; on the access switch check which uplink learned the virtual MAC.

Symptom to cause

What you see	Likely cause	Next check
Inter-switch link hot, one uplink per access switch idle	STP root and HSRP active on different switches	show spanning-tree vlan N on access, show standby brief
Po1(SD) on one side, members (I) or (s)	Channel mode mismatch (on versus LACP) or LACP not negotiating	show etherchannel summary both sides, show lacp neighbor
Preferred switch stays HSRP standby after recovery	Preempt missing	show standby, look for "Preemption enabled"
WAN router has one path to a user subnet	A distribution switch missing from OSPF	show ip ospf neighbor, show ip route ospf

Lab walkthrough: traffic takes the long way

Ticket: the DS1–DS2 port-channel is down and VLAN 10 users take a detour, going up to DS2 and hairpinning through AS1 to reach DS1. Design intent: DS1 owns VLAN 10, DS2 owns VLAN 20, the interconnect is a two-member LACP bundle.

Task 1: who is the root for VLAN 10?

AS1# show spanning-tree vlan 10
VLAN0010
  Root ID    Priority    24586
             Cost        4
             Port        2 (GigabitEthernet0/1)
AS1# show cdp neighbors
Device ID    Local Intrfce   Holdtme   Capability  Platform  Port ID
DS1          Gig 0/0         152       R S I                 Gig 0/0
DS2          Gig 0/1         147       R S I                 Gig 0/0

The root port is Gi0/1 and CDP says Gi0/1 connects to DS2, so the answer is DS2. The running configuration explains why: DS1 has spanning-tree vlan 10 priority 28672 and vlan 20 priority 24576, DS2 the opposite. The priorities are swapped, while HSRP (DS1 priority 110 for group 10, DS2 for group 20) follows the design.

Task 2: realign the roots

DS1(config)# spanning-tree vlan 10 root primary
DS1(config)# spanning-tree vlan 20 root secondary
DS2(config)# spanning-tree vlan 20 root primary
DS2(config)# spanning-tree vlan 10 root secondary

Because DS2 already had 24576 for VLAN 10, the root primary macro on DS1 writes 20480 (4096 lower). Check the result on AS1: VLAN 10 root port Gi0/0 (DS1), VLAN 20 root port Gi0/1 (DS2).

Task 3: repair the bundle

DS1# show etherchannel summary
Group  Port-channel  Protocol    Ports
1      Po1(SD)         LACP      Gi0/6(I)    Gi0/7(I)

DS2# show etherchannel summary
Group  Port-channel  Protocol    Ports
1      Po1(SU)          -        Gi0/6(P)    Gi0/7(P)

DS1 speaks LACP (mode active) and receives no LACP packets, so its ports stay stand-alone (I) and its Po1 is down. DS2 uses mode on: no protocol ("-"), so it bundles unconditionally and believes Po1 is up. The two sides disagree, the interconnect is unusable, and anything that must cross between DS1 and DS2 at Layer 2 is forced down through AS1 and back up: the hairpin in the ticket.

DS2(config)# interface range g0/6 - 7
DS2(config-if-range)# no channel-group 1
DS2(config-if-range)# channel-group 1 mode active

The Port-channel1 interface and its trunk settings stay in place, and the members inherit them when they rejoin. Verify Po1(SU) with both members (P) and protocol LACP on both switches.

Task 4: verify gateways and reachability

DS1# show standby brief
Interface   Grp  Pri P State   Active          Standby         Virtual IP
Vl10        10   110 P Active  local           10.1.10.3       10.1.10.1
Vl20        20   100 P Standby 10.1.20.3       local           10.1.20.1
PC1> ping 172.16.0.10
84 bytes from 172.16.0.10 icmp_seq=1 ttl=62 time=3.112 ms

TTL 62 means two routed hops (DS1 and R1), the direct path. Repeat from PC2 through DS2.

You have walked the whole city: lanes, arterial roads and highways; the rule that every district has two roads in; the traffic police that must stand where the cars actually arrive; the hot-standby pilot; the metro-style fabric; the rented cars of the cloud; the restaurant kitchens of wireless; and the sorting machine inside every switch. This chapter packs it into a checklist you can revise from the night before the exam or an interview.

Can you do all of this?

• Explain hierarchy, modularity, resiliency and flexibility, and give a network example of each.
• Define a failure domain and describe three ways to keep it small.
• Calculate an oversubscription ratio, counting only forwarding links, and compare it with 20:1 (access to distribution) and 4:1 (distribution to core).
• Describe the roles of access, distribution and core, and say when a collapsed core is enough.
• Compare L2 access, routed access and virtual switching with MEC.
• Align STP root and HSRP active per VLAN, with preempt, and bundle the interconnect with LACP.
• Build and verify the collapsed-core lab: Po1(SU), root ports on AS1, show standby brief, ECMP on R1, failover and the virtual MAC 0000.0c07.ac0a.
• Explain RPR, RPR+, SSO, NSF, graceful restart, NSR, StackWise and StackWise Virtual.
• Describe spine-leaf rules, how to scale a fabric, and the SD-Access planes.
• Choose between on-premises, cloud and hybrid for a given workload and describe how to connect them.
• Compare centralised, distributed, controller-less, controller-based, cloud and remote-branch WLAN models.
• Explain process switching, fast switching and CEF, and the RIB, FIB, adjacency table, CAM and TCAM.
• Run a design review and find a root/active mismatch and a channel-mode mismatch.

Mini glossary

Collapsed core

Two-tier design in which distribution and core run on one switch pair.

Failure domain

The part of the network affected by one failure.

Oversubscription

Downstream bandwidth divided by usable upstream bandwidth.

Routed access

Access switch is the default gateway and routes on its uplinks; no FHRP, no uplink blocking.

MEC

Multichassis EtherChannel: one bundle to two physical switches acting as one.

SSO / NSF / NSR

Stateful standby supervisor / keep forwarding on the last FIB / keep routing state on the standby.

Graceful restart

Protocol signalling that asks NSF-aware neighbours to keep routes during a restart.

Spine-leaf

Fabric in which every leaf connects to every spine; always leaf-spine-leaf.

FlexConnect

AP mode with control to a central WLC and data switched locally; survives WAN loss.

FIB / adjacency

CEF's forwarding table and its pre-built Layer 2 rewrites.

TCAM

Ternary memory with value, mask and result entries for routes, ACLs and QoS.

Most-tested facts

Topic	Remember
Collapsed core	Merges distribution and core
Oversubscription guidance	About 20:1 access to distribution, 4:1 distribution to core
Routed access	No FHRP, no STP blocking on uplinks, VLAN cannot span
Alignment	Same switch is STP root and HSRP active per VLAN, with preempt
Root macros	primary = 24576 or 4096 below current root; secondary = 28672
HSRPv1 MAC	0000.0c07.acXX (group in hex)
EtherChannel modes	on only with on; active with active or passive
SSO vs NSF vs NSR	state sync; forward on last FIB with GR helpers; routing state on standby, no helper
Spine-leaf	Add spines for bandwidth, leaves for ports
SD-Access planes	LISP control, VXLAN data, SGT policy
CAPWAP	UDP 5246 control, UDP 5247 data
CAM vs TCAM	binary exact match vs ternary with don't-care
CEF	topology-driven, per-destination load sharing by default

Command cheat-sheet

! Bundle
interface range g0/6 - 7
 channel-group 1 mode active
show etherchannel summary
show lacp neighbor
! STP and HSRP alignment
spanning-tree vlan 10 root primary
spanning-tree vlan 20 root secondary
interface vlan 10
 standby 10 priority 110
 standby 10 preempt
show spanning-tree vlan 10
show standby brief
! Routing and forwarding
show ip route ospf
show ip cef 10.1.20.0/24
show ip cef exact-route 172.16.0.10 10.1.20.12
show adjacency detail
show mac address-table vlan 10
! Device HA
redundancy
 mode sso
show redundancy states
show switch
show stackwise-virtual