SD-WAN: members, SLAs & rules

What you will learn in this module. You will learn how FortiGate SD-WAN bundles several internet or private links into one logical interface, measures their quality, and steers each type of traffic over the best link. You will build members and zones, health checks and SLA targets, SD-WAN rules, and understand the implicit rule, load balancing and failover. These are the SD-WAN objectives of the Fortinet NSE 4 / FortiOS 7.6 Administrator exam.

Prerequisites. Finish nse4-routing first: you must be comfortable with the routing table, distance, floating routes and firewall policies per interface.

Analogy: a delivery company with a live traffic app

A floating static route is like telling every driver: "always take the highway; if it is closed, take the village road". It cannot tell that the highway is open but jammed, and it treats an urgent medicine parcel and a catalogue the same. SD-WAN is a dispatcher with a live traffic app. It measures every road all the time (loss, delay, jitter), knows what each parcel is, and says: "medicine over the fastest road, catalogues over the cheap road, and if a road gets bad, switch."

The five building blocks

Block	What it is
Member	One link (interface and gateway) that joins SD-WAN.
Zone	A group of members; the object that routes and policies point to.
Health check	A probe (ping, DNS and so on) that measures each member.
SLA	The thresholds that decide whether a member is good enough.
Rule (service)	Which traffic goes over which members and by which strategy.

Traffic that matches no rule falls to the implicit rule, which uses the normal routing table and a load-balancing mode. The next chapters build each block in order.

Why SD-WAN and not floating routes

• Failover on quality, not only on a dead interface.
• Per-application steering: voice on the best link, backups on the cheap one.
• Use of all links at once instead of leaving the backup idle.
• One policy for the zone instead of one per link.

A member is a single WAN link described by an interface and, normally, the gateway of that link. A physical port, a VLAN, an aggregate or an IPsec tunnel interface can be a member. A zone is a named group of members. Out of the box FortiOS has one zone called virtual-wan-link; you can create more, for example UNDERLAY for ISP links and OVERLAY for VPN tunnels, so that rules and policies can address them separately.

! Try it: lab "Two ISPs become one SD-WAN zone"
config system sdwan
    set status enable
    config zone
        edit "virtual-wan-link"
        next
    end
    config members
        edit 1
            set interface "port1"
            set zone "virtual-wan-link"
            set gateway 203.0.113.1
        next
        edit 2
            set interface "port4"
            set zone "virtual-wan-link"
            set gateway 198.51.100.1
            set cost 10
        next
    end
end

The addresses are the same as the Pune branch of nse4-routing: ISP-A on port1 (203.0.113.2/30, gateway 203.0.113.1) and ISP-B on port4 (198.51.100.2/30, gateway 198.51.100.1). The member number is only an ID; it is the number you use later in rules and health checks.

Member attributes that matter

Attribute	Meaning
interface	The port, VLAN, aggregate or tunnel interface.
zone	The zone this member belongs to; default virtual-wan-link.
gateway	Next hop of this link. Not needed for DHCP or PPPoE links, which learn it.
cost	Used by the cost-based strategy (lowest cost link preferred).
priority	Preference of the member when the routing table must pick one.
weight	Share of sessions in weight-based load balancing.
status	enable or disable the member without deleting it.

The reference rule: clean the interface first

An interface cannot join SD-WAN while other parts of the configuration still use it by name. Before adding port1 and port4 as members you must remove or move every reference: static routes that point to the port, firewall policies that use it as source or destination, DHCP servers, and similar objects. Typical order of work on an existing branch:

1. Note existing routes, policies and NAT that use port1 and port4.
2. Delete the old default routes and the policy entries that name those ports.
3. Create the members and zone.
4. Create one default route for the zone and one policy to the zone (next chapter).
5. Test from a client, then add health checks and rules.

Members and zones alone carry no traffic. You need two more things that you already know from nse4-routing: a route and a policy. In SD-WAN, both point at the zone. The route tells the FortiGate "internet traffic leaves through the zone"; the policy says "the LAN may go to the zone, with NAT". SD-WAN then chooses the member.

! Try it: lab "Two ISPs become one SD-WAN zone"
config router static
    edit 1
        set sdwan-zone "virtual-wan-link"
        set comment "Default route over the SD-WAN zone"
    next
end
config firewall policy
    edit 10
        set name "LAN-to-SDWAN"
        set srcintf "port2"
        set dstintf "virtual-wan-link"
        set srcaddr "all"
        set dstaddr "all"
        set action accept
        set schedule "always"
        set service "ALL"
        set nat enable
    next
end

Compare this with the floating-route design of nse4-routing. There you needed two default routes with different distances and one policy that listed both interfaces. Here there is one default route and one policy, and the choice between ISP-A and ISP-B moves into SD-WAN where it can use quality measurements. The route has no gateway of its own because each member carries its gateway.

What changes in the routing table

With SD-WAN enabled, the zone's routes appear in the routing table through the member interfaces, and the table behaves like the ECMP example in chapter 6 of nse4-routing: the member gateways are the next hops. SD-WAN rules and the implicit rule decide which of them a given session uses. When a member is considered dead by its health check, its next hop is not used for new sessions even though the interface itself may still be up. That is the big difference from a plain static route.

Other routes and policies

• Routes to internal networks stay normal static routes (or OSPF/BGP) on the internal interfaces.
• A static route can name a member interface directly when you need a specific path, but then that traffic skips SD-WAN decisions.
• Policies to a zone apply to every member; you no longer need one per ISP.
• NAT still uses the address of the egress interface (see the failover story in nse4-routing).

A static route knows only whether the cable is up. SD-WAN wants to know whether the path works. A health check (called a performance SLA in the GUI) sends small probes through each member toward a reliable server, waits for answers and measures three things: packet loss, latency (delay) and jitter (variation of the delay). From those it derives a state for every member: alive or dead.

! Try it: lab "A performance SLA and failover when ISP-A degrades"
config system sdwan
    config health-check
        edit "INTERNET-PING"
            set server "192.0.2.53"
            set protocol ping
            set interval 1000
            set failtime 5
            set recoverytime 5
            set members 0
        next
    end
end

The probe target 192.0.2.53 is a documentation address standing for a stable, always-on internet server that you trust; in real life pick something reliable and not controlled by the ISP you are measuring, such as a public DNS service or a server you own.

The parameters

Parameter	Meaning
server	The probe target: an IP address or a name.
protocol	ping, tcp-echo, udp-echo, http, twamp, dns, tcp-connect, ftp.
interval	Milliseconds between probes (default 500).
failtime	Consecutive lost probes before a member is declared dead (default 5).
recoverytime	Consecutive good probes before a dead member is alive again (default 5).
members	Which members to probe; 0 means all.
update-static-route	Remove static routes of a dead member from the table.

How long does a failure take to detect?

Roughly interval times failtime. With an interval of 1000 ms and a failtime of 5, a member is declared dead after about five seconds of lost probes. Recovery works the same way with recoverytime. Short values react fast but make links "flap" on a single bad moment; long values are stable but slow. For most branches a few seconds is a good compromise. There is also a probe timeout: a probe that gets no answer within it counts as lost.

Choosing the protocol

• ping: simplest; proves the path to a host exists. Some servers or ISPs rate-limit ICMP.
• dns: asks a real name server, which proves that name resolution over that link works.
• http or tcp-connect: proves that a TCP service answers; useful for a specific SaaS application.
• twamp: standards-based latency measurement between FortiGates or a TWAMP responder.

Reading the result

On a real FortiGate diagnose sys sdwan health-check prints one block per health check and one line per member. Each line names the member, shows whether it is alive or dead, and lists loss, latency, jitter and an SLA map that says which SLA targets the member currently meets. Run it in the lab A performance SLA and failover when ISP-A degrades: the impair tool on a provider router adds delay, jitter or loss to a link, and you can watch the member's numbers and sla_map change. Probe latency in the simulator is one-way, so expect smaller numbers than a real round-trip probe.

"Alive" is not the same as "good". A link can answer probes but lose 8 percent of packets or take 400 ms, which ruins a voice call. An SLA target is a set of thresholds inside a health check: a member meets the SLA when its measured latency, jitter and packet loss are all within the thresholds. A member that is alive but misses the SLA is "out of SLA"; it still exists, but rules that require the SLA stop choosing it.

! Try it: lab "A performance SLA and failover when ISP-A degrades"
config system sdwan
    config health-check
        edit "INTERNET-PING"
            config sla
                edit 1
                    set link-cost-factor latency jitter packet-loss
                    set latency-threshold 100
                    set jitter-threshold 30
                    set packetloss-threshold 2
                next
            end
        next
    end
end

This target, number 1 inside the health check, says: latency up to 100 ms, jitter up to 30 ms, loss up to 2 percent. A health check can hold several SLA targets with different numbers, for example a strict one for voice and a relaxed one for web. Rules refer to a target as "health check name, SLA number".

link-cost-factor: ranking links by quality

The attribute link-cost-factor tells the "best quality" strategy which measurement to compare when several members are eligible: latency, jitter, packet-loss, inbound-bandwidth, outbound-bandwidth, bibandwidth, custom-profile-1 and others. Listing more than one makes the FortiGate combine them by weight. For a voice rule people usually choose latency and jitter; for a bulk-transfer rule, bandwidth.

Dead versus out of SLA

State	Cause	Effect
Dead	failtime probes lost in a row	Removed from every decision for new sessions
Alive, in SLA	All thresholds met	Eligible for all rules
Alive, out of SLA	A threshold exceeded	Skipped by rules that require this SLA; still usable by others

Two optional behaviours help in practice. Logging of SLA changes (sla-fail-log-period and sla-pass-log-period) writes an event when a member crosses a threshold, which is the evidence you need for an ISP complaint. A threshold that is too strict makes the decision flip back and forth; start from the application's real need (voice is comfortable under about 150 ms one way and a few percent loss) and tune with data.

An SD-WAN rule (a "service" in the CLI) says: "traffic that looks like this should use these members, chosen this way". It has a match part (source, destination, protocol and port, an Internet Service Database application such as a SaaS, or user) and a strategy part. Rules are read top to bottom and the first rule that matches decides; this is the same idea as a policy route, evaluated before the routing table.

! Try it: lab "A performance SLA and failover when ISP-A degrades"
config system sdwan
    config service
        edit 1
            set name "VOICE-LOW-LATENCY"
            set mode sla
            set dst "all"
            set src "VOICE-NET"
            set protocol 17
            set start-port 5060
            set end-port 5061
            config sla
                edit "INTERNET-PING"
                    set id 1
                next
            end
            set priority-members 1 2
        next
        edit 2
            set name "BACKUPS-CHEAP"
            set mode manual
            set dst "BACKUP-SERVERS"
            set priority-members 2
        next
    end
end

Rule 1 sends the voice network's SIP traffic over member 1 as long as it meets SLA 1 of the health check, otherwise over member 2. Rule 2 pins traffic to the backup servers to member 2, the cheap link (manual mode). In a rule, priority-members lists members in order of preference.

The strategies (modes)

GUI name	CLI mode	What it does
Manual	manual	Always use the listed member (if alive).
Best Quality	priority (link-cost-factor)	Use the member with the best measured value of the chosen factor, for example latency.
Lowest Cost (SLA)	sla	Use the first member in priority-members that meets the SLA.
Maximize Bandwidth (SLA)	load-balance	Share sessions over all members that meet the SLA.

Matching on applications

Instead of addresses a rule can match Internet Service Database (ISDB) objects, which FortiGuard keeps up to date with the addresses of well-known SaaS and cloud services. In the CLI that is set internet-service enable and set internet-service-name, choosing the name from the list shown by ? on your build. Application control signatures can be used as well, but they only work after the first packets have been inspected, so a session can start on one link and be steered later.

Tie-break

When two members are equally good, tie-break decides: zone (the default), cfg-order (order in the configuration), fib-best-match or input-device. Set it explicitly when the result matters.

What happens to traffic that matches none of your rules? It is handled by the implicit rule, the last, invisible rule of every SD-WAN configuration. The implicit rule does not look at quality. It uses the normal routing table and shares sessions among the members that have a route, using the load-balance mode of the SD-WAN settings. If you write no rules at all, everything uses the implicit rule, and SD-WAN behaves like ECMP with health-aware members.

! Try it: lab "A performance SLA and failover when ISP-A degrades"
config system sdwan
    set load-balance-mode weight-based
    config members
        edit 1
            set interface "port1"
            set weight 70
        next
        edit 2
            set interface "port4"
            set weight 30
        next
    end
end

load-balance-mode	How sessions are shared
source-ip-based	Default. Hash on the source address, so one client keeps one link.
weight-based	In proportion to member weights (70/30 above).
usage-based	Fill one link up to a limit, then use the next.
source-dest-ip-based	Hash on source and destination addresses.
measured-volume-based	Share by the volume each member has already carried.

Failover in three situations

1. Interface down. The member's routes are removed, exactly as with a floating route (nse4-routing chapter 5). No health check is needed.
2. Member dead. The health check lost failtime probes. The member is dropped from every decision for new sessions, although its interface stays up.
3. Member out of SLA. Only rules that require that SLA stop using it. The rule moves to the next member in priority-members that meets the SLA.

Two details to plan for. First, what a rule does when no member meets the SLA depends on the strategy and the firmware version, so build a last-resort member into the design and test it. Second, sessions that are already established are not always moved the moment a decision changes; behaviour for existing sessions has options and differs between failure types, so read the release notes of your version and test with a long download and a voice call.

Overlay members and routing protocols (a first look)

A member does not have to be an ISP port. IPsec tunnels towards a hub can be members of an overlay zone, so that one SD-WAN rule chooses between, say, an MPLS-based tunnel and an internet tunnel. Dynamic routing (BGP or OSPF over the tunnels, see nse4-dynamic) then supplies the routes to the other sites while SD-WAN decides which tunnel carries each flow. On a hub, SD-WAN can also use the BGP neighbours' state; that is an advanced topic beyond this exam.

SD-WAN adds three layers on top of ordinary routing: members, measurements and rules. A good workflow checks them in order and then falls back to the routing basics you practised in nse4-routing. The FortiOS commands for the SD-WAN layers all run in the simulator: practise them in the ticket lab Half the users cannot browse. The routing and flow outputs at the end were captured from the real nse4-routing labs.

Layer	Command (real FortiGate)	Question
Members	diagnose sys sdwan member	Are the right interfaces and gateways members of the right zone?
Zones	diagnose sys sdwan zone	Which members does each zone contain?
Health	diagnose sys sdwan health-check	Is each member alive, and does it meet the SLA?
Rules	diagnose sys sdwan service	Which members does each rule currently choose?
Routing	get router info routing-table all	Is the zone default route present?

Common causes and what you see

Symptom	Likely cause
Member shows dead although the ISP is fine	Probe target unreachable through that member, probe blocked, wrong protocol, or no route/NAT for the probe
Rule never chooses the expected member	An earlier rule matches first, or the SLA is too strict so no member qualifies
All traffic on one link	Only the first match or default source-ip hashing; check weights and rules
No internet after enabling SD-WAN	Missing zone default route or no policy to the zone
Interface refused as member	It is still referenced by a route, policy or DHCP server
SaaS application not steered	ISDB name wrong, or classification only happens after the first packets

The routing and flow checks (real output)

The two lab firewalls of nse4-routing have no SD-WAN, but the generic checks are identical. A dual-link default route with equal distance shows both next hops, which is what a zone with two members looks like in the table:

NK-PUNE-FGT # get router info routing-table details 0.0.0.0
Routing table for VRF=0
Routing entry for 0.0.0.0/0
  Known via "static", distance 10, metric 0, candidate default, best
  * vrf 0 203.0.113.1, via port1
  * vrf 0 198.51.100.1, via port4

And a missing policy shows up in the flow trace as a denied packet even when the route and the member are right. This is the same message you would see for a missing policy to the SD-WAN zone:

id=65308 trace_id=1 msg="find a route: flag=04000000 gw-198.51.100.1 via port4"
id=65308 trace_id=1 msg="in-[port2], out-[port4], skb_flags-02000000, vid-0, app_id: 0, url_cat_id: 0"
id=65308 trace_id=1 msg="Denied by forward policy check (policy 0)"

For an SD-WAN session the flow trace also shows a Match policy routing line naming the rule, and diagnose firewall proute list and diagnose sys session list (sdwan_mbr_seq, sdwan_service_id) show which rule and member a session used. The simulator prints these too.

This chapter gathers the SD-WAN module into a checklist, a glossary and the facts that appear most often in exam questions. The SD-WAN core in this module (zones, members, health checks, SLA, rules, failover and diagnose sys sdwan) runs in the three simulator labs; overlay IPsec, ADVPN, FortiManager and shaping are reference-only.

Can-do checklist

• Explain why SD-WAN beats floating routes: quality-based failover, per-application steering, all links in use.
• Create members and zones and list what must be removed from an interface first.
• Point one default route and one firewall policy at the zone.
• Configure a health check with server, protocol, interval, failtime and recoverytime.
• Define an SLA with latency, jitter and packet-loss thresholds.
• Write a rule with the right strategy and priority-members, and order rules correctly.
• Explain the implicit rule and the load-balance modes.
• Troubleshoot with diagnose sys sdwan commands and the routing basics.

Mini glossary

Member

A WAN link (interface plus gateway) in SD-WAN.

Zone

A named group of members; routes and policies point to it.

Health check

A probe that measures loss, latency and jitter per member.

SLA

Thresholds that define a good link.

Rule (service)

Match plus strategy, evaluated top-down.

Implicit rule

Handles unmatched traffic with routing and load balancing.

ISDB

Internet Service Database of SaaS and cloud addresses.

Most-tested facts

• Default zone: virtual-wan-link. Policies and the default route use the zone.
• Health check measures latency, jitter and packet loss; dead after failtime lost probes.
• A member can be alive and still fail the SLA.
• Strategies: manual, best quality, lowest cost (SLA), maximize bandwidth (SLA).
• Rules are matched top-down; unmatched traffic uses the implicit rule (default load balancing is source-ip-based).

Command cheat-sheet (reference)

! Try it: lab "Half the users cannot browse" and the other two labs of this module
config system sdwan
diagnose sys sdwan member
diagnose sys sdwan zone
diagnose sys sdwan health-check
diagnose sys sdwan service
get router info routing-table all
diagnose debug flow filter addr 10.0.1.10
diagnose debug flow trace start 5
diagnose debug enable

Where to practise

Repeat the three labs of this module until you can finish them without hints, then strengthen the pieces SD-WAN builds on: repeat both labs of nse4-routing (floating route and reverse-path check), practise tracking and policy routing in enarsi-pbr-vrf-bfd, and learn the dynamic routing used between SD-WAN sites in nse4-dynamic, enarsi-ospf, enarsi-bgp and jncis-ospf.