Fortigate SD-WAN ADVPN CLI Part 3

Part 2 in the series, I went through setting up the ADVPN between the Hub and spokes using the IPsec Wizard to build the VPN topology. In my lab, all sites have their own primary and secondary WAN links. This allows for redundancy and still maintains the ADVPN tunnels in the event of an outage in any of the links. Please do note, the beauty of a shortcut link between spoke sites stays up despite the spoke losing its connection to the hub site(parent tunnel).

If you are using the Fortigate VM downloaded from the fortigate website, they are good for 15 days and you’d have to rebuild them. That time has passed so I had to do a re-build and realised maybe I should do a lab this time using the CLI and be more granular.

The table below provides an IP addressing scheme for my lab environment. I know WAN links have private addresses, we’ll pretend they are publicly routable and RFC 1918 doesn’t exist 🙂

WAN 1

WAN 2

LAN

HW1SW1

HW1SW2

HW2SW1

HW2SW2

Hub 1

192.168.1.2

192.168.3.2

10.10.1.0

10.10.3.1

10.10.4.1

10.10.5.1

10.10.6.1

Spoke 1

192.168.4.2

192.168.5.2

10.10.2.0

10.10.3.3

10.10.4.3

10.10.5.3

10.10.6.3

Spoke 2

192.168.6.2

192.168.7.2

10.10.10.0

10.10.3.4

10.10.4.4

10.10.5.4

10.10.6.4

HUB Configuration

I won’t be totally doing everything hardcore CLI but more of a hybrid approach, its more of the building the ADVPN+BGP configuration I will be doing in CLI the rest like interfaces and firewall policies would still be via GUI as I assume most of us are familiar with it.

Most of the work will be done at the hub but once this is established you are well good to go. We will start by building the tunnels for it. We’ll start off with creating the ADVPN tunnels to the spokes.

Phase 1 IPsec Configuration

config vpn ipsec phase1-interface

edit “ADVPN”

set type dynamic

set interface “port1”

set peertype any

set net-device disable

set proposal des-md5 des-sha1

set add-route disable

set dpd on-idle

set auto-discovery-sender enable

set tunnel-search nexthop

set psksecret password1

next

edit “ADVPN2”

set type dynamic

set interface “port1”

set peertype any

set net-device disable

set proposal des-md5 des-sha1

set add-route disable

set dpd on-idle

set auto-discovery-sender enable

set tunnel-search nexthop

set psksecret password2

set dpd-retryinterval 3

next

edit “ADVPN3”

set type dynamic

set interface “port2”

set peertype any

set net-device disable

set proposal des-md5 des-sha1

set add-route disable

set dpd on-idle

set auto-discovery-sender enable

set tunnel-search nexthop

set psksecret password3

set dpd-retryinterval 3

next

edit “ADVPN4”

set type dynamic

set interface “port2”

set peertype any

set net-device disable

set proposal des-md5 des-sha1

set add-route disable

set dpd on-idle

set auto-discovery-sender enable

set tunnel-search nexthop

set psksecret password4

set dpd-retryinterval 3

next

end

Phase 2 IPsec Configuration

config vpn ipsec phase2-interface

edit “ADVPN”

set phase1name “ADVPN”

set proposal des-md5 des-sha1

next

edit “ADVPN2”

set phase1name “ADVPN2”

set proposal des-md5 des-sha1

next

edit “ADVPN3”

set phase1name “ADVPN3”

set proposal des-md5 des-sha1

next

edit “ADVPN4”

set phase1name “ADVPN4”

set proposal des-md5 des-sha1

next

end

Creating the ADVPN interface on the Hub is very critical and requires your attention with it as this mostly get missed including myself when I first building the labs. Make sure you associate the correct WAN (underlay) port to the correct ADVPN IP (overlay). If this gets missed you’ll get results were you’ll have multiple dialed interfaces to a single WAN link. You are after 1:1 connection between a hub and spoke.

ADVPN Interface Configuration

config system interface

edit “ADVPN”

set vdom “root”

set ip 10.10.3.1 255.255.255.255

set type tunnel

set remote-ip 10.10.3.2 255.255.255.0

set interface “port1”

next

edit “ADVPN2”

set vdom “root”

set ip 10.10.4.1 255.255.255.255

set type tunnel

set remote-ip 10.10.4.2 255.255.255.0

set interface “port1”

next

edit “ADVPN3”

set vdom “root”

set ip 10.10.5.1 255.255.255.255

set type tunnel

set remote-ip 10.10.5.2 255.255.255.0

set interface “port2”

next

edit “ADVPN4”

set vdom “root”

set ip 10.10.6.1 255.255.255.255

set type tunnel

set remote-ip 10.10.6.2 255.255.255.0

set interface “port2”

next

end

With the BGP configuration, if you were using the wizard with FortiOS 6.4 you’ll get individual neighbor line items for every spoke tunnel which can be very difficult to manage once you have a huge topology. I’ve done a neighbor-group command to group the command command per neighbor so moving forward I can just add a spoke together with the neighbor-group.

ADVPN BGP Configuration

config router bgp

set as 65400

config neighbor-group

edit “advpn”

set capability-default-originate enable

set link-down-failover enable

set remote-as 65400

set additional-path both

set adv-additional-path 4

set route-reflector-client enable

next

end

config neighbor-range

edit 1

set prefix 10.10.3.0 255.255.255.0

set neighbor-group “advpn”

next

edit 2

set prefix 10.10.4.0 255.255.255.0

set neighbor-group “advpn”

next

edit 3

set prefix 10.10.5.0 255.255.255.0

set neighbor-group “advpn”

next

edit 4

set prefix 10.10.6.0 255.255.255.0

set neighbor-group “advpn”

next

end

config network

edit 1

set prefix 10.10.1.0 255.255.255.0

next

end

Firewall Policy

Creating policy per tunnel can be tedious as well and would suggest creating a zone to group the tunnels. This is when I started using the GUI. Network > Interface > Create New > Zone. I named mine ADVPNZONES and added all the ADVPN tunnels I’ve created.

Then I created 3 policies, LAN to ADVPN tunnels,

Spoke Configuration

We will basically do the same thing for each spoke. I will only post one spoke as the configuration is the same for the next one once you have done one. The only thing that will change would be the tunnel overlay IP for the spoke.

For Phase 1, it is pretty much just like the Hub to make sure values match. Be mindful as to which port you are using to have remote gateway IP you are peering with. Once you get into a single DC with multiple WAN links this can get confusing.

NOTE: As per Fortinet best practice, ideally you should be peering with a proper data centre Hub public facing IP rather than a Head-Office as data centre WAN links have stability.

Phase 1 IPsec Configuration

config vpn ipsec phase1-interface

edit “ADVPN-SPOKE”

set interface “port1”

set peertype any

set net-device enable

set proposal des-md5 des-sha1

set add-route disable

set dpd on-idle

set auto-discovery-receiver enable

set remote-gw 192.168.1.2

set psksecret password1

next

edit “WAN2_H1”

set interface “port2”

set peertype any

set net-device enable

set proposal des-md5 des-sha1

set add-route disable

set dpd on-idle

set auto-discovery-receiver enable

set remote-gw 192.168.1.2

set psksecret password2

set dpd-retryinterval 3

next

edit “ADVPN3”

set interface “port1”

set peertype any

set net-device enable

set proposal des-md5 des-sha1

set add-route disable

set dpd on-idle

set auto-discovery-receiver enable

set remote-gw 192.168.3.2

set psksecret password3

set dpd-retryinterval 3

next

edit “WAN2_H2”

set interface “port2”

set peertype any

set net-device enable

set proposal des-md5 des-sha1

set add-route disable

set dpd on-idle

set auto-discovery-receiver enable

set remote-gw 192.168.3.2

set psksecret password4

next

end

Phase 2 IPsec Configuration

Note: Using the IPSec wizard in FortiOS 6.4 does not enable auto-negotiate by default so this should be enabled manually via the CLI.

config vpn ipsec phase2-interface

edit “ADVPN-SPOKE”

set phase1name “ADVPN-SPOKE”

set proposal des-md5 des-sha1

set auto-negotiate enable

next

edit “WAN2_H1”

set phase1name “WAN2_H1”

set proposal des-md5 des-sha1

set auto-negotiate enable

next

edit “ADVPN3”

set phase1name “ADVPN3”

set proposal des-md5 des-sha1

set auto-negotiate enable

next

edit “WAN2_H2”

set phase1name “WAN2_H2”

set proposal des-md5 des-sha1

set auto-negotiate enable

next

end

ADVPN Interface Configuration

config system interface

edit “ADVPN-SPOKE”

set vdom “root”

set ip 10.10.3.3 255.255.255.255

set type tunnel

set remote-ip 10.10.3.1 255.255.255.0

set interface “port1”

next

edit “WAN2_H1”

set vdom “root”

set ip 10.10.4.3 255.255.255.255

set type tunnel

set remote-ip 10.10.4.1 255.255.255.0

set interface “port2”

next

edit “ADVPN3”

set vdom “root”

set ip 10.10.5.3 255.255.255.255

set type tunnel

set remote-ip 10.10.5.1 255.255.255.0

set interface “port1”

next

edit “WAN2_H2”

set vdom “root”

set ip 10.10.6.3 255.255.255.255

set type tunnel

set remote-ip 10.10.6.1 255.255.255.0

set interface “port2”

next

end

ADVPN BGP Configuration

Note: I have enabled the multi-path feature on both ends, I didn’t mention it in the Hub configuration because there’s more details on the spoke. In this configuration I have allowed for routes to be learnt from as much as 4 paths if available.

config router bgp

set as 65400

set ibgp-multipath enable

set additional-path enable

set additional-path-select 4

config neighbor

edit “10.11.3.1”

set remote-as 65400

next

edit “10.10.3.1”

set advertisement-interval 1

set link-down-failover enable

set soft-reconfiguration enable

set remote-as 65400

set additional-path both

set adv-additional-path 4

next

edit “10.10.4.1”

set advertisement-interval 1

set link-down-failover enable

set soft-reconfiguration enable

set remote-as 65400

set additional-path both

set adv-additional-path 4

next

edit “10.10.5.1”

set advertisement-interval 1

set link-down-failover enable

set soft-reconfiguration enable

set remote-as 65400

set additional-path both

set adv-additional-path 4

next

edit “10.10.6.1”

set advertisement-interval 1

set link-down-failover enable

set soft-reconfiguration enable

set remote-as 65400

set additional-path both

set adv-additional-path 4

next

end

config network

edit 1

set prefix 10.10.2.0 255.255.255.0

next

end

end

Firewall Policies

Same deal as the Hub, I have created a zone to get all the tunnels in it and created the policies for it. I don’t really need the INTERADVPN policy at the bottom of that screenshot.

ADVPN Tunnels

Once everything is set-up, you should be able to see all four tunnels up.

From the Hub:

From Spoke1:

BGP routing table from spoke1, Both its interfaces from its primary and secondary WAN link at the moment is learning the Hub’s LAN network from the Hubs primary link.

Getting Spoke2 up and running, we can see more BGP routes pop-up!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s