The IPsec Architecture: SA, SPD, Transport and Tunnel Mode

RFC 4301 also lists a few flags for fragments and for the DSCP field, which carry no marks at this level.

The Security Policy Database

Policy comes first. Before any packet is protected, somebody has to decide which traffic needs protection, which may go out as it is, and which must never leave. That decision is the SPD, and RFC 4301 requires it to be consulted for all traffic crossing the IPsec boundary, protected or not.

Every entry ends in one of three actions:

• PROTECT: apply IPsec, using the SA the entry names or causes to be created.
• BYPASS: let the packet cross without IPsec.
• DISCARD: drop it.

Every entry begins with selectors, the values it matches in a packet's headers. RFC 4301 s.4.4.1.1 requires these:

The SPD is ordered. RFC 4301 says it plainly: the SPD "is an ordered database, consistent with the use of Access Control Lists (ACLs) or packet filters in firewalls". Entries overlap, so the first entry that matches decides. The run below shows what a wrong order does.

The SAD and the PAD

The SAD is the table of live SAs, one row per SA, with the parameters listed above. For outgoing traffic, the SPD entry that says PROTECT points to the SA; for incoming traffic, the SPI in the packet finds the SA directly.

RFC 4301 adds a third database, the Peer Authorisation Database (PAD), which records which peers may negotiate SAs and for which traffic. It links the key management protocol to the SPD, and is the reason a peer that authenticates successfully still cannot ask for protection of traffic it has no right to.

How a packet is processed

Outbound, from the protected side:

1. Match the packet against the SPD, in order, and take the first matching entry.
2. If the action is DISCARD, drop it; if BYPASS, send it unprotected.
3. If PROTECT, find the SA for this traffic in the SAD; if none exists yet, ask key management to create a pair.
4. Apply AH or ESP in the SA's mode, add the next sequence number, and send.

Inbound, from the unprotected side:

1. If the packet carries AH or ESP, use its SPI to find the SA; if there is none, discard it and log the event.
2. Check the sequence number against the anti-replay window, verify the integrity check, decrypt.
3. Check that the packet that comes out matches the SA's selectors. A key alone does not entitle a sender to deliver any traffic it likes through the SA.
4. If the packet was not protected, look it up in the SPD: it is accepted only if policy says BYPASS.

Transport mode and tunnel mode

Figure 72.1 Transport mode inserts the IPsec header into the packet; tunnel mode puts the whole packet inside a new one.

Transport mode protects what the IP packet carries. The IPsec header goes immediately after the original IP header and before the TCP or UDP header. The original addresses stay as they are, and the protection applies to the upper-layer protocol and its data. RFC 4301 describes it as "typically employed between a pair of hosts to provide end-to-end security services".

Tunnel mode protects the entire packet. The whole original packet, its header included, becomes the payload of a new IP packet with a new outer header. The outer header carries the addresses of the two IPsec endpoints, usually gateways; the inner header carries the real source and destination. RFC 4301: "whenever either end of a security association is a security gateway, the SA MUST be tunnel mode", apart from traffic addressed to the gateway itself.

What each protects, drawn from RFC 4301 s.4.1:

Why tunnel mode for gateways. A gateway protects traffic for many hosts behind it. The packet must reach the gateway that holds the SA, not the host behind it, so the outer header has to name the gateway. And the inner header, hidden inside, keeps an observer on the Internet from learning which host behind the gateway is talking to which: the traffic flow confidentiality of the previous chapter's list.

The run: a gateway's databases at work

The listing sets up the gateway of campus A, protecting 10.1.0.0/16 and talking to campus B's gateway at 203.0.113.2, with an SPD of five ordered entries. It sends seven packets through it, prints the pair of SAs the first protected packet caused, runs the same traffic under a wrongly ordered SPD, handles four incoming packets, and prices one 60-byte packet in each mode using AES-GCM's real overheads.

# A security gateway's two databases at work: the SPD decides, the SAD protects.
import ipaddress

def net(text):
    return ipaddress.ip_network(text)

ANY = None

class Rule:                               # one SPD entry: selectors, then an action
    def __init__(self, name, local, remote, proto, port, action, tunnel_to=None):
        self.name, self.local, self.remote = name, local, remote
        self.proto, self.port, self.action, self.tunnel_to = proto, port, action, tunnel_to

    def matches(self, p, inbound=False):  # local and remote swap roles for inbound packets
        mine, theirs = (p['dst'], p['src']) if inbound else (p['src'], p['dst'])
        return ((self.local is ANY or ipaddress.ip_address(mine) in net(self.local)) and
                (self.remote is ANY or ipaddress.ip_address(theirs) in net(self.remote)) and
                (self.proto is ANY or p['proto'] == self.proto) and
                (self.port is ANY or p['dport'] == self.port))

SPD = [Rule('campus A to campus B', '10.1.0.0/16', '10.2.0.0/16', ANY, ANY,
            'PROTECT', tunnel_to='203.0.113.2'),
       Rule('DNS', ANY, ANY, 'udp', 53, 'BYPASS'),
       Rule('no Telnet anywhere', ANY, ANY, 'tcp', 23, 'DISCARD'),
       Rule('HTTPS, already under TLS', ANY, ANY, 'tcp', 443, 'BYPASS'),
       Rule('everything else', ANY, ANY, ANY, ANY, 'DISCARD')]

SAD = {}                                   # SPI -> security association

def make_pair(rule):                       # IKE makes SAs in pairs, one each way
    for spi, direction, ends in ((0x2A01, 'outbound', ('198.51.100.1', rule.tunnel_to)),
                                 (0x5B01, 'inbound', (rule.tunnel_to, '198.51.100.1'))):
        SAD[spi] = {'SPI': spi, 'direction': direction, 'seq': 0, 'mode': 'tunnel',
                    'tunnel': '%s to %s' % ends, 'ESP': 'AES-GCM-16, 256-bit key',
                    'window': 64, 'lifetime': 'soft 3,000 s, hard 3,600 s',
                    'selectors': rule}

def outbound(p, spd):                      # RFC 4301 s.5.1: first matching entry decides
    rule = next(r for r in spd if r.matches(p))
    if rule.action != 'PROTECT':
        return rule, rule.action, None
    sa = next((s for s in SAD.values() if s['selectors'] is rule
               and s['direction'] == 'outbound'), None)
    if sa is None:                         # no SA yet: key management would make them
        make_pair(rule)
        sa = SAD[0x2A01]
    sa['seq'] += 1
    return rule, 'PROTECT', sa

def show(p, spd=SPD):
    rule, action, sa = outbound(p, spd)
    tail = ' SPI 0x%X, sequence %d' % (sa['SPI'], sa['seq']) if sa else ''
    print('  %-9s -> %-10s %-3s %-4s | "%s": %s%s' % (
        p['src'], p['dst'], p['proto'], p['dport'], rule.name, action, tail))

print('OUTBOUND, THROUGH THE GATEWAY OF CAMPUS A')
traffic = [dict(src='10.1.4.7', dst='10.2.8.1', proto='tcp', dport=80),
           dict(src='10.1.4.7', dst='10.2.8.1', proto='udp', dport=514),
           dict(src='10.1.4.7', dst='192.0.2.53', proto='udp', dport=53),
           dict(src='10.1.4.7', dst='192.0.2.80', proto='tcp', dport=443),
           dict(src='10.1.4.7', dst='192.0.2.80', proto='tcp', dport=23),
           dict(src='10.1.4.7', dst='192.0.2.80', proto='tcp', dport=8080),
           dict(src='10.1.9.2', dst='10.2.0.40', proto='tcp', dport=22)]
for p in traffic:
    show(p)

print()
print('THE PAIR OF SECURITY ASSOCIATIONS THE FIRST PACKET CAUSED, AS THE SAD HOLDS THEM')
for sa in SAD.values():
    print('  SPI 0x%X, %s' % (sa['SPI'], sa['direction']))
    for field in ('seq', 'mode', 'tunnel', 'ESP', 'window', 'lifetime'):
        print('    %-9s %s' % (field, sa[field]))

print()
print('THE SAME TRAFFIC, WITH TWO RULES IN THE WRONG ORDER')
wrong = [SPD[0], SPD[1], Rule('any TCP to the Internet', ANY, ANY, 'tcp', ANY, 'BYPASS'),
         SPD[2], SPD[4]]
for p in traffic[4:6]:
    show(p, wrong)

# ---- inbound: the SPI finds the SA; then the inner packet must fit the SA's selectors --
def inbound(spi, inner):
    sa = SAD.get(spi)
    if sa is None or sa['direction'] != 'inbound':
        return 'discarded and logged: no inbound SA with SPI 0x%X' % spi
    if not sa['selectors'].matches(inner, inbound=True):
        return 'discarded: %s is not a source SA 0x%X carries' % (inner['src'], spi)
    return 'accepted on SA 0x%X, delivered to %s' % (spi, inner['dst'])

print()
print('INBOUND, FROM CAMPUS B\'S GATEWAY 203.0.113.2')
reply = dict(src='10.2.8.1', dst='10.1.4.7', proto='tcp', dport=80)
spoof = dict(src='10.9.9.9', dst='10.1.4.7', proto='tcp', dport=80)
print('  SPI 0x5B01, inner 10.2.8.1 -> 10.1.4.7 :', inbound(0x5B01, reply))
print('  SPI 0x2A01, the same inner packet      :', inbound(0x2A01, reply))
print('  SPI 0x7777, the same inner packet      :', inbound(0x7777, reply))
print('  SPI 0x5B01, inner 10.9.9.9 -> 10.1.4.7 :', inbound(0x5B01, spoof))

# ---- what each mode costs one 60-byte packet (20 IP + 20 TCP + 20 data) ------------------
def esp_bytes(inner):                      # SPI 4, sequence 4, IV 8, trailer 2 + pad, ICV 16
    pad = (-(inner + 2)) % 4
    return 4 + 4 + 8 + inner + pad + 2 + 16

print()
print('ONE 60-BYTE PACKET, AES-GCM (8-BYTE IV, 16-BYTE ICV)')
print('  transport mode: 20 IP + ESP around the 40 bytes after it = %d bytes'
      % (20 + esp_bytes(40)))
print('  tunnel mode   : 20 new IP + ESP around all 60 bytes      = %d bytes'
      % (20 + esp_bytes(60)))

What the run establishes, in order.

Every packet meets exactly one decision, the first entry that matches. Traffic from campus A to campus B, of any protocol, is PROTECTED on SPI 0x2A01, with sequence numbers 1, 2 and 3 as the packets go. DNS and HTTPS are BYPASSED, the second because TLS already protects it. Telnet is DISCARDED by name, and port 8080 by the final catch-all entry.

The first protected packet caused a pair of SAs. SPI 0x2A01 carries traffic out from 198.51.100.1 to 203.0.113.2; SPI 0x5B01 carries traffic in, the other way. Each records its mode, its tunnel endpoints, its algorithm, a 64-packet replay window and a soft and hard lifetime; the outbound one has sent three packets.

Order is policy. Put a broad "any TCP may leave" entry above the Telnet entry and both Telnet and port 8080 go out unprotected. No rule was deleted; one was moved.

Inbound, the SPI decides, and then the selectors. A reply from 10.2.8.1 on the inbound SA 0x5B01 is accepted. The same packet on 0x2A01 is refused, because that SA runs the other way. An unknown SPI is discarded and logged. And a packet that decrypts correctly on 0x5B01 but claims to come from 10.9.9.9 is discarded, because that SA carries traffic only from campus B's addresses.

Tunnel mode costs a header more. The same 60-byte packet becomes 96 bytes in transport mode and 116 in tunnel mode: the difference is the new 20-byte outer header, the price of hiding the real addresses.

Distinctions that carry marks

What beginners get wrong here

Saying an SA is two-way. It is simplex. A two-way conversation needs two SAs, and the run's reply is refused on the outbound SPI.

Saying the SPI is chosen by the sender. The receiver chooses it, so that it can find the SA from it.

Thinking the SPD only lists protected traffic. It governs all traffic crossing the boundary, and BYPASS and DISCARD are as much its business as PROTECT.

Forgetting that SPD order matters. The first match decides, exactly as in a firewall.

Saying tunnel mode encrypts and transport mode authenticates. Mode says what is protected, and AH or ESP says how. Either protocol can be used in either mode.

Thinking the outer header in tunnel mode is protected by ESP. ESP protects the inner packet only; the outer header is covered by neither encryption nor ESP's integrity check.

Quick revision

• SA (RFC 4301): simplex; AH or ESP; a pair for two-way traffic, made by IKE; identified by SPI, destination address and protocol; SPI chosen by the receiver.
• SA parameters: SPI, sequence number counter, counter overflow, anti-replay window, AH information, ESP information, lifetime (soft and hard, time or bytes), mode, path MTU, tunnel endpoints.
• SPD: ordered; selectors (remote and local addresses, next-layer protocol, ports, name); actions PROTECT, BYPASS, DISCARD; consulted for all traffic.
• SAD: the live SAs; outbound found from the SPD, inbound found by SPI. PAD: which peers may negotiate what.
• Transport mode: IPsec header after the IP header; protects the payload; host to host.
• Tunnel mode: new outer header; protects the whole inner packet; required when either end is a gateway; hides the real addresses; costs a header more.

Test yourself

1. Define a security association and state how it is identified. A security association is a one-way relationship between a sender and a receiver that affords security services, by AH or by ESP but not both, to the traffic it carries. It is identified by the Security Parameters Index, a 32-bit value chosen by the receiver and carried in every AH or ESP header, together with the destination IP address and the security protocol identifier; for ordinary unicast traffic the SPI alone suffices.

2. Why do two hosts that talk both ways need two SAs? Because an SA is simplex: it carries traffic in one direction only, with its own SPI, keys and sequence numbers. One SA protects traffic from the first host to the second and another protects traffic back, which is why IKE creates SAs in pairs.

3. List the parameters an SA holds. Its Security Parameters Index; a sequence number counter and a flag saying what happens when it overflows; an anti-replay window; the AH algorithm and keys, or the ESP encryption and integrity algorithms, modes and keys; its lifetime, as time, byte count or both, with soft and hard limits; its mode, transport or tunnel; the path MTU; and, in tunnel mode, the addresses of the tunnel endpoints.

4. What is the SPD, what are its three actions, and why is its order important? The Security Policy Database is the ordered list of policy entries consulted for every packet crossing the IPsec boundary. Each entry has selectors, such as addresses, protocol and ports, and an action: protect the packet with IPsec, bypass IPsec, or discard it. Because entries overlap, the first entry that matches decides, so a broad entry placed above a narrower one can silently override it, exactly as in a firewall's rule list.

5. Distinguish transport mode from tunnel mode. In transport mode the AH or ESP header is placed after the original IP header and protects the upper-layer data; the original addresses remain visible, and it is typically used end to end between hosts. In tunnel mode the entire original packet is carried inside a new IP packet whose header names the IPsec endpoints; the whole inner packet is protected and the real addresses are hidden, at the cost of an extra header, and it must be used whenever either end of the SA is a security gateway.

6. A packet arrives with a valid SPI and decrypts correctly, but its inner source address lies outside the SA's selectors. What happens, and why? It is discarded. After processing, the receiver checks that the packet matches the selectors of the SA it arrived on; possessing the key does not entitle a sender to inject traffic the SA was not set up to carry, so a peer cannot use an SA for a network it was never authorised for.