IP Security: What It Is For

1. Host to host: two computers protect the traffic between themselves, end to end.
2. Gateway to gateway: two security gateways, routers or firewalls running IPsec, protect all traffic between two networks, and the hosts behind them need nothing at all.
3. Host to gateway: a single computer outside connects securely to a gateway, and through it to the network behind.

What it is used for

Each use is one of the three paths put to work.

• Joining branch offices over the Internet. A college with two campuses puts a security gateway at each; everything between the campuses travels encrypted across the public Internet as if on a private line. This is a virtual private network (VPN), gateway to gateway.
• Remote access. A staff member at home connects to the campus gateway, host to gateway, and works as though inside the campus network.
• Connecting with partner organisations. Two organisations join selected parts of their networks, with policy deciding exactly which traffic may pass.
• Protecting the network's own traffic. RFC 4301 gives the example of a router using IPsec to protect its routing protocol and its management traffic, the traffic that, if forged, lets an attacker redirect everyone else's.

The benefits, and why each follows

Transparent to applications. IPsec sits below the transport layer, so nothing in an application changes. A college need not wait for every program to add security.

Transparent to users. No user action is needed: no keys to manage, no settings to choose. The protection happens whether or not the user knows it exists.

One point of control at a gateway. Implemented in the firewall or router at the edge of a network, IPsec protects all traffic crossing the edge, and policy is set in one place rather than on every machine.

Protection for protocols that have none. Many protocols carried over IP were designed with no security. IPsec gives them integrity and confidentiality without redesigning them.

Where IPsec runs

RFC 4301 s.3.3 describes three ways to implement it:

The documents

IPsec is a suite of standards, not one document, and a question on the IPsec overview often asks for them.

Keeping the algorithms in separate documents is deliberate. RFC 4301 explains that the algorithm lists "will be periodically updated to keep pace with computational and cryptologic advances" without touching the protocols themselves.

Two things textbooks still get wrong

AH is optional. RFC 4301 requires implementations to support ESP and only permits them to support AH, "because experience has shown that there are very few contexts in which ESP cannot provide the requisite security services". ESP can provide integrity without encryption, which is what AH does. The AH chapter explains why ESP won.

IPsec is not mandatory in IPv6. Many notes say IPv6 requires IPsec. RFC 8504, the IPv6 Node Requirements of January 2019, says: "Previously, IPv6 mandated implementation of IPsec". RFC 6434 changed that to a recommendation, a SHOULD, and RFC 8504 keeps the SHOULD, noting that for some devices, such as constrained sensors, "the full IPsec Architecture is not justified".

The run: two applications that never learn IPsec is there

The listing builds real IPv4 headers, with the header checksum computed the way RFC 1071 describes and first checked against that RFC's worked example. Two applications run on a pair of hosts: a chat program sending a line over UDP, and a file transfer over TCP. The applications are run twice, once with IPsec off and once with an ESP security association in transport mode, and not one line of either application changes. Then an attacker on the path tries three things. The cipher is a stand-in; the integrity check is HMAC with SHA-256, truncated to 128 bits, as RFC 8221 requires of ESP.

# IPsec's whole point, run: two applications protected without a line of them changing.
import hashlib, hmac, random

rng = random.Random(710)

def ones_complement_sum(data):             # RFC 1071, the Internet checksum's core
    if len(data) % 2:
        data += b'\x00'
    total = sum(int.from_bytes(data[i:i + 2], 'big') for i in range(0, len(data), 2))
    while total >> 16:
        total = (total & 0xFFFF) + (total >> 16)
    return total

print('RFC 1071 s.3 example, sum of 0001 f203 f4f5 f6f7:',
      '%04x' % ones_complement_sum(bytes.fromhex('0001f203f4f5f6f7')))

def ipv4(src, dst, protocol, payload):     # a real RFC 791 header, checksum and all
    header = bytearray(20)
    header[0], header[8], header[9] = 0x45, 64, protocol
    header[2:4] = (20 + len(payload)).to_bytes(2, 'big')
    header[12:16] = bytes(map(int, src.split('.')))
    header[16:20] = bytes(map(int, dst.split('.')))
    header[10:12] = (~ones_complement_sum(bytes(header)) & 0xFFFF).to_bytes(2, 'big')
    return bytes(header) + payload

UDP, TCP, ESP = 17, 6, 50                  # IANA protocol numbers

# ---- an ESP security association, transport mode (stand-in cipher; HMAC-SHA-256-128) ---
class SA:
    def __init__(self, spi, enc_key, auth_key):
        self.spi, self.enc, self.auth = spi, enc_key, auth_key
        self.sent, self.highest = 0, 0

    def stream(self, iv, n):
        return b''.join(hashlib.sha256(self.enc + iv + i.to_bytes(4, 'big')).digest()
                        for i in range(n // 32 + 1))[:n]

def esp_protect(sa, protocol, payload):
    sa.sent += 1
    head = sa.spi.to_bytes(4, 'big') + sa.sent.to_bytes(4, 'big')
    iv = rng.randbytes(16)
    pad = (-(len(payload) + 2)) % 4                   # RFC 4303 s.2.4: end on a 4-byte boundary
    plain = payload + bytes(range(1, pad + 1)) + bytes([pad, protocol])
    body = head + iv + bytes(a ^ b for a, b in zip(plain, sa.stream(iv, len(plain))))
    return body + hmac.new(sa.auth, body, hashlib.sha256).digest()[:16]

def esp_accept(sa, data):
    body, icv = data[:-16], data[-16:]
    if not hmac.compare_digest(icv, hmac.new(sa.auth, body, hashlib.sha256).digest()[:16]):
        return None, 'dropped: integrity check failed'
    seq = int.from_bytes(body[4:8], 'big')
    if seq <= sa.highest:
        return None, 'dropped: sequence number %d already seen' % seq
    sa.highest = seq
    iv, ct = body[8:24], body[24:]
    plain = bytes(a ^ b for a, b in zip(ct, sa.stream(iv, len(ct))))
    pad, protocol = plain[-2], plain[-1]
    return (protocol, plain[:-2 - pad]), 'accepted'

# ---- the network, a host stack, and two applications that know nothing of IPsec --------
wire = []

class Host:
    def __init__(self, addr, sa=None):
        self.addr, self.sa, self.inbox = addr, sa, []

    def send(self, dst, protocol, payload):          # the IP layer
        if self.sa:
            packet = ipv4(self.addr, dst.addr, ESP, esp_protect(self.sa, protocol, payload))
        else:
            packet = ipv4(self.addr, dst.addr, protocol, payload)
        wire.append(packet)
        return dst.receive(packet)

    def receive(self, packet):
        protocol, payload = packet[9], packet[20:]
        if protocol == ESP:
            result, verdict = esp_accept(self.sa, payload)
            if result is None:
                return verdict
            protocol, payload = result
        self.inbox.append((protocol, payload))
        return 'delivered'

def chat_app(host, peer, text):                      # "UDP": a line of chat
    return host.send(peer, UDP, text.encode())

def file_app(host, peer, name, content):              # "TCP": a file transfer
    return host.send(peer, TCP, name.encode() + b'\x00' + content)

def run(with_ipsec):
    global wire
    wire = []
    if with_ipsec:
        keys = rng.randbytes(16), rng.randbytes(16)
        a, b = Host('10.1.0.5', SA(0x1001, *keys)), Host('10.2.0.9', SA(0x1001, *keys))
    else:
        a, b = Host('10.1.0.5'), Host('10.2.0.9')
    chat_app(a, b, 'Practical exam moved to Thursday')
    file_app(a, b, 'marks.csv', b'roll,marks\n101,17\n102,19\n')
    return a, b

for label, on in (('WITHOUT IPsec', False), ('WITH IPsec (ESP, transport mode)', True)):
    a, b = run(on)
    print()
    print(label)
    for protocol, payload in b.inbox:
        print('  application received, protocol %2d: %r' % (protocol, payload[:40]))
    for packet in wire:
        print('  on the wire: protocol field %2d, %3d bytes, text visible: %s'
              % (packet[9], len(packet), b'Thursday' in packet or b'roll,marks' in packet))

print()
print('AN ATTACKER ON THE PATH, AGAINST THE PROTECTED HOSTS')
recorded = wire[0]
print('  replays a captured packet      :', b.receive(recorded))
forged = bytearray(recorded)
forged[-20] ^= 0x01                                   # flip one bit of the ciphertext
print('  alters one bit of a packet     :', b.receive(bytes(forged)))
fake = ipv4('10.1.0.5', '10.2.0.9', ESP, (0x1001).to_bytes(4, 'big') + (99).to_bytes(4, 'big')
            + rng.randbytes(60))
print('  forges a packet from 10.1.0.5  :', b.receive(fake))
print('  headers still valid IPv4 (checksum sums to ffff):',
      all(ones_complement_sum(p[:20]) == 0xFFFF for p in wire))

What the run establishes, in order.

The checksum routine is the Internet's. It reproduces the sum ddf2 that RFC 1071 works by hand.

The applications cannot tell the difference. With IPsec off and on, the receiving side delivers exactly the same chat line and exactly the same file to the applications. Their code is identical in both runs.

The wire can. Without IPsec the packets carry protocol numbers 17 and 6, UDP and TCP, and the text is readable to anyone on the path. With IPsec every packet carries protocol 50, ESP, is 44 or 45 bytes larger, and no text is visible.

The three attacks all fail. A captured packet sent again is refused because its sequence number has been seen. A packet with one bit changed fails the integrity check. A packet forged to look as if it came from the protected host fails the integrity check too, because the forger has no key. And every header remains valid IPv4, so routers along the way carry the packets exactly as before.

Distinctions that carry marks

What beginners get wrong here

Saying IPsec only encrypts. Its services include access control, integrity, origin authentication and replay rejection; encryption is one of six.

Saying applications must be rewritten. The whole point of IPsec is that they need not be.

Saying IPsec is mandatory in IPv6. It was once; since RFC 6434, and in RFC 8504, it is a recommendation.

Saying IPsec replaces TLS. RFC 8504 says plainly that IPsec "is not viewed as the ideal security technology in all cases and is unlikely to displace the others".

Treating AH as required. ESP is required; AH is optional.

Quick revision

• IPsec: security at the IP layer, for IPv4 and IPv6; transparent to applications and users.
• Services (RFC 4301): access control, connectionless integrity, data origin authentication, rejection of replays, confidentiality, limited traffic flow confidentiality.
• Boundary: every packet is protected, bypassed or discarded, by policy.
• Paths: host to host, gateway to gateway (VPN), host to gateway (remote access).
• Uses: branch offices, remote access, partners, protecting routing and management traffic.
• Implementations: native, bump-in-the-stack, bump-in-the-wire.
• Documents: RFC 4301 architecture, 4302 AH, 4303 ESP, 7296 IKEv2, 8221 and 8247 algorithms, 6071 roadmap.
• Currency: ESP MUST, AH MAY; IPsec is a SHOULD, not a MUST, for IPv6 (RFC 8504, 2019).

Test yourself

1. What is IPsec, and what services does it provide? IPsec is the IETF's framework of standards for security at the IP layer, for IPv4 and IPv6, specified by RFC 4301 and its companion documents. It provides access control, connectionless integrity, data origin authentication, rejection of replayed packets, confidentiality, and limited traffic flow confidentiality.

2. Why is IPsec transparent to applications, and why does that matter? Because it operates below the transport layer, on the IP packets themselves, so applications hand data to TCP or UDP exactly as before and never see IPsec. It matters because every application on a machine, including old ones and ones with no security of their own, is protected at once, without being rewritten.

3. Describe three uses of IPsec. Joining the networks of two offices over the Internet through security gateways at each, a virtual private network; giving an individual remote access to an organisation's network from a host to its gateway; and protecting a router's own routing and management traffic, so that an attacker cannot forge routing updates. Connecting selected parts of partner organisations' networks is a fourth.

4. What are the three ways IPsec can be implemented? Natively, integrated into the operating system's IP stack; as a bump-in-the-stack, inserted between an existing IP stack and the network drivers, which suits systems whose stack cannot be changed; and as a bump-in-the-wire, a separate inline device, which may serve a single host or act as a gateway.

5. Is IPsec required in every IPv6 implementation? Not any more. IPv6 originally mandated IPsec, but RFC 6434 changed this to a recommendation, and RFC 8504, the current IPv6 Node Requirements of January 2019, keeps it as a SHOULD, recognising that for some constrained devices the full IPsec architecture is not justified. Where IPsec is implemented, ESP is required and AH is optional.

6. How does IPsec's position differ from that of TLS and of S/MIME? IPsec works at the network layer and protects all IP traffic between two points without any application changing. TLS works at the transport layer and protects one connection, and each application must use it. S/MIME works at the application layer and protects one email message inside the mail program.