IPsec Key Management: Oakley, ISAKMP and IKE

It also uses nonces, random values each party contributes, so that every exchange is fresh and a replayed old message fits no current exchange.

Authentication methods. Oakley's list is pre-shared keys, which it makes REQUIRED; public keys published in the DNS; RSA keys without a certification authority, as in PGP; and RSA or DSS keys with certificates. Textbooks group these as three: digital signatures, public-key encryption, and symmetric-key encryption with a pre-shared key.

Cookies

A cookie is how a responder makes an initiator prove it can receive replies at the address it claims, before the responder does any expensive work.

The responder answers a first request not with a Diffie-Hellman value but with a cookie, a short value computed from the request. The initiator must send its request again, including the cookie. A forger using someone else's address never receives the cookie, because it goes to the address it forged, so the responder never does the work for it.

ISAKMP, RFC 2408 s.2.5.3, sets three requirements, originally Phil Karn's:

1. The cookie must depend on the specific parties. Otherwise an attacker could obtain one real cookie and then use it with requests from any address.
2. Only the issuer can make cookies it will accept. So it must depend on a local secret, which must not be deducible from any cookie.
3. It must be fast to make. Otherwise the cookie itself becomes the target of the flood.

IKEv2 suggests exactly such a construction: the cookie is a version number of the secret followed by a hash of the initiator's nonce, its IP address, its SPI and the responder's secret.

ISAKMP

The Internet Security Association and Key Management Protocol, RFC 2408, is not a key exchange. It is a framework: the formats and procedures for establishing, negotiating, modifying and deleting security associations, whatever key exchange is used inside it.

Its header, fixed for every message:

Its payloads: security association, proposal, transform, key exchange, identification, certificate, certificate request, hash, signature, nonce, notification, delete and vendor ID.

Its exchange types: base, identity protection, authentication only, aggressive, and informational.

Two phases. First the two parties set up an ISAKMP SA to protect their own negotiations; then, under its protection, they negotiate SAs for AH or ESP. The expensive, authenticated work is done once, and each IPsec SA after it is cheap.

IKEv1

IKEv1, RFC 2409, is Oakley's exchange carried in ISAKMP's framework.

Phase 1, which authenticates the parties and builds the ISAKMP SA:

• Main Mode, six messages, an instance of ISAKMP's identity protection exchange. The first two negotiate policy, the next two exchange Diffie-Hellman values and nonces, and the last two authenticate, encrypted, so the identities are hidden.

Main Mode (signatures)                  Aggressive Mode (signatures)

HDR, SA                  -->            HDR, SA, KE, Ni, IDii         -->
             <--  HDR, SA                            <--  HDR, SA, KE, Nr, IDir,
HDR, KE, Ni              -->                                  [CERT,] SIG_R
             <--  HDR, KE, Nr           HDR, [CERT,] SIG_I            -->
HDR*, IDii, [CERT,] SIG_I -->
             <--  HDR*, IDir, [CERT,] SIG_R        (HDR* means the payloads are encrypted)

• Aggressive Mode, three messages. Faster, but the identities travel unprotected.

Phase 2: Quick Mode, three messages under the phase 1 SA's protection, negotiates the SAs for AH or ESP: HDR, HASH(1), SA, Ni then HDR, HASH(2), SA, Nr then HDR*, HASH(3), with optional fresh Diffie-Hellman values for perfect forward secrecy.

IKEv2, the protocol in use

IKEv2 was published as RFC 4306 in December 2005 and is now RFC 7296 (October 2014). It keeps the ideas, cuts the modes, and puts everything into request-and-response pairs.

The initial exchanges, four messages that replace Main Mode and Quick Mode together:

Initiator                                     Responder

IKE_SA_INIT   HDR, SAi1, KEi, Ni         -->
                                         <--  HDR, SAr1, KEr, Nr, [CERTREQ]
IKE_AUTH      HDR, SK {IDi, [CERT,] [CERTREQ,] [IDr,] AUTH, SAi2, TSi, TSr}   -->
                                         <--  HDR, SK {IDr, [CERT,] AUTH, SAr2, TSi, TSr}

• IKE_SA_INIT negotiates the algorithms (SA), exchanges Diffie-Hellman values (KE) and nonces (N). Both sides can now compute the shared secret.
• IKE_AUTH, encrypted and integrity-protected under the new keys (the SK { } notation), exchanges identities, proves them with AUTH, and sets up the first Child SA, the first ESP or AH SA, with its traffic selectors TSi and TSr, which become its SPD selectors.

Two more exchanges complete the protocol: CREATE_CHILD_SA, for further Child SAs or for rekeying, and INFORMATIONAL, for errors, deletions and checking that the peer is alive.

Where the keys come from. RFC 7296 s.2.14 derives everything from the Diffie-Hellman secret and both nonces:

SKEYSEED = prf(Ni | Nr, g^ir)
{SK_d | SK_ai | SK_ar | SK_ei | SK_er | SK_pi | SK_pr} = prf+(SKEYSEED, Ni | Nr | SPIi | SPIr)

where prf+ runs the pseudo-random function repeatedly to produce as many bytes as are needed. SK_d seeds the keys of Child SAs; SK_ai and SK_ar protect the integrity of IKE messages each way; SK_ei and SK_er encrypt them each way; SK_pi and SK_pr go into the AUTH values.

How AUTH stops the man in the middle. With a pre-shared key, the initiator's AUTH is a pseudo-random function, keyed from the shared secret, over its own first message, the responder's nonce, and a value computed with SK_pi. Its first message contains its Diffie-Hellman value. So the responder checks AUTH against the first message it received and the keys it derived. If anyone changed a Diffie-Hellman value in between, the two do not agree.

The run: IKEv2 from its formulas

The listing uses the 2048-bit group 14 that RFC 8247 requires, checked first as a safe prime, and HMAC-SHA-256 as the pseudo-random function. It runs an honest exchange, then a man in the middle, then a flood of spoofed requests with cookies off and on.

# IKEv2's first two exchanges, run: keys from RFC 7296's formulas, a MITM caught, cookies at work.
import hashlib, hmac, random

rng = random.Random(760)

# ---- Diffie-Hellman group 14 (RFC 3526), the group RFC 8247 says MUST be implemented -------
P = int(
    "FFFFFFFFFFFFFFFFC90FDAA22168C234C4C6628B80DC1CD129024E08"
    "8A67CC74020BBEA63B139B22514A08798E3404DDEF9519B3CD3A431B"
    "302B0A6DF25F14374FE1356D6D51C245E485B576625E7EC6F44C42E9"
    "A637ED6B0BFF5CB6F406B7EDEE386BFB5A899FA5AE9F24117C4B1FE6"
    "49286651ECE45B3DC2007CB8A163BF0598DA48361C55D39A69163FA8"
    "FD24CF5F83655D23DCA3AD961C62F356208552BB9ED529077096966D"
    "670C354E4ABC9804F1746C08CA18217C32905E462E36CE3BE39E772C"
    "180E86039B2783A2EC07A28FB5C55DF06F4C52C9DE2BCBF695581718"
    "3995497CEA956AE515D2261898FA051015728E5A8AACAA68FFFFFFFF"
    "FFFFFFFF", 16)
G = 2

def probably_prime(n, bases=(2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37)):
    d, r = n - 1, 0
    while d % 2 == 0:
        d, r = d // 2, r + 1
    for a in bases:
        x = pow(a, d, n)
        if x in (1, n - 1):
            continue
        for _ in range(r - 1):
            x = x * x % n
            if x == n - 1:
                break
        else:
            return False
    return True

print('group 14: %d-bit prime %s, and (P - 1) / 2 prime %s: a safe prime' % (
      P.bit_length(), probably_prime(P), probably_prime((P - 1) // 2)))

def prf(key, data):                         # PRF_HMAC_SHA2_256, RFC 8247 MUST
    return hmac.new(key, data, hashlib.sha256).digest()

def prf_plus(key, seed, length):            # RFC 7296 s.2.13
    out, t, i = b'', b'', 1
    while len(out) < length:
        t = prf(key, t + seed + bytes([i]))
        out, i = out + t, i + 1
    return out[:length]

SIZES = (('SK_d', 32), ('SK_ai', 32), ('SK_ar', 32), ('SK_ei', 16), ('SK_er', 16),
         ('SK_pi', 32), ('SK_pr', 32))

def ike_keys(ni, nr, shared, spii, spir):   # RFC 7296 s.2.14
    skeyseed = prf(ni + nr, shared.to_bytes(256, 'big'))
    stream = prf_plus(skeyseed, ni + nr + spii + spir, sum(n for _, n in SIZES))
    keys, at = {}, 0
    for name, n in SIZES:
        keys[name], at = stream[at:at + n], at + n
    return keys

def half(label):                             # one side's IKE_SA_INIT contribution
    x = rng.getrandbits(256)
    return {'who': label, 'x': x, 'KE': pow(G, x, P), 'N': rng.randbytes(32),
            'SPI': rng.randbytes(8)}

def auth(psk, message1, other_nonce, sk_p, identity):   # RFC 7296 s.2.15, pre-shared key
    signed = message1 + other_nonce + prf(sk_p, identity)
    return prf(prf(psk, b'Key Pad for IKEv2'), signed)

PSK = b'campus-vpn shared secret, 32 bytes!'

# ---- 1. an honest exchange ----------------------------------------------------------------
i, r = half('initiator'), half('responder')
msg1 = i['SPI'] + i['KE'].to_bytes(256, 'big') + i['N']        # HDR, SAi1, KEi, Ni (reduced)
ki = ike_keys(i['N'], r['N'], pow(r['KE'], i['x'], P), i['SPI'], r['SPI'])
kr = ike_keys(i['N'], r['N'], pow(i['KE'], r['x'], P), i['SPI'], r['SPI'])
print()
print('IKE_SA_INIT: two messages, then both sides derive seven keys from SKEYSEED')
for name, n in SIZES:
    print('  %-6s %2d bytes  same on both sides: %s' % (name, n, ki[name] == kr[name]))
a = auth(PSK, msg1, r['N'], ki['SK_pi'], b'asha@campus-a')
print('IKE_AUTH: the responder checks the initiator\'s AUTH:',
      a == auth(PSK, msg1, r['N'], kr['SK_pi'], b'asha@campus-a'))

# ---- 2. a man in the middle ---------------------------------------------------------------
i, r, m1, m2 = half('initiator'), half('responder'), half('mallory'), half('mallory')
msg1_sent = i['SPI'] + i['KE'].to_bytes(256, 'big') + i['N']           # what the initiator sent
msg1_seen = i['SPI'] + m2['KE'].to_bytes(256, 'big') + i['N']          # what reached the responder
k_i = ike_keys(i['N'], r['N'], pow(m1['KE'], i['x'], P), i['SPI'], r['SPI'])   # with Mallory
k_r = ike_keys(i['N'], r['N'], pow(m2['KE'], r['x'], P), i['SPI'], r['SPI'])   # with Mallory
forwarded = auth(PSK, msg1_sent, r['N'], k_i['SK_pi'], b'asha@campus-a')
print()
print('A MAN IN THE MIDDLE swaps in her own Diffie-Hellman values both ways')
mallory_i = ike_keys(i['N'], r['N'], pow(i['KE'], m1['x'], P), i['SPI'], r['SPI'])
mallory_r = ike_keys(i['N'], r['N'], pow(r['KE'], m2['x'], P), i['SPI'], r['SPI'])
print('  the two sides now hold different keys :', k_i['SK_ei'] != k_r['SK_ei'])
print('  and Mallory holds both sets of keys   :', mallory_i == k_i and mallory_r == k_r)
print('  the initiator\'s AUTH, relayed, checks :',
      forwarded == auth(PSK, msg1_seen, r['N'], k_r['SK_pi'], b'asha@campus-a'))
print('  so the responder refuses the IKE SA: AUTH covers the KE it received and the keys')
print('  it derived, and without the pre-shared key Mallory cannot make a new one')

# ---- 3. cookies against a flood of spoofed IKE_SA_INIT requests ---------------------------
SECRET = rng.randbytes(32)

def responder(ni, ip, spii, cookie=None, cookies_on=False):
    expect = b'\x01' + hashlib.sha256(ni + ip.encode() + spii + SECRET).digest()
    if cookies_on and cookie != expect:
        return 'N(COOKIE)', expect, 0             # no state kept, no exponentiation
    return 'SAr1, KEr, Nr', None, 1                # state kept, one exponentiation

print()
print('A FLOOD OF 200 IKE_SA_INIT REQUESTS FROM SPOOFED ADDRESSES')
for on in (False, True):
    work = sum(responder(rng.randbytes(32), '198.18.%d.%d' % (n // 250, n % 250),
                         rng.randbytes(8), cookies_on=on)[2] for n in range(200))
    print('  cookies %-3s: %3d half-open IKE SAs kept, %3d Diffie-Hellman computations'
          % ('on' if on else 'off', work, work))
ni, spii = rng.randbytes(32), rng.randbytes(8)
reply, cookie, _ = responder(ni, '203.0.113.50', spii, cookies_on=True)
print('  a real initiator is first told      :', reply)
print('  it repeats the request with the cookie:',
      responder(ni, '203.0.113.50', spii, cookie=cookie, cookies_on=True)[0])
print('  a spoofer never receives the cookie, because it goes to the address it forged')

What the run establishes, in order.

The group is sound. A 2048-bit prime whose half, less one, is also prime.

Both sides derive the same seven keys, of the sizes the algorithms need, from nothing but the Diffie-Hellman secret, the nonces and the SPIs; and the responder's check of the initiator's AUTH succeeds.

The man in the middle gets everything except acceptance. By swapping in her own Diffie-Hellman values both ways, Mallory leaves the two sides with different keys and holds both sets herself, exactly the attack of Module 1. But the initiator's AUTH, relayed to the responder, fails, because it was computed over a first message containing the initiator's Diffie-Hellman value, not the one the responder received, and with keys the responder does not have. Without the pre-shared key Mallory cannot compute a replacement.

Cookies turn a flood into nothing. Two hundred requests from forged addresses cost the responder 200 exponentiations and 200 half-open SAs with cookies off; with cookies on, it keeps no state and does no exponentiation, because it answers each with a cookie that goes to the forged address. A real initiator, told to present a cookie, does so and is served.

Where things stand

• IKEv1 is Historic. RFC 9395, April 2023: "IKEv1 has been deprecated, and RFCs 2407, 2408, and 2409 have been moved to Historic status", noting among IKEv2's improvements that IKEv1 is "vulnerable to amplification attacks". Systems still running IKEv1 "should be upgraded and reconfigured to run IKEv2".
• Algorithms for IKEv2, RFC 8247 (September 2017): Diffie-Hellman group 14, 2048-bit MODP, MUST; group 19, the 256-bit elliptic curve group, SHOULD; the 1,536-bit and 1,024-bit groups SHOULD NOT; the 768-bit group MUST NOT; HMAC-SHA-256 as the pseudo-random function MUST; AES-CBC encryption MUST.
• Curve25519 is defined for IKEv2 by RFC 8031 (December 2016) as group 31.

Distinctions that carry marks

What beginners get wrong here

Calling ISAKMP a key exchange. It defines how negotiations are carried, not how keys are agreed; Oakley, or IKE's own exchange, does that.

Saying cookies encrypt anything. They prove only that the initiator receives messages at its address, and they must be cheap.

Thinking Diffie-Hellman alone is safe. Unauthenticated, it falls to a man in the middle; IKE's AUTH is what binds the exchange to the parties.

Describing IKEv1's Main Mode as current practice. IKEv1 is Historic; IKEv2 is what is deployed.

Forgetting phase 2. The expensive, authenticated phase 1 sets up a protected channel once; the cheap phase 2 creates the SAs that actually protect traffic.

Quick revision

• Manual (by hand, small fixed setups) against automated (IKE) key management; RFC 4301 requires both be possible; RFC 8504: manual "not recommended".
• Threats to plain Diffie-Hellman: no authentication (man in the middle), clogging, replay.
• Oakley (RFC 2412): cookies, negotiated algorithms, authentication bound to the exponentials, chosen groups, nonces, perfect forward secrecy; authentication by pre-shared keys (required), signatures, public-key encryption.
• Cookies: depend on the parties, need the issuer's secret, fast to compute (RFC 2408, after Karn).
• ISAKMP (RFC 2408): a framework; header with two 8-byte cookies; payloads; exchange types; two phases.
• IKEv1 (RFC 2409): Main Mode 6, Aggressive Mode 3, Quick Mode 3. Historic since RFC 9395 (2023).
• IKEv2 (RFC 7296): IKE_SA_INIT (SA, KE, N) and IKE_AUTH (ID, AUTH, first Child SA, traffic selectors); CREATE_CHILD_SA; INFORMATIONAL. SKEYSEED = prf(Ni | Nr, g^ir); seven keys by prf+; AUTH defeats the man in the middle.
• RFC 8247: group 14 MUST, group 19 SHOULD, groups 2 and 5 SHOULD NOT, HMAC-SHA-256 PRF MUST.

Test yourself

1. Distinguish manual from automated key management in IPsec. In manual key management an administrator configures each system with its own keys and the keys of the systems it talks to; it suits small, fixed arrangements, and keys change only when someone retypes them. Automated key management uses a protocol, IKE, to negotiate security associations and derive fresh keys on demand, so it scales to many systems and supports regular rekeying; RFC 8504 describes manual keying as of limited applicability and not recommended.

2. What features does Oakley add to Diffie-Hellman? Cookies to resist clogging attacks; negotiation of the encryption, key derivation and authentication methods; authentication that binds the Diffie-Hellman exponentials to the identities of the parties, defeating a man in the middle; the choice of standard or user-defined groups; nonces against replay; and perfect forward secrecy. Its authentication may use pre-shared keys, digital signatures or public-key encryption.

3. State the requirements for ISAKMP cookies and explain how a cookie defeats clogging. A cookie must depend on the specific parties, must be something only the issuing entity can generate, using a local secret that cannot be deduced from any cookie, and must be fast to compute. The responder answers a first request with a cookie instead of doing a Diffie-Hellman computation, and does the expensive work only when the request is repeated with the cookie. A forger using another party's address never receives the cookie, so it can never make the responder do the work.

4. What is ISAKMP, and how does it differ from Oakley? ISAKMP is a framework that defines the message formats, payloads, exchange types and procedures for establishing, negotiating, modifying and deleting security associations, independently of any particular key exchange. Oakley is a key determination protocol, based on authenticated Diffie-Hellman, that actually produces the keys. IKEv1 combined them, Oakley's exchange carried in ISAKMP's messages.

5. Describe the IKEv2 initial exchanges. In IKE_SA_INIT the initiator sends its proposed algorithms, its Diffie-Hellman value and a nonce, and the responder replies with its chosen algorithms, its own Diffie-Hellman value and nonce, and optionally a certificate request; both can then compute SKEYSEED and the keys derived from it. In IKE_AUTH, encrypted and integrity-protected with those keys, each side sends its identity, optionally certificates, and an AUTH value that proves its identity and covers the earlier messages, and the pair sets up the first Child SA with its traffic selectors.

6. How does IKEv2's AUTH payload defeat a man in the middle? The AUTH value is computed over the sender's own first message, which contains its Diffie-Hellman value, over the other side's nonce, and over a value derived from the keys, using a secret the attacker lacks, a pre-shared key or a private key. A man in the middle who substituted his own Diffie-Hellman values leaves the responder checking AUTH against a first message and keys different from those the initiator used, so the check fails, and without the secret he cannot compute a valid replacement.

7. What is the status of IKEv1 today? It is deprecated. RFC 9395, of April 2023, moved RFCs 2407, 2408 and 2409, the documents defining IKEv1 and ISAKMP, to Historic status, stating that IKEv2, first published in December 2005 and now RFC 7296, is a full replacement, and that systems running IKEv1 should be upgraded to IKEv2.