VR on IPsec VPN Part 3 - Understanding Different IKEv2 Message Types

1. Introduction

In Part 1 we sketched why IKEv2 is a worthwhile target and listed the exchanges an implementation has to handle. Before we can start poking at any of them, we need to know exactly what each message type is supposed to do and what the protocol guarantees around it.

There are a total of 6 different types of messages:

• IKE_SA_INIT, IKE_AUTH, INFORMATIONAL, and CREATE_CHILD_SA are defined by the base IKEv2 specification, RFC 7296.
• IKE_SESSION_RESUME is the session-resumption extension defined by RFC 5723.
• IKE_INTERMEDIATE is defined by RFC 9242, and is the transport used by the Additional Key Exchanges (ADDKE) mechanism in RFC 9370 for post-quantum key agreement.

However, we are not going to cover all of them in this post. We will mainly focus on covering IKE_SA_INIT, IKE_AUTH and INFORMATIONAL extensively, because these messages can be sent during pre-authentication.

2. Message Types

2.1 IKE_SA_INIT

The IKE_SA_INIT exchange negotiates cryptographic parameters and generates a shared secret. These messages are sent in plaintext and are not yet encrypted. The following diagram shows a simplified call flow how the initiator and responder communicates with each other to establish a set of session keys, used for encrypting and decrypting messages in the later stages.

IKE_SA_INIT – Simplified Call Flow

Cool, now that you have a high level diagram to reference from, let’s get into the nerdy computation stuff for our initiator.

Assuming our initiator wants to do the key exchange with DH group 14, generating the key pair with python is as easy as the following snippet:

from diffiehellman import DiffieHellman

dh = DiffieHellman(group=14)
kei = dh.get_public_key()

The initiator also generates a random 8-bytes Security Parameter Index (SPI), along with a random 32-bytes nonce.

import os

spi_i = os.urandom(8)
n_i = os.urandom(32)

The initiator sends a list of proposals for responder to choose from. The following shows an example of the initiator proposing to use AES-CBC 128 bit encryption, SHA2-256-128 for integrity checks and using HMAC-SHA2-256 for Pseudo Random Function (PRF).

IKE_SA_INIT
├── SA
│   ├── Proposal 1
│   │   ├── Transform 1 (AES-CBC 128 bit Encryption)
│   │   ├── Transform 2 (SHA2-256-128 Integrity)
│   │   ├── Transform 3 (HMAC-SHA2-256 PRF)
│   │   └── Transform 4 (DH Group 14)
│   ├── Proposal 2
│   └── Proposal 3
├── KE (KEi)
├── Nonce (Ni)
...

This is what the responder’s IKE_SA_INIT message could look like, should it choose proposal 1. It also sends back its own public key (KE_r) and nonce (N_r) to the initiator.

IKE_SA_INIT
├── SA
│   └── Proposal 1
│       ├── Transform 1 (AES-CBC 128 bit Encryption)
│       ├── Transform 2 (SHA2-256-128 Integrity)
│       ├── Transform 3 (HMAC-SHA2-256 PRF)
│       └── Transform 4 (DH Group 14)
├── KE (KEr)
├── Nonce (Nr)
...

The initiator’s computes the DH shared secret (g^ir):

g_ir = dh.generate_shared_key(ke_r)

Then, it only needs to calculate something called the secret key seed (SKEYSEED) using the PRF chosen by the responder. Since the responder chose proposal 1 as shown in the previous listing, we will need to calculate SKEYSEED based on HMAC-SHA2-256. We can do so with the pycryptodome python library.

from Crypto.Hash import HMAC

skeyseed = HMAC.new(n_i + n_r, g_ir, SHA256).digest()

Next, the initiator generates a key material, which is comprised of different session keys in the following strict order:

Here’s the python snippet to generate the key material by using the PRF+ formula based on HMAC-SHA2-256:

"""
RFC 7296 PRF+ Definition
-------------------------
prf+ (K,S) = T1 | T2 | T3 | T4 | ...
where:
T1 = prf (K, S | 0x01)
T2 = prf (K, T1 | S | 0x02)
T3 = prf (K, T2 | S | 0x03)
T4 = prf (K, T3 | S | 0x04)
"""
def prf_plus(skeyseed: bytes, seed: bytes) -> bytes:
    keys = b''
    t = b''
    counter = 1

    # 1024 bytes is more than enough buffer
    while len(keys) < 1024:
        t = HMAC.new(skeyseed, t + seed + bytes([counter]), SHA256).digest()
        keys += t
        counter += 1

    return keys

seed = n_i + n_r + spi_i + spi_r
key_material = prf_plus(skeyseed, seed)

Then, partition the key material data for each kind of session key based on the chosen example proposal:

offset = 0

# len(sk_d) based on PRF HMAC-SHA2-SHA256 == 32 bytes
sk_d = key_material[offset:offset + 32]
offset += 32

# len(sk_ai) based on Integrity SHA2-256-128 == 32 bytes
sk_ai = key_material[offset:offset + 32]
offset += 32

# len(sk_ar) based on Integrity SHA2-256-128 == 32 bytes
sk_ar = key_material[offset:offset + 32]
offset += 32

# len(sk_ei) based on Encryption AES-CBC-128 == 16 bytes
sk_ei = key_material[offset:offset + 16]
offset += 16

# len(sk_er) based on Encryption AES-CBC-128 == 16 bytes
sk_er = key_material[offset:offset + 16]
offset += 16

# len(sk_pi) based on PRF HMAC-SHA2-SHA256 == 32 bytes
sk_pi = key_material[offset:offset + 32]
offset += 32

# len(sk_pr) based on PRF HMAC-SHA2-SHA256 == 32 bytes
sk_pr = key_material[offset:offset + 32]
offset += 32

From here on out, every other message that comes after IKE_SA_INIT will be encrypted, with strict verification of the messages’ integrity.

2.2 IKE_AUTH

After the session keys have been computed from a successful IKE_SA_INIT exchange, what follows after is typically IKE_AUTH. Depending on the authentication method, the payload types will differ slightly. We will mainly study how IKE_AUTH looks like when authenticating with PSK and digital sigantures.

2.2.1 Pre-Shared Key (PSK)

IKE_AUTH (PSK) – Simplified Call Flow

The calculation of the AUTH payload is kinda tricky here, and it took me awhile to really understand what’s going on. As usual, let us focus on how the computations are done by the initiator.

Based on RFC 7296, the initiator calculates the IKE_AUTH payload like so:

AUTH = prf( prf(Shared Secret, "Key Pad for IKEv2"), <InitiatorSignedOctets>)

And the initiator’s signed octets are defined as such:

InitiatorSignedOctets = RealMessage1 | NonceRData | MACedIDForI
GenIKEHDR = [ four octets 0 if using port 4500 ] | RealIKEHDR
RealIKEHDR =  SPIi | SPIr |  . . . | Length
RealMessage1 = RealIKEHDR | RestOfMessage1
NonceRPayload = PayloadHeader | NonceRData
InitiatorIDPayload = PayloadHeader | RestOfInitIDPayload
RestOfInitIDPayload = IDType | RESERVED | InitIDData
MACedIDForI = prf(SK_pi, RestOfInitIDPayload)

Convoluted much? But what the hell is RealMessage1? This took me some trial and error to figure out, but it’s essentially just the entire initiator’s IKE_SA_INIT message, starting from the IKEv2 header.

In python, the computation of the AUTH payload would look something like this:

from scapy.contrib.ikev2 import *

# Previously assembled IKE_SA_INIT message sent by initiator
ike_sa_init = IKEv2(exch_type="IKE_SA_INIT") / IKEv2_SA (...) / ...

# Get RestOfInitIDPayload
rest_of_init_id_payload = bytes(ike_auth[IKEv2_IDi].IDtype) + b"\x00\x00\x00" + ike_auth[IKEv2_IDi].ID.encode("utf-8")

# Calculate MACedIDforI
mac_id_i = HMAC.new(sk_pi, rest_of_init_id_payload, SHA256).digest()

# Get initiator's signed octets
signed_octets = bytes(ike_sa_init) + n_r + mac_id_i

# Calculate AUTH payload
PSK = b"lab-psk-change-me"
auth_payload = HMAC.new(HMAC.new(PSK, b"Key Pad for IKEv2", SHA256).digest(), signed_octets, SHA256).digest()

Once the initiator encrypts the IKE_AUTH plaintext data (after IKEv2 headers), it sends an encrypted message that on decryption, is minimally made up of:

• Initiator’s identifier (ID_i)
• Responder’s Identifier (ID_r)
• Authentication (AUTH)
• Configuration (CP)
• Security Association (SA)
• Proposal with its transforms
• Initiator’s Traffic Selector (TS_i)
• Responder’s Traffic Selector (TS_r)

The listing below shows the breakdown of its payload types as explained when it’s decrypted.

IKE_AUTH
└── Encrypted
    └── (Decrypted Data)
        ├── IDi          
        ├── IDr           
        ├── AUTH (PSK)   
        ├── CP            
        ├── SA            
        ├── Proposal 1
        │   ├── Transform 1 (AES-CBC 128 bit Encryption)
        │   ├── Transform 2 (SHA2-256-128 Integrity)
        │   └── Transform 3 (Extended Sequence Numbers)
        ├── TSi        
        ├── TSr
        ...

2.2.2 Digital Signatures

IKE_AUTH (Digital Signatures) – Simplified Call Flow

When using digital signatures instead of a pre-shared key, each side proves its identity by signing the same signed octets with its private key. The other side verifies the signature using the public key from the peer’s certificate. This section walks through how that works with ECDSA P-256 and SHA-256, which is what our lab’s certificate-based road warrior connection uses.

Good news: the signed octets are computed exactly the same way as in the PSK case (section 2.2.1). The InitiatorSignedOctets formula from RFC 7296 does not change – it is still RealMessage1 | NonceRData | MACedIDForI. What changes is what we do with those octets.

from scapy.contrib.ikev2 import *
from Crypto.Hash import HMAC, SHA256

# Previously assembled IKE_SA_INIT message sent by initiator
ike_sa_init = IKEv2(exch_type="IKE_SA_INIT") / IKEv2_SA (...) / ...

# IDi is an email address (ID_RFC822_ADDR) for certificate-based auth
rest_of_init_id_payload = bytes(ike_auth[IKEv2_IDi].IDtype) + b"\x00\x00\x00" + ike_auth[IKEv2_IDi].ID.encode("utf-8")

# MACedIDForI is still computed with prf(SK_pi, ...) -- same as PSK
mac_id_i = HMAC.new(sk_pi, rest_of_init_id_payload, SHA256).digest()

# The signed octets are identical to the PSK case
signed_octets = bytes(ike_sa_init) + n_r + mac_id_i

Here is where PSK and digital signatures diverge. Instead of a double-HMAC with the pre-shared key, we compute a real cryptographic signature over the exact same octets:

PSK:               AUTH = prf( prf(Shared Secret, "Key Pad for IKEv2"), <InitiatorSignedOctets>)
Digital Signature:  AUTH = sign( private_key, <InitiatorSignedOctets>)

In python, the signing with ECDSA P-256 looks like this:

from cryptography.hazmat.primitives import hashes, serialization
from cryptography.hazmat.primitives.asymmetric import ec

# Load the initiator's ECDSA P-256 private key
with open("clientKey.pem", "rb") as f:
    private_key = serialization.load_pem_private_key(f.read(), password=None)

# Sign the same signed octets we computed above
signature = private_key.sign(signed_octets, ec.ECDSA(hashes.SHA256()))

The resulting AUTH payload does not just contain the raw signature. RFC 7427 defines a new authentication data format that wraps the signature with metadata telling the receiver which algorithm was used:

RFC 7427 Authentication Data Format
────────────────────────────────────
Auth Method (1 byte):           14 (Digital Signature)
Reserved (3 bytes):             0x000000
ASN.1 Length (1 byte):          Length of AlgorithmIdentifier
AlgorithmIdentifier:            DER-encoded ASN.1 (identifies signature algorithm)
Signature Value:                The actual signature bytes

For ECDSA-SHA256, the ASN.1 AlgorithmIdentifier encodes OID 1.2.840.10045.4.3.2 (ecdsa-with-SHA256) into 12 bytes. The following snippet shows how to assemble the full authentication data:

# ASN.1 DER-encoded AlgorithmIdentifier for ecdsa-with-SHA256
# SEQUENCE { OID 1.2.840.10045.4.3.2 }
asn1_algo_id = bytes.fromhex("300a06082a8648ce3d040302")
asn1_length = bytes([len(asn1_algo_id)])

auth_data = asn1_length + asn1_algo_id + signature

Unlike PSK where both sides already know the shared secret, with digital signatures the responder needs a way to obtain the initiator’s public key. The initiator therefore includes a CERT payload containing its X.509 certificate, and a CERTREQ payload with the SHA-1 hash of the trusted CA’s public key to tell the responder which CA it trusts. The responder validates the initiator’s certificate chain against its own trusted CA, verifies the AUTH signature using the public key from the certificate, and then sends back its own certificate and AUTH signature in the IKE_AUTH response.

Once the initiator encrypts the IKE_AUTH plaintext data (after IKEv2 headers), it sends an encrypted message that on decryption, is minimally made up of:

• Initiator’s Identifier (ID_i)
• Certificate (CERT)
• Certificate Request (CERTREQ)
• Responder’s Identifier (ID_r)
• Authentication (AUTH)
• Configuration (CP)
• Security Association (SA)
• Proposal with its transforms
• Initiator’s Traffic Selector (TS_i)
• Responder’s Traffic Selector (TS_r)

The listing below shows the breakdown of its payload types as explained when it’s decrypted.

IKE_AUTH
└── Encrypted
    └── (Decrypted Data)
        ├── IDi (ID_RFC822_ADDR: road@lab)
        ├── CERT (X.509 Certificate)
        ├── CERTREQ (X.509, CA public key hash)
        ├── IDr (ID_IPV4_ADDR: 192.168.122.4)
        ├── AUTH (Digital Signature, ECDSA-SHA256)
        ├── CP (CFG_REQUEST)
        │   ├── INTERNAL_IP4_ADDRESS
        │   └── INTERNAL_IP4_DNS
        ├── SA
        │   └── Proposal 1
        │       ├── Transform 1 (AES-CBC 128 bit Encryption)
        │       ├── Transform 2 (SHA2-256-128 Integrity)
        │       └── Transform 3 (Extended Sequence Numbers)
        ├── TSi
        ├── TSr
        ...

2.3 INFORMATIONAL

2.3.1 Heartbeat

INFORMATIONAL – Heartbeat Simplified Call Flow

Once an IKE SA is established, either side may want to check whether the peer is still alive. RFC 7296 Section 2.4 calls this a “liveness check”, but it is commonly known as Dead Peer Detection (DPD). The mechanism is simple: send an empty INFORMATIONAL request and wait for an empty INFORMATIONAL response.

In our lab configuration, the server has dpd_delay = 30s, which means it sends a heartbeat every 30 seconds when the tunnel is idle. If the peer does not respond within dpd_timeout (120 seconds), the SA is considered dead and cleaned up.

The request and response are both encrypted and integrity-protected like any other post-IKE_SA_INIT message. The only difference is that the Encrypted payload contains no inner payloads at all – it is just padding.

from scapy.contrib.ikev2 import *

# Heartbeat request (empty INFORMATIONAL)
ike_heartbeat = IKEv2(
    init_SPI=initiator_spi,
    resp_SPI=responder_spi,
    exch_type="INFORMATIONAL",
    flags="Initiator",
    id=message_id
) / IKEv2_Encrypted(
    next_payload="NONE",
    load=iv + ciphertext + checksum
)

When decrypted, both the request and the response contain nothing:

INFORMATIONAL (Heartbeat)
├── Request
│   └── Encrypted
│       └── (Decrypted Data)
│           (empty)
└── Response
    └── Encrypted
        └── (Decrypted Data)
            (empty)

2.3.2 Teardown

INFORMATIONAL – Teardown Simplified Call Flow

To gracefully close an IKE SA, either side sends an INFORMATIONAL request containing a Delete payload (RFC 7296 Section 1.4). The peer acknowledges with an empty INFORMATIONAL response, and both sides tear down the SA along with all its Child SAs.

The Delete payload for an IKE SA sets the Protocol ID to IKE (1) with an SPI Size of 0 and no SPIs listed. This is because the IKE SA’s SPIs are already in the IKEv2 header – there is no ambiguity about which SA is being deleted.

from scapy.contrib.ikev2 import *

# IKE SA teardown request
ike_delete = IKEv2(
    init_SPI=initiator_spi,
    resp_SPI=responder_spi,
    exch_type="INFORMATIONAL",
    flags="Initiator",
    id=message_id
) / IKEv2_Encrypted(
    next_payload="Delete",
    load=iv + ciphertext + checksum
)

When decrypted, the request contains a single Delete payload, while the response is empty:

INFORMATIONAL (Teardown)
├── Request
│   └── Encrypted
│       └── (Decrypted Data)
│           └── Delete
│               ├── Protocol ID: IKE (1)
│               ├── SPI Size: 0
│               └── Number of SPIs: 0
└── Response
    └── Encrypted
        └── (Decrypted Data)
            (empty)

3. Encryption

After IKE_SA_INIT, IKEv2 protects the payloads inside each message. This section uses the algorithms negotiated in our earlier example: AES-CBC for encryption and HMAC-SHA2-256-128 for integrity. Other algorithms can use different IV and checksum sizes, and combined-mode algorithms such as AES-GCM package the data differently.

With AES-CBC and a separate integrity algorithm, the Encrypted payload defined by RFC 7296 Section 3.14 looks like this:

IKEv2 Header (28 bytes) | Encrypted Payload Header (4 bytes) | IV | Ciphertext | Checksum

• IV: a fresh 16-byte initialization vector for AES-CBC.
• Ciphertext: the inner IKE payloads, padded and encrypted.
• Checksum: a 16-byte Integrity Checksum Data field for HMAC-SHA2-256-128.

For a message sent by the original IKE initiator, encryption uses SK_ei and integrity protection uses SK_ai. Messages in the other direction use SK_er and SK_ar.

First, pad and encrypt the inner payloads. AES-CBC requires the input length to be a multiple of 16 bytes, so the final byte records how many padding octets come before it.

from Crypto.Cipher import AES
import os

iv = os.urandom(16)

# Pad plaintext to 16-byte boundary
pad_len = 16 - (len(plaintext) % 16)
padded = plaintext + os.urandom(pad_len - 1) + bytes([pad_len - 1])

cipher = AES.new(sk_ei, AES.MODE_CBC, iv)
ciphertext = cipher.encrypt(padded)

Next, build the message with a temporary all-zero checksum. Serializing the message at this point gives us the bytes covered by the integrity calculation.

from scapy.contrib.ikev2 import *

placeholder_checksum = b"\x00" * 16

ike_msg = IKEv2(
    init_SPI=initiator_spi,
    resp_SPI=responder_spi,
    exch_type="IKE_AUTH",
    flags="Initiator",
    id=message_id
) / IKEv2_Encrypted(
    next_payload="IDi",
    load=iv + ciphertext + placeholder_checksum
)

The HMAC covers the complete IKE message from the fixed IKEv2 header through the encrypted Pad Length byte. This includes any payloads before the Encrypted payload, but not the checksum field itself or the outer IP and UDP headers. In this example, the Encrypted payload is the only payload after the IKEv2 header. HMAC-SHA2-256-128 truncates the result to 16 bytes.

from Crypto.Hash import HMAC, SHA256

# bytes(ike_msg)[0:-16] == everything except the 16-byte placeholder checksum
h = HMAC.new(sk_ai, bytes(ike_msg)[0:-16], SHA256)
checksum = h.digest()[:16]

Finally, we swap the placeholder with the real checksum.

ike_msg[IKEv2_Encrypted].load = iv + ciphertext + checksum

The resulting message looks like this:

IKE_AUTH
├── IKEv2 Header (28 bytes)
└── Encrypted Payload
    ├── Encrypted Payload Header (4 bytes)
    ├── IV (16 bytes, random)
    ├── Ciphertext (padded plaintext, encrypted with SK_ei)
    └── Checksum (16 bytes, HMAC-SHA2-256-128 with SK_ai)

The receiver verifies the checksum before decrypting. If it is valid, the receiver decrypts the ciphertext and removes the padding:

# Strip IV (first 16 bytes) and checksum (last 16 bytes) from the payload
iv = payload[:16]
ciphertext = payload[16:-16]

if not ciphertext or len(ciphertext) % AES.block_size != 0:
    raise ValueError("Invalid ciphertext length")

cipher = AES.new(sk_ei, AES.MODE_CBC, iv)
padded = cipher.decrypt(ciphertext)

# Get padding length from the last byte and strip it
pad_len = padded[-1]
if pad_len + 1 > len(padded):
    raise ValueError("Invalid padding length")

plaintext = padded[:-pad_len - 1]

This Encrypted payload format can protect exchanges such as IKE_AUTH, INFORMATIONAL, and IKE_INTERMEDIATE. The inner payloads and key direction may change, but the AES-CBC/HMAC protection steps remain the same.

4. IKE Fragmentation

Large IKEv2 messages may trigger IP fragmentation. In practice, networks sometimes drop IP fragments or the ICMP messages needed for path-MTU discovery. IKE_AUTH is especially likely to hit this problem when it carries certificates or many proposals and traffic selectors.

RFC 7383 solves this with IKE-level fragmentation. Both sides signal support during IKE_SA_INIT by including a IKEV2_FRAGMENTATION_SUPPORTED notify payload:

IKEv2_Notify(type="IKEV2_FRAGMENTATION_SUPPORTED")

After both peers advertise support, a message containing an Encrypted payload can instead be sent as a set of smaller Encrypted Fragment payloads.

Fragmentation happens before encryption: the sender splits the original Encrypted payload’s plaintext content into chunks. A chunk may even end in the middle of an inner payload. Each chunk then gets its own padding, IV, encryption, and integrity checksum.

The fragment size must account for the entire IP datagram, not only the plaintext. For IPv4 over UDP port 500 with the algorithms used in this article, the fixed overhead is:

MTU Budget Breakdown (IPv4, UDP port 500)
─────────────────────────────────────────
IPv4 Header:                    20 bytes
UDP Header:                      8 bytes
IKEv2 Header:                  28 bytes
Encrypted Fragment Header:       8 bytes
IV (AES-CBC):                   16 bytes
Checksum (HMAC-SHA2-256-128):   16 bytes
─────────────────────────────────────────
Total Overhead:                 96 bytes

This 96-byte figure is not universal. UDP port 4500 adds a 4-byte Non-ESP Marker, and IPv6, tunnels, IP options, and different cryptographic algorithms change the overhead.

The remaining space must hold block-aligned ciphertext. The following calculation rounds down to an AES block boundary and reserves one complete block for padding. This is conservative and easy to reason about, although it may leave a few bytes unused.

IP_HDR_LEN = 20
UDP_HDR_LEN = 8
IKE_HDR_LEN = 28
ENC_FRAG_HDR_LEN = 8
IV_LEN = 16
CHECKSUM_LEN = 16
BLOCK_SIZE = 16

overhead = IP_HDR_LEN + UDP_HDR_LEN + IKE_HDR_LEN + ENC_FRAG_HDR_LEN + IV_LEN + CHECKSUM_LEN
max_ciphertext = ((mtu - overhead) // BLOCK_SIZE) * BLOCK_SIZE
max_plaintext_per_frag = max_ciphertext - BLOCK_SIZE

if max_plaintext_per_frag <= 0:
    raise ValueError("MTU is too small for an encrypted IKE fragment")

plaintext_frags = []
offset = 0
while offset < len(plaintext):
    plaintext_frags.append(plaintext[offset:offset + max_plaintext_per_frag])
    offset += max_plaintext_per_frag

frag_total = len(plaintext_frags)

For example, a 576-byte MTU leaves 480 bytes for ciphertext after the 96-byte overhead. Reserving one AES block gives a conservative chunk size of 464 plaintext bytes.

Every fragment uses the same IKE Message ID but has its own fragment number, IV, ciphertext, and checksum. The IKEv2 header points to an Encrypted_Fragment payload. Inside that payload, only fragment 1 identifies the first original inner payload; fragments 2 through N set next_payload to zero. This example fragments a message from the original IKE initiator, so it uses SK_ei and SK_ai; the other direction uses SK_er and SK_ar.

ike_hdr = IKEv2(
    init_SPI=unfrag_ike[IKEv2].init_SPI,
    resp_SPI=unfrag_ike[IKEv2].resp_SPI,
    next_payload="Encrypted_Fragment",
    version=unfrag_ike[IKEv2].version,
    exch_type=unfrag_ike[IKEv2].exch_type,
    flags=unfrag_ike[IKEv2].flags,
    id=unfrag_ike[IKEv2].id
)

for i in range(1, frag_total + 1):
    # encrypt_aes_cbc applies fresh padding and generates a fresh IV
    iv, ciphertext = encrypt_aes_cbc(sk_ei, plaintext_frags[i - 1])

    placeholder_checksum = b"\x00" * 16
    ike_frag = ike_hdr / IKEv2_Encrypted_Fragment(
        next_payload=(
            unfrag_ike[IKEv2_Encrypted].next_payload if i == 1 else 0
        ),
        frag_number=i,
        frag_total=frag_total,
        load=iv + ciphertext + placeholder_checksum
    )

    # Compute integrity checksum for this fragment
    h = HMAC.new(sk_ai, bytes(ike_frag)[0:-16], SHA256)
    checksum = h.digest()[:16]
    ike_frag[IKEv2_Encrypted_Fragment].load = iv + ciphertext + checksum

Each fragment ends up looking like this:

IKE_AUTH (Fragment 1 of N)
├── IKEv2 Header (next_payload = Encrypted_Fragment)
└── Encrypted Fragment Payload
    ├── Next Payload (original type in fragment 1; zero otherwise)
    ├── Fragment Number (1)
    ├── Total Fragments (N)
    ├── IV (16 bytes, random)
    ├── Ciphertext (fragment plaintext, encrypted with SK_ei)
    └── Checksum (16 bytes, HMAC-SHA2-256-128 with SK_ai)

Fragments can arrive out of order and are not acknowledged separately. For each one, the receiver verifies the checksum before decrypting, removes that fragment’s padding, and stores the plaintext chunk. Once every fragment is present, it concatenates the chunks by fragment number to recover the original Encrypted payload content.

An invalid fragment is silently discarded. The receiver may keep the other valid fragments while waiting for a valid replacement; reassembly fails if a complete set does not arrive before its timeout.

5. Conclusion

This post walked through the four core IKEv2 message types we care about for vulnerability research:

1. IKE_SA_INIT for negotiating cryptographic parameters in plaintext
2. IKE_AUTH for proving identity via PSK or digital signatures
3. INFORMATIONAL for housekeeping like heartbeats and teardown
4. Encryption and fragmentation when sending messages after SA has been established.

With this foundation, we now know exactly what each message carries, how the payloads are structured, and where the protocol’s complexity lives.

6. Resources

1. diagrams.drawio

7. References

1. RFC 7296 – Internet Key Exchange Protocol Version 2 (IKEv2)
2. RFC 5723 – IKEv2 Session Resumption
3. RFC 7383 – IKEv2 Message Fragmentation
4. RFC 9242 – Intermediate Exchange in IKEv2
5. RFC 9370 – Multiple Key Exchanges in IKEv2 (ADDKE)
6. RFC 7427 – Signature Authentication in the Internet Key Exchange Version 2 (IKEv2)