Internet Key Exchange (IKE) is one of the most widely deployed protocols for establishing IPsec VPN connections. Despite the growing popularity of modern alternatives such as WireGuard and OpenVPN, IKE remains an essential component of enterprise network infrastructure, particularly for site-to-site VPNs, remote-access VPN gateways and firewall appliances.

Unfortunately, IKE services are frequently overlooked during external security assessments. Their UDP-based communication, multiple protocol versions, complex negotiation process and vendor-specific configuration requirements make them considerably more difficult to discover and fingerprint than conventional network services.

In this article, we explore how IKE works, how to discover exposed IKE services, and how to audit their cryptographic configuration. We will also introduce IKESS – IKE Security Scanner, a Python 3 tool that automates the discovery, enumeration, fingerprinting and security assessment of IKE VPN servers.

GitHub - l4rm4nd/IKESS: A Python3 Script for Auditing IKE VPN Servers
A Python3 Script for Auditing IKE VPN Servers. Contribute to l4rm4nd/IKESS development by creating an account on GitHub.

Understanding Internet Key Exchange (IKE)

IKE (Internet Key Exchange) is the key management protocol used by IPsec to establish and maintain secure VPN connections.

While IPsec provides the actual network-layer protection through protocols such as Encapsulating Security Payload (ESP), IKE handles the negotiation and management of the cryptographic parameters required to establish those connections.

This includes:

  • Authentication of the communicating VPN peers.
  • Negotiation of encryption and integrity algorithms.
  • Diffie-Hellman key exchange.
  • Establishment of Security Associations (SAs).
  • Rekeying and maintenance of existing VPN connections.
  • NAT traversal negotiation.

IKE is available in two major versions:

  • IKEv1: The original protocol, specified in RFC 2409. It supports Main Mode and Aggressive Mode during Phase 1 negotiation.
  • IKEv2: The significantly improved successor, specified in RFC 7296. It simplifies the negotiation process, improves reliability and introduces a more structured exchange model.

IKEv2 should be preferred for modern deployments. IKEv1 has been formally deprecated and should be disabled wherever possible. Especially due to the insecure Aggressive Mode, which may leak the Pre-Shared-Key (PSK) hash.

Why is IKE still so common?

Although WireGuard and OpenVPN have become popular VPN technologies, IKE/IPsec remains heavily deployed, particularly in enterprise environments.

There are several reasons for this:

Native operating system support: IKE/IPsec is supported by Windows, macOS, Linux, iOS and Android. Many operating systems can establish IKEv2 connections without installing additional VPN software.

  • Site-to-site connectivity: IPsec is a long-established standard for connecting corporate networks, branch offices, cloud environments and data centers.
  • Firewall integration: Almost every major enterprise firewall vendor provides integrated IPsec VPN functionality. Products from Fortinet, Cisco, Palo Alto Networks, Check Point, Sophos and others commonly expose IKE services.
  • Hardware acceleration: Many networking appliances provide dedicated hardware acceleration for IPsec encryption and packet processing.
  • Interoperability: IKE/IPsec is standardized and supports interoperability between different vendors and platforms.

WireGuard offers a much simpler and more modern protocol design, while OpenVPN provides a flexible TLS-based VPN implementation. However, neither has displaced the enormous installed base of IKE/IPsec infrastructure.

For penetration testers, this makes IKE an important attack surface that should be included in external network assessments.

IKE Network Services: UDP/500 and UDP/4500

IKE primarily communicates over UDP using two well-known ports:

Port

Protocol

Purpose

UDP/500

IKE / ISAKMP

Initial IKE negotiation

UDP/4500

IKE / NAT-T / UDP-encapsulated ESP

NAT traversal and encapsulated IPsec traffic

UDP/500 – IKE and ISAKMP

UDP/500 is the standard port used for IKE communication.

Both IKEv1 and IKEv2 initially use this port to exchange negotiation messages.

Depending on the IKE version, the initial exchange looks different:

  • IKEv1 uses ISAKMP Phase 1 exchanges, including Main Mode and Aggressive Mode.
  • IKEv2 begins with an IKE_SA_INIT exchange, followed by IKE_AUTH.

During these exchanges, the VPN peers negotiate cryptographic parameters and establish the initial IKE Security Association.

UDP/4500 – NAT Traversal

UDP/4500 is used for NAT Traversal (NAT-T).

IPsec ESP normally operates directly at the IP layer using IP protocol number 50. This can cause problems when communicating through NAT devices, which commonly translate IP addresses and transport-layer ports.

NAT-T solves this problem by encapsulating ESP traffic inside UDP packets.

IKE traffic can also be transported over UDP/4500.

A useful distinction is the non-ESP marker:

  • IKE packets sent over UDP/4500 begin with four zero bytes.
  • Encapsulated ESP packets begin with a non-zero SPI value.

This allows the receiving endpoint to distinguish between IKE and ESP traffic on the same UDP port.

An important detail is that NAT-T is not necessarily restricted to connections where NAT has actually been detected. IKE implementations can also initiate communication directly over UDP/4500.

Consequently, a VPN gateway may respond on UDP/4500 even when UDP/500 is completely silent.

Discovering IKE Services with Nmap

The first step during an external assessment is identifying hosts that expose IKE services.

Nmap provides an NSE script called ike-version, which can identify IKE implementations and retrieve information such as vendor IDs and supported features.

A simple discovery command is:

nmap -n -Pn -sU -sV -p 500,4500 --script ike-version <TARGET>

Alternatively, for a straightforward IKE discovery attempt:

nmap -n -Pn -sU -p 500,4500 <TARGET>

The ike-version script sends several IKE probes, including Main Mode and Aggressive Mode requests, with different transform combinations.

A successful response might look like:

PORT    STATE SERVICE
500/udp open  isakmp
| ike-version:
|   vendor_id: Fortinet FortiGate
|   attributes:
|     Dead Peer Detection v1.0
|_    XAUTH

This already provides useful information about the exposed VPN service.

However, there are several important limitations.

Common pitfalls when detecting IKE

IKE is not always easily fingerprinted and detectable. Here are some reasons why.

UDP scanning is inherently unreliable

Unlike TCP, UDP does not provide a handshake that can reliably establish whether a service is listening. An unresponsive UDP port can be reported as open|filtered. This does not necessarily mean that no service exists. Firewalls may silently drop probes, rate-limit requests or only allow traffic from specific source addresses.

An IKE server may reject your proposal

An IKE server can be operational but reject all cryptographic proposals offered by your scanner. Depending on the implementation, it may return NO-PROPOSAL-CHOSEN rather than a successful handshake. This is still valuable evidence though, as the target is at least responding to IKE requests.

Some VPN gateways require a specific group ID

Certain gateways require clients to provide a configured VPN group name before they will negotiate. Without the correct group ID, the gateway may reject every proposal, even if the cryptographic parameters are otherwise correct.

Some gateways even respond with a fake PSK hash during an IKEv1 Aggressive Mode connection, even if the provided group name was wrong. This implementation aims at preventing group ID bruteforcing by responding with random, fake, non-crackable PSK hashes.

UDP/4500 might be the only externally accessible port

Some environments expose NAT-T while UDP/500 is filtered or not forwarded. A scanner that only checks UDP/500 will miss these services.

Source port restrictions

Some VPN implementations expect incoming IKE requests to originate from UDP/500. Probes originating from random source ports may be silently ignored.

Different IKE versions require different probing strategies

A scanner that only tests IKEv1 cannot reliably identify modern IKEv2-only deployments. Likewise, an IKEv1 transform enumeration result does not tell you which IKEv2 proposals are supported. IKE discovery is therefore more complicated than simply scanning two UDP ports.

Auditing IKE with ike-scan

One of the best-known tools for discovering and fingerprinting IKE services is ike-scan. It is available in Kali Linux and can also be installed on common Linux distributions.

Its primary functionality includes:

  • Discovering IKE servers.
  • Enumerating IKEv1 Phase 1 transforms.
  • Testing Main Mode and Aggressive Mode.
  • Displaying IKE Vendor IDs.
  • Fingerprinting implementations using retransmission backoff patterns.

A basic scan looks like this:

sudo ike-scan <TARGET>

To test Aggressive Mode:

sudo ike-scan -M -A <TARGET>

To display retransmission backoff information:

sudo ike-scan --showbackoff <TARGET>

To inspect the Vendor ID information:

sudo ike-scan -M <TARGET>

The tool sends crafted IKE negotiation requests and analyzes the responses.

Understanding NO-PROPOSAL-CHOSEN

A particularly important response during IKE auditing is:

0 returned handshake; 1 returned notify

Notify message 14 (NO-PROPOSAL-CHOSEN)
ike-scan fails to audit an IKE server

This means that the server responded to the IKE request but did not accept the offered cryptographic proposal. It does not mean that the IKE service is unavailable.

The problem is that ike-scan is a relatively old tool with legacy defaults, which could not provide the required cryptographic transforms to trigger a valid negotiation.

The Problems of IKE-SCAN

Ike-scan's traditional IKEv2 proposal does not adequately cover the modern cryptographic configurations used by many contemporary VPN gateways. Modern deployments may exclusively accept combinations involving AES-GCM, SHA-2 PRFs and elliptic-curve Diffie-Hellman groups.

Consequently, a server can support IKEv2 perfectly well while rejecting the proposals sent by an older scanner like ike-scan. This problem is documented in ike-scan Pull Request #47, which proposes modernizing the default IKEv2 proposals.

Another limitation is that ike-scan does not enumerate IKEv2 transforms. Its --trans functionality is intended for IKEv1 Phase 1 negotiation only. This means that even a successful IKEv2 discovery does not automatically provide a complete picture of the cryptographic configuration.

IKEv2 available but ike-scan cannot obtain handshake

Understanding IKEv1 Aggressive Mode

One of the most important security issues identified during IKE assessments is the combination of IKEv1, Aggressive Mode and Pre-Shared Key (PSK) authentication.

Aggressive Mode was designed to reduce the number of messages required during Phase 1 negotiation. However, unlike Main Mode, it does not provide identity protection during the initial exchange.

When PSK authentication is used, the responder's exchange exposes information that allows an attacker to perform offline dictionary or brute-force attacks against the pre-shared key.

This means an attacker does not need to continuously interact with the VPN server while guessing passwords. Once the relevant authentication material has been captured, password guessing can be performed offline.

A successful recovery of the PSK can have serious consequences, depending on the VPN configuration and the privileges associated with the connection. In the worst case, the attacker can connect to the IKEv1 VPN and gain unauthorized access to internal networks.

Why IKE Fingerprinting Is Difficult

Identifying an IKE responder is only the first step.

Determining the underlying VPN implementation and its actual configuration is considerably more complicated.

Unlike services such as SSH or HTTP, IKE does not necessarily expose an easily identifiable software banner.

Fingerprinting depends on several different characteristics:

  • Supported cryptographic transforms.
  • Vendor ID payloads.
  • Retransmission behavior.
  • IKE version and negotiation behavior.
  • Required VPN group IDs.
  • Source port handling.
  • Vendor-specific notification messages.
  • Network stack characteristics.

Furthermore, the supported transforms are not necessarily fixed characteristics of the VPN implementation. They depend heavily on the configuration of the backend VPN gateway.

A FortiGate appliance, for example, may support several encryption algorithms internally while only accepting a small subset configured by its administrator.

Therefore, transform enumeration alone cannot reliably identify a specific appliance or software version.

Vendor ID fingerprinting

IKE messages can contain Vendor ID (VID) payloads.

These payloads allow implementations to advertise vendor-specific capabilities and identify certain software implementations.

A scanner can compare received Vendor IDs against a database of known identifiers. For example, a response may contain a Vendor ID associated with Fortinet, Cisco or Check Point. This can provide useful information about the underlying VPN implementation.

However, Vendor IDs are not reliable proof of product identity. They can be omitted, modified or spoofed, and different implementations may reuse compatible vendor identifiers.

Treat VID fingerprinting as an indication rather than definitive identification.

UDP backoff fingerprinting

Another interesting fingerprinting technique is UDP retransmission backoff analysis.

IKE uses UDP, which does not guarantee delivery. Consequently, implementations need to retransmit messages when they do not receive a response. Different IKE implementations use different retransmission intervals and backoff strategies.

A simplified example might look like this:

Packet 1: 0 seconds
Packet 2: 2 seconds
Packet 3: 4 seconds
Packet 4: 6 seconds
Packet 5: 10 seconds
Packet 6: 14 seconds

The scanner records the arrival times and calculates the intervals between retransmissions. These intervals form a backoff pattern that can be compared against known implementation fingerprints. For example, certain older Check Point implementations have recognizable retransmission patterns.

There are limitations:

  • Packet loss can invalidate the fingerprint.
  • Firewalls and NAT devices can influence packet delivery.
  • Network latency can distort timing measurements.
  • Only responders that retransmit handshake responses can be fingerprinted this way.
  • A single notification response does not provide a retransmission pattern.

Nevertheless, backoff fingerprinting can be a useful supplementary technique when identifying otherwise difficult-to-classify IKE services.

Introducing IKESS – IKE Security Scanner

To simplify IKE discovery and security auditing, I have developed IKESS (IKE Security Scanner).

IKESS is a Python 3 wrapper around ike-scan that automates the discovery, enumeration, fingerprinting and security assessment of IKE VPN servers.

Instead of manually constructing different IKE proposals and interpreting the responses, IKESS performs a structured series of probes and generates a comprehensive security report.

Features

IKESS currently provides the following functionality:

  • Detects IKEv2 VPN servers.
  • Detects IKEv1 VPN servers and Aggressive Mode.
  • Enumerates supported IKEv1 transforms (ENC, HASH, AUTH and GROUP).
  • Saves results as JSON, XML and HTML reports.
  • Risk-rates findings and provides a security summary with recommendations.
  • Supports fingerprinting through Vendor IDs (VID).
  • Supports optional fingerprinting through UDP backoff patterns.
  • Supports probing over NAT-T / UDP/4500.
  • Detects NAT-T advertisement through the RFC 3947 Vendor ID on UDP/500 and automatically probes UDP/4500 to confirm NAT-T responsiveness.
  • Supports IKEv1 Aggressive Mode group VPN IDs through --id.
  • Optionally tests common VPN group IDs using a curated wordlist through --idfile.
  • Automatically switches to UDP/4500 discovery when UDP/500 is silent, allowing NAT-T-only responders to be detected and enumerated.

Installation and Usage

IKESS requires Python 3 and the ike-scan binary. It must be executed with root privileges.

Clone the repository:

git clone https://github.com/l4rm4nd/IKESS.git
cd IKESS

Run a scan against an IKE server:

sudo python3 ikess.py <TARGET>

That's it.

IKESS performs the discovery and enumeration process automatically and generates an HTML report containing the identified security issues, supported transforms, protocol versions and fingerprinting information.

The HTML report provides an overview of the VPN server's security posture, including findings and recommendations.

Enumerating Weak Cryptographic Configurations

One of the most useful capabilities is the automatic identification of weak cryptographic algorithms.

For example, if a gateway still supports IKEv1 with DES, 3DES, MD5, SHA1 or weak Diffie-Hellman groups, these configurations can be identified and reported.

IKESS also highlights problematic authentication configurations, particularly the combination of:

  • IKEv1
  • Aggressive Mode
  • Pre-Shared Key (PSK) authentication

This combination is particularly dangerous because Aggressive Mode exposes authentication material that can be used for offline password-guessing attacks.

The report helps distinguish individual weak transforms from more serious protocol-level weaknesses.

IKESS Security Dashboard

Custom Transform Enumeration

IKESS also supports custom cryptographic proposals.

For example, you can restrict the tested algorithms to specific encryption, hash, authentication and Diffie-Hellman groups:

sudo python3 ikess.py <TARGET> \
  --enc AES128,AES256,3DES \
  --hash SHA1,SHA256 \
  --auth PSK,RSA \
  --group G2,G14

This is useful when testing a particular security baseline or validating a VPN configuration against a predefined set of accepted algorithms.

For broader IKEv1 transform enumeration, IKESS provides:

sudo python3 ikess.py <TARGET> --fullalgs

This tests the full supported combination set and can generate considerably more traffic.

NAT-T Support

IKESS takes special care to detect servers that only respond over UDP/4500.

By default, it first probes UDP/500. If the server advertises NAT-T support, IKESS additionally checks UDP/4500 to confirm that the service responds there. If UDP/500 is completely silent, IKESS also attempts NAT-T discovery over UDP/4500.

This is particularly useful when assessing firewall appliances where only UDP/4500 is externally forwarded.

You can also explicitly force NAT-T probing:

sudo python3 ikess.py <TARGET> --nat-t

VPN Group ID Discovery

Another interesting feature is the ability to supply an IKEv1 group VPN ID.

Some gateways, including certain Cisco ASA, SonicWall, FortiGate and WatchGuard configurations, require a specific group name before they will negotiate with an IKEv1 Aggressive Mode client.

Without the correct identifier, the gateway may simply return NO-PROPOSAL-CHOSEN for every proposal.

IKESS supports supplying a known group ID through:

sudo python3 ikess.py <TARGET> --id <GROUP_ID>

For authorized assessments where the group name is unknown, IKESS also supports testing a curated list of common group identifiers using --idfile and the pre-generated wordlist from wordlists/group_ids.txt.

A scan may look like this:

sudo python3 ikess.py <TARGET> --idfile wordlists/group_ids.txt

This functionality is particularly useful when an otherwise responsive IKEv1 gateway refuses to negotiate because it requires a configured group identifier.

🚨
A successful-looking response does not necessarily prove that the supplied group ID is valid. Some firewall implementations deliberately return synthetic or misleading IKEv1 responses to mitigate Aggressive Mode PSK hash collection. Cisco VPN gateways, for example, may return fake hash material for invalid group IDs. Results therefore need to be interpreted carefully and, where possible, validated against the actual VPN configuration.

IKESS Attack Labs

The IKESS repository includes example and intentionally vulnerable Dockerized VPN server configurations in its test directory.

These environments can be started locally to test IKESS against actual IKE implementations without interacting with production VPN infrastructure.

From the IKESS repository's root, start a scenario using:

docker compose -f test/01-ikev1-aggressive/docker-compose.yml up --build

01-ikev1-aggressive

🏆
Demonstrate IKEv1 Aggressive Mode with PSK authentication. This scenario does not require a gateway-specific group ID and accepts an AES-256/SHA-1/PSK/DH14 proposal.

This lab is useful for demonstrating why IKEv1 Aggressive Mode with PSK authentication is considered insecure. Authentication material (PSK hash) can be collected and subjected to offline password guessing.

Scan the gateway:

python3 ikess.py 127.0.0.1 --sport 0

IKESS will audit the server and retrieve the IKEv1 PSK hash:

IKESS audits IKEv1 with Aggressive Mode and leaks the PSK hash
IKESS HTML Dashboard with results and raw PSK hash

Afterwards, one can try to crack the PSK hash using psk-crack. Just copy the raw hash from the HTML dashboard in a separate file, prepare your wordlist to use and run:

# dump psk hash into file
echo -n "c30a5247f55ea923fb40af634b3a8c767b7789da0dac45b1356d0a067c553046225aa09b57618cede0d737d7f4be67220ecd88798efac46b2610de1005889bfa2d4f27ec328a32d00622066adca07be2a6e5e408e6996fa2a30b7c197a823906dbc694f5aa132da8d6b10bad5450481a6cfbce89849ca631360877ab6c6954c2c68f576451a4424368e770e427545147bc54e672da422699574abb38b250baf9d857233b51996428dadbc776c5bfc7eac70f88852f1022fcdb8c5b1ce9820b5767f95bd5e343d96e06f47b485ba30896d7b6fa0023ad6e2c509b0e09ff651a877feaeb56f6a7b803252f40fc0c51938913a3cd7a6c4e2d8e4aaf79c72c972faa:a8efc999e92d92c4ab10f323c73a7978a23865dea90c269cb40005f8fc863a03f659e9e3c30dcb2d42772fa3e0f4b516dc5441fe752ee5b2f51717409d4e362f5356a267e7c68487a9abe71e0be3b8ba05818143d0cff77081dc5862f2bae69b6fe2491863ac2566699c518f6a0798a9c217e36ba75d44e509f1ec17657855deaae0e32ebf16e326a7aaf4e6878fe8d6abe73c1b577e58598719ccee2446e1fad989daa09fd9a7e09440c091ac9226999184a7187b4c518208dd36fdca78a6558bd6238a198366b4202925868765cee2639c96b4dbabb00878afe8f8dc09b9fd3f2403b137de07819710fd5560d86a5777c6691da37f19e607444ddc9ed1fe03:00ad7c99be16f434:430ec46db12025f5:0000000100000001000000300101000100000028010100008001000780020002800300018004000e800e0100800b0001000c000400007080:010000007f000001:a0d5dcc20acc6302b568954f458eb4afb122cde3:ac73e92c6f9968fc361bc18bec7d287d92c3799154f24f7e35b1ceace8a9eb33:13e8ea8045e4b54198ef28e8a89822021f85f0a5" > psk.hash

# crack the hash
psk-crack -d /path-to-wordlist.txt psk.hash

Cracking the PSK Hash

🚨
If you have not changed the compose file nor environment variables, the default PSK passphrase is ikess-test-only-change-this-psk-2026.

You typically won't find it in a public wordlist, so craft your own for testing purposes or just define a different PSK passphrase via the .env.example or VPN_IPSEC_PSK environment variable directly and rebuild + restart the docker image.

Remember, it's just a testing lab 😉

02-ikev1-ikev2-aes256

🏆
Test a VPN gateway supporting both IKEv1 Main Mode and IKEv2, with IKEv1 restricted to a single AES-256/SHA-256/PSK/DH14 proposal.

This scenario demonstrates how IKESS identifies multiple IKE versions and how custom transform enumeration can identify a narrowly configured cryptographic proposal.

python3 ikess.py 127.0.0.1 --sport 0

Note: Unfortunately, as the official ike-scan uses deprecated default proposals for IKEv2, those transforms cannot be enumerated by IKESS until my PR is reviewed and merged. It's not a limitation of IKESS but of ike-scan due to its age. If you use the IKESS Docker image, my forked ike-scan with full support will be used.

03-ikev1-ikev2-default

🏆
Test IKEv1 Main Mode and IKEv2 using the VPN implementation's default IKE proposal set.

Unlike the previous scenario, this gateway retains the upstream default proposal configuration. This allows you to compare a narrowly restricted VPN configuration against a more typical default configuration.

python3 ikess.py 127.0.0.1 --sport 0

Note: Unfortunately, as the official ike-scan uses deprecated default proposals for IKEv2, those transforms cannot be enumerated by IKESS until my PR is reviewed and merged. It's not a limitation of IKESS but of ike-scan due to its age. If you use the IKESS Docker image, my forked ike-scan with full support will be used.

04-ikev1-aggressive-group-vpn

🏆
Demonstrate an IKEv1 Aggressive Mode gateway that requires a specific VPN group ID.

The configured group ID is vpn. Without this identifier, the gateway will not negotiate the expected Aggressive Mode connection.

First, provide the known group ID explicitly:

python3 ikess.py 127.0.0.1 --sport 0 --id vpn

Alternatively, let IKESS test its curated group ID wordlist:

python3 ikess.py 127.0.0.1 --sport 0 --idfile wordlists/group_ids.txt

This demonstrates how group ID discovery can help identify otherwise difficult-to-enumerate IKEv1 Aggressive Mode configurations.

Note: The PSK hash is also leaked but as the VPN server uses a somewhat modern transform with AES256 and SHA256, there is currently no support in cracking the hash using official hashcat or psk-crack. Maybe time for another PR?

05-ikev1-3des-only

🏆
Demonstrate a legacy VPN gateway that exclusively supports 3DES encryption with SHA-1, PSK authentication and DH Group 14.

This scenario is particularly useful for demonstrating how IKESS identifies deprecated cryptographic algorithms and highlights weak VPN configurations.

python3 ikess.py 127.0.0.1 --sport 0

06-ikev2-only

🏆
Test an IKEv2-only VPN gateway.

IKEv1 is disabled, while IKEv2 remains available. This is a useful negative test to verify that IKESS distinguishes between the two protocol versions instead of incorrectly identifying an IKEv1 service.

Run a regular scan:

python3 ikess.py 127.0.0.1 --sport 0

Note: Unfortunately, as the official ike-scan uses deprecated default proposals for IKEv2, those transforms cannot be enumerated by IKESS until my PR is reviewed and merged. It's not a limitation of IKESS but of ike-scan due to its age. If you use the IKESS Docker image, my forked ike-scan with full support will be used.

07-ikev1-transform-matrix

🏆
Demonstrate a VPN gateway with multiple IKEv1 proposals, including both modern and deprecated cryptographic configurations

The configured proposals include:

  • AES-256 / SHA-256 / DH14
  • AES-128 / SHA-1 / DH5
  • 3DES / SHA-1 / DH14

This scenario is particularly interesting because it demonstrates that a VPN gateway can support strong cryptographic algorithms while simultaneously retaining weaker legacy alternatives.

Run an extended transform enumeration:

python3 ikess.py 127.0.0.1 --sport 0 --fullalgs

Hardening IKE VPN Services

IKE security is frequently overlooked because VPN services are often enabled through a graphical interface on firewall appliances.

Administrators can sometimes enable remote-access VPN functionality with only a few clicks. Unfortunately, the resulting configuration may retain compatibility settings for older VPN clients and protocols.

During penetration tests, it is not uncommon to encounter externally accessible VPN gateways that still support:

  • IKEv1
  • IKEv1 with Aggressive Mode
  • Insecure and deprecated algorithms like DES, 3DES, MD5 and SHA1
  • Weak Diffie-Hellman groups such as MODP768 and MODP1024
  • PSK authentication using human-readable passwords
  • Legacy VPN group identifiers

These configurations should be reviewed and modernized.

For new deployments, the preferred configuration should be based on IKEv2.

A sensible modern baseline includes:

Component

Recommended configuration

IKE version

IKEv2 only

Encryption

AES-GCM-128 or AES-GCM-256

PRF

HMAC-SHA-256 or stronger

Integrity

Integrated AEAD authentication with AES-GCM

Diffie-Hellman

ECP Group 19 or stronger

Authentication

X.509 certificates / ECDSA / RSA

ESP encryption

AES-GCM

PFS

Enabled

IKEv1

Disabled

Aggressive Mode

Disabled

For environments where AES-GCM is unavailable, AES-CBC with a separate SHA-2 integrity transform is a reasonable compatibility alternative. Do not configure a separate integrity algorithm alongside AES-GCM in the same proposal. GCM already provides authenticated encryption. The exact supported configuration should be validated against the relevant VPN implementation and current interoperability requirements.

Avoid Weak PSKs

Pre-shared Keys (PSK) should not be ordinary passwords or short group secrets.

They should be generated using a cryptographically secure random generator and contain sufficient entropy to resist offline guessing. Where possible, use certificate-based authentication or an appropriately configured EAP-based authentication mechanism instead.

IKEv2 does not automatically make weak PSKs safe. PSK-based IKEv2 authentication also has security considerations, particularly when secrets have insufficient entropy.

Disable Legacy Protocols and Algorithms

If there is no operational requirement for IKEv1, disable it completely.

If there is still need for IKEv1, ensure to disable the IKEv1 Aggressive mode.

Finally, it is recommended to disable these:

  • DES
  • 3DES
  • MD5
  • SHA1 where modern alternatives are supported
  • Weak MODP Diffie-Hellmann (DH) groups
  • Aggressive Mode with PSK authentication.

Review both the IKE SA proposals and the Child SA / ESP proposals.

Conclusion

IKE/IPsec remains an important part of modern enterprise network infrastructure.

Its widespread deployment, integration into firewall appliances and complex negotiation behavior make it an interesting and frequently overlooked attack surface during penetration tests.

Discovering an IKE service is only the beginning. Proper assessments need to consider protocol versions, supported transforms, NAT-T behavior, vendor fingerprints, VPN group identifiers and authentication mechanisms.

Traditional tools such as ike-scan remain useful, but their legacy defaults and limited IKEv2 capabilities can result in incomplete assessments. IKESS aims to make this process considerably easier.

By combining IKE discovery, IKEv1 transform enumeration, Aggressive Mode detection, NAT-T probing, fingerprinting and automated security reporting, IKESS provides a practical starting point for auditing externally exposed IPsec VPN gateways.

If you regularly perform external penetration tests, give it a try:

GitHub - l4rm4nd/IKESS: A Python3 Script for Auditing IKE VPN Servers
A Python3 Script for Auditing IKE VPN Servers. Contribute to l4rm4nd/IKESS development by creating an account on GitHub.

Cheers!


LRVT
Hey!If you enjoy my online content, please feel free to buy me a coffee or slice of pizza. Many thanks!
Sponsoring
I firmly believe in the importance of open access to knowledge and resources. That’s why all my blog content and online resources are freely available to everyone, fostering a space for learning and knowledge sharing without barriers. While this blog is not restricted by a paywall, I sincerely appreciate