Negotiating IPSec Connections :
Unfortunately, IP was not originally designed with authentication or encryption in mind. As the internet grew and TCP/IP became the network protocol of choice, this unsecured form of communication became the standard. IPSec allows computers to continue using IP, while adding authentication and encryption.
However, most computers on IP networks today do not have IPSec enabled. As a result, computers with IPSec enabled are usually configured to politely ask remote computers to use IPSec to improve the security of the connection. If the two computers determine that they both have IPSec configured, and can agree upon a set of security standards, they can begin to use IPSec. This process is known as IPSec negotiation.
Not all IPSec negotiations are successful. Often the negotiations will fail because one of the two computers is not capable of using IPSec. Alternatively, the computers might not have the same security protocols enabled, which would mean that they wouldn’t be able to agree on a set of standards. In these cases, the computers will either revert to unprotected IP communications or determine that they will not communicate at all if they cannot use IPSec.
Internet Key Exchange (IKE) is the algorithm by which the first secure Security Association, or SA (a secure channel), is negotiated. IKE is a combination of the Internet Security Association Key Management Protocol (ISAKMP) and the Oakley Key Determination protocol and performs a two-phase negotiation: Main Mode and Quick Mode.
Main Mode
The initial long form of the IKE negotiation (Main Mode or Phase 1) performs the authentication and generates the master key material to establish an ISAKMP SA between machines. The result is referred to as an ISAKMP SA or an IKE SA. After the ISAKMP SA is established, it will remain in place for the period of time defined on the host computers—by default, it will last for 8 hours on computers running Windows. If data is actively being transferred at the end of the 8 hours, the Main Mode security association (SA) will be renegotiated automatically.
Main Mode negotiation occurs in three parts:
1. Negotiation of protection suites
2. Diffie-Hellman exchange
3. Authentication
Quick Mode
Quick Mode (also known as Phase 2) IKE negotiation establishes a secure channel between two computers to protect data. Because this phase involves the establishment of SAs that are negotiated on behalf of the IPSec service, the SAs created during Quick Mode are called the IPSec SAs. Two SAs are established, each with its own Security Parameter Index (SPI) label. One IPSec SA is used for inbound traffic, and the other is used for outbound traffic. During Quick Mode, keying material is refreshed or, if necessary, new keys are generated. A protection suite that protects specific IP traffic is also selected.
IPSec hosts will perform IKE Quick Mode negotiation on a regular basis to reduce the risk of an attacker using brute force methods to determine the keys used in the communications. Each renegotiation re-establishes two new IPSec security associations with new keys and SPIs. By default, computers running Windows will perform Quick Mode negotiation every hour (3600 seconds) or after 100 megabytes have been transferred.
Either side of the connection can start the renegotiation process. Therefore, the site that first reaches the defined session key limit will initiate renegotiation. Lesson 3 describes how to specify session key limits.
Authentication Header and ESP :IPSec can use two protocols: Authentication Header (AH) and ESP. The protocols canbe used either separately or together. AH provides data origin authentication, dataintegrity, and anti-replay protection for the entire packet, including the IP header andthe data payload carried in the packet. Naturally, AH does not provide protection forthe fields in the IP header that are allowed to change in transit, such as the hop count.AH does not encrypt data, which means it does not provide privacy. Attackers can readthe contents of packets if they can intercept them, but the packets cannot be modified.ESP is more commonly used than AH because it provides data origin authentication,data integrity, anti-replay protection, and the option of privacy. While AH and ESP canbe used together, you will use ESP alone in most circumstances. You should chooseAH over ESP only when the data and header in the packet need to be protected frommodification and authentication but not encrypted. You might do this if you have anintrusion detection system, firewall, or quality of service (QoS) router that needs toinspect the contents of the packet. Otherwise, take advantage of the privacy providedby encryption, and use ESP. If IPSec traffic must traverse a NAT server, you must useESP, because ESP is the only IPSec protocol that supports NAT-T.
IPSec in Windows :
IPSec is natively available and can be used to protect network communications for Windows 2000, Windows XP Professional, and Windows Server 2003. Additionally, a legacy client is available for Microsoft Windows NT 4.0, Windows 98, and Windows Millennium Edition (ME). You can download the legacy client from
Unfortunately, IP was not originally designed with authentication or encryption in mind. As the internet grew and TCP/IP became the network protocol of choice, this unsecured form of communication became the standard. IPSec allows computers to continue using IP, while adding authentication and encryption.
However, most computers on IP networks today do not have IPSec enabled. As a result, computers with IPSec enabled are usually configured to politely ask remote computers to use IPSec to improve the security of the connection. If the two computers determine that they both have IPSec configured, and can agree upon a set of security standards, they can begin to use IPSec. This process is known as IPSec negotiation.
Not all IPSec negotiations are successful. Often the negotiations will fail because one of the two computers is not capable of using IPSec. Alternatively, the computers might not have the same security protocols enabled, which would mean that they wouldn’t be able to agree on a set of standards. In these cases, the computers will either revert to unprotected IP communications or determine that they will not communicate at all if they cannot use IPSec.
Internet Key Exchange (IKE) is the algorithm by which the first secure Security Association, or SA (a secure channel), is negotiated. IKE is a combination of the Internet Security Association Key Management Protocol (ISAKMP) and the Oakley Key Determination protocol and performs a two-phase negotiation: Main Mode and Quick Mode.
Main Mode
The initial long form of the IKE negotiation (Main Mode or Phase 1) performs the authentication and generates the master key material to establish an ISAKMP SA between machines. The result is referred to as an ISAKMP SA or an IKE SA. After the ISAKMP SA is established, it will remain in place for the period of time defined on the host computers—by default, it will last for 8 hours on computers running Windows. If data is actively being transferred at the end of the 8 hours, the Main Mode security association (SA) will be renegotiated automatically.
Main Mode negotiation occurs in three parts:
1. Negotiation of protection suites
2. Diffie-Hellman exchange
3. Authentication
Quick Mode
Quick Mode (also known as Phase 2) IKE negotiation establishes a secure channel between two computers to protect data. Because this phase involves the establishment of SAs that are negotiated on behalf of the IPSec service, the SAs created during Quick Mode are called the IPSec SAs. Two SAs are established, each with its own Security Parameter Index (SPI) label. One IPSec SA is used for inbound traffic, and the other is used for outbound traffic. During Quick Mode, keying material is refreshed or, if necessary, new keys are generated. A protection suite that protects specific IP traffic is also selected.
IPSec hosts will perform IKE Quick Mode negotiation on a regular basis to reduce the risk of an attacker using brute force methods to determine the keys used in the communications. Each renegotiation re-establishes two new IPSec security associations with new keys and SPIs. By default, computers running Windows will perform Quick Mode negotiation every hour (3600 seconds) or after 100 megabytes have been transferred.
Either side of the connection can start the renegotiation process. Therefore, the site that first reaches the defined session key limit will initiate renegotiation. Lesson 3 describes how to specify session key limits.
Authentication Header and ESP :IPSec can use two protocols: Authentication Header (AH) and ESP. The protocols canbe used either separately or together. AH provides data origin authentication, dataintegrity, and anti-replay protection for the entire packet, including the IP header andthe data payload carried in the packet. Naturally, AH does not provide protection forthe fields in the IP header that are allowed to change in transit, such as the hop count.AH does not encrypt data, which means it does not provide privacy. Attackers can readthe contents of packets if they can intercept them, but the packets cannot be modified.ESP is more commonly used than AH because it provides data origin authentication,data integrity, anti-replay protection, and the option of privacy. While AH and ESP canbe used together, you will use ESP alone in most circumstances. You should chooseAH over ESP only when the data and header in the packet need to be protected frommodification and authentication but not encrypted. You might do this if you have anintrusion detection system, firewall, or quality of service (QoS) router that needs toinspect the contents of the packet. Otherwise, take advantage of the privacy providedby encryption, and use ESP. If IPSec traffic must traverse a NAT server, you must useESP, because ESP is the only IPSec protocol that supports NAT-T.
IPSec in Windows :
IPSec is natively available and can be used to protect network communications for Windows 2000, Windows XP Professional, and Windows Server 2003. Additionally, a legacy client is available for Microsoft Windows NT 4.0, Windows 98, and Windows Millennium Edition (ME). You can download the legacy client from
http://www.microsoft.com /windows2000/server/evaluation/news/bulletins/l2tpclient.asp.
