Configuring ISA Server to Secure SMTP Traffic

How to Configure ISA Server to Secure SMTP Traffic
ISA Server provides three components for securing SMTP traffic. The first is the Mail Server Wizard, which can be used to publish the SMTP server to the Internet. The second component is the SMTP Message Screener, which can help reduce the amount of unwanted e-mail entering the organization. The third component is the SMTP application filter, which can be used to block buffer-overflow attacks or SMTP command based attacks on Exchange Server.

Mail Server Wizard
You can use the Mail Server Wizard to make Exchange Server computers available to Internet clients. The Mail Server Wizard includes several options, one of which is publishing an SMTP server. When publishing an Exchange Server computer as an SMTP server, you create a server publishing rule that accepts SMTP traffic on the ISA Server computer’s external interface and forwards the packets to the Exchange Server computer.

SMTP Application Filter
ISA Server 2004 provides application-layer filtering to help prevent Internet attackers from using buffer-overflow commands to disable or take control of your computer running Exchange Server. The SMTP application-layer filter inspects the commands included in all incoming SMTP communications. You can configure the SMTP filter to limit the size of the SMTP command sequences as well to block specific commands.

SMTP Message Screener
The ISA Server 2004 SMTP Message Screener can be used to control incoming SMTP mail by performing application-layer inspection of all SMTP messages. The Message Screener can scan the messages and examine the attachments and then block or hold messages for later inspection.

You can configure the SMTP Message Screener to block or hold incoming or outgoing e-mail using the following parameters:
1- Source or destination e-mail domain
2- Source or destination e-mail address
3- Attachment size, file extension, or file name
4- Keywords in the mail subject or body
The SMTP Message Screener can block or hold messages sent from the internal network in the same way that it does for messages entering the network.

Configuring Intrusion Detection and IP Preferences

Intrusion-Detection Configuration Options
To protect your network, you will also need to know how to configure your ISA Server for intrusion detection. Intrusion detection identifies when an attack is attempted against your network and performs a set of configured actions, or alerts, in case of an attack. To detect potential attacks, ISA Server compares network traffic and log entries to well-known attacks. When ISA Server detects suspicious activities, it triggers an alert. You can configure the actions that ISA Server will perform in the event of an alert. These actions include connection termination, service termination, e-mail alerts, logging, and others.

Intrusion Detection at the Application Layer
ISA Server also provides built-in application filters that detect DNS networking protocol and Post Office Protocol (POP) intrusions. The DNS intrusion-detection filter detects the following known DNS exploits:

- DNS host name overflow A DNS host name overflow occurs when a DNS response for a host name exceeds a certain fixed length (255 bytes). Applications that do not check the length of the host names may overflow internal buffers on the server when copying this host name, allowing a remote attacker to execute arbitrary commands on a targeted computer. This filter inspects the response that an internal client receives from an external DNS server.

- DNS length overflow DNS responses for IP addresses contain a length field,which should be 4 bytes. By formatting a DNS response with a larger value, some applications executing DNS lookups will overflow internal buffers, potentially allowing a remote attacker to execute arbitrary commands on a targeted computer. This filter inspects the response that an internal client receives from an external
DNS server.

- DNS zone transfer A malicious user executes a zone transfer to gather a list of all the host names in a domain. This filter detects when an Internet user attempts to execute a zone transfer from an internal DNS server through ISA Server.

The POP filter intercepts and analyzes POP traffic destined for the published servers. The application filter checks for POP buffer overflow attacks. A POP buffer overflow attack occurs when a remote attacker attempts to gain root access to a POP server by overflowing an internal buffer on the server.

IP Preferences Configuration Options
Another option on ISA Server 2004 that you can use to improve security is to configure the IP preferences. IP preferences are used to configure how ISA Server will handle specific types of IP packets. Configuring IP preferences is more complicated than configuring intrusion detection because, in most cases, IP preferences can be used to block normal packets that may or may not be used by attackers. You can configure the following IP preferences on ISA Server:

- IP option You can configure ISA Server to refuse all packets that have the IP options flag set in the header, or you can configure ISA Server to drop packets with only specific IP options enabled. The IP options flags that are most commonly used by attackers are the source routing options. The source route option in the IP header allows the sender to override routing decisions that are normally
made by the routers between the source and destination machines. An attacker can use source routing to reach addresses on the internal network that normally are not reachable from other networks, by routing the traffic through another computer that is reachable from both the other network and the internal network. Because source routing can be used in this way, you should disable source routing on your ISA Server computer.

- IP fragments You can also configure ISA Server to drop all IP fragments. A single IP datagram can be separated into multiple datagrams of smaller sizes known as IP fragments. If you enable this option, then all fragmented packets are dropped when ISA Server filters packet fragments. A common attack that uses IP fragments is the teardrop. In the teardrop attack, multiple IP fragments are sent to a server. However, the IP fragments are modified so that the offset fields within the packet overlap. When the destination computer tries to reassemble these packets, it is unable to do so. It may fail, stop responding, or restart. Enabling IP fragment filtering can interfere with streaming audio and video. In addition, Layer Two Tunneling Protocol (L2TP) over IPSec connections may not be established successfully because packet fragmentation may take place during certificate exchange.

- IP routing When IP routing is enabled, ISA Server sends the original network packet from one network to another. ISA Server can filter the network packet. When IP routing is disabled, ISA Server sends only the data (and not the original network packet) to the destination. Also, when IP routing is disabled, ISA Server sends each packet through the firewall in user mode. Disabling IP routing is more secure, but can also decrease router performance.

Introduction to ISA Server as a Firewall

What Is Application-Layer Filtering?
Application-layer filtering enables the firewall to inspect the application data in a TCP/IP packet for unacceptable commands and data. For example, a Simple Mail Transport Protocol (SMTP) filter intercepts network traffic on Port 25 and inspects it to make sure the SMTP commands are authorized before passing the communication to the destination server. An HTTP filter performs the same function on all HTTP packets. Firewalls that are capable of application-layer filtering can stop dangerous code at the edge of the network before it can do any damage.

Advantages and Disadvantages of Application-Layer Filtering
Application-layer filtering can be used to stop attacks from sources such as viruses and worms. To the packet-filtering firewall, most worms look like legitimate network traffic. The headers of the packets are identical in format to those of legitimate traffic. It is the payload that is malicious; only when all the packets are put together can the worm be identified as malicious code, so these exploits often travel straight through to the private network because the firewall allowed what appeared to be legitimate application data.

But the advantages of application-layer filtering transcend the prevention of attacks. It can also be used to protect your network and systems from the harmful actions often taken by unaware employees. For example, you can configure filters that prevent potentially harmful programs from being downloaded through the Internet, or ensure that critical customer data does not leave the network in an e-mail.

Application-layer filtering can also be used to more broadly limit employee actions on the network. You can use an application filter to restrict common types of inappropriate communication on your network. For example, you can block peer-to-peer fileexchange services. These types of services can consume substantial network resources and raise legal liability concerns for your organization.

What Is Intrusion Detection?
Intrusion detection is a means of detecting when an attack against a network is attempted or in progress. If you detect an intrusion attempt early enough, you may be able to prevent a successful intrusion. If an intrusion does occur, you must be alerted as soon as possible to reduce the potential impact of the intrusion and to eliminate the vulnerability in your network security.

An intrusion-detection system (IDS) that is located at the edge of a network inspects all traffic in and out of the network and identifies patterns that may indicate a network or system attack. An IDS is usually configured with information about a wide variety of known attacks, then monitors the network traffic for signatures indicating that a known attack is occurring. An IDS can also be configured with information about normal network traffic and then be configured to detect variations from the normal traffic.

A complete IDS includes several layers. One device may be located at the network perimeter and monitor all traffic entering and leaving the network. Additional devices may be deployed on the internal networks, or on routers connecting networks. A final layer of protection is provided by host-based systems in which an IDS is configured on individual computers. The most sophisticated IDS can collect information from all the layers and correlate data to make the most accurate intrusion-detection decisions.

Intrusion-detection systems also provide options for configuring alerts or responses to intrusion attempts. At the very least, an IDS should alert an administrator when an attack is detected. More sophisticated IDSs provide additional responses to attacks, including shutting down or limiting the functionality of the systems under attack.

ISA Server and Intrusion Detection
ISA Server includes intrusion-detection functionality that monitors for several wellknown vulnerabilities. ISA Server detects intrusions at two different network layers. First, ISA Server detects intrusions at the network layer. This enables ISA Server to detect vulnerabilities that are inherent to the IP layer. Second, ISA Server uses application filters to detect intrusions at the application layer. You can use third-party application filters to add more intrusion detection or create your own application filters using the filter application programming interfaces (APIs) defined in the ISA Server software development kit (SDK).

MCP 70-350 : Introduction to ISA Server 2004

Lesson 1: Overview of ISA Server Functionality

ISA Server 2004 is a valuable component in an overall plan to secure an organization’s network. Because ISA Server is deployed at the connecting point between an internal network and the Internet, ISA Server’s role is critical. Almost all organizations provide some level of access to the Internet for its users. ISA Server can be used to enforce security policies dealing with the types of access users should have to the Internet. At the same time, many organizations also allow remote users some type of access to internal servers. For example, almost all organizations allow e-mail servers on the Internet to connect to internal e-mail servers to send Internet e-mail. Many companies also host internal Web sites, or want employees to be able to access internal resources
from the Internet. ISA Server 2004 can be used to ensure that access to these internal resources is secure.

How ISA Server Works—An Overview
ISA Server is designed to secure the perimeter of an organization’s network. In most cases, this perimeter is between the organization’s internal local area network (LAN) and a public network such as the Internet.
The internal network, or protected network, is usually located on an organization’s premises and is under the control of the organization’s IT staff. The internal network is considered to be relatively
secure; that is, normally only authorized users have physical access to the internal network.
Also, the IT staff have a great deal of control over the types of traffic that are allowed on the internal network.

An organization has no control over who is accessing the Internet or over the security of network traffic on the Internet. Anyone in the world with an Internet connection can locate and access any other Internet connection using almost any application or protocol. Also, network packets sent via the Internet are not secure because they can be captured and inspected by anyone running a packet sniffer on an Internet network segment. A packet sniffer is an application that can be used to capture and view all the network traffic on a network. In order to capture network traffic, the packet sniffer must be connected to a network segment located between two routers.

How ISA Server Works as a Firewall
A firewall is a device that is located between one segment of a network and another, and allows only authorized traffic to pass between the segments. The firewall is configured with traffic filtering rules that define the types of network traffic that will be allowed to pass through. A firewall may be positioned and configured to protect an organization from the Internet, or it may be positioned internally to protect specific sections of an organization’s corporate network.

In most cases, firewalls are deployed at the network perimeter. The primary purpose of a firewall in this configuration is to ensure that no traffic from a publicly accessible network like the Internet can enter an organization’s internal network unless it has been explicitly permitted. For example, the organization may have an internal Web server that needs to be accessible to Internet users. The firewall can be configured to allow Internet traffic to access only that Web server.

ISA Server 2004 provides firewall functionality. By default, when you deploy ISA Server, it will block all traffic between networks that are attached to the server, including internal networks, perimeter networks (also known as demilitarized zones, or DMZs), and the Internet. ISA Server 2004 uses three types of filtering rules to block or allow network traffic: packet filtering, stateful filtering, and application-layer filtering.

Packet Filtering
Packet filtering works by examining the header information for each network packet that arrives at the firewall. When the packet arrives at the ISA Server network interface, ISA Server opens the packet header and checks information such as the source and destination addresses and the source and destination ports. ISA Server compares this information against its firewall rules that define which packets are allowed. If the source and destination addresses are allowed, and if the source and destination ports are allowed, the packet passes through the firewall to the destination network. If the addresses and the ports are not explicitly allowed, the packet is dropped and not forwarded through the firewall.

Stateful Filtering
Stateful filtering uses a more thorough examination of the network packet to make decisions on whether to forward it or not. When ISA Server uses a stateful inspection, it examines the Internet Protocol (IP) and the Transmission Control Protocol (TCP) headers to determine the state of a packet within the context of previous packets that have passed through ISA Server, or within the context of a TCP session. For example, a user on the internal network may send a request to a Web server on the Internet. The Web server sends a reply to that request. When the reply packet arrives at the firewall, the firewall inspects the TCP session information that is part of the packet. The firewall
determines that the packet is part of a currently active session that was initiated by the internal user, so the packet is forwarded to the user’s computer. If a user from outside the network attempts to connect to a computer inside the organization’s network, the firewall determines that the packet is not part of a currently active session and the packet is dropped.

Application-Layer Filtering
ISA Server also uses application-layer filtering to determine whether a packet is allowed or not. Application-layer filtering examines the actual content of a packet to determine if the packet can be forwarded through the firewall. An application filter opens the entire packet and examines the actual data in it before making a forwarding decision. For example, a user on the Internet may request a page from the internal Web server using the Hypertext Transfer Protocol (HTTP) GET command. When the packet arrives at the firewall, the application filter inspects the packet and detects the GET command. The application filter checks its policy to determine if the GET command is allowed. In most cases, the GET command is allowed and the packet is forwarded to the internal Web server.

11 Deploying, Configuring, and Managing SSL Certificates

Renewing SSL Certificates :
Like any other public key certificate, each SSL certificate has a lifetime. At some point in the future, the certificate will expire. You should plan to renew the certificate three to six months prior to the expiration to ensure that there is no period during which the certificate is invalid.

The specific process you use for renewing the certificate will vary. If you are using a certificate issued by a public CA, the CA will provide a renewal process. If you are using a certificate issued by Certificate Services, you can renew the certificate by using Web enrollment, the Certificates snap-in, or the Web Server Certificate Wizard.

Configuring Firewalls :
Applications use a unique port number for SSL-protected communications. As a result,you must change your firewall configuration to allow the encrypted traffic.

There are two approaches to allowing SSL traffic through a firewall. The first approach is to open the firewall to allow all traffic with a designated port. The typical ports that various applications use for SSL are listed in Table 11.2. Although this will allow SSL sessions to be established through the firewall, the firewall will not be able to analyze the contents of the SSL-encrypted packets. As a result, the firewall will be able to use only the origin and destination of the packet to determine whether to let packets through.