ISA Server has one primary job: It protects your network against what could be classified as the most hostile environment imaginable -- the Internet. The good news is that ISA Server is relatively secure, give or take a few unchecked buffer vulnerabilities that were discovered and quickly patched. The bad news is that ISA Server security holes can often be blamed on security at the network interface level, which is why I've created this list of configuration don'ts to avoid when securing ISA Server.
Obviously, you should never run ISA Server as a domain controller or leave Windows unpatched -- but there is much more to ISA Server security than that. Assuming you haven't made any major security blunders at the Windows level, your biggest concern should be the network interface. When you install Windows on a server, the operating system assumes the server is being placed on a fairly safe corporate network. This assumption just doesn't work for ISA Server.
Don't keep all network components connected to the Internet
Most ISA Servers have two network interface cards (NICs). One of those NICs is connected to the corporate network; the other is often directly connected to the Internet. You really don't want Windows to assume that the NIC connecting the server to the Internet is being used to connect the server to a safe network.
In a Windows 2000 Server environment, several components are bound to a NIC by default. Typically, these components include the TCP/IP protocol, File and Printer Sharing for Microsoft Networks, the Client for Microsoft Networks and possibly a QoS Packet Scheduler. All of these components, except for the TCP/IP protocol, are intended for use on an internal network and have no business being bound to a NIC that's directly connected to the Internet. Do you really want to enable File and Print Sharing over the Internet? I recommend uninstalling all network components except the TCP/IP protocol for the NIC that's connected to the Internet.
Uninstalling these network components goes a long way to making your ISA Server more secure, but I recommend examining your TCP/IP configuration as well.
Don't enable NetBIOS over TCP/IP at the NIC level
When most people configure TCP/IP, they set it to get IP address information from a DHCP server and then call it a day. Few people take the time to look at the advanced settings. If you click the advanced settings button on the TCP/IP properties sheet, you will see the Advanced TCP/IP Settings properties sheet, which contains four basically empty tabs. Although these tabs may initially appear empty, take a closer look at the WINS tab. By default, it is set to enable NetBIOS over TCP/IP. NetBIOS over TCP/IP allows WINS clients to resolve host names and send or receive browser list service announcements. Users who connect to the machine over the Internet should not be doing any of those things.
Technically, ISA Server is configured by default to block NetBIOS over TCP/IP. You must remember, though, that the NIC configuration exists at a much lower level than ISA Server. This means that you are initially allowing NetBIOS requests to enter your organization from the outside world, and you're then filtering those requests at the ISA Server level. Your server will be more secure if you block those requests at the NIC level.
Don't forget to disable DNS default settings
Another major configuration issue exists at the DNS tab of the Advanced TCP/IP Settings properties sheet. By default, the Append Primary and Connection Specific DNS Suffixes option is enabled. Often malicious communications directed to a server use the HTTP header followed by the server's IP address. If, however, a request comes in using WWW in the http header, then Windows automatically attaches the domain name of the private network to it. For example, if a request arrived at my ISA Server in the form of HTTP://www, then Windows would automatically append my domain name (brienposey.com) to the request, turning it into http://www.brienposey.com.
At first this sounds harmless, but if you stop and think about it, my ISA Server would likely be configured with a Web publishing rule that includes an entry for brienposey.com. Therefore, the malicious request would pass right through ISA Server. Sure the person making the request would just be directed to my Web site, but that's not really the point. The point is that someone is entering an unqualified request that Windows qualifies and forwards. Such a request could be structured in a way that allows an attacker to gain access to something other than a Web site.
To get around this problem, select the Append These DNS Suffixes option rather than the Append Primary and Connection Specific DNS Suffixes option. You are then free to enter a bogus DNS name, such as fake.com. Now if someone were to send an unqualified DNS request, they would be directed to a non-existent domain rather than to your real domain.
Overall, ISA Server is a fairly secure product, but to really lock it down, you need to make sure no one can exploit Windows at the NIC level.
Brien Posey is a regular contributor to SearchWindowsSecurity.com.
For More Information:
Find out why other experts advice against using ISA Server to browse the Internet.
Check out the Best Web Links on ISA Server security.
Get help accessing the remote VPN through ISA Server.
