Use IPSec Policies

This excerpt from Chapter 11 of Roberta Bragg's "Hardening Windows Systems" describes how to use IPSec policies to provide encryption of communications between two computers, and manage and prevent connections.

Hardening Windows Systems Get a glimpse inside Roberta Bragg's book "Hardening Windows Systems" with this series of book excerpts. Below is the introductory excerpt from Chapter 11, "Harden Communications." Click for the complete book excerpt series or purchase the book.

Use IPSec Policies

IPSec is a security protocol built in to the Windows TCP/IP stack of Windows XP, Windows Server 2003, and Windows 2000. An IPSec policy can be configured and assigned that will protect communications by providing mutual computer authentication, encryption, integrity, protection from replay attacks, and message origination authentication. It is also widely used as a security protocol in VPNs. Its use in Windows-based VPNs is discussed in the later section "Use L2TP/IPSec VPNs."

Here are three major uses for IPSec in Windows LANs:

  • To provide encryption of communications between two computers

  • To manage connections on the basis of IP address and protocol used

  • To prevent connections to network resources from rogue computers

IPSec policies are created using Group Policy. A policy can be developed and assigned to a single computer at a time using the local group policy, or configured in a GPO linked to an OU or entire domain and thus implemented on any number of computers.

IPSec is a complex protocol, and to thoroughly understand and troubleshoot IPSec is beyond the scope of this book. A few simple facts, however, will allow you to write and use the simple policies outlined here. These facts are easier to understand by following the policy steps, but these are their basics:

  • A policy is composed of rules, filters, and filter actions.

  • Rules are composed of settings and a list of filters.

  • Filters specify source and destination IP addresses and protocols.

  • Filter actions determine what happens if a rule's filter is matched.

  • Possible filter actions are: Block, Permit, and Negotiate. Rules are often referred to by their filter action.

  • Each rule can have only one filter action; however, a policy may be composed of one or more rules.

  • In order for Allow and Negotiate policies to work, each computer involved must have an IPSec policy assigned.

  • IPSec policies are not in effect until the policy is assigned.

  • Policies may be scripted, or the IPSec Policy Wizard can be used.

  • Three methods of authentication are available. Kerberos (only in Windows domains), certificates (all computers must have certificates and must be able to validate them), preshared key (the weakest, but good for testing).

It is possible to create an IPSec policy that can so successfully shut down communications that recovery of the computer system may be a difficult chore. To prevent complications, always test an IPSec policy in a test environment and always start by implementing the policy on one test computer at a time, then moving to a test domain.

Use IPSec for Confidentiality

To protect communications between two computers, use an IPSec negotiation policy. The following steps outline how to build a policy that encrypts communication between computer A with an IP address of and computer B, which has an IP address of

1. Add the IP Security Policy Management snap-in to an MMC console on computer A.
2. Right-click the IP Security Policies on Local Computer container, as shown here, and select Create an IP Security Policy.
3. Click Next on the Welcome page.
4. Enter the name Encrypt1 for the policy and click Next.
5. Uncheck Activate the Default Response Rule.
6. Click Next; then click Finish.
7. On the Encrypt1 Rules page, click Add, as shown here, to add a new rule:

8. On the New Rule Properties IP Filter List page, click Add to create the filter list.

9. Enter Encrypt to name the filter list.
10. Uncheck the Use Add Wizard box and click Add to add a filter.
11. In the Source address drop-down list box, select A Specific IP Address.
12. Enter the IP address of computer B,
13. In the Destination address drop-down list box, select My IP address, as shown here:

14. Click OK to close the IP Filter Properties list page and click OK to close the IP Filter List page.
15. In the IP Filter List tab, select the Encrypt entry (the list you just created), as shown in the following illustration, and then click the Filter Action tab.

16. Click to deselect the Use Add Wizard button and click Add to add a filter action.
17. On the New Filter Action Properties page, select Negotiate Security.
18. Click Add to add a security method. The default selection, Integrity and Encryption, is acceptable. By default, 3DES and SHA1 are selected. Click OK.
19. Click Accept Unsecured Communication, But Always Respond Using IPSec, as shown here:

20. Select the General page and enter Negotiate for the Filter Action name; then click OK.
21. Select Negotiate on the Filter Action page.
22. Select the Authentication Methods page and click Add.
23. Select Use This String (Preshared Key). Enter a long, complex key and then click OK.
24. Select Kerberos in the Authentication Method Preference Order box and click Remove. Click OK to respond to the pop-up. Note in the following illustration that the shared key is partially visible in the interface.

25. Click Close twice to exit the policy.
26. Export the policy and import it on computer B, or re-create the policy on computer B and in both cases change the Source address to that of computer A.
27. On computer A, in the IPSec console, right-click the policy, and select Assign to assign the policy. Until you assign the policy, it is not in effect.
28. Repeat on computer B. (Don't forget to change the IP address you entered in step 11.)

Click for the next excerpt in this series: Use IPSec to Manage Connections.

Click for book details or purchase the book.

Dig Deeper on Enterprise desktop management

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.