Checklist: Measuring patch management metrics

At this point, most organizations of any size have implemented some type of patch management program -- usually including the associated tools. A smaller subset has taken things a step further and rolled out full-fledged vulnerability management programs. No matter where your company is in the process, it's important to consider how you'll measure the progress of your efforts.

The value of metrics and measuring performance is well established in the IT industry, though given patch management's relatively recent emergence, measurement is not as widely adopted in this space. To get a good handle on ROI and the effectiveness of your patch management program, you should be tracking and analyzing key performance indicators.

So, what are the indicators? From a broad perspective, you should track information in several areas (adjustable based on each particular environment). To keep things simple and applicable to multiple scenarios, we'll divide them into the following categories:

Checklist: Measuring patch management metrics

Coverage. This metric category refers to the number or proportion of systems that any particular patch effort is able to cover. Coverage is one of the most important metrics,

since it relates directly to the amount of risk that exists and is addressed. An accurate asset management system produces the ideal baseline for this measurement, though ad hoc

scanning will also produce useful results. At a program level, you can use the coverage metric to track the number of systems and applications that are covered by any given patch

management tool (e.g., the percentage of systems with a patch agent installed, the percentage of systems using auto update features).

Effort. This measurement tracks the level of effort expended for each patch. Typical items covered in this category are the number of support calls a patch generates, the amount

of man-hours involved in deployment and the number of in-person visits required (manual intervention versus automated deployment).

Speed. There are several dimensions to the speed category. One metric tracks how quickly a patch is deployed after the vendor releases it. If the patch is a security update and

there is an exploit available, this can be referred to as an exposure or vulnerability window. Another component of this category measures the amount of time it takes to roll

out the patch in your organization. This is most often measured in increments -- for example, the amount of time it takes to patch 50%, 75% or 95% of affected systems.

Impact. Business owners show a lot of interest in this metric. Impact refers to the impact on your organization -- measured most often in terms of downtime and failures related to

patch deployment. Organizations with a mature risk management program can also measure impact in terms of revenue or similar means.

Quality. This is somewhat related to the effort category, but is important enough to break out on its own. Quality metrics include time spent testing (installation and backout plans)

and the percentage of successful installations.

Of course, each organization puts its own value on particular metrics. Regardless of the metrics you choose to measure, it's important to build appropriate reporting and tracking mechanisms into your patch management process. Doing so will move your organization forward not only in the patch management discipline, but also in overall risk and vulnerability management.

About the author: Jason Chan serves as a co-moderator for the patchmanagement.org mailing list. He has written articles for publications such as SysAdmin Magazine and Computer Fraud and Security. He has been a security consultant for over six years. In this role, Chan works with many of the world's largest enterprises to help ensure the security of their digital assets. Prior to consulting, he worked in the Information Warfare Engineering group with the U.S. Navy Space and Naval Warfare Center.

Ask Jason Chan your patch management questions

More information from SearchWindowsSecurity.com