Checklist: Measuring patch management metrics

How effective is your patch management program? In this checklist, patch management expert Jason Chan recommends the most important patch management metrics to track.

At this point, most organizations of any size have implemented some type of patch management program -- usually

including the associated tools. A smaller subset has taken things a step further and rolled out full-fledged vulnerability management programs. No matter where your company is in the process, it's important to consider how you'll measure the progress of your efforts.

The value of metrics and measuring performance is well established in the IT industry, though given patch management's relatively recent emergence, measurement is not as widely adopted in this space. To get a good handle on ROI and the effectiveness of your patch management program, you should be tracking and analyzing key performance indicators.

So, what are the indicators? From a broad perspective, you should track information in several areas (adjustable based on each particular environment). To keep things simple and applicable to multiple scenarios, we'll divide them into the following categories:

 Checklist: Measuring patch management metrics
Coverage. This metric category refers to the number or proportion of systems that any particular patch effort is able to cover. Coverage is one of the most important metrics,
since it relates directly to the amount of risk that exists and is addressed. An accurate asset management system produces the ideal baseline for this measurement, though ad hoc
scanning will also produce useful results. At a program level, you can use the coverage metric to track the number of systems and applications that are covered by any given patch
management tool (e.g., the percentage of systems with a patch agent installed, the percentage of systems using auto update features).
Effort. This measurement tracks the level of effort expended for each patch. Typical items covered in this category are the number of support calls a patch generates, the amount
of man-hours involved in deployment and the number of in-person visits required (manual intervention versus automated deployment).
Speed. There are several dimensions to the speed category. One metric tracks how quickly a patch is deployed after the vendor releases it. If the patch is a security update and
there is an exploit available, this can be referred to as an exposure or vulnerability window. Another component of this category measures the amount of time it takes to roll
out the patch in your organization. This is most often measured in increments -- for example, the amount of time it takes to patch 50%, 75% or 95% of affected systems.
Impact. Business owners show a lot of interest in this metric. Impact refers to the impact on your organization -- measured most often in terms of downtime and failures related to
patch deployment. Organizations with a mature risk management program can also measure impact in terms of revenue or similar means.
Quality. This is somewhat related to the effort category, but is important enough to break out on its own. Quality metrics include time spent testing (installation and backout plans)
and the percentage of successful installations.

Of course, each organization puts its own value on particular metrics. Regardless of the metrics you choose to measure, it's important to build appropriate reporting and tracking mechanisms into your patch management process. Doing so will move your organization forward not only in the patch management discipline, but also in overall risk and vulnerability management.

About the author: Jason Chan serves as a co-moderator for the patchmanagement.org mailing list. He has written articles for publications such as SysAdmin Magazine and Computer Fraud and Security. He has been a security consultant for over six years. In this role, Chan works with many of the world's largest enterprises to help ensure the security of their digital assets. Prior to consulting, he worked in the Information Warfare Engineering group with the U.S. Navy Space and Naval Warfare Center.

Ask Jason Chan your patch management questions


More information from SearchWindowsSecurity.com

  • Step-by-step Guide: Patch management must-do list
  • Article: Don't have a patch attack
  • Tip: Stay on top of automated patching


  • This was first published in October 2005

    Dig deeper on Patches, alerts and critical updates

    Pro+

    Features

    Enjoy the benefits of Pro+ membership, learn more and join.

    0 comments

    Oldest 

    Forgot Password?

    No problem! Submit your e-mail address below. We'll send you an email containing your password.

    Your password has been sent to:

    SearchVirtualDesktop

    SearchWindowsServer

    SearchExchange

    Close