Tip

Checklist: Measuring patch management metrics

At this point, most organizations of any size have implemented some type of patch management program -- usually including the associated tools. A smaller subset has taken things a step further and rolled out full-fledged vulnerability management programs. No matter where your company is in the process, it's important to consider how you'll measure the progress of your efforts.

The value of metrics and measuring performance is well established in the IT industry, though given patch management's relatively recent emergence, measurement is not as widely adopted in this space. To get a good handle on ROI and the effectiveness of your patch management program, you should be tracking and analyzing key performance indicators.

So, what are the indicators? From a broad perspective, you should track information in several areas (adjustable based on each particular environment). To keep things simple and applicable to multiple scenarios, we'll divide them into the following categories:

 Checklist: Measuring patch management metrics
Coverage. This metric category refers to the number or proportion of systems that any particular patch effort is able to cover. Coverage is one of the most important metrics,
since it relates directly to the amount of risk that exists and is addressed. An accurate asset management system produces the ideal baseline for this measurement, though ad hoc
scanning will also produce useful results. At a program level, you can use the coverage metric to track the number of systems and applications that are covered by any given patch
management tool (e.g., the percentage of systems with a patch agent installed, the percentage of systems using auto update features).
Effort. This measurement tracks the level of effort expended for each patch. Typical items covered in this category are the number of support calls a patch generates, the amount
of man-hours involved in deployment and the number of in-person visits required (manual intervention versus automated deployment).
Speed. There are several dimensions to the speed category. One metric tracks how quickly a patch is deployed after the vendor releases it. If the patch is a security update and
there is an exploit available, this can be referred to as an exposure or vulnerability window. Another component of this category measures the amount of time it takes to roll
out the patch in your organization. This is most often measured in increments -- for example, the amount of time it takes to patch 50%, 75% or 95% of affected systems.
Impact. Business owners show a lot of interest in this metric. Impact refers to the impact on your organization -- measured most often in terms of downtime and failures related to
patch deployment. Organizations with a mature risk management program can also measure impact in terms of revenue or similar means.
Quality. This is somewhat related to the effort category, but is important enough to break out on its own. Quality metrics include time spent testing (installation and backout plans)
and the percentage of successful installations.

Of course, each organization puts its own value on particular metrics. Regardless of the metrics you choose to measure, it's important to build appropriate reporting and tracking mechanisms into your patch management process. Doing so will move your organization forward not only in the patch management discipline, but also in overall risk and vulnerability management.

About the author: Jason Chan serves as a co-moderator for the patchmanagement.org mailing list. He has written articles for publications such as SysAdmin Magazine and Computer Fraud and Security. He has been a security consultant for over six years. In this role, Chan works with many of the world's largest enterprises to help ensure the security of their digital assets. Prior to consulting, he worked in the Information Warfare Engineering group with the U.S. Navy Space and Naval Warfare Center.

Ask Jason Chan your patch management questions


More information from SearchWindowsSecurity.com

  • Step-by-step Guide: Patch management must-do list
  • Article: Don't have a patch attack
  • Tip: Stay on top of automated patching


  • This was first published in October 2005

    There are Comments. Add yours.

     
    TIP: Want to include a code block in your comment? Use <pre> or <code> tags around the desired text. Ex: <code>insert code</code>

    REGISTER or login:

    Forgot Password?
    By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
    Sort by: OldestNewest

    Forgot Password?

    No problem! Submit your e-mail address below. We'll send you an email containing your password.

    Your password has been sent to:

    Disclaimer: Our Tips Exchange is a forum for you to share technical advice and expertise with your peers and to learn from other enterprise IT professionals. TechTarget provides the infrastructure to facilitate this sharing of information. However, we cannot guarantee the accuracy or validity of the material submitted. You agree that your use of the Ask The Expert services and your reliance on any questions, answers, information or other materials received through this Web site is at your own risk.