Rootkit detection and malware removal FAQ

This malware FAQ guide provides several tips and tools on rootkit detection, prevention and removal.

For years, IT administrators have had to keep up with the ever-evolving threats to Windows systems in the enterprise. Attacks range from blue-screen annoyances and proof-of-concept attacks to keyloggers and spyware designed to steal critical business data. Keep your desktops, networks and mobile devices secure by looking at the following expert tips on rootkit protection. This knowledge should be part of desktop admin toolboxes, along...

with antivirus tools, passwords, rootkit detection and removal best practices and more.

What is a rootkit?

Rootkits are tools or programs that enable administrator-level access to a computer or network. They can provide access to the BIOS of a system and aren't always designed with malicious intent, but BIOS rootkit attacks can require a hard drive to be wiped and reimaged.

In fact, rootkits can come from surprising sources, and they may be as troublesome as viruses and spyware. Security expert Mark Russinovich of Sysinternals once found that a digital rights management (DRM) component on a Sony audio DC had installed a rootkit on his own computer.

"This creates opportunities for virus writers," said Mikko Hypponen, director of antivirus research at Finnish firm F-Secure Corp. "These rootkits can be exploited by any malware, and when it's used this way, it's harder for firms like ours to distinguish the malicious from the legitimate."

In addition to 64-bit Windows systems, virtual machines and smartphones have been proven to be vulnerable to rootkits, so it's a good idea to learn and follow best practices for rootkit detection, as well as rootkit removal.

How can I find a rootkit?

Rootkit technologies have cropped up in a variety of places, including commercial security products and seemingly benign, third-party application extensions. Finding and removing rootkit installations is not an exact science because rootkits can be installed on a computer in many ways. No single tool can correctly identify all rootkits and rootkit-like behavior.

To determine if a rootkit is operating behind the scenes, use a system process analyzer such as Sysinternals' ProcessExplorer or, better yet, a network analyzer. You'll likely be surprised to find what programs are doing and what's going in and out of your network adapter. You may also discover that you simply have an overtaxed system running with too little memory or a severely fragmented hard drive. Check your system configuration, and run a rootkit scan.

  1. Search your system memory. Monitor all ingress points for a process as it is invoked, keeping track of imported library calls (from Dynamic Link Libraries) that may be hooked or redirected to other functions, loading device drivers, etc. The drawback to this approach to rootkit detection is that it is tedious, time-consuming and cannot account for all possible avenues in which a rootkit can be introduced into the system.
  2. Seek the truth -- expose API dishonesty. One rootkit detection application for Windows was the RootkitRevealer by Windows security analysts Bryce Cogswell and Mark Russinovich. This lightweight binary scouted file system locations and registry hives, looking for information kept hidden from the Windows application programming interface (API), the Master File Table and directory index. In addition, Jamie Butler, author of Subverting the Windows Kernel: Rootkits, created a tool called VICE, which systematically hunted down hooks in APIs, call tables and function pointers.
  3. Keep abreast of the latest antivirus and malware protection software. Security vendors such as F-Secure offer standalone rootkit detection tools. Even Microsoft has implemented rootkit detection features in its own Malicious Software Removal Tool. It's important to choose the best scanning tool for rootkit detection as part of your overall security defenses, but you may also need to conduct manual searches.
  4. Update your firewall protection. Remember, for a potential attack to be effectively concealed, the hacker must be able to get back into a machine once it has been compromised. Although firewalls do nothing to mitigate application-level risks, they can pose a significant challenge to attackers when they prohibit re-entry into a victim machine.
  5. Harden your workstation or server against attack. This proactive step prevents an attacker from installing a rootkit in the first place. The National Security Agency publishes guidelines for hardening Windows environments.

How can I remove rootkits?

Rootkits are relatively easy to install on victim hosts. To upload a rootkit, a determined attacker can do everything from exploit a Windows vulnerability to crack a password or even obtain physical system access. They can even execute a phishing attack, where a hacker cons a user into running an executable file in an email attachment or via a hyperlink distributed via email or instant messaging. Once they're in place, rootkits aren't so easy to get rid of.

Since the rootkit threat is not as widespread as viruses and spyware, removing rootkits is largely a reactive process. Once you've found the malware, you'll need to clean up contaminated Windows files and double-check after removing rootkits.

What are some rootkit removal tools?

Microsoft tools, such as Windows encryption, Microsoft Security Essentials and BitLocker, can help with rootkit removal. In addition, Windows 8 will include strengthened security features, according to Microsoft.

In addition to the Sysinternals and F-Secure products mentioned above, there are several other third-party security suites for removing rootkits from Windows systems.

For example, Sophos Anti-Rootkit has an install routine, and you have to manually run the executable afterwards. It allows for more user interactivity than BlackLight, but it is slower to scan your system. Another rootkit scanner is Rootkit Hook Analyzer. Try all of them to see which one(s) best suit your needs.

Why are backups necessary for rootkit removal?

Don't forget that proper backups will make it easier to restore systems if needed after rootkit and botnet removal. Removing a rootkit with cleaning tools may actually leave Windows in an unstable or inoperable state, depending on which files were infected and subsequently cleaned, according to security expert Kevin Beaver. Or, worse, a well-coded rootkit could detect the removal process and self-destruct, taking your data out with it.

Read the current user instructions for your scanning tools to see what special steps you need to take before, during and after the cleanup process. Then, after you've found and cleaned a rootkit, rescan the system once you reboot to double-check that it was fully cleaned and the malware hasn't returned.

In addition, the best way to address Windows security threats is to prevent malware from affecting enterprise systems and those that connect to enterprise networks in the first place.

Still a little paranoid about rootkit infections? Want to be sure your system is truly clean? The best and most reliable method is to repartition, reformat and reload Windows. Restoring Windows systems can be painful, but it's the best way to go if you really need some closure.

This was first published in July 2012

Dig deeper on Network intrusion detection and prevention and malware removal

Pro+

Features

Enjoy the benefits of Pro+ membership, learn more and join.

0 comments

Oldest 

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to:

-ADS BY GOOGLE

SearchVirtualDesktop

SearchWindowsServer

SearchExchange

Close