It has been alleged that the latest addition to the Windows family, Windows XP, has not measured up to the performance levels of Windows 2000 and Windows NT 4.0 when used with SCSI and Fibre Channel disk devices. The way I see it, the problem is not isolated to XP but with the Windows NT family of products. The phrase "Windows NT" is used here in a generic sense to refer to the Windows NT 4.0, Windows 2000, Windows XP and Windows .NET Server 2003 products. This article investigates the details of the performance problem, as well as the solution and related implications.
Operating system and storage vendors have implemented caching in an effort to maximize system throughput. In general, smaller I/O requests (read or write) have a noticeable impact on system throughput. The amount of instructions required to be executed to transfer -- say 512 bytes of data -- is approximately the same as, 64 kb of data. Also, each request involves a number of context switches. This means that the overhead of transferring data in smaller request sizes can be appreciably more. To alleviate this, operating system and disk manufacturers have implemented caching. On hardware devices, this involves some amount of faster access memory, usually RAM, that buffers requests and in some cases performs other optimizations. One of the ways caching enhances system throughput is by coalescing small I/O requests into fewer bigger I/O requests. Another way caching enhances system throughput is by allowing parallel operations. One I/O request may be accomplished between spinning media and cache while a second I/O request is simultaneously completed between an application data buffer and the cache.
While caching can enhance system throughput, some forms of write caching can increase the chances for data corruption. Data within a cache may be lost for a number of reasons including power failure, a system or bus reset, etc. In general, essential data such as database log files or file system meta-data should not be cached. This article is concerned with write caching only; read caching is unaffected.
Caching within the file system
Microsoft has implemented caching in the Windows NT family of operating systems. The cache manager sub-system implements caching co-operatively with file systems. The Windows NT Cache Manager sub-system exports an interface for use by file systems and it is also used by network file systems. The cache manager uses a large portion of system memory to buffer read and write data. Not all write data is buffered and applications can bypass this function.
Caching within storage sub-systems and devices Many storage sub-systems implement their own hardware cache, and all disk drives also contain a small amount of cache as well (up to 8MB) When caching is enabled, data is copied into the cache in the device or subsystem. The type of write caching used may be:
When no caching is implemented, the disk controller completes an I/O request only after the write data has been transferred to the spinning media. When caching is implemented, the disk controller completes an I/O request when the data is copied into its cache. The data will be transferred later on to the spinning media. Disk or subsystem controller caching in SCSI environments can be enabled through the use of the WCE or write caching enable bit. Benchmarks have shown that enabling write caching can have a noticeable impact on Windows system throughput. On SCSI devices that have write cache enabled, each data transfer operation can individually set a parameter called "Force Unit Access" (FUA) which is part of the SCSI-3 block device (SBC) specification. Support for FUA is optional, as is support for the WCE bit. RAID subsystems and host controllers, particularly ones with battery backup, may choose to ignore these bits. (IDE devices support write cache but do not yet have the notion of FUA; this has been proposed by Microsoft to the ANSI INCITS-T13 committee.)
The advantage with disk controller caching is that write performance can show appreciable improvement in throughput. The disadvantage is that the data in the cache may be lost because of various reasons including power failure, a SCSI bus reset (some controllers do flush their cache in response to a bus reset request), spinning media error etc. In general, disk controller caches have a comparatively smaller lifetime than file system caches (i.e. the data gets flushed much sooner). This means the risk of data loss is smaller. Data in a disk cache also have a greater chance of being committed to media in the event of a system crash or hang since most disk and subsystems implement some form of "cache sweeping" where data waiting to be written is automatically sent to media during idle periods.
In part two of "Nuances of Windows NT and SCSI disk performance", Dilip will cover some of the shortcomings Windows storage has, especially how certain APIs affect application caching. Part two will help you optimize and side step some of the errors in Windows that may cause you grief.
About the author:
Dilip C. Naik has more than fourteen years of experience in various roles at Microsoft, including software engineer, program manager, and technical evangelist. His contributions include writing CIFS/SMB code, CIFS-related RFCs, code and documentation for the Windows NT Installable File System Kit, as well as Windows Management Instrumentation (WMI) and performance/management (including storage management) features for the Windows platform. Dilip has also represented Microsoft on a number of industry standards organizations. Dilip has recently authored "Inside Windows Storage" from Addison-Wesley Professional. For a free chapter on Windows backup, click here.
Do you want to see more articles or insights from noted industry observers? Visit the complete Bits & Bytes column library.