Error recovery control

Lore & Background

Modern hard drives can recover from read/write errors by internally remapping sectors and performing self-test and recovery, a process that may take seconds or minutes during which the drive is unresponsive. Hardware RAID controllers and software RAID implementations are designed to recognize a drive that does not respond within a few seconds and mark it as unreliable, withdrawing it from use and rebuilding the array from parity data—a long process that degrades performance and can be catastrophic if more drives fail under the additional workload. ERC features prevent a disk from being unnecessarily marked as failed by limiting the time spent on correcting detected errors before advising the array controller of a failed operation, allowing the controller to handle data recovery for the limited amount involved rather than marking the entire drive as faulty.

Western Digital enterprise drives such as Raptor, Caviar RE2 and RE2-GP come with TLER read and write enabled (7 seconds), while desktop drives such as Caviar SE, SE16, and GP come with TLER disabled (0 seconds). In a RAID array, enabling TLER prevents the recovery time from exceeding the RAID implementation's timeout threshold, avoiding the need to manually re-add the drive and rebuild the array. In a standalone configuration, TLER should be disabled because the drive is not redundant and reporting segments as failed increases manual intervention.

The ZFS filesystem was designed to immediately write data to a sector that reports as bad or takes an excessively long time to read, which usually forces an immediate sector remap on a weak sector in most drives. In software RAID, Linux mdadm holds and lets the drive complete its recovery, but the default command timeout for the SCSI Disk layer is 30 seconds, after which it attempts to reset the drive and, if that fails, puts the drive offline. FreeBSD's ATA/CAM stack progressively increases timeouts as they occur, preventing premature removal of a desktop disk without TLER from the array.

Reader's Guide

Error recovery control is significant because it addresses a fundamental tension between drive-level error recovery and RAID-level fault tolerance. Without ERC, a drive that spends too long recovering from a bad sector can be mistaken for a failed drive by the RAID controller, triggering a lengthy and risky array rebuild. By capping the recovery time, ERC allows the RAID controller to handle the error for only the affected data segment, preserving the drive's place in the array and avoiding unnecessary rebuilds. This feature is especially important in enterprise RAID environments where uptime and data integrity are critical.

Its legacy is mixed: while ERC is standard on enterprise drives, many desktop drives lack support, and some manufacturers have removed ERC support from newer desktop models to encourage sales of more expensive RAID/enterprise models. The ATA-8 standard defined a SCT command for changing the error-recovery timeout, and the smartctl utility can modify it on drives that implement the interface. However, not all drives support the ERC parameter, and Western Digital retracted ERC support on some models without notice. Vendor utilities exist for Western Digital, Hitachi, and Seagate drives, though Western Digital's utility is no longer available and the company warns that using it on newer drives can damage firmware. The feature remains a key differentiator between consumer and enterprise storage products.

Did You Know?

• Western Digital enterprise drives such as Raptor and Caviar RE2 come with TLER read and write enabled at 7 seconds, while desktop drives have TLER disabled.
• The ZFS filesystem writes data immediately to a sector that reports as bad or takes an excessively long time to read, forcing an immediate sector remap.
• Linux mdadm holds and lets the drive complete its recovery, but the SCSI Disk layer default command timeout is 30 seconds.
• Western Digital retracted ERC support on some newer drives without notice, and its TLER utility is no longer available from the company.